Intel Intrinsics

Learning Outcomes

• Understand how Intel Intrinsics exposes the Intel SIMD ISA(s) in the high-level C language.
• Write C code that leverages Intel Intrinsics.
• Given an Intel intrinsic, interpret register width and (if applicable) packed data type.

We have discussed earlier that Intel SIMD instruction set architectures (ISAs) are extensions to the base Intel x86/x87 architecture. ISAs specify assembly instructions. In this section, we discuss at a high-level the assembly instruction format, then focus more on how to call these assembly instructions from our high-level C program using Intel Intrinsics.

Intel ISA, continued

SIMD instructions act as extensions to a base instruction set, with different systems supporting different SIMD instructions.

Intel SIMD Registers

The “wide” registers that Intel SIMD architectures use are separate from the general-purpose and floating-point registers used in x86/x87. For example, the SSE2 extension has 128-bit registers. As shown in #fig-intel-simd-regs, these 128-bit-wide registers can be interpreted as values packed in different ways: two 64-bit words, four 32-bit words, and so on.

Intel SSE/AVX-128 128-bit-wide registers and AVX 256-bit-wide registers pack different numbers of data types. On Intel architectures, words are 16-bits, so single-precision floating point is a double-word (32bit) and double-precision floating point is a quadword (64-bit).

As a side note, registers from legacy extensions operate on the lower bits of modern extensions, as shown in #fig-intel-simd-regs-compatible.

AVX 256-bit-wide YMM registers. Legacy SSE instructions (which use the XMM registers) can still be used to operate on the lower 128 bits of the YMM registers.

Intel SIMD Assembly Instructions

Assembly instructions, e.g., in SSE, operate on SSE registers. Expand the below code for examples of SSE extension assembly instructions.

// pseudocode
vec_res.x = v1.x + v2.x;
vec_res.y = v1.y + v2.y;
vec_res.z = v1.z + v2.z;
vec_res.w = v1.w + v2.w;

# v1.w | v1.z | v1.y | v1.x -> xmm0
movaps  address-of-v1, %xmm0 

# v1.w+v2.w | v1.z+v2.z | v1.y+v2.y | v1.x+v2.x -> xmm0
addps   address-of-v2, %xmm0  

movaps  %xmm0, address-of-vec_res

We will not discuss Intel assembly instructions too much in this course. Instead, because we leverage a compiler like gcc to translate C into assembly, we will directly write such instructions into our high-level C programs using Intel intrinsics.

Final comments

RISC-V doesn’t have a standard vector library, so we’re using x86’s SIMD extension operators on the course hive machines. In practice, this doesn’t matter too much since arithmetic syntax works similarly to RISC-V.

Some notes:

• There’s still only one PC, so we can’t vectorize branch or jump instructions.
• Since we only have limited instructions available, we can’t do different math operations to vector components.
• The biggest list of SIMD architectures comes from efficient loading and storing (though of course there are small efficiencies from performing arithmetic operations in parallel). However, with SIMD registers we can only easily load/store to/from memory consecutive chunks of memory.¹
• Each instruction needs its own circuitry, so we’re limited to the set of instructions that came with the CPU. Large registers require significant amounts of circuitry, so they are expensive to implement, and often have higher cycles/instruction than standard instructions.

Intel Intrinsics

Intel Intrinsics are C functions and procedures that provide access to assembly language. With intrinsics, we can program using assembly instructions indirectly. There is a one-to-one correspondence between a given Intel intrinsic and an Intel SIMD extension assembly instruction (e.g., SSE, AVX).

Variable Declaration

C has typed variables (in contrast to assembly, which only has hardware registers that store bits). To use Intel intrinsics, we must declare registers as C variables of a specific Intel intrisic variable type.

In the Intel AVX family of extensions, registers are 256-bits wide. The corresponding Intel intrinsic __m256d reg; declaration indicates that reg is a 256-bit-wide AVX register that packs four double-precision floating point values.

Once declared, we can use the Intel intrinsic name similarly to C variables. Importantly, the name is associated with available registers, so we can’t just initialize, say, an array of __mm256ds.

Procedures

Once we have declared Intel intrisic variables, we can call Intel intrinsics. While these look like functions and procedures, each and every function call maps directly to an assembly instruction for the SIMD hardware.

Understanding Intel Intrinsics Format

Luckily, most intel intrinsic procedures and data types are formatted similarly. #fig-intrinsics-format provides a guide.

Intel Instructions and Formats.

Example

void simd_example_pseudo() {
    int arr[8] = {3, 1, 4, 1, 5, 9, 2, 6};
    // 1. 
    sse128_t sum_sse = sse_set_zero();
    // sum_sse: {0, 0, 0, 0}

    // 2. P
    sse128_t tmp = sse_load(arr);
    sum_sse = sse_add(sum_sse, tmp);
    // sum_sse: {3, 1, 4, 1}

    // 3. 
    tmp = sse_load(arr + 4);
    // tmp: {5, 9, 2, 6}
    sum_sse = sse_add(sum_sse, tmp);
    // sum_sse: {3 + 5, 1 + 9, 4 + 2, 1 + 6}
    // sum_sse: {8, 10, 6, 7}

    // 4. 
    int tmp_arr[4];
    sse_store(tmp_arr, sum_sse);
    int sum = tmp_arr[0] + tmp_arr[1] + tmp_arr[2] + tmp_arr[3];
    printf("sum: %d\n", sum);
}

Let’s rewrite this pseudocode with Intel Intrinsics, can you explain how the below code implements the vector pseudocode from earlier?

int simd_example_sse() {
    int arr[8] = {3, 1, 4, 1, 5, 9, 2, 6};

    // 1.
    __m128i sum_sse = _mm_setzero_si128();
    // sum_sse: {0, 0, 0, 0}

    // 2.
    __m128i tmp = _mm_loadu_si128((__m128i *) arr);
    sum_sse = _mm_add_epi32(sum_sse, tmp);
    // sum_sse: {3, 1, 4, 1}

    // 3.
    tmp = _mm_loadu_si128((__m128i *) (arr + 4));
    sum_sse = _mm_add_epi32(sum_sse, tmp);
    // sum_sse: {3 + 5, 1 + 9, 4 + 2, 1 + 6}

    // 4.
    int tmp_arr[4];
    _mm_storeu_si128((__m128i *) tmp_arr, sum_sse);
    int sum = tmp_arr[0] + tmp_arr[1] + tmp_arr[2] + tmp_arr[3];

    printf("sum: %d\n", sum);
    return sum;
}

#tab-simd-intel describes some more common operations. Note we use a vector pseudocode here; this is consistent with many final exam examples.

Common mistakes with SIMD instructions

• Trying to directly access a 32-bit chunk of a SIMD register (such as through typecasting)

  • Need to do an explicit load/store, since registers are different from memory

• Trying to _mm_load or _mm_store with unaligned addresses

  • Use loadu or storeu if you must, or try to get your addresses aligned.
  • For mallocs, aligned_alloc (from C <stdlib.h>, C11 or later) gives you an aligned address.

• Forgetting the “tail” case

  • If your data isn’t an array whose length is a multiple of your SIMD register size (e.g., 4), you need to handle the last iterations of your dataset one-by-one instead of, e.g., 4 at a time.

• Using too many SIMD registers (or creating a large array of registers)

  • Ends up throttling your code because the compiler ends up trying load/store SIMD registers to the stack a bunch of times.