Technology: SIMD / MMX / SSE / SSE2 / SSE3 / 3DNow!

An increasing number of newer processors come with extensions to their instruction set commonly referred to as SIMD instructions. SIMD stands for Single Instruction Multiple Data meaning that a single instruction, such as "add", operates on a number of data items in parallel. A typical SIMD instruction for example will add 8 16 bit values in parallel. Obviously this can increase execution speed dramatically which is why these instruction set extensions were created.

You can either read this page sequentially or jump directly to the following subsections:

• MMX Instruction Set
• SSE Instruction Set
• SSE2 Instruction Set
• SSE3 Instruction Set (not yet covered here)
• General Issues with SIMD Instructions
• Speed-Ups
• Conclusion

[The 3DNow! instruction set extension is not covered, however, the SSE instruction set is similar.]

MMX Instruction Set

In 1997 Intel launched the Pentium Processor with MMX Technology. Subsequently the Pentium II, III and 4 were launched and other processors, such as the AMD Athlon and Duron, added this instruction set extension.

The MMX instruction set introduces 8 new 64 bit wide registers named mm0 .. mm7; however, these are mapped on the FPU registers, so FPU and MMX instructions can not be intermixed. The registers can hold the following data types:

Corresponding integer instructions were introduced:

All operations are either register, register or register, memory operations. No memory, register or memory, memory operations are available.

Here come some examples:

As can be seen from the above instruction list there are a number of limitations on this initial set of instructions:

• Not all data types are supported for all operations, i.e. unsigned multiplication, byte-wide shifts are missing etc.
• Various interesting operations are missing, i.e. min/max
• The registers are shared with the FPU registers, prohibiting any FPU operations in parallel
• No floating point operations are possible
• The registers are only 64 bits wide
• There are only 8 registers
• There are not prefetch or write-through instructions
• There are no real provisions for unaligned access
• Constants are not supported

The subsequent introduction of the SSE and SSE2 instruction set extensions remove many of these limitations.

Jump to top of page

SSE Instruction Set

The Pentium III processor, introduced in 1999, was the first processor with the SSE instruction set extension. However, AMD at that time already had added the similar 3DNow! instruction set extension to their AMD K6 CPU.

The SSE instruction set in fact consists of 3 distinct enhancements:

• Additional MMX instructions such as min/max
• Prefetch and write-through instructions for optimizing data movement from and to the L2/L3 caches and main memory
• New 128 bit XMM registers and corresponding 32 bit floating point (single precision) instructions

The SSE instruction set introduces 8 new 128 bit wide registers named xmm0 .. xmm7. The registers can hold the following data types:

The SSE instruction set includes the MMX instruction set; however, the following table lists only the additional SSE instructions:

Here come some examples:

As can be seen from the above instruction list there are still a number of limitations on this set of instructions:

• Not all data types are supported for all operations, i.e. 32 bit integer multiplies, byte-wide shifts are missing etc.
• Only 32 bit floating point data types are available
• There are no integer instructions for the XMM registers
• There are only 8 registers
• There are no real provisions for unaligned access
• Constants are not supported

The subsequent introduction of the SSE2 instruction set extension removes some of these limitations.

Jump to top of page

SSE2 Instruction Set

The Pentium 4 processor introduced the SSE2 instruction set extensions in 2000.

Two important improvements were made:

• Support 64 bit (double precision) floating point data types
• Integer instruction for the XMM registers 

Because the SSE2 instruction set introduces integer instructions to the XMM registers and double precision floating data types the XMM registers can now hold following data types:

The SSE2 instruction set includes the SSE instruction set (the MMX and SSE integer instructions also apply to the XMM registers now and the single precision floating point instructions of the SSE instruction set also apply to the double precision data types); the following table lists only the additional SSE2 instructions:

Here come some examples:

As can be seen from the above instruction list there are still a number of limitations on this set of instructions:

• Not all data types are supported for all operations, i.e. 32 bit saturated adds, byte-wide shifts are missing etc.
• There are only 8 registers
• There are no real provisions for unaligned access
• Constants are not supported

It is unclear whether future instruction set extensions will remove some of these limitations.

Nevertheless the SSE2 instruction set in effect is a parallel version of the typical integer and floating point instructions found in the basic instruction set of most processors, thereby enabling impressive speed-ups.

Jump to top of page

General Issues with SIMD Instructions

The basic characteristic of SIMD instructions is that they operate on n data items in parallel, n typically being 1, 2, 4, 8 or 16. There are mainly 5 problems with this:

• If the data layout does not match the SIMD requirements SIMD instructions can either not be used or data rearrangement code is necessary
• In case of unaligned data the performance will suffer dramatically; this problem leads to massively more complicated alignment code being necessary
• If the problem does not match the possible values of n, for example in case of a filter which would require n = 5, the solution is rather complicated 
• (Parallel) Table lookups are not possible
• Compare operations are limited

Basically there are solutions to these problems, but they require some clever or additional code. Due to these problems even compilers which generate SIMD code will in many cases not do so or not be able to do so.

Jump to top of page

Speed-Ups

SIMD instructions have the potential to speed-up software by factors of 2 .. 16 or even more. This especially applies to compute-intensive software dealing with arrays of data.

Please also see the case studies which give numbers on speed-up factors.

Jump to top of page

Conclusion

SIMD instructions have the potential to speed-up software by factors of 2 .. 16 or even more.

However, as most compilers do not utilize the (full) potential of these instructions these speed-ups can generally only be achieved by an optimization expert.

Hayes Technologies offers services to help your software realize the high potential of the SIMD instruction sets.

Jump to top of page