The EVEX prefix (enhanced vector extension) and corresponding coding scheme is an extension to the 32-bit x86 (IA-32) and 64-bit x86-64 (AMD64) instruction set architecture. EVEX is based on, but should not be confused with the MVEX prefix used by the Knights Corner processor.

The EVEX scheme is a 4-byte extension to the VEX scheme which supports the AVX-512 instruction set and allows addressing new 512-bit ZMM registers and new 64-bit operand mask registers.

With Advanced Performance Extensions, the Extended EVEX prefix redefines the semantics of several payload bits.

Features
EVEX coding can address 8 operand mask registers, 16 general-purpose registers and 32 vector registers in 64-bit mode (otherwise, 8 general-purpose and 8 vector), and can support up to 4 operands.

Like the VEX coding scheme, the EVEX prefix unifies existing opcode prefixes and escape codes, memory addressing and operand length modifiers of the x86 instruction set.

The following features are carried over from the VEX scheme: 
 Direct encoding of three SIMD registers (XMM, YMM, or ZMM) as source operands (MMX or x87 registers are not supported);
 Compacted REX prefix for 64-bit mode;
 Compacted SIMD prefix (66h, F2h, F3h), escape opcode (0Fh) and two-byte escape (0F38h, 0F3Ah);
 Less strict memory alignment requirements for memory operand

EVEX also extends VEX with additional capabilities:
 Extended SIMD register encoding: a total of 32 new 512-bit SIMD registers ZMM0–ZMM31 in 64-bit mode;
 Operand mask encoding: 8 new 64-bit opmask registers k0–k7 for conditional execution and merging of destination operands;
 Broadcasting from source to destination for instructions that take memory vector as a source operand: the second operand is broadcast before being used in the actual operation;
 Direct embedded rounding control for instructions that operate on floating-point SIMD registers with rounding semantics;
 Embedded exceptions control for floating-point instructions without rounding semantics;
 Compressed displacement (Disp8 × N), new memory addressing mode to improve encoding density of instruction byte stream; the scale factor N depends on vector length and broadcast mode.

For example, the EVEX encoding scheme allows conditional vector addition in the form of

 VADDPS zmm1 {k1}{z}, zmm2, zmm3

where {k1} modifier next to the destination operand encodes the use of opmask register k1 for conditional processing and updates to destination, and {z} modifier (encoded by EVEX.z) provides the two types of masking (merging and zeroing),  with merging as default when no modifier is attached.

Technical description
The EVEX coding scheme uses a code prefix consisting of 4 bytes; the first byte is always 62h and derives from an unused opcode of the 32-bit BOUND instruction, which is not supported in 64-bit mode. 

The ModR/M byte specifies one operand (always a register) with reg field, and the second operand is encoded with mod and r/m fields, specifying either a register or a location in memory.  Base-plus-index and scale-plus-index addressing require the SIB byte, which encodes 2-bit scale factor as well as 3-bit index and 3-bit base registers. Depending on the addressing mode, Disp8/Disp16/Disp32 field may follow with displacement that needs to be added to the address.

The EVEX prefix retains fields introduced in the VEX prefix:

 Four bits R̅, X̅, B̅ and W from the VEX prefix, stored in inverted form. W expands the operand size to 64 bits or serves as an additional opcode, R expands reg, B expands r/m or reg, and X and B expand index and base in the SIB byte. 
 Four bits named v̅, stored in inverted form. vvvv specifies a second non-destructive source register operand. 
 Bit L specifying 256-bit vector length.
 Two bits named p to replace operand size prefixes and operand type prefixes (66h, F2h, F3h).
 Two of the m bits for replacing existing escape codes (0Fh, 0F 38h and 0F 3Ah).

New functions of the existing fields:
 Bit X now expands r/m along with bit B when the SIB byte is not present, which allows 32 SIMD registers.

There are several new bit fields:
 Bit R̅’ in inverted form; R’ expands reg.
 Bit V̅’ in inverted form; V’ expands vvvv.
 Three bits named a, specifying the operand mask register (k0–k7) for vector instructions.
 Bit z for specifying merging mode (merge or zero).
 Bit b for source broadcast, rounding control (combined with L’L), or suppress exceptions. 
 Bit L’ for specifying 512-bit vector length, or rounding control mode when combined with L.

The encoding of the EVEX prefix is as follows:

The following table lists possible register addressing combinations (bit 4 is always zero when encoding the 16 general purpose registers):

A few VEX-encoded AVX blending instructions have 4 operands. To accommodate this, VEX has IS4 addressing mode, which encodes 4th operand (a vector register) in bits Imm8[7:4] of the immediate constant. Similar EVEX-encoded blend instructions have their 4th operand in a mask register. No EVEX-encoded instruction uses IS4 addressing mode encoding.

Extended EVEX prefix 
Intel Advanced Performance Extensions introduce several new variants of the 3-byte payload in the EVEX prefix, which are used to encode Extended GPR registers R16-R31 and new conditional instructions.

EVEX extension of EVEX instructions:

 R̅3, X̅3 and B̅3 bits are inversions of the REX2 prefix's R3, X3 and B3 bits. These are the same as R̅, X̅ and B̅ bits from VEX and EVEX prefixes.
 R̅4, X̅4, B4 bits are used to encode the 32 EGPR registers. Stored in inverted form, except for B4.
 Five bits named v̅, stored in inverted form. vvvvv specifies additional source register index, which can encode the 32 EGPR registers. 
 Z, m, b, L, p bits are the same as in the legacy EVEX prefix.

EVEX extension of VEX instructions:

EVEX extension for legacy instructions:

 NF is status flags update suppression ("no flags") for several BMI instructions (ANDN, BEXTR, BLSI, BLSMSK, BLSR, BZHI).
 ND is new data destination (NDD) flag. When ND = 1, EGPR register index is encoded by v̅ bits.

EVEX prefix for conditional CMP and TEST:

 SC bits are source condition code (SCC).
 OF, SF, ZF, CF are overflow, sign, zero, and carry flags to test (there is no encoding for the parity flag).

When the new EGPR registers and operand destinations can be encoded by both extended EVEX and REX2 prefixes, the latter is preferred.

References

X86 instructions
SIMD computing