# Z/Architecture

z/Architecture, initially and briefly called ESA Modal Extensions (ESAME), is IBM's 64-bit complex instruction set computer (CISC) instruction set architecture, implemented by its mainframe computers. IBM introduced its first z/Architecture-based system, the z900, in late 2000. Later z/Architecture systems include the IBM z800, z990, z890, System z9, System z10, zEnterprise 196, zEnterprise 114, zEC12, zBC12, z13, z14, z15 and z16.
z/Architecture retains backward compatibility with previous 32-bit-data/31-bit-addressing architecture ESA/390 and its predecessors back to the 32-bit-data/24-bit-addressing System/360. The IBM z13 is the last z Systems server to support running an operating system in ESA/390 architecture mode. However, all 24-bit and 31-bit problem-state application programs originally written to run on the ESA/390 architecture will be unaffected by this change.

## Features

z/Architecture includes almost all of the features of ESA/390, and adds some new features. Among the features of z/Architecture are
    A channel subsystem with the architecture introduced by S/370-XA
    Branch relative instructions introduced by ESA/390
    Trimodal (24/31/64-bit) addresses
    16 32-bit access registers (ARs) introduced by ESA/370
    16 64-bit general registers (GRs), 32-bit on older architectures
    16 64-bit control registers (CRs) introduced by System/370 as 32-bit
    16 64-bit floating-point registers (FPRs)
    32 128-bit vector registers (VRs); bits 0-63 of VR0-VR15 contain FPR0-FPR15
    1 32-bit floating point control (FPC) register
    1 128-bit program-status word (PSW), which includes a 64-bit instruction address
    An 8-KiB prefix storage area (PSA)
    Cryptographic Facility
    IEEE Binary-floating-point instructions added by ESA/390
    IEEE Decimal-floating point instructions
For information on when each feature was introduced, consult the Principles of Operation.

### Vector facility

The z13 introduced the Vector Facility for z/Architecture. It adds 32 vector registers, each 128 bits wide; the existing 16 floating-point registers are overlaid on the new vector registers. The new architecture adds over 150 new instructions to operate on data in vector registers, including integer, floating-point, and string data types. The z13 implementation includes two independent SIMD units to operate on vector data.

### Neural-network-processing-assist facility

The z16 introduced the Neural-network-processing-assist facility, which introduces several instructions performing operations on model-dependent data types. For the z16 this is the 16-bit NNP-Data-Type-1 Format.
The new instruction provide tensor operations useful for AI and neural network applications.

## Registers

Each processor has these registers
* Access registers
* Breaking-event-address register (BEAR)
* Control registers
* Floating point Control (FPC) register
* Floating point registers
* General registers
* Prefix register
* Program status word (PSW)
* Vector registers

### Access registers

Each CPU has 16 32-bit access registers. When a program running in AR mode specifies register 1-15 as a base register or as a register operand containing an address, the CPU uses the associated access register during address translation.

### Breaking-event-address register (BEAR)

The 64-bit BEAR contains the address of the last instruction that broke the sequential execution of instructions; an interrupt stores the BEAR in the doubleword at real address 272 (110$_{16}$). After an Execute of a branch, the BEAR contains the address of the execute, not that of the branch.

### Control registers

The 16 64-bit control registers provide controls over and the status of a CPU, except for information included in the PSW. They are an evolutionary enhancement to the control registers on the earlier ESA/390 on the IBM S/390 processors. For details on which fields are dependent on specific features, consult the Principles of Operation. Because z/Architecture expands the control registers from 32 bits to 64, the bit numbering differs from that in ESA/390.
z/Architecture mode control registers :
| CR | bits | Field |
| 0 | 8 | Transactional-execution control |
| 0 | 9 | Transactional-execution program-interruption filtering override |
| 0 | 10 | Clock-comparator sign control |
| 0 | 13 | Cryptography counter control |
| 0 | 14 | Processor-activity-instrumentation-extension control |
| 0 | 15 | Measurement-counter-extraction-authorization control |
| 0 | 30 | Warning-track subclass mask |
| 0 | 32 | TRACE TOD-clock control |
| 0 | 33 | SSM-suppression |
| 0 | 34 | TOD-clock-sync control |
| 0 | 35 | Low-address-protection control |
| 0 | 36 | Extraction-authority control |
| 0 | 37 | Secondary-space control |
| 0 | 38 | Fetch-protection-override control |
| 0 | 39 | Storage-protection-override control |
| 0 | 40 | Enhanced-DAT-enablement control |
| 0 | 43 | Instruction-execution-protection-enablement control |
| 0 | 44 | ASN-and-LX-reuse control |
| 0 | 45 | AFP-register control |
| 0 | 46 | Vector enablement control |
| 0 | 48 | Malfunction-alert subclass mask |
| 0 | 48 | Malfunction-alert subclass mask |
| 0 | 49 | Emergency-signal subclass mask |
| 0 | 50 | External-call subclass mask |
| 0 | 52 | Clock-comparator subclass mask |
| 0 | 53 | CPU-timer subclass mask |
| 0 | 54 | Service-signal subclass mask |
| 0 | 56 | Initialized to 1 |
| 0 | 57 | Interrupt-key subclass mask |
| 0 | 58 | Measurement-alert subclass mask |
| 0 | 59 | Timing-alert subclass mask |
| 0 | 61 | Crypto control |
| 1 | 0-51 | Primary Address-Space Control Element (ASCE) Primary region-table origin Primary segment-table origin Primary real-space token origin |
| 1 | 54 | Primary subspace-group control |
| 1 | 55 | Primary private-space control |
| 1 | 56 | Primary storage-alteration-event |
| 1 | 57 | Primary space-switch-event control |
| 1 | 58 | Primary real-space control |
| 1 | 60-61 | Primary designation-type control |
| 1 | 62-63 | Primary table length |
| 2 | 33-57 | Dispatchable-unit-control-table origin |
| 2 | 59 | Guarded-storage-facility enablement control |
| 2 | 61 | Transaction diagnostic scope |
| 2 | 62-63 | Transaction diagnostic control |
| 3 | 0-31 | Secondary ASN-second-table-entry instance number |
| 3 | 32-47 | PSW-key mask |
| 3 | 48-63 | Secondary ASN |
| 4 | 0-31 | Primary ASN-second-table-entry instance number |
| 4 | 32-47 | Authorization index |
| 4 | 48-63 | Primary ASN |
| 5 | 33-57 | Primary-ASN-second-table-entry origin |
| 6 | 32-39 | I/O-interruption subclass mask |
| 7 | 0-51 | Secondary Address-Space Control Element (ASCE) Secondary region-table origin Secondary segment-table origin Secondary real-space token origin |
| 7 | 54 | Secondary subspace-group control |
| 7 | 55 | Secondary private-space control |
| 7 | 56 | Secondary storage-alteration-event control |
| 7 | 58 | Secondary real-space control |
| 7 | 60-61 | Secondary designation-type control |
| 7 | 62-63 | Secondary table length |
| 8 | 16-31 | Enhanced-monitor masks |
| 8 | 32-47 | Extended authorization index |
| 8 | 48-63 | Monitor masks |
| 9 | 32 | Successful-branching-event mask |
| 9 | 33 | Instruction-fetching-event mask |
| 9 | 34 | Storage-alteration-event mask |
| 9 | 35 | Storage-key-alteration-event mask |
| 9 | 36 | Store-using-real-address-event mask |
| 9 | 37 | Zero-address-detection-event mask |
| 9 | 38 | Transaction-end event mask |
| 9 | 39 | Instruction-fetching-nullification-event mask |
| 9 | 40 | Branch-address control |
| 9 | 41 | PER-event-suppression control |
| 9 | 43 | Storage-alteration-space control |
| 10 | 0-63 | PER starting address |
| 11 | 0-63 | PER ending address |
| 12 | 0 | Branch-trace control |
| 12 | 1 | Mode-trace control |
| 12 | 2-61 | Trace-entry address |
| 12 | 62 | ASN-trace control |
| 12 | 63 | Explicit-trace control |
| 13 | 0-51 | Home Address-Space Control Element (ASCE) Home region-table origin Home segment-table origin Home real-space token origin |
| 13 | 55 | Home private-space control |
| 13 | 56 | Home storage-alteration-eventl |
| 13 | 57 | Home space-switch-event control |
| 13 | 58 | Secondary real-space control |
| 13 | 60-61 | Home designation-type control |
| 13 | 62-63 | Home table length |
| 14 | 32 | Set to 1 |
| 14 | 33 | Set to 1 |
| 14 | 34 | Extended save-area control (ESA/390-compatibility mode only) |
| 14 | 35 | Channel-report-pending subclass mask |
| 14 | 36 | Recovery subclass mask |
| 14 | 37 | Degradation subclass mask |
| 14 | 38 | External-damage subclass mask |
| 14 | 39 | Warning subclass mask |
| 14 | 42 | TOD-clock-control-override control |
| 14 | 44 | ASN-translation control |
| 14 | 45-63 | ASN-first-table origin |
| 15 | 0-60 | Linkage-stack-entry address |

### Floating point Control (FPC) register

The FPC register contains Interrupt Masks (IM), Status Flags (SF), Data Exception Code (DXC), Decimal Rounding Mode (DRM) and Binary Rounding Mode (BRM). An interruption only stores the DXC if the FPC register if the AFP-register (additional floating-point register) control bit, bit 13 of control register 0, is one. Also, while individual bits of the DXC usually have significance, programs should normally treat it as an 8-bit integer rather than querying individual bits.
FPC fields :
| Byte name | Bits | Field name | Use |
| masks | 0 | IMi | IEEE-invalid-operation mask |
| masks | 1 | IMz | IEEE-division-by-zero mask |
| masks | 2 | IMo | IEEE-overflow mask |
| masks | 3 | IMu | IEEE-underflow mask |
| masks | 4 | IMx | IEEE-inexact mask |
| masks | 5 | IMq | Quantum-exception mask |
| flags | 8 | SFi | IEEE-invalid-operation flag |
| flags | 9 | SFz | IEEE-division-by-zero |
| flags | 10 | SFo | IEEE-overflow flag |
| flags | 11 | SFu | IEEE-underflow flag |
| flags | 12 | SFx | IEEE-inexact flag |
| flags | 13 | SFq | Quantum-exception flag |
| DXC | 16-23 | DXC | Data-exception code |
| DXC | 16 | i | IEEE-invalid-operation |
| DXC | 17 | z | IEEE-division-by-zero |
| DXC | 18 | o | IEEE-overflow |
| DXC | 19 | u | IEEE-underflow mask |
| DXC | 20 | x | IEEE-inexact mask |
| DXC | 21 | y/q | Quantum-exception mask |
| | 25-27 | DRM | DFP rounding mode |
| | 29-31 | BRM | BFP rounding mode |

### Floating point registers

Each CPU had 16 64-bit floating point registers; FP0-15 occupy bits 0-63 of VR0-15.

### General registers

Each CPU has 16 64-bit general registers, which serve as accumulators, base registers and index registers. Instructions designated as Grandé operate on all 64 bits; some instructions added by the Extended-Immediate Facility operate on any halfword or word in the register; most other instructions do not change or use bits 0-31.

### Prefix register

The prefix register is used in translating a real address to an absolute address. In z/Architecture mode, the PSA is 2 pages (8 KiB). Bits 0-32 and 51-63 are always zero. If bits 0-50 of a real address are zero then they are replaced by bits 0-50 of the prefix register; if bits 0-50 of the real address are equal to bits 0-50 of the prefix register then they are replaced with zeros.

### Program status word (PSW)

The PSW holds the instruction address and other fields reflecting the status of the program currently running on a CPU. The status of the program is also affected by the contents of the Control registers.

### Vector registers

Each CPU has 32 128-bit vector registers; bits 0-63 of VR0-15 are also FPR0-15. A vector register may contain 16 8-bit fields, 8 16-bit fields, 4 32-bit fields, 2 64-bit fields or 1 128-bit field.

## Memory

IBM classifies memory in z/Architecture into Main Storage and Expanded Storage.
Main storage is addressed in 8-bit bytes (octets), with larger aligned groupings:

### Halfword

    Two bytes
    16 bits

### Word

    Four bytes
    32 bits

### Doubleword

    8 bytes
    64 bits

### Quadword

    16 bytes
    128 bits

### Page

    4096 bytes
Although z/Architecture allows real and virtual addresses from 0 to 2$^{64}$-1, engineering constraints limit current and planned models to far less.
Expanded storage is address in 4 KiB blocks, with block numbers ranging fom 0 to 2$^{32}$.

## Addressing
### Types of main storage addresses

There are three types of main storage addresses in z/Architecture

#### Virtual address

    The address as seen by application programs. It is an offset into an address space and is subject to address translation via page and segment tables.

#### Real address

    The address after address translation, or the address seen by an OS component running with translation off. It is subject to prefixing.

#### Absolute address

    The address after prefixing references to the first two pages via the prefix register.

### Address encoding

z/Architecture uses the same truncated addressing as ESA, with some additional instruction formats. As with ESA, in AR mode each nonzero base register is associated with a base register specifying the address space. Depending on the instruction, an address may be provided in several different formats.

#### R

    The address is contained in a general register

#### Relative

    A signed 16-bit halfword offset from the current instruction.

#### Relative long

    A signed 32-bit halfword offset from the current instruction.

#### RS

    A base register and a 12-bit displacement

#### RX

    A base register, an index register, and a 12-bit displacement

#### Y

    A base register, an index register, and a 20-bit displacement; colloquially known as "Yonder".

### Addressing modes

In addition to the two addressing modes supported by S/370-XA and ESA, a/Architecture has an extended addressing mode with 64-bit virtual addresses. The addressing mode is controlled by the EA (bit 31) and BA (bit 32) bits in the PSW. The valid combinations are
* 00 24-bit addressing
* 01 31-bit addressing
* 11 64-bit addressing

### Translation modes

z/Architecture supports four virtual translation modes, controlled by bit 5, the DAT-mode bit, and bits 16-17, the Address-Space Control (AS) bits, of the PSW.

#### Primary-space mode

    All storage references use the translation tables for the primary address space

#### Access-register mode

    All storage references use the translation tables designated by the access register associated with the base register.

#### Secondary-space mode

    All storage references use the translation tables for the secondary address space

#### Home-space mode

    All storage references use the translation tables for the home address space

## Operating system support

IBM's operating systems z/OS, z/VSE, z/TPF, and z/VM are versions of MVS, VSE, Transaction Processing Facility (TPF), and VM that support z/Architecture. Older versions of z/OS, z/VSE, and z/VM continued to support 32-bit systems; z/OS version 1.6 and later, z/VSE Version 4 and later, and z/VM Version 5 and later require z/Architecture.
Linux also supports z/Architecture with Linux on IBM Z.
z/Architecture supports running multiple concurrent operating systems and applications even if they use different address sizes. This allows software developers to choose the address size that is most advantageous for their applications and data structures.
On July 7, 2009, IBM on occasion of announcing a new version of one of its operating systems implicitly stated that Architecture Level Set 4 (ALS 4) exists, and is implemented on the System z10 and subsequent machines. The ALS 4 is also specified in LOADxx as ARCHLVL 3, whereas the earlier z900, z800, z990, z890, System z9 specified ARCHLVL 2. Earlier announcements of System z10 simply specified that it implements z/Architecture with some additions: 50+ new machine instructions, 1 MB page frames, and hardware decimal floating point unit (HDFU).
Most operating systems for the z/Architecture, including z/OS, generally restrict code execution to the first 2 GB (31 address bits, or 2$^{31}$ addressable bytes) of each virtual address space for reasons of efficiency and compatibility rather than because of architectural limits. Linux on IBM Z allows code to execute within 64-bit address ranges.

### z/OS

Each z/OS address space, called a 64-bit address space, is 16 exabytes in size.

#### Code (or mixed) spaces

The z/OS implementation of the Java programming language is an exception. The z/OS virtual memory implementation supports multiple 2 GB address spaces, permitting more than 2 GB of concurrently resident program code.

#### Data-only spaces

Data-only spaces are memory regions that can be read from and written to, but not used as executable code. (Similar to the NX bit on other modern processors.) By default, the z/Architecture memory space is indexed by 64-bit pointers, allowing up to 16 exabytes of memory to be visible to an executing program.

##### Dataspaces and hiperspaces

Applications that need more than a 16 exabyte data address space can employ extended addressability techniques, using additional address spaces or data-only spaces. The data-only spaces that are available for user programs are called:
* dataspaces (sometimes referred to as "data spaces") and
* hiperspaces (High performance space).
These spaces are similar in that both are areas of virtual storage that a program can create, and can be up to 2 gigabytes. Unlike an address space, a dataspace or hiperspace contains only user data; it does not contain system control blocks or common areas. Program code cannot run in a dataspace or a hiperspace.
A dataspace differs from a hiperspace in that dataspaces are byte-addressable, whereas hiperspaces are page-addressable.

### IBM mainframe expanded storage

Traditionally IBM Mainframe memory has been byte-addressable. This kind of memory is termed "Central Storage". IBM Mainframe processors through much of the 1980s and 1990s supported another kind of memory: Expanded Storage. It was first introduced with the IBM 3090 high-end mainframe series in 1985.
Expanded Storage is 4KB-page addressable. When an application wants to access data in Expanded Storage it must first be moved into Central Storage. Similarly, data movement from Central Storage to Expanded Storage is done in multiples of 4KB pages. Initially page movement was performed using relatively expensive instructions, by paging subsystem code.
The overhead of moving single and groups of pages between Central and Expanded Storage was reduced with the introduction of the MVPG (Move Page) instruction and the ADMF (Asynchronous Data Mover Facility) capability.
The MVPG instruction and ADMF are explicitly invoked—generally by middleware in z/OS or z/VM (and ACP?)—to access data in expanded storage. Some uses are namely:
* MVPG is used by VSAM Local Shared Resources (LSR) buffer pool management to access buffers in a hiperspace in Expanded Storage.
* Both MVPG and ADMF are used by IBM Db2 to access hiperpools. Hiperpools are portions of a buffer pool located in a hiperspace.
* VM Minidisk Caching.
Until the mid-1990s Central and Expanded Storage were physically different areas of memory on the processor. Since the mid-1990s Central and Expanded Storage were merely assignment choices for the underlying processor memory. These choices were made based on specific expected uses: For example, Expanded Storage is required for the Hiperbatch function (which uses the MVPG instruction to access its hiperspaces).
In addition to the hiperspace and paging cases mentioned above there are other uses of expanded storage, including:
* Virtual I/O (VIO) to Expanded Storage which stored temporary data sets in simulated devices in Expanded Storage. (This function has been replaced by VIO in Central Storage.)
* VM Minidisk Caching.
z/OS removed the support for Expanded Storage. All memory in z/OS is now Central Storage. z/VM 6.4 fulfills Statement of Direction to drop support for all use of Expanded Storage.

### MVPG and ADMF
#### MVPG

IBM described MVPG as "moves a single page and the central processor cannot execute any other instructions until the page move is completed."
The MVPG mainframe instruction (MoVe PaGe, opcode X'B254') has been compared to the MVCL (MoVe Character Long) instruction, both of which can move more than 256 bytes within main memory using a single instruction. These instructions do not comply with definitions for atomicity, although they can be used as a single instruction within documented timing and non-overlap restrictions.$^{:Note8,page7–27}$
The need to move more than 256 bytes within main memory had historically been addressed with software (MVC loops), MVCL, which was introduced with the 1970 announcement of the System/370, and MVPG, patented and announced by IBM in 1989, each have advantages.

#### ADMF

ADMF (Asynchronous Data Mover Facility), which was introduced in 1992, goes beyond the capabilities of the MVPG (Move Page) instruction, which is limited to a single page, and can move groups of pages between Central and Expanded Storage.
A macro instruction named IOSADMF, which has been described as an API that avoids "direct, low-level use of ADMF", can be used to read or write data to or from a hiperspace. Hiperspaces are created using DSPSERV CREATE.
To provide reentrancy, IOSADMF is used together with a "List form" and "Execute form."

## Non-IBM implementations

Platform Solutions Inc. (PSI) previously marketed Itanium-based servers which were compatible with z/Architecture. IBM bought PSI in July 2008, and the PSI systems are no longer available. FLEX-ES, zPDT and the Hercules emulator also implement z/Architecture. Hitachi mainframes running newer releases of the VOS3 operating system implement ESA/390 plus Hitachi-unique CPU instructions, including a few 64-bit instructions. While Hitachi formally collaborated with IBM on the z900-G2/z800 CPUs introduced in 2002, Hitachi's machines are not z/Architecture-compatible.