Chapter 11. Preparing a RHEL installation on 64-bit IBM Z This section describes how to install Red Hat Enterprise Linux on the 64-bit IBM Z architecture. 11.1. Planning for installation on 64-bit IBM Z Red Hat Enterprise Linux 8 runs on z13 or later IBM mainframe systems. The installation process assumes that you are familiar with the 64-bit IBM Z and can set up logical partitions (LPARs) and z/VM guest virtual machines. For installation of Red Hat Enterprise Linux on 64-bit IBM Z, Red Hat supports Direct Access Storage Device (DASD), Fiber Channel Protocol (FCP) storage devices, and virtio-blk and virtio-scsi devices. When using FCP devices, Red Hat recommends using them in multipath configuration for better reliability. Important DASDs are disks that allow a maximum of three partitions per device. For example, dasda can have partitions dasda1 , dasda2 , and dasda3 . Pre-installation decisions Whether the operating system is to be run on an LPAR, KVM, or as a z/VM guest operating system. Network configuration. Red Hat Enterprise Linux 8 for 64-bit IBM Z supports the following network devices: Real and virtual Open Systems Adapter (OSA) Real and virtual HiperSockets LAN channel station (LCS) for real OSA virtio-net devices RDMA over Converged Ethernet (RoCE) Ensure you select machine type as ESA for your z/VM VMs, because selecting any other machine types might prevent RHEL from installing. See the IBM documentation . Note When initializing swap space on a Fixed Block Architecture (FBA) DASD using the SWAPGEN utility, the FBAPART option must be used. Additional resources For additional information about system requirements, see RHEL Technology Capabilities and Limits For additional information about 64-bit IBM Z, see IBM documentation . For additional information about using secure boot with Linux on IBM Z, see Secure boot for Linux on IBM Z . For installation instructions on IBM Power Servers, refer to IBM installation documentation . To see if your system is supported for installing RHEL, refer to https://catalog.redhat.com . 11.2. Boot media compatibility for IBM Z servers The following table provides detailed information about the supported boot media options for installing Red Hat Enterprise Linux (RHEL) on 64-bit IBM Z servers. It outlines the compatibility of each boot medium with different system types and indicates whether the zipl boot loader is used. This information helps you determine the most suitable boot medium for your specific environment. System type / Boot media Uses zipl boot loader z/VM KVM LPAR z/VM Reader No Yes N/A N/A SE or HMC (remote SFTP, FTPS, FTP server, DVD) No N/A N/A Yes DASD Yes Yes Yes Yes FCP SCSI LUNs Yes Yes Yes Yes FCP SCSI DVD Yes Yes Yes Yes N/A indicates that the boot medium is not applicable for the specified system type. 11.3. Supported environments and components for IBM Z servers The following tables provide information about the supported environments, network devices, machine types, and storage types for different system types when installing Red Hat Enterprise Linux (RHEL) on 64-bit IBM Z servers. Use these tables to identify the compatibility of various components with your specific system configuration. Table 11.1. Network device compatibility for system types Network device z/VM KVM LPAR Open Systems Adapter (OSA) Yes N/A Yes HiperSockets Yes N/A Yes LAN channel station (LCS) Yes N/A Yes virtio-net N/A Yes N/A RDMA over Converged Ethernet (RoCE) Yes Yes Yes N/A indicates that the component is not applicable for the specified system type. Table 11.2. Machine type compatibility for system types Machine type z/VM KVM LPAR ESA Yes N/A N/A s390-virtio-ccw N/A Yes N/A N/A indicates that the component is not applicable for the specified system type. Table 11.3. Storage type compatibility for system types Storage type z/VM KVM LPAR DASD Yes Yes Yes FCP SCSI Yes Yes [a] Yes virtio-blk N/A Yes N/A [a] Conditional support based on configuration N/A indicates that the component is not applicable for the specified system type. 11.4. Overview of installation process on 64-bit IBM Z servers You can install Red Hat Enterprise Linux on 64-bit IBM Z interactively or in unattended mode. Installation on 64-bit IBM Z differs from other architectures as it is typically performed over a network, and not from local media. The installation consists of three phases: Booting the installation Connect to the mainframe Customize the boot parameters Perform an initial program load (IPL), or boot from the media containing the installation program Connecting to the installation system From a local machine, connect to the remote 64-bit IBM Z system using SSH, and start the installation program using Virtual Network Computing (VNC) Completing the installation using the RHEL installation program 11.5. Boot media for installing RHEL on 64-bit IBM Z servers After establishing a connection with the mainframe, you need to perform an initial program load (IPL), or boot, from the medium containing the installation program. This document describes the most common methods of installing Red Hat Enterprise Linux on 64-bit IBM Z. In general, any method may be used to boot the Linux installation system, which consists of a kernel ( kernel.img ) and initial RAM disk ( initrd.img ) with parameters in the generic.prm file supplemented by user defined parameters. Additionally, a generic.ins file is loaded which determines file names and memory addresses for the initrd, kernel and generic.prm . The Linux installation system is also called the installation program in this book. You can use the following boot media only if Linux is to run as a guest operating system under z/VM: z/VM reader You can use the following boot media only if Linux is to run in LPAR mode: SE or HMC through a remote SFTP, FTPS or FTP server SE or HMC DVD You can use the following boot media for both z/VM and LPAR: DASD SCSI disk device that is attached through an FCP channel FCP-attached SCSI DVD If you use DASD or an FCP-attached SCSI disk device as boot media, you must have a configured zipl boot loader. 11.6. Customizing boot parameters Before the installation can begin, you must configure some mandatory boot parameters. When installing through z/VM, these parameters must be configured before you boot into the generic.prm file. When installing on an LPAR, the rd.cmdline parameter is set to ask by default, meaning that you will be given a prompt on which you can enter these boot parameters. In both cases, the required parameters are the same. All network configuration can either be specified by using a parameter file, or at the prompt. Installation source An installation source must always be configured. Use the inst.repo option to specify the package source for the installation. Network devices Network configuration must be provided if network access will be required during the installation. If you plan to perform an unattended (Kickstart-based) installation by using only local media such as a disk, network configuration can be omitted. ip= Use the ip= option for basic network configuration, and other options as required for RoCE configuration. rd.znet= Also use the rd.znet= kernel option, which takes a network protocol type, a comma delimited list of sub-channels, and, optionally, comma delimited sysfs parameter and value pairs for qeth devices. This parameter can be specified multiple times to activate multiple network devices. For example: When specifying multiple rd.znet boot options, only the last one is passed on to the kernel command line of the installed system. This does not affect the networking of the system since all network devices configured during installation are properly activated and configured at boot. The qeth device driver assigns the same interface name for Ethernet and Hipersockets devices: enc <device number> . The bus ID is composed of the channel subsystem ID, subchannel set ID, and device number, separated by dots; the device number is the last part of the bus ID, without leading zeroes and dots. For example, the interface name will be enca00 for a device with the bus ID 0.0.0a00 . net.naming-scheme= The udev service renames network devices to assign, preferably, consistent names. By using the net.naming-scheme= parameter, you can influence the naming scheme. For details, see Implementing consistent network interface naming . Note If you use Remote direct memory access (RDMA) over Converged Ethernet (RoCE) devices, it depends on multiple factors whether Red Hat Enterprise Linux (RHEL) assigns a predictable or unpredictable name. However, during the installation, RHEL always assigns an unpredictable name to RoCE devices that are enumerated by a function identifier (FID), but you can configure RHEL after the installation to assign a predictable name to these RoCE devices. For details about the factors that influence the naming of RoCE devices and how to configure a consistent name after the installation for RoCE devices enumerated by FID, see Determining a predictable RoCE device name on the IBM Z platform . Storage devices At least one storage device must always be configured for text mode installations. The rd.dasd= option takes a Direct Access Storage Device (DASD) adapter device bus identifier. For multiple DASDs, specify the parameter multiple times, or use a comma separated list of bus IDs. To specify a range of DASDs, specify the first and the last bus ID. For example: The rd.zfcp= option takes a SCSI over FCP (zFCP) adapter device bus identifier, a world wide port name (WWPN), and a FCP LUN, then activates the device. This parameter can be specified multiple times to activate multiple zFCP devices. For example: Since 8, a target world wide port name (WWPN) and an FCP LUN have to be provided only if the zFCP device is not configured in NPIV mode or when auto LUN scanning is disabled by the zfcp.allow_lun_scan=0 kernel module parameter. It provides access to all SCSI devices found in the storage area network attached to the FCP device with the specified bus ID. This parameter needs to be specified at least twice to activate multiple paths to the same disks. Kickstart options If you are using a Kickstart file to perform an automatic installation, you must always specify the location of the Kickstart file using the inst.ks= option. For an unattended, fully automatic Kickstart installation, the inst.cmdline option is also useful. An example customized generic.prm file containing all mandatory parameters look similar to the following example: Example 11.1. Customized generic.prm file Some installation methods also require a file with a mapping of the location of installation data in the file system of the DVD or FTP server and the memory locations where the data is to be copied. The file is typically named generic.ins , and contains file names for the initial RAM disk, kernel image, and parameter file ( generic.prm ) and a memory location for each file. An example generic.ins will look similar to the following example: Example 11.2. Sample generic.ins file A valid generic.ins file is provided by Red Hat along with all other files required to boot the installer. Modify this file only if you want to, for example, load a different kernel version than default. Additional resources For a list of all boot options to customize the installation program's behavior, see Boot options reference . 11.7. Parameters and configuration files on 64-bit IBM Z This section contains information about the parameters and configuration files on 64-bit IBM Z. 11.7.1. Required configuration file parameters on 64-bit IBM Z Several parameters are required and must be included in the parameter file. These parameters are also provided in the file generic.prm in directory images/ of the installation DVD. ro Mounts the root file system, which is a RAM disk, read-only. ramdisk_size= size Modifies the memory size reserved for the RAM disk to ensure that the Red Hat Enterprise Linux installation program fits within it. For example: ramdisk_size=40000 . The generic.prm file also contains the additional parameter cio_ignore=all,!condev . This setting speeds up boot and device detection on systems with many devices. The installation program transparently handles the activation of ignored devices. 11.7.2. 64-bit IBM z/VM configuration file Under z/VM, you can use a configuration file on a CMS-formatted disk. The purpose of the CMS configuration file is to save space in the parameter file by moving the parameters that configure the initial network setup, the DASD, and the FCP specification out of the parameter file. Each line of the CMS configuration file contains a single variable and its associated value, in the following shell-style syntax: variable = value . You must also add the CMSDASD and CMSCONFFILE parameters to the parameter file. These parameters point the installation program to the configuration file: CMSDASD= cmsdasd_address Where cmsdasd_address is the device number of a CMS-formatted disk that contains the configuration file. This is usually the CMS user's A disk. For example: CMSDASD=191 CMSCONFFILE= configuration_file Where configuration_file is the name of the configuration file. This value must be specified in lower case. It is specified in a Linux file name format: CMS_file_name . CMS_file_type . The CMS file REDHAT CONF is specified as redhat.conf . The CMS file name and the file type can each be from one to eight characters that follow the CMS conventions. For example: CMSCONFFILE=redhat.conf 11.7.3. Installation network, DASD and FCP parameters on 64-bit IBM Z These parameters can be used to automatically set up the preliminary network, and can be defined in the CMS configuration file. These parameters are the only parameters that can also be used in a CMS configuration file. All other parameters in other sections must be specified in the parameter file. NETTYPE=" type " Where type must be one of the following: qeth , lcs , or ctc . The default is qeth . Choose qeth for: OSA-Express features HiperSockets Virtual connections on z/VM, including VSWITCH and Guest LAN Select ctc for: Channel-to-channel network connections SUBCHANNELS=" device_bus_IDs " Where device_bus_IDs is a comma-separated list of two or three device bus IDs. The IDs must be specified in lowercase. Provides required device bus IDs for the various network interfaces: For example (a sample qeth SUBCHANNEL statement): PORTNO=" portnumber " You can add either PORTNO="0" (to use port 0) or PORTNO="1" (to use port 1 of OSA features with two ports per CHPID). LAYER2=" value " Where value can be 0 or 1 . Use LAYER2="0" to operate an OSA or HiperSockets device in layer 3 mode ( NETTYPE="qeth" ). Use LAYER2="1" for layer 2 mode. For virtual network devices under z/VM this setting must match the definition of the GuestLAN or VSWITCH to which the device is coupled. To use network services that operate on layer 2 (the Data Link Layer or its MAC sublayer) such as DHCP, layer 2 mode is a good choice. The qeth device driver default for OSA devices is now layer 2 mode. To continue using the default of layer 3 mode, set LAYER2="0" explicitly. VSWITCH=" value " Where value can be 0 or 1 . Specify VSWITCH="1" when connecting to a z/VM VSWITCH or GuestLAN, or VSWITCH="0" (or nothing at all) when using directly attached real OSA or directly attached real HiperSockets. MACADDR=" MAC_address " If you specify LAYER2="1" and VSWITCH="0" , you can optionally use this parameter to specify a MAC address. Linux requires six colon-separated octets as pairs lower case hex digits - for example, MACADDR=62:a3:18:e7:bc:5f . This is different from the notation used by z/VM. If you specify LAYER2="1" and VSWITCH="1" , you must not specify the MACADDR , because z/VM assigns a unique MAC address to virtual network devices in layer 2 mode. CTCPROT=" value " Where value can be 0 , 1 , or 3 . Specifies the CTC protocol for NETTYPE="ctc" . The default is 0 . HOSTNAME=" string " Where string is the host name of the newly-installed Linux instance. IPADDR=" IP " Where IP is the IP address of the new Linux instance. NETMASK=" netmask " Where netmask is the netmask. The netmask supports the syntax of a prefix integer (from 1 to 32) as specified in IPv4 classless interdomain routing (CIDR). For example, you can specify 24 instead of 255.255.255.0 , or 20 instead of 255.255.240.0 . GATEWAY=" gw " Where gw is the gateway IP address for this network device. MTU=" mtu " Where mtu is the Maximum Transmission Unit (MTU) for this network device. DNS=" server1:server2:additional_server_terms:serverN " Where " server1:server2:additional_server_terms:serverN " is a list of DNS servers, separated by colons. For example: SEARCHDNS=" domain1:domain2:additional_dns_terms:domainN " Where " domain1:domain2:additional_dns_terms:domainN " is a list of the search domains, separated by colons. For example: You only need to specify SEARCHDNS= if you specify the DNS= parameter. DASD= Defines the DASD or range of DASDs to configure for the installation. The installation program supports a comma-separated list of device bus IDs, or ranges of device bus IDs with the optional attributes ro , diag , erplog , and failfast . Optionally, you can abbreviate device bus IDs to device numbers with leading zeros stripped. Any optional attributes should be separated by colons and enclosed in parentheses. Optional attributes follow a device bus ID or a range of device bus IDs. The only supported global option is autodetect . This does not support the specification of non-existent DASDs to reserve kernel device names for later addition of DASDs. Use persistent DASD device names such as /dev/disk/by-path/name to enable transparent addition of disks later. Other global options such as probeonly , nopav , or nofcx are not supported by the installation program. Only specify those DASDs that need to be installed on your system. All unformatted DASDs specified here must be formatted after a confirmation later on in the installation program. Add any data DASDs that are not needed for the root file system or the /boot partition after installation. For example: FCP_ n =" device_bus_ID [ WWPN FCP_LUN ]" For FCP-only environments, remove the DASD= option from the CMS configuration file to indicate no DASD is present. Where: n is typically an integer value (for example FCP_1 or FCP_2 ) but could be any string with alphabetic or numeric characters or underscores. device_bus_ID specifies the device bus ID of the FCP device representing the host bus adapter (HBA) (for example 0.0.fc00 for device fc00). WWPN is the world wide port name used for routing (often in conjunction with multipathing) and is as a 16-digit hex value (for example 0x50050763050b073d ). FCP_LUN refers to the storage logical unit identifier and is specified as a 16-digit hexadecimal value padded with zeroes to the right (for example 0x4020400100000000 ). Note A target world wide port name (WWPN) and an FCP_LUN have to be provided if the zFCP device is not configured in NPIV mode, when auto LUN scanning is disabled by the zfcp.allow_lun_scan=0 kernel module parameter or when installing RHEL-8.6 or older releases. Otherwise only the device_bus_ID value is mandatory. These variables can be used on systems with FCP devices to activate FCP LUNs such as SCSI disks. Additional FCP LUNs can be activated during the installation interactively or by means of a Kickstart file. An example value looks similar to the following: Each of the values used in the FCP parameters (for example FCP_1 or FCP_2 ) are site-specific and are normally supplied by the FCP storage administrator. 11.7.4. Miscellaneous parameters on 64-bit IBM Z The following parameters can be defined in a parameter file but do not work in a CMS configuration file. rd.live.check Turns on testing of an ISO-based installation source; for example, when using inst.repo= with an ISO on local disk or mounted with NFS. inst.nompath Disables support for multipath devices. proxy=[ protocol ://][ username [: password ]@] host [: port ] Specify a proxy to use with installation over HTTP, HTTPS or FTP. inst.rescue Boot into a rescue system running from a RAM disk that can be used to fix and restore an installed system. inst.stage2= URL Specifies a path to a tree containing install.img , not to the install.img directly. Otherwise, follows the same syntax as inst.repo= . If inst.stage2 is specified, it typically takes precedence over other methods of finding install.img . However, if Anaconda finds install.img on local media, the inst.stage2 URL will be ignored. If inst.stage2 is not specified and install.img cannot be found locally, Anaconda looks to the location given by inst.repo= or method= . If only inst.stage2= is given without inst.repo= or method= , Anaconda uses whatever repos the installed system would have enabled by default for installation. Use the option multiple times to specify multiple HTTP, HTTPS or FTP sources. The HTTP, HTTPS or FTP paths are then tried sequentially until one succeeds: inst.syslog= IP/hostname [: port ] Sends log messages to a remote syslog server. The boot parameters described here are the most useful for installations and trouble shooting on 64-bit IBM Z, but only a subset of those that influence the installation program. 11.7.5. Sample parameter file and CMS configuration file on 64-bit IBM Z To change the parameter file, begin by extending the shipped generic.prm file. Example of generic.prm file: Example of redhat.conf file configuring a QETH network device (pointed to by CMSCONFFILE in generic.prm ): 11.7.6. Using parameter and configuration files on 64-bit IBM Z The 64-bit IBM Z architecture can use a customized parameter file to pass boot parameters to the kernel and the installation program. You need to change the parameter file if you want to: Install unattended with Kickstart. Choose non-default installation settings that are not accessible through the installation program's interactive user interface, such as rescue mode. The parameter file can be used to set up networking non-interactively before the installation program ( Anaconda ) starts. The kernel parameter file is limited to 3754 bytes plus an end-of-line character. The parameter file can be variable or fixed record format. Fixed record format increases the file size by padding each line up to the record length. Should you encounter problems with the installation program not recognizing all specified parameters in LPAR environments, you can try to put all parameters in one single line or start and end each line with a space character. The parameter file contains kernel parameters, such as ro , and parameters for the installation process, such as vncpassword=test or vnc . 11.8. Preparing an installation in a z/VM guest virtual machine Use the x3270 or c3270 terminal emulator, to log in to z/VM from other Linux systems, or use the IBM 3270 terminal emulator on the 64-bit IBM Z Hardware Management Console (HMC). If you are running Microsoft Windows operating system, there are several options available, and can be found through an internet search. A free native Windows port of c3270 called wc3270 also exists. Ensure you select machine type as ESA for your z/VM VMs, because selecting any other machine types might prevent installing RHEL. See the IBM documentation . Procedure Log on to the z/VM guest virtual machine chosen for the Linux installation. optional: If your 3270 connection is interrupted and you cannot log in again because the session is still active, you can replace the old session with a new one by entering the following command on the z/VM logon screen: + Replace user with the name of the z/VM guest virtual machine. Depending on whether an external security manager, for example RACF, is used, the logon command might vary. If you are not already running CMS (single-user operating system shipped with z/VM) in your guest, boot it now by entering the command: Be sure not to use CMS disks such as your A disk (often device number 0191) as installation targets. To find out which disks are in use by CMS, use the following query: You can use the following CP (z/VM Control Program, which is the z/VM hypervisor) query commands to find out about the device configuration of your z/VM guest virtual machine: Query the available main memory, which is called storage in 64-bit IBM Z terminology. Your guest should have at least 1 GiB of main memory. Query available network devices by type: osa OSA - CHPID type OSD, real or virtual (VSWITCH or GuestLAN), both in QDIO mode hsi HiperSockets - CHPID type IQD, real or virtual (GuestLAN type Hipers) lcs LCS - CHPID type OSE For example, to query all of the network device types mentioned above, run: Query available DASDs. Only those that are flagged RW for read-write mode can be used as installation targets: Query available FCP devices (vHBAs):

Commands:
rd.znet=qeth,0.0.0600,0.0.0601,0.0.0602,layer2=1,portno= <number>
rd.dasd=0.0.0200 rd.dasd=0.0.0202(ro),0.0.0203(ro:failfast),0.0.0205-0.0.0207
rd.zfcp=0.0.4000,0x5005076300C213e9,0x5022000000000000 rd.zfcp=0.0.4000
ro ramdisk_size=40000 cio_ignore=all,!condev inst.repo=http://example.com/path/to/repository rd.znet=qeth,0.0.0600,0.0.0601,0.0.0602,layer2=1,portno=0,portname=foo ip=192.168.17.115::192.168.17.254:24:foobar.systemz.example.com:enc600:none nameserver=192.168.17.1 rd.dasd=0.0.0200 rd.dasd=0.0.0202 rd.zfcp=0.0.4000,0x5005076300c213e9,0x5022000000000000 rd.zfcp=0.0.5000,0x5005076300dab3e9,0x5022000000000000 inst.ks=http://example.com/path/to/kickstart
images/kernel.img 0x00000000 images/initrd.img 0x02000000 images/genericdvd.prm 0x00010480 images/initrd.addrsize 0x00010408
qeth: SUBCHANNELS=" read_device_bus_id , write_device_bus_id , data_device_bus_id " lcs or ctc: SUBCHANNELS=" read_device_bus_id , write_device_bus_id "
SUBCHANNELS="0.0.f5f0,0.0.f5f1,0.0.f5f2"
DNS="10.1.2.3:10.3.2.1"
SEARCHDNS="subdomain.domain:domain"
DASD="eb1c,0.0.a000-0.0.a003,eb10-eb14(diag),0.0.ab1c(ro:diag)"
FCP_ n =" device_bus_ID [ WWPN FCP_LUN ]"
FCP_1="0.0.fc00 0x50050763050b073d 0x4020400100000000" FCP_2="0.0.4000"
inst.stage2=http://hostname/path_to_install_tree/ inst.stage2=http://hostname/path_to_install_tree/ inst.stage2=http://hostname/path_to_install_tree/
ro ramdisk_size=40000 cio_ignore=all,!condev CMSDASD="191" CMSCONFFILE="redhat.conf" inst.vnc inst.repo=http://example.com/path/to/dvd-contents
NETTYPE="qeth" SUBCHANNELS="0.0.0600,0.0.0601,0.0.0602" PORTNAME="FOOBAR" PORTNO="0" LAYER2="1" MACADDR="02:00:be:3a:01:f3" HOSTNAME="foobar.systemz.example.com" IPADDR="192.168.17.115" NETMASK="255.255.255.0" GATEWAY="192.168.17.254" DNS="192.168.17.1" SEARCHDNS="systemz.example.com:example.com" DASD="200-203"
logon user here
cp ipl cms
query disk
cp query virtual storage
cp query virtual osa
cp query virtual dasd
cp query virtual fcp