Project: Metal³ (Provisioning / Automation & Configuration)
File: dev-setup.md

Install and launch minikube <https://kubernetes.io/docs/setup/minikube/> Create a namespace to host the operator ```bash kubectl create namespace baremetal-operator-system ``` Install cert-manager ```bash kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.4/cert-manager.yaml ``` Verify that it is deployed correctly. Note: don't move to the next step before you see all three pods in Running state. ```bash kubectl get pods --namespace cert-manager NAME READY STATUS RESTARTS AGE cert-manager-5c6866597-zw7kh 1/1 Running 0 2m cert-manager-cainjector-577f6d9fd7-tr77l 1/1 Running 0 2m cert-manager-webhook-787858fcdb-nlzsq 1/1 Running 0 2m ``` Install operator in the cluster ```bash eval $(go env) mkdir -p $GOPATH/src/github.com/metal3-io cd $GOPATH/src/github.com/metal3-io git clone https://github.com/metal3-io/baremetal-operator.git cd baremetal-operator kustomize build config/default | kubectl apply -f - ``` OR Launch the operator locally ```bash export OPERATOR_NAME=baremetal-operator export DEPLOYKERNELURL=http://172.22.0.1/images/ironic-python-agent.kernel export DEPLOYRAMDISKURL=http://172.22.0.1/images/ironic-python-agent.initramfs export IRONIC_ENDPOINT=http://localhost:6385/v1/ make run ``` Create the CR ```bash kubectl create namespace metal3 kubectl apply -f examples/example-host.yaml -n metal3 ``` In environments where Ironic is not available, and the only real need is to be able to have some test data, use the test fixture provisioner instead of the real Ironic provisioner by passing `-test-mode` to the operator when launching it. ```bash make run-test-mode ``` There is a script available that will run a set of containers locally using `podman` to stand up Ironic for development and testing. See `tools/runlocalironic.sh`. Note that this script may need customizations to some of the `podman run` commands, to include environment variables that configure the containers for your environment. All ironic related environment variables are set by default if they are not passed through the environment. The following environment variables can be passed to configure the ironic: HTTP_PORT - port used by httpd server (default 6180) PROVISIONING_IP - provisioning interface IP address to use for ironic, dnsmasq(dhcpd) and httpd (default 172.22.0.1) CLUSTERPROVISIONINGIP - cluster provisioning interface IP address (default 172.22.0.2) PROVISIONING_INTERFACE - interface to use for ironic, dnsmasq(dhcpd) and httpd (default ironicendpoint) CLUSTERDHCPRANGE - dhcp range to use for provisioning (default 172.22.0.10-172.22.0.100) DEPLOYKERNELURL - the URL of the kernel to deploy ironic-python-agent DEPLOYRAMDISKURL - the URL of the ramdisk to deploy ironic-python-agent IRONIC_ENDPOINT - the endpoint of the ironic CACHEURL - the URL of the cached images IRONICFASTTRACK - whether to enable fast_track provisioning or not (default true) IRONICKERNELPARAMS - Kernel parameters to pass to IPA (default console=ttyS0) IRONICINSPECTORVLAN_INTERFACES - VLAN interfaces included in introspection, `all` - all VLANs on all interfaces, using LLDP information (default), interface - all VLANs on an interface, using LLDP information,

- a particular VLAN interface, not using LLDP IRONICBOOTISO_SOURCE - where the boot iso image will be served from, possible values are: local (default), to download the image, prepare it and serve it from the conductor; http, to serve it directly from its HTTP URL. IPADOWNLOADENABLED - Enables the use of the Ironic Python Agent Downloader container to download IPA archive (default true) USELOCALIPA - Enables the use of locally supplied IPA archive. This condition is handled by BMO and this has effect only when IPADOWNLOADENABLED is "false", otherwise IPADOWNLOADENABLED takes precedence. (default false) LOCALIPAPATH - This has effect only when USELOCALIPA is set to "true", points to the directory where the IPA archive is located. This variable is handled by BMO. The variable should contain an arbitrary path pointing to the directory that contains the `ironic-python-agent.tar`. NOTES In case both `IPADOWNLOADENABLED` and `USELOCALIPA` are set to true then the IPA archive will be downloaded and the use of local IPA archive will be skipped. In case both `IPADOWNLOADENABLED` and `USELOCALIPA` are set to false then the IPA download/copy process won't start at all. Skipping the IPA download/copy process could help in avoiding unwanted overwrites of the "ironic-python-agent.tar" in case somebody is testing some functionality where the unwanted overwrite of the archive would cause issues. In case you want to run the local ironic containers with TLS and basic authentication enabled, you also need to export the following variables: IRONICCACERTFILE IRONICCERTFILE IRONICKEYFILE IRONIC_USERNAME IRONIC_PASSWORD The names of these variables are self explanatory. TLS variables expect the path of the corresponding certificate/key file as their value. Basic authentication variables expect the corresponding value as string. Note that, these variables do not have any default value. So if they are not set, the ironic container will run with TLS and basic authentication disabled. It is easy to use Tilt for BMO deployment. Once you have a local instance of Ironic running, just run ```sh make tilt-up ``` and clean it with ```sh make kind-reset ``` It is also possible to develop Baremetal Operator using Tilt with CAPM3. Please refer to and specially the In case the standard Tilt setup does not work as expected, you can use the following alternative method: Create a Kind Cluster: ```sh kind create cluster --name bmo ``` Install Cert-Manager: ```sh kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.2/cert-manager.yaml ``` Launch Tilt: ```sh tilt up ``` Virtinst, libvirt-clients, libvirt-daemon-system, and are required to to create BareMetalHosts this

The network and VBMC needed for making a BareMetalHosts can be initialized with ```sh tools/bmhtest/runlocalbmhtest_setup.sh ``` When Tilt is up, it is possible to make BareMetalHosts by pressing a button in the Tilt localhost interface. This is currently only supported for Unix based systems. This button runs the content of file ```sh tools/bmhtest/createbmh.sh <NAME> <VBMC_PORT> ``` and adds the values given to the button as arguments. Controlplane host can be created with ```sh tools/bmhtest/createbmh.sh <NAME> <VBMCPORT> <CONSUMER> <CONSUMERNAMESPACE> ``` The network, VBMC, and virtual machines can be cleaned with ```sh tools/bmhtest/cleanlocalbmhtest_setup.sh ``` In order to use VMs as hosts, they need to be connected to and the `bootMACAddress` field needs to be set to the MAC address of the network interface that will PXE boot. For example: ```yaml apiVersion: metal3.io/v1alpha1 kind: BareMetalHost metadata: name: worker-0 spec: online: true bmc: address: libvirt://192.168.122.1:6233/ credentialsName: worker-0-bmc-secret bootMACAddress: 00:73:49:3a:76:8e ``` The `make-virt-host` utility can be used to generate a YAML file for registering a host. It takes as input the name of the `virsh` domain and produces as output the basic YAML to register that host properly, with the boot MAC address and BMC address filled in. ```bash $ go run cmd/make-virt-host/main.go openshiftworker1 apiVersion: v1 kind: Secret metadata: name: openshift-worker-1-bmc-secret type: Opaque data: username: YWRtaW4= password: cGFzc3dvcmQ= apiVersion: metal3.io/v1alpha1 kind: BareMetalHost metadata: name: openshift-worker-1 spec: online: true bmc: address: libvirt://192.168.122.1:6234/ credentialsName: openshift-worker-1-bmc-secret bootMACAddress: 00:1a:74:74:e5:cf ``` The output can be passed directly to `kubectl apply` like this: ```bash go run cmd/make-virt-host/main.go openshiftworker1 | kubectl apply -f - ``` When the host is a controlplane host, include the `-consumer` and `-consumer-namespace` options to associate the host with the existing `Machine` object. ```bash $ go run cmd/make-virt-host/main.go -consumer ostest-controlplane-1 \ -consumer-namespace openshift-machine-api openshiftcontrolplane1 apiVersion: v1 kind: Secret metadata: name: openshift-controlplane-1-bmc-secret type: Opaque data: username: YWRtaW4= password: cGFzc3dvcmQ= apiVersion: metal3.io/v1alpha1 kind: BareMetalHost metadata: name: openshift-controlplane-1 spec: online: true bmc: address: libvirt://192.168.122.1:6231/ credentialsName: openshift-controlplane-1-bmc-secret bootMACAddress: 00:c9:a0:f2:e0:59 consumerRef: name: ostest-controlplane-1 namespace: openshift-machine-api ``` The `make-bm-worker` tool may be a more convenient way of creating YAML definitions for workers than editing the files directly. If deploying baremetal hosts you might want to consider setting IRONICKERNELPARAMS="", when deploying ironic as the default directs the console to ttyS0 and is lost in Bare Metal Hosts. ```bash $ go run cmd/make-bm-worker/main.go -address 1.2.3.4 \ -password password -user admin worker-99 apiVersion: v1 kind: Secret metadata: name: worker-99-bmc-secret type: Opaque data: username: YWRtaW4= password: cGFzc3dvcmQ= apiVersion: metal3.io/v1alpha1 kind: BareMetalHost metadata: name: worker-99 spec: online: true bmc: address: 1.2.3.4 credentialsName: worker-99-bmc-secret disableCertificateVerification: true ```