Repository: vmware/govmomi (go)

Issue:
vC Sim should bind mount BIOS UUID to DMI files when simulating lifecycle with containers
# Overview

This issue tracks a feature request for vC Simulator when it is simulating a VM's lifecycle with a container.

## Use case

The request is for vC Simulator to take the VM's BIOS UUID, ex. `04F419FE-8688-4066-A16D-CBC4193D5B93` and bind mount it to two files inside of the container:

1. `/sys/class/dmi/id/product_uuid`

    This file's content should mirror the value of the BIOS UUID

    ```shell
    $ cat /sys/class/dmi/id/product_uuid
    04F419FE-8688-4066-A16D-CBC4193D5B93
    ```

2. `/sys/class/dmi/id/product_serial`

    This file's content is a transformed version of the BIOS UUID.

    ```shell
    $ cat /sys/class/dmi/id/product_serial
    VMware-fe 19 f4 04 88 86 66 40-a1 6d cb c4 19 3d 5b 93
    ```

    * The UUID is prefixed with `VMware-`
    * The 32-character hexadecimal UUID string is formatted into distinct pairs of characters, each pair representing a byte value (`2^8`).
    * The [UUID should be encoded](https://en.wikipedia.org/wiki/Universally_unique_identifier#Encoding) where the first three components are little-endian, and the final two components are big-endian.
    * The 32 characters are separated by a `-` character after eight bytes (16 characters).
      * Please note that all characters before the `-` separator are little-endian, and all characters after the `-` separator are big-endian.

## Requirements

This feature is needed for the following reasons:

* The aforementioned files are read-only and cannot be modified once the guest is running. The only way to supply the correct data to these files when the guest is a container is to bind mount the data into the container (there is a similar feature for [setting these specific values when running a VM with VirtualBox or Fusion](https://github.com/vmware/simple-k8s-test-env/blob/1d251afbabbc27f9eab01cb70d9060e017734fc4/vagrant/Vagrantfile#L289-L302)).
* The VM lifecycle simulation feature of vC Sim is useful for performing integration or end-to-end (e2e) testing with Kubernetes because of the aforementioned files' relationship with the discovery of the _node ID_.
  * The node ID is used as the basis for the provider ID for:
    * The [Kubernetes cloud provider for vSphere](https://github.com/kubernetes/cloud-provider-vsphere)
    * The [Kubernetes cluster API provider for vSphere](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere)
  * Unless specified as a flag when starting the kubelet, Kubernetes determines a node's initial ID by reading the DMI information.
    1. The kubelet attempts to gather information about the node's status, [including the node's SystemUUID](https://github.com/kubernetes/kubernetes/blob/0599ca2bcfcae7d702f95284f3c2e2c2978c7772/pkg/kubelet/nodestatus/setters.go#L263). The method used to obtain this information depends on the cAdvisor `Interface`.
    1. The cAdvisor `Interface` [is initialized with a real system filesystem implementation](https://github.com/kubernetes/kubernetes/blob/9488fbef64afd580d053347e57819817e51ac0f9/pkg/kubelet/cadvisor/cadvisor_linux.go#L87) (`SysFs`).
    1. The cAdvisor `Interface` initializes a new `Manager` [with the `SysFs` instance](https://github.com/kubernetes/kubernetes/blob/9488fbef64afd580d053347e57819817e51ac0f9/pkg/kubelet/cadvisor/cadvisor_linux.go#L104).
    1. The cAdvisor `Manager` [gathers information about the machine](https://github.com/kubernetes/kubernetes/blob/3b440dfd5550b13278e40975a7cba13a3f129475/vendor/github.com/google/cadvisor/manager/manager.go#L188) with `machine.Info(SysFs,...)`.
    1. The `machine.Info` call uses the `SysFs` instance to [get the system's UUID via `sysinfo.GetSystemUUID`](https://github.com/kubernetes/kubernetes/blob/3b440dfd5550b13278e40975a7cba13a3f129475/vendor/github.com/google/cadvisor/machine/info.go#L138).
    1. The `sysinfo.GetSystemUUID` function is simply [a wrapper](https://github.com/kubernetes/kubernetes/blob/d1e8702d36947ccc23ad820eaae7f4afbaf0b058/vendor/github.com/google/cadvisor/utils/sysinfo/sysinfo.go#L201-L203) for the `SysFs` instance's `GetSystemUUID` function.
    1. And the real `SysFs` implementation of the `GetSystemUUID` function reads the system UUID via several methods, [starting with the DMI information](https://github.com/kubernetes/kubernetes/blob/c466fd9eaf31a6a12f901369d13129f6a8853da4/vendor/github.com/google/cadvisor/utils/sysfs/sysfs.go#L237-L249).

## Notes

* The Kubernetes cloud provider for vSphere [implements this feature with Kind for doing integration testing](https://github.com/kubernetes/cloud-provider-vsphere/tree/master/test/integration).

cc @dougm @imikushin @corrieb 

Pull request:
vcsim: bind mount BIOS UUID DMI files
For use within container simulated VMs

Fixes #1709

Patch:
diff --git a/simulator/container.go b/simulator/container.go
index f3051541..7b0175cd 100644
--- a/simulator/container.go
+++ b/simulator/container.go
@@ -17,13 +17,17 @@ limitations under the License.
 package simulator
 
 import (
+	"archive/tar"
 	"bytes"
+	"encoding/hex"
 	"encoding/json"
 	"fmt"
 	"log"
 	"os/exec"
 	"strings"
+	"time"
 
+	"github.com/google/uuid"
 	"github.com/vmware/govmomi/vim25/types"
 )
 
@@ -88,6 +92,46 @@ func (c *container) inspect(vm *VirtualMachine) error {
 	return nil
 }
 
+// createDMI writes BIOS UUID DMI files to a container volume
+func (c *container) createDMI(vm *VirtualMachine, name string) error {
+	cmd := exec.Command("docker", "run", "--rm", "-i", "-v", name+":"+"/"+name, "busybox", "tar", "-C", "/"+name, "-xf", "-")
+	stdin, err := cmd.StdinPipe()
+	if err != nil {
+		return err
+	}
+
+	err = cmd.Start()
+	if err != nil {
+		return err
+	}
+
+	tw := tar.NewWriter(stdin)
+
+	dmi := []struct {
+		name string
+		val  func(uuid.UUID) string
+	}{
+		{"product_uuid", productUUID},
+		{"product_serial", productSerial},
+	}
+
+	for _, file := range dmi {
+		val := file.val(vm.uid)
+		_ = tw.WriteHeader(&tar.Header{
+			Name:    file.name,
+			Size:    int64(len(val) + 1),
+			Mode:    0444,
+			ModTime: time.Now(),
+		})
+		_, _ = fmt.Fprintln(tw, val)
+	}
+
+	_ = tw.Close()
+	_ = stdin.Close()
+
+	return cmd.Wait()
+}
+
 // start runs the container if specified by the RUN.container extraConfig property.
 func (c *container) start(vm *VirtualMachine) {
 	if c.id != "" {
@@ -132,7 +176,14 @@ func (c *container) start(vm *VirtualMachine) {
 	}
 
 	run := append([]string{"docker", "run", "-d", "--name", vm.Name}, env...)
-	run = append(run, "--env", "VMX_CONFIG_UUID="+vm.Config.Uuid)
+
+	volume := fmt.Sprintf("vcsim-%s-%s", vm.Name, vm.uid)
+	if err := c.createDMI(vm, volume); err != nil {
+		log.Printf("%s: %s", vm.Name, err)
+		return
+	}
+	run = append(run, "-v", fmt.Sprintf("%s:%s:ro", volume, "/sys/class/dmi/id"))
+
 	args = append(run, args...)
 	cmd := exec.Command(shell, "-c", strings.Join(args, " "))
 	out, err := cmd.Output()
@@ -181,9 +232,52 @@ func (c *container) remove(vm *VirtualMachine) {
 		return
 	}
 
-	cmd := exec.Command("docker", "rm", "-f", c.id)
+	cmd := exec.Command("docker", "rm", "-v", "-f", c.id)
 	err := cmd.Run()
 	if err != nil {
 		log.Printf("%s %s: %s", vm.Name, cmd.Args, err)
 	}
 }
+
+// productSerial returns the uuid in /sys/class/dmi/id/product_serial format
+func productSerial(id uuid.UUID) string {
+	var dst [len(id)*2 + len(id) - 1]byte
+
+	j := 0
+	for i := 0; i < len(id); i++ {
+		hex.Encode(dst[j:j+2], id[i:i+1])
+		j += 3
+		if j < len(dst) {
+			s := j - 1
+			if s == len(dst)/2 {
+				dst[s] = '-'
+			} else {
+				dst[s] = ' '
+			}
+		}
+	}
+
+	return fmt.Sprintf("VMware-%s", string(dst[:]))
+}
+
+// productUUID returns the uuid in /sys/class/dmi/id/product_uuid format
+func productUUID(id uuid.UUID) string {
+	var dst [36]byte
+
+	hex.Encode(dst[0:2], id[3:4])
+	hex.Encode(dst[2:4], id[2:3])
+	hex.Encode(dst[4:6], id[1:2])
+	hex.Encode(dst[6:8], id[0:1])
+	dst[8] = '-'
+	hex.Encode(dst[9:11], id[5:6])
+	hex.Encode(dst[11:13], id[4:5])
+	dst[13] = '-'
+	hex.Encode(dst[14:16], id[7:8])
+	hex.Encode(dst[16:18], id[6:7])
+	dst[18] = '-'
+	hex.Encode(dst[19:23], id[8:10])
+	dst[23] = '-'
+	hex.Encode(dst[24:], id[10:])
+
+	return strings.ToUpper(string(dst[:]))
+}
diff --git a/simulator/object.go b/simulator/object.go
index e6ce2545..17041874 100644
--- a/simulator/object.go
+++ b/simulator/object.go
@@ -51,7 +51,12 @@ func SetCustomValue(ctx *Context, req *types.SetCustomValue) soap.HasFault {
 	return body
 }
 
-// newUUID returns a stable UUID based on input s
+// newUUID returns a stable UUID string based on input s
 func newUUID(s string) string {
-	return uuid.NewSHA1(uuid.NameSpaceOID, []byte(s)).String()
+	return sha1UUID(s).String()
+}
+
+// sha1UUID returns a stable UUID based on input s
+func sha1UUID(s string) uuid.UUID {
+	return uuid.NewSHA1(uuid.NameSpaceOID, []byte(s))
 }
diff --git a/simulator/virtual_machine.go b/simulator/virtual_machine.go
index 2a01b43f..9594dd4d 100644
--- a/simulator/virtual_machine.go
+++ b/simulator/virtual_machine.go
@@ -46,6 +46,7 @@ type VirtualMachine struct {
 	log string
 	sid int32
 	run container
+	uid uuid.UUID
 	imc *types.CustomizationSpec
 }
 
@@ -96,12 +97,13 @@ func NewVirtualMachine(parent types.ManagedObjectReference, spec *types.VirtualM
 	}
 
 	dsPath := path.Dir(spec.Files.VmPathName)
+	vm.uid = sha1UUID(spec.Files.VmPathName)
 
 	defaults := types.VirtualMachineConfigSpec{
 		NumCPUs:           1,
 		NumCoresPerSocket: 1,
 		MemoryMB:          32,
-		Uuid:              newUUID(spec.Files.VmPathName),
+		Uuid:              vm.uid.String(),
 		InstanceUuid:      newUUID(strings.ToUpper(spec.Files.VmPathName)),
 		Version:           esx.HardwareVersion,
 		Files: &types.VirtualMachineFileInfo{


Test patch:
diff --git a/govc/test/vcsim.bats b/govc/test/vcsim.bats
index a160a1c1..dd98e688 100755
--- a/govc/test/vcsim.bats
+++ b/govc/test/vcsim.bats
@@ -262,6 +262,19 @@ load test_helper
   run govc vm.destroy $vm
   assert_success
 
+  vm=DC0_C0_RP0_VM0
+  run govc vm.change -vm $vm -e RUN.container="busybox grep VMware- /sys/class/dmi/id/product_serial"
+  assert_success
+
+  run govc vm.power -on $vm
+  assert_success
+
+  run docker inspect -f '{{.State.ExitCode}}' $vm
+  assert_success "0"
+
+  run govc vm.destroy $vm
+  assert_success
+
   vm=DC0_C0_RP0_VM1
   run govc vm.change -vm $vm -e RUN.container="busybox sh -c 'sleep \$VMX_GUESTINFO_SLEEP'" -e guestinfo.sleep=500
   assert_success
diff --git a/simulator/container_test.go b/simulator/container_test.go
new file mode 100644
index 00000000..d00a263c
--- /dev/null
+++ b/simulator/container_test.go
@@ -0,0 +1,42 @@
+/*
+Copyright (c) 2019 VMware, Inc. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package simulator
+
+import (
+	"testing"
+
+	"github.com/google/uuid"
+)
+
+func TestEncodeDMI(t *testing.T) {
+	id := uuid.Must(uuid.Parse("423e36c8-723d-2b24-8590-0eb1d9180aa2"))
+
+	tests := []struct {
+		f      func(uuid.UUID) string
+		expect string
+	}{
+		{productSerial, "VMware-42 3e 36 c8 72 3d 2b 24-85 90 0e b1 d9 18 0a a2"},
+		{productUUID, "C8363E42-3D72-242B-8590-0EB1D9180AA2"},
+	}
+
+	for _, test := range tests {
+		val := test.f(id)
+		if val != test.expect {
+			t.Errorf("%q != %q", val, test.expect)
+		}
+	}
+}