Repository: osbuild/osbuild-composer
Pull request #1188 (2021-01-29 15:10:09)

Add sysconfig stage
This pull request includes:

- [x] adequate testing for the new functionality or fixed issue
- [x] adequate documentation informing people about the change



The org.osbuild.sysconfig stage is now supported. Config updates can be made to the kernel and network files. Currently, only the default values are used for all image types in rhel84. The image-info script is updated to allow testing the sysconfig info.

The default values are:
```
  kernel:
    UPDATEDEFAULT=yes
    DEFAULTKERNEL=kernel
    
  network:
    NETWORKING=yes
    NOZEROCONF=yes
```

gicmo: Schutzi is unhappy because the osbuild requirements was not updated so the sysconfig stage is actually not there, hence you get missing schema errors.
gicmo: One other little other thing, in your commit you write:
```
Currently, only the default values are used for all image types in rhel84.
```
I stumbled a bit over `default`, since if they were the default we would not have to set them, right? 
jkozol: > I stumbled a bit over default, since if they were the default we would not have to set them, right?

@gicom I see what you're saying here. I changed the wording to just indicate that we are setting values for all rhel 8.4 image types.
gicmo: > @gicom

That's not my name! *sing* 😄 
ondrejbudai: Blocked until osbuild 24 is in both Fedora 32 and Fedora 33:

https://bodhi.fedoraproject.org/updates/FEDORA-2021-1f28f28ea7
https://bodhi.fedoraproject.org/updates/FEDORA-2021-1f070f8684
gicmo: > Blocked until osbuild 24 is in both Fedora 32 and Fedora 33:
> 
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-1f28f28ea7
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-1f070f8684

On its way now for F33, but needs karma or time for F32.
jkozol: Also blocked until osbuild 24 is in rhel 8.4 nightly.

Patch:
diff --git a/README.md b/README.md
index 4f6d581d2..1eb7630dc 100644
--- a/README.md
+++ b/README.md
@@ -45,7 +45,7 @@ provided infrastructure and services.
 
 The requirements for this project are:
 
- * `osbuild >= 23`
+ * `osbuild >= 24`
  * `systemd >= 244`
 
 At build-time, the following software is required:
diff --git a/Schutzfile b/Schutzfile
index 08d6581a2..b541882d1 100644
--- a/Schutzfile
+++ b/Schutzfile
@@ -9,7 +9,7 @@
   "rhel-8.3": {
     "dependencies": {
       "osbuild": {
-        "commit": "5aee7b9fa724575daa010a55cad0558fbb7b9ad1"
+        "commit": "20a142d8f9b8b5d0a69f4d91631dc94118d209ca"
       }
     },
     "dependants": {
diff --git a/docs/news/unreleased/osbuild-24.md b/docs/news/unreleased/osbuild-24.md
new file mode 100644
index 000000000..b5693f2b9
--- /dev/null
+++ b/docs/news/unreleased/osbuild-24.md
@@ -0,0 +1,5 @@
+# OSBuild: spec: update to osbuild version 24
+
+In order to add the newly supported sysconfig stage, the osbuild dependency 
+needs to be updated to version 24. This update is reflected in both the spec 
+file dependency and in the testing dependency.
\ No newline at end of file
diff --git a/docs/news/unreleased/sysconfig.md b/docs/news/unreleased/sysconfig.md
new file mode 100644
index 000000000..0df54ccf5
--- /dev/null
+++ b/docs/news/unreleased/sysconfig.md
@@ -0,0 +1,13 @@
+# RHEL 8.4: add support for org.osbuild.sysconfig stage
+
+The kernel and network sysconfigs need to have certain values set in RHEL 8.4.
+Currently, the following values are set for all image types in 8.4:
+
+  kernel:
+    UPDATEDEFAULT=yes
+    DEFAULTKERNEL=kernel
+    
+  network:
+    NETWORKING=yes
+    NOZEROCONF=yes
+    
diff --git a/internal/distro/rhel84/distro.go b/internal/distro/rhel84/distro.go
index d4f175313..1a5877b1e 100644
--- a/internal/distro/rhel84/distro.go
+++ b/internal/distro/rhel84/distro.go
@@ -329,6 +329,18 @@ func (t *imageType) pipeline(c *blueprint.Customizations, options distro.ImageOp
 
 	p.AddStage(osbuild.NewSELinuxStage(t.selinuxStageOptions()))
 
+	// These are the current defaults for the sysconfig stage. This can be changed to be image type exclusive if different configs are needed.
+	p.AddStage(osbuild.NewSysconfigStage(&osbuild.SysconfigStageOptions{
+		Kernel: osbuild.SysconfigKernelOptions{
+			UpdateDefault: true,
+			DefaultKernel: "kernel",
+		},
+		Network: osbuild.SysconfigNetworkOptions{
+			Networking: true,
+			NoZeroConf: true,
+		},
+	}))
+
 	if t.rpmOstree {
 		p.AddStage(osbuild.NewRPMOSTreeStage(&osbuild.RPMOSTreeStageOptions{
 			EtcGroupMembers: []string{
diff --git a/internal/osbuild/sysconfig_stage.go b/internal/osbuild/sysconfig_stage.go
new file mode 100644
index 000000000..d083f7c45
--- /dev/null
+++ b/internal/osbuild/sysconfig_stage.go
@@ -0,0 +1,25 @@
+package osbuild
+
+type SysconfigStageOptions struct {
+	Kernel  SysconfigKernelOptions  `json:"kernel,omitempty"`
+	Network SysconfigNetworkOptions `json:"network,omitempty"`
+}
+
+type SysconfigNetworkOptions struct {
+	Networking bool `json:"networking,omitempty"`
+	NoZeroConf bool `json:"no_zero_conf,omitempty"`
+}
+
+type SysconfigKernelOptions struct {
+	UpdateDefault bool   `json:"update_default,omitempty"`
+	DefaultKernel string `json:"default_kernel,omitempty"`
+}
+
+func (SysconfigStageOptions) isStageOptions() {}
+
+func NewSysconfigStage(options *SysconfigStageOptions) *Stage {
+	return &Stage{
+		Name:    "org.osbuild.sysconfig",
+		Options: options,
+	}
+}
diff --git a/osbuild-composer.spec b/osbuild-composer.spec
index ef09e55c4..39d09be9a 100644
--- a/osbuild-composer.spec
+++ b/osbuild-composer.spec
@@ -272,8 +272,8 @@ The core osbuild-composer binary. This is suitable both for spawning in containe
 Summary:    The worker for osbuild-composer
 Requires:   systemd
 Requires:   qemu-img
-Requires:   osbuild >= 23
-Requires:   osbuild-ostree >= 23
+Requires:   osbuild >= 24
+Requires:   osbuild-ostree >= 24
 
 # remove in F34
 Obsoletes: golang-github-osbuild-composer-worker < %{version}-%{release}
diff --git a/tools/image-info b/tools/image-info
index 61c84830e..cc874705f 100755
--- a/tools/image-info
+++ b/tools/image-info
@@ -358,6 +358,30 @@ def read_fstab(tree):
             result = sorted([line.split() for line in f if line and not line.startswith("#")])
     return result
 
+# Create a nested dictionary for all supported sysconfigs 
+def read_sysconfig(tree):
+    result = {}
+    sysconfig_paths = {
+        "kernel": f"{tree}/etc/sysconfig/kernel",
+        "network": f"{tree}/etc/sysconfig/network"
+    }
+    # iterate through supported configs
+    # based on https://github.com/osbuild/osbuild/blob/main/osbuild/util/osrelease.py#L17
+    for name, path in sysconfig_paths.items():
+        with contextlib.suppress(FileNotFoundError):
+            with open(path) as f:
+                # if file exists start with empty array of values
+                result[name] = {}
+                for line in f:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    if line[0] == "#":
+                        continue
+                    key, value = line.split("=", 1)
+                    result[name][key] = value.strip('"')
+    return result
+ 
 
 def append_filesystem(report, tree, *, is_ostree=False):
     if os.path.exists(f"{tree}/etc/os-release"):
@@ -389,6 +413,10 @@ def append_filesystem(report, tree, *, is_ostree=False):
         if fstab:
             report["fstab"] = fstab
 
+        sysconfig = read_sysconfig(tree)
+        if sysconfig:
+            report["sysconfig"] = sysconfig
+            
         with open(f"{tree}/etc/passwd") as f:
             report["passwd"] = sorted(f.read().strip().split("\n"))
 


Test patch:
diff --git a/internal/osbuild/sysconfig_stage_test.go b/internal/osbuild/sysconfig_stage_test.go
new file mode 100644
index 000000000..b1c30b024
--- /dev/null
+++ b/internal/osbuild/sysconfig_stage_test.go
@@ -0,0 +1,16 @@
+package osbuild
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestNewSysconfigStage(t *testing.T) {
+	expectedStage := &Stage{
+		Name:    "org.osbuild.sysconfig",
+		Options: &SysconfigStageOptions{},
+	}
+	actualStage := NewSysconfigStage(&SysconfigStageOptions{})
+	assert.Equal(t, expectedStage, actualStage)
+}
diff --git a/test/data/manifests/fedora_32-x86_64-qcow2-customize.json b/test/data/manifests/fedora_32-x86_64-qcow2-customize.json
index c204bad20..76419be2a 100644
--- a/test/data/manifests/fedora_32-x86_64-qcow2-customize.json
+++ b/test/data/manifests/fedora_32-x86_64-qcow2-customize.json
@@ -24,7 +24,7 @@
         }
       ],
       "customizations": {
-        "hosname": "my-host",
+        "hostname": "my-host",
         "kernel": {
           "append": "debug"
         },
@@ -3442,6 +3442,12 @@
             "keymap": "dvorak"
           }
         },
+        {
+          "name": "org.osbuild.hostname",
+          "options": {
+            "hostname": "my-host"
+          }
+        },
         {
           "name": "org.osbuild.timezone",
           "options": {
@@ -9274,6 +9280,7 @@
       "video:x:39:",
       "wheel:x:10:"
     ],
+    "hostname": "my-host",
     "image-format": "qcow2",
     "os-release": {
       "ANSI_COLOR": "0;34",
diff --git a/test/data/manifests/fedora_33-x86_64-qcow2-customize.json b/test/data/manifests/fedora_33-x86_64-qcow2-customize.json
index 06d7a9eb4..fb99ccb9a 100644
--- a/test/data/manifests/fedora_33-x86_64-qcow2-customize.json
+++ b/test/data/manifests/fedora_33-x86_64-qcow2-customize.json
@@ -24,7 +24,7 @@
         }
       ],
       "customizations": {
-        "hosname": "my-host",
+        "hostname": "my-host",
         "kernel": {
           "append": "debug"
         },
@@ -3495,7 +3495,7 @@
         {
           "name": "org.osbuild.hostname",
           "options": {
-            "hostname": "localhost.localdomain"
+            "hostname": "my-host"
           }
         },
         {
@@ -9443,7 +9443,7 @@
       "video:x:39:",
       "wheel:x:10:"
     ],
-    "hostname": "localhost.localdomain",
+    "hostname": "my-host",
     "image-format": "qcow2",
     "os-release": {
       "ANSI_COLOR": "0;38;2;60;110;180",
diff --git a/test/data/manifests/rhel_8-x86_64-qcow2-customize.json b/test/data/manifests/rhel_8-x86_64-qcow2-customize.json
index 78c3a37b1..7a47ab5d5 100644
--- a/test/data/manifests/rhel_8-x86_64-qcow2-customize.json
+++ b/test/data/manifests/rhel_8-x86_64-qcow2-customize.json
@@ -27,7 +27,7 @@
         }
       ],
       "customizations": {
-        "hosname": "my-host",
+        "hostname": "my-host",
         "kernel": {
           "append": "debug"
         },
@@ -3508,6 +3508,12 @@
             "keymap": "dvorak"
           }
         },
+        {
+          "name": "org.osbuild.hostname",
+          "options": {
+            "hostname": "my-host"
+          }
+        },
         {
           "name": "org.osbuild.timezone",
           "options": {
@@ -9649,6 +9655,7 @@
       "video:x:39:",
       "wheel:x:10:"
     ],
+    "hostname": "my-host",
     "image-format": "qcow2",
     "os-release": {
       "ANSI_COLOR": "0;31",
diff --git a/test/data/manifests/rhel_84-x86_64-ami-boot.json b/test/data/manifests/rhel_84-x86_64-ami-boot.json
index f37ed5ef7..2b390e5a4 100644
--- a/test/data/manifests/rhel_84-x86_64-ami-boot.json
+++ b/test/data/manifests/rhel_84-x86_64-ami-boot.json
@@ -3285,6 +3285,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -9700,6 +9713,16 @@
       "udisks2.service",
       "unbound-anchor.timer"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "New_York"
   }
 }
\ No newline at end of file
diff --git a/test/data/manifests/rhel_84-x86_64-openstack-boot.json b/test/data/manifests/rhel_84-x86_64-openstack-boot.json
index 62ffc1c95..8a8aad0dc 100644
--- a/test/data/manifests/rhel_84-x86_64-openstack-boot.json
+++ b/test/data/manifests/rhel_84-x86_64-openstack-boot.json
@@ -3528,6 +3528,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -10321,6 +10334,16 @@
       "udisks2.service",
       "unbound-anchor.timer"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "New_York"
   }
 }
\ No newline at end of file
diff --git a/test/data/manifests/rhel_84-x86_64-qcow2-boot.json b/test/data/manifests/rhel_84-x86_64-qcow2-boot.json
index 94605512a..5478748d6 100644
--- a/test/data/manifests/rhel_84-x86_64-qcow2-boot.json
+++ b/test/data/manifests/rhel_84-x86_64-qcow2-boot.json
@@ -3468,6 +3468,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -10168,6 +10181,16 @@
       "tuned.service",
       "unbound-anchor.timer"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "New_York"
   }
 }
\ No newline at end of file
diff --git a/test/data/manifests/rhel_84-x86_64-qcow2-customize.json b/test/data/manifests/rhel_84-x86_64-qcow2-customize.json
index 57fd4a47e..1d46d4f67 100644
--- a/test/data/manifests/rhel_84-x86_64-qcow2-customize.json
+++ b/test/data/manifests/rhel_84-x86_64-qcow2-customize.json
@@ -29,7 +29,7 @@
         }
       ],
       "customizations": {
-        "hosname": "my-host",
+        "hostname": "my-host",
         "kernel": {
           "append": "debug"
         },
@@ -3499,6 +3499,12 @@
             "keymap": "dvorak"
           }
         },
+        {
+          "name": "org.osbuild.hostname",
+          "options": {
+            "hostname": "my-host"
+          }
+        },
         {
           "name": "org.osbuild.timezone",
           "options": {
@@ -3567,6 +3573,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -9644,6 +9663,7 @@
       "video:x:39:",
       "wheel:x:10:"
     ],
+    "hostname": "my-host",
     "image-format": "qcow2",
     "os-release": {
       "ANSI_COLOR": "0;31",
@@ -10271,6 +10291,16 @@
       "tuned.service",
       "unbound-anchor.timer"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "London"
   }
 }
\ No newline at end of file
diff --git a/test/data/manifests/rhel_84-x86_64-vhd-boot.json b/test/data/manifests/rhel_84-x86_64-vhd-boot.json
index a46f6c34e..a3641f266 100644
--- a/test/data/manifests/rhel_84-x86_64-vhd-boot.json
+++ b/test/data/manifests/rhel_84-x86_64-vhd-boot.json
@@ -3499,6 +3499,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -10248,6 +10261,16 @@
       "unbound-anchor.timer",
       "waagent.service"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "New_York"
   }
 }
\ No newline at end of file
diff --git a/test/data/manifests/rhel_84-x86_64-vmdk-boot.json b/test/data/manifests/rhel_84-x86_64-vmdk-boot.json
index 5b52eb087..bf9e12ac9 100644
--- a/test/data/manifests/rhel_84-x86_64-vmdk-boot.json
+++ b/test/data/manifests/rhel_84-x86_64-vmdk-boot.json
@@ -3357,6 +3357,19 @@
           "options": {
             "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts"
           }
+        },
+        {
+          "name": "org.osbuild.sysconfig",
+          "options": {
+            "kernel": {
+              "update_default": true,
+              "default_kernel": "kernel"
+            },
+            "network": {
+              "networking": true,
+              "no_zero_conf": true
+            }
+          }
         }
       ],
       "assembler": {
@@ -9873,6 +9886,16 @@
       "vgauthd.service",
       "vmtoolsd.service"
     ],
+    "sysconfig": {
+      "kernel": {
+        "DEFAULTKERNEL": "kernel",
+        "UPDATEDEFAULT": "yes"
+      },
+      "network": {
+        "NETWORKING": "yes",
+        "NOZEROCONF": "yes"
+      }
+    },
     "timezone": "New_York"
   }
 }
\ No newline at end of file
diff --git a/tools/test-case-generators/generate-test-cases b/tools/test-case-generators/generate-test-cases
index 18b341c57..b806628a9 100755
--- a/tools/test-case-generators/generate-test-cases
+++ b/tools/test-case-generators/generate-test-cases
@@ -109,7 +109,7 @@ CUSTOMIZATIONS_BLUEPRINT =  {
         }
     ],
     "customizations": {
-        "hosname": "my-host",
+        "hostname": "my-host",
         "kernel": {
             "append": "debug"
         },