Repository: cobbler/cobbler

Issue:
Cobbler v3.3.0 does not honer the "auto_migrate_settings" setting
If I change the "auto_migrate_settings" to "false", and restart cobblerd, the settings.yaml 
is still overwritten and "auto_migrate_settings" changes to "true":
New settings.yaml file is always generated (and old one backed up) when "cobblerd" service restarts, no matter what is the value of "auto_migrate_settings".


Way to reproduce (removed yum and build-and-install-rpms.sh -output):

```bash
[root@localhost cobbler]# git show
commit d8f60bbf14a838c8c8a1dba98086b223e35fe70a (HEAD -> v3.3.0, tag: v3.3.0, origin/release33)
Merge: 36c2ba14 452e4714
Author: Enno G <matrixfueller@gmail.com>
Date:   Mon Sep 20 16:55:41 2021 +0200

    Merge pull request #2794 from cobbler/fix/file-rce
    
    Fix multiple security vulnerabilites

[root@localhost cobbler]# ./docker/rpms/build-and-install-rpms.sh el8 docker/rpms/CentOS_8/CentOS8.dockerfile
[root@localhost cobbler]# yum install rpm-build/cobbler-3.3.0-1.el8.noarch.rpm
[root@localhost cobbler]# cd /etc/cobbler/
[root@localhost cobbler]# sed -i 's/auto_migrate_settings:.*/auto_migrate_settings: false/' settings.yaml 
[root@localhost cobbler]# grep auto_migrate_settings settings.yaml 
auto_migrate_settings: false
[root@localhost cobbler]# ls -dl settings*
drwxr-xr-x. 2 root root   119 10.10. 18:28 settings.d
-rw-r--r--. 1 root root 22389 10.10. 18.33 settings.yaml
[root@localhost cobbler]# systemctl restart cobblerd
[root@localhost cobbler]# grep auto_migrate_settings settings.yaml 
auto_migrate_settings: true
[root@localhost cobbler]# ls -l settings*
-rw-r--r--  1 root root 22389 10.10. 18:33 settings_20211010_18-33-50.yaml
drwxr-xr-x. 2 root root   119 10.10. 18:28 settings.d
-rw-r--r--. 1 root root  4844 10.10. 18:33 settings.yaml
[root@localhost cobbler]# cobbler --version
Cobbler 3.3.0
  source: ?, ?
  build time: Sun Oct 10 15:26:12 2021
[root@localhost cobbler]# cat /etc/redhat-release 
Rocky Linux release 8.4 (Green Obsidian)
```







Discussion:
Okay the root cause of this is a design error in the system how I imagined it:

Currently we pass as a default value `True`, so we do migrate per default. So far this is fine. The problem is in that chain that the value from the `settings.yaml` file is read but not taken into consideration. Thus the value from the daemon is used as the only source of truth. Since we did add this key to the settings, we do want to respect the value from the settings.

Workaround: Use the CLI argument to the `cobblerd` executable to disable the settings auto-migration.
I define the desired behavior as:

- Default value is (if option not present anywhere): Do execute the auto-migration.
- Daemon Flag overpowers the value in `settings.yaml`.

Resolving patch:
diff --git a/cobbler/api.py b/cobbler/api.py
--- a/cobbler/api.py
+++ b/cobbler/api.py
@@ -28,6 +28,8 @@
 from pathlib import Path
 from typing import Dict, List, Optional, Union
 
+from schema import SchemaError
+
 from cobbler.actions import status, hardlink, sync, replicate, report, log, acl, check, reposync, mkloaders
 from cobbler.actions.buildiso.standalone import StandaloneBuildiso
 from cobbler.actions.buildiso.netboot import NetbootBuildiso
@@ -129,18 +131,26 @@ def __init__(self, is_cobblerd: bool = False, settingsfile_location: str = "/etc
 
     def __generate_settings(self, settings_path: Path,
                             execute_settings_automigration: bool = True) -> settings.Settings:
-        # Read in YAML file and get dict
         yaml_dict = settings.read_yaml_file(settings_path)
 
-        # Add or update auto migrate settings value
-        yaml_dict["auto_migrate_settings"] = execute_settings_automigration
-
-        # Take dict and use it in migrations
-        migrated_settings = settings.migrate(yaml_dict, settings_path)
+        if execute_settings_automigration is not None:
+            self.logger.info('Daemon flag overwriting other possible values from "settings.yaml" for automigration!')
+            yaml_dict["auto_migrate_settings"] = execute_settings_automigration
 
-        # After the migration is done, save result in the settings object
+        if yaml_dict.get("auto_migrate_settings", True):
+            self.logger.info("Automigration executed")
+            normalized_settings = settings.migrate(yaml_dict, settings_path)
+        else:
+            self.logger.info("Automigration NOT executed")
+            # In case we have disabled the auto-migration, we still check if the settings are valid.
+            try:
+                normalized_settings = settings.validate_settings(yaml_dict)
+            except SchemaError as error:
+                raise ValueError("Settings are invalid and auto-migration is disabled. Please correct this manually!") \
+                    from error
+        # Use normalized or migrated dict and create settings object
         new_settings = settings.Settings()
-        new_settings.from_dict(migrated_settings)
+        new_settings.from_dict(normalized_settings)
 
         # save to disk
         new_settings.save(settings_path)


Test patch:
diff --git a/tests/api/miscellaneous_test.py b/tests/api/miscellaneous_test.py
--- a/tests/api/miscellaneous_test.py
+++ b/tests/api/miscellaneous_test.py
@@ -1,10 +1,40 @@
+import logging
 from unittest.mock import create_autospec
 
+import pytest
+
+from cobbler import settings
 from cobbler.api import CobblerAPI
 from cobbler.actions.buildiso.netboot import NetbootBuildiso
 from cobbler.actions.buildiso.standalone import StandaloneBuildiso
 
 
+@pytest.mark.parametrize("input_automigration,result_migrate_count,result_validate_count", [
+    (None, 1, 2),
+    (True, 1, 2),
+    (False, 0, 3),
+])
+def test_settings_migration(caplog, mocker, input_automigration, result_migrate_count, result_validate_count):
+    # Arrange
+    caplog.set_level(logging.DEBUG)
+    # TODO: Create test where the YAML is missing the key
+    spy_migrate = mocker.spy(settings, "migrate")
+    spy_validate = mocker.spy(settings, "validate_settings")
+    # Override private class variables to have a clean slate on all runs
+    CobblerAPI._CobblerAPI__shared_state = {}
+    CobblerAPI._CobblerAPI__has_loaded = False
+
+    # Act
+    CobblerAPI(execute_settings_automigration=input_automigration)
+
+    # Assert
+    assert len(caplog.records) > 0
+    if input_automigration is not None:
+        assert 'Daemon flag overwriting other possible values from "settings.yaml" for automigration!' in caplog.text
+    assert spy_migrate.call_count == result_migrate_count
+    assert spy_validate.call_count == result_validate_count
+
+
 def test_buildiso(mocker):
     # Arrange
     netboot_stub = create_autospec(spec=NetbootBuildiso)
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -7,20 +7,20 @@
 import pytest
 
 
-def pytest_addoption(parser):
-    parser.addoption("-E", action="store", metavar="NAME", help="only run tests matching the environment NAME.")
-
-
-def pytest_configure(config):
-    # register an additional marker
-    config.addinivalue_line("markers", "env(name): mark test to run only on named environment")
-
-
 @contextmanager
 def does_not_raise():
     yield
 
 
+@pytest.fixture(scope="function", autouse=True)
+def reset_settings_yaml(tmp_path):
+    filename = "settings.yaml"
+    filepath = "/etc/cobbler/%s" % filename
+    shutil.copy(filepath, tmp_path.joinpath(filename))
+    yield
+    shutil.copy(tmp_path.joinpath(filename), filepath)
+
+
 @pytest.fixture(scope="function")
 def create_testfile(tmp_path):
     def _create_testfile(filename):