Repository: OSInside/kiwi

Issue:
Disable timeout for checksum test failure
We have noticed that if the checksum test at the end of the installation from an ISO built using kiwi fails, the installation process will wait on the checksum test failure screen for user confirmation for a short period of time (less than 1 minute), but eventually times out and will continue booting into the OS. This means that unless someone is sitting through the whole installation process (which can take hours depending on the ISO size and hardware specs) and watches the checksum verification process and makes sure it passes, the checksum verification can actually fail because of some data corruption that is not big enough to break the OS boot, and by the time user gets back to the system, the checksum test failure screen could have timed out and the corrupted OS installation could have booted up, leaving the user confused as to why there are random failures happening across different parts of the OS.

Is there a way to disable the timeout of the checksum failure screen, and force the installation to stall on that screen until the failure is acknowledged through user input?

Discussion:
@schaefi: The timeout value was a fixed value of 60 seconds. I created a PR which allows you to set

```xml
<type ... kernelcmdline="rd.kiwi.dialog.timeout=off" .../>
```@zohourih: That is very helpful, thank you.@zohourih: @schaefi By the way, does this disable the timeout just for case of checksum failure, or does it also disable the timeout for successful verification?@schaefi: > [@schaefi](https://github.com/schaefi) By the way, does this disable the timeout just for case of checksum failure, or does it also disable the timeout for successful verificastion?

I think I can't follow you. In the process of calculating the checksum there is no timeout. Once the calculation is done the calculated checksum value is compared with the expected one. If there is a mismatch you see that error dialog `Image checksum test failed` and without any dialog timeout setting it will disappear after 60sec and try to boot and pray. With `rd.kiwi.dialog.timeout=off` the dialog with the error information will stay. I thought this is what you requested, no ?

If there is no error and the checksum matches, we directly boot into the system, there is no dialog then

@schaefi: One thing to mention `rd.kiwi.dialog.timeout=` will affect all dialogs that uses a timeout in the process of your deployment. It's not specific to the checksum verification. I believe this is not an issue as we use timeout dialogs only for `show_log_and_quit` and `report_and_quit` which are only used in error condition cases.

If rd.debug is set all dialogs will end in a shell, for debugging. So rd.debug in combination with rd.kiwi.dialog.timeout= will have no effect@zohourih: > If there is no error and the checksum matches, we directly boot into the system, there is no dialog then

If that is the case, it should be fine. I had a vague recollection that there is also a dialogue on successful verification which immediately exits, but it might be a false memory. The behavior I expect is indeed to boot directly into the OS upon successful verification, but wait indefinitely for user input if the verification fails.
@schaefi: > > If there is no error and the checksum matches, we directly boot into the system, there is no dialog then
> 
> If that is the case, it should be fine. I had a vague recollection that there is also a dialogue on successful verification which immediately exits, but it might be a false memory. The behavior I expect is indeed to boot directly into the OS upon successful verification, but wait indefinitely for user input if the verification fails.

yep that's how it will work.

* See https://github.com/OSInside/kiwi/blob/main/dracut/modules.d/90kiwi-dump/kiwi-dump-image.sh#L366

There is no additional dialog unless you have specified that in kiwi. See:

* https://github.com/OSInside/kiwi/blob/main/dracut/modules.d/99kiwi-dump-reboot/kiwi-dump-reboot-system.sh#L23

and those ask_and_reboot or ask_and_shutdown dialogs are not timeout based.

Solution patch:
diff --git a/doc/source/concept_and_workflow/customize_the_boot_process.rst b/doc/source/concept_and_workflow/customize_the_boot_process.rst
index e7b9606d..acbab806 100644
--- a/doc/source/concept_and_workflow/customize_the_boot_process.rst
+++ b/doc/source/concept_and_workflow/customize_the_boot_process.rst
@@ -194,6 +194,12 @@ the available kernel boot parameters for these modules:
   Activates the debug log file for the {kiwi} part of
   the boot process in :file:`/run/initramfs/log/boot.kiwi`.
 
+``rd.kiwi.dialog.timeout=seconds|off``
+  Sets a timeout value for dialogs invoked by kiwi dracut
+  modules. By default the timeout is set to 60 seconds.
+  If set to the special value `off`, the dialog will never
+  timeout.
+
 ``rd.kiwi.install.pxe``
   Instructs an OEM installation image to lookup the system
   image on a remote location specified in `rd.kiwi.install.image`.
diff --git a/doc/source/image_description/elements.rst b/doc/source/image_description/elements.rst
index b084793b..d8a87049 100644
--- a/doc/source/image_description/elements.rst
+++ b/doc/source/image_description/elements.rst
@@ -1729,7 +1729,7 @@ is installed using the `tar` program. Thus the file name is
 expected to be a tar archive. The compression of the archive is
 detected automatically by the tar program. The optional target_dir
 attribute can be used to specify a target directory to unpack the
-archive.
+archive inside the image root tree.
 
 <packages><ignore>
 ~~~~~~~~~~~~~~~~~~
diff --git a/dracut/modules.d/99kiwi-lib/kiwi-dialog-lib.sh b/dracut/modules.d/99kiwi-lib/kiwi-dialog-lib.sh
index f343e680..2e32ac8b 100644
--- a/dracut/modules.d/99kiwi-lib/kiwi-dialog-lib.sh
+++ b/dracut/modules.d/99kiwi-lib/kiwi-dialog-lib.sh
@@ -126,11 +126,25 @@ function _fbiterm_ok {
     return 0
 }
 
+function dialog_timeout {
+    local timeout=60
+    custom_timeout=$(getarg "rd.kiwi.dialog.timeout=")
+    [ -n "${custom_timeout}" ] && timeout="${custom_timeout}"
+    echo "${timeout}"
+}
+
 function show_log_and_quit {
     local text_message="$1"
     local log_file="$2"
-    run_dialog --timeout 60 --backtitle "\"${text_message}\"" \
-        --textbox "${log_file}" 15 80
+    local timeout
+    timeout=$(dialog_timeout)
+    if [ "${timeout}" = "off" ];then
+        run_dialog --backtitle "\"${text_message}\"" \
+            --textbox "${log_file}" 15 80
+    else
+        run_dialog --timeout "${timeout}" --backtitle "\"${text_message}\"" \
+            --textbox "${log_file}" 15 80
+    fi
     if getargbool 0 rd.debug; then
         die "${text_message}"
     else
@@ -140,7 +154,13 @@ function show_log_and_quit {
 
 function report_and_quit {
     local text_message="$1"
-    run_dialog --timeout 60 --msgbox "\"${text_message}\"" 5 80
+    local timeout
+    timeout=$(dialog_timeout)
+    if [ "${timeout}" = "off" ];then
+        run_dialog --msgbox "\"${text_message}\"" 5 80
+    else
+        run_dialog --timeout "${timeout}" --msgbox "\"${text_message}\"" 5 80
+    fi
     if getargbool 0 rd.debug; then
         die "${text_message}"
     else
diff --git a/kiwi/bootloader/config/grub2.py b/kiwi/bootloader/config/grub2.py
index 51f2da3c..c81c0461 100644
--- a/kiwi/bootloader/config/grub2.py
+++ b/kiwi/bootloader/config/grub2.py
@@ -610,7 +610,7 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
                     config_file, grub_config_file_for_efi_boot
                 )
             )
-            shutil.copy(
+            shutil.copy2(
                 config_file, grub_config_file_for_efi_boot
             )
 
@@ -685,10 +685,10 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
         if target_type and os.path.exists(zipl2grub_config_file):
             if os.path.exists(zipl2grub_config_file_orig):
                 # reset the original template file first
-                shutil.copy(zipl2grub_config_file_orig, zipl2grub_config_file)
+                shutil.copy2(zipl2grub_config_file_orig, zipl2grub_config_file)
             else:
                 # no copy of the original template, create it
-                shutil.copy(zipl2grub_config_file, zipl2grub_config_file_orig)
+                shutil.copy2(zipl2grub_config_file, zipl2grub_config_file_orig)
             with open(zipl2grub_config_file) as zipl_config_file:
                 zipl_config = zipl_config_file.read()
                 zipl_config = re.sub(
@@ -1069,7 +1069,7 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
             log.debug(
                 f'Copy earlyboot to media path: {early_boot_script_on_media}'
             )
-            shutil.copy(
+            shutil.copy2(
                 early_boot_script, early_boot_script_on_media
             )
         mkimage_call = Command.run(
@@ -1099,7 +1099,7 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
                 f'Copy grub image to media path: {efi_image_media_file}'
             )
             Path.create(os.path.dirname(efi_image_media_file))
-            shutil.copy(efi_image_root_file, efi_image_media_file)
+            shutil.copy2(efi_image_root_file, efi_image_media_file)
 
     def _create_efi_config_search(self, uuid=None, mbrid=None):
         efi_boot_config = self.efi_boot_path + '/grub.cfg'
@@ -1130,7 +1130,7 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
             Path.create(
                 os.path.dirname(early_boot_script_on_media)
             )
-            shutil.copy(
+            shutil.copy2(
                 early_boot_script, early_boot_script_on_media
             )
         mkimage_call = Command.run(
@@ -1166,7 +1166,7 @@ class BootLoaderConfigGrub2(BootLoaderConfigBase):
                 f'Copy grub image to media path: {bios_image_media_file}'
             )
             Path.create(os.path.dirname(bios_image_media_file))
-            shutil.copy(bios_image_root_file, bios_image_media_file)
+            shutil.copy2(bios_image_root_file, bios_image_media_file)
 
     def _create_early_boot_script_for_uuid_search(self, filename, uuid):
         with open(filename, 'w') as early_boot:
diff --git a/kiwi/bootloader/install/grub2.py b/kiwi/bootloader/install/grub2.py
index e1701576..ac18c888 100644
--- a/kiwi/bootloader/install/grub2.py
+++ b/kiwi/bootloader/install/grub2.py
@@ -18,6 +18,7 @@
 import glob
 import os
 import re
+import shutil
 import logging
 from contextlib import ExitStack
 
@@ -400,17 +401,13 @@ class BootLoaderInstallGrub2(BootLoaderInstallBase):
                 [grub2_install, '.orig']
             )
             grub2_install_noop = '/bin/true'
-            Command.run(
-                [
-                    'chroot', root_path,
-                    'cp', '-p', grub2_install, grub2_install_backup
-                ]
+            shutil.copy2(
+                os.path.normpath(f'{root_path}/{grub2_install}'),
+                os.path.normpath(f'{root_path}/{grub2_install_backup}')
             )
-            Command.run(
-                [
-                    'chroot', root_path,
-                    'cp', grub2_install_noop, grub2_install
-                ]
+            shutil.copy2(
+                os.path.normpath(f'{root_path}/{grub2_install_noop}'),
+                os.path.normpath(f'{root_path}/{grub2_install}')
             )
 
     def _enable_grub2_install(self, root_path):
diff --git a/kiwi/system/setup.py b/kiwi/system/setup.py
index 7d6f6c17..353b0be2 100644
--- a/kiwi/system/setup.py
+++ b/kiwi/system/setup.py
@@ -284,7 +284,8 @@ class SystemSetup:
         overlay_archive = self.description_dir + '/root.tar.gz'
         if os.path.exists(overlay_directory):
             self._sync_overlay_files(
-                overlay_directory, follow_links, preserve_owner_group
+                f'{os.path.normpath(overlay_directory)}/',
+                follow_links, preserve_owner_group
             )
         elif os.path.exists(overlay_archive):
             log.info('Extracting user defined files from archive to image tree')
@@ -355,7 +356,7 @@ class SystemSetup:
                 ['chroot', self.root_dir, 'chkstat', '--system', '--set']
             )
         else:
-            log.warning(
+            log.debug(
                 'chkstat not found in image. File Permissions Check skipped'
             )
 
@@ -587,7 +588,7 @@ class SystemSetup:
             Command.run(
                 [
                     'chroot', self.root_dir, 'setfiles',
-                    '-F', '-p', '-c', self._find_selinux_policy_file(
+                    '-T0', '-F', '-p', '-c', self._find_selinux_policy_file(
                         self.xml_state.build_type.get_selinux_policy() or 'targeted'
                     )
                 ] + exclude + [


Test patch:
diff --git a/test/data/example_config_target_dir.xml b/test/data/example_config_target_dir.xml
index f90e5bcd..5e6d6ad7 100644
--- a/test/data/example_config_target_dir.xml
+++ b/test/data/example_config_target_dir.xml
@@ -214,6 +214,7 @@
         <namedCollection name="bootstrap-collection"/>
         <product name="kiwi"/>
         <archive name="bootstrap.tgz" target_dir="foo"/>
+        <archive name="some.tgz" target_dir="/etc"/>
     </packages>
     <packages type="delete">
         <package name="kernel-debug"/>
diff --git a/test/unit/bootloader/config/grub2_test.py b/test/unit/bootloader/config/grub2_test.py
index 6df953f6..486ab526 100644
--- a/test/unit/bootloader/config/grub2_test.py
+++ b/test/unit/bootloader/config/grub2_test.py
@@ -173,9 +173,9 @@ class TestBootLoaderConfigGrub2:
     @patch('kiwi.bootloader.config.grub2.DataSync')
     @patch('kiwi.bootloader.config.grub2.Path.which')
     @patch('os.path.exists')
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     def test_setup_install_boot_images_raises_no_efigrub(
-        self, mock_shutil_copy, mock_exists, mock_Path_which,
+        self, mock_shutil_copy2, mock_exists, mock_Path_which,
         mock_sync, mock_command, mock_grub, mock_shim
     ):
         Defaults.set_platform_name('x86_64')
@@ -349,14 +349,14 @@ class TestBootLoaderConfigGrub2:
         ]
 
     @patch('glob.iglob')
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     @patch('kiwi.bootloader.config.grub2.Path.create')
     @patch('kiwi.bootloader.config.grub2.Command.run')
     @patch('kiwi.bootloader.config.grub2.Defaults.get_signed_grub_loader')
     @patch('kiwi.bootloader.config.grub2.Path.which')
     def test_copy_grub_config_to_efi_path(
         self, mock_Path_which, mock_get_signed_grub_loader, mock_Command,
-        mock_Path_create, mock_shutil_copy, mock_glob
+        mock_Path_create, mock_shutil_copy2, mock_glob
     ):
         mock_get_signed_grub_loader.return_value = None
         mock_glob.return_value = []
@@ -381,12 +381,12 @@ class TestBootLoaderConfigGrub2:
         mock_Path_create.assert_called_once_with(
             'root_dir/EFI/BOOT'
         )
-        mock_shutil_copy.assert_called_once_with(
+        mock_shutil_copy2.assert_called_once_with(
             'config_file', 'root_dir/EFI/BOOT/grub.cfg'
         )
 
         # 3. run: check with vendor directory read from grub binary
-        mock_shutil_copy.reset_mock()
+        mock_shutil_copy2.reset_mock()
         mock_Path_create.reset_mock()
         strings_out = Mock()
         strings_out.output = '/EFI/ubuntu'
@@ -401,12 +401,12 @@ class TestBootLoaderConfigGrub2:
         mock_Path_create.assert_called_once_with(
             'root_dir/EFI/ubuntu'
         )
-        mock_shutil_copy.assert_called_once_with(
+        mock_shutil_copy2.assert_called_once_with(
             'config_file', 'root_dir/EFI/ubuntu/grub.cfg'
         )
 
         # 4. run: check with installed vendor directory
-        mock_shutil_copy.reset_mock()
+        mock_shutil_copy2.reset_mock()
         mock_Path_create.reset_mock()
         mock_get_signed_grub_loader.return_value = None
         mock_glob.return_value = ['root_dir/EFI/fedora/shim.efi']
@@ -417,15 +417,15 @@ class TestBootLoaderConfigGrub2:
         mock_Path_create.assert_called_once_with(
             'root_dir/EFI/fedora'
         )
-        mock_shutil_copy.assert_called_once_with(
+        mock_shutil_copy2.assert_called_once_with(
             'config_file', 'root_dir/EFI/fedora/grub.cfg'
         )
 
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     @patch('os.path.exists')
     @patch('kiwi.bootloader.config.grub2.Command.run')
     def test_setup_zipl2grub_conf_512_byte_target(
-        self, mock_Command_run, mock_exists, mock_shutil_copy
+        self, mock_Command_run, mock_exists, mock_shutil_copy2
     ):
         path_return_values = [True, False]
 
@@ -457,26 +457,26 @@ class TestBootLoaderConfigGrub2:
                 '    targetblocksize = 512\n' \
                 '    targetoffset = 2048' \
                 in file_handle.write.call_args[0][0]
-        mock_shutil_copy.assert_called_once_with(
+        mock_shutil_copy2.assert_called_once_with(
             'root_dir/etc/default/zipl2grub.conf.in',
             'root_dir/etc/default/zipl2grub.conf.in.orig'
         )
         path_return_values = [True, True]
-        mock_shutil_copy.reset_mock()
+        mock_shutil_copy2.reset_mock()
         with patch('builtins.open', create=True) as mock_open:
             file_handle = mock_open.return_value.__enter__.return_value
             file_handle.read.return_value = zipl_config
             self.bootloader._setup_zipl2grub_conf()
-        mock_shutil_copy.assert_called_once_with(
+        mock_shutil_copy2.assert_called_once_with(
             'root_dir/etc/default/zipl2grub.conf.in.orig',
             'root_dir/etc/default/zipl2grub.conf.in'
         )
 
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     @patch('os.path.exists')
     @patch('kiwi.bootloader.config.grub2.Command.run')
     def test_setup_zipl2grub_conf_4096_byte_target(
-        self, mock_Command_run, mock_exists, mock_shutil_copy
+        self, mock_Command_run, mock_exists, mock_shutil_copy2
     ):
         path_return_values = [True, False]
         command_return_values = [
@@ -1754,9 +1754,9 @@ class TestBootLoaderConfigGrub2:
     @patch('kiwi.bootloader.config.grub2.Path.create')
     @patch('kiwi.bootloader.config.grub2.DataSync')
     @patch('os.path.exists')
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     def test_setup_install_boot_images_ppc(
-        self, mock_shutil_copy, mock_exists, mock_sync,
+        self, mock_shutil_copy2, mock_exists, mock_sync,
         mock_Path_create, mock_Path_which, mock_command,
         mock_get_grub_bios_core_loader, mock_get_unsigned_grub_loader,
         mock_get_boot_path
@@ -1817,9 +1817,9 @@ class TestBootLoaderConfigGrub2:
     @patch('kiwi.bootloader.config.grub2.Path.create')
     @patch('kiwi.bootloader.config.grub2.DataSync')
     @patch('os.path.exists')
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     def test_setup_install_boot_images_efi(
-        self, mock_shutil_copy, mock_exists, mock_sync,
+        self, mock_shutil_copy2, mock_exists, mock_sync,
         mock_Path_create, mock_Path_which, mock_command,
         mock_get_grub_bios_core_loader, mock_get_unsigned_grub_loader,
         mock_get_boot_path
@@ -1959,7 +1959,7 @@ class TestBootLoaderConfigGrub2:
             call(exclude=['*.module'], options=['-a']),
             call(exclude=['*.module'], options=['-a'])
         ]
-        assert mock_shutil_copy.call_args_list == [
+        assert mock_shutil_copy2.call_args_list == [
             call(
                 'boot_dir/boot/grub2/earlyboot.cfg',
                 'root_dir/boot/grub2/earlyboot.cfg'
@@ -2229,9 +2229,9 @@ class TestBootLoaderConfigGrub2:
     @patch('kiwi.bootloader.config.grub2.DataSync')
     @patch('os.path.exists')
     @patch('kiwi.defaults.Defaults.get_grub_path')
-    @patch('shutil.copy')
+    @patch('shutil.copy2')
     def test_setup_install_boot_images_with_theme_not_existing(
-        self, mock_shutil_copy, mock_get_grub_path,
+        self, mock_shutil_copy2, mock_get_grub_path,
         mock_exists, mock_sync, mock_Path_which, mock_command
     ):
         Defaults.set_platform_name('x86_64')
diff --git a/test/unit/bootloader/install/grub2_test.py b/test/unit/bootloader/install/grub2_test.py
index aeebda16..73f17d23 100644
--- a/test/unit/bootloader/install/grub2_test.py
+++ b/test/unit/bootloader/install/grub2_test.py
@@ -403,8 +403,9 @@ class TestBootLoaderInstallGrub2:
     @patch('kiwi.bootloader.install.grub2.MountManager')
     @patch('os.path.exists')
     @patch('os.access')
+    @patch('shutil.copy2')
     def test_secure_boot_install(
-        self, mock_access, mock_exists,
+        self, mock_shutil_copy2, mock_access, mock_exists,
         mock_mount_manager, mock_command
     ):
         mock_access.return_value = True
@@ -419,16 +420,18 @@ class TestBootLoaderInstallGrub2:
 
         self.bootloader.secure_boot_install()
 
+        assert mock_shutil_copy2.call_args_list == [
+            call(
+                'tmp_root/usr/sbin/grub2-install',
+                'tmp_root/usr/sbin/grub2-install.orig'
+            ),
+            call(
+                'tmp_root/bin/true',
+                'tmp_root/usr/sbin/grub2-install'
+            )
+        ]
+
         assert mock_command.call_args_list == [
-            call([
-                'chroot', 'tmp_root',
-                'cp', '-p', '/usr/sbin/grub2-install',
-                '/usr/sbin/grub2-install.orig'
-            ]),
-            call([
-                'chroot', 'tmp_root',
-                'cp', '/bin/true', '/usr/sbin/grub2-install'
-            ]),
             call([
                 'chroot', 'tmp_root', 'shim-install', '--removable',
                 '/dev/some-device'
diff --git a/test/unit/system/prepare_test.py b/test/unit/system/prepare_test.py
index 48dc7170..3908f29f 100644
--- a/test/unit/system/prepare_test.py
+++ b/test/unit/system/prepare_test.py
@@ -399,7 +399,10 @@ class TestSystemPrepare:
 
         self.system.install_bootstrap(self.manager)
 
-        tar.extract.assert_called_once_with('root_dir/foo')
+        assert tar.extract.call_args_list == [
+            call('root_dir/foo'),
+            call('root_dir/etc')
+        ]
 
     @patch('kiwi.xml_state.XMLState.get_bootstrap_packages_sections')
     def test_install_bootstrap_skipped(self, mock_bootstrap_section):
diff --git a/test/unit/system/setup_test.py b/test/unit/system/setup_test.py
index d8f07ac8..0101f57d 100644
--- a/test/unit/system/setup_test.py
+++ b/test/unit/system/setup_test.py
@@ -1652,7 +1652,7 @@ class TestSystemSetup:
         mock_command.assert_called_once_with(
             [
                 'chroot', 'root_dir',
-                'setfiles', '-F', '-p',
+                'setfiles', '-T0', '-F', '-p',
                 '-c', '/etc/selinux/targeted/policy/policy.some_policy',
                 '-e', '/proc', '-e', '/sys', '-e', '/dev',
                 'security_context_file', '/'