Repository: opensearch-project/opensearch-build

Issue:
[RPM M1] Add a new block to call the generation code for RPM
Tasks | Estimate | Status | Notes | Dependencies
-- | -- | -- | -- | --
The generation code should pull the artifacts from the build workflow to a temporary location | 1 | Complete |   | Build workflow must provide usable artifacts
The code will call existing install function to install plugins on min artifacts | 1 | Complete |   |  
After installation, the code will execute a tool or utility to wrap all the content into a RPM package | 5 | Complete | Require writing a script to utilize FPM to start with and later implement in pure python code. <br><br>20220204: We might change to rpmbuild directly without using FPM. See comments. | FPM usages
The code will also add dependencies to the RPM package so that things like JDK and additional libs for plugins can be installed and pulled separately | 5 | Complete | Need to study on RPM dependency setups | RPM Build Dependencies and the locations of each dependent artifact
The code will move the RPM package from the temp location to dist folder | 2 | Complete |   |  




Discussion:
20220204: FPM is having some issues along the way:
* Messy handling of folder structure, as it will add every single file under `%files` section instead of owning folders, so erase of the package will not do proper clean up.
* Need to specifically set `--rpm-summary` for RPM, and debian has a different command, which is not suitable for the usages.
* If build on non-linux machine, `--rpm-os` is needed as another specific command that debian version does not own.
* Very large rpm spec file generated during runtimes, lack of many settings, need manual editing during runtime of the tool.
* Cannot source a spec file directly for a build, have to use the existing template.
* Documentation kind of lackluster as there are duplication of commands man page, and I need to dig through issues and issues to find some commands for my needs.

Overall, it is a great tool for quick building an RPM without spec file.
However, for our production usages I think using rpmbuild directly probably better here.

Still testing.

Resolving patch:
diff --git a/src/assemble_workflow/bundle_rpm.py b/src/assemble_workflow/bundle_rpm.py
--- a/src/assemble_workflow/bundle_rpm.py
+++ b/src/assemble_workflow/bundle_rpm.py
@@ -89,7 +89,10 @@ def build(self, name: str, dest: str, archive_path: str, build_cls: BuildManifes
         os.environ.pop('OPENSEARCH_PATH_CONF', None)
 
         # Restore config file and core folder to original location
-        shutil.move(f"{min_bin_env_path}.backup", min_bin_env_path)
+        if os.path.exists(f"{min_bin_env_path}.backup"):
+            logging.info(f"Restore {min_bin_env_path}.backup to {min_bin_env_path}")
+            shutil.move(f"{min_bin_env_path}.backup", min_bin_env_path)
+
         shutil.move(min_dest_path, min_source_path)
 
         # Run bundle rpmbuild


Test patch:
diff --git a/tests/tests_assemble_workflow/test_bundle_rpm.py b/tests/tests_assemble_workflow/test_bundle_rpm.py
--- a/tests/tests_assemble_workflow/test_bundle_rpm.py
+++ b/tests/tests_assemble_workflow/test_bundle_rpm.py
@@ -41,11 +41,12 @@ def test_extract_rpm(self, check_call_mock: Mock, shutil_copy2_mock: Mock, shuti
         self.assertEqual(os_path_exists_mock.call_count, 1)
         self.assertEqual(os.environ['OPENSEARCH_PATH_CONF'], os.path.join(self.artifacts_path, 'etc', 'opensearch'))
 
+    @patch("os.path.exists", return_value=True)
     @patch("os.walk")
     @patch("builtins.open")
     @patch("shutil.move")
     @patch("subprocess.check_call")
-    def test_build_rpm(self, check_call_mock: Mock, shutil_move_mock: Mock, builtins_open: Mock, os_walk_mock: Mock) -> None:
+    def test_build_rpm(self, check_call_mock: Mock, shutil_move_mock: Mock, builtins_open: Mock, os_walk_mock: Mock, os_path_exists: Mock) -> None:
 
         self.bundle_rpm.build(self.package_path, self.artifacts_path, os.path.join(self.artifacts_path, 'opensearch-1.3.0'), self.manifest.build)
         args_list = check_call_mock.call_args_list