Repository: packit-service/packit
Pull request #634 (2019-12-11 13:22:45)

Add methods for finding spec file

softwarefactory-project-zuul[bot]: Build succeeded.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/cfb7b2560d10417b9f3d11f42b27a3e3) : SUCCESS in 1m 54s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/02f8d504d5e949ebbf4a11533360be1c) : SUCCESS in 3m 38s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/7d0210006af14b038aa8697f81d682e6) : SUCCESS in 4m 19s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/1a336fc7082c46d9b511c0818fd06588) : SUCCESS in 4m 18s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/1c7c636cc7114fe181f49300fe87bf59) : SUCCESS in 2m 27s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/6ecd34a2ebee4034b05ffd0c4a6c30f9) : SUCCESS in 2m 33s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/3c58621e095f4f45a4f1e74efca92864) : SUCCESS in 2m 48s

packit-as-a-service[bot]: Congratulations! The build [has finished](https://copr.fedorainfracloud.org/coprs/packit/packit-service-packit-634/build/1128046/) successfully. :champagne:

You can install the built RPMs by following these steps:

* `sudo yum install -y dnf-plugins-core` on RHEL 8
* `sudo dnf install -y dnf-plugins-core` on Fedora
* `dnf copr enable packit/packit-service-packit-634`
* And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/0d26aca88fa44467a5dc6691eae72f04) : FAILURE in 1m 55s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/fa63724af13a410aac7a0e37102c6d57) : FAILURE in 5m 18s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/549e53f39d424397b927752b749ce053) : FAILURE in 5m 22s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/5bc9d0a368254dcb82412dc19dd83bd9) : SUCCESS in 5m 09s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/403c390724bc48aba23e501bafa0ebf3) : SUCCESS in 2m 35s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/847f9a0ad3664f868871bc05ad48bff2) : SUCCESS in 2m 55s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/dc2048625bc049279570bbaf490641b0) : SUCCESS in 2m 58s

softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/1dc083123bc44a359c814e48deaea94a) : SUCCESS in 1m 45s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/277e1fe1632047c9b61c899b2cdafe41) : FAILURE in 4m 43s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/70e580b73af8450fb03a0300f4b9cc10) : FAILURE in 5m 05s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/9e438855fa7f423a85516a13891c2d8e) : SUCCESS in 5m 16s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/ae4bae4c7cf0425fbc0e05cbcf05a503) : SUCCESS in 1m 57s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/2e10fc1296a74615bf2b90267cbee1b4) : SUCCESS in 2m 32s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/4aa4127229454c379a4a4357503d14f9) : SUCCESS in 3m 17s

lbarcziova: > I see some test fails
I think it is because the function get_files() in ogr was just merged and now it is only in master, where the tests pass.
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/6a254b97078043b58fd094d2e5313a99) : SUCCESS in 2m 09s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/e0aeb2e47f6d4b2288d6d0b8f66ca2f3) : FAILURE in 6m 51s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/9591f323dfb949d5b7e556f61093a160) : FAILURE in 5m 00s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/0e1c7755594b48318155fab1084d1e19) : SUCCESS in 5m 33s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/5c5864bdf2634152b25680ca9e345762) : SUCCESS in 1m 49s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/7fce02bbc55b4c7baec037d19a6a42da) : SUCCESS in 3m 07s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/46fb2404a2a647cb890fdd67d6fa1371) : SUCCESS in 2m 53s

jpopelka: > I think it is because the function get_files() in ogr was just merged and now it is only in master, where the tests pass.

Ok, it makes sense.

The thing is that in the `packit-service-worker` image (which is based on `packit` image) we use the `python3-ogr` RPM from Fedora.
So we can't merge this until that `get_files()` change is part of `python3-ogr` in Fedora if we don't want to break `stg`/`prod`.

Time for new `ogr` release @lachmanfrantisek ?


lachmanfrantisek: > Time for new `ogr` release @lachmanfrantisek ?

Sure. I'll take a look at the opened PR and we can release a new OGR...
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/e8a4770bcc7c48918cb50af2c61d9816) : SUCCESS in 1m 59s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/211b97be9b4c45aeaf4103f1c4eb00fc) : FAILURE in 4m 40s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/3eb4a539ecd94c3da769b6678eb221b6) : FAILURE in 5m 30s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/64eb6309c96f4abb84f4dbf66c7052cd) : SUCCESS in 5m 22s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/9d800c396b5746938ff687f0f1942959) : SUCCESS in 2m 07s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/765a66767c9a4f7e8a4a62303fb8e375) : SUCCESS in 2m 19s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/fcea2dbccfed4502b68820ed6f68532a) : SUCCESS in 2m 39s

TomasTomecek: needs rebase :sun_with_face: 
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/fac3bd43cfa44c51ae6df480b5efabbe) : FAILURE in 2m 40s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/cde60714d5a94a9bbde8560a95ae9c8e) : FAILURE in 5m 31s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/c28583c520b849279d8aec12622fdfa6) : FAILURE in 5m 16s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/77f35fe988b448f68a60f34111dc8dd2) : SUCCESS in 6m 07s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/c6fd51126bc04f86b5f8a91ec754cf15) : SUCCESS in 1m 51s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/513ecd98773140c5a3d7e3fe9e5c5107) : SUCCESS in 2m 24s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/0d7de567fdd64da4a344643adbfb469a) : SUCCESS in 3m 14s

packit-as-a-service[bot]: There was an error while creating SRPM. You can re-trigger build by adding a comment (`/packit copr-build`) into this pull request.

Output:
```
Preparing of the upstream to the SRPM build failed: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'Date': 'Wed, 29 Jan 2020 13:45:16 GMT', 'Content-Length': '465'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"persistentvolumeclaims \"sandcastle--sandcastle-20200129-134516488485-pvc\" is forbidden: exceeded quota: 74746f6d6563656b407265646861742e636f6d-noncompute, requested: requests.storage=1Gi, used: requests.storage=23Gi, limited: requests.storage=23Gi","reason":"Forbidden","details":{"name":"sandcastle--sandcastle-20200129-134516488485-pvc","kind":"persistentvolumeclaims"},"code":403}


```
lbarcziova: /packit copr-build
packit-as-a-service[bot]: Congratulations! One of the builds has completed. :champagne:

You can install the built RPMs by following these steps:

* `sudo yum install -y dnf-plugins-core` on RHEL 8
* `sudo dnf install -y dnf-plugins-core` on Fedora
* `dnf copr enable packit/packit-service-packit-634`
* And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/5ea5c6afd8c94c16ac967efe22d13280) : FAILURE in 2m 14s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/e3cf9ecda12e44e194a8e2f3934c49d2) : FAILURE in 5m 38s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/397e22f6c4704f3daca9904481abfe66) : SUCCESS in 5m 42s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/b1b67b990c34432ba524df7265f0f9e9) : SUCCESS in 6m 42s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/1f2ab85b13874777a8e645966d6b28cb) : SUCCESS in 2m 20s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/8e121523a3fa480dbf60461189e1c940) : SUCCESS in 2m 25s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/291bd29031f64ca5827a83a91ffb0525) : SUCCESS in 2m 46s

softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/06f2bbd18307446591bcabd93d381166) : SUCCESS in 1m 39s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/15b5ab93f8444be69897af21f9c00d60) : FAILURE in 7m 01s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/e7086145872246619fb59ad5bed74b83) : SUCCESS in 8m 10s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/598b292d528a483985786acc045a5463) : SUCCESS in 6m 58s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/2502633224eb4036803eaa16519d57c4) : SUCCESS in 3m 20s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/7402a02c73b140be992c141aa859b924) : SUCCESS in 5m 06s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/7efc9ce10bcb498e8d21c5a84b25f575) : SUCCESS in 4m 44s

jpopelka: recheck
jpopelka: /packit build
softwarefactory-project-zuul[bot]: Build failed.

- [packit-pre-commit ](https://softwarefactory-project.io/zuul/t/local/build/54bb6a6db3614750afe907c54cbf5e20) : SUCCESS in 1m 54s
- [packit-tests-rpm ](https://softwarefactory-project.io/zuul/t/local/build/d0af963649494e93be59863891f72492) : FAILURE in 5m 45s
- [packit-tests-pip-deps ](https://softwarefactory-project.io/zuul/t/local/build/cc29295b22864753b6df7c94badc2f18) : SUCCESS in 6m 22s
- [packit-tests-git-master ](https://softwarefactory-project.io/zuul/t/local/build/391fe6b080404d619c81df944d48cf6a) : SUCCESS in 5m 50s
- [packit-tests-rpm-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/682fb87a4c9445d9a817e84ff024074b) : SUCCESS in 2m 10s
- [packit-tests-pip-deps-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/e05437a5d1b74470989e2cca92045afe) : SUCCESS in 3m 44s
- [packit-tests-git-master-sess-rec ](https://softwarefactory-project.io/zuul/t/local/build/d1263a3dad7349acbfc7c2e72abe0e25) : SUCCESS in 2m 55s

jpopelka: >     * packit-tests-rpm  : FAILURE in 5m 45s

#694 should address that. I'll rebase once it's merged.



Patch:
diff --git a/packit/config/package_config.py b/packit/config/package_config.py
index 0d1360aa..d1d51db0 100644
--- a/packit/config/package_config.py
+++ b/packit/config/package_config.py
@@ -22,6 +22,8 @@
 
 import json
 import logging
+import os
+from glob import glob
 from pathlib import Path
 from typing import Optional, List, Dict, Union
 
@@ -118,7 +120,11 @@ class PackageConfig:
 
     @classmethod
     def get_from_dict(
-        cls, raw_dict: dict, config_file_path: str = None, repo_name: str = None
+        cls,
+        raw_dict: dict,
+        config_file_path: str = None,
+        repo_name: str = None,
+        spec_file_path: str = None,
     ) -> "PackageConfig":
         # required to avoid cyclical imports
         from packit.schema import PackageConfigSchema
@@ -128,6 +134,12 @@ class PackageConfig:
 
         package_config = PackageConfigSchema(strict=True).load(raw_dict).data
 
+        if not getattr(package_config, "specfile_path", None):
+            if spec_file_path:
+                package_config.specfile_path = spec_file_path
+            else:
+                raise PackitConfigException("Spec file was not found!")
+
         if not getattr(package_config, "upstream_package_name", None) and repo_name:
             package_config.upstream_package_name = repo_name
 
@@ -216,6 +228,7 @@ def get_local_package_config(
                     loaded_config=loaded_config,
                     config_file_path=str(config_file_name),
                     repo_name=repo_name,
+                    spec_file_path=get_local_specfile_path(directories),
                 )
 
             logger.debug(f"The local config file '{config_file_name_full}' not found.")
@@ -258,11 +271,15 @@ def get_package_config_from_repo(
         loaded_config=loaded_config,
         config_file_path=config_file_name,
         repo_name=sourcegit_project.repo,
+        spec_file_path=get_specfile_path_from_repo(sourcegit_project),
     )
 
 
 def parse_loaded_config(
-    loaded_config: dict, config_file_path: str = None, repo_name: str = None
+    loaded_config: dict,
+    config_file_path: str = None,
+    repo_name: str = None,
+    spec_file_path: str = None,
 ) -> PackageConfig:
     """Tries to parse the config to PackageConfig."""
     logger.debug(f"Package config:\n{json.dumps(loaded_config, indent=4)}")
@@ -272,8 +289,38 @@ def parse_loaded_config(
             raw_dict=loaded_config,
             config_file_path=config_file_path,
             repo_name=repo_name,
+            spec_file_path=spec_file_path,
         )
         return package_config
     except Exception as ex:
         logger.error(f"Cannot parse package config. {ex}.")
         raise PackitConfigException(f"Cannot parse package config: {ex}.")
+
+
+def get_local_specfile_path(directories: Union[List[str], List[Path]]):
+    """
+    Get the relative path of the local spec file if present.
+    :param directories: dirs to find the spec file
+    :return: str relative path of the spec file
+    """
+    for dir in directories:
+        files = [
+            os.path.relpath(path, dir) for path in glob(os.path.join(dir, "*.spec"))
+        ]
+        if len(files) > 0:
+            return files[0]
+
+    return None
+
+
+def get_specfile_path_from_repo(project: GitProject):
+    """
+    Get the path of the spec file in the given repo if present.
+    :param project: GitProject
+    :return: str path of the spec file
+    """
+    try:
+        return project.get_files(filter_regex=".*.spec")[0]
+    except Exception as ex:
+        logger.debug(f"Cannot find the spec file: {ex}")
+        return None
diff --git a/packit/schema.py b/packit/schema.py
index c20d3a4a..efbaec14 100644
--- a/packit/schema.py
+++ b/packit/schema.py
@@ -152,7 +152,7 @@ class PackageConfigSchema(Schema):
     """
 
     config_file_path = fields.String()
-    specfile_path = fields.String(required=True)
+    specfile_path = fields.String()
     downstream_package_name = fields.String()
     upstream_project_url = fields.String(missing=None)
     upstream_package_name = fields.String()
@@ -274,5 +274,4 @@ class UserConfigSchema(Schema):
 
     @post_load
     def make_instance(self, data, **kwargs):
-
         return Config(**data)


Test patch:
diff --git a/tests/spellbook.py b/tests/spellbook.py
index a9bb01c0..104d6779 100644
--- a/tests/spellbook.py
+++ b/tests/spellbook.py
@@ -51,6 +51,7 @@ SOURCEGIT_SOURCEGIT = DATA_DIR / "sourcegit" / "source_git"
 DG_OGR = DATA_DIR / "dg-ogr"
 SPECFILE = DATA_DIR / "upstream_git/beer.spec"
 UPSTREAM_SPEC_NOT_IN_ROOT = DATA_DIR / "spec_not_in_root/upstream"
+SYNC_FILES = DATA_DIR / "sync_files"
 
 
 def git_set_user_email(directory):
diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py
index 4f64b9aa..959969b1 100644
--- a/tests/unit/test_config.py
+++ b/tests/unit/test_config.py
@@ -41,6 +41,9 @@ from packit.config import (
     SyncFilesItem,
 )
 
+from packit.config.package_config import get_local_specfile_path
+from tests.spellbook import UP_OSBUILD, SYNC_FILES
+
 
 def get_job_config_dict_simple():
     return {"job": "build", "trigger": "release"}
@@ -233,7 +236,6 @@ def test_package_config_not_equal(not_equal_package_config):
 @pytest.mark.parametrize(
     "raw,is_valid",
     [
-        ({}, False),
         (
             {
                 "specfile_path": "fedora/package.spec",
@@ -241,7 +243,6 @@ def test_package_config_not_equal(not_equal_package_config):
             },
             False,
         ),
-        ({"jobs": [{"trigger": "release", "job": "propose_downstream"}]}, False),
         (
             {
                 "specfile_path": "fedora/package.spec",
@@ -300,12 +301,11 @@ def test_package_config_validate(raw, is_valid):
 @pytest.mark.parametrize(
     "raw",
     [
-        {},
         # {"specfile_path": "test/spec/file/path", "something": "different"},
         {
             "specfile_path": "test/spec/file/path",
             "jobs": [{"trigger": "release", "release_to": ["f28"]}],
-        },
+        }
     ],
 )
 def test_package_config_parse_error(raw):
@@ -580,6 +580,19 @@ def test_get_package_config_from_repo(content):
     assert config.create_pr
 
 
+@pytest.mark.parametrize("content", ["{}"])
+def test_get_package_config_from_repo_spec_file_not_defined(content):
+    gp = flexmock(GitProject)
+    gp.should_receive("full_repo_name").and_return("a/b")
+    gp.should_receive("get_file_content").and_return(content)
+    gp.should_receive("get_files").and_return(["packit.spec"])
+    git_project = GitProject(repo="", service=GitService(), namespace="")
+    config = get_package_config_from_repo(sourcegit_project=git_project, ref="")
+    assert isinstance(config, PackageConfig)
+    assert Path(config.specfile_path).name == "packit.spec"
+    assert config.create_pr
+
+
 def test_get_user_config(tmpdir):
     user_config_file_path = Path(tmpdir) / ".packit.yaml"
     user_config_file_path.write_text(
@@ -695,3 +708,8 @@ def test_user_config_fork_token(tmpdir, recwarn):
 )
 def test_get_all_files_to_sync(package_config, all_synced_files):
     assert package_config.get_all_files_to_sync() == all_synced_files
+
+
+def test_get_local_specfile_path():
+    assert get_local_specfile_path([UP_OSBUILD]) == "osbuild.spec"
+    assert not get_local_specfile_path([SYNC_FILES])