Repository: ansible/ansible
Pull request: fix installing collections from git repos that contain MANIFEST.json
##### SUMMARY
Fixes #79796

As of 595413d11346b6f26bb3d9df2d8e05f2747508a3, collections in git repos can contain a galaxy.yml or MANIFEST.json (just like non-git repo directories).

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
- Bugfix Pull Request

##### COMPONENT NAME
ansible-galaxy collection install
```


Problem statement:
2.14 Issue with installing custom collections
### Summary

In 2.14+ we are getting errors when installing our local collections
Multiple versions of 2.13 (including 2.13.7) were tested against those same collections without any errors.
v2.14.0 Release notes do not include anything that I can see would have made breaking changes to ansible-galaxy command.

### Issue Type

Bug Report

### Component Name

ansible-galaxy

### Ansible Version

```console
$ ansible-galaxy --version
ansible-galaxy [core 2.14.0]
  config file = /home/REDACTED/venv/ansible_2140/meta/ansible.cfg
  configured module search path = ['/home/REDACTED/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/REDACTED/venv/ansible_2140/collections
  executable location = /home/REDACTED/venv/ansible_2140/bin/ansible-galaxy
  python version = 3.9.7 (default, Apr 11 2022, 06:30:15) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10.0.1)] (/home/REDACTED/venv/ansible_2140/bin/python3)
  jinja version = 3.1.2
  libyaml = True
```


### Configuration

```console
# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all
COLLECTIONS_PATHS(/home/REDACTED/venv/ansible_2140/meta/ansible.cfg) = ['/home/REDACTED/venv/ansible_2140/collections']
CONFIG_FILE() = /home/REDACTED/venv/ansible_2140/meta/ansible.cfg
```


### OS / Environment

Oracle Linux Server 8.6

### Steps to Reproduce

```
$ ansible-galaxy collection install -r /tmp/requirements.yml -vvv
ansible-galaxy [core 2.14.0]
  config file = /home/REDACTED/venv/ansible_2140/meta/ansible.cfg
  configured module search path = ['/home/REDACTED/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/REDACTED/venv/ansible_2140/collections
  executable location = /home/REDACTED/venv/ansible_2140/bin/ansible-galaxy
  python version = 3.9.7 (default, Apr 11 2022, 06:30:15) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10.0.1)] (/home/REDACTED/venv/ansible_2140/bin/python3)
  jinja version = 3.1.2
  libyaml = True
Using /home/REDACTED/venv/ansible_2140/meta/ansible.cfg as config file
Reading requirement file at '/tmp/requirements.yml'
Starting galaxy collection install process
Found installed collection community.general:6.2.0 at '/home/REDACTED/venv/ansible_2140/collections/ansible_collections/community/general'
Process install dependency map
Cloning into '/home/REDACTED/.ansible/tmp/ansible-local-13046752reyjqax/tmpx57_tb93/REDACTED.REDACTEDbwwa6z2y'...
remote: Enumerating objects: 335, done.
remote: Counting objects: 100% (335/335), done.
remote: Compressing objects: 100% (171/171), done.
remote: Total 503 (delta 128), reused 281 (delta 97), pack-reused 168
Receiving objects: 100% (503/503), 78.81 KiB | 530.00 KiB/s, done.
Resolving deltas: 100% (190/190), done.
Already on 'master'
Your branch is up to date with 'origin/master'.
Starting collection install process
Installing 'REDACTED.REDACTED:1.2.0' to '/home/REDACTED/venv/ansible_2140/collections/ansible_collections/REDACTED/REDACTED'
ERROR! Unexpected Exception, this is probably a bug: 'manifest'
the full traceback was:

Traceback (most recent call last):
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/cli/__init__.py", line 647, in cli_executor
    exit_code = cli.run()
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/cli/galaxy.py", line 681, in run
    return context.CLIARGS['func']()
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/cli/galaxy.py", line 116, in method_wrapper
    return wrapped_method(*args, **kwargs)
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/cli/galaxy.py", line 1344, in execute_install
    self._execute_install_collection(
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/cli/galaxy.py", line 1381, in _execute_install_collection
    install_collections(
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/galaxy/collection/__init__.py", line 771, in install_collections
    install(concrete_coll_pin, output_path, artifacts_manager)
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/galaxy/collection/__init__.py", line 1446, in install
    install_src(collection, b_artifact_path, b_collection_path, artifacts_manager)
  File "/home/REDACTED/venv/ansible_2140/lib64/python3.9/site-packages/ansible/galaxy/collection/__init__.py", line 1578, in install_src
    collection_meta['manifest'],
KeyError: 'manifest'

$ cat /tmp/requirements.yml
collections:
  - name: git@REDACTED:REDACTED/automation/ansible/collections/REDACTED.REDACTED.git
    type: git
    version: master
```

### Expected Results

Expect the same thing that happens in Ansible versions < 2.14.0
```$ ansible-galaxy --version
ansible-galaxy [core 2.13.4]
  config file = /home/REDACTED/venv/ee-rcstandard-rhel8-183/meta/ansible.cfg
  configured module search path = ['/home/REDACTED/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/REDACTED/venv/ee-rcstandard-rhel8-183/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/REDACTED/venv/ee-rcstandard-rhel8-183/collections
  executable location = /home/REDACTED/venv/ee-rcstandard-rhel8-183/bin/ansible-galaxy
  python version = 3.9.7 (default, Apr 11 2022, 06:30:15) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10.0.1)]
  jinja version = 3.1.2
  libyaml = True

$ ansible-galaxy install -r /tmp/requirements.yml
Starting galaxy collection install process
Process install dependency map
Cloning into '/home/REDACTED/.ansible/tmp/ansible-local-1304752dfq9iqso/tmpmo9tw51i/REDACTED.REDACTEDbkddz8qw'...
remote: Enumerating objects: 335, done.
remote: Counting objects: 100% (335/335), done.
remote: Compressing objects: 100% (171/171), done.
remote: Total 503 (delta 128), reused 281 (delta 97), pack-reused 168
Receiving objects: 100% (503/503), 78.81 KiB | 537.00 KiB/s, done.
Resolving deltas: 100% (190/190), done.
Already on 'master'
Your branch is up to date with 'origin/master'.
Starting collection install process
Installing 'REDACTED.REDACTED:1.2.0' to '/home/REDACTED/venv/ee-rcstandard-rhel8-183/collections/ansible_collections/REDACTED/REDACTED'
Created collection for REDACTED.REDACTED:1.2.0 at /home/REDACTED/venv/ee-rcstandard-rhel8-183/collections/ansible_collections/REDACTED/REDACTED
REDACTED.REDACTED:1.2.0 was installed successfully
```


### Actual Results

```console
See steps to reproduce
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

Hints from issue discussion:
Files identified in the description:
* `lib/ansible/galaxy`

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Commit message: add null 'manifest' key to metadata for git repo collections containing MANIFEST.json

changelog

Patch to review:
diff --git a/changelogs/fragments/ansible-galaxy-install-git-src-manifest.yml b/changelogs/fragments/ansible-galaxy-install-git-src-manifest.yml
new file mode 100644
index 00000000000000..c306777df999fe
--- /dev/null
+++ b/changelogs/fragments/ansible-galaxy-install-git-src-manifest.yml
@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy - fix installing collections in git repositories which contain a MANIFEST.json file instead of galaxy.yml (https://github.com/ansible/ansible/issues/79796).
diff --git a/lib/ansible/galaxy/collection/__init__.py b/lib/ansible/galaxy/collection/__init__.py
index fa9ccd3350a9ba..e14bb8782e1d77 100644
--- a/lib/ansible/galaxy/collection/__init__.py
+++ b/lib/ansible/galaxy/collection/__init__.py
@@ -1626,6 +1626,7 @@ def install_src(collection, b_collection_path, b_collection_output_path, artifac
     if 'build_ignore' not in collection_meta:  # installed collection, not src
         # FIXME: optimize this? use a different process? copy instead of build?
         collection_meta['build_ignore'] = []
+        collection_meta['manifest'] = None
     collection_manifest = _build_manifest(**collection_meta)
     file_manifest = _build_files_manifest(
         b_collection_path,


Code review comments:
diff_hunk: @@ -1626,6 +1626,7 @@ def install_src(collection, b_collection_path, b_collection_output_path, artifac
     if 'build_ignore' not in collection_meta:  # installed collection, not src
         # FIXME: optimize this? use a different process? copy instead of build?
         collection_meta['build_ignore'] = []
+        collection_meta['manifest'] = None
path: lib/ansible/galaxy/collection/__init__.py
text: @user1:
I think this needs to be `Sentinel`, as having it set to `None` will enable it with defaults.  See:

* https://github.com/ansible/ansible/blob/10eda5801ad11f66985251b5c3de481e7b917d3c/lib/ansible/galaxy/data/collections_galaxy_meta.yml#L113-L120
* https://github.com/ansible/ansible/blob/10eda5801ad11f66985251b5c3de481e7b917d3c/lib/ansible/galaxy/collection/__init__.py#L1074

@author:
I forgot about that!