Repository: ansible-collections/community.general

Issue:
community.general.gitlab_runner SSLError cert verify failed
### Summary

I am trying to register some gitlab runners. On new openstack instances.
I get a SSL certificate verify failed error as the return.

If I curl the gitlab url from the command line of the box that I am trying to register I get a response back no TLS issues. 
The gitlab servers certificate is signed by an internal CA, which is installed into the system ca-trust /etc/pki/ca-trust/source/anchors and update-ca-trust has been run.

I am unable to copy and paste the actual out from the servers. Due to interal work policy

### Issue Type

Bug Report

### Component Name

gitlab_runner

### Ansible Version

```console (paste below)
$ ansible --version

```
ansible 2.9.33
python version 3.9.5

### Configuration

```

```


### OS / Environment

CentOS 8 on Openstack train
Gitlab 14.0.1 runner same version


### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
- name: register gitlab runner
  community.general.gitlab_runner
    api_url: https://gitlab.internal.addressing
    api_token: abc
    registration_token: def
    description: "{{ anisble_hostname }}"
    state: present
    active: false
    tag_list: shell
    run_untagged: false
    locked: false
```


### Expected Results

The ansible task to connect to the gitlab server and register the runner

### Actual Results

```console (paste below)
SSLError "Certificate_verify_failed"
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/source_control/gitlab/gitlab_runner.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/source_control/gitlab/gitlab_runner.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Lunik @SamyCoenen @Shaps @dj-wasabi @marwatk @metanovii @scodeman @waheedi @zanssa
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
It looks like Python isn't using the system ca-trust then.
I have the same problem, only tested with community.general.gitlab_group, but if needed I can try with community.general.gitlab_users as well. 

The problem seems to come from the python-gitlab module. 

Here is what I tried: 

### On Python
```python
import socket
import ssl

hostname = 'gitlab.domain.local'
context = ssl.create_default_context()

with socket.create_connection((hostname, 443)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as ssock:
        print(ssock.version())

```

I get no error. If the certificate is absent from my system, then I would have one.

### On python-gitlab
I defined the $HOME/.python-gitlab.cfg as following:
```
[global]
default = gitlab.domain.local
timeout = 5
[gitlab.domain.local]
api_version = 4
url = https://gitlab.domain.local
private_token = XXXXXXXXXXXXXXXX
```
When I run `gitlab group list` I have the following: 
```
HTTPSConnectionPool(host='gitlabl.domain.local', port=443): Max retries exceeded with url: /api/v4/user (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))
```

The problem is fixed when I add the variable `ssl_verify = 'MyCertificate.cer'` in the python-gitlab.cfg.

### On Ansible
I then tried to add an extra variable in the community.general.gitlab_group module, called certs, and specified my certificate in my playbook, and it worked.

So while the python-gitlab module can be fixed regarding that matter, this is a solution that could be implemented for all gitlab ansiblle modules.

My Ansible task looks like that:
```yaml
- name: "Create GitLab Group"
  community.general.gitlab_group:
    api_url: "{{ gitlab_url }}"
    #validate_certs: no  # not working when set to yes
    certs: "/CA-ROOT.cer"
    api_token: "{{ gitlab_api_token }}"
    name: "{{ g_group }}"
    path: "{{ g_group }}"
    state: present
    parent: "{{ gitlab_group_parent }}"
```

#### The changes in the Ansible module
In community/general/plugins/module_utils/gitlab.py:81
Change to `module.params['certs']`
```python
def gitlabAuthentication(module):
    gitlab_url = module.params['api_url']
    validate_certs = module.params['certs']
    gitlab_user = module.params['api_username']
    gitlab_password = module.params['api_password']
    gitlab_token = module.params['api_token']
```

In community/general/plugins/modules/source_control/gitlab/gitlab_group.py:256
Add a line to define certs:
```python
def main():
    argument_spec = basic_auth_argument_spec()
    argument_spec.update(dict(
        api_token=dict(type='str', no_log=True),
        name=dict(type='str', required=True),
        certs=dict(type='str'),
        path=dict(type='str'),
        description=dict(type='str'),
        state=dict(type='str', default="present", choices=["absent", "present"]),
        parent=dict(type='str'),
        visibility=dict(type='str', default="private", choices=["internal", "private", "public"]),
    ))
```

That variable, `certs` can be either the certificate file's path, or no/yes.
@jayooin thanks for confirming the issue isn't just with my setup.
@jayooin there should be two variables: `ca_path` (type `path`) for allowing to specify a path to a CA certificate, and `validate_certs` (type `bool`) for enabling/disabling certificate validation. These two inputs need to be combined together to the parameter for python-gitlab.
@dj-wasabi Is there a way to implement that solution ? (see above)
All we need is to be able to pass a certificate path, like I did with _certs_.

From my understanding, validate_certs only accepts boolean. So we need something like ca_path.
Then, because the python-gitlab argument ssl_verify takes both a boolean AND a path, there should be a test to pass either.

e.g:

```
- name: register gitlab runner
  community.general.gitlab_runner
    validate_certs: True
    ca_path: /path/to/ca
```

Then the module should, when `validate_certs` is set to True, check if there's a value for `ca_path` and pass.
cc @sh0shin
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
FYI: I guess having an option for this would be more explicit, but you don't need to patch the module for this to work. 

python-gitlab uses the `requests` library, so certs are validated using the `certifi` package. For self-signed/internal CAs, you can use the `REQUESTS_CA_BUNDLE` environment variable. This is documented in python-gitlab and upstream requests:
- https://python-gitlab.readthedocs.io/en/stable/api-usage.html#ssl-certificate-verification
- https://2.python-requests.org/en/master/user/advanced/#ssl-cert-verification
cc @lgatellier
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @suukit
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Files identified in the description:
* [`plugins/modules/gitlab_runner.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/modules/gitlab_runner.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Since `validate_certs` option exists in all gitlab modules, I see two options :
- We consider the `REQUESTS_CA_BUNDLE` environment variable as a good solution for self-signed CAs, and add a little mention in all gitlab modules documentation to explicit this use case
- We do not want to rely on certifi use (` requests` library may change certificates provider one day, who knows ?), and we consider to add a new `ca_path` option

@felixfontein what's your opinion, regarding the collection policy in that case ?
I would prefer a `ca_path` option that abstracts knowing implementation details (the module indirectly uses requests, and requests uses certifi) away. There is no official policy I guess, but having a `ca_path` option instead of some environment variable magic would theoretically allow to switch out the python-gitlab library, or make that library able to switch out requests, or make requests switch out certifi (even if all of them are not likely to happen).

Resolving patch:
diff --git a/plugins/doc_fragments/gitlab.py b/plugins/doc_fragments/gitlab.py
--- a/plugins/doc_fragments/gitlab.py
+++ b/plugins/doc_fragments/gitlab.py
@@ -29,4 +29,9 @@ class ModuleDocFragment(object):
       - GitLab CI job token for logging in.
     type: str
     version_added: 4.2.0
+  ca_path:
+    description:
+      - The CA certificates bundle to use to verify GitLab server certificate.
+    type: str
+    version_added: 8.1.0
 '''
diff --git a/plugins/module_utils/gitlab.py b/plugins/module_utils/gitlab.py
--- a/plugins/module_utils/gitlab.py
+++ b/plugins/module_utils/gitlab.py
@@ -34,6 +34,7 @@
 
 def auth_argument_spec(spec=None):
     arg_spec = (dict(
+        ca_path=dict(type='str'),
         api_token=dict(type='str', no_log=True),
         api_oauth_token=dict(type='str', no_log=True),
         api_job_token=dict(type='str', no_log=True),
@@ -76,6 +77,7 @@ def ensure_gitlab_package(module):
 def gitlab_authentication(module):
     gitlab_url = module.params['api_url']
     validate_certs = module.params['validate_certs']
+    ca_path = module.params['ca_path']
     gitlab_user = module.params['api_username']
     gitlab_password = module.params['api_password']
     gitlab_token = module.params['api_token']
@@ -84,23 +86,25 @@ def gitlab_authentication(module):
 
     ensure_gitlab_package(module)
 
+    verify = ca_path if validate_certs and ca_path else validate_certs
+
     try:
         # python-gitlab library remove support for username/password authentication since 1.13.0
         # Changelog : https://github.com/python-gitlab/python-gitlab/releases/tag/v1.13.0
         # This condition allow to still support older version of the python-gitlab library
         if LooseVersion(gitlab.__version__) < LooseVersion("1.13.0"):
-            gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
+            gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=verify, email=gitlab_user, password=gitlab_password,
                                             private_token=gitlab_token, api_version=4)
         else:
             # We can create an oauth_token using a username and password
             # https://docs.gitlab.com/ee/api/oauth2.html#authorization-code-flow
             if gitlab_user:
                 data = {'grant_type': 'password', 'username': gitlab_user, 'password': gitlab_password}
-                resp = requests.post(urljoin(gitlab_url, "oauth/token"), data=data, verify=validate_certs)
+                resp = requests.post(urljoin(gitlab_url, "oauth/token"), data=data, verify=verify)
                 resp_data = resp.json()
                 gitlab_oauth_token = resp_data["access_token"]
 
-            gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, private_token=gitlab_token,
+            gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=verify, private_token=gitlab_token,
                                             oauth_token=gitlab_oauth_token, job_token=gitlab_job_token, api_version=4)
 
         gitlab_instance.auth()


Test patch: