Repository: ansible-collections/community.general

Issue:
Request to WAF/CDN with enhanced bot protection using url module gets HTTP Error 403: Forbidden
### Summary

The specific scenario is checked with Keycloak Module, where my Keycloak Instance is hosted with Cloudflare Proxy.

When I try run keycloak related task with ansible playbook, it fails with HTTP Error 403: Forbidden. 
The reason being the bot protection in CDN Services, Cloudflare in my case. The request made by the keycloak module, doesn't add User-Agent header, which is blocked. I came across this issue when I was testing out keycloak realm creation with ansible playbook, It worked for my local test server, gave the following error for my cloudflare protected keycloak. 
```bash
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Could not obtain access token from https://kc.korifi.run/realms/master/protocol/openid-connect/token: HTTP Error 403: Forbidden"}
```
I debugged down to the module and tried out the snippet to get token with a smaller python script.
Just adding the Useragent header with appropriate default value, and it worked

Many of the WAF Protected servers won't allow Requests without User-Agent headers, considering them as bot generated and block.

Adding  User-Agent header to the request will work for most of the scenarios that use `ansible.module_uitls.urls` library
Following Script demonstrates the exception and fix for the same with snippet taken from https://github.com/ansible-collections/community.general/blob/1c167ab8948afe429fd4591ea4434f14ab40a862/plugins/module_utils/identity/keycloak/keycloak.py#L140

```python

from __future__ import absolute_import, division, print_function

__metaclass__ = type

import json
import traceback
import urllib
import ssl
from ansible.module_utils.urls import open_url
from ansible.module_utils.six.moves.urllib.parse import urlencode, quote
from ansible.module_utils.common.text.converters import to_native, to_text
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE


request_url = "https://kc.korifi.run/realms/master/protocol/openid-connect/token" #Your Keycloak URL


## Taking out Snipper from 
temp_payload = {
            'grant_type': 'password',
            'client_id': "admin-cli",
            'username': "admin",
            'password': "admin",
}
payload = dict(
    (k, v) for k, v in temp_payload.items() if v is not None)

try:  #Request without User-Agent header will return 403 Forbidden
    r = json.loads(to_native(open_url(request_url, method='POST',
                                              validate_certs='False',
                                              data=urlencode(payload)).read()))
    print(r)
except Exception as e:
    print(e,"\n")

try: #Request with User-Agent header will work
    rh = json.loads(to_native(open_url(request_url, method='POST',
                                              validate_certs='False',
                                              data=urlencode(payload), headers={"User-Agent": "Ansible"}).read()))
    print(rh)
except Exception as e:
    print(e,"\n")



### Demonstrating Behavior with URL Lib

data=urllib.parse.urlencode(payload)
data=data.encode('ascii')

try: #Request without User-Agent header will return 403 Forbidden
    req = urllib.request.Request(url=request_url, data=data)

    with urllib.request.urlopen(req, context=ctx) as f:
        print("Response with no User-Agent Header")
        print(json.loads(to_native(f.read())))
except Exception as e:
    print("Exception with no User-Agent Header")
    print(e,"\n")

try: #Request with User-Agent header will work
    req_header = urllib.request.Request(url=request_url,headers={"User-Agent": "Ansible"},data=data)

    with urllib.request.urlopen(req_header, context=ctx) as f:
        print("Response with User-Agent Header\n")
        print(f.read())
except Exception as e:
    print(e,"\n")
```

> One suggestion is to implicitly set User-Agent headers value to something like Ansible-<version>
> I will be  working on applying the fix myself will create a PR once completed. 

### Issue Type

Feature request

### Component Name

community.general(Keycloak)

### Ansible Version

```console (paste below)
$ ansible --version
ansible [core 2.12.7]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/dishant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  ansible collection location = /home/dishant/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.5 (main, Jun  9 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]
  jinja version = 3.0.3
  libyaml = True
```


### Community.general Version

```console (paste below)
$ ansible-galaxy collection list community.general

# /usr/lib/python3.10/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 4.8.2  

# /home/dishant/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 5.3.0 
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

DEFAULT_GATHERING(/etc/ansible/ansible.cfg) = smart
```


### OS / Environment

Linux fedora 5.18.13-200.fc36.x86_64

### Steps to Reproduce

Pre-requisites:

Keycloak Server Hosted with Cloudflare Proxy 

```yaml (paste below)
- name: Create or update Keycloak realm (minimal example)
  community.general.keycloak_realm:
    auth_client_id: "{{auth_client_id}}"
    auth_keycloak_url: "{{auth_keycloak_url}}"
    auth_realm: "{{auth_realm}}"
    auth_username: "{{auth_username}}"
    auth_password: "{{auth_password}}"
    id: "{{realm}}"
    realm: "{{realm}}"
    state: "{{state}}"
    validate_certs: false
```


### Expected Results

It shoud be successful

### Actual Results

```console (paste below)
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Could not obtain access token from https://kc.korifi.run/realms/master/protocol/openid-connect/token: HTTP Error 403: Forbidden"}
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
None

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Resolving patch:
diff --git a/plugins/doc_fragments/keycloak.py b/plugins/doc_fragments/keycloak.py
--- a/plugins/doc_fragments/keycloak.py
+++ b/plugins/doc_fragments/keycloak.py
@@ -68,4 +68,10 @@ class ModuleDocFragment(object):
         type: int
         default: 10
         version_added: 4.5.0
+    http_agent:
+        description:
+            - Configures the HTTP User-Agent header.
+        type: str
+        default: Ansible
+        version_added: 5.4.0
 '''
diff --git a/plugins/module_utils/identity/keycloak/keycloak.py b/plugins/module_utils/identity/keycloak/keycloak.py
--- a/plugins/module_utils/identity/keycloak/keycloak.py
+++ b/plugins/module_utils/identity/keycloak/keycloak.py
@@ -104,6 +104,7 @@ def keycloak_argument_spec():
         validate_certs=dict(type='bool', default=True),
         connection_timeout=dict(type='int', default=10),
         token=dict(type='str', no_log=True),
+        http_agent=dict(type='str', default='Ansible'),
     )
 
 
@@ -123,6 +124,7 @@ def get_token(module_params):
     """
     token = module_params.get('token')
     base_url = module_params.get('auth_keycloak_url')
+    http_agent = module_params.get('http_agent')
 
     if not base_url.lower().startswith(('http', 'https')):
         raise KeycloakError("auth_url '%s' should either start with 'http' or 'https'." % base_url)
@@ -149,7 +151,7 @@ def get_token(module_params):
             (k, v) for k, v in temp_payload.items() if v is not None)
         try:
             r = json.loads(to_native(open_url(auth_url, method='POST',
-                                              validate_certs=validate_certs, timeout=connection_timeout,
+                                              validate_certs=validate_certs, http_agent=http_agent, timeout=connection_timeout,
                                               data=urlencode(payload)).read()))
         except ValueError as e:
             raise KeycloakError(
@@ -233,6 +235,7 @@ def __init__(self, module, connection_header):
         self.validate_certs = self.module.params.get('validate_certs')
         self.connection_timeout = self.module.params.get('connection_timeout')
         self.restheaders = connection_header
+        self.http_agent = self.module.params.get('http_agent')
 
     def get_realm_info_by_id(self, realm='master'):
         """ Obtain realm public info by id
@@ -243,7 +246,8 @@ def get_realm_info_by_id(self, realm='master'):
         realm_info_url = URL_REALM_INFO.format(url=self.baseurl, realm=realm)
 
         try:
-            return json.loads(to_native(open_url(realm_info_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(realm_info_url, method='GET', http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -268,7 +272,7 @@ def get_realm_by_id(self, realm='master'):
         realm_url = URL_REALM.format(url=self.baseurl, realm=realm)
 
         try:
-            return json.loads(to_native(open_url(realm_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(realm_url, method='GET', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -293,7 +297,7 @@ def update_realm(self, realmrep, realm="master"):
         realm_url = URL_REALM.format(url=self.baseurl, realm=realm)
 
         try:
-            return open_url(realm_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(realm_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(realmrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update realm %s: %s' % (realm, str(e)),
@@ -307,7 +311,7 @@ def create_realm(self, realmrep):
         realm_url = URL_REALMS.format(url=self.baseurl)
 
         try:
-            return open_url(realm_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(realm_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(realmrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create realm %s: %s' % (realmrep['id'], str(e)),
@@ -322,7 +326,7 @@ def delete_realm(self, realm="master"):
         realm_url = URL_REALM.format(url=self.baseurl, realm=realm)
 
         try:
-            return open_url(realm_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(realm_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not delete realm %s: %s' % (realm, str(e)),
@@ -340,7 +344,8 @@ def get_clients(self, realm='master', filter=None):
             clientlist_url += '?clientId=%s' % filter
 
         try:
-            return json.loads(to_native(open_url(clientlist_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(clientlist_url, http_agent=self.http_agent, method='GET', headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of clients for realm %s: %s'
@@ -371,7 +376,8 @@ def get_client_by_id(self, id, realm='master'):
         client_url = URL_CLIENT.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return json.loads(to_native(open_url(client_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(client_url, method='GET', http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -410,7 +416,7 @@ def update_client(self, id, clientrep, realm="master"):
         client_url = URL_CLIENT.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return open_url(client_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(client_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clientrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update client %s in realm %s: %s'
@@ -425,7 +431,7 @@ def create_client(self, clientrep, realm="master"):
         client_url = URL_CLIENTS.format(url=self.baseurl, realm=realm)
 
         try:
-            return open_url(client_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(client_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clientrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create client %s in realm %s: %s'
@@ -441,7 +447,7 @@ def delete_client(self, id, realm="master"):
         client_url = URL_CLIENT.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return open_url(client_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(client_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not delete client %s in realm %s: %s'
@@ -456,7 +462,8 @@ def get_client_roles_by_id(self, cid, realm="master"):
         """
         client_roles_url = URL_CLIENT_ROLES.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return json.loads(to_native(open_url(client_roles_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(client_roles_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch rolemappings for client %s in realm %s: %s"
@@ -488,7 +495,8 @@ def get_client_rolemapping_by_id(self, gid, cid, rid, realm='master'):
         """
         rolemappings_url = URL_CLIENT_ROLEMAPPINGS.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            rolemappings = json.loads(to_native(open_url(rolemappings_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            rolemappings = json.loads(to_native(open_url(rolemappings_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                         timeout=self.connection_timeout,
                                                          validate_certs=self.validate_certs).read()))
             for role in rolemappings:
                 if rid == role['id']:
@@ -508,7 +516,8 @@ def get_client_available_rolemappings(self, gid, cid, realm="master"):
         """
         available_rolemappings_url = URL_CLIENT_ROLEMAPPINGS_AVAILABLE.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            return json.loads(to_native(open_url(available_rolemappings_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(available_rolemappings_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch available rolemappings for client %s in group %s, realm %s: %s"
@@ -524,7 +533,8 @@ def get_client_composite_rolemappings(self, gid, cid, realm="master"):
         """
         available_rolemappings_url = URL_CLIENT_ROLEMAPPINGS_COMPOSITE.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            return json.loads(to_native(open_url(available_rolemappings_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(available_rolemappings_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch available rolemappings for client %s in group %s, realm %s: %s"
@@ -541,7 +551,7 @@ def add_group_rolemapping(self, gid, cid, role_rep, realm="master"):
         """
         available_rolemappings_url = URL_CLIENT_ROLEMAPPINGS.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            open_url(available_rolemappings_url, method="POST", headers=self.restheaders, data=json.dumps(role_rep),
+            open_url(available_rolemappings_url, method="POST", http_agent=self.http_agent, headers=self.restheaders, data=json.dumps(role_rep),
                      validate_certs=self.validate_certs, timeout=self.connection_timeout)
         except Exception as e:
             self.module.fail_json(msg="Could not fetch available rolemappings for client %s in group %s, realm %s: %s"
@@ -558,7 +568,7 @@ def delete_group_rolemapping(self, gid, cid, role_rep, realm="master"):
         """
         available_rolemappings_url = URL_CLIENT_ROLEMAPPINGS.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            open_url(available_rolemappings_url, method="DELETE", headers=self.restheaders,
+            open_url(available_rolemappings_url, method="DELETE", http_agent=self.http_agent, headers=self.restheaders,
                      validate_certs=self.validate_certs, timeout=self.connection_timeout)
         except Exception as e:
             self.module.fail_json(msg="Could not delete available rolemappings for client %s in group %s, realm %s: %s"
@@ -573,7 +583,7 @@ def get_client_templates(self, realm='master'):
         url = URL_CLIENTTEMPLATES.format(url=self.baseurl, realm=realm)
 
         try:
-            return json.loads(to_native(open_url(url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(url, method='GET', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of client templates for realm %s: %s'
@@ -592,7 +602,7 @@ def get_client_template_by_id(self, id, realm='master'):
         url = URL_CLIENTTEMPLATE.format(url=self.baseurl, id=id, realm=realm)
 
         try:
-            return json.loads(to_native(open_url(url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(url, method='GET', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain client templates %s for realm %s: %s'
@@ -638,7 +648,7 @@ def update_client_template(self, id, clienttrep, realm="master"):
         url = URL_CLIENTTEMPLATE.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return open_url(url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clienttrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update client template %s in realm %s: %s'
@@ -653,7 +663,7 @@ def create_client_template(self, clienttrep, realm="master"):
         url = URL_CLIENTTEMPLATES.format(url=self.baseurl, realm=realm)
 
         try:
-            return open_url(url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clienttrep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create client template %s in realm %s: %s'
@@ -669,7 +679,7 @@ def delete_client_template(self, id, realm="master"):
         url = URL_CLIENTTEMPLATE.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return open_url(url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not delete client template %s in realm %s: %s'
@@ -686,7 +696,8 @@ def get_clientscopes(self, realm="master"):
         """
         clientscopes_url = URL_CLIENTSCOPES.format(url=self.baseurl, realm=realm)
         try:
-            return json.loads(to_native(open_url(clientscopes_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(clientscopes_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch list of clientscopes in realm %s: %s"
@@ -703,7 +714,8 @@ def get_clientscope_by_clientscopeid(self, cid, realm="master"):
         """
         clientscope_url = URL_CLIENTSCOPE.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return json.loads(to_native(open_url(clientscope_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(clientscope_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -748,7 +760,7 @@ def create_clientscope(self, clientscoperep, realm="master"):
         """
         clientscopes_url = URL_CLIENTSCOPES.format(url=self.baseurl, realm=realm)
         try:
-            return open_url(clientscopes_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(clientscopes_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clientscoperep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg="Could not create clientscope %s in realm %s: %s"
@@ -763,7 +775,7 @@ def update_clientscope(self, clientscoperep, realm="master"):
         clientscope_url = URL_CLIENTSCOPE.format(url=self.baseurl, realm=realm, id=clientscoperep['id'])
 
         try:
-            return open_url(clientscope_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(clientscope_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(clientscoperep), validate_certs=self.validate_certs)
 
         except Exception as e:
@@ -801,7 +813,7 @@ def delete_clientscope(self, name=None, cid=None, realm="master"):
         # should have a good cid by here.
         clientscope_url = URL_CLIENTSCOPE.format(realm=realm, id=cid, url=self.baseurl)
         try:
-            return open_url(clientscope_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(clientscope_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
 
         except Exception as e:
@@ -819,7 +831,8 @@ def get_clientscope_protocolmappers(self, cid, realm="master"):
         """
         protocolmappers_url = URL_CLIENTSCOPE_PROTOCOLMAPPERS.format(id=cid, url=self.baseurl, realm=realm)
         try:
-            return json.loads(to_native(open_url(protocolmappers_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(protocolmappers_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch list of protocolmappers in realm %s: %s"
@@ -838,7 +851,8 @@ def get_clientscope_protocolmapper_by_protocolmapperid(self, pid, cid, realm="ma
         """
         protocolmapper_url = URL_CLIENTSCOPE_PROTOCOLMAPPER.format(url=self.baseurl, realm=realm, id=cid, mapper_id=pid)
         try:
-            return json.loads(to_native(open_url(protocolmapper_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(protocolmapper_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -885,7 +899,7 @@ def create_clientscope_protocolmapper(self, cid, mapper_rep, realm="master"):
         """
         protocolmappers_url = URL_CLIENTSCOPE_PROTOCOLMAPPERS.format(url=self.baseurl, id=cid, realm=realm)
         try:
-            return open_url(protocolmappers_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(protocolmappers_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(mapper_rep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg="Could not create protocolmapper %s in realm %s: %s"
@@ -901,7 +915,7 @@ def update_clientscope_protocolmappers(self, cid, mapper_rep, realm="master"):
         protocolmapper_url = URL_CLIENTSCOPE_PROTOCOLMAPPER.format(url=self.baseurl, realm=realm, id=cid, mapper_id=mapper_rep['id'])
 
         try:
-            return open_url(protocolmapper_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(protocolmapper_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(mapper_rep), validate_certs=self.validate_certs)
 
         except Exception as e:
@@ -918,7 +932,8 @@ def get_groups(self, realm="master"):
         """
         groups_url = URL_GROUPS.format(url=self.baseurl, realm=realm)
         try:
-            return json.loads(to_native(open_url(groups_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(groups_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg="Could not fetch list of groups in realm %s: %s"
@@ -935,7 +950,8 @@ def get_group_by_groupid(self, gid, realm="master"):
         """
         groups_url = URL_GROUP.format(url=self.baseurl, realm=realm, groupid=gid)
         try:
-            return json.loads(to_native(open_url(groups_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(groups_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
 
         except HTTPError as e:
@@ -981,7 +997,7 @@ def create_group(self, grouprep, realm="master"):
         """
         groups_url = URL_GROUPS.format(url=self.baseurl, realm=realm)
         try:
-            return open_url(groups_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(groups_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(grouprep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg="Could not create group %s in realm %s: %s"
@@ -996,7 +1012,7 @@ def update_group(self, grouprep, realm="master"):
         group_url = URL_GROUP.format(url=self.baseurl, realm=realm, groupid=grouprep['id'])
 
         try:
-            return open_url(group_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(group_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(grouprep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update group %s in realm %s: %s'
@@ -1033,7 +1049,7 @@ def delete_group(self, name=None, groupid=None, realm="master"):
         # should have a good groupid by here.
         group_url = URL_GROUP.format(realm=realm, groupid=groupid, url=self.baseurl)
         try:
-            return open_url(group_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(group_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg="Unable to delete group %s: %s" % (groupid, str(e)))
@@ -1046,7 +1062,8 @@ def get_realm_roles(self, realm='master'):
         """
         rolelist_url = URL_REALM_ROLES.format(url=self.baseurl, realm=realm)
         try:
-            return json.loads(to_native(open_url(rolelist_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(rolelist_url, method='GET', http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of roles for realm %s: %s'
@@ -1064,7 +1081,7 @@ def get_realm_role(self, name, realm='master'):
         """
         role_url = URL_REALM_ROLE.format(url=self.baseurl, realm=realm, name=quote(name))
         try:
-            return json.loads(to_native(open_url(role_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(role_url, method="GET", http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except HTTPError as e:
             if e.code == 404:
@@ -1084,7 +1101,7 @@ def create_realm_role(self, rolerep, realm='master'):
         """
         roles_url = URL_REALM_ROLES.format(url=self.baseurl, realm=realm)
         try:
-            return open_url(roles_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(roles_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(rolerep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create role %s in realm %s: %s'
@@ -1098,7 +1115,7 @@ def update_realm_role(self, rolerep, realm='master'):
         """
         role_url = URL_REALM_ROLE.format(url=self.baseurl, realm=realm, name=quote(rolerep['name']))
         try:
-            return open_url(role_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(role_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(rolerep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update role %s in realm %s: %s'
@@ -1112,7 +1129,7 @@ def delete_realm_role(self, name, realm='master'):
         """
         role_url = URL_REALM_ROLE.format(url=self.baseurl, realm=realm, name=quote(name))
         try:
-            return open_url(role_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(role_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Unable to delete role %s in realm %s: %s'
@@ -1131,7 +1148,8 @@ def get_client_roles(self, clientid, realm='master'):
                                       % (clientid, realm))
         rolelist_url = URL_CLIENT_ROLES.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return json.loads(to_native(open_url(rolelist_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(rolelist_url, method='GET', http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of roles for client %s in realm %s: %s'
@@ -1155,7 +1173,7 @@ def get_client_role(self, name, clientid, realm='master'):
                                       % (clientid, realm))
         role_url = URL_CLIENT_ROLE.format(url=self.baseurl, realm=realm, id=cid, name=quote(name))
         try:
-            return json.loads(to_native(open_url(role_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(role_url, method="GET", http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except HTTPError as e:
             if e.code == 404:
@@ -1181,7 +1199,7 @@ def create_client_role(self, rolerep, clientid, realm='master'):
                                       % (clientid, realm))
         roles_url = URL_CLIENT_ROLES.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return open_url(roles_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(roles_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(rolerep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create role %s for client %s in realm %s: %s'
@@ -1201,7 +1219,7 @@ def update_client_role(self, rolerep, clientid, realm="master"):
                                       % (clientid, realm))
         role_url = URL_CLIENT_ROLE.format(url=self.baseurl, realm=realm, id=cid, name=quote(rolerep['name']))
         try:
-            return open_url(role_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(role_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(rolerep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update role %s for client %s in realm %s: %s'
@@ -1220,7 +1238,7 @@ def delete_client_role(self, name, clientid, realm="master"):
                                       % (clientid, realm))
         role_url = URL_CLIENT_ROLE.format(url=self.baseurl, realm=realm, id=cid, name=quote(name))
         try:
-            return open_url(role_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(role_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Unable to delete role %s for client %s in realm %s: %s'
@@ -1237,7 +1255,8 @@ def get_authentication_flow_by_alias(self, alias, realm='master'):
             authentication_flow = {}
             # Check if the authentication flow exists on the Keycloak serveraders
             authentications = json.load(open_url(URL_AUTHENTICATION_FLOWS.format(url=self.baseurl, realm=realm), method='GET',
-                                                 headers=self.restheaders, timeout=self.connection_timeout, validate_certs=self.validate_certs))
+                                                 http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout, validate_certs=self.validate_certs))
             for authentication in authentications:
                 if authentication["alias"] == alias:
                     authentication_flow = authentication
@@ -1256,7 +1275,7 @@ def delete_authentication_flow_by_id(self, id, realm='master'):
         flow_url = URL_AUTHENTICATION_FLOW.format(url=self.baseurl, realm=realm, id=id)
 
         try:
-            return open_url(flow_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(flow_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not delete authentication flow %s in realm %s: %s'
@@ -1279,7 +1298,7 @@ def copy_auth_flow(self, config, realm='master'):
                     realm=realm,
                     copyfrom=quote(config["copyFrom"])),
                 method='POST',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(new_name),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1288,7 +1307,7 @@ def copy_auth_flow(self, config, realm='master'):
                     URL_AUTHENTICATION_FLOWS.format(url=self.baseurl,
                                                     realm=realm),
                     method='GET',
-                    headers=self.restheaders,
+                    http_agent=self.http_agent, headers=self.restheaders,
                     timeout=self.connection_timeout,
                     validate_certs=self.validate_certs))
             for flow in flow_list:
@@ -1318,7 +1337,7 @@ def create_empty_auth_flow(self, config, realm='master'):
                     url=self.baseurl,
                     realm=realm),
                 method='POST',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(new_flow),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1328,7 +1347,7 @@ def create_empty_auth_flow(self, config, realm='master'):
                         url=self.baseurl,
                         realm=realm),
                     method='GET',
-                    headers=self.restheaders,
+                    http_agent=self.http_agent, headers=self.restheaders,
                     timeout=self.connection_timeout,
                     validate_certs=self.validate_certs))
             for flow in flow_list:
@@ -1353,7 +1372,7 @@ def update_authentication_executions(self, flowAlias, updatedExec, realm='master
                     realm=realm,
                     flowalias=quote(flowAlias)),
                 method='PUT',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(updatedExec),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1374,7 +1393,7 @@ def add_authenticationConfig_to_execution(self, executionId, authenticationConfi
                     realm=realm,
                     id=executionId),
                 method='POST',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(authenticationConfig),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1399,7 +1418,7 @@ def create_subflow(self, subflowName, flowAlias, realm='master'):
                     realm=realm,
                     flowalias=quote(flowAlias)),
                 method='POST',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(newSubFlow),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1423,7 +1442,7 @@ def create_execution(self, execution, flowAlias, realm='master'):
                     realm=realm,
                     flowalias=quote(flowAlias)),
                 method='POST',
-                headers=self.restheaders,
+                http_agent=self.http_agent, headers=self.restheaders,
                 data=json.dumps(newExec),
                 timeout=self.connection_timeout,
                 validate_certs=self.validate_certs)
@@ -1447,7 +1466,7 @@ def change_execution_priority(self, executionId, diff, realm='master'):
                             realm=realm,
                             id=executionId),
                         method='POST',
-                        headers=self.restheaders,
+                        http_agent=self.http_agent, headers=self.restheaders,
                         timeout=self.connection_timeout,
                         validate_certs=self.validate_certs)
             elif diff < 0:
@@ -1458,7 +1477,7 @@ def change_execution_priority(self, executionId, diff, realm='master'):
                             realm=realm,
                             id=executionId),
                         method='POST',
-                        headers=self.restheaders,
+                        http_agent=self.http_agent, headers=self.restheaders,
                         timeout=self.connection_timeout,
                         validate_certs=self.validate_certs)
         except Exception as e:
@@ -1480,7 +1499,7 @@ def get_executions_representation(self, config, realm='master'):
                         realm=realm,
                         flowalias=quote(config["alias"])),
                     method='GET',
-                    headers=self.restheaders,
+                    http_agent=self.http_agent, headers=self.restheaders,
                     timeout=self.connection_timeout,
                     validate_certs=self.validate_certs))
             for execution in executions:
@@ -1493,7 +1512,7 @@ def get_executions_representation(self, config, realm='master'):
                                 realm=realm,
                                 id=execConfigId),
                             method='GET',
-                            headers=self.restheaders,
+                            http_agent=self.http_agent, headers=self.restheaders,
                             timeout=self.connection_timeout,
                             validate_certs=self.validate_certs))
                     execution["authenticationConfig"] = execConfig
@@ -1509,7 +1528,7 @@ def get_identity_providers(self, realm='master'):
         """
         idps_url = URL_IDENTITY_PROVIDERS.format(url=self.baseurl, realm=realm)
         try:
-            return json.loads(to_native(open_url(idps_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(idps_url, method='GET', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of identity providers for realm %s: %s'
@@ -1526,7 +1545,7 @@ def get_identity_provider(self, alias, realm='master'):
         """
         idp_url = URL_IDENTITY_PROVIDER.format(url=self.baseurl, realm=realm, alias=alias)
         try:
-            return json.loads(to_native(open_url(idp_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(idp_url, method="GET", http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except HTTPError as e:
             if e.code == 404:
@@ -1546,7 +1565,7 @@ def create_identity_provider(self, idprep, realm='master'):
         """
         idps_url = URL_IDENTITY_PROVIDERS.format(url=self.baseurl, realm=realm)
         try:
-            return open_url(idps_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(idps_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(idprep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create identity provider %s in realm %s: %s'
@@ -1560,7 +1579,7 @@ def update_identity_provider(self, idprep, realm='master'):
         """
         idp_url = URL_IDENTITY_PROVIDER.format(url=self.baseurl, realm=realm, alias=idprep['alias'])
         try:
-            return open_url(idp_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(idp_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(idprep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update identity provider %s in realm %s: %s'
@@ -1573,7 +1592,7 @@ def delete_identity_provider(self, alias, realm='master'):
         """
         idp_url = URL_IDENTITY_PROVIDER.format(url=self.baseurl, realm=realm, alias=alias)
         try:
-            return open_url(idp_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(idp_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Unable to delete identity provider %s in realm %s: %s'
@@ -1587,7 +1606,8 @@ def get_identity_provider_mappers(self, alias, realm='master'):
         """
         mappers_url = URL_IDENTITY_PROVIDER_MAPPERS.format(url=self.baseurl, realm=realm, alias=alias)
         try:
-            return json.loads(to_native(open_url(mappers_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(mappers_url, method='GET', http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of identity provider mappers for idp %s in realm %s: %s'
@@ -1605,7 +1625,8 @@ def get_identity_provider_mapper(self, mid, alias, realm='master'):
         """
         mapper_url = URL_IDENTITY_PROVIDER_MAPPER.format(url=self.baseurl, realm=realm, alias=alias, id=mid)
         try:
-            return json.loads(to_native(open_url(mapper_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(mapper_url, method="GET", http_agent=self.http_agent, headers=self.restheaders,
+                                                 timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except HTTPError as e:
             if e.code == 404:
@@ -1626,7 +1647,7 @@ def create_identity_provider_mapper(self, mapper, alias, realm='master'):
         """
         mappers_url = URL_IDENTITY_PROVIDER_MAPPERS.format(url=self.baseurl, realm=realm, alias=alias)
         try:
-            return open_url(mappers_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(mappers_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(mapper), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not create identity provider mapper %s for idp %s in realm %s: %s'
@@ -1641,7 +1662,7 @@ def update_identity_provider_mapper(self, mapper, alias, realm='master'):
         """
         mapper_url = URL_IDENTITY_PROVIDER_MAPPER.format(url=self.baseurl, realm=realm, alias=alias, id=mapper['id'])
         try:
-            return open_url(mapper_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(mapper_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(mapper), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update mapper %s for identity provider %s in realm %s: %s'
@@ -1655,7 +1676,7 @@ def delete_identity_provider_mapper(self, mid, alias, realm='master'):
         """
         mapper_url = URL_IDENTITY_PROVIDER_MAPPER.format(url=self.baseurl, realm=realm, alias=alias, id=mid)
         try:
-            return open_url(mapper_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(mapper_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Unable to delete mapper %s for identity provider %s in realm %s: %s'
@@ -1672,7 +1693,7 @@ def get_components(self, filter=None, realm='master'):
             comps_url += '?%s' % filter
 
         try:
-            return json.loads(to_native(open_url(comps_url, method='GET', headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(comps_url, method='GET', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except ValueError as e:
             self.module.fail_json(msg='API returned incorrect JSON when trying to obtain list of components for realm %s: %s'
@@ -1689,7 +1710,7 @@ def get_component(self, cid, realm='master'):
         """
         comp_url = URL_COMPONENT.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return json.loads(to_native(open_url(comp_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(comp_url, method="GET", http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except HTTPError as e:
             if e.code == 404:
@@ -1709,13 +1730,13 @@ def create_component(self, comprep, realm='master'):
         """
         comps_url = URL_COMPONENTS.format(url=self.baseurl, realm=realm)
         try:
-            resp = open_url(comps_url, method='POST', headers=self.restheaders, timeout=self.connection_timeout,
+            resp = open_url(comps_url, method='POST', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(comprep), validate_certs=self.validate_certs)
             comp_url = resp.getheader('Location')
             if comp_url is None:
                 self.module.fail_json(msg='Could not create component in realm %s: %s'
                                           % (realm, 'unexpected response'))
-            return json.loads(to_native(open_url(comp_url, method="GET", headers=self.restheaders, timeout=self.connection_timeout,
+            return json.loads(to_native(open_url(comp_url, method="GET", http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                                                  validate_certs=self.validate_certs).read()))
         except Exception as e:
             self.module.fail_json(msg='Could not create component in realm %s: %s'
@@ -1732,7 +1753,7 @@ def update_component(self, comprep, realm='master'):
             self.module.fail_json(msg='Cannot update component without id')
         comp_url = URL_COMPONENT.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return open_url(comp_url, method='PUT', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(comp_url, method='PUT', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             data=json.dumps(comprep), validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Could not update component %s in realm %s: %s'
@@ -1745,7 +1766,7 @@ def delete_component(self, cid, realm='master'):
         """
         comp_url = URL_COMPONENT.format(url=self.baseurl, realm=realm, id=cid)
         try:
-            return open_url(comp_url, method='DELETE', headers=self.restheaders, timeout=self.connection_timeout,
+            return open_url(comp_url, method='DELETE', http_agent=self.http_agent, headers=self.restheaders, timeout=self.connection_timeout,
                             validate_certs=self.validate_certs)
         except Exception as e:
             self.module.fail_json(msg='Unable to delete component %s in realm %s: %s'


Test patch: