Repository: ansible-collections/community.general

Issue:
Error when searching in Active Directory due to ldap referrals return
##### SUMMARY
<!--- Explain the problem briefly below -->
I try to use ldap_search to search for computer entries matching specific CN in an AD directory.
Playbook failed because of python-ldap library chasing of referrals

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
ldap_search
##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```
ansible 2.9.12
  config file = /home/mike/.ansible.cfg
  configured module search path = ['/home/mike/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ansible
  executable location = /home/mike/.venvs/ansible2.9.12/bin/ansible
  python version = 3.7.9 (default, Aug 19 2020, 17:05:11) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```
DEFAULT_STDOUT_CALLBACK(/home/mike/.ansible.cfg) = yaml
HOST_KEY_CHECKING(/home/mike/.ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
OS : Fedora 31

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
Search for a computer in an Active Directory.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- name: "LDAP REGISTRATION"
  hosts: localhost
  gather_facts: false
    
  tasks:
  - name: Search LDAP computer object
    delegate_to: localhost
    community.general.ldap_search:
      dn: "DC=test,DC=intra,DC=acme,DC=fr"
      filter: '(cn=myComputerName)'
      scope: children
      server_uri: "ldap://myad.acme.fr"
      bind_dn: "cn=username,cn=users,DC=test,DC=acme,DC=fr"
      bind_pw: "myPassword"
    register: search
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
Ldap search have to return the list of find objects.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
Playbook failed with error :
fatal: [myserver.acme.fr]: FAILED! => changed=false 
  details: '{''msgtype'': 100, ''msgid'': 2, ''result'': 1, ''desc'': ''Operations error'', ''ctrls'': [], ''info'': ''000004DC: LdapErr: DSID-0C09075A, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1''}'
  msg: Attribute action failed.
  
<!--- Paste verbatim command output between quotes -->
```
ansible-playbook main.yml -vvv
ansible-playbook 2.9.12
  config file = /home/mike/.ansible.cfg
  configured module search path = ['/home/mike/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ansible
  executable location = /home/mike/.venvs/ansible2.9.12/bin/ansible-playbook
  python version = 3.7.9 (default, Aug 19 2020, 17:05:11) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)]
Using /home/mike/.ansible.cfg as config file
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAYBOOK: main.yml ***********************************************************************************************************************************************
1 plays in main.yml

PLAY [LDAP REGISTRATION] *****************************************************************************************************************************************
META: ran handlers

TASK [Searching LDAP object] *************************************************************************************************************************************
task path: /home/mike/dev/snippet/ansible/ldap/main.yml:8
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mike
<localhost> EXEC /bin/sh -c 'echo ~mike && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mike/.ansible/tmp `"&& mkdir "` echo /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288 `" && echo ansible-tmp-1602250671.1481202-64238-190095329907288="` echo /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288 `" ) && sleep 0'
Using module file /home/mike/.ansible/collections/ansible_collections/community/general/plugins/modules/ldap_search.py
<localhost> PUT /home/mike/.ansible/tmp/ansible-local-64229rk_8c5ts/tmp41ux3u7k TO /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288/AnsiballZ_ldap_search.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288/ /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288/AnsiballZ_ldap_search.py && sleep 0'
<localhost> EXEC /bin/sh -c '/home/mike/.venvs/ansible2.9.12/bin/python3 /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288/AnsiballZ_ldap_search.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mike/.ansible/tmp/ansible-tmp-1602250671.1481202-64238-190095329907288/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_community.general.ldap_search_payload_xokng1tu/ansible_community.general.ldap_search_payload.zip/ansible_collections/community/general/plugins/modules/ldap_search.py", line 111, in main
  File "/tmp/ansible_community.general.ldap_search_payload_xokng1tu/ansible_community.general.ldap_search_payload.zip/ansible_collections/community/general/plugins/modules/ldap_search.py", line 163, in main
  File "/tmp/ansible_community.general.ldap_search_payload_xokng1tu/ansible_community.general.ldap_search_payload.zip/ansible_collections/community/general/plugins/modules/ldap_search.py", line 173, in perform_search
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 870, in search_s
    return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 864, in search_ext_s
    return self.result(msgid,all=1,timeout=timeout)[1]
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 756, in result
    resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 760, in result2
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 767, in result3
    resp_ctrl_classes=resp_ctrl_classes
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 774, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 340, in _ldap_call
    reraise(exc_type, exc_value, exc_traceback)
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/compat.py", line 46, in reraise
    raise exc_value
  File "/home/mike/.venvs/ansible2.9.12/lib64/python3.7/site-packages/ldap/ldapobject.py", line 324, in _ldap_call
    result = func(*args,**kwargs)
fatal: [localhost]: FAILED! => changed=false 
  details: '{''msgtype'': 101, ''msgid'': 2, ''result'': 1, ''desc'': ''Operations error'', ''ctrls'': [], ''info'': ''000004DC: LdapErr: DSID-0C09075A, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1''}'
  invocation:
    module_args:
      attrs:
      - dn
      bind_dn: cn=username,cn=users,DC=test,DC=acme,DC=fr
      bind_pw: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
      dn: "DC=test,DC=intra,DC=acme,DC=fr"
      filter: (cn=myComputerName)
      schema: false
      scope: children
      server_uri: ldap://myad.acme.fr
      start_tls: false
      validate_certs: true
  msg: Attribute action failed.

PLAY RECAP *******************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0  
```
After some investigations, it seems Active Directory returns referrals that are cheased by ldap client.
Extract from [python-ldap FAQ](https://www.python-ldap.org/en/python-ldap-3.3.0/faq.html)
![image](https://user-images.githubusercontent.com/4119982/95605949-f42eb180-0a59-11eb-8bcb-113d60935ef7.png)


After modifying ldap.py in module_utils like this:
```python
    def _connect_to_ldap(self):
        if not self.verify_cert:
            ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
        connection = ldap.initialize(self.server_uri)
        connection.set_option(ldap.OPT_REFERRALS, 0)
```
Playbooks run successfully.

Regards.


Discussion:
Files identified in the description:
* [`plugins/modules/net_tools/ldap/ldap_search.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/net_tools/ldap/ldap_search.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @eryx12o45 @jtyr @nerzhul
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Looks like a duplicate of #963.
Not sure this issue duplicated of #963 which describe the behaviour of the module when trying to parse the results returned by the ldap query when result contains referrals. On #963 the parse method have to be modified to exclude referrals from the result of the search.
In this issue, the module failed before the parsing because of the ldap client (python-ldap lib) try to follow the referral sent back from the ldap server.
On this issue the best could be to have a new module attribut like follow_referrals : yes/no to allow or deny to follow referrals.
@lamm Would you be interested in opening new PR for this? Thanks in advance.

Resolving patch:
diff --git a/plugins/doc_fragments/ldap.py b/plugins/doc_fragments/ldap.py
--- a/plugins/doc_fragments/ldap.py
+++ b/plugins/doc_fragments/ldap.py
@@ -27,6 +27,15 @@ class ModuleDocFragment(object):
     description:
       - The DN of the entry to add or remove.
     type: str
+  referrals_chasing:
+    choices: [disabled, anonymous]
+    default: anonymous
+    type: str
+    description:
+      - Set the referrals chasing behavior.
+      - C(anonymous) follow referrals anonymously. This is the default behavior.
+      - C(disabled) disable referrals chasing. This sets C(OPT_REFERRALS) to off.
+    version_added: 2.0.0
   server_uri:
     description:
       - A URI to the LDAP server.
diff --git a/plugins/module_utils/ldap.py b/plugins/module_utils/ldap.py
--- a/plugins/module_utils/ldap.py
+++ b/plugins/module_utils/ldap.py
@@ -26,6 +26,7 @@ def gen_specs(**specs):
         'bind_dn': dict(),
         'bind_pw': dict(default='', no_log=True),
         'dn': dict(required=True),
+        'referrals_chasing': dict(type='str', default='anonymous', choices=['disabled', 'anonymous']),
         'server_uri': dict(default='ldapi:///'),
         'start_tls': dict(default=False, type='bool'),
         'validate_certs': dict(default=True, type='bool'),
@@ -41,6 +42,7 @@ def __init__(self, module):
         self.bind_dn = self.module.params['bind_dn']
         self.bind_pw = self.module.params['bind_pw']
         self.dn = self.module.params['dn']
+        self.referrals_chasing = self.module.params['referrals_chasing']
         self.server_uri = self.module.params['server_uri']
         self.start_tls = self.module.params['start_tls']
         self.verify_cert = self.module.params['validate_certs']
@@ -61,6 +63,10 @@ def _connect_to_ldap(self):
 
         connection = ldap.initialize(self.server_uri)
 
+        if self.referrals_chasing == 'disabled':
+            # Switch off chasing of referrals (https://github.com/ansible-collections/community.general/issues/1067)
+            connection.set_option(ldap.OPT_REFERRALS, 0)
+
         if self.start_tls:
             try:
                 connection.start_tls_s()
diff --git a/plugins/modules/net_tools/ldap/ldap_search.py b/plugins/modules/net_tools/ldap/ldap_search.py
--- a/plugins/modules/net_tools/ldap/ldap_search.py
+++ b/plugins/modules/net_tools/ldap/ldap_search.py
@@ -168,10 +168,14 @@ def perform_search(self):
                 attrlist=self.attrlist,
                 attrsonly=self.attrsonly
             )
-            if self.schema:
-                return [dict(dn=result[0], attrs=list(result[1].keys())) for result in results]
-            else:
-                return [_extract_entry(result[0], result[1]) for result in results]
+            ldap_entries = []
+            for result in results:
+                if isinstance(result[1], dict):
+                    if self.schema:
+                        ldap_entries.append(dict(dn=result[0], attrs=list(result[1].keys())))
+                    else:
+                        ldap_entries.append(_extract_entry(result[0], result[1]))
+            return ldap_entries
         except ldap.NO_SUCH_OBJECT:
             self.module.fail_json(msg="Base not found: {0}".format(self.dn))
 


Test patch: