Repository: netbox-community/netbox

Issue:
Integration with AUTH_LDAP_FIND_GROUP_PERMS does not work
### Environment

* Python version: 
* NetBox version: 2.9.10

### Steps to Reproduce

1. Configure LDAP authentication
2. Enable `AUTH_LDAP_FIND_GROUP_PERMS`
3. Create in the admin panel group that would be returned by LDAP and grant it some permissions (e.g. users.add_token)
4. **DO NOT** add user into the group, this should be done explicitly by the `django-auth-ldap` module

### Expected Behavior

User is able to use those permissions.

<!-- What happened instead? -->
### Observed Behavior

User is getting permission denied on creating api token.

### Analyze

I've spent many hours trying to find out why users can't have permissions in our netbox installation but it does seem to me like a bug in Netbox.

What django-auth-ldap is doing:

```python
    def _load_group_permissions(self):
        """
        Populates self._group_permissions based on LDAP group membership and
        Django group permissions.
        """
        group_names = self._get_groups().get_group_names()
 
        perms = Permission.objects.filter(group__name__in=group_names)
        ...
```

Here it gets empty `QuerySet`.

Enabling `DEBUG` and inspecting SQL:

```sql
SELECT "auth_permission"."id",
       "auth_permission"."name",
       "auth_permission"."content_type_id",
       "auth_permission"."codename"
  FROM "auth_permission"
 INNER JOIN "auth_group_permissions"
    ON ("auth_permission"."id" = "auth_group_permissions"."permission_id")
 INNER JOIN "auth_group"
    ON ("auth_group_permissions"."group_id" = "auth_group"."id")
 INNER JOIN "django_content_type"
    ON ("auth_permission"."content_type_id" = "django_content_type"."id")
 WHERE "auth_group"."name" IN ('github-private-cloud-elders', 'gapps', 'team-privatecloud', 'github-na-elders', 'mgmt-prod', 'confluence-users', 'github-keyring-elders', 'pagerduty-users', 'ipausers', 'github-tools-s3provision-hackers', 'github-gooddata-hackers', 'jira-gooddata-users', 'role-swdeveloper')
 ORDER BY "django_content_type"."app_label" ASC, "django_content_type"."model" ASC, "auth_permission"."codename" ASC
 LIMIT 21
```

It is trying to get info from the `auth_group_permissions`, but inspecting the database it shows completely empty table. Even there is some permissions assigned to the group.

```
netbox=# select * from auth_group;
 id |               name
----+----------------------------------
  3 | ipausers
 16 | github-gooddata-hackers
 17 | github-keyring-elders
 18 | pagerduty-users
 19 | gapps
 20 | role-swdeveloper
 21 | jira-gooddata-users
 22 | github-tools-s3provision-hackers
 23 | github-na-elders
 24 | mgmt-prod
 25 | github-private-cloud-elders
 26 | team-privatecloud
 27 | confluence-users
(13 rows)
 
netbox=# select * from auth_group_permissions;
 id | group_id | permission_id
----+----------+---------------
(0 rows)
 
netbox=#  
```

Dumping the DB, removing group permission, dumping DB again and comparing shows something interesting:

```diff
@@ -5522,6 +5522,7 @@
 37     2020-12-11 13:35:00.497982+01   10      lubomir.tomecek 2       [{"changed": {"fields": ["Groups"]}}]   80      3                                                                                           
 38     2020-12-11 13:37:53.0537+01     10      lubomir.tomecek 2       [{"changed": {"fields": ["Groups"]}}]   80      3                                                                                           
 39     2020-12-11 13:42:45.743739+01   3       Allow managing API tokens       2       [{"changed": {"fields": ["Users"]}}]    81      3                                                                           
+40     2020-12-11 13:43:36.651894+01   3       Allow managing API tokens       2       [{"changed": {"fields": ["Groups"]}}]   81      3                                                                           
 \.
 
 
@@ -6273,7 +6274,6 @@
 --
 
 COPY public.users_objectpermission_groups (id, objectpermission_id, group_id) FROM stdin;
-3      3       3
 \.
 
 
@@ -6667,7 +6667,7 @@
 -- Name: django_admin_log_id_seq; Type: SEQUENCE SET; Schema: public; Owner: netbox
 --
 
-SELECT pg_catalog.setval('public.django_admin_log_id_seq', 39, true);
+SELECT pg_catalog.setval('public.django_admin_log_id_seq', 40, true);
```

So it does change `users_objectpermission_groups` and not the `auth_permission_groups`. And that's why django-auth-ldap can't find any permissions (without explicitly creating group).

Seems that `ObjectPermissionBackend` written in netbox is behaving differently than the standard django one.


Discussion:
NetBox v2.9 introduced object-based permissions, which employ a custom ObjectPermission model rather than Django's built-in Permission. NetBox's LDAPBackend class likely needs to be extended to support the described behavior. Marking this as `needs owner` for anyone who would like to volunteer.
This might be a duplicate of #5125.
@jeremystretch yep, that seems to be same case, though linked ticket is missing information *why* it is broken so if you decide to close one as dupe of another - IMO would be better to close *that one* as a duplicate (even though it is older).

> NetBox's LDAPBackend class likely needs to be extended to support the described behavior

Do you have any hint about this? I could try to make a patch, but quick look into the code does not help me to guess even where to look.
I wasn't able to replicate this on v2.10.2: The group assignment works automatically as expected. Here's what I did:

1. Create a group in NetBox and assign it permission to create sites
2. Create a group of the same name in OpenLDAP
3. Create a user in OpenLDAP and assign it to the group
4. Log into NetBox using that user's LDAP credentials

The LDAP user account is automatically created and assigned to the specified group. After logging in as the user, I am granted the permissions assigned to the group.

Here's my LDAP configuration:

```
import ldap
from django_auth_ldap.config import LDAPSearch, PosixGroupType

# Server URI
AUTH_LDAP_SERVER_URI = "ldap://$SERVER_NAME_OR_IP"

# Set the DN and password for the NetBox service account.
AUTH_LDAP_BIND_DN = "userid=netbox,dc=example,dc=com"
AUTH_LDAP_BIND_PASSWORD = "netbox_ldap"

LDAP_IGNORE_CERT_ERRORS = True

AUTH_LDAP_USER_DN_TEMPLATE = "cn=%(user)s,ou=users,dc=example,dc=com"

AUTH_LDAP_USER_ATTR_MAP = {
    "first_name": "givenName",
    "last_name": "sn",
}

AUTH_LDAP_GROUP_SEARCH = LDAPSearch("dc=example,dc=com", ldap.SCOPE_SUBTREE, "(objectClass=posixGroup)")
AUTH_LDAP_GROUP_TYPE = PosixGroupType()

AUTH_LDAP_MIRROR_GROUPS = True
AUTH_LDAP_FIND_GROUP_PERMS = True

AUTH_LDAP_USER_FLAGS_BY_GROUP = {
    "is_staff": "cn=staff,ou=groups,dc=example,dc=com",
}
```

Note that I'm using `PosixGroupType` rather than `GroupOfNamesType` as given in the documentation, but I believe this is dependent on the specific LDAP backend.
> AUTH_LDAP_MIRROR_GROUPS = True

I believe this is the key. This will sync all groups of a user into the netbox and will "add user to the group".

https://django-auth-ldap.readthedocs.io/en/latest/permissions.html#group-mirroring

> The second way to turn LDAP group memberships into permissions is to mirror the groups themselves. This approach has some important disadvantages and should be avoided if possible.

@jeremystretch , it is supposed to work without that option but that is not compatible with netbox user backend.
Ok, I think I see the disconnect. My expectation is just to have the LDAP user assigned to the groups that exist in Django, so that it gets granted the associated permissions by proxy. But it looks like you're trying to assign those permissions to the user directly?

As you point out, django-auth-ldap's `_load_group_permissions()` method references Django's built-in `Permission` class, which NetBox v2.9+ no longer uses. So if the goal is to _replicate_ permissions to the individual user, this method needs to be overridden to work with NetBox's ObjectPermission class.
> But it looks like you're trying to assign those permissions to the user directly?

If I understand how `AUTH_LDAP_FIND_GROUP_PERMS` works, when user tries to do something - django-auth-ldap checks in which groups the user is and give him permissions from the group that is already created in netbox. Without adding user specifically to the group or creating new one.
That's my impression as well. This seems undesirable, though: IMO I'd rather just assign the user to the appropriate Django group(s) so that they inherit all associated permissions assigned to each group. It looks like `AUTH_LDAP_MIRROR_GROUPS` is required for that though?
@jeremystretch I don't think it actually "assigns" them. It just overrides "get_group_permissions()" from Django to return "extra" permissions without assigning user to a group. So it actually does not modify permissions DB at all, merely just returns more permissions on the fly.

That seems quite desirable because you don't have to force re-login users when you change some group assignment and so on.
Hey, any update on this ? 
We are facing this issue. New users dont get the group added to them when being auto-created by ldap.
I can see that the ldap debug shows they are member of the group.. (I tried creating groups in netbox with the whole CN and lots of other lower/uppercase combinations, but non work)

I dont want all of the ldap groups to be created in netbox wirh 'AUTH_LDAP_MIRROR_GROUPS '


We are also facing this issue.

We've started on 2.10 with a blank installation and neither `AUTH_LDAP_MIRROR_GROUPS`, `AUTH_LDAP_MIRROR_GROUPS_EXCEPT` or `AUTH_LDAP_FIND_GROUP_PERMS` are currently working.

As @PieterL75 we've tried numerous combinations of DNs, plain "True" on `AUTH_LDAP_MIRROR_GROUPS` and multiple manually created, DN-equal, groups, but no luck.

While testing, we've debug-logged django-auth-ldap & Django and ran Netbox itself in debug-Mode, but no error messages were thrown on logins. Logs show, that all infos on a DN are found and processed.

@PieterL75 
If `AUTH_LDAP_MIRROR_GROUPS` would be working, you could define Groups to mirror, if you provide a list of DNs instead of "True" or "False". And combine it with a list of undesired Group-DNs in `AUTH_LDAP_MIRROR_GROUPS_EXCEPT`.

Edit: We've had a pretty obvious configuration issue in our deployment, soooo... MIRROR_GROUPS is working flawlessy now. :)

Mirror groups does seem to workaround this issue (and other LDAP issues I've had) - however I'd recommend also setting `AUTH_LDAP_MIRROR_GROUPS` ([docs](https://django-auth-ldap.readthedocs.io/en/latest/reference.html#auth-ldap-mirror-groups)) to a list of the groups you want NetBox to mirror, otherwise you'll get unrelated LDAP groups being created in NetBox. That setting should probably be added to the NetBox docs as well.

I'd initially not used the group mirroring since the `django-auth-ldap` [docs](https://django-auth-ldap.readthedocs.io/en/latest/permissions.html?highlight=mirror#group-mirroring) had described it as less ideal. In the past I'd solely relied on `AUTH_LDAP_FIND_GROUP_PERMS` since I was manually creating the NetBox-specific groups in NetBox to line up with the LDAP groups.

However - in NetBox's case ~it~ mirroring does have advantages; API tokens will work for LDAP authenticated users, and as established in this thread, it works with the new permissions model without having to create a custom LDAP handler (though I imagine there will be some users that really don't want to mirror groups for good reasons and then will need a custom handler).
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
Please don't close it as it is real issue.
@ignatenkobrain would you like to volunteer to work on this?

Resolving patch:
diff --git a/netbox/netbox/api/authentication.py b/netbox/netbox/api/authentication.py
--- a/netbox/netbox/api/authentication.py
+++ b/netbox/netbox/api/authentication.py
@@ -25,6 +25,15 @@ def authenticate_credentials(self, key):
         if not token.user.is_active:
             raise exceptions.AuthenticationFailed("User inactive")
 
+        # When LDAP authentication is active try to load user data from LDAP directory
+        if settings.REMOTE_AUTH_BACKEND == 'netbox.authentication.LDAPBackend':
+            from netbox.authentication import LDAPBackend
+            ldap_backend = LDAPBackend()
+            user = ldap_backend.populate_user(token.user.username)
+            # If the user is found in the LDAP directory use it, if not fallback to the local user
+            if user:
+                return user, token
+
         return token.user, token
 
 
diff --git a/netbox/netbox/authentication.py b/netbox/netbox/authentication.py
--- a/netbox/netbox/authentication.py
+++ b/netbox/netbox/authentication.py
@@ -11,7 +11,7 @@
 from utilities.permissions import permission_is_exempt, resolve_permission, resolve_permission_ct
 
 
-class ObjectPermissionBackend(ModelBackend):
+class ObjectPermissionMixin():
 
     def get_all_permissions(self, user_obj, obj=None):
         if not user_obj.is_active or user_obj.is_anonymous:
@@ -20,13 +20,16 @@ def get_all_permissions(self, user_obj, obj=None):
             user_obj._object_perm_cache = self.get_object_permissions(user_obj)
         return user_obj._object_perm_cache
 
+    def get_permission_filter(self, user_obj):
+        return Q(users=user_obj) | Q(groups__user=user_obj)
+
     def get_object_permissions(self, user_obj):
         """
         Return all permissions granted to the user by an ObjectPermission.
         """
         # Retrieve all assigned and enabled ObjectPermissions
         object_permissions = ObjectPermission.objects.filter(
-            Q(users=user_obj) | Q(groups__user=user_obj),
+            self.get_permission_filter(user_obj),
             enabled=True
         ).prefetch_related('object_types')
 
@@ -86,6 +89,10 @@ def has_perm(self, user_obj, perm, obj=None):
         return model.objects.filter(constraints, pk=obj.pk).exists()
 
 
+class ObjectPermissionBackend(ObjectPermissionMixin, ModelBackend):
+    pass
+
+
 class RemoteUserBackend(_RemoteUserBackend):
     """
     Custom implementation of Django's RemoteUserBackend which provides configuration hooks for basic customization.
@@ -133,11 +140,27 @@ def has_perm(self, user_obj, perm, obj=None):
         return False
 
 
+# Create a new instance of django-auth-ldap's LDAPBackend with our own ObjectPermissions
+try:
+    from django_auth_ldap.backend import LDAPBackend as LDAPBackend_
+
+    class NBLDAPBackend(ObjectPermissionMixin, LDAPBackend_):
+        def get_permission_filter(self, user_obj):
+            permission_filter = super().get_permission_filter(user_obj)
+            if (self.settings.FIND_GROUP_PERMS and
+                    hasattr(user_obj, "ldap_user") and
+                    hasattr(user_obj.ldap_user, "group_names")):
+                permission_filter = permission_filter | Q(groups__name__in=user_obj.ldap_user.group_names)
+            return permission_filter
+except ModuleNotFoundError:
+    pass
+
+
 class LDAPBackend:
 
     def __new__(cls, *args, **kwargs):
         try:
-            from django_auth_ldap.backend import LDAPBackend as LDAPBackend_, LDAPSettings
+            from django_auth_ldap.backend import LDAPSettings
             import ldap
         except ModuleNotFoundError as e:
             if getattr(e, 'name') == 'django_auth_ldap':
@@ -163,8 +186,7 @@ def __new__(cls, *args, **kwargs):
                 "Required parameter AUTH_LDAP_SERVER_URI is missing from ldap_config.py."
             )
 
-        # Create a new instance of django-auth-ldap's LDAPBackend
-        obj = LDAPBackend_()
+        obj = NBLDAPBackend()
 
         # Read LDAP configuration parameters from ldap_config.py instead of settings.py
         settings = LDAPSettings()


Test patch: