Repository: netbox-community/netbox
Pull request: 11508 Add group assignments for Azure SSO
### Fixes: #11508 

Creates a custom Social Auth pipeline to allow mapping Azure group-id's to NetBox groups so groups, staff and superuser assignments are automatically done when using Azure SSO.

Problem statement:
Using AzureAD authentication I cannot get the group members for the user that authenticates
### NetBox version

v3.3.8

### Python version

3.9

### Steps to Reproduce

1. I authenticate using AzureAD with my user
2. I get authenticated
3. I check the user groups in admin with a superuser account
4. I don't see any group even if the user has several groups in AzureAD
5. I check in the user id_token in User social auths (inside the meny PYTHON SOCIAL AUTH) and I verify that I get the group id

### Expected Behavior

The expected behaviour would be to see the member groups related to the user that is authenticated using AzureAD

### Observed Behavior

No groups appeared related to the user.

My configurations related with authentication is:

REMOTE_AUTH_BACKEND = 'social_core.backends.azuread.AzureADOAuth2'

SOCIAL_AUTH_AZUREAD_OAUTH2_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
SOCIAL_AUTH_REDIRECT_IS_HTTPS = True

Hints from issue discussion:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. **Do not** attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
> The expected behaviour would be to see the member groups related to the user that is authenticated using AzureAD

Neither the NetBox documentation nor the documentation for the upstream python-social-auth package suggests that this functionality exists. This should have been submitted as a feature request.

I'm marking this as `needs owner` for anyone who would like to dig into this, however some preliminary analysis suggests that the upstream Azure AD authentication backend would need to be extended to support group assignment.
I tried to do it for Keycloak, and it broke things badly. Here's a potted history in case it helps.

In https://github.com/python-social-auth/social-core/pull/668, in an attempt to be helpful I also picked up the "groups" claim and returned it in the user structure from all OIDC responses:

```
    def get_user_details(self, response):
        username_key = self.setting('USERNAME_KEY', default=self.USERNAME_KEY)
        return {
            'username': response.get(username_key),
            'email': response.get('email'),
            'fullname': response.get('name'),
            'first_name': response.get('given_name'),
            'last_name': response.get('family_name'),
            'groups': response.get('groups'),    # <<<<<<<<< returns a list of strings
        }
```

But it turns out that if you do this, Django barfs, and this change had to be rolled back: see https://github.com/python-social-auth/social-core/issues/690 / https://github.com/python-social-auth/social-core/pull/692

The actual error from Django is:

```
Direct assignment to the forward side of a many-to-many set is prohibited. Use groups.set() instead.
```

If you can work out where that's coming from, you may be able to fix this issue. There's a full backtrace at https://github.com/netbox-community/netbox/discussions/8579#discussioncomment-2200174
Thanks @candlerb, that's very helpful. I suspected that this sort of change would be necessary so it's nice to have that confirmed.

> If you can work out where that's coming from, you may be able to fix this issue.

I assume that the dictionary of attributes returned by the backend is being applied to the user instance blindly. `groups` would cause an issue because it's a reverse manager (rather than a normal attribute). So I'm not sure how it's working (_if_ it's working) for other backends as you pointed out [here](https://github.com/python-social-auth/social-core/issues/690#issuecomment-1168627022).
Hi, just in case that helps, the workaround I found was not using AzureAD but SAML against AzureAD and store groups in SOCIAL_AUTH_SAML_EXTRA_DATA to use them with a custom pipeline. Might be considered as "dirty" (and the code far from perfect)
https://www.reddit.com/r/Netbox/comments/104ouqz/netbox_saml_with_azure_ad_authorization/
There's now a feature request to implement this ability natively within `python-social-auth`: https://github.com/python-social-auth/social-core/issues/794

Commit message: 11508 remove addition to settings

Patch to review:
diff --git a/docs/administration/authentication/microsoft-azure-ad.md b/docs/administration/authentication/microsoft-azure-ad.md
index ee24e8232bb..a97c22b2d78 100644
--- a/docs/administration/authentication/microsoft-azure-ad.md
+++ b/docs/administration/authentication/microsoft-azure-ad.md
@@ -61,6 +61,43 @@ Restart the NetBox services so that the new configuration takes effect. This is
 sudo systemctl restart netbox
 ```
 
+## Group Assignment
+
+If you want NetBox to assign groups based on Azure AD groups, then some additonal configuration is needed. Enter the following configuration parameters in `configuration.py`, substituting your own values:
+
+```python
+SOCIAL_AUTH_AZUREAD_OAUTH2_RESOURCE = 'https://graph.microsoft.com/'
+SOCIAL_AUTH_PIPELINE = (
+    'social_core.pipeline.social_auth.social_details',
+    'social_core.pipeline.social_auth.social_uid',
+    'social_core.pipeline.social_auth.social_user',
+    'social_core.pipeline.user.get_username',
+    'social_core.pipeline.social_auth.associate_by_email',
+    'social_core.pipeline.user.create_user',
+    'social_core.pipeline.social_auth.associate_user',
+    'netbox.authentication.user_default_groups_handler',
+    'social_core.pipeline.social_auth.load_extra_data',
+    'social_core.pipeline.user.user_details',
+    'netbox.authentication.azuread_map_groups',
+)
+SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS = True
+
+# Define special user types using groups. Exercise great caution when assigning superuser status.
+SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP = {
+    "is_staff": ['{AZURE_GROUP_ID}',],
+    "is_superuser": ['{AZURE_GROUP_ID}',]
+}
+
+SOCIAL_AUTH_AZUREAD_GROUP_MAP = {
+    '{AZURE_GROUP_ID}': '{NETBOX_GROUP}',
+}
+```
+**SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP.is_staff**: users who are in any of the Azure AD group-ids in the array will have staff permission assigned to them.
+
+**SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP.is_superuser**: users who are in any of the Azure AD group-ids in the array will have superuser permission assigned to them.
+
+**SOCIAL_AUTH_AZUREAD_GROUP_MAP**: Any user with the given Azure AD group-id is included in the given NetBox group name.
+
 ## Testing
 
 Log out of NetBox if already authenticated, and click the "Log In" button at top right. You should see the normal login form as well as an option to authenticate using Azure AD. Click that link.
diff --git a/netbox/netbox/authentication.py b/netbox/netbox/authentication.py
index 61dfe2fdbdc..acc3a178311 100644
--- a/netbox/netbox/authentication.py
+++ b/netbox/netbox/authentication.py
@@ -1,4 +1,5 @@
 import logging
+import requests
 from collections import defaultdict
 
 from django.conf import settings
@@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
+        return
+
+    flags_by_group = getattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP", False)
+    if not flags_by_group:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP is not defined."
+        )
+
+    group_mapping = getattr(settings, "SOCIAL_AUTH_AZUREAD_GROUP_MAP", False)
+    if not group_mapping:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_GROUP_MAP is not defined."
+        )
+
+    url = 'https://graph.microsoft.com/v1.0/me'
+
+    access_token = response.get('access_token')
+    headers = {
+        "Content-Type": "application/x-www-form-urlencoded",
+        "Accept": "application/json",
+        'Authorization': f'Bearer {access_token}',
+    }
+
+    response = requests.get(
+        url,
+        headers=headers,
+    )
+    uid = response.json().get('id')
+    url = f"https://graph.microsoft.com/v1.0/users/{uid}/memberOf"
+    response = requests.get(
+        url,
+        headers=headers,
+    )
+
+    is_superuser = False
+    is_staff = False
+    values = response.json().get('value', [])
+
+    for value in values:
+        # AD response contains both directories and groups - we only want groups
+        if value.get('@odata.type') == '#microsoft.graph.group':
+            group_id = value.get('id', None)
+
+            if group_id in flags_by_group['is_superuser']:
+                is_superuser = True
+
+            if group_id in flags_by_group['is_staff']:
+                is_staff = True
+
+            if group_id in group_mapping:
+                group = Group.objects.get(name=group_mapping[group_id])
+                if group:
+                    group.user_set.add(user)
+                else:
+                    logger.info(f"Azure group mapping - group: {group_mapping[group_id]} not found.")
+
+    user.is_superuser = is_superuser
+    user.is_staff = is_staff
+    user.save()


Code review comments:
diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
path: netbox/netbox/authentication.py
text: @user1:
Is this used anywhere?

diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
+        return
+
+    flags_by_group = getattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP", False)
+    if not flags_by_group:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP is not defined."
+        )
+
+    group_mapping = getattr(settings, "SOCIAL_AUTH_AZUREAD_GROUP_MAP", False)
+    if not group_mapping:
path: netbox/netbox/authentication.py
text: @user1:
```suggestion
    if not hasattr(settings, "SOCIAL_AUTH_AZUREAD_GROUP_MAP"):
```

diff_hunk: @@ -61,6 +61,43 @@ Restart the NetBox services so that the new configuration takes effect. This is
 sudo systemctl restart netbox
 ```
 
+## Group Assignment
+
+If you want NetBox to assign groups based on Azure AD groups, then some additonal configuration is needed. Enter the following configuration parameters in `configuration.py`, substituting your own values:
+
+```python
+SOCIAL_AUTH_AZUREAD_OAUTH2_RESOURCE = 'https://graph.microsoft.com/'
+SOCIAL_AUTH_PIPELINE = (
+    'social_core.pipeline.social_auth.social_details',
+    'social_core.pipeline.social_auth.social_uid',
+    'social_core.pipeline.social_auth.social_user',
+    'social_core.pipeline.user.get_username',
+    'social_core.pipeline.social_auth.associate_by_email',
+    'social_core.pipeline.user.create_user',
+    'social_core.pipeline.social_auth.associate_user',
+    'netbox.authentication.user_default_groups_handler',
+    'social_core.pipeline.social_auth.load_extra_data',
+    'social_core.pipeline.user.user_details',
+    'netbox.authentication.azuread_map_groups',
+)
+SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS = True
+
+# Define special user types using groups. Exercise great caution when assigning superuser status.
+SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP = {
path: docs/administration/authentication/microsoft-azure-ad.md
text: @user1:
I recall we talked about the parameter naming strategy last week, but I'm having second thoughts. I worry that introducing our own paramters prefixed with `SOCIAL_AUTH_AZUREAD` may cause confusion among users expecting to see it the `python-social-auth` docs. I don't have a better suggestion though; open to other opinions.

@user2:
Why not prefix with `NETBOX_`?

@author:
changed to REMOTE_AUTH_BACKEND_ and auto-imported everything starting with this as we will probably be adding more so can group them all together.

diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
+        return
+
+    flags_by_group = getattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP", False)
+    if not flags_by_group:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP is not defined."
+        )
+
+    group_mapping = getattr(settings, "SOCIAL_AUTH_AZUREAD_GROUP_MAP", False)
+    if not group_mapping:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_GROUP_MAP is not defined."
+        )
+
+    url = 'https://graph.microsoft.com/v1.0/me'
+
+    access_token = response.get('access_token')
+    headers = {
+        "Content-Type": "application/x-www-form-urlencoded",
+        "Accept": "application/json",
+        'Authorization': f'Bearer {access_token}',
+    }
+
+    response = requests.get(
+        url,
+        headers=headers,
+    )
+    uid = response.json().get('id')
+    url = f"https://graph.microsoft.com/v1.0/users/{uid}/memberOf"
+    response = requests.get(
+        url,
+        headers=headers,
+    )
+
+    is_superuser = False
+    is_staff = False
+    values = response.json().get('value', [])
+
+    for value in values:
+        # AD response contains both directories and groups - we only want groups
+        if value.get('@odata.type') == '#microsoft.graph.group':
+            group_id = value.get('id', None)
+
+            if group_id in flags_by_group['is_superuser']:
+                is_superuser = True
+
+            if group_id in flags_by_group['is_staff']:
+                is_staff = True
+
+            if group_id in group_mapping:
+                group = Group.objects.get(name=group_mapping[group_id])
+                if group:
+                    group.user_set.add(user)
+                else:
+                    logger.info(f"Azure group mapping - group: {group_mapping[group_id]} not found.")
path: netbox/netbox/authentication.py
text: @user1:
`logger` has not been defined.

diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
+        return
+
+    flags_by_group = getattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP", False)
+    if not flags_by_group:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP is not defined."
+        )
+
+    group_mapping = getattr(settings, "SOCIAL_AUTH_AZUREAD_GROUP_MAP", False)
+    if not group_mapping:
+        raise ImproperlyConfigured(
+            "Azure group mapping has been configured, but SOCIAL_AUTH_AZUREAD_GROUP_MAP is not defined."
+        )
+
+    url = 'https://graph.microsoft.com/v1.0/me'
+
+    access_token = response.get('access_token')
+    headers = {
+        "Content-Type": "application/x-www-form-urlencoded",
+        "Accept": "application/json",
+        'Authorization': f'Bearer {access_token}',
+    }
+
+    response = requests.get(
+        url,
+        headers=headers,
+    )
+    uid = response.json().get('id')
+    url = f"https://graph.microsoft.com/v1.0/users/{uid}/memberOf"
+    response = requests.get(
+        url,
+        headers=headers,
+    )
path: netbox/netbox/authentication.py
text: @user1:
There doesn't seem to be any error handling in place. At a minimum, we should catch any fatal errors and re-raise them with some context for the user. Some comments would also be helpful to explain what requests are being made and why.

Additionally, we should avoid declaring `url` multiple times: If the value isn't being reused, we can pass the string(s) directly to `requests.get()` instead.

@author:
For error checking I think the error from the API call would contain the most info so I've added extra logging but re-raise the error.

diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
path: netbox/netbox/authentication.py
text: @user1:
We probably don't need this setting. Adding/removing this function to/from the authentication pipeline should be enough to enable or disable the functionality.

diff_hunk: @@ -386,3 +387,74 @@ def user_default_groups_handler(backend, user, response, *args, **kwargs):
             user.groups.add(*group_list)
         else:
             logger.info(f"No valid group assignments for {user} - REMOTE_AUTH_DEFAULT_GROUPS may be incorrectly set?")
+
+
+class AuthFailed(Exception):
+    pass
+
+
+def azuread_map_groups(response, user, backend, *args, **kwargs):
+    '''
+    Map Azure AD group ID to Netbox group
+    Also set is_superuser or is_staff based on config map
+    '''
+    if not getattr(settings, "SOCIAL_AUTH_AZUREAD_MAP_GROUP_PERMS", False):
+        return
+
+    flags_by_group = getattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP", False)
+    if not flags_by_group:
path: netbox/netbox/authentication.py
text: @user1:
```suggestion
    if not hasattr(settings, "SOCIAL_AUTH_AZUREAD_USER_FLAGS_BY_GROUP"):
```