Repository: ImperialCollegeLondon/imperial_coldfront_plugin
Pull request #216: :recycle: Streamline allocation creation
Author: login: dalonsoa
Created: 2025-05-07T10:30:09Z | Merged: 2025-05-08T09:56:40Z

# Description

Updates the storage allocation process to also create a project - and a user, if needed -, so things are simpler for the end user. 

Fixes #213 

## Type of change

- [ ] Documentation (non-breaking change that adds or improves the documentation)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Optimization (non-breaking, back-end change that speeds up the code)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Breaking change (whatever its nature)

## Key checklist

- [ ] All tests pass (eg. `python -m pytest`)
- [ ] The documentation builds and looks OK (eg. `python -m sphinx -b html docs docs/build`)
- [x] Pre-commit hooks run successfully (eg. `pre-commit run --all-files`)

## Further checks

- [x] Code is commented, particularly in hard-to-understand areas
- [ ] Tests added or an issue has been opened to tackle that in the future. (Indicate issue here: # (issue))


Linked issue #213: Update add_rdf_storage_allocation to possibly also create a Project
At the moment we populate the `project` field of `RDFAllocationForm` with a list of existing projects within which to create the storage allocation. This basically assumes that the project owner has already logged in and created the project themselves. In order to provide a simpler setup process we're going to support creating a user for the group owner and a project before creating and adding the storage allocation.

To do:
- Replace `RDFAllocationForm.project` with a `username` CharField.
- In `add_rdf_storage_allocation` update the form data processing:
  - To try and lookup a user in the database using the username. If it a user doesn't exist, create one using data from `MicrosoftGraphClient.user_profile` (created user should have the username, email, first and last names set as a minimum).
  - Check if that user has an existing project. If not then create one.
  - Add the storage allocation to the retrieved/created project.

Diff:
diff --git a/imperial_coldfront_plugin/forms.py b/imperial_coldfront_plugin/forms.py
index 375b9f3e..ef3f79db 100644
--- a/imperial_coldfront_plugin/forms.py
+++ b/imperial_coldfront_plugin/forms.py
@@ -97,9 +97,8 @@ def get_initial_department_choices() -> Iterable[tuple[str, str]]:
 class RDFAllocationForm(forms.Form):
     """Form for creating a new RDF allocation."""
 
-    project = forms.ChoiceField(
-        choices=get_project_choices,
-        widget=forms.Select(attrs={"class": "js-example-basic-single"}),
+    username = forms.CharField(
+        help_text="Name of the user associated to this allocation.",
     )
     faculty = forms.ChoiceField(
         choices=get_faculty_choices,
diff --git a/imperial_coldfront_plugin/views.py b/imperial_coldfront_plugin/views.py
index 98c8973a..84d8c088 100644
--- a/imperial_coldfront_plugin/views.py
+++ b/imperial_coldfront_plugin/views.py
@@ -11,7 +11,13 @@
     AllocationStatusChoice,
     AllocationUserStatusChoice,
 )
-from coldfront.core.project.models import Project
+from coldfront.core.project.models import (
+    Project,
+    ProjectStatusChoice,
+    ProjectUser,
+    ProjectUserRoleChoice,
+    ProjectUserStatusChoice,
+)
 from coldfront.core.resource.models import Resource
 from coldfront.core.user.utils import UserSearch
 from django.conf import settings
@@ -524,7 +530,11 @@ def add_rdf_storage_allocation(request):
             rdf_resource = Resource.objects.get(name="RDF Active")
 
             allocation_active_status = AllocationStatusChoice.objects.get(name="Active")
-            project = get_object_or_404(Project, pk=form.cleaned_data["project"])
+
+            # We create a new user and an associated project, if they don't exist.
+            user = get_or_create_user(form.cleaned_data["username"])
+            project = get_or_create_project(user)
+
             rdf_allocation = Allocation.objects.create(
                 project=project,
                 status=allocation_active_status,
@@ -645,3 +655,52 @@ def task_stat_view(request, group: str, allocation_pk: int):
             "allocation_pk": allocation_pk,
         },
     )
+
+
+def get_or_create_user(username: str) -> User:
+    """Get user from the database or creates one using data from Graph.
+
+    Args:
+        username: The username of the user to be retrieved or created.
+
+    Return:
+        The user, already existing or newly created.
+    """
+    try:
+        user = User.objects.get(username=username)
+    except User.DoesNotExist:
+        client = get_graph_api_client()
+        user_data = client.user_profile(username)
+        user = User.objects.create(
+            username=user_data["username"],
+            first_name=user_data["first_name"],
+            last_name=user_data["last_name"],
+            email=user_data["email"],
+        )
+    return user
+
+
+def get_or_create_project(user: User) -> Project:
+    """Get project from the database or creates one.
+
+    Args:
+        user: The user object that will own a project.
+
+    Return:
+        The project, already existing or newly created.
+    """
+    project, project_created = Project.objects.get_or_create(
+        pi=user,
+        defaults=dict(
+            title=f"{user.get_full_name()}'s Research Group",
+            status=ProjectStatusChoice.objects.get_or_create(name="Active")[0],
+        ),
+    )
+    if project_created:
+        ProjectUser.objects.create(
+            user=user,
+            project=project,
+            role=ProjectUserRoleChoice.objects.get_or_create(name="Manager")[0],
+            status=ProjectUserStatusChoice.objects.get_or_create(name="Active")[0],
+        )
+    return project
diff --git a/tests/conftest.py b/tests/conftest.py
index 51a09ef2..d3d0041c 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -336,7 +336,7 @@ def pi_project(pi):
 
     return Project.objects.create(
         pi=pi,
-        title="project title",
+        title=f"{pi.get_full_name()}'s Research Group",
         status=project_active_status,
         field_of_science=field_of_science_other,
     )
diff --git a/tests/test_forms.py b/tests/test_forms.py
index e4cbc28f..f6a3f72e 100644
--- a/tests/test_forms.py
+++ b/tests/test_forms.py
@@ -48,7 +48,7 @@ def rdf_form_data(pi_project):
     faculty_id = "foe"
     department_id = DEPARTMENTS_IN_FACULTY[faculty_id][0]
     return dict(
-        project=pi_project.pk,
+        username=pi_project.pi.username,
         faculty=faculty_id,
         department=department_id,
         end_date=datetime.max.date(),
diff --git a/tests/test_views.py b/tests/test_views.py
index 1c19f217..a7037786 100644
--- a/tests/test_views.py
+++ b/tests/test_views.py
@@ -818,7 +818,7 @@ def test_post(
         response = superuser_client.post(
             self._get_url(),
             data=dict(
-                project=pi_project.pk,
+                username=pi_project.pi.username,
                 end_date=end_date,
                 size=size,
                 department=department,
@@ -970,3 +970,43 @@ def test_the_right_tasks_returned(self, superuser_client):
         assert response.status_code == HTTPStatus.OK
         assertTemplateUsed(response, "imperial_coldfront_plugin/task_list.html")
         assert len(response.context["completed"]) == 3
+
+
+def test_get_or_create_project(pi):
+    """Test get_or_create_project function."""
+    from coldfront.core.field_of_science.models import FieldOfScience
+    from coldfront.core.project.models import (
+        Project,
+        ProjectStatusChoice,
+        ProjectUser,
+        ProjectUserRoleChoice,
+        ProjectUserStatusChoice,
+    )
+
+    from imperial_coldfront_plugin.views import get_or_create_project
+
+    FieldOfScience.objects.create(pk=FieldOfScience.DEFAULT_PK)
+
+    assert not Project.objects.filter(pi=pi)
+    assert not ProjectStatusChoice.objects.filter(name="Active")
+    assert not ProjectUserRoleChoice.objects.filter(name="Manager")
+    assert not ProjectUserStatusChoice.objects.filter(name="Active")
+
+    project = get_or_create_project(pi)
+
+    assert project == Project.objects.get(pi=pi)
+    ProjectUser.objects.get(user=pi, project=project)
+    ProjectStatusChoice.objects.get(name="Active")
+    ProjectUserRoleChoice.objects.get(name="Manager")
+    ProjectUserStatusChoice.objects.get(name="Active")
+
+
+def test_get_or_create_user(
+    get_graph_api_client_mock, parsed_profile, django_user_model
+):
+    """Test get_or_create_user function."""
+    from imperial_coldfront_plugin.views import get_or_create_user
+
+    assert not django_user_model.objects.filter(username=parsed_profile["username"])
+    user = get_or_create_user(parsed_profile["username"])
+    assert user == django_user_model.objects.get(username=parsed_profile["username"])


Reviews:
user: login: cc-a
body: Thanks @dalonsoa. Looking good, just a couple of small tweaks suggested.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-07T10:52:15Z
user: login: cc-a
body: Thanks @dalonsoa. Looks great. I have one small tweak.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-08T09:46:29Z
user: login: dalonsoa
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T09:50:12Z

Review comments:
diff_hunk: @@ -524,7 +524,14 @@ def add_rdf_storage_allocation(request):
             rdf_resource = Resource.objects.get(name="RDF Project Storage Space")
 
             allocation_active_status = AllocationStatusChoice.objects.get(name="Active")
-            project = get_object_or_404(Project, pk=form.cleaned_data["project"])
+
+            user = get_or_create_user(form.cleaned_data["username"])
+            project = Project.objects.get_or_create(
+                pi=user,
+                title=user.username,
path: imperial_coldfront_plugin/views.py
user: login: cc-a
body: Maybe let's go with:
```suggestion
                title=f"{user.get_full_name()}'s Research Group",
```
created_at: 2025-05-07T10:43:00Z
author_association: COLLABORATOR
diff_hunk: @@ -524,7 +524,14 @@ def add_rdf_storage_allocation(request):
             rdf_resource = Resource.objects.get(name="RDF Project Storage Space")
 
             allocation_active_status = AllocationStatusChoice.objects.get(name="Active")
-            project = get_object_or_404(Project, pk=form.cleaned_data["project"])
+
+            user = get_or_create_user(form.cleaned_data["username"])
+            project = Project.objects.get_or_create(
+                pi=user,
+                title=user.username,
+                status=ProjectStatusChoice.objects.get(name="Active"),
+            )
+
path: imperial_coldfront_plugin/views.py
user: login: cc-a
body: Something I forgot to mention in the issue... You'll need to also create a `ProjectUserStatus` for `user`. In typical convoluted Coldfront fashion this should be something like:
```suggestion
            ProjectUserStatus.objects.create(
                user=user,
                project=project,
                role=ProjectUserRoleChoice.objects.get(name="Manager"),
                status=ProjectUserStatusChoice.objects.get(name="Active"),
           )
```
created_at: 2025-05-07T10:49:23Z
author_association: COLLABORATOR
diff_hunk: @@ -631,3 +641,50 @@ def task_stat_view(request, group: str):
     return render(
         request, "imperial_coldfront_plugin/task_list.html", context={"tasks": task_qs}
     )
+
+
+def get_or_create_user(username: str) -> User:
+    """Get user from the database or creates one using data from Graph.
+
+    Args:
+        username: The username of the user to be retrieved or created.
+
+    Return:
+        The user, already existing or newly created.
+    """
+    try:
+        user = User.objects.get(username=username)
+    except User.DoesNotExist:
+        client = get_graph_api_client()
+        user_data = client.user_profile(username)
+        user = User.objects.create(
+            username=user_data["username"],
+            first_name=user_data["first_name"],
+            last_name=user_data["last_name"],
+            email=user_data["email"],
+        )
+    return user
+
+
+def get_or_create_project(user: User) -> Project:
+    """Get project from the database or creates one.
+
+    Args:
+        user: The user object that will own a project.
+
+    Return:
+        The project, already existing or newly created.
+    """
+    project, project_created = Project.objects.get_or_create(
+        pi=user,
+        title=f"{user.get_full_name()}'s Research Group",
+        status=ProjectStatusChoice.objects.get_or_create(name="Active")[0],
+    )
path: imperial_coldfront_plugin/views.py
user: login: cc-a
body: I'd just make a tiny tweak here to avoid creating a second group in the event that a project title gets edited or we decide to change the naming pattern later.
```suggestion
    project, project_created = Project.objects.get_or_create(
        pi=user,
        defaults=dict(
            title=f"{user.get_full_name()}'s Research Group",
            status=ProjectStatusChoice.objects.get_or_create(name="Active")[0],
        )
    )
```
created_at: 2025-05-08T09:45:32Z
author_association: COLLABORATOR
diff_hunk: @@ -631,3 +641,50 @@ def task_stat_view(request, group: str):
     return render(
         request, "imperial_coldfront_plugin/task_list.html", context={"tasks": task_qs}
     )
+
+
+def get_or_create_user(username: str) -> User:
+    """Get user from the database or creates one using data from Graph.
+
+    Args:
+        username: The username of the user to be retrieved or created.
+
+    Return:
+        The user, already existing or newly created.
+    """
+    try:
+        user = User.objects.get(username=username)
+    except User.DoesNotExist:
+        client = get_graph_api_client()
+        user_data = client.user_profile(username)
+        user = User.objects.create(
+            username=user_data["username"],
+            first_name=user_data["first_name"],
+            last_name=user_data["last_name"],
+            email=user_data["email"],
+        )
+    return user
+
+
+def get_or_create_project(user: User) -> Project:
+    """Get project from the database or creates one.
+
+    Args:
+        user: The user object that will own a project.
+
+    Return:
+        The project, already existing or newly created.
+    """
+    project, project_created = Project.objects.get_or_create(
+        pi=user,
+        title=f"{user.get_full_name()}'s Research Group",
+        status=ProjectStatusChoice.objects.get_or_create(name="Active")[0],
+    )
path: imperial_coldfront_plugin/views.py
user: login: dalonsoa
body: Well spotted!
created_at: 2025-05-08T09:50:11Z
author_association: CONTRIBUTOR

PR comments:
user: login: dalonsoa
created_at: 2025-05-07T10:31:36Z
body: @cc-a could you have a look at this? If it makes sense, I'll work on the tests. My main unknown is the creation of a new project. I'm not sure if the input arguments are the ones that shall be use. In particular, `title` and `status` are mandatory, and I don't know if I've made reasonable choices there. 
author_association: CONTRIBUTOR
user: login: dalonsoa
created_at: 2025-05-07T12:57:34Z
body: It seems that the default field of science is not available, which makes it quite useless as default field of science. @cc-a  o you have any suggestion about how to proceed? Maybe getting or creating the default field when creating a project?
author_association: CONTRIBUTOR
user: login: cc-a
created_at: 2025-05-07T13:06:23Z
body: > It seems that the default field of science is not available, which makes it quite useless as default field of science. @cc-a o you have any suggestion about how to proceed? Maybe getting or creating the default field when creating a project?

Odd,... Why does it complain in the test teardown? They've done something really gross with a [hardcoded pk value](https://github.com/ubccr/coldfront/blob/c766ee41a36c4e9a8f38740489624b2133d21f7b/coldfront/core/field_of_science/models.py#L22) for some reason. Hopefully we can just handle it in the same way as we have some previously prerequisite models via creating it in the `rdf_allocation_dependencies` fixture. 
author_association: COLLABORATOR
user: login: dalonsoa
created_at: 2025-05-08T05:03:03Z
body: Quick and dirty solution in the test that is causing trouble. Not sure why that one causes trouble, though...
author_association: CONTRIBUTOR