Repository: carpentries/amy

Issue:
Do not allow concurrent same community roles
People should not have the same concurrent community roles.  For example if a person has an Instructor role from 2018-02-01 through 2020-06-01, any other role must end before 2018-02-01 and start after 2020-06-01.


Discussion:
@maneesha To be sure: if the person is an instructor in 2022 (2022-01-01 to 2022-12-31) they can't have *any* other community role in that time? Or is this specific to the same role (in this case "instructor")?
They can not have the **same** concurrent role.  For example, they should not have two concurrent Instructor roles, but the may have concurrent Instructor and Maintainer roles.
@maneesha thank you for this explanation.
Sorry I'm just seeing this now - this is actually not correct for Maintainers. A person can be a Maintainer on more than one lesson, example https://amy.carpentries.org/workshops/person/20352/ 
@ErinBecker thanks for noticing this.

> A person can be a Maintainer on more than one lesson

How would that be represented using current mechanism for Community Roles? @maneesha 
Thanks for bringing this up @ErinBecker 
Could this be resolved by allowing the same role only if they have associated key/value pairs? 

EDIT to note we would have to ensure that all curricula are set up in the admin panel.  Right now the difference in the two roles is noted only in the URL -- and if we have lessons properly set up, the URL should become redundant.
@maneesha @ErinBecker 

I can see two options here:

**Option 1**: extend the check for concurrent roles to cover "generic relation object".
For example:
![obraz](https://user-images.githubusercontent.com/72821/191227784-198f2d4a-f471-4f55-adcc-c2ffa777e202.png)
You can see a user with maintainer role for DC/Cloud lesson (table on the bottom). The form above is prepared to create another maintainer role but this time for DC/Python lesson. Currently this will fail because condition is only checking for "Maintainer" Role Name and the same person, and some date constraints.
If I extended the condition to also check for the same "generic relation object", we'd be able to create multiple Maintainer roles for the same user but with different lessons listed in "generic relation object".

**Option 2**: when editing an existing role there are custom keys available. In this example I set up "Maintainer" role configuration to have "lesson url" custom key available for all "Maintainer" roles.
![obraz](https://user-images.githubusercontent.com/72821/191228650-a8881612-32ba-4daf-a64b-42db9230e416.png)
We can extend the condition for concurrent roles to cover this field(s) too.

These "custom keys" are only available when editing an existing role.
@pbanaszkiewicz  I am trying to test out these options.  I've created test custom keys in the admin panel for Maintainer and Instructor role. Could we make these available when initially assigning a Person to the Role? 

We are still trying to figure out how to catalog all our curricula in AMY.  The current list of lessons is incomplete, and including ALL lessons would make other interfaces overwhelming.
Because of this, I would lean towards option 2.  It's not ideal, though, because it requires an extra step in the data entry process if we can not have this available when initially assigning a Person to the Role. 
@maneesha 
>  I've created test custom keys in the admin panel for Maintainer and Instructor role. Could we make these available when initially assigning a Person to the Role?

Not easily.

This form is dynamic, it changes with the choice of role type. It was way easier to omit custom keys in this form and only show them in "edit role" form which doesn't allow to change the role type.
As decided with @pbanaszkiewicz  and @ErinBecker  we will check for the URL as already defined  in the model (not as a new key/value pair).  
Additionally, the URL should be mandatory.

Resolving patch:
diff --git a/amy/communityroles/forms.py b/amy/communityroles/forms.py
--- a/amy/communityroles/forms.py
+++ b/amy/communityroles/forms.py
@@ -85,6 +85,7 @@ def clean(self) -> Union[dict[str, Any], None]:
         )
         start_date: Optional[date] = cleaned_data.get("start")
         end_date: Optional[date] = cleaned_data.get("end")
+        url: Optional[str] = cleaned_data.get("url")
 
         # Config is required, but field validation for 'config' should raise
         # validation error first.
@@ -116,10 +117,10 @@ def clean(self) -> Union[dict[str, Any], None]:
                 ValidationError(f"Membership is required with community role {config}")
             )
 
-        # Additional URL supported?
-        if not config.additional_url and cleaned_data.get("url"):
+        # Additional URL supported and required?
+        if config.additional_url and not url:
             errors["url"].append(
-                ValidationError(f"URL is not supported for community role {config}")
+                ValidationError(f"URL is required for community role {config}")
             )
 
         # Generic relation object must exist
@@ -148,7 +149,7 @@ def clean(self) -> Union[dict[str, Any], None]:
         # Person should not have any concurrent Community Roles of the same type in the
         # same time.
         if concurrent_roles := self.find_concurrent_roles(
-            config, person, start_date, end_date
+            config, person, start_date, end_date, url
         ):
             errors["person"].append(
                 ValidationError(
@@ -187,16 +188,23 @@ def find_concurrent_roles(
         person: Optional[Person] = None,
         start_date: Optional[date] = None,
         end_date: Optional[date] = None,
+        url: Optional[str] = None,
     ) -> Optional[QuerySet[CommunityRole]]:
         """Lookup concurrent Community Roles."""
         # These are required fields in the form, so they should be present.
         if config and person and start_date:
-            same_time = Q(end__gt=start_date) | Q(end__isnull=True)
+            initial_conditions = Q(end__gt=start_date) | Q(end__isnull=True)
+
+            # if `end_date` is present, introduce additional condition
             if end_date:
-                # if `end_date` is present, introduce additional condition
-                same_time &= Q(start__lt=end_date) | Q(start__isnull=True)
-            roles = CommunityRole.objects.filter(person=person, config=config).filter(
-                same_time
+                initial_conditions &= Q(start__lt=end_date) | Q(start__isnull=True)
+
+            # if configuration requires URL, add URL to conditions
+            if config.additional_url:
+                initial_conditions &= Q(url=url)
+
+            roles = CommunityRole.objects.filter(
+                initial_conditions, person=person, config=config
             )
             return roles
         return None
@@ -226,6 +234,7 @@ def find_concurrent_roles(
         person: Optional[Person] = None,
         start_date: Optional[date] = None,
         end_date: Optional[date] = None,
+        url: Optional[str] = None,
     ) -> Optional[QuerySet[CommunityRole]]:
         """When updating a CommunityRole, we shouldn't check for concurrent roles."""
         return None
diff --git a/amy/workshops/management/commands/migrate_to_single_instructor_badge.py b/amy/workshops/management/commands/migrate_to_single_instructor_badge.py
--- a/amy/workshops/management/commands/migrate_to_single_instructor_badge.py
+++ b/amy/workshops/management/commands/migrate_to_single_instructor_badge.py
@@ -5,25 +5,33 @@
 
 
 class Command(BaseCommand):
+    OLD_INSTRUCTOR_BADGE_NAMES = (
+        "swc-instructor",
+        "dc-instructor",
+        "lc-instructor",
+    )
+    NEW_INSTRUCTOR_BADGE_NAME = "instructor"
+
     def __init__(self) -> None:
         super().__init__()
-        self.instructor_badge = Badge.objects.get(name="instructor")
+        self.instructor_badge = Badge.objects.get(name=self.NEW_INSTRUCTOR_BADGE_NAME)
 
     def find_instructors(self) -> QuerySet[Person]:
         return (
             Person.objects.filter(
-                award__badge__name__in=[
-                    "swc-instructor",
-                    "dc-instructor",
-                    "lc-instructor",
-                ]
+                award__badge__name__in=self.OLD_INSTRUCTOR_BADGE_NAMES,
             )
-            .exclude(award__badge__name="instructor")
+            .exclude(award__badge__name=self.NEW_INSTRUCTOR_BADGE_NAME)
             .distinct()
         )
 
     def earliest_award(self, person: Person) -> Award:
-        return person.award_set.order_by("awarded").first()
+        return person.award_set.order_by("awarded").first()  # type: ignore
+
+    def remove_awards_for_old_instructor_badges(self) -> None:
+        Award.objects.filter(
+            badge__name__in=self.OLD_INSTRUCTOR_BADGE_NAMES,
+        ).exclude(badge__name=self.NEW_INSTRUCTOR_BADGE_NAME).delete()
 
     def create_instructor_award(self, person: Person) -> Award:
         earliest_award = self.earliest_award(person)
@@ -58,3 +66,7 @@ def handle(self, *args, **kwargs):
 
         Award.objects.bulk_create(new_awards)
         self.log(no_output, "Done.")
+
+        self.log(no_output, "Removing old instructor awards...")
+        self.remove_awards_for_old_instructor_badges()
+        self.log(no_output, "Done.")


Test patch:
diff --git a/amy/communityroles/tests/test_community_role_forms.py b/amy/communityroles/tests/test_community_role_forms.py
--- a/amy/communityroles/tests/test_community_role_forms.py
+++ b/amy/communityroles/tests/test_community_role_forms.py
@@ -259,7 +259,7 @@ def test_start_end_dates_valid(self):
                 self.assertEqual(form.cleaned_data.get("end"), p2)
                 self.assertNotIn("end", form.errors.keys())
 
-    def test_additional_url_supported(self):
+    def test_additional_url_required(self):
         # Arrange
         test_config = CommunityRoleConfig.objects.create(
             name="test",
@@ -267,7 +267,7 @@ def test_additional_url_supported(self):
             link_to_award=True,
             award_badge_limit=None,
             link_to_membership=True,
-            additional_url=False,
+            additional_url=True,
             generic_relation_content_type=None,
         )
         data = {
@@ -278,7 +278,7 @@ def test_additional_url_supported(self):
             "end": "2022-11-14",
             "inactivation": None,
             "membership": self.membership.pk,
-            "url": "https://example.org",  # should be empty
+            "url": "",  # shouldn't be empty
             "generic_relation_content_type": None,
             "generic_relation_pk": None,
         }
@@ -291,9 +291,40 @@ def test_additional_url_supported(self):
         self.assertEqual(form.errors.keys(), {"url"})
         self.assertEqual(
             form.errors["url"],
-            ["URL is not supported for community role Test"],
+            ["URL is required for community role Test"],
         )
 
+    def test_additional_url_not_required(self):
+        # Arrange
+        test_config = CommunityRoleConfig.objects.create(
+            name="test",
+            display_name="Test",
+            link_to_award=True,
+            award_badge_limit=None,
+            link_to_membership=True,
+            additional_url=False,
+            generic_relation_content_type=None,
+        )
+        data = {
+            "config": test_config.pk,
+            "person": self.hermione.pk,
+            "award": self.award.pk,
+            "start": "2021-11-14",
+            "end": "2022-11-14",
+            "inactivation": None,
+            "membership": self.membership.pk,
+            "url": "",  # it's okay
+            "generic_relation_content_type": None,
+            "generic_relation_pk": None,
+        }
+
+        # Act
+        form = CommunityRoleForm(data)
+
+        # Assert
+        self.assertTrue(form.is_valid())  # errors not expected
+        self.assertEqual(form.errors.keys(), set())
+
     def test_generic_relation_object_doesnt_exist(self):
         # Arrange
         ct = ContentType.objects.get_for_model(Lesson)
@@ -368,6 +399,62 @@ def test_end_date_required_when_inactivation_selected(self):
             form.errors["end"], ["Required when Reason for inactivation selected."]
         )
 
+    def test_find_concurrent_roles(self) -> None:
+        # Arrange
+        config1 = CommunityRoleConfig.objects.create(
+            name="test1",
+            display_name="Test1",
+            link_to_award=False,
+            link_to_membership=False,
+            additional_url=False,
+        )
+        config2 = CommunityRoleConfig.objects.create(
+            name="test2",
+            display_name="Test2",
+            link_to_award=False,
+            link_to_membership=False,
+            additional_url=True,
+        )
+        role1 = CommunityRole.objects.create(
+            person=self.hermione,
+            config=config1,
+            start=date(2022, 9, 29),
+            end=date(2023, 9, 29),
+        )
+        role2 = CommunityRole.objects.create(
+            person=self.hermione,
+            config=config2,
+            start=date(2022, 9, 29),
+            end=date(2023, 9, 29),
+            url="https://example.org/",
+        )
+        # Act
+        roles1 = CommunityRoleForm.find_concurrent_roles(
+            config1, self.hermione, date(2022, 1, 1), date(2022, 1, 31)
+        )
+        roles2 = CommunityRoleForm.find_concurrent_roles(
+            config1, self.hermione, date(2022, 10, 1), date(2022, 10, 31)
+        )
+        roles3 = CommunityRoleForm.find_concurrent_roles(
+            config2, self.hermione, date(2022, 10, 1), date(2022, 10, 31), url=None
+        )
+        roles4 = CommunityRoleForm.find_concurrent_roles(
+            config2, self.hermione, date(2022, 10, 1), date(2022, 10, 31), url=""
+        )
+        roles5 = CommunityRoleForm.find_concurrent_roles(
+            config2,
+            self.hermione,
+            date(2022, 10, 1),
+            date(2022, 10, 31),
+            url="https://example.org/",
+        )
+        # Assert
+        self.assertEqual(list(roles1), [])  # type: ignore
+        self.assertEqual(list(roles2), [role1])  # type: ignore
+        self.assertEqual(list(roles3), [])  # type: ignore
+        self.assertEqual(list(roles4), [])  # type: ignore
+        self.assertEqual(list(roles5), [role2])  # type: ignore
+
     def test_concurrent_community_roles_disallowed__validation_errors(self) -> None:
         """"""
         # Arrange
@@ -599,7 +686,7 @@ def test_clean_success(self) -> None:
             "end": "",
             "inactivation": None,
             "membership": "",
-            "url": "",
+            "url": "https://example.org/",
             "generic_relation_content_type": "",
             "generic_relation_pk": "",
         }
@@ -634,7 +721,7 @@ def test_custom_keys_field(self) -> None:
             "end": "",
             "inactivation": None,
             "membership": "",
-            "url": "",
+            "url": "https://example.org/",
             "generic_relation_content_type": "",
             "generic_relation_pk": "",
             "custom_keys": ["", "another value"],
diff --git a/amy/workshops/tests/test_commands.py b/amy/workshops/tests/test_commands.py
--- a/amy/workshops/tests/test_commands.py
+++ b/amy/workshops/tests/test_commands.py
@@ -98,7 +98,7 @@ def test_fetching_activity(self):
         # include people who don't want to be contacted (other option is tested
         # in `self.test_fetching_activity_may_contact_only`)
         results = self.cmd.fetch_activity(may_contact_only=False)
-        instructor_badges = Badge.objects.instructor_badges()
+        instructor_badges = Badge.objects.instructor_badges()  # type: ignore
 
         persons = [d["person"] for d in results]
         lessons = [list(d["lessons"]) for d in results]
@@ -155,7 +155,7 @@ def setUp(self):
         self.fake_cmd = FakeDatabaseCommand()
         self.seed = 12345
         Faker.seed(self.seed)
-        self.fake_cmd.stdout = StringIO()
+        self.fake_cmd.stdout = StringIO()  # type: ignore
 
         self.fake_cmd.fake_organizations()
 
@@ -481,6 +481,35 @@ def test_earliest_award(self) -> None:
         # Assert
         self.assertEqual(earliest_award, award)
 
+    def test_remove_awards_for_old_instructor_badges(self) -> None:
+        # Arrange
+        person1 = Person.objects.create(
+            username="test1", personal="Test1", family="User", email="test1@example.org"
+        )
+        Award.objects.create(person=person1, badge=self.swc_instructor)
+        person2 = Person.objects.create(
+            username="test2", personal="Test2", family="User", email="test2@example.org"
+        )
+        Award.objects.create(person=person2, badge=self.swc_instructor)
+        Award.objects.create(person=person2, badge=self.dc_instructor)
+        person3 = Person.objects.create(
+            username="test3", personal="Test3", family="User", email="test3@example.org"
+        )
+        Award.objects.create(person=person3, badge=self.swc_instructor)
+        Award.objects.create(person=person3, badge=self.dc_instructor)
+        Award.objects.create(person=person3, badge=self.lc_instructor)
+        instructor_award = Award.objects.create(
+            person=person3, badge=self.instructor_badge
+        )
+
+        # Act
+        self.command.remove_awards_for_old_instructor_badges()
+
+        # Assert
+        self.assertEqual(list(Award.objects.filter(person=person1)), [])
+        self.assertEqual(list(Award.objects.filter(person=person2)), [])
+        self.assertEqual(list(Award.objects.filter(person=person3)), [instructor_award])
+
     def test_create_instructor_award(self) -> None:
         # Arrange
         person = Person.objects.create(
@@ -563,7 +592,7 @@ def test_handle(self) -> None:
         self.command.handle(no_output=True)
 
         # Assert
-        for db, exp in zip(list(Award.objects.order_by("-pk")[:3]), expected):
+        for db, exp in zip(list(Award.objects.order_by("-pk")), expected):
             # Can't compare db == exp, since exp isn't from database and
             # doesn't contain a PK.
             self.assertEqual(db.person, exp.person)