Repository: oppia/oppia

Issue:
Update the CODEOWNER check to ensure new files don’t fall under blanket code owners.
For newly added files/directories in the code base, currently the `codeowner_linter` checks pass even for those file whose ownership is defined only by the [blanket codeowners](https://github.com/oppia/oppia/blob/ab5d7eb8a6e3013805e7cab05aa05ccd39e54ab6/.github/CODEOWNERS#L29), and are not being overriden by a specific codeowner. This can possibly result in a large number of files being owned only by blanket codeowners, which was not their original role. It also leads to them being assigned to a large number of PRs, since we did not ensure specific codeowners exist for them.

We need to modify the [check](https://github.com/oppia/oppia/blob/ab5d7eb8a6e3013805e7cab05aa05ccd39e54ab6/scripts/linters/codeowner_linter.py#L181) to ensure that for each file there must exist an additional code owner (specific for that file or some set of files) apart from the blanket code owner.

Following files do not have any **non-blanket** codeowners assigned to them (at the time of writing/last editing this comment):
- [ ] ./core/domain/base_model_validators.py
- [ ] ./core/domain/improvements_validators.py
- [ ] ./core/domain/audit_validators.py
- [ ] ./core/domain/rule_registry_test.py
- [ ] ./core/domain/rights_domain.py
- [ ] ./core/domain/interaction_jobs_one_off.py
- [ ] ./core/domain/prod_validators_test.py
- [ ] ./core/domain/job_validators_test.py
- [ ] ./core/domain/caching_services_test.py
- [ ] ./core/domain/improvements_validators_test.py
- [ ] ./core/domain/statistics_validators.py
- [ ] ./core/domain/caching_services.py
- [ ] ./core/domain/taskqueue_services_test.py
- [ ] ./core/domain/auth_validators_test.py
- [ ] ./core/domain/auth_services.py
- [ ] ./core/domain/auth_domain.py
- [ ] ./core/domain/audit_validators_test.py
- [ ] ./core/domain/interaction_jobs_one_off_test.py
- [ ] ./core/domain/caching_domain.py
- [ ] ./core/domain/classroom_domain.py
- [ ] ./core/domain/classroom_services.py
- [ ] ./core/domain/taskqueue_services.py
- [ ] ./core/domain/auth_validators.py
- [ ] ./core/domain/prod_validators.py
- [ ] ./core/domain/statistics_validators_test.py
- [ ] ./core/domain/rules_registry.py
- [ ] ./core/domain/job_validators.py
- [ ] ./core/domain/auth_domain_test.py
- [ ] ./core/domain/rights_domain_test.py
- [ ] ./core/domain/classroom_services_test.py
- [ ] ./core/domain/base_model_validators_test.py
- [ ] ./core/domain/auth_services_test.py
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.ts
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.spec.ts
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.html
- [ ] ./core/templates/components/common-layout-directives/common-elements/attribution-guide.component.spec.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.spec.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.html
- [ ] ./core/templates/domain/promo_bar/promo-bar.model.ts
- [ ] ./core/templates/domain/promo_bar/promo-bar.model.spec.ts
- [ ] ./core/templates/services/external-save.service.ts
- [ ] ./core/templates/services/auth-interceptor.service.ts
- [ ] ./core/templates/services/external-rte-save.service.ts
- [ ] ./core/templates/services/search-backend-api.service.ts
- [ ] ./core/templates/services/schema-form-submitted.service.ts
- [ ] ./core/templates/services/auth.service.ts
- [ ] ./core/templates/services/attribution.service.ts
- [ ] ./core/templates/pages/community-dashboard-page/modal-templates/login-required-modal.directive.html
Update the CODEOWNER check to ensure new files don’t fall under blanket code owners.
For newly added files/directories in the code base, currently the `codeowner_linter` checks pass even for those file whose ownership is defined only by the [blanket codeowners](https://github.com/oppia/oppia/blob/ab5d7eb8a6e3013805e7cab05aa05ccd39e54ab6/.github/CODEOWNERS#L29), and are not being overriden by a specific codeowner. This can possibly result in a large number of files being owned only by blanket codeowners, which was not their original role. It also leads to them being assigned to a large number of PRs, since we did not ensure specific codeowners exist for them.

We need to modify the [check](https://github.com/oppia/oppia/blob/ab5d7eb8a6e3013805e7cab05aa05ccd39e54ab6/scripts/linters/codeowner_linter.py#L181) to ensure that for each file there must exist an additional code owner (specific for that file or some set of files) apart from the blanket code owner.

Following files do not have any **non-blanket** codeowners assigned to them (at the time of writing/last editing this comment):
- [ ] ./core/domain/base_model_validators.py
- [ ] ./core/domain/improvements_validators.py
- [ ] ./core/domain/audit_validators.py
- [ ] ./core/domain/rule_registry_test.py
- [ ] ./core/domain/rights_domain.py
- [ ] ./core/domain/interaction_jobs_one_off.py
- [ ] ./core/domain/prod_validators_test.py
- [ ] ./core/domain/job_validators_test.py
- [ ] ./core/domain/caching_services_test.py
- [ ] ./core/domain/improvements_validators_test.py
- [ ] ./core/domain/statistics_validators.py
- [ ] ./core/domain/caching_services.py
- [ ] ./core/domain/taskqueue_services_test.py
- [ ] ./core/domain/auth_validators_test.py
- [ ] ./core/domain/auth_services.py
- [ ] ./core/domain/auth_domain.py
- [ ] ./core/domain/audit_validators_test.py
- [ ] ./core/domain/interaction_jobs_one_off_test.py
- [ ] ./core/domain/caching_domain.py
- [ ] ./core/domain/classroom_domain.py
- [ ] ./core/domain/classroom_services.py
- [ ] ./core/domain/taskqueue_services.py
- [ ] ./core/domain/auth_validators.py
- [ ] ./core/domain/prod_validators.py
- [ ] ./core/domain/statistics_validators_test.py
- [ ] ./core/domain/rules_registry.py
- [ ] ./core/domain/job_validators.py
- [ ] ./core/domain/auth_domain_test.py
- [ ] ./core/domain/rights_domain_test.py
- [ ] ./core/domain/classroom_services_test.py
- [ ] ./core/domain/base_model_validators_test.py
- [ ] ./core/domain/auth_services_test.py
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.ts
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.spec.ts
- [ ] ./core/templates/components/keyboard-shortcut-help/keyboard-shortcut-help-modal.component.html
- [ ] ./core/templates/components/common-layout-directives/common-elements/attribution-guide.component.spec.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.spec.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.ts
- [ ] ./core/templates/components/statistics-directives/completion-graph.component.html
- [ ] ./core/templates/domain/promo_bar/promo-bar.model.ts
- [ ] ./core/templates/domain/promo_bar/promo-bar.model.spec.ts
- [ ] ./core/templates/services/external-save.service.ts
- [ ] ./core/templates/services/auth-interceptor.service.ts
- [ ] ./core/templates/services/external-rte-save.service.ts
- [ ] ./core/templates/services/search-backend-api.service.ts
- [ ] ./core/templates/services/schema-form-submitted.service.ts
- [ ] ./core/templates/services/auth.service.ts
- [ ] ./core/templates/services/attribution.service.ts
- [ ] ./core/templates/pages/community-dashboard-page/modal-templates/login-required-modal.directive.html


Discussion:

Resolving patch:
diff --git a/scripts/linters/codeowner_linter.py b/scripts/linters/codeowner_linter.py
--- a/scripts/linters/codeowner_linter.py
+++ b/scripts/linters/codeowner_linter.py
@@ -195,14 +195,21 @@ def check_codeowner_file(self):
         # Checks whether every pattern in the CODEOWNERS file matches at
         # least one dir/file.
         critical_file_section_found = False
+        inside_blanket_codeowners_section = False
         important_rules_in_critical_section = []
         file_patterns = []
-        dir_patterns = []
+        ignored_dir_patterns = []
         for line_num, line in enumerate(self.file_cache.readlines(
                 CODEOWNER_FILEPATH)):
             stripped_line = line.strip()
             if '# Critical files' in line:
                 critical_file_section_found = True
+            if '# Blanket codeowners' in line:
+                inside_blanket_codeowners_section = True
+            # An empty line after the Blanket codeowners section marks its end.
+            if inside_blanket_codeowners_section is True and not stripped_line:
+                inside_blanket_codeowners_section = False
+                continue
             if stripped_line and stripped_line[0] != '#':
                 if '@' not in line:
                     error_message = (
@@ -271,14 +278,20 @@ def check_codeowner_file(self):
                     # leading '/' to aid in the glob pattern matching in
                     # the next part of the check wherein the valid patterns
                     # are used to check if they cover the entire codebase.
-                    if os.path.isdir(line_in_concern):
-                        dir_patterns.append(line_in_concern)
-                    else:
-                        file_patterns.append(line_in_concern)
+                    # Also we do not populate the lists if we are currently in
+                    # the blanket codeowners section, because that would allow
+                    # even those files and directories to pass the check whose
+                    # ownership is defined by blanket codeowners only and is not
+                    # overridden by a specific codeowner.
+                    if not inside_blanket_codeowners_section:
+                        if os.path.isdir(line_in_concern):
+                            ignored_dir_patterns.append(line_in_concern)
+                        else:
+                            file_patterns.append(line_in_concern)
 
         # Checks that every file (except those under the dir represented by
-        # the dir_patterns) is covered under CODEOWNERS.
-        for file_paths in self._walk_with_gitignore('.', dir_patterns):
+        # the ignored_dir_patterns) is covered under CODEOWNERS.
+        for file_paths in self._walk_with_gitignore('.', ignored_dir_patterns):
             for file_path in file_paths:
                 match = False
                 for file_pattern in file_patterns:


Test patch:
diff --git a/core/domain/rule_registry_test.py b/core/domain/rules_registry_test.py
similarity index 100%
rename from core/domain/rule_registry_test.py
rename to core/domain/rules_registry_test.py
diff --git a/scripts/linters/codeowner_linter_test.py b/scripts/linters/codeowner_linter_test.py
--- a/scripts/linters/codeowner_linter_test.py
+++ b/scripts/linters/codeowner_linter_test.py
@@ -48,6 +48,8 @@
     LINTER_TESTS_DIR, 'invalid_wildcard_used_codeowner')
 INVALID_FILEPATH_MISSING_FROM_DIRECTORY = os.path.join(
     LINTER_TESTS_DIR, 'invalid_file_path_missing_codeowner')
+INVALID_FILEPATH_WITH_BLANKET_CODEOWNER_ONLY = os.path.join(
+    LINTER_TESTS_DIR, 'invalid_file_path_with_blanket_codeowner_only')
 
 
 CODEOWNER_IMPORTANT_PATHS = [
@@ -74,7 +76,21 @@ def mock_listdir(unused_arg):
                 '.github/CODEOWNERS',
                 '.github/stale.yml']
 
+        def mock_listdir_with_blanket_codeowner_only_file(unused_arg):
+            return [
+                'manifest.json',
+                'package.json',
+                'yarn.lock',
+                'scripts/install_third_party_libs.py',
+                '.github/CODEOWNERS',
+                '.github/stale.yml',
+                'core/domain/new_file.py']
+
         self.listdir_swap = self.swap(os, 'listdir', mock_listdir)
+        self.listdir_swap_file_with_blanket_codeowner_only = (
+            self.swap(
+                os, 'listdir', mock_listdir_with_blanket_codeowner_only_file)
+        )
 
     def test_missing_important_codeowner_path_from_list(self):
         # Remove '/.github/stale.yml' important path from the end of list
@@ -173,6 +189,24 @@ def test_check_codeowner_file_with_success_message(self):
         self.assertEqual('CODEOWNERS', lint_task_report.name)
         self.assertFalse(lint_task_report.failed)
 
+    def test_check_file_with_only_blanket_codeowner_defined_fails(self):
+        codeowner_swap = self.swap(
+            codeowner_linter, 'CODEOWNER_FILEPATH',
+            INVALID_FILEPATH_WITH_BLANKET_CODEOWNER_ONLY)
+
+        with self.listdir_swap_file_with_blanket_codeowner_only, codeowner_swap:
+            linter = codeowner_linter.CodeownerLintChecksManager(FILE_CACHE)
+            lint_task_report = linter.check_codeowner_file()
+
+        error_message = (
+            './core/domain/new_file.py is not listed in the .github/CODEOWNERS '
+            'file.')
+        self.assert_same_list_elements(
+            error_message,
+            lint_task_report.trimmed_messages)
+        self.assertEqual('CODEOWNERS', lint_task_report.name)
+        self.assertTrue(lint_task_report.failed)
+
     def test_check_codeowner_file_without_codeowner_name(self):
         codeowner_swap = self.swap(
             codeowner_linter, 'CODEOWNER_FILEPATH',
diff --git a/scripts/linters/test_files/invalid_file_path_with_blanket_codeowner_only b/scripts/linters/test_files/invalid_file_path_with_blanket_codeowner_only
new file mode 100644
--- /dev/null
+++ b/scripts/linters/test_files/invalid_file_path_with_blanket_codeowner_only
@@ -0,0 +1,33 @@
+# We will list a new file in /core/domain, so that it becomes a file that is covered by blanket codeowners only.
+
+# Blanket codeowners
+# This is for the case when new files are created in any directories that aren't
+# covered as a whole, since in these cases, codeowners are not recognized for
+# those files when reviewing, even if the PR does add it. Unless new
+# files/folders are created in the following directories, these codeowners would
+# be superseded by the relevant codeowners mentioned elsewhere in this file.
+# (Reference: https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners)
+/core/controllers/ @aks681
+/core/domain/ @DubeySandeep
+/core/templates/components/ @srijanreddy98
+/core/templates/domain/ @kevintab95
+/core/templates/pages/ @srijanreddy98
+/core/templates/services/ @nithusha21
+/extensions/ @vojtechjelinek
+/scripts/ @DubeySandeep
+
+# Critical files.
+#
+# IMPORTANT NOTE: Please ensure that these files are always located at the end
+# of the CODEOWNERS file, since later rows override earlier ones. For details,
+# see the example at the bottom of this page:
+#
+#     https://help.github.com/en/articles/about-code-owners
+#
+# On modifying this list make sure to keep the CODEOWNER_IMPORTANT_PATHS list
+# in scripts/linters/pre_commit_linter.py in sync with the modifications.
+/manifest.json @vojtechjelinek
+/package.json @vojtechjelinek
+/yarn.lock @vojtechjelinek
+/scripts/install_third_party_libs.py @vojtechjelinek
+/.github/CODEOWNERS @DubeySandeep