Repository: repobee/repobee

Issue:
Refactor try_register_plugin
It can be simplified such that it's both better tested, and smaller.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=h1) Report
> Merging [#455](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=desc) into [master](https://codecov.io/gh/repobee/repobee/commit/a65391b8ab3df39bec05ba16b68f99d05321475e&el=desc) will **decrease** coverage by `3.88%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/repobee/repobee/pull/455/graphs/tree.svg?width=650&height=150&src=pr&token=5vOKgkphZe)](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #455      +/-   ##
==========================================
- Coverage   92.69%   88.81%   -3.89%     
==========================================
  Files          40       40              
  Lines        2164     2145      -19     
  Branches      369      366       -3     
==========================================
- Hits         2006     1905     -101     
- Misses        125      206      +81     
- Partials       33       34       +1     
```


| [Impacted Files](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/\_repobee/plugin.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL3BsdWdpbi5weQ==) | `94.73% <100.00%> (+1.45%)` | :arrow_up: |
| [src/\_repobee/ext/gitlab.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2V4dC9naXRsYWIucHk=) | `63.86% <0.00%> (-28.56%)` | :arrow_down: |
| [src/\_repobee/git.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2dpdC5weQ==) | `96.62% <0.00%> (-3.38%)` | :arrow_down: |
| [src/\_repobee/command/repos.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2NvbW1hbmQvcmVwb3MucHk=) | `92.62% <0.00%> (-2.46%)` | :arrow_down: |
| [src/\_repobee/ext/query.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2V4dC9xdWVyeS5weQ==) | `48.48% <0.00%> (-1.52%)` | :arrow_down: |
| [src/\_repobee/cli/parsing.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2NsaS9wYXJzaW5nLnB5) | `92.26% <0.00%> (-1.20%)` | :arrow_down: |
| [src/\_repobee/ext/github.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL19yZXBvYmVlL2V4dC9naXRodWIucHk=) | `90.29% <0.00%> (-0.11%)` | :arrow_down: |
| [src/repobee\_plug/\_corehooks.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL3JlcG9iZWVfcGx1Zy9fY29yZWhvb2tzLnB5) | `100.00% <0.00%> (ø)` | |
| [src/repobee\_plug/\_exceptions.py](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree#diff-c3JjL3JlcG9iZWVfcGx1Zy9fZXhjZXB0aW9ucy5weQ==) | `100.00% <0.00%> (ø)` | |
| ... and [3 more](https://codecov.io/gh/repobee/repobee/pull/455/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=footer). Last update [a65391b...2a83506](https://codecov.io/gh/repobee/repobee/pull/455?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/docs/code.rst b/docs/code.rst
index e0a8edf..dbae3b5 100644
--- a/docs/code.rst
+++ b/docs/code.rst
@@ -59,39 +59,39 @@ util
     :members:
 
 
-Core plugins
-============
+Default plugins
+===============
+These plugins are loaded by default.
 
-defaults
---------
+configwizard
+------------
+
+.. automodule:: _repobee.ext.defaults.configwizard
+    :members:
 
-.. automodule:: _repobee.ext.defaults
+genreviews
+----------
+
+.. automodule:: _repobee.ext.defaults.genreviews
     :members:
 
+
 github
 ------
 
-.. automodule:: _repobee.ext.github
+.. automodule:: _repobee.ext.defaults.github
     :members:
 
+Additional plugins
+==================
+These plugins ship with RepoBee, but are not loaded by default.
+
 gitlab
 ------
 
 .. automodule:: _repobee.ext.gitlab
    :members:
 
-
-
-pairwise
---------
-
-.. automodule:: _repobee.ext.pairwise
-    :members:
-
-
-Extension plugins
-=================
-
 javac
 -----
 
@@ -104,11 +104,12 @@ pylint
 .. automodule:: _repobee.ext.pylint
     :members:
 
-config-wizard
--------------
 
-.. automodule:: _repobee.ext.configwizard
-   :members:
+pairwise
+--------
+
+.. automodule:: _repobee.ext.pairwise
+    :members:
 
 query
 -----
diff --git a/src/_repobee/cli/mainparser.py b/src/_repobee/cli/mainparser.py
index 2523bd1..a2e2668 100644
--- a/src/_repobee/cli/mainparser.py
+++ b/src/_repobee/cli/mainparser.py
@@ -122,7 +122,7 @@ def create_parser_for_docs() -> argparse.ArgumentParser:
     """
     daiquiri.setup(level=logging.FATAL)
     # load default plugins
-    plugin.initialize_plugins([_repobee.constants.DEFAULT_PLUGIN])
+    plugin.initialize_default_plugins()
     ext_commands = plug.manager.hook.create_extension_command()
     return create_parser(
         show_all_opts=True,
@@ -158,6 +158,7 @@ def create_parser(
             _versioned_plugin_name(p)
             for p in plug.manager.get_plugins()
             if isinstance(p, types.ModuleType)
+            and not plugin.is_default_plugin(p)
         ]
     )
 
diff --git a/src/_repobee/constants.py b/src/_repobee/constants.py
index 0b1a1a9..5a55a27 100644
--- a/src/_repobee/constants.py
+++ b/src/_repobee/constants.py
@@ -40,5 +40,3 @@ CONFIGURABLE_ARGS = set(ORDERED_CONFIGURABLE_ARGS)
 
 TOKEN_ENV = "REPOBEE_TOKEN"
 TOKEN_ENV_OLD = "REPOBEE_OAUTH"
-
-DEFAULT_PLUGIN = "defaults"
diff --git a/src/_repobee/ext/defaults/__init__.py b/src/_repobee/ext/defaults/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/_repobee/ext/configwizard.py b/src/_repobee/ext/defaults/configwizard.py
similarity index 100%
rename from src/_repobee/ext/configwizard.py
rename to src/_repobee/ext/defaults/configwizard.py
diff --git a/src/_repobee/ext/defaults.py b/src/_repobee/ext/defaults/genreviews.py
similarity index 66%
rename from src/_repobee/ext/defaults.py
rename to src/_repobee/ext/defaults/genreviews.py
index 4ba1c9b..797d4ff 100644
--- a/src/_repobee/ext/defaults.py
+++ b/src/_repobee/ext/defaults/genreviews.py
@@ -1,17 +1,7 @@
-"""The defaults plugin contains all default hook implementations.
+"""The default implementation of generate_review_allocations.
 
-The goal is to make core parts of repobee pluggable using hooks that only
-return the first result that is not None. The standard behavior will be
-provided by the default plugin (this one), which implements all of the required
-hooks.  The default plugin will always be run last, so any user-defined hooks
-will run before it and therefore effectively override the default hooks.
-
-Currently, only the peer review related generate_review_allocations hook has a
-default implementation.
-
-.. module:: defaults
-    :synopsis: Plugin that provides the default behavior for core repobee
-        functionality.
+.. module:: genreviews
+    :synopsis: The default implementation of generate_review_allocations
 
 .. moduleauthor:: Simon Larsén
 """
@@ -21,10 +11,6 @@ from typing import List
 
 import repobee_plug as plug
 
-# import other default plugins
-from _repobee.ext.github import DefaultAPIHooks  # noqa: F401
-from _repobee.ext.configwizard import create_extension_command  # noqa: F401
-
 
 @plug.repobee_hook
 def generate_review_allocations(
diff --git a/src/_repobee/ext/github.py b/src/_repobee/ext/defaults/github.py
similarity index 100%
rename from src/_repobee/ext/github.py
rename to src/_repobee/ext/defaults/github.py
diff --git a/src/_repobee/ext/gitlab.py b/src/_repobee/ext/gitlab.py
index 8bad8fd..a1059a2 100644
--- a/src/_repobee/ext/gitlab.py
+++ b/src/_repobee/ext/gitlab.py
@@ -25,7 +25,7 @@ import requests.exceptions
 import repobee_plug as plug
 
 from _repobee import exception
-from _repobee.ext.github import DEFAULT_REVIEW_ISSUE
+from _repobee.ext.defaults.github import DEFAULT_REVIEW_ISSUE
 
 LOGGER = daiquiri.getLogger(__file__)
 
diff --git a/src/_repobee/main.py b/src/_repobee/main.py
index fe9a164..8b994a6 100644
--- a/src/_repobee/main.py
+++ b/src/_repobee/main.py
@@ -17,7 +17,6 @@ import _repobee.cli.parsing
 import _repobee.cli.preparser
 from _repobee import plugin
 from _repobee import exception
-from _repobee import constants
 from _repobee import config
 from _repobee.cli.preparser import separate_args
 
@@ -46,19 +45,19 @@ def main(sys_args: List[str]):
         )
         config_file = parsed_preparser_args.config_file
 
-        if parsed_preparser_args.no_plugins:
-            LOGGER.info("Non-default plugins disabled")
-            plugin.initialize_plugins([constants.DEFAULT_PLUGIN])
-        else:
+        # IMPORTANT: the default plugins must be loaded before user-defined
+        # plugins to ensure that the user-defined plugins override the defaults
+        # in firstresult hooks
+        LOGGER.debug("Initializing default plugins")
+        plugin.initialize_default_plugins()
+
+        if not parsed_preparser_args.no_plugins:
+            LOGGER.debug("Initializing user plugins")
             plugin_names = (
                 parsed_preparser_args.plug
                 or config.get_plugin_names(config_file)
             ) or []
-            # IMPORTANT: the default plugin MUST be loaded last to ensure that
-            # any user-defined plugins override the firstresult hooks
-            plugin.initialize_plugins(
-                plugin_names + [constants.DEFAULT_PLUGIN]
-            )
+            plugin.initialize_plugins(plugin_names)
 
         config.execute_config_hooks(config_file)
         ext_commands = plug.manager.hook.create_extension_command()
diff --git a/src/_repobee/plugin.py b/src/_repobee/plugin.py
index ee2d5de..2fff203 100644
--- a/src/_repobee/plugin.py
+++ b/src/_repobee/plugin.py
@@ -11,8 +11,7 @@ import collections
 import contextlib
 import shutil
 import tempfile
-
-import types
+import pkgutil
 import pathlib
 import importlib
 from types import ModuleType
@@ -21,6 +20,7 @@ from typing import List, Optional, Iterable, Mapping, Union
 import daiquiri
 
 import _repobee
+import _repobee.ext.defaults
 from _repobee import exception
 
 import repobee_plug as plug
@@ -38,7 +38,9 @@ def _external_plugin_qualname(plugin_name):
     )
 
 
-def load_plugin_modules(plugin_names: Iterable[str]) -> List[ModuleType]:
+def load_plugin_modules(
+    plugin_names: Iterable[str], allow_qualified: bool = False
+) -> List[ModuleType]:
     """Load the given plugins. Plugins are loaded such that they are executed
     in the same order that they are specified in the plugin_names list.
 
@@ -54,9 +56,18 @@ def load_plugin_modules(plugin_names: Iterable[str]) -> List[ModuleType]:
         # import nr 2
         from repobee_javac import javac
 
-    Args:
-        plugin_names: A list of plugin names. Overrides the config file.
+    If ``allow_qualified`` is True, an additional import using the provided
+    plugin names as-is is also attempted.
+
+    .. code-block:: python
+
+        # import nr 3 (only if allow_qualified)
+        import javac
 
+    Args:
+        plugin_names: A list of plugin names.
+        allow_qualified: If True, attempts to import modules using the plugin
+            names as qualified paths.
     Returns:
         a list of loaded modules.
     """
@@ -64,9 +75,11 @@ def load_plugin_modules(plugin_names: Iterable[str]) -> List[ModuleType]:
     LOGGER.debug("Loading plugins: " + ", ".join(plugin_names))
 
     for name in plugin_names:
-        plug_mod = _try_load_module(
-            _plugin_qualname(name)
-        ) or _try_load_module(_external_plugin_qualname(name))
+        plug_mod = (
+            _try_load_module(_plugin_qualname(name))
+            or _try_load_module(_external_plugin_qualname(name))
+            or (allow_qualified and _try_load_module(name))
+        )
         if not plug_mod:
             msg = "failed to load plugin module " + name
             raise exception.PluginLoadError(msg)
@@ -176,6 +189,8 @@ def try_register_plugin(
     """
     expected_plugin_classes = set(plugin_classes or [])
     newly_registered = register_plugins([plugin_module])
+    for reg in newly_registered:
+        plug.manager.unregister(reg)
 
     registered_modules = [
         reg for reg in newly_registered if isinstance(reg, ModuleType)
@@ -184,39 +199,35 @@ def try_register_plugin(
         cl.__class__ for cl in newly_registered if cl not in registered_modules
     }
 
-    errors = []
-    if len(registered_modules) != 1 or registered_modules[0] != plugin_module:
-        errors.append(
-            f"Expected only module {plugin_module}, got {registered_modules}"
-        )
-    elif expected_plugin_classes != registered_classes:
-        errors.append(
+    assert len(registered_modules) == 1, "Module was not registered"
+    if expected_plugin_classes != registered_classes:
+        raise plug.PlugError(
             f"Expected plugin classes {expected_plugin_classes}, "
             f"got {registered_classes}"
         )
 
-    for reg in newly_registered:
-        plug.manager.unregister(reg)
-
-    if errors:
-        raise plug.PlugError(errors)
-
 
 def initialize_plugins(
-    plugin_names: List[str] = None,
+    plugin_names: List[str] = None, allow_qualified: bool = False,
 ) -> List[Union[ModuleType, type]]:
     """Load and register plugins.
 
     Args:
         plugin_names: An optional list of plugin names that overrides the
-        configuration file's plugins.
+            configuration file's plugins.
+        allow_qualified: Allows the plugin names to be qualified.
     Returns:
         A list of registered modules and classes.
+    Raises:
+        :py:class:`_repobee.exception.PluginLoadError`
     """
+    if not allow_qualified:
+        _check_no_qualified_names(plugin_names)
+
     registered_plugins = plug.manager.get_plugins()
     plug_modules = [
         p
-        for p in load_plugin_modules(plugin_names=plugin_names)
+        for p in load_plugin_modules(plugin_names, allow_qualified)
         if p not in registered_plugins
     ]
     registered = register_plugins(plug_modules)
@@ -224,7 +235,15 @@ def initialize_plugins(
     return registered
 
 
-def resolve_plugin_version(plugin_module: types.ModuleType,) -> Optional[str]:
+def _check_no_qualified_names(names: List[str]):
+    qualified_names = [name for name in names if "." in name]
+    if qualified_names:
+        raise exception.PluginLoadError(
+            f"Qualified names not allowed: {qualified_names}"
+        )
+
+
+def resolve_plugin_version(plugin_module: ModuleType) -> Optional[str]:
     """Return the version of the top-level package containing the plugin, or
     None if it is not defined.
     """
@@ -235,6 +254,53 @@ def resolve_plugin_version(plugin_module: types.ModuleType,) -> Optional[str]:
     )
 
 
+def is_default_plugin(module: ModuleType) -> Optional[str]:
+    """Check if the provided module is a default module.
+
+    Args:
+        module: A Python module.
+    Returns:
+        True iff the provided module is a default plugin.
+    """
+    return module.__package__ == _repobee.ext.defaults.__name__
+
+
+def initialize_default_plugins() -> None:
+    """Initialize the default plugin modules."""
+    default_plugin_qualnames = get_qualified_module_names(
+        _repobee.ext.defaults
+    )
+    initialize_plugins(default_plugin_qualnames, allow_qualified=True)
+
+
+def get_qualified_module_names(pkg: ModuleType) -> List[str]:
+    """Return a list of all python modules in the given package. Only considers
+    the modules directly in this package, and not in subpackages.
+
+    Args:
+        pkg: The package to resolve modules in.
+    Returns:
+        All modules in the given package.
+    """
+    return [f"{pkg.__name__}.{name}" for name in get_module_names(pkg)]
+
+
+def get_module_names(pkg: ModuleType) -> List[str]:
+    """Get the unqualified module names from the given package.
+
+    Args:
+        pkg: The package to resolve modules in.
+    Returns:
+        All modules in the given package.
+    """
+    return [
+        name
+        for file_finder, name, _ in pkgutil.iter_modules(pkg.__path__)
+        # only include modules (i.e. files), not subpackages
+        if (pathlib.Path(file_finder.path) / (name + ".py")).is_file()
+    ]
+
+
 def execute_clone_tasks(
     repo_names: List[str], api: plug.API, cwd: Optional[pathlib.Path] = None
 ) -> Mapping[str, List[plug.Result]]:


Test patch:
diff --git a/tests/unit_tests/repobee/conftest.py b/tests/unit_tests/repobee/conftest.py
index 52c8a11..8d004ed 100644
--- a/tests/unit_tests/repobee/conftest.py
+++ b/tests/unit_tests/repobee/conftest.py
@@ -199,3 +199,14 @@ def config_mock(empty_config_mock, students_file):
     )
     empty_config_mock.write(config_contents)
     yield empty_config_mock
+
+
+@pytest.fixture
+def load_default_plugins():
+    """Load the default plugins."""
+    default_plugin_names = _repobee.plugin.get_qualified_module_names(
+        _repobee.ext.defaults
+    )
+    _repobee.plugin.initialize_plugins(
+        default_plugin_names, allow_qualified=True
+    )
diff --git a/tests/unit_tests/repobee/plugin_tests/test_configwizard.py b/tests/unit_tests/repobee/plugin_tests/test_configwizard.py
index c748f9a..fe7f5ce 100644
--- a/tests/unit_tests/repobee/plugin_tests/test_configwizard.py
+++ b/tests/unit_tests/repobee/plugin_tests/test_configwizard.py
@@ -9,7 +9,7 @@ from unittest.mock import patch
 import pytest
 
 import _repobee.constants
-from _repobee.ext import configwizard
+from _repobee.ext.defaults import configwizard
 
 
 @pytest.fixture
diff --git a/tests/unit_tests/repobee/plugin_tests/test_defaults.py b/tests/unit_tests/repobee/plugin_tests/test_genreviews.py
similarity index 83%
rename from tests/unit_tests/repobee/plugin_tests/test_defaults.py
rename to tests/unit_tests/repobee/plugin_tests/test_genreviews.py
index b1eb76f..ef9560a 100644
--- a/tests/unit_tests/repobee/plugin_tests/test_defaults.py
+++ b/tests/unit_tests/repobee/plugin_tests/test_genreviews.py
@@ -1,10 +1,10 @@
-"""Tests for the defaults plugin."""
+"""Tests for the genreviews plugin."""
 import itertools
 import collections
 
 import pytest
 
-from _repobee.ext import defaults
+from _repobee.ext.defaults import genreviews
 
 import constants
 
@@ -16,7 +16,7 @@ class TestGenerateReviewAllocations:
     def test_raises_when_too_many_reviews(self, num_students, num_reviews):
         teams = list(constants.STUDENTS[:num_students])
         with pytest.raises(ValueError) as exc_info:
-            defaults.generate_review_allocations(teams, num_reviews)
+            genreviews.generate_review_allocations(teams, num_reviews)
 
         assert "num_reviews must be less than len(teams)" in str(
             exc_info.value
@@ -24,7 +24,7 @@ class TestGenerateReviewAllocations:
 
     def test_raises_when_too_few_reviews(self):
         with pytest.raises(ValueError) as exc_info:
-            defaults.generate_review_allocations(list(constants.STUDENTS), 0)
+            genreviews.generate_review_allocations(list(constants.STUDENTS), 0)
         assert "num_reviews must be greater than 0" in str(exc_info.value)
 
     @pytest.mark.parametrize(
@@ -37,7 +37,7 @@ class TestGenerateReviewAllocations:
         students = list(constants.STUDENTS[:num_students])
         assert len(students) == num_students, "pre-test assert"
 
-        allocations = defaults.generate_review_allocations(
+        allocations = genreviews.generate_review_allocations(
             students, num_reviews
         )
 
@@ -60,7 +60,9 @@ class TestGenerateReviewAllocations:
         teams = list(constants.STUDENTS[:num_students])
         expected_reviewed_teams = list(teams)
 
-        allocations = defaults.generate_review_allocations(teams, num_reviews)
+        allocations = genreviews.generate_review_allocations(
+            teams, num_reviews
+        )
 
         assert sorted(expected_reviewed_teams) == sorted(
             [alloc.reviewed_team for alloc in allocations]
diff --git a/tests/unit_tests/repobee/plugin_tests/test_github.py b/tests/unit_tests/repobee/plugin_tests/test_github.py
index 84ead19..a8a6b47 100644
--- a/tests/unit_tests/repobee/plugin_tests/test_github.py
+++ b/tests/unit_tests/repobee/plugin_tests/test_github.py
@@ -7,11 +7,9 @@ import github
 
 import repobee_plug as plug
 
-import _repobee
-import _repobee.ext
-import _repobee.ext.github
+import _repobee.ext.defaults.github as github_plugin
 from _repobee import exception
-from _repobee.ext.github import REQUIRED_TOKEN_SCOPES
+from _repobee.ext.defaults.github import REQUIRED_TOKEN_SCOPES
 
 import constants
 import functions
@@ -314,20 +312,20 @@ def issues(repos):
 
 @pytest.fixture(scope="function")
 def api(happy_github, organization, no_teams):
-    return _repobee.ext.github.GitHubAPI(BASE_URL, TOKEN, ORG_NAME, USER)
+    return github_plugin.GitHubAPI(BASE_URL, TOKEN, ORG_NAME, USER)
 
 
 class TestInit:
     def test_raises_on_empty_user_arg(self):
         with pytest.raises(TypeError) as exc_info:
-            _repobee.ext.github.GitHubAPI(BASE_URL, TOKEN, ORG_NAME, "")
+            github_plugin.GitHubAPI(BASE_URL, TOKEN, ORG_NAME, "")
 
         assert "argument 'user' must not be empty" in str(exc_info.value)
 
     @pytest.mark.parametrize("url", ["https://github.com", constants.HOST_URL])
     def test_raises_when_url_is_bad(self, url):
         with pytest.raises(plug.PlugError) as exc_info:
-            _repobee.ext.github.GitHubAPI(url, TOKEN, ORG_NAME, USER)
+            github_plugin.GitHubAPI(url, TOKEN, ORG_NAME, USER)
 
         assert (
             "invalid base url, should either be https://api.github.com or "
@@ -338,7 +336,7 @@ class TestInit:
         "url", ["https://api.github.com", constants.BASE_URL]
     )
     def test_accepts_valid_urls(self, url):
-        api = _repobee.ext.github.GitHubAPI(url, TOKEN, ORG_NAME, USER)
+        api = github_plugin.GitHubAPI(url, TOKEN, ORG_NAME, USER)
 
         assert isinstance(api, plug.API)
 
@@ -756,7 +754,7 @@ def team_to_repos(api, no_repos, organization):
 class TestAddReposToReviewTeams:
     def test_with_default_issue(self, team_to_repos, organization, api):
         num_teams = len(team_to_repos)
-        default_issue = _repobee.ext.github.DEFAULT_REVIEW_ISSUE
+        default_issue = github_plugin.DEFAULT_REVIEW_ISSUE
         assert num_teams, "pre-test assert"
         team_repo_tuples = [
             (team, *repos) for team, repos in team_to_repos.items()
@@ -829,7 +827,7 @@ class TestVerifySettings:
 
     def test_happy_path(self, happy_github, organization, api):
         """Tests that no exceptions are raised when all info is correct."""
-        _repobee.ext.github.GitHubAPI.verify_settings(
+        github_plugin.GitHubAPI.verify_settings(
             USER, ORG_NAME, BASE_URL, TOKEN
         )
 
@@ -837,7 +835,7 @@ class TestVerifySettings:
         self, happy_github, monkeypatch, api
     ):
         with pytest.raises(exception.BadCredentials) as exc_info:
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 USER, ORG_NAME, BASE_URL, ""
             )
 
@@ -845,7 +843,7 @@ class TestVerifySettings:
 
     def test_incorrect_info_raises_not_found_error(self, github_bad_info, api):
         with pytest.raises(exception.NotFoundError):
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 USER, ORG_NAME, BASE_URL, TOKEN
             )
 
@@ -853,14 +851,14 @@ class TestVerifySettings:
         type(happy_github).oauth_scopes = PropertyMock(return_value=["repo"])
 
         with pytest.raises(exception.BadCredentials) as exc_info:
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 USER, ORG_NAME, BASE_URL, TOKEN
             )
         assert "missing one or more access token scopes" in str(exc_info.value)
 
     def test_not_owner_raises(self, happy_github, organization, api):
         with pytest.raises(exception.BadCredentials) as exc_info:
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 NOT_OWNER, ORG_NAME, BASE_URL, TOKEN
             )
 
@@ -883,7 +881,7 @@ class TestVerifySettings:
         happy_github.get_user.side_effect = lambda _: User(login=None)
 
         with pytest.raises(exception.UnexpectedException) as exc_info:
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 USER, ORG_NAME, BASE_URL, TOKEN
             )
 
@@ -907,7 +905,7 @@ class TestVerifySettings:
         ]
 
         with pytest.raises(exception.UnexpectedException) as exc_info:
-            _repobee.ext.github.GitHubAPI.verify_settings(
+            github_plugin.GitHubAPI.verify_settings(
                 USER, ORG_NAME, BASE_URL, TOKEN
             )
 
diff --git a/tests/unit_tests/repobee/test_cli.py b/tests/unit_tests/repobee/test_cli.py
index 21a64a2..f8479f3 100644
--- a/tests/unit_tests/repobee/test_cli.py
+++ b/tests/unit_tests/repobee/test_cli.py
@@ -13,7 +13,7 @@ import _repobee
 import _repobee.cli.dispatch
 import _repobee.cli.parsing
 import _repobee.ext
-import _repobee.ext.github
+import _repobee.ext.defaults.github
 import _repobee.ext.query
 import _repobee.constants
 import _repobee.plugin
@@ -77,7 +77,7 @@ VALID_PARSED_ARGS = dict(
 
 @pytest.fixture(autouse=True)
 def api_instance_mock():
-    instance_mock = mock.MagicMock(spec=_repobee.ext.github.GitHubAPI)
+    instance_mock = mock.MagicMock(spec=_repobee.ext.defaults.github.GitHubAPI)
 
     def _get_repo_urls(repo_names, org_name=None, teams=None):
         return [generate_repo_url(name, org_name) for name in repo_names]
@@ -91,18 +91,19 @@ def api_instance_mock():
 
 @pytest.fixture(autouse=True)
 def api_class_mock(mocker, api_instance_mock):
-    class_mock = mocker.patch("_repobee.ext.github.GitHubAPI", autospec=True)
+    class_mock = mocker.patch(
+        "_repobee.ext.defaults.github.GitHubAPI", autospec=True
+    )
     class_mock.return_value = api_instance_mock
     return class_mock
 
 
 @pytest.fixture(autouse=True)
-def load_default_plugins(api_instance_mock):
-    """Load the default plugins after mocking the GitHubAPI."""
-    loaded = _repobee.plugin.load_plugin_modules(
-        [_repobee.constants.DEFAULT_PLUGIN]
-    )
-    _repobee.plugin.register_plugins(loaded)
+def load_default_plugins_auto(load_default_plugins):
+    """The only point of this fixture is to make load_default_plugins
+    autouse=True.
+    """
+    yield
 
 
 @pytest.fixture
@@ -709,7 +710,7 @@ class TestExtensionCommands:
         command.
         """
         yield mock.MagicMock(
-            spec=_repobee.ext.configwizard.create_extension_command
+            spec=_repobee.ext.defaults.configwizard.create_extension_command
         )
 
     @pytest.fixture
diff --git a/tests/unit_tests/repobee/test_command.py b/tests/unit_tests/repobee/test_command.py
index f5185e8..1bd7849 100644
--- a/tests/unit_tests/repobee/test_command.py
+++ b/tests/unit_tests/repobee/test_command.py
@@ -7,7 +7,7 @@ from unittest.mock import patch, MagicMock, call, PropertyMock
 import pytest
 
 import _repobee
-import _repobee.ext.github
+import _repobee.ext.defaults.github
 from _repobee import command
 from _repobee import git
 from _repobee import util
@@ -147,8 +147,10 @@ def api_mock(request, mocker):
     def url_from_repo_info(repo_info):
         return generate_repo_url(repo_info.name)
 
-    mock = MagicMock(spec=_repobee.ext.github.GitHubAPI)
-    api_class = mocker.patch("_repobee.ext.github.GitHubAPI", autospec=True)
+    mock = MagicMock(spec=_repobee.ext.defaults.github.GitHubAPI)
+    api_class = mocker.patch(
+        "_repobee.ext.defaults.github.GitHubAPI", autospec=True
+    )
     api_class.return_value = mock
 
     mock.get_repo_urls.side_effect = partial(
@@ -158,7 +160,7 @@ def api_mock(request, mocker):
         map(url_from_repo_info, repo_infos)
     )
     mock.get_issues = MagicMock(
-        spec="_repobee.ext.github.GitHubAPI.get_issues",
+        spec="_repobee.ext.defaults.github.GitHubAPI.get_issues",
         side_effect=_get_issues,
     )
     type(mock).token = PropertyMock(return_value=TOKEN)
@@ -701,11 +703,11 @@ def test_purge_review_teams(master_names, students, api_mock):
 # TODO add more test cases
 class TestAssignPeerReviewers:
     @pytest.fixture(autouse=True)
-    def load_default_plugins(self):
-        modules = plugin.load_plugin_modules(
-            [_repobee.constants.DEFAULT_PLUGIN]
-        )
-        plugin.register_plugins(modules)
+    def load_default_plugins_auto(self, load_default_plugins):
+        """The only point of this fixture is to make load_default_plugins
+        autouse=True.
+        """
+        yield
 
     @pytest.mark.parametrize(
         "num_students, num_reviews",
diff --git a/tests/unit_tests/repobee/test_main.py b/tests/unit_tests/repobee/test_main.py
index 4b818ce..5056633 100644
--- a/tests/unit_tests/repobee/test_main.py
+++ b/tests/unit_tests/repobee/test_main.py
@@ -1,18 +1,19 @@
 import argparse
 import tempfile
 import pathlib
-from unittest.mock import MagicMock, patch
+from unittest.mock import MagicMock, patch, call
 from collections import namedtuple
 
 import pytest
 
 import _repobee.cli.preparser
+import _repobee.ext.defaults
 from functions import raise_
 import _repobee.constants
 from _repobee.cli import mainparser
 from _repobee import main
 from _repobee import plugin
-from _repobee.ext import configwizard
+from _repobee.ext.defaults import configwizard
 
 import constants
 
@@ -41,6 +42,7 @@ CLONE_ARGS = "clone --mn week-2 -s slarse".split()
 module = namedtuple("module", ("name",))
 
 DEFAULT_EXT_COMMANDS = [configwizard.create_extension_command()]
+DEFAULT_PLUGIN_NAMES = plugin.get_qualified_module_names(_repobee.ext.defaults)
 
 
 @pytest.fixture
@@ -55,7 +57,7 @@ def api_instance_mock(mocker):
 
 @pytest.fixture
 def init_plugins_mock(mocker):
-    def init_plugins(plugs=None):
+    def init_plugins(plugs=None, allow_qualified=False):
         list(map(module, plugs or []))
 
     return mocker.patch(
@@ -89,7 +91,6 @@ def dispatch_command_mock(mocker):
 def test_happy_path(
     api_instance_mock, handle_args_mock, dispatch_command_mock, no_config_mock
 ):
-
     sys_args = ["just some made up arguments".split()]
 
     main.main(sys_args)
@@ -156,8 +157,12 @@ def test_plugins_args(
 
     main.main(sys_args)
 
-    init_plugins_mock.assert_called_once_with(
-        ["javac", "pylint", _repobee.constants.DEFAULT_PLUGIN]
+    init_plugins_mock.assert_has_calls(
+        [
+            call(["javac", "pylint"]),
+            call(DEFAULT_PLUGIN_NAMES, allow_qualified=True),
+        ],
+        any_order=True,
     )
     handle_args_mock.assert_called_once_with(
         CLONE_ARGS,
@@ -178,7 +183,7 @@ def test_no_plugins_arg(
     main.main(sys_args)
 
     init_plugins_mock.assert_called_once_with(
-        [_repobee.constants.DEFAULT_PLUGIN]
+        DEFAULT_PLUGIN_NAMES, allow_qualified=True
     )
     handle_args_mock.assert_called_once_with(
         CLONE_ARGS,
@@ -199,7 +204,7 @@ def test_no_plugins_with_configured_plugins(
     main.main(sys_args)
 
     init_plugins_mock.assert_called_once_with(
-        [_repobee.constants.DEFAULT_PLUGIN]
+        DEFAULT_PLUGIN_NAMES, allow_qualified=True
     )
     handle_args_mock.assert_called_once_with(
         CLONE_ARGS,
@@ -216,8 +221,12 @@ def test_configured_plugins_are_loaded(
 
     main.main(sys_args)
 
-    init_plugins_mock.assert_called_once_with(
-        [*constants.PLUGINS, _repobee.constants.DEFAULT_PLUGIN]
+    init_plugins_mock.assert_has_calls(
+        [
+            call(["javac", "pylint"]),
+            call(DEFAULT_PLUGIN_NAMES, allow_qualified=True),
+        ],
+        any_order=True,
     )
     handle_args_mock.assert_called_once_with(
         CLONE_ARGS,
@@ -234,8 +243,12 @@ def test_plugin_with_subparser_name(
 
     main.main(sys_args)
 
-    init_plugins_mock.assert_called_once_with(
-        ["javac", "clone", _repobee.constants.DEFAULT_PLUGIN]
+    init_plugins_mock.assert_has_calls(
+        [
+            call(["javac", "clone"]),
+            call(DEFAULT_PLUGIN_NAMES, allow_qualified=True),
+        ],
+        any_order=True,
     )
     handle_args_mock.assert_called_once_with(
         CLONE_ARGS,
@@ -263,17 +276,19 @@ def test_invalid_plug_options(dispatch_command_mock, init_plugins_mock):
 
     Note that the default plugins must be loaded for this test to work.
     """
-    # load default plugins
-    loaded = plugin.load_plugin_modules([_repobee.constants.DEFAULT_PLUGIN])
-    plugin.register_plugins(loaded)
+    plugin.initialize_plugins(
+        plugin.get_qualified_module_names(_repobee.ext.defaults),
+        allow_qualified=True,
+    )
 
     sys_args = ["repobee", "-p", "javac", "-f", *CLONE_ARGS]
 
     with pytest.raises(SystemExit):
         main.main(sys_args)
 
-    init_plugins_mock.assert_called_once_with(
-        ["javac", _repobee.constants.DEFAULT_PLUGIN]
+    init_plugins_mock.assert_has_calls(
+        [call(["javac"]), call(DEFAULT_PLUGIN_NAMES, allow_qualified=True)],
+        any_order=True,
     )
     assert not dispatch_command_mock.called
 
diff --git a/tests/unit_tests/repobee/test_plugin.py b/tests/unit_tests/repobee/test_plugin.py
index 2e207f0..3a2f2ba 100644
--- a/tests/unit_tests/repobee/test_plugin.py
+++ b/tests/unit_tests/repobee/test_plugin.py
@@ -16,6 +16,7 @@ import pytest
 import repobee_plug as plug
 
 import _repobee.constants
+import _repobee.ext.defaults
 from _repobee import plugin
 from _repobee import exception
 
@@ -37,23 +38,34 @@ def unregister_plugins():
 class TestLoadPluginModules:
     """Tests for load_plugin_modules."""
 
-    def test_load_all_bundled_plugins(self):
-        """Test load the bundled plugins, i.e. the ones listed in
-        constants.PLUGINS.
-        """
-        plugin_names = [*PLUGINS, _repobee.constants.DEFAULT_PLUGIN]
-        plugin_qualnames = list(map(plugin._plugin_qualname, plugin_names))
+    def test_load_default_plugins(self):
+        default_plugin_qualnames = plugin.get_qualified_module_names(
+            _repobee.ext.defaults
+        )
+
+        modules = plugin.load_plugin_modules(
+            default_plugin_qualnames, allow_qualified=True
+        )
 
-        modules = plugin.load_plugin_modules(plugin_names)
         module_names = [mod.__name__ for mod in modules]
+        assert module_names == default_plugin_qualnames
 
-        assert module_names == plugin_qualnames
+    def test_load_bundled_plugins(self):
+        """Test load the bundled plugins that are not default plugins."""
+        bundled_plugin_qualnames = plugin.get_qualified_module_names(
+            _repobee.ext
+        )
+        bundled_plugin_names = plugin.get_module_names(_repobee.ext)
+
+        modules = plugin.load_plugin_modules(bundled_plugin_names)
+
+        module_names = [mod.__name__ for mod in modules]
+        assert module_names == bundled_plugin_qualnames
 
     def test_load_no_plugins(self, no_config_mock):
         """Test calling load plugins when no plugins are specified results in
         no plugins being loaded."""
         modules = plugin.load_plugin_modules([])
-
         assert modules == []
 
     def test_raises_when_loading_invalid_module(self, empty_config_mock):
@@ -69,6 +81,19 @@ class TestLoadPluginModules:
             exc_info.value
         )
 
+    def test_raises_when_loading_default_plugins_without_allow_qualified(self):
+        """Default plugins can only be loaded by their qualified names, and it
+        should only be allowed if allow_qualify is True.
+        """
+        default_plugin_qualnames = plugin.get_qualified_module_names(
+            _repobee.ext.defaults
+        )
+
+        with pytest.raises(exception.PluginLoadError) as exc_info:
+            plugin.load_plugin_modules(default_plugin_qualnames)
+
+        assert "failed to load plugin module" in str(exc_info.value)
+
 
 class TestRegisterPlugins:
     """Tests for register_plugins."""
@@ -214,3 +239,10 @@ class TestInitializePlugins:
             plugin.initialize_plugins([mod_name])
 
         assert warning_mock.called
+
+    def test_raises_on_qualified_names_by_default(self):
+        qualname = "_repobee.ext.query"
+        with pytest.raises(exception.PluginLoadError) as exc_info:
+            plugin.initialize_plugins([qualname])
+
+        assert "Qualified names not allowed" in str(exc_info.value)