Repository: getpelican/pelican

Issue:
Plugins run twice in autoreload mode
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this, let’s make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill them first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I have read the [Filing Issues](https://docs.getpelican.com/en/latest/contribute.html#filing-issues) and subsequent “How to Get Help” sections of the documentation.
- [x] I have searched the [issues](https://github.com/getpelican/pelican/issues?q=is%3Aissue) (including closed ones) and believe that this is not a duplicate.

<!--
  Once the above boxes are checked, if you are able to fill in the following list
  with your information, it would be very helpful for maintainers.
-->

- **OS version and name**: Ubuntu 18.04.5 LTS (WSL)
- **Python version**: 3.8.6
- **Pelican version**: e4d9c41a

## To reproduce

Create these files:

```
.
├── content
│   └── 1.rst
├── pelicanconf.py
└── plugins
    └── test_plugin.py
```

```rst
.. content/1.rst
####
test
####

:date: 1970-01-01
:modified: 1970-01-01

TEST
```

```python
# pelicanconf.py
PLUGINS = [
    'test_plugin',
]
PLUGIN_PATHS = [
    'plugins',
]
```

```python
# plugins/test_plugin.py
import logging

from pelican import signals

logger = logging.getLogger(__name__)


def test_function(content):
    logger.info('test plugin loaded')
    test = content._content
    test += 'TEST'
    content._content = test
    logger.info(content._content)


def register():
    signals.content_object_init.connect(test_function)
```

Then run:

```console
$ pelican -rD --logs-dedup-min-level DEBUG content
[Not showing unrelated logs]
-> test plugin loaded
-> <p>TEST</p>
  | TEST
-> test plugin loaded
-> <p>TEST</p>
  | TESTTEST

$ pelican -D --logs-dedup-min-level DEBUG content
[Not showing unrelated logs]
-> test plugin loaded
-> <p>TEST</p>
  | TEST
```

The plugin runs twice with `-r`, but only once without `-r`.

## The casue of the issue

After a bisect, I found that commit ed1eca16 introduced this issue.

```console
$ git checkout ed1eca16^
$ pelican -rD --logs-dedup-min-level DEBUG content
[Not showing unrelated logs]
-> test plugin loaded
-> <p>TEST</p>
  | TEST

$ git checkout ed1eca16
$ pelican -rD --logs-dedup-min-level DEBUG content
[Not showing unrelated logs]
-> test plugin loaded
-> <p>TEST</p>
  | TEST
-> test plugin loaded
-> <p>TEST</p>
  | TESTTEST
```

After adding this line, I found that the plugin was registered twice:

```diff
diff --git a/pelican/contents.py b/pelican/contents.py
index 594cd3b5..ff991be9 100644
--- a/pelican/contents.py
+++ b/pelican/contents.py
@@ -139,6 +139,7 @@ class Content(object):
         if 'summary' in metadata:
             self._summary = metadata['summary']

+        logger.info(str(signals.content_object_init.receivers))
         signals.content_object_init.send(self)

     def __str__(self):
```

```console
$ pelican -rD --logs-dedup-min-level DEBUG content
[Not showing unrelated logs]
-> {140542259499216: <weakref at 0x7fd28a2b8b80; to 'function' at 0x7fd28b7410d0 (test_function)>, 140542259600400: <weakref at 0x7fd28b7525e0; to 'function' at 0x7fd28b759c10 (test_function)>}
-> test plugin loaded
-> <p>TEST</p>
  | TEST
-> test plugin loaded
-> <p>TEST</p>
  | TESTTEST
```

But why? The reason:

```python
>>> # Implementation of ed1eca16^
>>> import sys
>>> sys.path.insert(0, 'plugins')
>>> plugin = __import__('test_plugin', globals(), locals(), str('module'))
>>> plugin_ = __import__('test_plugin', globals(), locals(), str('module'))
>>> plugin is plugin_
True
>>>
>>> # Implementation of ed1eca16
>>> import importlib
>>> spec = importlib.machinery.PathFinder.find_spec('test_plugin', ['plugins'])
>>> plugin = importlib.util.module_from_spec(spec)
>>> plugin_ = importlib.util.module_from_spec(spec)
>>> plugin is plugin_
False
```

## How to fix

To fix it, simply avoid loading the same plugin twice:

```diff
diff --git a/pelican/plugins/_utils.py b/pelican/plugins/_utils.py
index 4e6ec3c5..699192d3 100644
--- a/pelican/plugins/_utils.py
+++ b/pelican/plugins/_utils.py
@@ -53,6 +53,11 @@ def load_legacy_plugin(plugin, plugin_paths):
     if spec is None:
         raise ImportError('Cannot import plugin `{}`'.format(plugin))
     else:
+        # Avoid loading the same plugin twice
+        try:
+            return sys.modules[spec.name]
+        except KeyError:
+            pass
         # create module object from spec
         mod = importlib.util.module_from_spec(spec)
         # place it into sys.modules cache
```

I will create a pull request after writing some tests for it.




Discussion:
Thanks. I would expect old module to be deleted/garbage collected since it is overridden in the `sys.modules`. Returning module if it is already present makes sense, but `try/except` is a bit unnecessary. A simple `if` would do:

```python
if spec.name in sys.modules:
    return sys.modules[spec.name]
```
> Thanks. I would expect old module to be deleted/garbage collected since it is overridden in the `sys.modules`. Returning module if it is already present makes sense, but `try/except` is a bit unnecessary. A simple `if` would do:
> 
> ```python
> if spec.name in sys.modules:
>     return sys.modules[spec.name]
> ```

I used `try/except` to make it consistent with: https://github.com/getpelican/pelican/blob/e4d9c41a77b45cb7ff6b8d1732623990adf65931/pelican/plugins/_utils.py#L65-L69

I will use `if` instead in my commit.

Resolving patch:
diff --git a/pelican/plugins/_utils.py b/pelican/plugins/_utils.py
--- a/pelican/plugins/_utils.py
+++ b/pelican/plugins/_utils.py
@@ -53,6 +53,9 @@ def load_legacy_plugin(plugin, plugin_paths):
     if spec is None:
         raise ImportError('Cannot import plugin `{}`'.format(plugin))
     else:
+        # Avoid loading the same plugin twice
+        if spec.name in sys.modules:
+            return sys.modules[spec.name]
         # create module object from spec
         mod = importlib.util.module_from_spec(spec)
         # place it into sys.modules cache


Test patch:
diff --git a/pelican/tests/test_plugins.py b/pelican/tests/test_plugins.py
--- a/pelican/tests/test_plugins.py
+++ b/pelican/tests/test_plugins.py
@@ -131,6 +131,17 @@ def get_plugin_names(plugins):
                  'normal subpackage plugin'},
                 get_plugin_names(plugins))
 
+            # ensure normal plugins are loaded only once
+            SETTINGS = {
+                'PLUGINS': ['normal_plugin'],
+                'PLUGIN_PATHS': [self._NORMAL_PLUGIN_FOLDER],
+            }
+            plugins = load_plugins(SETTINGS)
+            for plugin in load_plugins(SETTINGS):
+                # The second load_plugins() should return the same plugin
+                # objects as the first one
+                self.assertIn(plugin, plugins)
+
             # namespace plugin short
             SETTINGS = {
                 'PLUGINS': ['ns_plugin']