Repository: getpelican/pelican

Issue:
Pelican leaks child processes when killed with SIGINT
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this, let’s make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill them first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I have read the [Filing Issues](https://docs.getpelican.com/en/latest/contribute.html#filing-issues) and subsequent “How to Get Help” sections of the documentation.
- [x] I have searched the [issues](https://github.com/getpelican/pelican/issues?q=is%3Aissue) (including closed ones) and believe that this is not a duplicate.

<!--
  Once the above boxes are checked, if you are able to fill in the following list
  with your information, it would be very helpful for maintainers.
-->

- **OS version and name**: macOS Ventura 13.3
- **Python version**: 3.11
- **Pelican version**: 4.8.0
- **Link to theme**: Custom (check repo)
- **Links to plugins**: markdown codhilite only
- **Link to your site**: N/A
- **Link to your source**: [source](https://github.com/marcusdesai/marcusdesai.github.io/tree/a7ade3c803dab91adb44db5cada716d3b9e378b6)

## Issue
When running pelican in the following way:
```bash
pelican -lr <CONTENT> -o <OUTPUT> -s <CONF>
```
it is possible to make pelican leak child processes by killing the pelican process manually, with the following command:
```bash
kill -SIGINT <pelican_pid>
```

### Cause
Running pelican with the options `-lr` leads to the following section of the main function (`pelican/__init__.py::main`) to be run:
```python
if args.autoreload and args.listen:
    excqueue = multiprocessing.Queue()
    p1 = multiprocessing.Process(
        target=autoreload,
        args=(args, excqueue))
    p2 = multiprocessing.Process(
        target=listen,
        args=(settings.get('BIND'), settings.get('PORT'),
              settings.get("OUTPUT_PATH"), excqueue))
    p1.start()
    p2.start()
    exc = excqueue.get()
    p1.terminate()
    p2.terminate()
    if exc is not None:
        logger.critical(exc)
```
the leaked children are specifically `p1` and `p2` as initialised here. When shutting down pelican by hitting `Ctrl+C` on the keyboard these processes will be terminated as desired. But when sending `SIGINT` using `kill`, these two process do not get terminated, even though the main pelican process is ended.

### Fix
A simple fix is available, which covers more than just the SIGINT signal being sent.
```python
if args.autoreload and args.listen:
    excqueue = multiprocessing.Queue()
    p1 = multiprocessing.Process(
        target=autoreload,
        args=(args, excqueue))
    p2 = multiprocessing.Process(
        target=listen,
        args=(settings.get('BIND'), settings.get('PORT'),
              settings.get("OUTPUT_PATH"), excqueue))
    try:
        p1.start()
        p2.start()
        exc = excqueue.get()
        if exc is not None:
            logger.critical(exc)
    finally:
        p1.terminate()
        p2.terminate()
```
This ensures cleanup occurs.

I'm happy to submit a PR for this, but the issue is testing the fix. Unfortunately in the time I gave it I did not find a way to run & kill a pelican process in a test, checking for cleanup of child processes. This sort of testing is always a bit touchy 😅. If you're happy to use this fix without a test then let me know and I'll get it ready.
Pelican leaks child processes when killed with SIGINT
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this, let’s make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill them first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I have read the [Filing Issues](https://docs.getpelican.com/en/latest/contribute.html#filing-issues) and subsequent “How to Get Help” sections of the documentation.
- [x] I have searched the [issues](https://github.com/getpelican/pelican/issues?q=is%3Aissue) (including closed ones) and believe that this is not a duplicate.

<!--
  Once the above boxes are checked, if you are able to fill in the following list
  with your information, it would be very helpful for maintainers.
-->

- **OS version and name**: macOS Ventura 13.3
- **Python version**: 3.11
- **Pelican version**: 4.8.0
- **Link to theme**: Custom (check repo)
- **Links to plugins**: markdown codhilite only
- **Link to your site**: N/A
- **Link to your source**: [source](https://github.com/marcusdesai/marcusdesai.github.io/tree/a7ade3c803dab91adb44db5cada716d3b9e378b6)

## Issue
When running pelican in the following way:
```bash
pelican -lr <CONTENT> -o <OUTPUT> -s <CONF>
```
it is possible to make pelican leak child processes by killing the pelican process manually, with the following command:
```bash
kill -SIGINT <pelican_pid>
```

### Cause
Running pelican with the options `-lr` leads to the following section of the main function (`pelican/__init__.py::main`) to be run:
```python
if args.autoreload and args.listen:
    excqueue = multiprocessing.Queue()
    p1 = multiprocessing.Process(
        target=autoreload,
        args=(args, excqueue))
    p2 = multiprocessing.Process(
        target=listen,
        args=(settings.get('BIND'), settings.get('PORT'),
              settings.get("OUTPUT_PATH"), excqueue))
    p1.start()
    p2.start()
    exc = excqueue.get()
    p1.terminate()
    p2.terminate()
    if exc is not None:
        logger.critical(exc)
```
the leaked children are specifically `p1` and `p2` as initialised here. When shutting down pelican by hitting `Ctrl+C` on the keyboard these processes will be terminated as desired. But when sending `SIGINT` using `kill`, these two process do not get terminated, even though the main pelican process is ended.

### Fix
A simple fix is available, which covers more than just the SIGINT signal being sent.
```python
if args.autoreload and args.listen:
    excqueue = multiprocessing.Queue()
    p1 = multiprocessing.Process(
        target=autoreload,
        args=(args, excqueue))
    p2 = multiprocessing.Process(
        target=listen,
        args=(settings.get('BIND'), settings.get('PORT'),
              settings.get("OUTPUT_PATH"), excqueue))
    try:
        p1.start()
        p2.start()
        exc = excqueue.get()
        if exc is not None:
            logger.critical(exc)
    finally:
        p1.terminate()
        p2.terminate()
```
This ensures cleanup occurs.

I'm happy to submit a PR for this, but the issue is testing the fix. Unfortunately in the time I gave it I did not find a way to run & kill a pelican process in a test, checking for cleanup of child processes. This sort of testing is always a bit touchy 😅. If you're happy to use this fix without a test then let me know and I'll get it ready.


Discussion:
Your proposed solution looks reasonable to me. Like you said it's tricky to test things like this and off the top of my head, I can't think of a good way to test this. I'm OK with a PR without tests :).
Your proposed solution looks reasonable to me. Like you said it's tricky to test things like this and off the top of my head, I can't think of a good way to test this. I'm OK with a PR without tests :).

Resolving patch:
diff --git a/pelican/__init__.py b/pelican/__init__.py
--- a/pelican/__init__.py
+++ b/pelican/__init__.py
@@ -543,13 +543,15 @@ def main(argv=None):
                 target=listen,
                 args=(settings.get('BIND'), settings.get('PORT'),
                       settings.get("OUTPUT_PATH"), excqueue))
-            p1.start()
-            p2.start()
-            exc = excqueue.get()
-            p1.terminate()
-            p2.terminate()
-            if exc is not None:
-                logger.critical(exc)
+            try:
+                p1.start()
+                p2.start()
+                exc = excqueue.get()
+                if exc is not None:
+                    logger.critical(exc)
+            finally:
+                p1.terminate()
+                p2.terminate()
         elif args.autoreload:
             autoreload(args)
         elif args.listen:


Test patch: