Repository: encode/uvicorn

Issue:
Sending SIGTERM to parent process when running with --workers hangs indefinitely
### Checklist

<!-- Please make sure you check all these items before submitting your bug report. -->

- [x] The bug is reproducible against the latest release and/or `master`.
- [x] There are no similar issues or pull requests to fix it yet.

### Describe the bug

<!-- A clear and concise description of what the bug is. -->
When running uvicorn with multiple workers, sending a SIGTERM ie `kill -15 ppid` there's no graceful shutdown and the process hangs indefinitely on https://github.com/encode/uvicorn/blob/ff4af12d6902bc9d535fe2a948d1df3ffa02b0d3/uvicorn/supervisors/multiprocess.py#L57

### To reproduce

<!-- Provide a *minimal* example with steps to reproduce the bug locally.

1. Run `uvicorn app:app --workers 2 --log-level=debug`
2. Get the ppid in the logs `INFO:     Started parent process [38237]`
3. Send the SIGTERM `kill -15 38237`

NOTE: try to keep any external dependencies *at an absolute minimum* .
In other words, remove anything that doesn't make the bug go away.

-->

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
A graceful shutdown

### Actual behavior

<!-- A clear and concise description of what actually happens. -->

### Debugging material

<!-- Any tracebacks, screenshots, etc. that can help understanding the problem.

NOTE:
- Please list tracebacks in full (don't truncate them).
- If relevant, consider turning on DEBUG or TRACE logs for additional details (see the Logging section on https://www.uvicorn.org/settings/ specifically the `log-level` flag).
- Consider using `<details>` to make tracebacks/logs collapsible if they're very large (see https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d).
-->

### Environment

- OS / Python / Uvicorn version: just run `uvicorn --version`
`Running uvicorn 0.12.2 with CPython 3.8.6 on Linux`
- The exact command you're running uvicorn with, all flags you passed included. If you run it with gunicorn please do the same. If there is a reverse-proxy involved and you cannot reproduce without it please give the minimal config of it to reproduce.

### Additional context

<!-- Any additional information that can help understanding the problem.

Eg. linked issues, or a description of what you were trying to achieve. -->

originally discovered in https://github.com/encode/uvicorn/issues/364 and mistakenly taken for a docker issue, it's in fact a SIGTERM issue afaiu


Discussion:
Ugh deja vu from last year. Child processes no longer getting signals, and not exiting cleanly.

It's unfortunate your additions were reverted.

Can confirm this bug as re-introduced (and hanging linux container deployments such as docker).
yep we got a problem with SIGTERM, and not just in docker: it was discovered 1st as a docker issue because it's the default signal sent to exit a container , but the issue exists without containers, so the digging performed in #364 that ended up in a potential fix in #620 was not the right move.

I think #853 is the correct way to handle multi process exits cleanly, it is just too big to review, cant find a way not time to slice it properly
Commenting for notifications - I'd really like to see #853's work accepted 'n' merged.

(But am too stupid to help of course).
I've been hit by this problem recently and one thing I noticed is that in the `--reload` state specifically, the restart happens fine, but the shutdown does not.

Looking at the code I can see that `restart()` calls `self.process.terminate()` before `self.process.join()`:
https://github.com/encode/uvicorn/blob/87da6cf4082cd28306bdb78d7d42552d131cc179/uvicorn/supervisors/basereload.py#L69-L70

 but `shutdown()` does not:
https://github.com/encode/uvicorn/blob/87da6cf4082cd28306bdb78d7d42552d131cc179/uvicorn/supervisors/basereload.py#L77-L78

In the multiprocess module, `shutdown()` doesn't call `process.terminate()` either:

https://github.com/encode/uvicorn/blob/87da6cf4082cd28306bdb78d7d42552d131cc179/uvicorn/supervisors/multiprocess.py#L65-L67

As far as I can tell, without first calling `terminate()`, `join()` will hang because all `join()` does is wait for the process to terminate, but nothing has told it to. My multiprocessing knowledge is really rusty however, so I'm prepared for all of this to be wrong!


@sgsabbage Seems easy enough to test your theory right?  Spin up a docker environment and see if it hangs on termination, then make your change and try again.  If it fixes it and tests still pass, toss up a PR and we might just be away.

Resolving patch:
diff --git a/uvicorn/supervisors/basereload.py b/uvicorn/supervisors/basereload.py
--- a/uvicorn/supervisors/basereload.py
+++ b/uvicorn/supervisors/basereload.py
@@ -75,6 +75,7 @@ def restart(self) -> None:
         self.process.start()
 
     def shutdown(self) -> None:
+        self.process.terminate()
         self.process.join()
         message = "Stopping reloader process [{}]".format(str(self.pid))
         color_message = "Stopping reloader process [{}]".format(
diff --git a/uvicorn/supervisors/multiprocess.py b/uvicorn/supervisors/multiprocess.py
--- a/uvicorn/supervisors/multiprocess.py
+++ b/uvicorn/supervisors/multiprocess.py
@@ -64,6 +64,7 @@ def startup(self) -> None:
 
     def shutdown(self) -> None:
         for process in self.processes:
+            process.terminate()
             process.join()
 
         message = "Stopping parent process [{}]".format(str(self.pid))


Test patch: