Repository: celery/celery

Issue:
Please use `sys.exit()` or flush STDERR if failed on `platform.check_privileges`
https://github.com/celery/celery/blob/a3c377474ab1109a26de5169066a4fae0d30524b/celery/platforms.py#L794:24

The way it is now, the message could be lost because `os._exit()` does not flushes STDERR.

Why are we using `os._exit()` instead of the recommended `sys.exit()`? I can provide a PR if asked, but please clarify why the `os._exit()` before the patch is written.


Discussion:
cc: @ask @sbneto
Can you check the history of the file to see if there is a comment in the commit introducing this function?
The commit that introduced this code says `worker/beat: --uid + --gid now works even without --detach`. I could not find anything related to the use of `os._exit()` specifically. Also, this was the first and only version of the code written about 4 years ago. Not sure where else to look for clues on why this is as it is. 
This was by design. See [here](https://stackoverflow.com/a/9591397/920374) and if you notice in the [documentation on sys.exit](https://docs.python.org/3.5/library/sys.html#sys.exit), it raises an Exception which could potentially be caught somewhere in the code. On the other hand, [os._exit](https://docs.python.org/3.5/library/os.html#os._exit) causes an immediate shutdown, I believe using a system call.
Alright 👍. I could advocate for sys.exit(), but not strongly.

However, the `sys.stderr` is not being flushed and this is a real problem. When running on Docker I can only see the thing reboot on loop. No sign of why. Because the error message is not flushed.

On dev machine the thing works. On docker it cycles, because I forgot that it runs as `root` by default. Had to exec into the container to discover why by accident.

Could I provide a PR flushing `sys.stderr`?
https://docs.python.org/3.5/library/exceptions.html#SystemExit:
> **exception SystemExit**
> This exception is raised by the sys.exit() function. It inherits from BaseException instead of Exception so that it is not accidentally caught by code that catches Exception.
>...
@alanjds sorry to hear about the issues. I don't see any reason why not to flush output streams before exiting, so please feel free to open a PR.
I had the same issue (celery exiting with error code 1 without printing anything), and it was quite painful to figure out what was happening (basically had to isolate the bug by adding dozens of print() statements in celery to figure out where it was dying, because I had no access to a debugger in that environment).

Resolving patch:
diff --git a/celery/platforms.py b/celery/platforms.py
--- a/celery/platforms.py
+++ b/celery/platforms.py
@@ -791,6 +791,7 @@ def check_privileges(accept_content):
                         uid=uid, euid=euid, gid=gid, egid=egid,
                     ), file=sys.stderr)
                 finally:
+                    sys.stderr.flush()
                     os._exit(1)
         warnings.warn(RuntimeWarning(ROOT_DISCOURAGED.format(
             uid=uid, euid=euid, gid=gid, egid=egid,
diff --git a/celery/utils/threads.py b/celery/utils/threads.py
--- a/celery/utils/threads.py
+++ b/celery/utils/threads.py
@@ -74,6 +74,7 @@ def run(self):
                         self.on_crash('{0!r} crashed: {1!r}', self.name, exc)
                         self._set_stopped()
                     finally:
+                        sys.stderr.flush()
                         os._exit(1)  # exiting by normal means won't work
         finally:
             self._set_stopped()
diff --git a/celery/utils/timer2.py b/celery/utils/timer2.py
--- a/celery/utils/timer2.py
+++ b/celery/utils/timer2.py
@@ -91,6 +91,7 @@ def run(self):
                 pass
         except Exception as exc:
             logger.error('Thread Timer crashed: %r', exc, exc_info=True)
+            sys.stderr.flush()
             os._exit(1)
 
     def stop(self):


Test patch: