Repository: pyinstaller/pyinstaller

Issue:
Bootloader does not forward most signals to child process
## The problem

Binaries built with Pyinstaller don't appear to gracefully handle some common
signals, such as SIGHUP and SIGUSR1.

Briefly: the bootloader [handles a few termination signals][sigs] (SIGINT, SIGTERM),
ignores others (SIGABRT, SIGBREAK), and leaves the rest as their default
behavior, which often is to terminate the process. This causes problems with
daemon processes that use other signals for external control. For example,
SIGHUP is commonly used to tell a daemon to reload its configuration from a
file on disk.

I'll be working on a PR later today to forward more signals. If there are any problems
with that approach, or if anyone has other suggestions, please let me know.

[sigs]: https://github.com/pyinstaller/pyinstaller/blob/b7638e423ce4ac48394f37961d73ed039e9a5ea6/bootloader/src/pyi_utils.c#L904

## How to reproduce

I've made a [test repo] with a `test.sh` script that:

1. Starts up a Python process that sleeps in a loop
   - This process has signal handlers set for the signals
     that the test script will send
1. Sends a signal to the process
1. Notes `pass` if the process exited gracefully; `fail` if it exited with error

The test repeats this for a couple different signals and prints the results.
It also runs the Python process with and without using Pyinstaller to show
the difference in behavior.

## Results

This [TravisCI build] shows the results. Curiously, the signals are handled as expected
on macOS; the crash manifests on Linux.

**Update**: further testing shows that the signals are handled correctly on macOS
only in the default `--onedir` operation, because the bootloader does not fork.
In `--onefile` mode (which is now part of the test), the bootloader does fork,
and the crash occurs.

[TravisCI build]: https://travis-ci.org/brettdh/pyinstaller-signal-handling

## Running the test manually

See the [test repo].

[test repo]: https://github.com/brettdh/pyinstaller-signal-handling/


Discussion:

Resolving patch:
diff --git a/PyInstaller/building/api.py b/PyInstaller/building/api.py
--- a/PyInstaller/building/api.py
+++ b/PyInstaller/building/api.py
@@ -300,6 +300,13 @@ def __init__(self, *args, **kwargs):
         kwargs
             Possible keywork arguments:
 
+            bootloader_ignore_signals
+                Non-Windows only. If True, the bootloader process will ignore
+                all ignorable signals. If False (default), it will forward
+                all signals to the child process. Useful in situations where
+                e.g. a supervisor process signals both the bootloader and
+                child (e.g. via a process group) to avoid signalling the
+                child twice.
             console
                 On Windows or OSX governs whether to use the console executable
                 or the windowed executable. Always True on Linux/Unix (always
@@ -332,6 +339,8 @@ def __init__(self, *args, **kwargs):
 
         # Available options for EXE in .spec files.
         self.exclude_binaries = kwargs.get('exclude_binaries', False)
+        self.bootloader_ignore_signals = kwargs.get(
+            'bootloader_ignore_signals', False)
         self.console = kwargs.get('console', True)
         self.debug = kwargs.get('debug', False)
         self.name = kwargs.get('name', None)
@@ -391,6 +400,10 @@ def __init__(self, *args, **kwargs):
         if self.runtime_tmpdir is not None:
             self.toc.append(("pyi-runtime-tmpdir " + self.runtime_tmpdir, "", "OPTION"))
 
+        if self.bootloader_ignore_signals:
+            # no value; presence means "true"
+            self.toc.append(("pyi-bootloader-ignore-signals", "", "OPTION"))
+
         if is_win:
             filename = os.path.join(CONF['workpath'], CONF['specnm'] + ".exe.manifest")
             self.manifest = winmanifest.create_manifest(filename, self.manifest,
diff --git a/PyInstaller/building/makespec.py b/PyInstaller/building/makespec.py
--- a/PyInstaller/building/makespec.py
+++ b/PyInstaller/building/makespec.py
@@ -293,11 +293,20 @@ def __add_options(parser):
                         "The ``_MEIxxxxxx``-folder will be created here. "
                         "Please use this option only if you know what you "
                         "are doing.")
+    g.add_argument("--bootloader-ignore-signals", action="store_true",
+                   default=False,
+                   help=("Tell the bootloader to ignore signals rather "
+                         "than forwarding them to the child process. "
+                         "Useful in situations where e.g. a supervisor "
+                         "process signals both the bootloader and child "
+                         "(e.g. via a process group) to avoid signalling "
+                         "the child twice."))
 
 
 def main(scripts, name=None, onefile=None,
          console=True, debug=None, strip=False, noupx=False,
          runtime_tmpdir=None, pathex=None, version_file=None, specpath=None,
+         bootloader_ignore_signals=False,
          datas=None, binaries=None, icon_file=None, manifest=None, resources=None, bundle_identifier=None,
          hiddenimports=None, hookspath=None, key=None, runtime_hooks=None,
          excludes=None, uac_admin=False, uac_uiaccess=False,
@@ -403,6 +412,7 @@ def main(scripts, name=None, onefile=None,
         'noarchive': 'noarchive' in debug,
         'options': [('v', None, 'OPTION')] if 'imports' in debug else [],
         'debug_bootloader': 'bootloader' in debug,
+        'bootloader_ignore_signals': bootloader_ignore_signals,
         'strip': strip,
         'upx': not noupx,
         'runtime_tmpdir': runtime_tmpdir,
diff --git a/PyInstaller/building/templates.py b/PyInstaller/building/templates.py
--- a/PyInstaller/building/templates.py
+++ b/PyInstaller/building/templates.py
@@ -37,6 +37,7 @@
           %(options)s,
           name='%(name)s',
           debug=%(debug_bootloader)s,
+          bootloader_ignore_signals=%(bootloader_ignore_signals)s,
           strip=%(strip)s,
           upx=%(upx)s,
           runtime_tmpdir=%(runtime_tmpdir)r,
@@ -66,6 +67,7 @@
           exclude_binaries=True,
           name='%(name)s',
           debug=%(debug_bootloader)s,
+          bootloader_ignore_signals=%(bootloader_ignore_signals)s,
           strip=%(strip)s,
           upx=%(upx)s,
           console=%(console)s %(exe_options)s)


Test patch:
diff --git a/tests/functional/conftest.py b/tests/functional/conftest.py
--- a/tests/functional/conftest.py
+++ b/tests/functional/conftest.py
@@ -60,6 +60,7 @@
 if is_py2:
     import subprocess32
     psutil.subprocess = subprocess32
+    subprocess.TimeoutExpired = psutil.TimeoutExpired
 
 # Globals
 # =======
diff --git a/tests/functional/test_signals.py b/tests/functional/test_signals.py
new file mode 100644
--- /dev/null
+++ b/tests/functional/test_signals.py
@@ -0,0 +1,106 @@
+# -*- coding: utf-8 -*-
+#-----------------------------------------------------------------------------
+# Copyright (c) 2005-2018, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+
+# Library imports
+# ---------------
+import signal
+
+# Third-party imports
+# -------------------
+import pytest
+from PyInstaller.utils.tests import skipif_win
+
+signals = sorted([
+    key for key in dir(signal)
+    if key.startswith('SIG') and not key.startswith('SIG_')
+])
+
+
+@skipif_win
+@pytest.mark.parametrize('signame', signals)
+@pytest.mark.parametrize('ignore', [True, False])
+def test_signal_handled(pyi_builder, signame, ignore):
+    # xfail tests for signals that the bootloader does NOT forward
+    if signame in ['SIGKILL', 'SIGSTOP']:
+        pytest.xfail('{} cannot be caught'.format(signame))
+    elif signame in ['SIGCHLD', 'SIGCLD']:
+        pytest.xfail(
+            'Messing with {} interferes with bootloader'.format(signame)
+        )
+
+    verb = 'ignored' if ignore else 'handled'
+    app_name = 'test_signal_{}_{}'.format(verb, signame)
+    pyi_args = ['--bootloader-ignore-signals'] if ignore else []
+
+    pyi_builder.test_source(
+        """
+        from __future__ import print_function
+        import psutil
+        import signal
+        import sys
+        import time
+        from signal import {signame}
+
+        def eprint(*args): print(*args, file=sys.stderr)
+
+        p = psutil.Process()
+        eprint('[test_signal_handled_{signame}] process tree:')
+        while p:
+            eprint('-', p.name(), '(%s)' % p.pid)
+            if p == p.parent():
+                break
+            p = p.parent()
+
+        signalled = False
+
+        def handle(signum, *args):
+            eprint('handled signal', signum)
+            global signalled
+            signalled = True
+
+        signal.signal({signame}, handle)
+
+        ignore = {ignore}
+
+        child = psutil.Process()
+        parent = child.parent()
+
+        if parent.name() == '{app_name}':
+            # We are the forked child of the bootloader process.
+            # Signal our parent process to mimic the behavior
+            # of an external program signalling the process running
+            # the executable that pyinstaller produced.
+            target = parent
+        elif ignore:
+            # can't pytest.skip() from inside this process
+            print('Bootloader did not fork; test is invalid')
+            sys.exit(0)
+        else:
+            target = child
+
+        eprint('signalling', target.name(), '(%s)' % target.pid)
+        target.send_signal({signame})
+
+        # sleep a bit to avoid exiting before the signal is delivered
+        time.sleep(1)
+
+        eprint('ignore:', ignore)
+        eprint('signalled:', signalled)
+        if ignore and signalled:
+            raise Exception('signal {signame} not ignored')
+        elif not ignore and not signalled:
+            raise Exception('signal handler not called for {signame}')
+
+        msg = 'ignored' if ignore else 'handled'
+        eprint('bootloader', msg, 'signal successfully.')
+        """.format(signame=signame, app_name=app_name, ignore=ignore),
+        app_name=app_name,
+        runtime=5,
+        pyi_args=pyi_args)
diff --git a/tests/scripts/test-docker.sh b/tests/scripts/test-docker.sh
--- a/tests/scripts/test-docker.sh
+++ b/tests/scripts/test-docker.sh
@@ -22,4 +22,4 @@ fi
 
 _image="pyinstaller-functional-tests"
 docker build -t ${_image} -f "$(dirname $0)/Dockerfile" .
-docker run -it -v "$(pwd):/src" --rm ${_image} ${_args}
+docker run -it --rm ${_image} ${_args}