Repository: pyinstaller/pyinstaller

Issue:
Options related to spec generation are silently ignored if an existing .spec file is given
## Description of the issue

When invoking PyInstaller, if a `.spec` file is passed, arguments like `--add-binary` are discarded silently, leaving no indication to the user that these arguments **have no effect**. More specifically, `--add-binary` is not consumed by [`building.build_main.main()`](https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/building/build_main.py#L802-L827) (invoked [here](https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/__main__.py#L118-L122)).

I discovered this behavior while working on build automation for an application that has some custom logic in the `.spec` file, but which will also want to add somewhat-arbitrary data to the `binaries` for the build. I ended up generating the final `.spec` from a template that fills in the binaries, but I would have been saved a bit of _"what the heck?!"_ time if PyInstaller issued a warning or error for me that what I was trying to do was nonsense.

### Context information (for bug reports)

* Output of `pyinstaller --version`: ```(paste here)```
* Version of Python: 4.9
* Platform: Ubuntu 20.04.3
* How you installed Python: pyenv
* Did you also try this on another platform? Does it work there? Yes. No.

### A minimal example program which shows the error

```
$ echo "print('hello world')" > repro.py
$ touch data1.txt data2.txt
$ python3 -m PyInstaller --add-binary=data1.txt:data repro.py
...
$ tree dist/
dist/                                                                             
└── repro                                                                         
    ├── base_library.zip                                                          
    ├── data                             
    │   └── data1.txt
    ├── libbz2.so.1.0
...
$ # data1 is part of the generated spec file, naively try to add data2
$ python3 -m PyInstaller --add-binary=data2.txt:data repro.spec
... (no warning or error about ignored argument) ...
$ tree dist/                                                                      
dist/                                                                             
└── repro           
    ├── base_library.zip    
    ├── data                             
    │   └── data1.txt
    ├── libbz2.so.1.0
```
Options related to spec generation are silently ignored if an existing .spec file is given
## Description of the issue

When invoking PyInstaller, if a `.spec` file is passed, arguments like `--add-binary` are discarded silently, leaving no indication to the user that these arguments **have no effect**. More specifically, `--add-binary` is not consumed by [`building.build_main.main()`](https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/building/build_main.py#L802-L827) (invoked [here](https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/__main__.py#L118-L122)).

I discovered this behavior while working on build automation for an application that has some custom logic in the `.spec` file, but which will also want to add somewhat-arbitrary data to the `binaries` for the build. I ended up generating the final `.spec` from a template that fills in the binaries, but I would have been saved a bit of _"what the heck?!"_ time if PyInstaller issued a warning or error for me that what I was trying to do was nonsense.

### Context information (for bug reports)

* Output of `pyinstaller --version`: ```(paste here)```
* Version of Python: 4.9
* Platform: Ubuntu 20.04.3
* How you installed Python: pyenv
* Did you also try this on another platform? Does it work there? Yes. No.

### A minimal example program which shows the error

```
$ echo "print('hello world')" > repro.py
$ touch data1.txt data2.txt
$ python3 -m PyInstaller --add-binary=data1.txt:data repro.py
...
$ tree dist/
dist/                                                                             
└── repro                                                                         
    ├── base_library.zip                                                          
    ├── data                             
    │   └── data1.txt
    ├── libbz2.so.1.0
...
$ # data1 is part of the generated spec file, naively try to add data2
$ python3 -m PyInstaller --add-binary=data2.txt:data repro.spec
... (no warning or error about ignored argument) ...
$ tree dist/                                                                      
dist/                                                                             
└── repro           
    ├── base_library.zip    
    ├── data                             
    │   └── data1.txt
    ├── libbz2.so.1.0
```


Discussion:
Here's an example patch that introduces the user-facing warning I found myself wishing I'd seen while debugging my build script. I don't know if this attempted usage actually constitutes an error, and this fault applies to other arguments (e.g. `--add-datas`), but it seems like a simple enough change that could maybe be generalized:

```diff
diff --git a/PyInstaller/__main__.py b/PyInstaller/__main__.py
index 6157308..8422eea 100644
--- a/PyInstaller/__main__.py
+++ b/PyInstaller/__main__.py
@@ -117,6 +117,8 @@ def run(pyi_args=None, pyi_config=None):
 
         # Skip creating .spec when .spec file is supplied.
         if args.filenames[0].endswith('.spec'):
+            if args.binaries:
+                logger.warning("Cannot add binaries to existing .spec (%s)", args.binaries)
             spec_file = args.filenames[0]
         else:
             spec_file = run_makespec(**vars(args))
```

With this diff applied, the output of PyInstaller looks like:

```
$ python3 -m PyInstaller --add-binary=data2.txt:data repro.spec
236 INFO: PyInstaller: 5.0.dev0+g9db19c9.mod
237 INFO: Python: 3.8.10
252 INFO: Platform: Linux-5.4.0-90-generic-x86_64-with-glibc2.29
252 WARNING: Cannot add binaries to existing .spec ([('data2.txt', 'data')])
...
```
Yeah it's not great. #6460 would have sorted it but I doubt we'll ever go through with that one. If we are going to start slapping checks in, I'd rather that they were errors - the signal to noise ratio of PyInstaller's warnings is already so low that I'm amazed when people even look at them.
Here's an example patch that introduces the user-facing warning I found myself wishing I'd seen while debugging my build script. I don't know if this attempted usage actually constitutes an error, and this fault applies to other arguments (e.g. `--add-datas`), but it seems like a simple enough change that could maybe be generalized:

```diff
diff --git a/PyInstaller/__main__.py b/PyInstaller/__main__.py
index 6157308..8422eea 100644
--- a/PyInstaller/__main__.py
+++ b/PyInstaller/__main__.py
@@ -117,6 +117,8 @@ def run(pyi_args=None, pyi_config=None):
 
         # Skip creating .spec when .spec file is supplied.
         if args.filenames[0].endswith('.spec'):
+            if args.binaries:
+                logger.warning("Cannot add binaries to existing .spec (%s)", args.binaries)
             spec_file = args.filenames[0]
         else:
             spec_file = run_makespec(**vars(args))
```

With this diff applied, the output of PyInstaller looks like:

```
$ python3 -m PyInstaller --add-binary=data2.txt:data repro.spec
236 INFO: PyInstaller: 5.0.dev0+g9db19c9.mod
237 INFO: Python: 3.8.10
252 INFO: Platform: Linux-5.4.0-90-generic-x86_64-with-glibc2.29
252 WARNING: Cannot add binaries to existing .spec ([('data2.txt', 'data')])
...
```
Yeah it's not great. #6460 would have sorted it but I doubt we'll ever go through with that one. If we are going to start slapping checks in, I'd rather that they were errors - the signal to noise ratio of PyInstaller's warnings is already so low that I'm amazed when people even look at them.

Resolving patch:
diff --git a/PyInstaller/__main__.py b/PyInstaller/__main__.py
--- a/PyInstaller/__main__.py
+++ b/PyInstaller/__main__.py
@@ -15,6 +15,7 @@
 import argparse
 import os
 import platform
+from collections import defaultdict
 
 from PyInstaller import __version__
 from PyInstaller import log as logging
@@ -68,7 +69,56 @@ def __add_options(parser):
     )
 
 
-def generate_parser() -> argparse.ArgumentParser:
+class _PyiArgumentParser(argparse.ArgumentParser):
+    def __init__(self, *args, **kwargs):
+        self._pyi_action_groups = defaultdict(list)
+        super().__init__(*args, **kwargs)
+
+    def _add_options(self, __add_options: callable, name: str = ""):
+        """
+        Mutate self with the given callable, storing any new actions added in a named group
+        """
+        n_actions_before = len(getattr(self, "_actions", []))
+        __add_options(self)  # preserves old behavior
+        new_actions = getattr(self, "_actions", [])[n_actions_before:]
+        self._pyi_action_groups[name].extend(new_actions)
+
+    def _option_name(self, action):
+        """
+        Get the option name(s) associated with an action
+
+        For options that define both short and long names, this function will
+        return the long names joined by "/"
+        """
+        longnames = [name for name in action.option_strings if name.startswith("--")]
+        if longnames:
+            name = "/".join(longnames)
+        else:
+            name = action.option_strings[0]
+        return name
+
+    def _forbid_options(self, args: argparse.Namespace, group: str, errmsg: str = ""):
+        """Forbid options from a named action group"""
+        options = defaultdict(str)
+        for action in self._pyi_action_groups[group]:
+            dest = action.dest
+            name = self._option_name(action)
+            if getattr(args, dest) is not self.get_default(dest):
+                if dest in options:
+                    options[dest] += "/"
+                options[dest] += name
+
+        # if any options from the forbidden group are not the default values,
+        # the user must have passed them in, so issue an error report
+        if options:
+            sep = "\n  "
+            bad = sep.join(options.values())
+            if errmsg:
+                errmsg = "\n" + errmsg
+            raise SystemExit(f"option(s) not allowed:{sep}{bad}{errmsg}")
+
+
+def generate_parser() -> _PyiArgumentParser:
     """
     Build an argparse parser for PyInstaller's main CLI.
     """
@@ -77,13 +127,14 @@ def generate_parser() -> argparse.ArgumentParser:
     import PyInstaller.building.makespec
     import PyInstaller.log
 
-    parser = argparse.ArgumentParser(formatter_class=_SmartFormatter)
+    parser = _PyiArgumentParser(formatter_class=_SmartFormatter)
     parser.prog = "pyinstaller"
-    __add_options(parser)
 
-    PyInstaller.building.makespec.__add_options(parser)
-    PyInstaller.building.build_main.__add_options(parser)
-    PyInstaller.log.__add_options(parser)
+    parser._add_options(__add_options)
+    parser._add_options(PyInstaller.building.makespec.__add_options, name="makespec")
+    parser._add_options(PyInstaller.building.build_main.__add_options, name="build_main")
+    parser._add_options(PyInstaller.log.__add_options, name="log")
+
     parser.add_argument(
         'filenames',
         metavar='scriptname',
@@ -117,6 +168,9 @@ def run(pyi_args=None, pyi_config=None):
 
         # Skip creating .spec when .spec file is supplied.
         if args.filenames[0].endswith('.spec'):
+            parser._forbid_options(
+                args, group="makespec", errmsg="makespec options not valid when a .spec file is given"
+            )
             spec_file = args.filenames[0]
         else:
             spec_file = run_makespec(**vars(args))
diff --git a/PyInstaller/building/makespec.py b/PyInstaller/building/makespec.py
--- a/PyInstaller/building/makespec.py
+++ b/PyInstaller/building/makespec.py
@@ -204,7 +204,7 @@ def __add_options(parser):
         "--onedir",
         dest="onefile",
         action="store_false",
-        default=False,
+        default=None,
         help="Create a one-folder bundle containing an executable (default)",
     )
     g.add_argument(
@@ -212,7 +212,7 @@ def __add_options(parser):
         "--onefile",
         dest="onefile",
         action="store_true",
-        default=False,
+        default=None,
         help="Create a one-file bundled executable.",
     )
     g.add_argument(
@@ -434,7 +434,7 @@ def __add_options(parser):
         "--nowindowed",
         dest="console",
         action="store_true",
-        default=True,
+        default=None,
         help="Open a console window for standard i/o (default). On Windows this option has no effect if the first "
         "script is a '.pyw' file.",
     )
@@ -444,6 +444,7 @@ def __add_options(parser):
         "--noconsole",
         dest="console",
         action="store_false",
+        default=None,
         help="Windows and Mac OS X: do not provide a console window for standard i/o. On Mac OS this also triggers "
         "building a Mac OS .app bundle. On Windows this option is automatically set if the first script is a '.pyw' "
         "file. This option is ignored on *NIX systems.",
@@ -642,6 +643,13 @@ def main(
     argv_emulation=False,
     **_kwargs
 ):
+    # Default values for onefile and console when not explicitly specified on command-line (indicated by None)
+    if onefile is None:
+        onefile = False
+
+    if console is None:
+        console = True
+
     # If appname is not specified - use the basename of the main script as name.
     if name is None:
         name = os.path.splitext(os.path.basename(scripts[0]))[0]
diff --git a/PyInstaller/utils/conftest.py b/PyInstaller/utils/conftest.py
--- a/PyInstaller/utils/conftest.py
+++ b/PyInstaller/utils/conftest.py
@@ -174,6 +174,7 @@ def __init__(self, tmpdir, request, bundle_mode):
         self._specdir = str(tmpdir)
         self._distdir = str(tmpdir / 'dist')
         self._builddir = str(tmpdir / 'build')
+        self._is_spec = False
 
     def test_spec(self, specfile, *args, **kwargs):
         """
@@ -182,6 +183,7 @@ def test_spec(self, specfile, *args, **kwargs):
         __tracebackhide__ = True
         specfile = os.path.join(_get_spec_dir(self._request), specfile)
         # 'test_script' should handle .spec properly as script.
+        self._is_spec = True
         return self.test_script(specfile, *args, **kwargs)
 
     def test_source(self, source, *args, **kwargs):
@@ -236,7 +238,8 @@ def marker(line):
             app_args = []
 
         if app_name:
-            pyi_args.extend(['--name', app_name])
+            if not self._is_spec:
+                pyi_args.extend(['--name', app_name])
         else:
             # Derive name from script name.
             app_name = os.path.splitext(os.path.basename(script))[0]
@@ -400,22 +403,29 @@ def _test_building(self, args):
 
         Return True if build succeded False otherwise.
         """
-        default_args = [
-            '--debug=bootloader',
-            '--noupx',
-            '--specpath', self._specdir,
-            '--distpath', self._distdir,
-            '--workpath', self._builddir,
-            '--path', _get_modules_dir(self._request),
-            '--log-level=INFO',
-        ]   # yapf: disable
-
-        # Choose bundle mode.
-        if self._mode == 'onedir':
-            default_args.append('--onedir')
-        elif self._mode == 'onefile':
-            default_args.append('--onefile')
-        # if self._mode is None then just the spec file was supplied.
+        if self._is_spec:
+            default_args = [
+                '--distpath', self._distdir,
+                '--workpath', self._builddir,
+                '--log-level=INFO',
+            ]  # yapf: disable
+        else:
+            default_args = [
+                '--debug=bootloader',
+                '--noupx',
+                '--specpath', self._specdir,
+                '--distpath', self._distdir,
+                '--workpath', self._builddir,
+                '--path', _get_modules_dir(self._request),
+                '--log-level=INFO',
+            ]  # yapf: disable
+
+            # Choose bundle mode.
+            if self._mode == 'onedir':
+                default_args.append('--onedir')
+            elif self._mode == 'onefile':
+                default_args.append('--onefile')
+            # if self._mode is None then just the spec file was supplied.
 
         pyi_args = [self.script] + default_args + args
         # TODO: fix return code in running PyInstaller programatically.


Test patch: