Project: FOSSA (Provisioning / Security & Compliance)
File: setuptools.md

requirements.txt, alongside setup.py, is the most common -- yet imprecise -- approach to dependency management in python projects. `requirements.txt`: Find all files matching the glob: `req*.txt` `setup.py`: Find all files named `setup.py` For the CLI to identify transitive dependencies from a requirements.txt file, certain conditions must be met. First, the specified packages must be installed using a command like `python -m pip install -r requirements.txt`. The packages should be installed within a virtual environment, ensuring a clean and isolated environment. Subsequently, the CLI should be executed within the same virtual environment the packages were installed in. Finally, the environment should include both `python` and `pip`, as the CLI parses the output from executing `python -m pip list` and `python -m pip show` to determine transitive dependencies. Importantly, the CLI only reports transitive dependencies for packages explicitly listed in the requirements.txt file and installed within the current environment. If neither python nor pip are installed the CLI will only to reporting only direct dependencies. requirements.txt contains direct dependencies, and is parsed compliant to its . pip-cli options, URLs, absolute paths, and relative paths are ignored -- though this may be revisited in the future. Dependencies found in requirements.txt have a spec defined by . Dependencies often have version ranges and environment markers (e.g. python version, OS, ...). The resulting graph contains packages tagged with environment markers. For the CLI to detect transitives dependencies for packages specified in a `setup.py` file's `install_requires` field, certain conditions must be met. The packages must be installed with command similar to `python -m pip install .`. The packages should be installed within a virtual environment, ensuring a clean and isolated environment. Subsequently, the CLI should be executed with the same environment. Finally, the environment should include both `python` and `pip`, as the CLI will parse the output from executing `python -m pip list` and `python -m pip show` to determine the transitive dependencies. The CLI will naively scan a setup.py file for a `name` attribute and attempt to match the name with an installed package. If the name matches an installed package, the CLI will exclusively report the transitive dependencies required by the found package. If a name is not matched then the CLI will naively scan for install_requires as explained in the next section. If neither python nor pip are installed the CLI will report dependencies found by naively scanning for install_requires as explained in the next section. setup.py is naively scanned for its

field, which often fails on projects encountered in the wild. Short of implementing a robust python parser, or running a python script in their environment (which may have unintended consequences!), reliable output from setup.py is difficult to obtain. Entries in the `install_requires` array are parsed compliant to the spec, similar to requirements.txt If `setup.cfg` exists in the same directory as `setup.py`, `fossa-cli` also naively scans for its `install_requires=[...]` attributes, similar to `setup.py`. If both `setup.cfg` and `setup.py` exists and both have `install_requires` attribute, `fossa-cli` concatenates requirements from both files. Python requirements files and setup.py files do not provide any data about edges between dependencies. Requirements files can be completely different than an existing setup.py specification, as there is no built-in synchronization between them. The CLI will catch variables* named `installrequires`, as long as they are declared earlier in the file than the `installrequires` keyword argument to `setup`. Because the CLI doesn't actually run `setup.py` or do its own interpretation of the Python code therein, `install_requires` not defined as a literal array of string literals done in the `setup.py` file will hide the true `installrequires` list from the CLI's view. For example, FOSSA CLI does not have a general way to find `installrequires` set up this way: ```python a = ['package1==1.0.0'] b = ['package2==2.0.0'] install_requires = a + b ``` Often, the `requirements.txt` file entirely overlaps the `setup.py` file. This is almost always by design. Assuming no virtual environment, given the following files: `setup.py` (manually created): ```python setup( name='Foo-project', version='1.0', description='Python example project', author='Jeff Jefferson', author_email='bug-catcher@butterfly.net', url='https://this.url/means#nothing', packages=['foo'], install_requires=[ "requests", ], ) ``` `requirements.txt`: Note: your requirements file may have different versions. This file is just a reference example. ```txt certifi==2021.5.30 chardet==4.0.0 idna==2.10 requests==2.25.1 urllib3==1.26.6 ``` We will produce a list of these direct dependencies with no edges between them (see ): ```txt certifi==2021.5.30 chardet==4.0.0 idna==2.10 requests==2.25.1 urllib3==1.26.6 ``` `fossa-cli` uses `python -m pip list` and `python -m pip show` command to infer transitive dependencies, and it's edges. It's paramount that you are in project's prior to invoking `fossa analyze` command. If you are not in project's virtual environment, `fossa-cli` will use global python environment to infer transitive dependencies, edges, and dependency versions. For example, you should always invoke `fossa-cli` in project's python's virtual environment. ```bash python -m venv ~/envs/my-venv source ~/envs/my-venv/bin/activate # this command may differ depending on your python version! pip install -r requirements.txt fossa analyze ``` If you are unable to get any transitive dependencies, ensure you can run following command in project's directory: `python -m pip list --format json` `python -m pip show`