Package: creosote
creosoteIdentify unused dependencies and avoid a bloated virtual environment.🌀 Migration guide: creosote 2.x to 3.xExpand me to read the guide.Creosote was updated to 3.0.0 because the way arguments were supplied has now changed. This also bringspyproject.tomlconfiguration support.Argument name changeThe argument naming has changed:2.x argument name3.x argument name--exclude-deps--exclude-dep--paths--path--sections--sectionMultiple argument valuesWith creosote 2.x, you were able to provide multiple values following some arguments, example:creosote-pfile1.pyfile2.pyWith creosote 3.x, you must now provide multiple arguments as a key/value pair:creosote-pfile1.py-pfile2.pyThis new creosote 3.x behavior applies to the following 3.x CLI arguments:--venv--exclude-dep-por--path-sor--section⚡️ QuickstartInstall creosote in separate virtual environment (using e.g.pipx):pipxinstallcreosoteScan virtual environment for unused dependencies (PEP-621example below, butPoetry,Pipenv,PDMandrequirements.txtfiles are also supported,see this table):$ creosote
Found dependencies in pyproject.toml: dotty-dict, loguru, pip-requirements-parser, requests, toml
Oh no, bloated venv! 🤢 🪣
Unused dependencies found: requestsAnd after having removed/uninstalledrequests:$ creosote
Found dependencies in pyproject.toml: dotty-dict, loguru, pip-requirements-parser, toml
No unused dependencies found! ✨✋ Note that you will likely not be able to runcreosoteas-is, but will have to configure it so it understands your project structure.Get help:creosote--help⚙️ ConfigurationYou can configure creosote using commandline arguments or in yourpyproject.toml.Using commandline argumentsRequired argumentsArgumentDefault valueDescription--venvPath to activated virtual environment or.venvThe path(s) to your virtual environment or site-packages folder.--pathsrcThe path(s) to your source code, one or more files/folders.--deps-filepyproject.tomlThe path to the file specifying your dependencies, likepyproject.toml,requirements_*.txt | .in.--sectionproject.dependenciesThe toml section(s) to parse, e.g.project.dependencies.Optional argumentsArgumentDefault valueDescription--exclude-depDependencies you wish to not scan for.--formatdefaultThe output format, valid values aredefault,no-colororporcelain.Usingpyproject.toml[tool.creosote]venvs=[".venv"]paths=["src"]deps-file="pyproject.toml"sections=["project.dependencies"]exclude-deps=["pyodbc","pg8000",]🤔 How this worksThe creosote tool will first scan the given python file(s) for all its imports. Then it fetches all dependency names (from the dependencies spec file). Finally, all imports are associated with their corresponding dependency name (requires the virtual environment for resolving and the ability to read the dependency'sRECORDortop_level.txtfile). If a dependency does not have any imports associated, it is considered unused.See themainfunction incli.pyfor a terse overview of the logic.🌶️ FeaturesThese optional features enable new/experimental functionality, that may be backward incompatible and may be removed/changed at any time. Some features may become mandatory for a target release version e.g. the next major release. Enable using--use-feature <FEATURE>. Use at your own risk!FeatureDescriptionTarget versionfail-excluded-and-not-installedWhen excluding a dependency from the scan (using--exclude-dep) and if the dependency is removed from the dependency specification file (e.g.pyproject.toml), return with exit code 1.N/A😤 Known limitationsimportlibimports are not detected by the AST parser (a great first contribution for anyone inclined 😄, reach out or start looking atparsers.py:get_module_info_from_python_file.🥧 History and ambitionThis project was inspired by security vulnerability reports about production dependencies that were shipped into production but turned out to be unused. Creosote aims to help prevent such occurrences and reduce noise from bots likeDependabotorRenovatefor simply unused dependencies.The intent is to run Creosote in CI (or withpre-commit) to detect cases where developers forget to remove unused dependencies, especially during refactorings. Creosote can identify both unused production dependencies and developer dependencies, depending on your objectives.🤨 FAQWhich dependency specification tooling/standards are supported?Tool/standardSupported--deps-filevalueExample--sectionvaluesPDMandPEP-582✅pyproject.tomlproject.dependencies,project.optional-dependencies.<GROUP>,tool.pdm.dev-dependenciesPipenv✅pyproject.tomlpackages,dev-packagesPoetry✅pyproject.tomltool.poetry.dependencies,tool.poetry.dev-dependencies(legacy),tool.poetry.group.<GROUP>.dependenciesLegacy Setuptools (setup.py)❌PEP-508(requirements.txt,pip-tools)✅*.[txt|in]N/APEP-621✅pyproject.tomlproject.dependencies,project.optional-dependencies.<GROUP>📔 Notes onPEP-508(requirements.txt)When usingrequirements.txtfiles to specify dependencies, there is no way to tell which part ofrequirements.txtspecifies production vs developer dependencies. Therefore, you have to break yourrequirements.txtfile into e.g.requirements-prod.txtandrequirements-dev.txtand use any of them as input. When usingpip-tools, you likely want to point Creosote to scan your*.infile(s).📓 Notes onPEP-582(__pypackages__)Creosote supports the__pypackages__folder, although PEP-582 was rejected. There is no reason to remove support for this today, but in case supporting this becomes cumbersome in the future, supporting PEP-582 might be dropped.creosote--venv__pypackages__Can I specify multiple toml sections?Yes, you can specify a list of sections after the--sectionargument. It all depends on what your setup looks like and what you set out to achieve.$creosote--sectionproject.dependencies--sectionproject.optional-dependencies.lint--sectionproject.optional-dependencies.testCan I exclude dependencies from the scan?Yes, you can use the--exclude-depargument to specify one or more dependencies you do not wish to get warnings for.This feature is intended for dependencies you must specify in your dependencies spec file, but which you don't import in your source code. An example of such a dependency are database drivers, which are commonly only defined in connection strings and will signal to the ORM which driver to use.$creosote--exclude-deppyodbc--exclude-deppg8000Can I run Creosote in a GitHub Action workflow?Yes, please see theactionjob example in.github/workflows/test.yml.Can I run Creosote withpre-commit?Yes, see example in.pre-commit-config.yaml.Here's another example setup, if already have Creosote installed onto $PATH (via e.g. pipx).# .pre-commit-config.yamlrepos:-repo:localhooks:-id:systemname:creosoteentry:creosote --venv .venv --path src --deps-file pyproject.toml --section project.dependenciespass_filenames:falsefiles:\.(py|toml|txt|in|lock)$language:systemWhat's with the name "creosote"?This tool has borrowed its name from theMonty Python scene about Mr. Creosote.📰 Creosote in the "news"Because it makes me happy to see this tool can help others! 🥰Creosote - Identify unused dependencies and avoid a bloated virtual environment— Reddit👩‍🔬 Development/debugging infoInstall in-development buildsYou can run in-development versions of Creosote. Examples below:# Creosote build from main branch$pipxinstall--suffix=@main--forcegit+https://github.com/fredrikaverpil/creosote.git@main
$creosote@main--venv.venv...
$pipxuninstallcreosote@main# Creosote build from PR #123$pipxinstall--suffix=@123--forcegit+https://github.com/fredrikaverpil/creosote.git@refs/pull/123/head
$creosote@123--venv.venv...
$pipxuninstallcreosote@123🚀 ReleasingBump version insrc/creosote/__about__.pyand.pre-commit-config.yaml.GitHub Action will run automatically on creatinga releaseand deploy the release onto PyPi.