Package: pypants
pypantsCLI for working with Python packages and BUILD files in aPantsmonorepo.Features:Auto-generate BUILD files based on the package type and import statementsGenerate new Python package folders through an interactive CLICompute a topologically-sorted list of dependencies for a given Python build targetTable of Contents:InstallationGuideCommandsProject ConfigurationPackage ConfigurationPackage TypesRegistering Extra TargetsPackage GeneratorsDevelopmentInstallationpypants requires Python 3.6 or abovepipinstallpypantsGuideCommandspypants process-requirementsUpdate3rdparty/python/import-map.jsonusing the entries in3rdparty/python/requirements.txt. All this does is convert the published package name to an import name. Execute this command when you add a new requirement torequirements.txt.pypants process-packagesAuto-generate all relevant BUILD files in the project/repo. You should execute this command in a git pre-commit or pre-push hook so your BUILD files are kept up to date. You can also run it on demand after you add a new dependency to an internal package.pypants generate-packageStarts an interactive CLI that generates a new package folder. This depends on the package generators you registered.Project ConfigurationTo configure your project, add a file named.pypants.cfgto the root of your Git repo and paste the example below. You should define thetop_dirsoption at a minimum.[project]; **************; COMMON OPTIONS; **************; REQUIRED: Top-level directories to search for Python packages. These are relative; to your project/repo root. This is a JSON list of strings.top_dirs=["."]; Prefix to use for names of packages generated by pypants. e.g. foobar_; python_package_name_prefix =; Never look for or process files in these directories. This is a JSON list of; strings. e.g. ["node_modules", "generators"]; ignore_dirs = []; ****************; UNCOMMON OPTIONS; ****************; Set of target package names to ignore when collecting build targets. This is a; JSON list of strings.; ignore_targets = []; Path to the location of the import-map.json file relative to the project root; third_party_import_map_path = 3rdparty/python/import-map.json; Path to the requirements.txt relative to the project root. The default value is; the default that Pants uses.; third_party_requirements_path = 3rdparty/python/requirements.txtBesides the JSON lists, other options are parsed with Python's built-inConfigParser.Package Configurationpypantscurrently expects the Python package to be structured like:mypackage/
├── setup.py
├── .pypants.cfg <---- this is the pypants config file
├── src/
    ├── BUILD <---- pypants will generate this file
    ├── mypackage/
        ├── __init__.py
        ├── ...source code...
├── tests/
    ├── unit/
        ├── BUILD <---- pypants will generate this file
        ├── ...unit tests...
    ├── functional/
        ├── BUILD <---- pypants will generate this file
        ├── ...functional tests...To configure each package, add a file named.pypants.cfgto the package folder and paste the example below. You should define thetypeoption at a minimum.[package]; **************; COMMON OPTIONS; **************; REQUIRED: Package type. See Package Types section for available values.type=library; ****************; UNCOMMON OPTIONS; ****************; Extra set of dependencies to include in the python_library target. This is a; JSON list of strings.; extra_dependencies = []; Extra set of tags to include in the Pants build targets. This is a JSON list of; strings.; extra_tags = []; Flag denoting whether to generate a BUILD file.; generate_build_file = true; Flag denoting whether to generate a pex_binary target for local.py. This is; essentially an extra entry point. It's only used for specific package types.; generate_local_binary = false; Flag denoting whether to include a pex_binary target for pytest; generate_pytest_binary = falsePackage TypesEach of the package types will result in a different BUILD file.libraryThe BUILD file for internal Python libraries has one target defined. For example:python_library(dependencies=["3rdparty/python:arrow","3rdparty/python:isoweek","lib/code/src","lib/logger/src",],sources=["my_library/**/*"],tags={"code","lib","python"},)There is no name provided so this target can be referenced just by its containing folder path. In this case it would be"<TOPDIR>/my_library/src".binaryA binary target can be used for executable scripts (CLIs and servers) and usually depend on internal libraries. The BUILD has a library and binary target defined:python_library(name="lib",dependencies=["3rdparty/python:boto3","3rdparty/python:cfn-flip","3rdparty/python:Click","3rdparty/python:jsonschema","lib/logger/src",],sources=["cli_deploy/**/*"],tags={"apps","code","python"},)pex_binary(name="deploy",dependencies=[":lib"],source="cli_deploy/cli.py",tags={"apps","code","python"},)Thepython_librarytarget is pretty much the same as an internal Python library packageThepex_binarytarget defines an explicit name. This is because when we go to build the PEX file, we want to define the filename. In this example, running./pants binary apps/cli_deploy/src:deploywill result indist/deploy.pex.The only dependency for the binary should be the library. The library will then include all the dependencies.sourcepoints to the entry point of the binary. This module should handle theif __name__ == "__main__"condition to kick off the script.testpypants looks for subfolders named unit, functional, or component within a package'stests/folder. The BUILD file for test folders have a few targets defined. For example:python_library(name="lib/time_utils/tests/unit",dependencies=["3rdparty/python:arrow","lib/python_core/src","lib/time_utils/src"],sources=["**/*"],tags={"lib","python","tests","unit"},)python_tests(dependencies=[":lib/time_utils/tests/unit"],sources=["**/*.py"],tags={"lib","python","tests","unit"},)pex_binary(name="unittest",entry_point="unittest",dependencies=[":lib/time_utils/tests/unit"])Thepython_librarytarget is mostly here to define the unit tests dependencies in a single place so the other two targets can point to itThepython_teststarget lets us run pytest against the test files that match**/*.pyThepex_binarytarget lets us run the unittest module directly. We won't actually package up this target via./pants binary. Setting the entry_point to"unittest"is essentially the same as runningpython -m unittest test_something.pyfrom the command line.lambda_functionThe BUILD file for the Lambda handler contains a special-purpose build target:python_awslambda. This target is a wrapper aroundlambdex. It creates a PEX like thepex_binarytarget (you can execute it) but it modifies the PEX to work with a Lambda Function. For example:python_library(name="my-lambda-lib",sources=["lambda_handler/**/*"],dependencies=["3rdparty/python:requests","lib/logger/src",],)pex_binary(name="my-lambda-bin",source="lambda_handler/lambda_handler.py",dependencies=[":my-lambda-lib"],)python_awslambda(name="my-lambda",binary=":my-lambda-bin",handler="lambda_handler.lambda_handler:lambda_handler",)This BUILD file will be placed in the same folder as the.pypants.cfgfile.migrationThe BUILD file for anAlembicmigration uses thepython_apptarget to include the loose version files:python_library(name="lib",dependencies=["3rdparty/python:alembic","3rdparty/python:SQLAlchemy","lib/core/src",],sources=["**/*"],tags={"code","db","migration","python"},)pex_binary(name="alembic",entry_point="alembic.config",dependencies=[":lib"])python_app(name="migrations-my-database-name",archive="tar",binary=":alembic",bundles=[bundle(fileset=["alembic.ini"]),bundle(fileset=["env.py"]),bundle(fileset=["versions/*.py"]),],tags={"code","db","migration","python"},)This BUILD file will be placed in the same folder as the.pypants.cfgfile.behaveThe BUILD file for abehavetest package includes a library target with test dependencies and a binary target that wraps behave. For example:python_library(name="lib",dependencies=["3rdparty/python:requests","lib/application_config/src",],sources=["**/*"],tags={"integration","python","tests","tests-integration"},)pex_binary(source="behave_cli.py",dependencies=[":lib"],tags={"integration","python","tests","tests-integration"},)This BUILD file will be placed in the same folder as the.pypants.cfgfile.Thebehave_cli.pysource references a wrapper script that you should add to the folder:"""Programmatic entrypoint to running behave from the command line"""importosimportsysfrombehave.__main__importmainasbehave_mainif__name__=="__main__":cwd=os.getcwd()os.chdir(os.path.dirname(__file__))try:exit_code=behave_main(sys.argv[1:])finally:os.chdir(cwd)sys.exit(exit_code)py2sfn_projectpy2sfn is a framework that simplifies the creation and deployment of workflows toAWS Step Functions. The BUILD file for a project only includes a generic target with the set of task dependencies:target(dependencies=["stepfunctions/projects/example-project/tasks/lambda_fetchjoke/src:lib","stepfunctions/projects/example-project/tasks/lambda_generatelist/src:lib","stepfunctions/projects/example-project/tasks/lambda_rankcharactersbyjoke/src:lib",],tags={"py2sfn-project","python","stepfunctions/projects"},)This BUILD file will be placed in the same folder as the.pypants.cfgfile.Registering Extra TargetsIf your project contains internal packages that don't aren't represented cleanly by the.pypants.cfgfile, you can register extra targets programmatically.In your repo, create a new file at.pypants/targets.pyDefine a top-level function calledregister_extra_targets. Within that function, instantiate your extra build targets and return a dictionary that maps package name toBuildTarget.For example, if you have several Alembic database folders:"""Module that defines extra pypants build targets"""fromtypingimportDictfrompypants.configimportPROJECT_CONFIGfrompypants.build_targetsimportAlembicMigrationPackagedefregister_extra_targets()->Dict[str,"pypants.build_targets.base.PythonPackage"]:"""Register extra targets specific to MyProject"""targets={}# Register task targets for Alembic database migration targets## * For migrations, this searches db/ looking for eny.py files. If it finds one,#   it means we've found an Alembic migration folder and can register a build#   target.env_py_paths=PROJECT_CONFIG.config_dir_path.joinpath("db").glob("**/env.py")forenv_py_pathinenv_py_paths:alias=env_py_path.parent.name.replace("_db","").replace("_","-")package_name=f"migrations-{alias}"target=AlembicMigrationPackage(target_type="code",build_template="migration",top_dir_name="db",package_dir_name=env_py_path.parent.name,package_path=str(env_py_path.parent),package_name=package_name,build_dir=str(env_py_path.parent),extra_tags={"migration"},)targets[package_name]=targetreturntargetsPackage GeneratorsThegenerate-packagecommand can be used to create a new package on disk. It sources package "generators" (folders that define the package boilerplate) from the.pypants/generatorsfolder in your repo. To create a new package generator, copy one of the folders fromexamples/generators/to<your repo>/.pypants/generators/<name>and modify it as needed. The generators use a tool calledcookiecutterto rendere templates.DevelopmentIf you're working on pypants locally and want to test out how your changes impact your target project:Activate the virtualenv for your target projectInstall flit:pip install flitKeeping your virtualenv activated, install pypants viacd /path/to/pypants/repo && flit install -s