Package: private-pypi
Private PyPIThe private PyPI server powered by flexible backends.Private PyPIWhat is it?DesignUsageInstall from PyPIUsing the docker image (recommended)Run the serverServer APIUpdate indexBackend-specific commandsEnvironment modeBackendsGitHubFile systemWhat is it?private-pypiallows you to deploy a PyPI server privately and keep your artifacts safe by leveraging the power (confidentiality, integrity and availability) of your storage backend. The backend mechanism is designed to be flexible so that the developer could support a new storage backend at a low cost.Supported backends:GitHub. (Yes, you can now host your Python package in GitHub by usingprivate-pypi. )File system.... (Upcoming)DesignTheprivate-pypiserver serves as an abstraction layer between Python package management tools (pip/poetry/twine) and the storage backends:Package management tools communicate withprivate-pypiserver, followingPEP 503 -- Simple Repository APIfor searching/downloading package, andLegacy APIfor uploading package.private-pypiserver  then performs file search/download/upload operations with some specific storage backend.UsageInstall from PyPIpipinstallprivate-pypi==0.2.0This should bring the execuableprivate_pypito your environment.$private_pypi--help
SYNOPSISprivate_pypi<command><command_flags>

SUPPORTEDCOMMANDSserverupdate_indexgithub.init_pkg_repogithub.gen_gh_pagesUsing the docker image (recommended)Docker image:privatepypi/private-pypi:0.2.0. The image tag is the same as the package version in PyPI.$dockerrun--rmprivatepypi/private-pypi:0.2.0--help
SYNOPSISprivate_pypi<command><command_flags>

SUPPORTEDCOMMANDSserverupdate_indexgithub.init_pkg_repogithub.gen_gh_pagesRun the serverTo run the server, use the commandprivate_pypi server.SYNOPSIS
    private_pypi server ROOT <flags>

POSITIONAL ARGUMENTS
    ROOT (str):
        Path to the root folder. This folder is for logging,
        file-based lock and any other file I/O.

FLAGS
    --config (Optional[str]):
        Path to the package repository config (TOML),
        or the file content if --config_or_admin_secret_can_be_text is set.
        Default to None.
    --admin_secret (Optional[str]):
        Path to the admin secrets config (TOML) with read/write permission.
        or the file content if --config_or_admin_secret_can_be_text is set.
        This field is required for local index synchronization.
        Default to None.
    --config_or_admin_secret_can_be_text (Optional[bool]):
        Enable passing the file content to --config or --admin_secret.
        Default to False.
    --auth_read_expires (int):
        The expiration time (in seconds) for read authentication.
        Default to 3600.
    --auth_write_expires (int):
        The expiration time (in seconds) for write authentication.
        Default to 300.
    --extra_index_url (str):
        Extra index url for redirection in case package not found.
        If set to empty string explicitly redirection will be suppressed.
        Default to 'https://pypi.org/simple/'.
    --debug (bool):
        Enable debug mode.
        Default to False.
    --host (str):
        The interface to bind to.
        Default to '0.0.0.0'.
    --port (int):
        The port to bind to.
        Default to 8888.
    **waitress_options (Dict[str, Any]):
        Optional arguments that `waitress.serve` takes.
        Details in https://docs.pylonsproject.org/projects/waitress/en/stable/arguments.html.
        Default to {}.In short, the configuration passed to--configdefines mappings frompkg_repo_nameto backend-specific settings. In other words, a single server instance can be configured to connect to multiple backends.Exampe of the configuration file passed to--config:[private-pypi-pkg-repo]type="github"owner="private-pypi"repo="private-pypi-pkg-repo"[local-file-system]type="file_system"read_secret="foo"write_secret="bar"Exampe of the admin secret file passed to--admin_secret:[private-pypi-pkg-repo]type="github"raw="<personal-access-token>"[local-file-system]type="file_system"raw="foo"Example run:dockerrun--rm\-v/path/to/root:/private-pypi-root\-v/path/to/config.toml:/config.toml\-v/path/to/admin_secret.toml:/admin_secret.toml\-p8888:8888\privatepypi/private-pypi:0.2.0\server\/private-pypi-root\--config=/config.toml\--admin_secret=/admin_secret.tomlServer APIAuthentication in shellUser must provide thepkg_repo_nameand their secret in most of the API calls so that the server can find which backend to operate and determine whether the operation is permitted or not. Thepkg_repo_nameand the secret should be provided inbasic access authentication.Some package management tools will handle the authentication behind the screen, for example,Twine: to set the environment variablesTWINE_USERNAMEandTWINE_PASSWORD.refPoetry:Configuring credentials.Some will not, for example,Pip: you need to prepend<pkg_repo_name>:<secret>@to the hostname in the URL manually like thishttps://[username[:password]@]pypi.company.com/simple.refAuthentication in browserYou need to visit/loginpage to submitpkg_repo_nameand the secret, since most of the browsers today don't support prepending<username>:<password>@to the hostname in the URL. Thepkg_repo_nameand the secret will be stored in the session cookies. To reset, visit/logout.Example:http://localhost:8888/login/PEP-503, Legacy APIThe server followsPEP 503 -- Simple Repository APIandLegacy APIto define APIs for searching/downloading/uploading package:GET /simple/: List all distributions.GET /simple/<distrib>/: List all packages in a distribution.GET /simple/<distrib>/<filename>: Download a package file.POST /simple/: Upload a package file.In a nutshell, you need to set the "index url / repository url / ..." tohttp://<host>:<port>/simple/for the package management tool.Private PyPI server managementGET /index_mtimeGet the last index index synchronization timestamp.$curlhttp://debug:foo@localhost:8888/index_mtime/1584379892POST /initializeSubmit configuration and (re-)initialize the server. User can change the package repository configuration on-the-fly with this API.# POST the file content.$curl\-d"config=${CONFIG}&admin_secret=${ADMIN_SECRET}"\-XPOST\http://localhost:8888/initialize/# Or, POST the file.$curl\-F'config=@/path/to/config.toml'\-F'admin_secret=@/path/to/admin_secret.toml'\http://localhost:8888/initialize/Update indexIndex file is used to track all published packages in a specific time:Remote index file: the index file sotred in the backend. By design, this file is only updated by a standaloneupdate indexservice and will not be updated by theprivate-pypiserver.Local index file: the index file synchronized from the remote index file by theprivate-pypiserverTo update the remote index file, use the commandprivate_pypi update_index:SYNOPSIS
    private_pypi update_index TYPE NAME <flags>

POSITIONAL ARGUMENTS
    TYPE (str):
        Backend type.
    NAME (str):
        Name of config.

FLAGS
    --secret (Optional[str]):
        The secret with write permission.
    --secret_env (Optional[str]):
        Instead of passing the secret through --secret,
        the secret could be loaded from the environment variable.
    **pkg_repo_configs (Dict[str, Any]):
        Any other backend-specific configs are allowed.Backend developer could setup anupdate indexservice by invokingprivate_pypi update_indexcommand.Backend-specific commandsThe backend registration mechanism will hook up the backend-specific commands toprivate_pypi. As illustrated, commandsgithub.init_pkg_repoandgithub.gen_gh_pagesare registered bygithubbackend.$private_pypi--help
SYNOPSISprivate_pypi<command><command_flags>

SUPPORTEDCOMMANDSserverupdate_indexgithub.init_pkg_repogithub.gen_gh_pagesEnvironment modeIf no argument is passed,private_pypiwill try to load the arguments from the environment variables. This mode would be helpful if passing argument in shell is not possible.The format:PRIVATE_PYPI_COMMAND: to set<command>.PRIVATE_PYPI_COMMAND_<FLAG>: to set the flag of<command>.BackendsGitHubIntroductionprivate-pypiwill help you setup a new GitHub repository to host your package. You package will be published as repository release and secured by personal access token. Takehttps://github.com/private-pypi/private-pypi-pkg-repoandhttps://private-pypi.github.io/private-pypi-pkg-repo/as an example.Configuration and secretPackage repository configuration of GitHub backend:type: must set togithub.owner: repository owner.repo: repository name.branch(optional): the branch to store the remote index file. Default tomaster.index_filename(optional): the name of remote index file. Default toindex.toml.max_file_bytes(optional): limit the maximum size (in bytes) of package. Default to2147483647sinceeach file included in a release must be under 2 GB,as restricted by GitHub.sync_index_interval(optional): the sleep time interval (in seconds) before taking the next local index file synchronization. Default to60.Example configuration ofhttps://github.com/private-pypi/private-pypi-pkg-repo:[private-pypi-pkg-repo]type="github"owner="private-pypi"repo="private-pypi-pkg-repo"The GitHub backend acceptspersonal access tokenas the repository secret. Theprivate-pypiserver calls GitHub API with PAT to operate on packages. You can authorize user with read or write permission based onteam role.Initialize the repositoryTo initialize a GitHub repository as the storage backend, run the commandgithub.init_pkg_repo:dockerrun--rmprivatepypi/private-pypi:0.2.0\github.init_pkg_repo\--nameprivate-pypi-pkg-repo\--ownerprivate-pypi\--repoprivate-pypi-pkg-repo\--token<personal-access-token>This will:Create a new repository<owner>/<repo>.Setup the GitHub workflow to update the remote index file if new package is published.Print the configuration for you.If you want to host the index in GitHub page, likehttps://private-pypi.github.io/private-pypi-pkg-repo/, add--enable_gh_pagesto command execution.GitHub workflow integrationTo useprivate-pypiwith GitHub workflow, takethie main.ymlas an example.Firstly, run the server as job service:services:private-pypi:image:privatepypi/private-pypi:0.2.0ports:-8888:8888volumes:-private-pypi-root:/private-pypi-rootenv:PRIVATE_PYPI_COMMAND:serverPRIVATE_PYPI_COMMAND_ROOT:/private-pypi-rootSecondly, initialize the server with configuration and admin secret (Note: remember toadd the admin secret to your repositorybefore using it):steps:-name:Setup private-pypirun:|curl \-d "config=${CONFIG}&admin_secret=${ADMIN_SECRET}" \-X POST \http://localhost:8888/initialize/env:CONFIG:|[private-pypi-pkg-repo]type = "github"owner = "private-pypi"repo = "private-pypi-pkg-repo"ADMIN_SECRET:|[private-pypi-pkg-repo]type = "github"raw = "${{ secrets.PRIVATE_PYPI_PKG_REPO_TOKEN }}"Afterward, sethttp://localhost:8888/simple/as the repository url, and you are good to go.File systemIntroductionYou can configure this backend to host the packages in the local file system.Configuration and secretPackage repository configuration of GitHub backend:type: must set tofile_system.read_secret: defines the secret with read only permission.write_secret: defines the secret with write permission.max_file_bytes(optional): limit the maximum size (in bytes) of package. Default to5368709119(5 GB).sync_index_interval(optional): the sleep time interval (in seconds) before taking the next local index file synchronization. Default to60.Example configuration:[local-file-system]type="file_system"read_secret="foo"write_secret="bar"To use the API, user must provide eitherread_secretorwrite_secret.Initialize the package repositoryA folder will be created automatically to store the packages, with the path<ROOT>/cache/<pkg_repo_name>/storage.