Package: pythonloc
pythonloc: Drop-in Python replacement that imports packages from local directorypythonlocis a drop in replacement forpythonandpipthat automatically recognizes a__pypackages__directory and prefers importing packages installed in this location over user or global site-packages. If you are familiar with node,__pypackages__works similarly tonode_modules.So instead of runningpythonyou runpythonlocand the__pypackages__path will automatically be searched first for packages. And instead of runningpipyou runpiplocand it will install/uninstall from__pypackages__.This is an alternative to using Virtual Environments.This is a Python implementation ofPEP 582, "Python local packages directory". The goal of pythonloc is to make an accessible tool while discussion takes place around adding this functionality to CPython itself. If you prefer, you canbuild your own CPythonwith these changes instead of usingpythonloc.Please note that PEP 582 has not been accepted. It may or not be accepted in the long term.pythonlocis experimental and its API may change in the future.ExamplesScript# myapp.pyimportrequestsprint(requests)>piplocinstallrequests
Installingcollectedpackages:urllib3,certifi,chardet,idna,requests
Successfullyinstalledcertifi-2018.11.29chardet-3.0.4idna-2.8requests-2.21.0urllib3-1.24.1

>pipfreezelocrequests==2.21.0

>pythonlocmyapp.py# works!<module'requests'from'/tmp/demo/__pypackages__/3.6/lib/requests/__init__.py'>TestimonialsFeatured onepisode #117of the Python bytes podcast."Chad has been working and writing some exciting python tools and articles in the packaging/pip space."—Jeff Triplett, Python Software Foundation Director"I’m very enthusiastic about how__pypackages__could help simplify and streamline the Python dependencies workflow. Well done on bringing an early prototype implementation for people to test!"— Florimond Manca, Creator ofBocadillo ProjectSystem RequirementsPython 2.7+pipInstallation: What's in the box?After installing withpip install --user pythonlocorpython3 -m pip install --user pythonlocyou will have four CLI tools available to you:pythonloc,piploc,pipx, andpipfreezeloc.pythonlocShort for "python local", it is a drop-in replacement for python with one important difference: the local directory__pypackages__/<version>/libis added to the front ofsys.path.<version>is the Python version, something like3.7. All arguments are forwarded topython.So instead of runningpython ...you would runpythonloc ...If PEP 582 is adopted,pythonitself will have this behavior.piplocShort for "pip local", it invokes pip with the samesys.pathaspythonloc. If installing a package, the target installation directory is modified to be__pypackages__instead of the globalsite-packages.If__pypackages__directory does not exist it will be created.All arguments are forwarded topip.So instead of runningpip ...you would runpiploc ...If PEP 582 is adopted, I thinkpipshould default to working in the appropriate__pypackages__directory. A flag can be added to install to site-packages, if desired.pipxNote: pipx is included with pythonloc for Python 3.6+ only.Installing packages that have so called "entry points" to__pypackages__presents a problem. The entry points, or "binaries", are no longer available on your $PATH as they would be if you installed in a virtual environment or to your system. These binaries are massively popular and useful. Examples of binaries areblack,pytest,tox,flake8,mypy,poetry, andpipenv(and indeedpythonlocitself).pipxis a binary installer and runner for Python that, when run, searches for a binary in the appropriate__pypackages__location and runs it. If you are familiar with JavaScript'snpx, it's similar to that.So instead of runningBINARY [BINARY ARGS]you would runpipx run BINARY [BINARY ARGS]If not found, pipx will install and run it from a temporary directory. If you require the binary to be found in the__pypackages__directory, you can runpipx run --pypackages BINARY [BINARY ARGS]If the binary is not found, and error will be presented.Note: When installing a new package to an existing__pypackages__directory, the entry points will not be created in.../3.6/lib/bin, for example, if something is already there. To do that, you need to runpiploc install -U PACKAGE. When you do that, the entire contents of the directory will be replaced. Fixing this would require a modification topipitself.If PEP 582 is adopted,pipxwill be a good companion tool to run binaries.pipfreezelocRunningpip freezepresents a problem because it shows all installed python packages: those insite-packagesas well as in__pypackages__. You likely only want to output the packages installed to__pypackages__and that is exactly whatpipfreezelocdoes.It is the equivalent ofpip freezebut only outputs packages in__pypackages__. This is required because there is no built-in way to do this with standard pip. For example, the commandpip freeze --target __pypackages__does not exist.No arguments are handled withpipfreezeloc.So instead of runningpip freeze > requirements.txtyou would runpipfreezeloc > requirements.txtIf PEP 582 is adopted, a more robust solution to freezing the state of__pypackages__should be created.Installing from requirements.txt/LockfilesThis works just like it does in pip. You just need arequirements.txtfile to install from.Installing fromrequirements.txtpiploc install -r requirements.txt
pythonloc <app>Installing frompoetry.lockpip cannot read poetry.lock files, so you'll have to generate a requirements.txt file.Poetry 1.x providesexportcommand:poetryself:update--preview#  install 1.x version of Poetrypoetryexport-frequirements.txt
piplocinstall-rrequirements.txt
pythonloc<app>For Poetry 0.x you can adopt the next approach:poetryrunpipfreeze>requirements.txt
piplocinstall-rrequirements.txt
pythonloc<app>Installing fromPipfile.lockpip cannot readPipfiles yet, only pipenv can. So you will need to generate requirements.txt using pipenv.pipenv lock --requirements
pipenv lock --requirements --dev
piploc install -r requirements.txt
pythonloc <app>In the long term tools will be able to install directly to__pypackages__or piploc will be able to read various lockfile formats.CLIYou can run any python command with pythonloc and it will just run python under the hood:>pythonloc--help
>pythonloc--versionAnother example showing how imports work:>ls

>pythonloc-c"import requests; print(requests)"Traceback(mostrecentcalllast):File"<string>",line1,in<module>
ModuleNotFoundError:Nomodulenamed'requests'>piplocinstallrequests# installs to __pypackages__/3.6/lib/requestsInstallingcollectedpackages:urllib3,certifi,chardet,idna,requests
Successfullyinstalledcertifi-2018.11.29chardet-3.0.4idna-2.8requests-2.21.0urllib3-1.24.1

>pythonloc-c"import requests; print(requests)"# requests is now found<module'requests'from'/tmp/demo/__pypackages__/3.6/lib/requests/__init__.py'>

>piplocuninstallrequests# uninstalls from __pypackages__/3.6/lib/requestsSuccessfullyuninstalledrequests-2.21.0Entry Points / Binaries> piploc install cowsay
Collecting cowsay
  Using cached https://files.pythonhosted.org/packages/e7/e7/e93f311adf63ac8936beb962223771b1ab61227ae3d9ec86e8d9f8f9da1c/cowsay-2.0-py2.py3-none-any.whl
Installing collected packages: cowsay
Successfully installed cowsay-2.0

> pipx run cowsay moooo from local __pypackages__!
  ________________________________
< moooo from local __pypackages__! >
  ================================
                                     \
                                      \
                                        ^__^
                                        (oo)\_______
                                        (__)\       )\/       ||----w |
                                            ||     ||Downsides?While this PEP is pretty exciting, there are a some things it doesn't solve.OS-dependent packages: The directory structure in__pypackages__is namespaced on python version, so packages for Python 3.6 will not mix with 3.7, which is great. But sometimes packages install differently for different OS's, so Windows may not match mac, etc.site-packages: This PEP first looks to__pypackages__but will fall back to looking insite-packages. This is not entirely hermetic and could lead to some confusion around which packages are being used. I would prefer the default search path beonly__pypackages__and nothing else.perceived downside -- bloat: Many have brought this up in various forums, comparing it tonode_modules, but I don't think it applies here. For one, the same if not more content is installed into a virtual environment, so this just moves it into a local directory. No additional bloat. In fact, it is more obvious and can be deleted because it's not hidden away in a virtual env directory.  But more importantly, I think the assumption that it is bloated or will be abused stems from JavaScript's ecosystem. JavaScript has a notoriously limited standard library, and developers need to reach for third party packages more often. In addition, the JavaScript community heavily relies on many plugins and transpilation. Python does not. I do not find the bloat argument convincing.Some pip installation idiosyncracies. For example,pip installwith--targetwill wipe out content in thelib/bindirectory when the-Uflag is passed, but not put anything there when it's not passed.FAQHow is this different from a virtual environment?A virtual environment may or may not include system packages, whereaspythonlocwill first look for packages in., then__pypackages__, then in other locations such as user or site-packages.pythonlocdoes not require activation or deactivationpythonloconly looks for a local directory called__pypackages__. On the other hand, virtual environment activation modifies yourPATHso you can access virtual environment packages no matter which directory you're in.How does it work?It's quite simple and clocks in at less than 100 lines of code. It uses features already built into Python and pip.All it does is provide a slight level of indirection when invoking Python and pip. It modifies thePYTHONPATHenvironment variable when running Python to include__pypackages__.If you consult the output ofpython --help, you'll see this:PYTHONPATH is a ':'-separated list of directories prefixed to the default module search path.  The result is sys.path.pythonloc is an alias forPYTHONPATH=.:__pypackages__/<version>/lib:$PYTHONPATH python PYTHONARGSTo install packages to the__pypackages__directory, it uses pip and runsPYTHONPATH=.:__pypackages__/<version>/lib:$PYTHONPATHpython-mpipPIPARGSwherePIPARGSare whatever arguments you pass it, such aspiploc install requests.It will insert the arguments--target __pypackages__if you are installing a package.What actually gets put in__pypackages__?The installed packages go there. This includes their source code, for example therequestsdirectory below. metadata about the package is stored in the*.dist-infodirectories.If you want to modify or debug the source of an installed package, it's very easy to do so. Just open the appropriate file in__pypackages__and edit away!>piplocinstallrequests
Installingcollectedpackages:urllib3,certifi,chardet,idna,requests
Successfullyinstalledcertifi-2018.11.29chardet-3.0.4idna-2.8requests-2.21.0urllib3-1.24.1


>ls# note __pypackages__ was created__pypackages__

>ls__pypackages__/3.6/lib
binidna-2.8.dist-info
certifirequests
certifi-2018.11.29.dist-inforequests-2.21.0.dist-info
chardeturllib3
chardet-3.0.4.dist-infourllib3-1.24.1.dist-info
idnaHow do I uninstall packages from__pypackages__?piplocwill automatically add__pypackages__to$PYTHONPATH, sopiploc uninstall PACKAGEwill work.If you get the errorNot uninstalling PACKAGE at ..., outside environment ...then rundeactivateto make sure you are not using a virtual environment, then try again.Can I makepythondo this instead of callingpythonloc?An easy way to get this behavior is to create a symlink in your local directoryln -s `which pythonloc` python
ln -s `which piploc` pipThen run them with./python
./pipOtherwise you'll have to build CPython yourself with thereference implementationon GitHub.Why not use the reference implementation of PEP 582?There is more overhead involved in building and distributing a custom CPython build than installing a pip package.You are encouraged to check it out if you are interested though, it's pretty cool!If it gets accepted and added to CPython thenpythonlocmay not be needed anymore.PEP 582reference CPython implementationon GitHub