Package: httpimport
httpimportPython's missing feature!The feature has been suggested in Python Mailing ListRemote,in-memoryPythonpackage/moduleimportingthrough HTTP/SA feature thatPythonmissesand has become popular in other languages is theremote loading of packages/modules.httpimportlets Python packages and modules to beinstalledandimporteddirectly in Python interpreter's process memory, throughremoteURIs, andmore...Python2 support has been discontinued. Last version that supports Python2 is1.1.0.Basic UsageLoad package/module accessible through any HTTP/S locationwithhttpimport.remote_repo('http://my-codes.example.com/python_packages'):importpackage1Load a module from PyPI:withhttpimport.pypi_repo():importdistlib# https://pypi.org/project/distlib/print(distlib.__version__)# '0.3.6' <-- currently latest (https://github.com/pypa/distlib/blob/0.3.6/distlib/__init__.py#L9)Load directly from a GitHub/BitBucket/GitLab repowithhttpimport.github_repo('operatorequals','httpimport',ref='master'):importhttpimportashttpimport_upstream# Also works with 'bitbucket_repo' and 'gitlab_repo'Load a Python module from a Github Gist (usingthis gist):url="https://gist.githubusercontent.com/operatorequals/ee5049677e7bbc97af2941d1d3f04ace/raw/e55fa867d3fb350f70b2897bb415f410027dd7e4"withhttpimport.remote_repo(url):importhellohello.hello()# Hello worldLoad a package/module directly to a variable# From HTTP/S URLhttp_module=httpimport.load('package1','https://my-codes.example.com/python_packages')print(http_module)<module'package1'from'https://my-codes.example.com/python_packages/package1/__init__.py'># From PyPIpypi_module=httpimport.load('distlib',importer_class=httpimport.PyPIImporter)print(pypi_module)<module'distlib'from'https://files.pythonhosted.org/packages/76/cb/6bbd2b10170ed991cf64e8c8b85e01f2fb38f95d1bc77617569e0b0b26ac/distlib-0.3.6-py2.py3-none-any.whl#distlib/__init__.py'>Load Python packages from archives served through HTTP/SNo file is touching the disk in the process# with httpimport.remote_repo('https://example.com/packages.tar'):# with httpimport.remote_repo('https://example.com/packages.tar.bz2'):# with httpimport.remote_repo('https://example.com/packages.tar.gz'):# with httpimport.remote_repo('https://example.com/packages.tar.xz'):withhttpimport.remote_repo('https://example.com/packages.zip'):importtest_packageServing a package through HTTP/SAny package can be served forhttpimportusing a simple HTTP/S Server:echo'print("Hello httpimport!")'>module.py
python-mhttp.server
ServingHTTPon0.0.0.0port8000...>>>importhttpimport>>>withhttpimport.remote_repo("http://127.0.0.1:8000"):...importmodule...Hellohttpimport!ProfilesAfterv1.0.0it is possible to set HTTP Authentication, Custom Headers, Proxies and several other things usingURLandNamed Profiles!URL ProfilesURL Profiles are INI configurations, setting specific per-URL options, as below:[http://127.0.0.1:8000]allow-plaintext:yes; also 'true' and '1' evaluate to True[https://example.com]proxy-url:https://127.0.0.1:8080; values must not be in quotes (')Now, requests tohttp://127.0.0.1:8000will be allowed (HTTP URLs do not work by default) and requests tohttps://example.comwill be sent to an HTTP Proxy.withhttpimport.remote_repo("https://example.com"):# URL matches the URL profileimportmodule_accessed_through_proxyNamed ProfilesNamed Profiles are like URL profiles but do not specify a URL and need to be explicitly used:[github]headers:Authorization:token <Github-Token>And the above can be used as follows:withhttpimport.github_repo('operatorequals','httpimport-private-test',profile='github'):importsecret_moduleGithub Tokens look likegithub_pat_<gibberish>and can be issued here:https://github.com/settings/tokens/newProfiles for PyPIWhen importing from PyPI extra options can be used, as described in the profile below:[pypi]# The location of a 'requirements.txt' file# to use for PyPI project versionsrequirements-file:requirements-dev.txt# Inline 'requirements.txt' syntax appendedrequirements:distlib==0.3.5sampleproject==3.0.0# Only version pinning notation is supported ('==')# with 'requirements' and 'requirements-file' options# A map that contains 'module': 'PyPI project' tuples# i.e: 'import sample' --> search 'sample' module at 'sampleproject' PyPI Project:# https://pypi.org/project/sampleproject/project-names:sample:sampleprojectThe PyPI Profiles can be used exactly like allNamed Profiles:withhttpimport.pypi_repo(profile='pypi'):importdistlibimportsampledistlib.__version__# '0.3.5' <-- pinned in the profile 'requirements' optionsample.__url__# 'https://files.pythonhosted.org/packages/ec/a8/5ec62d18adde798d33a170e7f72930357aa69a60839194c93eb0fb05e59c/sampleproject-3.0.0-py3-none-any.whl#sample/__init__.py' <-- loaded from 'sampleproject'Additionally, all other options cascade to PyPI profiles, such as HTTPS Proxy (HTTP proxies won't work, as PyPI is hosted with HTTPS),headers, etc.NOTE: The values in Profiles MUST NOT be quoted (',")Profile CreationProfiles can be provided as INI strings to theset_profilefunction and used in allhttpimportfunctions:httpimport.set_profile("""[profile1]proxy-url: https://my-proxy.example.comheaders:Authorization: Basic ...X-Hello-From: httpimportX-Some-Other: HTTP header""")withhttpimport.remote_repo("https://code.example.com",profile='profile1'):importmodule_accessed_through_proxyAdvancedProfiles are INI configuration strings parsed using Pythonconfigparsermodule.TheConfigParserobject forhttpimportis the global variablehttpimport.CONFIGand can be used freely:importhttpimporthttpimport.CONFIG.read('github.ini')# Read profiles from a filewithhttpimport.github_repo('operatorequals','httpimport-private-test',profile='github'):importsecret_moduleDefault ProfilesThehttpimportmodule automatically loads Profiles found in$HOME/.httpimport.iniand under the$HOME/.httpimport/directory. Profiles under$HOME/.httpimport/override ones found in$HOME/.httpimport.ini.Profile Options:SupportedHTTP optionszip-password-v1.0.0proxy-url-v1.0.0headers-v1.0.0allow-plaintext-v1.0.0ca-verify-v1.3.0ca-file-v1.3.0PyPI-only optionsproject-names-v1.2.0requirements-v1.2.0requirements-file-v1.2.0Not yet (subject to change)allow-compiledauthauth-typetls-certtls-keytls-passphraseDebugging...importhttpimportimportlogginglogging.getLogger('httpimport').setLevel(logging.DEBUG)Beware:Huge Security Implications!Using thehttpimportwith plainHTTP URLsis highly discouragedAs HTTP traffic can be read and changed from all intermediate hosts (unlike HTTPS), it is possible for a remote adversary to alter the HTTP responses consumed byhttpimportand add arbitraryPythoncode to the downloadedpackages/modules. This directly results in arbitraryRemote Code Executionon your current user's context of your host!In other words, using plain HTTP through the Internet can compromise your hostwithout a way to notice it.You have been warned! Use onlyHTTPS URLswithhttpimport!Contributorsldsink- TheRELOADflag and Bug Fixeslavvy- theload()functionsuperloach- Deprecation ofimpmodule in Python3 in favour ofimportlibyanliakos- Bug Fixrkbennett- Relative Imports fix, Proxy supportDonationsIn case my work helped you, you can always buy me a beer or a liter of gasthrough the Internetor in case you meet me personally.In the second case we can talk about any video ofInternet HistorianorOrdinary Things, while listening to aLofi Girl Playlist, like the citizens of the Internet that we are.