Package: numba-namespace-extension
Numba Namespace ExtensionThe idea of this package is to facilitate the creation of compiled numba extensionswithoutmodifying the original code implementation. The recommended approach is to use python namespaces so that
we can have the compiled version of the main package with the same access pattern. This looks like the following:Have the original package in a namespace module calledmy_package:$ pip install -e my_packagefrommy_package.utilsimportmy_functionSimple main package configuration using `my_package` namespace.
├── setup.py
├── README.md
└── src
    └──my_package
        └── utils.pyCreate a compiled version of the functions of the original package, in an independent python package using the samemy_pacakgenamespace.$ pip install -e my_package_compiledfrommy_package.compiled.utilsimportmy_functionSimple compiled package configuration using `my_package` namespace.
├── setup.py
├── README.md
├── registry.json
└── src
    └──my_package
        └── compiled
            └── __init__.pyBy following the namespace suggestion, you can keep your original code in themy_packagelibrary and have an extra
independent python package with the compiled version but sharing the same access point (namespace).Shared configuration betweenmy_packageandmy_package_compiled:They both use amy_packagenamespace. In practice, this means that there's no__init__.pyon under themy_packagedirectory.They are probably using thesetuptools.find_namespace_packagesfunction in thesetup.pyfile.Differences between themy_packageandmy_package_compiledimplementations:Themy_package_compiledlibrary depends on already having themy_pacakgelibrary installed.Theregistry.jsonfile contains the information needed for the numba compiler (for each function).Themy_package_compileddoes not has custom-made code; only the compiled version of the functions defined in theregistry.jsonfile.What are python namespace packages?In short, namespace modules are like regular python modules but
without the__init__.pyfile. This allows you to share the same "module name" among different packages.
Consider the following example:Shared namespace:my_packagePackage 1 (package-1): Implement hello world in englishPackage 2 (package-2): Implement hello world in spanishPackage 1: contains ahello_world.pypython submodule within themy_packagenamespace.README.md
setup.py
src/
    my_package/
        hello_world.pyPackage 2: contains ahola_mundo.pypython submodule within themy_packagenamespace.README.md
setup.py
src/
    my_package/
        hola_mundo.pyExpected behavior: you can install each package independently. Nonetheless, they will share a common namespace.
Take a look at a complete working implementationhere.Learn more about namespace packages:In thistwitter thread.From thedocumentation.InstallationYou can install this package viapipby running:$ pip install numba_namespace_extensionAlternatively, for development installation, clone this repo and run:$ pip install -e .UsageThe suggested usage is to create a python package with a matching namespace to the library with the original code.
You can do this by manually creating the new package or leveraging our cookiecutter template with a minimal
python namespace package setup.Install cookiecutter (i.e.,pip install cookiecutter) and execute the template:(Option 1)Referencing Github:cookiecutter https://github.com/RHDZMOTA/numba-namespace-extension.git(Option 2)Or locally if you have this repo already cloned:cookiecutter path/to/repoYou can now start registering functions in theregistry.jsonfile. For each function, provide the following:module (str): the module name where the original function is located.function_name (str): the original function name.signature: the numba type signature (read more thishere).Compilation is done when installing your package. You can install it locally by running:Replacepath/to/my_packagewith the full or relative path to your package (i.e., directory where thesetup.pyis located).pip install -e path/to/my_packageKey components:TheRegistryclass is basically a wrapper over theNumba AOTimplementation that allows us to define source functions on the configuration file (i.e.,registry.json)Use thefrom_jsonstatic method to create aRegistryinstance referencing the json config file.Once you have an instance, run theext_modulesand pass the result to theext_modulesargument from thesetupfunction.Consider that all the modules referenced by theregistry.jsonfile MUST be available in the installation runtime.Example of minimalsetup.pyfile:fromsetuptoolsimportsetup,find_namespace_packagesfromnumba_namespace_extension.registryimportRegistrysetup(name="<package-name>",version="0.1.0",packages=find_namespace_packages(where="src"),package_dir={"":"src"},ext_modules=Registry.from_json("registry.json").ext_modules())