Package: merak
MerakMerak is a package building toolkit.This project is started as an attempt to implement a tool that builds a single Cython extension from a Python package, based on the discussion on StackOverflow -Collapse multiple submodules to one Cython extension. See theIdeasection below.More features and functionalities may be added in the future.InstallTo install the current release:$pipinstallmerakTo upgrade Merak to the latest version, add--upgradeflag to the above command.UsageCurrently, Merak only supports thecythonizecommand for building binary extension from a Python package. More features and functionalities may be added in the future.To build a binary extension from a Python package:$merakcythonizePACKAGE_PATHOUTPUT_PATHThe package built will be placed at<OUTPUT_PATH>/<PACKAGE_NAME>. If-fis specified, any existing file / directory at this path will be overwritten.usage: merak cythonize [-h] [-v] [-k] [-s SEP] [-f] path output

positional arguments:
  path               Python package path
  output             Output directory

optional arguments:
  -h, --help         show this help message and exit
  -v, --verbose      Log verbosity level. Default -> WARNING, -v -> INFO, -vv
                     or above -> DEBUG.
  -k, --color        Display logging messages in colors.
  -s SEP, --sep SEP  Module layer separator, must be Python identifier.
                     Defaults to '_'
  -f, --force        Force overwrite if target path exists
  --py-cmd PY_CMD    Python interpreter to be used for building Cython
                     package. Defaults to value of environment variable
                     "PYTHON_CMD", or "python" if "PYTHON_CMD" not specified.ExampleAn example packagefoois included in theexamples/directory. It consists of one subpackagebarwith a modulebazcontaining a functiondo()in it.To build thefoopackage, run the following command in the project root:$merakcythonizeexamples/foofoo-buildThefoobinary package can then be found atfoo-build/foo. Change directory tofoo-buildand use an interactive Python session to try it out:>>> from foo.bar import baz
__main__:1: DeprecationWarning: Deprecated since Python 3.4. Use importlib.util.find_spec() instead.
>>> baz.do()
Running: foo.bar.baz.do()The deprecation warning seems to originate from the import logic in the compiled__init__extension by Cython. It should cause no execution problems at all.The binary package can be built into a Python distribution viasetuptoolsby simply adding asetup.pyin the output directory that includes the cython extension as package data. For this example, addsetup.pytofoo-build/with the following content:importsetuptoolssetuptools.setup(name="foo",version="0.1.0",packages=["foo"],include_package_data=True,package_data={"foo":["*"]},)and run$pythonsetup.pybdist_wheelThe distribution can be found atfoo-build/dist/.IdeaBased onthis answer, it appears that it is possible to build a single Cython extension with multiple modules included in it.However, it does NOT work with multi-level packages. Cython builds a C source file for each module with an initializer namedPyInit_xxx, which depends on the base name of the module. As the function is defined in the global scope, a name collision would happen if the same base name is used for different modules. For instance, the following package would have a name collision for__init__.pyandbase.py:foo/
  __init__.py
  bar/
    __init__.py
    base.py
  baz/
    __init__.py
    base.pyHere, we solve the problem in two steps:Module Flattening:We move all modules to the base layer, with name constructed from their original relative path:path.replace(path_separator, sep), wheresepis a legal Python identifier. For example,foo/bar/base.py->foo/bar_sep_base.pyifsep="_sep_".Import Redirection:We inject a finder inside the main__init__.pythat redirects dotted-paths to their flattened counterparts. Using the above example, the finder redirects the importfoo.bar.basetofoo.bar_sep_base.The injected finder is based onthis answerwith some modifications. See thetemplatefor implementation detail.The result would contain a single__init__extension inside the package folder. The package folder is still required for the builtin importer to load it as a package, rather than a module. The above example would result in afoo/folder with a single__init__Cython extension in it.ResourcesCython.orgStackoverflow - Collapse multiple submodules to one Cython ExtensionLicenseApache License 2.0