Package: julia-project
julia_projectThis package provides the classJuliaProjectfor managing aJulia projectthat lives inside
a Python package.julia_projectsupports two libraries for calling Julia from Python,pyjulia(the Python module "julia")
andjuliacall.julia_projectis in pypi; it can be installed viapip install julia_project. It is meant to be used as a library
in other projects.julia_projectis meant to provide some automation, hand holding, and error checking in managing
a Julia dependency in a Python package.For the user of a package that usesjulia_projectSuppose the Python modulemymoduleusesjulia_projectto manage its Julia dependency.
The user ofmymodulecan do the following to import mymodule and install and initialize
the Julia project thatmymoduledepends on.importmymodulemymodule.project.ensure_init()See the docstring forensure_initfor optional arguments.
The author ofmymodulemay have already calledensure_initas step peformed whenimport mymoduleis executed. In this case, callingensure_initagain is a no-op.To compile, or recompile, the Julia project, the user callsmymodule.project.compile().
The compiled Julia system image will be used the next timemymoduleis imported, speeding up
both startup and the first execution of code.Callingmymodule.project.clean()removes the compiled system image and some other files.
This will force again resolving the Julia package requirements on the nextimport mymodule.
Callingmymodule.project.clean_all()will remove the entire project tree.
This is a kind of "factory reset". The next time you runproject.ensure_init()a new directory will be created
and populated with files from the installation directly.Callingmymodule.project.update()checks for compatible updates of Julia packages that
are direct or indirect dependencies ofmymodule, and performs the update.If you want to handle installation and initialization of the Julia project and packages yourself, you can doimportmymodulemymodule.project.disable_init()Then subsequent calls toensure_init, explicit or otherwise will do nothing.project.enable_init()will enable initialization if it has been disabled.If someone else has calledmymodule.project.disable_init()and you want to override it, you
can callmymodule.project.enable_init().Choosing pyjulia or juliacallPass either "juliacall" or "pyjulia" as the argumentcalljuliatoensure_init.
For exampleimportmymodulemyjuliamod.project.ensure_init(calljulia="juliacall")Usingjulia_projectto call Julia functionsA Python-package author can usefind_juliato provide a custom interface to Julia resources.
The author may provide an full-featured or thin interface. In any case it is sometimes
useful to access the Julia/Python interface library directly.
You can also get the imported Python library, eitherjulia(i.e.pyjulia) orjuliacalllike thismyjuliamod.project.juliaFor example, the Julia moduleMainmay be accessed like this.Main=myjuliamod.project.julia.MainMain.sind(90)# 1.0The semantics and syntax of Python modulesjuliaandjuliacallare quite different.
But,julia_projectprovides a minimal common layer.
For example,Example=project.simple_import("Example")imports the Julia moduleExample.Some parts of managing the Julia project are particular to eitherpyjuliaorjuliacall. These are handled by the classesPyJuliaandJuliaCall.
Andproject.calljuliais an instance of one of these.For the author of a package usingjulia_projectThe intended use is as follows.
You want to create a Python package that calls some Julia packages via pyjulia.
You create a directory representing the top level of a Python package,
with asetup.pyandrequirements.txtand the Python code
in a directorymymodule.
You create a file./mymodule/Project.tomldescribing the Julia packages for the project.
In a Python source file in./mymodule/, you create an instance ofjulia_project.JuliaProjectthat manages the Julia project. Call this instanceprojectand import it into mymodule.
For example, in_julia_project.py, you might haveproject = julia_project.JuliaProject([args]).
And in__init__.pyofmymoduleyou havefrom _julia_project.py import project.
(Seethe example directory).What julia_project doesThenimport mymodule; mymodule.project.ensure_init()will do the followingLook for the Julia executable in various places usingfind_juliaOffer to download and install Julia if it is not found.Optionally create a private Julia depot formymoduleto avoid possible issues withPyCallin different Python environments.Check that thejulia(orjuliacall) package is installed.
I.e. check thatPyCall, orPythonCallis installed and built, etc.Optionally download and install a Julia registry.Optionally load a custom Julia system image.Instantiate the Julia project.Provide a Python function that compiles a system image that will be found the next
timemymoduleis imported. The scripts and environment for compilation are found in
a specified subdirectory of the Python project.Write info about all of the above to a log fileUsing julia_project to create a projectHere is a brief example. Seethe example directoryfor a complete example.Include the following in a file loaded in./mymodule/, that is, the directory found byimport mymodule.importosmymodule_path=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))# This just creates an object, but does none of the steps above.project=JuliaProject(name="mymodule",package_path=mymodule_path,registries={"MyModuleRegistry":"git@github.com:myuser/MyModuleRegistry.git"},logging_level=logging.INFO# or WARN, or ERROR,)# If the following is omitted, the user of mymodule must call it explicitly.project.ensure_init()# This exectutes all the management features listed aboveCreate./mymodule/Project.toml(or./mymodule/JuliaProject.toml)  for the Julia project.CompilingMake a folder./mymodule/sys_image/. Add a file./mymodule/sys_image/Project.toml(or./mymodule/sys_image/JuliaProject.toml)
This typically contains the same dependencies as the top-levelProject.toml. Perhaps a few more or less.
Add a script./mymodule/sys_image/packages.jlcontaining anArray{Symbol}of packages to be
included in the image.[:APackage,:AnotherPackage]You don't need to includePyCallorPythonCall.Optionally include a filecompile_exercise_script.jlthat will passed asprecompile_execution_file.After compiling, the system image file will be renamed fromsys_julia_project.so(ordll, ordylib) to a name that includes the version of the julia exectuable that
built it. The latter is the file name that will be searched for the next time you
importmymodule.The project is compiled by calling the methodJuliaProject.compileeither explicitly or during the installation.Arguments to JuliaProjectname,package_path,registries=None,version_spec="^1.6",strict_version=True,sys_image_dir="sys_image",sys_image_file_base=None,calljulia="pyjulia",env_prefix="JULIA_PROJECT_",post_init_hook=None,depot=None,logging_level=None,console_logging=Falsename-- the name of the module, e.g. "mymodule". Used only in the logger and the name of the system image.package_path-- path to the top level ofmymodule.registry_url-- ifNonethen no registry will be installed (other than
the General registry, if not already installed.)version_spec-- A juliaversion compatibility specification. The julia
executable must satisfy this specification.strict_version-- IfTrueprerelease (development) versions of Julia are disallowed when applyingversion_spec.sys_image_dir-- the directory in which scripts for compiling a system image, and the system images, are found. This is
relative to the top level ofmymodule.sys_image_file_base-- the base name of the Julia system image. The system image file will besys_image_file_base + "-" + a_julia_version_string + ".ext",
whereextis the dynamic lib extension for your platform.calljulia-- The julia-from-python interface library. One of two Python packages "pyjulia" and "juliacall".env_prefix-- Prefix for environment variables to set project optionsdepot-- IfTrue, then a private depot in themymoduleinstallation directory will be used.post_init_hook-- A function that will be called immediately beforeensure_initreturns.logging_level-- ifNonethen no logging will be done. iflogging.INFO, then detailed info will be loggedconsole_logging-- ifTrue, then the log messages are echoed to the console.Environment variablesIn the following, the prefixJULIA_PROJECT_may be changed with the argumentenv_prefixdescribed above. This allows you
to set environment variables specific to each project that do not interfere.JULIA_PROJECT_JULIA_PATHmay be set to the path to a Julia executable. This will override other possible paths to a Julia executable.JULIA_PROJECT_INSTALL_JULIAmay be set toyorn. If set, then no interactive query is done to install Julia viajill.py.
Instead the valueyornis used.JULIA_PROJECT_COMPILEmay be set toyorn. If set, then no interactive query is done to compile a system image
after installing packages. Instead the valueyornis used.JULIA_PROJECT_LOG_PATHmay be set to the path to the log file.JULIA_PROJECT_DEPOT-- If set toy, then a private Julia depot will be created in a directorydepotunder themymoduleinstallation directory. The depot contains all downloaded registries, packages, precompiled packages, and
many other data related to your julia installation. Set tonto use the standard depot. If it is unset, you may
be prompted for your choice.Location of julia executableJuliaProjectwill look in the following locations, in orderThe environment variableJULIA_PROJECT_JULIA_PATH. WithJULIA_PROJECT_optionally replaced byenv_prefixdescribed above.In the package top level for the installation./julia/A julia installation fromjill.py, with preferred versions specified as above.Your system or shell PATH variable.A fresh installation of julia viajill.pyafter asking if you want to download and install.BuildingPyCallInstalling and usingPyCallis sometimes easy and sometimes confusing. The latter happens if
you try to usePyCallwith different Python environments. The whole issue can be avoided
by using a private, or Python-package-specific Julia "depot".
Any of the following will create and use such a depot.Enable a new depot by passing the argumentdepot=Truewhen initializing yourJuliaProjectinstance.The user can set the environment variableJULIA_PROJECT_DEPOTdescribed above.If noPyCall.jlis found, the option to create the package-specific depot will be given.If there is a libpython conflict detected during installation you will be prompted to
create a depot.The new depot will be used each timemymoduleis imported. Remove or rename the directorymymodule/depotto prevent this.When using a new depot, registries, packges, cached precompiled files, and many other things are stored
in the installation directory of the project, e.g.mymodule.This is a heavy solution because it involves duplicating many files if you use Julia for other projects, with Python or not.
But, it does not require that the end user understand anything about the status of your Julia installation, libpython,PyCall.jl,
etc.Using a private depot should also allowjulia_projectto work with conda environments.TestingTESTS ARE OUTDATED!You can run tests like this:pytest-pjulia.pytestpluginjulia_project/testsTests that don't require a Julia installation may be run like this:pytest--no-julia-pjulia.pytestpluginjulia_project/testsWarningThis package is very new and is neither well tested nor documented.