Package: ork.build
ORK.BUILD TOOLS (OBT)DESCRIPTIONork.buildis a posix (Linux,OSX)container basedbuild environment which provides shared functionality for common cross-language build automation tasks.ork.buildalso has a set of dependency providers for useful libraries, docker containers, and "subspaces". Unlike homebrew and apt the dependency provider interface is consistent regardless if you are on OSX or Linux - in general the entire interface is consistent on both OSX and Linux. ork.build is implemented primarily in python3. If you need to compose a set of build products (implemented in many languages) with a known working set of versions and configuration data, then ork.build might be for you. It is also important to realize that ork.build is not a replacement for docker style containers. ork.build is specifically abuild containerenvironment, as opposed to a machine or microservice container environment. For example, one might use ork.build to prep content for use in a docker container. If you are wondering when you might choose OBT over another package manager, that might be missing the point. OBT wraps multiple other package managers - The philosphy being when you need a library or some other piece of infrastructure, you need it. The maintainers of said infrastructure made thier choice on package management / distribution or build systems. OBT does not attempt to better those package mgmt/build systems, it attempts toassimilateall of them into a single consistent interface.HISTORYork.buildhistorically derives from orkid's old build system,micro_ork's build systemin conjunction with concepts from orkid's'tozkit'dependency provider system,homebrew,apt, and other build systems and package managers I have worked with over the years.DEFINITIONSStaging Folder - The container which consists of a top level folder in which all build products go and a set of environment variablesModule - a python script in OBT or OBT_SEARCH_PATH that describes and implements a subspace, dependency, target and SDK. There aredep,docker, andsubspacemodules - each providing a different subset of functionality.Subspace - a subdivision of a staging folder containing build products for a specific target, and/or products for a foreign environment such as conda, and/or docker containers (for services or other items best left in a docker container).Host Subspace - The subspace containing products designed for distribution on the build host. Typically used for native apps that require high performance and low latency or hardware access (eg. game engines, realtime software). This is the default subspace. Even this subspace has a custom python (Built from source). Python based deps will build against this subspace's python when this subspace is active.Conda Subspace - Generic Anaconda based subspace. Inherits most paths from Host Subspace, except the python environment. Typically used for data science, ML or python based services running on the host natively (meaning - not in docker).Dependency - a recipe for building a package into a subspace, for a target, using an SDK.Host - the OS instance that is executing OBT in a shell.Target - the OS that code is being generated for (via an SDK)SDK - recipes for how to build products for a given targetSUPPORTED HOSTSLinux x86-64 (tested with ubuntu 20.04 and 22.04)Linux arm64 (tested with ubuntu 18.04 and 20.04)MacOs x86-64 (tested with monterey and big sur)MacOs arm64 (tested with monterey)Top Level Dependencies (User Installed)If using docker functionality - Docker (tryrootless)If using FPGA functionality -Vivado/Quartus[in /opt]If using Houdini functionality -Houdini[in /opt]Top Level Dependencies (apt/homebrew managed)Visit For Reference (Linux)https://github.com/tweakoz/ork.build/blob/develop/bin_pub/obt.ix.installdeps.ubuntu_x86_64.pyVisit For Reference (MacOs)https://github.com/tweakoz/ork.build/blob/develop/bin_pub/obt.osx.installdeps.pyPip Managed dependencies (Per python env)yarltoposortUSAGE Via PIP (to python user directory)Install OBTpip3install--userork.buildInstall system scoped dependencies (requires sudo)OBT will want to assume several packages are present for baseline operationSome dependencies are outside the scope of python, hence we are not using pip.On Ubuntu-x86_64 (20.04,22.04) - sudoREQUIREDReview the script that will be executed if you would like to know what it is doing (especially since sudo is involved)Visithttps://github.com/tweakoz/ork.build/blob/develop/bin_pub/obt.ix.installdeps.ubuntu_x86_64.pyobt.ix.installdeps.ubuntu_x86_64.py# will ask for sudoOn MacOs (Ventura - x86 or arm) - sudoNOT requiredInstall and/or update HomeBrew (https://brew.sh/)Install latest Xcode (Via AppStore)View the OBT system install file if you would like to know what it is doingVisithttps://github.com/tweakoz/ork.build/blob/develop/bin_pub/obt.osx.installdeps.pyobt.osx.installdeps.pyCommonUSAGE (from git, will still require system deps from above..)Clone itgitclonehttps://github.com/tweakoz/ork.buildInstall system scoped dependenciesOn Ubuntu 19.04/20.04obt.ix.installdeps.ubuntu22.pyUbuntu may require a few deps to be installed first, like wget, for example..or On MacOs (only MacOs Catalina Intel tested, atm...)obt.osx.installdeps.pyMacOs will require a few deps to be installed first, such as homebrew and macos commandline build tools.To create an environmentobt.env.create.py--stagedir<staging_folder>--inplaceNote that creating a staging environment will build a few core dependencies, such as python and a python virtual environment.to launch an environment container(the container remembers and references the original ork.build folder)<staging_folder>/obt-launch-envLaunching an environment container will push the launching shell onto the shell process stack, and invoke the modified shell on the next stack level.to exit an environment containerjust exit the shell and you will return to the environment's untouched parent shell.to get list of obt commands: (from bash shell)we use bash's command line completion and ork.build's convention of prefixing all public commands withobtobt.<tabtab>which returns: (example)obt.dep.list.pyobt.dep.require.pyobt.find.pyto get list of obt dependency providersobt.dep.list.pywhich returns something like the followingapitrace : ApiTrace (github-master)
   arachnepnr : arachnepnr (git-c40fb2289952f4f120cc10a5a4c82a6fb88442dc)
arm64_binutils : Arm64 BinUtils (source)
    arm64_gcc : Arm64-Gcc
       assimp : assimp (git-obt-v5.0.1)
  astcencoder : ARM ASTC encoder (github-1.7)
    audiofile : audiofile (git-master)
 avr_binutils : Avr BinUtils (source)
      avr_gcc : Avr GCC (source)
     avr_libc : Avr GCC (source)
        blosc : blosc (git-v1.21.1)
        boost : boost (git-boost-1.77.0)
       bullet : bullet (git-2.89)
         calf : CALF (github-master)
         cgal : cgal (git-releases/CGAL-5.0.2)
        clang : Clang(From LLVM)
        cmake : cmake (git-v3.22.1)
       cppzmq : cppzmq (github-v4.7.1)
    csvparser : csvparser (git-master)
         cuda : cuda (system)
       curlpp : curlpp (git-v0.8.1)
       cycles : cycles (git-master)
     drawtext : drawtext (git-v0.5)
     easyprof : easyprof (git-v2.1.0)
        eigen : eigen (git-3.4.0)
       embree : embree (git-v3.9.0)
        faust : faust (github-2.20.2)
     fcollada : fcollada
         fltk : fltk (git-release-1.3.5)
   fluidsynth : CALF (github-v2.1.0)
 frameretrace : FrameRetrace (github)
    gcode_gpr : A simple C++ G-code parser
    gitpython : gitpython (pip3)
         glfw : glfw (git-216d5e8402513b582563d5b8433fefb449a1593e)
          glm : glm (git-master)
       gnutar : gnutar (wget: ftp.gnu.org/gnu/tar/tar-1.34.tar.xz)
      houdini : <houdini.houdini object at 0x7fac5c655c80>
     icestorm : icestorm (git-83b8ef947f77723f602b706eac16281e37de278c)
          igl : igl (git-master)
irix65_binutils : MIPS/IRIX65 BinUtils (source)
   irix65_gcc : irix65 GCC (source)
     irrlicht : irrlicht (homebrew)
         ispc : ispc (wget: https://github.com/ispc/ispc/releases/download/v1.13.0/ispc-v1.13.0-linux.tar.gz)
     ispctexc : ispctexc (git-master)
    jpegturbo : jpegturbo (git-2.1.2)
       lapack : lapack (git-v3.9.0)
   lemongraph : lemongraph (git-master)
    lexertl14 : lexertl14 (git-e9fd6c95b530f3a3840c65e74e79627732cfd4a7)
      libfive : libfive (git-master)
      libpqpp : libpqpp (git-7.4.1)
librealsense2 : librealsense2 (github-v2.42.0)
    libsocket : libsocket (git-master)
     linuxcnc : <linuxcnc.linuxcnc object at 0x7fac5c74b910>
        litex : <litex.litex object at 0x7fac5c6acc30>
         llvm : llvm (git-llvmorg-12.0.1)
lm32_binutils : <lm32_binutils.lm32_binutils object at 0x7fac5c687e60>
     lm32_gcc : <lm32_gcc.lm32_gcc object at 0x7fac5c6d62d0>
          lua : lua (lua.org-source-v5.2.1)
       luajit : LuaJit (luajit.org-source-v2.1)
m68k_amiga_binutils : 68K BinUtils (source)
m68k_amiga_gcc : Amiga-68k-Gcc
     minetest : Minetest (github-commit-03edcafdda550e55e29bf48a682097028ae01306-source)
      nextpnr : nextpnr (git-67bd349e8f38d91a15f54340b29cc77ef156727f)
 nlohmannjson : nlohmannjson (git-v3.6.1)
          nss : nss (git-NSS_3_63_BRANCH)
         nvtt : nvtt (git-toz_orkdotbuild)
         ocio : ocio (git-v2.0.1)
         oiio : oiio (git-release)
       opencv : OpenCV (github-4.1.0)
       opendb : opendb (git-develop)
      openexr : openexr (git-v2.4.1)
     openjpeg : openjpeg (git-v2.4.0)
     openroad : openroad (git-toztest)
   opensubdiv : opensubdiv (git-release)
      openvdb : openvdb (git-v9.0.0)
       openvr : openvr (git-v1.11.11)
  osgeolaszip : osgeolaszip (git-v2.2.0)
  osgeoliblas : osgeoliblas (git-e6a1aaed412d638687b8aec44f7b12df7ca2bbbb)
    osgeoproj : osgeoproj (git-a892e23d9a444e86b35fc67d0fb84e4acca05c2f)
    osgeotiff : osgeotiff (git-3467bd7b49cca8df29efd606a554b5caf910a3d4)
          osl : osl (git-v1.11.16.0)
   parsertl14 : parsertl14 (git-master)
       pillar : pillar-python-sdk (https://github.com/armadillica/pillar)
    pkgconfig : <pkgconfig.pkgconfig object at 0x7fac5c729d70>
   postgresql : Postgresql (10.4-source)
         ptex : ptex (git-v2.4.1)
      pugixml : pugixml (git-v1.11.4)
     pybind11 : pybind11 (git-v2.7.1)
   pydefaults : <pydefaults.pydefaults object at 0x7fac5c5ea780>
     pyopengl : <pyopengl.pyopengl object at 0x7fac5c6bbf50>
        pyqt5 : pyqt5 ()
      pyside2 : <pyside2.pyside2 object at 0x7fac5c6879b0>
       python : Python3 (3.9.4-source)
          qt5 : QT5
        qt5ct : qt5ct (svn-trunk)
 qt5forpython : qt5forpython (git-5.12)
    rapidjson : rapidjson (git-toz-orkid)
         root : <root.root object at 0x7fac5c6bb460>
       rtmidi : rtmidi (git-4.0.0)
rv32_binutils : <rv32_binutils.rv32_binutils object at 0x7fac5c7578c0>
     rv32_gcc : <rv32_gcc.rv32_gcc object at 0x7fac5c6aca50>
       simavr : <simavr.simavr object at 0x7fac5c6bb9b0>
   unittestpp : UnitTestPP (github-master)
          usd : usd (git-release)
       vivado : <vivado.vivado object at 0x7fac5c6e88c0>
          vpf : VPF (github-master)
          vrx : VRX (github-master)
      vst3sdk : VST3SDK (github-master)
       vulkan : Vulkan (lunarg-1.2.170.0)
          wt4 : WT4 (github-4.1.2)
         yarl : YARL (pip3)
        yosys : yosys (git-96b6410dcb7a82e7be8d4a2025835936f2ca84a7)
       zephyr : ZEPHYR (github-tweakoz/litex-edition)
          zmq : zeromq and bindings (github-v4.3.4)to install a dependency (into the container)eg. boostobt.dep.build.pyboostyou can force a dep wipe and rebuild like this:obt.dep.build.pyboost--force--wipeor an incremntal build (on supported deps).Incremental dep builds are useful when you are modifying the source of dep - typically for fixing bugs or build issues.obt.dep.build.pyboost--incrementalTo check the status of a given dep like thisobt.dep.status.pyoiiowhich would return something likeoiio (git-release)
############################################################################################################################
Dependency(RevTopoOrder)   Supported      Manifest    SrcPresent    BinPresent                                    SourceRoot
############################################################################################################################
0. oiio                         True          True          True          True                            ${OBT_BUILDS}/oiio
1. openexr                      True          True          True          True                         ${OBT_BUILDS}/openexr
2. pybind11                     True          True          True          True                        ${OBT_BUILDS}/pybind11
3. fltk                         True          True          True          True                            ${OBT_BUILDS}/fltk
4. cmake                        True          True          True          True                           ${OBT_BUILDS}/cmake
5. jpegturbo                    True          True          True         False                       ${OBT_BUILDS}/jpegturbo
6. pkgconfig                    True          True          True          True                       ${OBT_BUILDS}/pkgconfig
7. root                         True          True          True          True                            ${OBT_BUILDS}/root
8. pydefaults                   True          True          True          True                      ${OBT_BUILDS}/pydefaults
9. python                       True          True          True          True                          ${OBT_BUILDS}/pythonTo list available subspacesobt.subspace.list.pywhich should return something like..host.py : {'name': 'host'}
     conda.py : {'name': 'conda'}To build a subspaceobt.subspace.build.pycondaTo launch a subspace child shellobt.subspace.launch.pycondaPython: To invoke a conda command in subprocess (without polluting parent process)fromobtimportsubspacesubspace.descriptor("conda").command(["list"])returns..Running Command [conda, list] In Conda Subspace
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py39hecd8cb5_1  
aiohttp                   3.8.1            py39hca72f7f_1  
aiosignal                 1.2.0              pyhd3eb1b0_0  
alabaster                 0.7.12             pyhd3eb1b0_0  
<snipped>To list available docker modulesobt.docker.list.pywhich should return something like..sagemath : {'container_name': 'obt-sagemath', 'image_name': 'obt/sagemath-jupyter', 'version': '9.6'}
   androiddev : {'image_name': 'obt-androiddev:latest'}
     ub-focal : {'image_name': 'obt-focal:latest'}
         cicd : {}
       ps1dev : {'image_name': 'obt-ps1dev:latest'}To launch a docker moduleobt.docker.build.pyps1devIf you have problems building modules, try doing a:dockersystemprune--all