Package: oreum_core
Oreum Core Toolsoreum_core1. Description and ScopeThis is an ever-growing package of core tools for use on client projects by
Oreum Industries.Provides an essential workflow for data curation, EDA, basic ML using the core
scientific Python stack incl.numpy,scipy,matplotlib,seaborn,pandas,scikit-learn,umap-learnOptionally provides an advanced Bayesian modelling workflow in R&D and
Production using a leading probabilistic programming stack incl.pymc,pytensor,arviz(dopip install oreum_core[pymc])Optionally enables a generalist black-box ML workflow in R&D using a leading
Gradient Boosted Trees stack incl.catboost,xgboost,optuna,shap(dopip install oreum_core[tree])Also includes seevral utilities for text cleaning, sql scripting, file handlingThis packageis:A work in progress (v0.y.z) and liable to breaking changes and inconvenience
to the userSolely designed for ease of use and rapid development by employees of
Oreum Industries, and selected clients with guidanceThis packageis not:Intended for public usage and will not be supported for public usageIntended for contributions by anyone not an employee of Oreum Industries,
and unsolicitied contributions will not be accepted.NotesProject began on 2021-01-01TheREADME.mdis MacOS and POSIX orientedSeeLICENCE.mdfor licensing and copyright detailsSeeCONTRIBUTORS.mdfor list of contributorsThis uses a logger named'oreum_core', feel free to incorporate or ignoreHosting:Source code repo onGitHubSource code release onGitHubPackage release onPyPi2. Instructions to Create Dev EnvironmentFor local development on MacOS2.0 Pre-requisite installs viahomebrewInstall Homebrew, see instuctions athttps://brew.shInstalldirenv,git,git-lfs,graphviz,zsh$>brewupdate&&upgrade
$>brewinstalldirenvgitgit-lfsgraphvizzsh2.1 Git clone the repoAssumesdirenv,git,git-lfsandzshinstalled as above$>gitclonehttps://github.com/oreum-industries/oreum_core
$>cdoreum_coreThen allowdirenvon MacOS to autorun file.envrcupon directory open2.2 Create virtual environment and install dev packagesNotes:We usecondavirtual envs controlled bymamba(quicker thanconda)We install packages usingminiforge(sourced from theconda-forgerepo)
wherever possible and only usepipfor packages that are handled better bypipand/or more up-to-date onpypiPackages might not be the very latest because we want stability forpymcwhich is usually in a state of development fluxSeecheat sheet of conda commandsTheMakefilecreates a dev env and will also download and preinstallminiforgeif not yet installed on your system2.2.1 Create the dev environmentFrom the dir aboveoreum_core/project dir:$>make-Coreum_core/devThis will also create some files to help confirm / diagnose successful installation:dev/install_log/blas_info.txtfor theBLAS MKLinstallation fornumpydev/install_log/pipdeptree[_rev].txtlists installed package deps (and reversed)LICENSES_THIRD_PARTY.mddetails the license for each package used2.2.2 (Optional best practice) Test successful installation of dev environmentFrom the dir aboveoreum_core/project dir:$>make-Coreum_core/test-dev-envThis will also add filesdev/install_log/[numpy|scipy].txtwhich detail
successful installation (or not) fornumpy,scipy2.2.3 (Useful during env install experimentation): To remove the dev environmentFrom the dir aboveoreum_core/project dir:$>make-Coreum_core/uninstall-env2.3 Code Linting & Repo Control2.3.1 Pre-commitWe usepre-committo run a suite of automated tests
for code linting & quality control and repo control prior to commit on local
development machines.Precommit is already installed by themake devcommand (which itself callspip install -e .[dev])The pre-commit script will then run on your system upongit commitSee this project's.pre-commit-config.yamlfor details2.3.2 Github ActionsWe useGithub Actionsaka
Github Workflows to run:A suite of automated tests for commits received at the origin (i.e. GitHub)Publishing to PyPi upon creating a GH ReleaseSeeMakefilefor the CLI commands that are issuedSee.github/workflows/*for workflow detailsCopyright 2024 Oreum OÜ t/a Oreum Industries. All rights reserved.
See LICENSE.md.Oreum OÜ t/a Oreum Industries, Sepapaja 6, Tallinn, 15551, Estonia,
reg.16122291,oreum.ioOreum OÜ © 2024