Package: mldock
MLDOCKA docker tool that helps put machine learning in places that empower ml developersWhat is MLDOCK?MLDOCK builds in conveniences and the power of docker and frames it around the core machine learning tasks related to production.As a tool this means MLDOCK's goals are:Provide tooling to improve the ML development workflow. ✅Enable portability of ml code betwen platforms and vendors (Sagemaker, AI Platform, Kubernetes, other container services). ✅Lower the barrier to entry by developing containers from templates. ✅Be ready out the box, using templates to get you started quickly. Bring only your code. ✅For any ML frameworks, runs in any orchestrator and on any cloud. (as long as it integrates with docker) ✅What it is not:Service orchestrator ❌Training Scheduler ❌Hyperparameter tuner ❌Experiment Tracking ❌Inspired bySagify,Sagemaker Training ToolkitandAmazon Sagemaker.Getting StartedSet up your environment(Optional) Use virtual environment to manage dependencies.Installdotenveasily configure environment.pip install --user python-dotenv[cli]note: dotenv allows configuring of environment through the.envfile. MLDOCK uses ENVIRONMENT VARIABLES in the environment to find yourDOCKER_HOST,DOCKERHUBcredentials and evenAWS/GCPcredentials.Create an .env with the following:# for windows and if you are using WSL1
DOCKER_HOST=tcp://127.0.0.1

# for WSL2 and linux (this is default and should work out of the box)
# but for consistency, set this dockerhost

DOCKER_HOST=unix://var/run/docker.socknote: Now to switch environments just use dotenv as follows:dotenv -f "/path/to/.env" run mldock local build --dir <my-project-path>Overview of MLDOCK command lineThe MLDOCK command line utility provides a set of commands to streamline the machine learning container image development process.
The commands are grouped in to 3 functionality sets, namely:Command GroupDescriptioncontainerA set of commands that support creating new containers, initialize and update containers. Also, provides commands for created new MLDOCK supported templates from previously built container images.localA set of commands to use during the development phase. Creating your trainer, prediction scripts and debugging the execution of scripts.registryA set of tools to help you push, pull and interact with image registries.Create your first container image projectInstall MLDOCKThe pip install is the only supported package manager at present. It is recommended that you use an environment manager, either virtualenv or conda will work.pipinstallmldock[cli]Setup local config for the mldock cliThis command sets up mldock cli with some nice to have defaults. It may even prompt you for some set up.mldockconfigureinitInitialize or create your first containerYou will see a some of prompts to set up container.mldockprojectinit--dirmy_ml_containernote:Just hit Return/Enter to accept all the defaults.Build your container image locallymldocklocalbuild--dirmy_ml_containerRun your training locallymldocklocaltrain--dirmy_ml_containerRun your training locallymldocklocaldeploy--dirmy_ml_containerPutting your model in the cloudPush to DockerhubAdd the following to.envDOCKERHUB_USERNAME=<your/user/name>
DOCKERHUB_PASSWORD=<your/dockerhub/password>
DOCKERHUB_REGISTRY=https://index.docker.io/v1/
DOCKERHUB_REPO=<your/user/repo/name>Push your container to dockerhubmldockregistrypush--dirmy_ml_container--providerdockerhub--buildnote: The flags allow you to stipulate configuration changes in the command.--buildsays build the image before pushing. This is required initially since the dockerhub registry will prefix your container name.--providertells MLDOCK to authenticate to dockerhub and push the container there.hintIn addition toDockerHub, bothAWS ECR&GCP GCRare also supported.helpful tipsdocker compose sees my files as directories in mounted volume -USE "./path/to/file" format|https://stackoverflow.com/questions/42248198/how-to-mount-a-single-file-in-a-volumesimlinks from my container have broken permissions in WSL2 |https://github.com/microsoft/WSL/issues/1475