Package: deepr
DeepR is a library for Deep Learning on top of Tensorflow 1.x that focuses on production capabilities. It makes it easy to define pipelines (via theJobabstraction), preprocess data (via thePreproabstraction), design models (via theLayerabstraction) and train them either locally or on a Yarn cluster. It also integrates nicely with MLFlow and Graphite, allowing for production ready logging capabilities.It can be seen as a collection of generic tools and abstractions to be extended for more specific use cases. See theUse DeepRsection for more information.Submitting jobs and defining flexible pipelines is made possible thanks to a config system based off simple dictionaries and import strings. It is similar toThinc config systemorgin configin a lot of ways.To start with deepr read theblogpostthen go toquickstart on colabWhy a Deep Learning Library based on TF1.xTensorflow 1.x provides great production oriented capabilities, centered around thetf.EstimatorAPI. It makes it possible to deploy models using aprotobufwith nopythoncode, and optimize computational graphs with XLA compilation.AlthoughDeepRcomes with aLayerinterface (most similar togoogle TRAXand very close to most modern frameworks) that makes it easy to define models using a functional programming approach, most of its capabilities are orthogonal to it. Most of the building blocks expect genericpythontypes (for example, aLayeris merely a functionfn(tensors, mode)).Use DeepRYou can useDeepRas a simple python library, reusing only a subset of the concepts (the config system is generic for example) or build your own extension as a standalone python package that depends ondeepr.Have a look at the submoduleexamplesofdeeprthat illustrates what packages built on top of deepr would look like. It defines custom jobs, layers, preprocessors, macros as well asconfigs. Once your custom components are packaged in a library, it is easy to run configs withdeepr run config.json macros.jsonMovieLens ExampleYou can try using DeepR on the MovieLens dataset, consisting of movie ratings aggregated by users. The submodulemovielensimplements an AverageModel, a Transformer Model and a BPR loss as well as jobs to build and evaluate on this dataset.You can jump to the notebook onColabor use the command line.pip install deepr[cpu] faiss_cpu
cd deepr/examples/movielens/configs
wget http://files.grouplens.org/datasets/movielens/ml-20m.zip
unzip ml-20m.zip
deepr run config.json macros.jsonInstallationPrerequisitesMake sure you usepython>=3.6and an up-to-date version ofpipandsetuptoolspython --version
pip install -U pip setuptoolsIt is recommended to installdeeprin a new virtual environment. For examplepython -m venv deepr
source deepr/bin/activate
pip install -U pip setuptools
pip install deepr[cpu]Using PipIf installing using pip and your ownrequirements.txtfile, be aware thatTensorflowis listed inextras_requirein thesetup.py, which means thatpip install deeprWON’T INSTALL Tensorflow. This is because the Tensorflow requirement is different depending on the platform (GPU or CPU-only).You can specify which extras to use using the[cpu]or[gpu]argument like in the following examplespip install deepr[cpu]
pip install deepr[gpu]
pip install -e ".[cpu]"
pip install -e ".[gpu]"Or alternatively, pre-install Tensorflow separately like sopip install tensorflow==1.15.2
pip install deeprFrom SourceFirst, clone thedeeprrepo on your local machine withgit clone https://github.com/criteo/deepr.git
cd deeprTo install from source in editable mode, runmake install-cpuOr to install on a GPU enabled machinemake install-gpuTo install development tools and test requirements, runmake install-devTestTo run unit tests in your current environment, runmake testTo run integration tests in your current environment, runmake integrationTo run lint + unit and integration tests in a fresh virtual environment, runmake venv-lint-test-integrationLintTo runmypy,pylintandblack--check:make lintTo auto-format the code usingblackmake blackCommand Line ToolsTo get a list of available commands, rundeepr --helpContributingSeeCONTRIBUTINGChange logSeeCHANGELOGMain contributorsMain contributors and maintainers for deepr areGuillaume Genthial,Romain Beaumont,Denis Kuzin,Amine Benhalloum