Package: ml4ir
ml4ir Python QuickstartFor more detailed usage documentation checkml4ir.readthedocs.ioContentsInstallationUsageRunning TestsInstallationUsing ml4ir as a libraryRequirementspython3.{6,7} (tf2.0.3 is not available for python3.8)pip3ml4ir can be installed as a pip package by using the following commandpip3 install ml4irThis will installml4ir-0.1.3(the current version) from PyPI.To install optional dependencies likepygraphviz, use the following command:pip3installml4ir[visualization]To use pre-built pipelines that come with ml4ir, make sure to install it as follows (this installs pyspark and pygraphviz as well)pip install ml4ir[all]Using ml4ir as a toolkit or contributing to ml4irFirstly, clone ml4irgit clone https://github.com/salesforce/ml4irYou can use and develop on ml4ir either using docker or virtualenvDocker (Recommended)Requirementsdocker(18.09+ tested)docker-composeWe have set up adocker-compose.ymlfile for building and using docker containers to train models.Change the working directory to the python packagecd path/to/ml4ir/python/To build the docker image and run unit testsdocker-compose up --buildTo only build the ml4ir docker image without running testsdocker-compose buildVirtual EnvironmentRequirementspython3.{6,7} (tf2.0.3 is not available for python3.8)pip3Change the working directory to the python packagecd path/to/ml4ir/python/Install virtualenvpip3 install virtualenvCreate new python3 virtual environment inside your git repo (it's .gitignored, don't worry)python3 -m venv env/.ml4ir_venv3Activate virtualenvsource env/.ml4ir_venv3/bin/activateInstall all dependenciespip3 install --upgrade setuptools
pip install --upgrade pip
pip3 install -r requirements.txtSet the PYTHONPATH environment variable to point to the python packageexport PYTHONPATH=$PYTHONPATH:`pwd`Contributing to ml4irInstall python dependencies from therequirements.txtanddev-requirements.txtto setup the dependencies required for pre-commit hooks.pre-commit-hooksare required, and installed as a requirement for contributing to ml4ir.
If an error results that they didn't install, executepre-commit installto install git hooks in your .git/ directory.Usageml4ir as a toolkitThe entrypoint into the training or evaluation functionality of ml4ir is throughml4ir/base/pipeline.pyand for application specific overrides, look at `ml4ir/applications/<eg: ranking>/pipeline.pyPipelines currently supported:ml4ir/applications/ranking/pipeline.pyml4ir/applications/classification/pipeline.pyTo run the ml4ir ranking pipeline to train, evaluate and/or test, usedocker-compose run ml4ir \
    python3 ml4ir/applications/ranking/pipeline.py \
    <args>An example ranking training predict and evaluate pipelinedocker-compose run ml4ir \
	python3 ml4ir/applications/ranking/pipeline.py \
	--data_dir ml4ir/applications/ranking/tests/data/tfrecord \
	--feature_config ml4ir/applications/ranking/tests/data/configs/feature_config.yaml \
	--run_id test \
	--data_format tfrecord \
	--execution_mode train_inference_evaluateFor more examples of usage, check:RankingQuery Classificationml4ir as a libraryTo use ml4ir as a deep learning library to build relevance models, look at the following walkthroughs undernotebooks/Learning to Rank: ThePointwiseRankingDemonotebook walks you through building, training, saving, and the entire life cycle of aRelevanceModelfrom the bottom up. You can also find details regarding the architecture of ml4ir in it.Text Classification: TheEntityPredictionDemonotebook walks you through training a model to predict entity type given a user context and query.Ranking Explanations: TheRanking_Explanationsnotebook walks you through per-query explanations for a trained ml4ir modelEnter the following command to spin up Jupyter notebook on your browser to run the above notebookscd path/to/ml4ir/python/
source env/.ml4ir_venv3/bin/activate
pip3 install notebook
jupyter-notebookRunning TestsTo run all the python based tests underml4irUsing dockerdocker-compose upUsing virtualenvpython3 -m pytestTo run specific tests,python3 -m pytest /path/to/test/moduleBuildWe are using CircleCi for the build process.
For code coverage for python, we are usingcoveragePython coverage scores for each PR are calculated by the build and are available in the "Artifacts" section
of thebuild_test_coveragejob.