Package: merlin-systems
Merlin SystemsMerlin Systems provides tools for combining recommendation models with other elements of production recommender systems like feature stores, nearest neighbor search, and exploration strategies into end-to-end recommendation pipelines that can be served withTriton Inference Server.QuickstartMerlin Systems uses the Merlin Operator DAG API, the same API used inNVTabularfor feature engineering, to create serving ensembles. To combine a feature engineering workflow and a Tensorflow model into an inference pipeline:importtensorflowastffrommerlin.systems.dagimportEnsemblefrommerlin.systems.dag.opsimportPredictTensorflow,TransformWorkflowfromnvtabular.workflowimportWorkflow# Load saved NVTabular workflow and TensorFlow modelworkflow=Workflow.load(nvtabular_workflow_path)model=tf.keras.models.load_model(tf_model_path)# Remove target/label columns from feature processing workflowkworkflow=workflow.remove_inputs([<target_columns>])# Define ensemble pipelinepipeline=(workflow.input_schema.column_names>>TransformWorkflow(workflow)>>PredictTensorflow(model))# Export artifacts to diskensemble=Ensemble(pipeline,workflow.input_schema)ensemble.export(export_path)After you export your ensemble, you reference the directory to run an instance of Triton Inference Server to host your ensemble.tritonserver--model-repository=/export_path/Refer to theMerlin Example Notebooksfor exploring notebooks that demonstrate
how to train and evaluate a ranking model with Merlin Models and then how to serve it as an ensemble onTriton Inference Server.For training models with XGBoost and Implicit, and then serving with Systems, you can visit theseexamples.Building a Four-Stage Recommender PipelineMerlin Systems can also build more complex serving pipelines that integrate multiple models and external tools (like feature stores and nearest neighbor search):# Load artifacts for the pipelineretrieval_model=tf.keras.models.load_model(retrieval_model_path)ranking_model=tf.keras.models.load_model(ranking_model_path)feature_store=feast.FeatureStore(feast_repo_path)# Define the fields expected in requestsrequest_schema=Schema([ColumnSchema("user_id",dtype=np.int32),])# Fetch user features, use them to a compute user vector with retrieval model,# and find candidate items closest to the user vector with nearest neighbor searchuser_features=request_schema.column_names>>QueryFeast.from_feature_view(store=feature_store,view="user_features",column="user_id")retrieval=(user_features>>PredictTensorflow(retrieval_model_path)>>QueryFaiss(faiss_index_path,topk=100))# Filter out candidate items that have already interacted with# in the current session and fetch item features for the restfiltering=retrieval["candidate_ids"]>>FilterCandidates(filter_out=user_features["movie_ids"])item_features=filtering>>QueryFeast.from_feature_view(store=feature_store,view="movie_features",column="filtered_ids",)# Join user and item features for the candidates and use them to predict relevance scorescombined_features=item_features>>UnrollFeatures("movie_id",user_features,unrolled_prefix="user")ranking=combined_features>>PredictTensorflow(ranking_model_path)# Sort candidate items by relevance score with some randomized explorationordering=combined_features["movie_id"]>>SoftmaxSampling(relevance_col=ranking["output"],topk=10,temperature=20.0)# Create and export the ensembleensemble=Ensemble(ordering,request_schema)ensemble.export("./ensemble")Refer to theExample Notebooksfor exploringbuilding-and-deploying-multi-stage-RecSysnotebooks with Merlin Models and Systems.InstallationMerlin Systems requires Triton Inference Server and Tensorflow. The simplest setup is to use theMerlin Tensorflow Inference Docker container, which has both pre-installed.Installing Merlin Systems Using PipYou can install Merlin Systems withpip:pipinstallmerlin-systemsInstalling Merlin Systems from SourceMerlin Systems can be installed from source by cloning the GitHub repository and runningsetup.pygitclonehttps://github.com/NVIDIA-Merlin/systems.gitcdsystems&&pythonsetup.pydevelopRunning Merlin Systems from DockerMerlin Systems is installed on multiple Docker containers that are available from the NVIDIA GPU Cloud (NGC) catalog.
The following table lists the containers that include Triton Inference Server for use with Merlin.Container NameContainer LocationFunctionalitymerlin-hugectrhttps://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-hugectrMerlin frameworks, HugeCTR, and Triton Inference Servermerlin-tensorflowhttps://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-tensorflowMerlin frameworks selected for only Tensorflow support and Triton Inference ServerIf you want to add support for GPU-accelerated workflows, you will first need to install theNVIDIA Container Toolkitto provide GPU support for Docker. You can use the NGC links referenced in the table above to obtain more information about how to launch and run these containers.Feedback and SupportTo report bugs or get help, pleaseopen an issue.