Package: collie-recs
collie_recsCollie is a library for preparing, training, and evaluating implicit deep learning hybrid recommender systems, named after the Border Collie dog breed.Collie offers a collection of simple APIs for preparing and splitting datasets, incorporating item metadata directly into a model architecture or loss, efficiently evaluating a model's performance on the GPU, and so much more. Above all else though, Collie is built with flexibility and customization in mind, allowing for faster prototyping and experimentation.See thedocumentationfor more details."We adopted 2 Border Collies a year ago and they are about 3 years old. They are completely obsessed with fetch and tennis balls and it's getting out of hand. They live in the fenced back yard and when anyone goes out there they instantly run around frantically looking for a tennis ball. If there is no ball they will just keep looking and will not let you pet them. When you do have a ball, they are 100% focused on it and will not notice anything else going on around them, like it's their whole world."--A Reddit thread on r/DogTrainingInstallationpipinstallcollie_recsQuick StartImplicit DataCreating and evaluating a matrix factorization model withimplicitMovieLens 100K data is simple with Collie:fromcollie_recs.cross_validationimportstratified_splitfromcollie_recs.interactionsimportInteractionsfromcollie_recs.metricsimportauc,evaluate_in_batches,mapk,mrrfromcollie_recs.modelimportMatrixFactorizationModel,CollieTrainerfromcollie_recs.movielensimportread_movielens_dffromcollie_recs.utilsimportconvert_to_implicit# read in explicit MovieLens 100K datadf=read_movielens_df()# convert the data to implicitdf_imp=convert_to_implicit(df)# store data as ``Interactions``interactions=Interactions(users=df_imp['user_id'],items=df_imp['item_id'],allow_missing_ids=True)# perform a data splittrain,val=stratified_split(interactions)# train an implicit ``MatrixFactorization`` modelmodel=MatrixFactorizationModel(train=train,val=val,embedding_dim=10,lr=1e-1,loss='adaptive',optimizer='adam')trainer=CollieTrainer(model,max_epochs=10)trainer.fit(model)model.eval()# evaluate the modelauc_score,mrr_score,mapk_score=evaluate_in_batches(metric_list=[auc,mrr,mapk],test_interactions=val,model=model)print(f'AUC:{auc_score}')print(f'MRR:{mrr_score}')print(f'MAP@10:{mapk_score}')More complicated examples of implicit pipelines can be viewedfor MovieLens 100K data here,in notebooks here, anddocumentationhere.Explicit DataCollie also handles the situation when you instead haveexplicitdata, such as star ratings. Note how similar the pipeline and APIs are compared to the implicit example above:fromcollie_recs.cross_validationimportstratified_splitfromcollie_recs.interactionsimportExplicitInteractionsfromcollie_recs.metricsimportexplicit_evaluate_in_batchesfromcollie_recs.modelimportMatrixFactorizationModel,CollieTrainerfromcollie_recs.movielensimportread_movielens_dffromtorchmetricsimportMeanAbsoluteError,MeanSquaredError# read in explicit MovieLens 100K datadf=read_movielens_df()# store data as ``Interactions``interactions=ExplicitInteractions(users=df['user_id'],items=df['item_id'],ratings=df['rating'])# perform a data splittrain,val=stratified_split(interactions)# train an implicit ``MatrixFactorization`` modelmodel=MatrixFactorizationModel(train=train,val=val,embedding_dim=10,lr=1e-2,loss='mse',optimizer='adam')trainer=CollieTrainer(model,max_epochs=10)trainer.fit(model)model.eval()# evaluate the modelmae_score,mse_score=explicit_evaluate_in_batches(metric_list=[MeanAbsoluteError(),MeanSquaredError()],test_interactions=val,model=model)print(f'MAE:{mae_score}')print(f'MSE:{mse_score}')Comparison With Other Open-Source Recommendation LibrariesOn some smaller screens, you might have to scroll right to see the full table.➡️Aspect Included in LibrarySurpriseLightFMFastAISpotlightRecBoleTensorFlow RecommendersCollieImplicit data supportfor when we only know when a user interacts with an item or not, not the explicit rating the user gave the item✓✓✓✓✓Explicit data supportfor when we know the explicit rating the user gave the item✓✓✓✓✓✓✓Support forside-dataincorporated directly into the models✓✓✓Support aflexible framework for new model architecturesand experimentation✓✓✓✓✓Deep learninglibraries utilizing speed-ups with a GPU and able to implement new, cutting-edge deep learning algorithms✓✓✓✓✓Automatic support for multi-GPU training✓Actively supported and maintained✓✓✓✓✓✓Type annotationsfor classes, methods, and functions✓✓Scalable for larger, out-of-memory datasets✓✓Includesmodel zoowith two or more model architectures implemented✓✓✓Includesimplicit loss functionsfor training andmetric functionsfor model evaluation✓✓✓✓Includesadaptive loss functionsfor multiple negative examples✓✓✓Includesloss functions that account for side-data✓The following table notes shows the results of an experiment training and evaluating recommendation models in some popular implicit recommendation model frameworks on a commonMovieLens 10Mdataset. The data was split via a 90/5/5 stratified data split. Each model was trained for a maximum of 40 epochs using an embedding dimension of 32. For each model, we used default hyperparameters (unless otherwise noted below).ModelMAP@10 ScoreNotesRandomly initialized, untrained model0.0001Logistic MF0.0128Using the CUDA implementation.LightFMwith BPR Loss0.0180ALS0.0189Using the CUDA implementation.BPR0.0301Using the CUDA implementation.Spotlight0.0376Using adaptive hinge loss.LightFMwith WARP Loss0.0412CollieMatrixFactorizationModel0.0425Using a separate SGD bias optimizer.At ShopRunner, we have found Collie models outperform comparable LightFM models with up to64% improved MAP@10 scores.DevelopmentTo run locally, begin by creating a data path environment variable:# Define where on your local hard drive you want to store data. It is best if this# location is not inside the repo itself. An example is belowexportDATA_PATH=$HOME/data/collie_recsRun development from within the Docker container:dockerbuild-tcollie_recs.# run the container in interactive mode, leaving port ``8888`` open for Jupyterdockerrun\-it\--rm\-v"${DATA_PATH}:/data"\-v"${PWD}:/collie_recs"\-p8888:8888\collie_recs/bin/bashRun on a GPU:dockerbuild-tcollie_recs.# run the container in interactive mode, leaving port ``8888`` open for Jupyterdockerrun\-it\--rm\--gpusall\-v"${DATA_PATH}:/data"\-v"${PWD}:/collie_recs"\-p8888:8888\collie_recs/bin/bashStart JupyterLabTo run JupyterLab, start the container and execute the following:jupyterlab--ip0.0.0.0--no-browser--allow-rootConnect to JupyterLab here:http://localhost:8888/labUnit TestsLibrary unit tests in this repo are to be run in the Docker container:# execute unit testspytest--cov-reportterm--cov=collie_recsNote that a handful of tests require theMovieLens 100K datasetto be downloaded (~5MB in size), meaning that either before or during test time, there will need to be an internet connection. This dataset only needs to be downloaded a single time for use in both unit tests and tutorials.DocsThe Collie library supports Read the Docs documentation. To compile locally,cddocs
makehtml# open local docsopenbuild/html/index.htmlLicenseCopyright (c) 2021 ShopRunner, Inc.Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.