Package: irspack
irspack - Implicit recommender systems for practitionersDocsirspackis a Python package for recommender systems based on implicit feedback, designed to be used by practitioners.Some of its features include:Efficient parameter tuning enabled by C++/Eigen implementations of core recommender algorithms andoptuna.In particular, if an early stopping scheme is available, optuna can prune out unpromising trial based on the intermediate validation scores.Various utility functions, includingID/index mapping utilitiesFast, multithreaded argsort for batch recommendation retrievalEfficient and configurable evaluation of recommender system performanceInstallation & Optional DependenciesIn most cases, you can install the pre-build binaries viapipinstallirspackThe binaries have been compiled to use AVX instruction. If you want to use AVX2/AVX512 or your environment does not support AVX (like Rosetta 2 on Apple M1), install it from source likeCFLAGS="-march=native"pipinstallgit+https://github.com/tohtsky/irspack.gitIn that case, you must have a decent version of C++ compiler (with C++11 support). If it doesn't work, feel free to make an issue!Optional DependenciesI have also prepared a wrapper class (BPRFMRecommender) to train/optimize BPR/warp loss Matrix factorization implemented inlightfm. To use it you have to installlightfmseparately, e.g. bypipinstalllightfmIf you want to use Mult-VAE, you'll need the following additional (pip-installable) packages:jaxjaxlibIf you want to use GPU, follow the installation guidehttps://github.com/google/jax#installationdm-haikuoptaxBasic UsageStep 1. Train a recommenderTo begin with, we represent the user/item interaction as ascipy.sparsematrix. Then we can feed it into recommender classes:importnumpyasnpimportscipy.sparseasspsfromirspackimportIALSRecommender,df_to_sparsefromirspack.datasetimportMovieLens100KDataManagerdf=MovieLens100KDataManager().read_interaction()# Convert pandas.Dataframe into scipy's sparse matrix.# The i'th row of `X_interaction` corresponds to `unique_user_id[i]`# and j'th column of `X_interaction` corresponds to `unique_movie_id[j]`.X_interaction,unique_user_id,unique_movie_id=df_to_sparse(df,'userId','movieId')recommender=IALSRecommender(X_interaction)recommender.learn()# for user 0 (whose userId is unique_user_id[0]),# compute the masked score (i.e., already seen items have the score of negative infinity)# of items.recommender.get_score_remove_seen([0])Step 2. Evaluation on a validation setTo evaluate the performance of a recommenderm we have to split the dataset to train and validation sets:fromirspack.splitimportrowwise_train_test_splitfromirspack.evaluationimportEvaluator# Random splitX_train,X_val=rowwise_train_test_split(X_interaction,test_ratio=0.2,random_state=0)evaluator=Evaluator(ground_truth=X_val)recommender=IALSRecommender(X_train)recommender.learn()evaluator.get_score(recommender)This will print something like{'appeared_item':435.0,'entropy':5.160409123151053,'gini_index':0.9198367595008214,'hit':0.40084835630965004,'map':0.013890322881619916,'n_items':1682.0,'ndcg':0.07867240014767263,'precision':0.06797454931071051,'recall':0.03327028758587522,'total_user':943.0,'valid_user':943.0}Step 3. Hyperparameter optimizationNow that we can evaluate the recommenders' performance against the validation set, we can useoptuna-backed hyperparameter optimization.best_params,trial_dfs=IALSRecommender.tune(X_train,evaluator,n_trials=20)# maximal ndcg around 0.43 ~ 0.45trial_dfs["ndcg@10"].max()Of course, we have to hold-out another interaction set for test, and measure the performance of tuned recommender against the test set.Seeexamples/for more complete examples.TODOsmore benchmark dataset