Package: pytorchcml
PyTorchCMLPyTorchCML is a library of PyTorch implementations of matrix factorization (MF) and collaborative metric learning (CML), algorithms used in recommendation systems and data mining.日本語版READMEはこちらWhat is CML ?CML is an algorithm that combines metric learning and MF. It allows us to embed elements of two sets, such as user-item or document-word, into a joint distance metric space using their relational data.In particular, CML is known to capture user-user and item-item relationships more precisely than MF and can achieve higher accuracy and interpretability than MF for recommendation systems [1]. In addition, the embeddings can be used for secondary purposes such as friend recommendations on SNS and similar item recommendations on e-commerce sites.For more details, please refer to this reference [1].InstallationYou can install PyTorchCML using Python's package manager pip.pipinstallPyTorchCMLYou can also download the source code directly and build your environment with poetry.。gitclonehttps://github.com/hand10ryo/PyTorchCML
poetoryinstalldependenciesThe dependencies are as followspython = ">=3.7.10,<3.9"torch = "^1.8.1"scikit-learn = "^0.22.2"scipy = "^1.4.1"numpy = "^1.19.5"pandas = "^1.1.5"tqdm = "^4.41.1"UsageExampleThisis a jupyter notebook example using the Movielens 100k dataset.OverviewThis library consists of the following six modules.trainersmodelssamplerslossesregularizersevaluatorsBy combining these modules, you can implement a variety of algorithms.The following figure shows the relationship between these modules.The most straightforward implementation is as follows.importtorchfromtorchimportoptimimportnumpyasnpfromPyTorchCMLimportlosses,models,samplers,trainersdevice=torch.device("cuda:0"iftorch.cuda.is_available()else"cpu")# train dataset (whose columns are [user_id, item_id].)train_set=np.array([[0,0],[0,1],[1,2],[1,3]])train_set_torch=torch.LongTensor(train_set).to(device)n_user=train_set[:,0].max()+1n_item=train_set[:,1].max()+1# model settingsmodel=models.CollaborativeMetricLearning(n_user,n_item,n_dim=10).to(device)optimizer=optim.Adam(model.parameters(),lr=1e-3)criterion=losses.MinTripletLoss(margin=1).to(device)sampler=samplers.BaseSampler(train_set_torch,n_user,n_item,device=device)trainer=trainers.BaseTrainer(model,optimizer,criterion,sampler)# runtrainer.fit(n_batch=256,n_epoch=3)The inputtrain_setrepresents a two-column NumPy array whose records are the user IDs and item IDs that received positive feedback.Then_userandn_itemare the number of users and items. Here, we assume that user ID and item ID start from 0 and that all users and items are included in the train_set.Then, define model, optimizer, criterion, and sampler, input them to a trainer and run the trainer's fit method to start learning CMmodelsThe models is the module that handles the embeddings.There are currently two models to choose from as follows.models.CollaborativeMetricLearningmodels.LogitMatrixFactorizationYou can predict the relationship between the target user and the item with thepredictmethod.CML uses vector distance, while MF uses the inner product to represent the relationship.You can also set the maximum norm and initial value of the embeddings.For example, inLogitMatrixFactorization, this is how it works.model=models.LogitMatrixFactorization(n_user,n_item,n_dim,max_norm=5,user_embedding_init=torch.Tensor(U),# shape = (n_user, n_dim)item_embedding_init=torch.Tensor(V.T),# shape = (n_dim, n_item)).to(device)lossesThe losses module is for handling the loss function for learning embeddings.
We can mainly divide the loss function into PairwiseLoss and TripletLoss.PairwiseLoss is the loss for each user-item pair.TripletLoss is the loss per.
Here,is a positive pair, andis a negative pair.In general, CML uses triplet loss, and MF uses pairwise loss.samplersThe samplers is a module that handles the sampling of mini-batches during training.There are two types of sampling done by the sampler.Sampling of positive user-item pairsSampling of negative itemsThe default setting is to sample both with a uniform random probability.It is also possible to weigh both positively and negatively.For example, if you want to weigh the items by their popularity, you can follow.item_ids,item_popularity=np.unique(train_set[:,1],return_counts=True)sampler=samplers.BaseSampler(train_set_torch,neg_weight=item_popularity,n_user,n_item,device=device)trainersThe trainers is the module that handles training.You can train by setting up a model, optimizer, loss function, and sampler.evaluatorsThe evaluators is a module for evaluating performance after learning.You can evaluate your model as follows.fromPyTorchCMLimportevaluators# test set (whose columns are [user_id, item_id, rating].)test_set=np.array([[0,2,3],[0,3,4],[1,0,2],[1,1,5]])test_set_torch=torch.LongTensor(test_set).to(device)# define metrics and evaluatorscore_function_dict={"nDCG":evaluators.ndcg,"MAP":evaluators.average_precision,"Recall":evaluators.recall}evaluator=evaluators.UserwiseEvaluator(test_set_torch,score_function_dict,ks=[3,5])# calc scoresscores=evaluator.score(model)Thetest_setis a three-column NumPy array with user ID, item ID, and rating records.Thescore_function_dictis a dictionary of evaluation metrics. Its key is a name, and its value is a function to compute the evaluation metric. The evaluators module implements nDCG@k, MAP@k, and Recall@k as its functions. In this example, those three are set, but you can set any number of evaluation indicators.Theevaluatortakes input test data, evaluation metrics, and a list with @k types.You can calculate the scores by running the method.score()with the model as input.  Its outputscoreswill be a single row pandas.DataFrame with each score. In this example, its columns are["nDCG@3", "MAP@3", "Recall@3", "nDCG@5", "MAP@5", "Recall@5"].Also, inputting the evaluator to thevalid_evaluatorargument of the fit method of the trainer will allow you to evaluate the learning progress.
This system is helpful for hyperparameter tuning.valid_evaluator=evaluators.UserwiseEvaluator(test_set_torch,# eval setscore_function_dict,ks=[3,5])trainer.fit(n_batch=50,n_epoch=15,valid_evaluator=valid_evaluator)regularizersThe regularizers is a module that handles the regularization terms of embedded vectors.You can implement the L2 norm, etc., by entering a list of regularizer instances as the argument of the loss function, as shown below.fromPyTorchCMLimportregularizersregs=[regularizers.L2Regularizer(weight=1e-2)]criterion=losses.MinTripletLoss(margin=1,regularizers=regs).to(device)It is also possible to introduce multiple regularizations by increasing the length of the list.adaptorsThe adaptors is a module for realizing domain adaptation.Domain adaptation in CML is achieved by addingto the loss for featureof item. The same is true for the user. This allows us to reflect attribute information in the embedding vector.MLPAdaptor is a class of adaptors that assumes a multilayer perceptron in function.You can set up the adaptor as shown in the code belowfromPyTorchCMLimportadaptors# item_feature.shape = (n_item, n_feature)item_feature_torch=torch.Tensor(item_feature)adaptor=adaptors.MLPAdaptor(item_feature_torch,n_dim=10,n_hidden=[20],weight=1e-4)model=models.CollaborativeMetricLearning(n_user,n_item,n_dim,item_adaptor=adaptor).to(device)Developmentpipinstallpoetry
pipinstallpoetry-dynamic-versioning

poetryinstall
poetrybuild
poetrylockCitationYou may use PyTorchCML under MIT License. If you use this program in your research then please cite:@misc{matsui2021pytorchcml,
  author = {Ryo, Matsui},
  title = {PyTorchCML},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {https://github.com/hand10ryo/PyTorchCML}
}References[1] Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin.Collaborative metric learning. InProceedings of the 26th International Conference on World WideWeb, pp. 193–201, 2017.