Repository: RUCAIBox/RecBole (language: python)

Issue:
FEA: add SimpleX in general model
add SimpleX in general model


Discussion / hints:

Fix patch:
diff --git a/docs/source/asset/simplex.png b/docs/source/asset/simplex.png
new file mode 100644
index 000000000..aa667de86
Binary files /dev/null and b/docs/source/asset/simplex.png differ
diff --git a/docs/source/recbole/recbole.model.general_recommender.rst b/docs/source/recbole/recbole.model.general_recommender.rst
index 2e349f7cf..d9a3aa951 100644
--- a/docs/source/recbole/recbole.model.general_recommender.rst
+++ b/docs/source/recbole/recbole.model.general_recommender.rst
@@ -21,6 +21,7 @@ recbole.model.general\_recommender
    recbole.model.general_recommender.neumf
    recbole.model.general_recommender.ngcf
    recbole.model.general_recommender.pop
-   recbole.model.general_recommender.spectralcf
    recbole.model.general_recommender.sgl
-   recbole.model.general_recommender.ncl
\ No newline at end of file
+   recbole.model.general_recommender.simplex
+   recbole.model.general_recommender.spectralcf
+   recbole.model.general_recommender.ncl
diff --git a/docs/source/recbole/recbole.model.general_recommender.simplex.rst b/docs/source/recbole/recbole.model.general_recommender.simplex.rst
new file mode 100644
index 000000000..71a4a2ab8
--- /dev/null
+++ b/docs/source/recbole/recbole.model.general_recommender.simplex.rst
@@ -0,0 +1,4 @@
+.. automodule:: recbole.model.general_recommender.simplex
+   :members:
+   :undoc-members:
+   :show-inheritance:
\ No newline at end of file
diff --git a/docs/source/user_guide/model/general/simplex.rst b/docs/source/user_guide/model/general/simplex.rst
new file mode 100644
index 000000000..72fdb8232
--- /dev/null
+++ b/docs/source/user_guide/model/general/simplex.rst
@@ -0,0 +1,91 @@
+SimpleX
+===========
+
+Introduction
+---------------------
+
+`[paper] <https://dl.acm.org/doi/10.1145/3459637.3482297>`_
+
+**Title:** SimpleX: A Simple and Strong Baseline for Collaborative Filtering
+
+**Authors:** Kelong Mao, Jieming Zhu, Jinpeng Wang, Quanyu Dai, Zhenhua Dong, Xi Xiao, Xiuqiang He
+
+**Abstract:**  Collaborative filtering (CF) is a widely studied research topic in recommender systems.
+The learning of a CF model generally depends on three major components, namely interaction encoder, loss
+function, and negative sampling. While many existing studies focus on the design of more powerful
+interaction encoders, the impacts of loss functions and negative sampling ratios have not yet been well
+explored. In this work, we show that the choice of loss function as well as negative sampling ratio is
+equivalently important. More specifically, we propose the cosine contrastive loss (CCL) and further
+incorporate it to a simple unified CF model, dubbed SimpleX. Extensive experiments have been conducted
+on 11 benchmark datasets and compared with 29 existing CF models in total. Surprisingly, the results show
+that, under our CCL loss and a large negative sampling ratio, SimpleX can surpass most sophisticated
+state-of-the-art models by a large margin. We believe that SimpleX could not only serve as a simple
+strong baseline to foster future research on CF, but also shed light on the potential research direction
+towards improving loss function and negative sampling.
+
+.. image:: ../../../asset/simplex.png
+    :width: 500
+    :align: center
+
+Running with RecBole
+-------------------------
+
+**Model Hyper-Parameters:**
+
+- ``embedding_size (int)`` : The embedding size of users, items. Defaults to ``64``.
+- ``margin (float)`` : The margin to filter negative samples in CCL loss. Range
+  in ``[-1, 1]``. Defaults to ``0.9``.
+- ``negative_weight (int)`` : Control the relative weights of positive-sample loss and negative-sample 
+  loss in CCL loss. Defaults to ``10``.
+- ``gamma (float)`` : The weights for fusion of user representations and historically interacted
+  items representations. Range in ``[0, 1]``. Defaults to ``0.5``.
+- ``aggregator (str)`` : The ways to aggregate historically interacted items representations. Range
+  in ``['mean', 'user_attention', 'self_attention']``. Defaults to ``'mean'``.
+- ``history_len (int)`` : The length of the user's historical interaction items. Defaults to ``50``.
+- ``reg_weight (float)`` : The L2 regularization weights. Defaults to ``1e-05``.
+
+
+**A Running Example:**
+
+Write the following code to a python file, such as `run.py`
+
+.. code:: python
+
+  from recbole.quick_start import run_recbole
+
+  run_recbole(model='SimpleX', dataset='ml-100k')
+
+And then:
+
+.. code:: bash
+
+  python run.py
+
+Tuning Hyper Parameters
+-------------------------
+
+If you want to use ``HyperTuning`` to tune hyper parameters of this model, you can copy the following settings and name it as ``hyper.test``.
+
+.. code:: bash
+
+  margin choice [0,0.5,0.9]
+  negative_weight choice [0,10,50]
+  gamma choice [0.3,0.5,0.7]
+
+Note that we just provide these hyper parameter ranges for reference only, and we can not guarantee that they are the optimal range of this model.
+
+Then, with the source code of RecBole (you can download it from GitHub), you can run the ``run_hyper.py`` to tuning:
+
+.. code:: bash
+
+	python run_hyper.py --model=[model_name] --dataset=[dataset_name] --config_files=[config_files_path] --params_file=hyper.test
+
+For more details about Parameter Tuning, refer to :doc:`../../../user_guide/usage/parameter_tuning`.
+
+
+If you want to change parameters, dataset or evaluation settings, take a look at
+
+- :doc:`../../../user_guide/config_settings`
+- :doc:`../../../user_guide/data_intro`
+- :doc:`../../../user_guide/train_eval_intro`
+- :doc:`../../../user_guide/usage`
\ No newline at end of file
diff --git a/docs/source/user_guide/model_intro.rst b/docs/source/user_guide/model_intro.rst
index d88678361..cbc88a25d 100644
--- a/docs/source/user_guide/model_intro.rst
+++ b/docs/source/user_guide/model_intro.rst
@@ -40,6 +40,7 @@ task of top-n recommendation. All the collaborative filter(CF) based models are
    model/general/sgl
    model/general/admmslim
    model/general/nceplrec
+   model/general/simplex
    model/general/ncl
 
 
diff --git a/recbole/model/general_recommender/__init__.py b/recbole/model/general_recommender/__init__.py
index 1b8aacbd9..71cf8cbb1 100644
--- a/recbole/model/general_recommender/__init__.py
+++ b/recbole/model/general_recommender/__init__.py
@@ -27,4 +27,5 @@
 from recbole.model.general_recommender.nceplrec import NCEPLRec
 from recbole.model.general_recommender.sgl import SGL
 from recbole.model.general_recommender.admmslim import ADMMSLIM
+from recbole.model.general_recommender.simplex import SimpleX
 
diff --git a/recbole/model/general_recommender/simplex.py b/recbole/model/general_recommender/simplex.py
new file mode 100644
index 000000000..9532a6902
--- /dev/null
+++ b/recbole/model/general_recommender/simplex.py
@@ -0,0 +1,254 @@
+# -*- coding: utf-8 -*-
+# @Time   : 2022/3/25 13:38
+# @Author : HaoJun Qin
+# @Email  : 18697951462@163.com
+
+r"""
+SimpleX
+################################################
+
+Reference:
+    Kelong Mao et al. "SimpleX: A Simple and Strong Baseline for Collaborative Filtering." in CIKM 2021.
+
+Reference code:
+    https://github.com/xue-pai/TwoToweRS
+"""
+import torch
+from torch import nn
+import torch.nn.functional as F
+
+from recbole.model.init import xavier_normal_initialization
+from recbole.model.abstract_recommender import GeneralRecommender
+from recbole.model.loss import EmbLoss
+from recbole.utils import InputType
+
+
+class SimpleX(GeneralRecommender):
+    r"""SimpleX is a simple, unified collaborative filtering model.
+
+    SimpleX presents a simple and easy-to-understand model. Its advantage lies 
+    in its loss function, which uses a larger number of negative samples and 
+    sets a threshold to filter out less informative samples, it also uses 
+    relative weights to control the balance of positive-sample loss 
+    and negative-sample loss.
+
+    We implement the model following the original author with a pairwise training mode.
+    """
+    input_type = InputType.PAIRWISE
+
+    def __init__(self, config, dataset):
+        super(SimpleX, self).__init__(config, dataset)
+
+        # Get user transaction history
+        self.history_item_id, _, self.history_item_len = dataset.history_item_matrix()
+        self.history_item_id = self.history_item_id.to(self.device)
+        self.history_item_len = self.history_item_len.to(self.device)
+
+        # load parameters info
+        self.embedding_size = config['embedding_size']
+        self.margin = config['margin']
+        self.negative_weight = config['negative_weight']
+        self.gamma = config['gamma']
+        self.neg_seq_len = list(config['neg_sampling'].values())[0]
+        self.reg_weight = config['reg_weight']
+        self.aggregator = config['aggregator']
+        if self.aggregator not in ['mean', 'user_attention', 'self_attention']:
+            raise ValueError('aggregator must be mean, user_attention or self_attention')
+        self.history_len=min(config['history_len'],self.history_item_len.shape[0])
+        
+        # user embedding matrix
+        self.user_emb = nn.Embedding(self.n_users, self.embedding_size)
+        # item embedding matrix
+        self.item_emb = nn.Embedding(
+            self.n_items, self.embedding_size, padding_idx=0)
+        # feature space mapping matrix of user and item
+        self.UI_map = nn.Linear(self.embedding_size,
+                                self.embedding_size, bias=False)
+        if self.aggregator in ['user_attention', 'self_attention']:
+            self.W_k = nn.Sequential(nn.Linear(self.embedding_size, self.embedding_size),
+                                     nn.Tanh())
+            if self.aggregator == 'self_attention':
+                self.W_q = nn.Linear(self.embedding_size, 1, bias=False)
+        # dropout
+        self.dropout = nn.Dropout(0.1)
+        self.require_pow = config['require_pow']
+        # l2 regularization loss
+        self.reg_loss = EmbLoss()
+
+        # parameters initialization
+        self.apply(xavier_normal_initialization)
+        # get the mask
+        self.item_emb.weight.data[0, :] = 0
+            
+    def get_UI_aggregation(self, user_e, history_item_e, history_len):
+        r"""Get the combined vector of user and historically interacted items
+
+        Args:
+            user_e (torch.Tensor): User's feature vector, shape: [user_num, embedding_size]
+            history_item_e (torch.Tensor): History item's feature vector, 
+                shape: [user_num, max_history_len, embedding_size]
+            history_len (torch.Tensor): User's history length, shape: [user_num]
+
+        Returns:
+            torch.Tensor: Combined vector of user and item sequences, shape: [user_num, embedding_size]
+        """
+        if self.aggregator == 'mean':
+            pos_item_sum = history_item_e.sum(dim=1)
+            # [user_num, embedding_size]
+            out = pos_item_sum / (history_len + 1.e-10).unsqueeze(1)
+        elif self.aggregator in ['user_attention', 'self_attention']:
+            # [user_num, max_history_len, embedding_size]
+            key = self.W_k(history_item_e)
+            if self.aggregator == 'user_attention':
+                # [user_num, max_history_len]
+                attention = torch.matmul(key, user_e.unsqueeze(2)).squeeze(2)
+            elif self.aggregator == 'self_attention':
+                # [user_num, max_history_len]
+                attention = self.W_q(key).squeeze(2)
+            e_attention = torch.exp(attention)
+            mask = (history_item_e.sum(dim=-1) != 0).int()
+            e_attention = e_attention*mask
+            # [user_num, max_history_len]
+            attention_weight = e_attention / (e_attention.sum(dim=1, keepdim=True)+1.e-10)
+            # [user_num, embedding_size]
+            out = torch.matmul(attention_weight.unsqueeze(1), history_item_e).squeeze(1)
+        # Combined vector of user and item sequences
+        out = self.UI_map(out)
+        g = self.gamma
+        UI_aggregation_e = g*user_e+(1-g)*out
+        return UI_aggregation_e
+
+    def get_cos(self, user_e, item_e):
+        r"""Get the cosine similarity between user and item
+
+        Args: 
+            user_e (torch.Tensor): User's feature vector, shape: [user_num, embedding_size]
+            item_e (torch.Tensor): Item's feature vector, 
+                shape: [user_num, item_num, embedding_size]
+
+        Returns:
+            torch.Tensor: Cosine similarity between user and item, shape: [user_num, item_num]
+        """
+        user_e = F.normalize(user_e, dim=1)
+        # [user_num, embedding_size, 1]
+        user_e = user_e.unsqueeze(2)
+        item_e = F.normalize(item_e, dim=2)
+        UI_cos = torch.matmul(item_e, user_e)
+        return UI_cos.squeeze(2)
+
+    def forward(self, user, pos_item, history_item, history_len, neg_item_seq):
+        r"""Get the loss
+
+        Args:
+            user (torch.Tensor): User's id, shape: [user_num]
+            pos_item (torch.Tensor): Positive item's id, shape: [user_num]
+            history_item (torch.Tensor): Id of historty item, shape: [user_num, max_history_len]
+            history_len (torch.Tensor): History item's length, shape: [user_num]
+            neg_item_seq (torch.Tensor): Negative item seq's id, shape: [user_num, neg_seq_len]
+
+        Returns:
+            torch.Tensor: Loss, shape: []
+        """
+        # [user_num, embedding_size]
+        user_e = self.user_emb(user)
+        # [user_num, embedding_size]
+        pos_item_e = self.item_emb(pos_item)
+        # [user_num, max_history_len, embedding_size]
+        history_item_e = self.item_emb(history_item)
+        # [nuser_num, neg_seq_len, embedding_size]
+        neg_item_seq_e = self.item_emb(neg_item_seq)
+
+        # [user_num, embedding_size]
+        UI_aggregation_e = self.get_UI_aggregation(user_e, history_item_e, history_len)
+        UI_aggregation_e = self.dropout(UI_aggregation_e)
+
+        pos_cos = self.get_cos(UI_aggregation_e, pos_item_e.unsqueeze(1))
+        neg_cos = self.get_cos(UI_aggregation_e, neg_item_seq_e)
+
+        # CCL loss
+        pos_loss = torch.relu(1-pos_cos)
+        neg_loss = torch.relu(neg_cos-self.margin)
+        neg_loss = neg_loss.mean(1, keepdim=True)*self.negative_weight
+        CCL_loss = (pos_loss+neg_loss).mean()
+
+        # l2 regularization loss
+        reg_loss = self.reg_loss(
+            user_e, pos_item_e, history_item_e, neg_item_seq_e, require_pow=self.require_pow)
+
+        loss = CCL_loss+self.reg_weight*reg_loss.sum()
+        return loss
+
+    def calculate_loss(self, interaction):
+        r"""Data processing and call function forward(), return loss
+
+        To use SimpleX, a user must have a historical transaction record, 
+        a pos item and a sequence of neg items. Based on the RecBole 
+        framework, the data in the interaction object is ordered, so 
+        we can get the data quickly.
+        """
+        user = interaction[self.USER_ID]
+        pos_item = interaction[self.ITEM_ID]
+        neg_item = interaction[self.NEG_ITEM_ID]
+
+        # get the sequence of neg items
+        neg_item_seq = neg_item.reshape((self.neg_seq_len, -1))
+        neg_item_seq = neg_item_seq.T
+        user_number = int(len(user)/self.neg_seq_len)
+        # user's id
+        user = user[0:user_number]
+        # historical transaction record
+        history_item = self.history_item_id[user]
+        history_item = history_item[:, :self.history_len]
+        # positive item's id
+        pos_item = pos_item[0:user_number]
+        # history_len
+        history_len = self.history_item_len[user]
+        history_len = torch.minimum(
+            history_len, torch.zeros(1, device=self.device)+self.history_len)
+
+        loss = self.forward(user, pos_item, history_item, history_len, neg_item_seq)
+        return loss
+
+    def predict(self, interaction):
+        user = interaction[self.USER_ID]
+        history_item = self.history_item_id[user]
+        history_item = history_item[:, :self.history_len]
+        history_len = self.history_item_len[user]
+        history_len = torch.minimum(history_len, torch.zeros(
+            1, device=self.device)+self.history_len)
+        test_item = interaction[self.ITEM_ID]
+
+        # [user_num, embedding_size]
+        user_e = self.user_emb(user)
+        # [user_num, embedding_size]
+        test_item_e = self.item_emb(test_item)
+        # [user_num, max_history_len, embedding_size]
+        history_item_e = self.item_emb(history_item)
+
+        # [user_num, embedding_size]
+        UI_aggregation_e = self.get_UI_aggregation(user_e, history_item_e, history_len)
+
+        UI_cos = self.get_cos(UI_aggregation_e, test_item_e.unsqueeze(1))
+        return UI_cos.squeeze(1)
+
+    def full_sort_predict(self, interaction):
+        user = interaction[self.USER_ID]
+        history_item = self.history_item_id[user]
+        history_item = history_item[:, :self.history_len]
+        history_len = self.history_item_len[user]
+        history_len = torch.minimum(
+            history_len, torch.zeros(1, device=self.device)+self.history_len)
+
+        # [user_num, embedding_size]
+        user_e = self.user_emb(user)
+        # [user_num, max_history_len, embedding_size]
+        history_item_e = self.item_emb(history_item)
+
+        # [user_num, embedding_size]
+        UI_aggregation_e = self.get_UI_aggregation(user_e, history_item_e, history_len)
+
+        UI_aggregation_e = F.normalize(UI_aggregation_e, dim=1)
+        all_item_emb = self.item_emb.weight
+        all_item_emb = F.normalize(all_item_emb, dim=1)
+        UI_cos = torch.matmul(UI_aggregation_e, all_item_emb.T)
+        return UI_cos
diff --git a/recbole/properties/model/SimpleX.yaml b/recbole/properties/model/SimpleX.yaml
new file mode 100644
index 000000000..19e4eca83
--- /dev/null
+++ b/recbole/properties/model/SimpleX.yaml
@@ -0,0 +1,7 @@
+embedding_size: 64
+margin: 0.5
+negative_weight: 10
+gamma: 0.5
+aggregator: 'mean'
+history_len: 50
+reg_weight: 1e-05
\ No newline at end of file


Test patch:
diff --git a/tests/model/test_model_auto.py b/tests/model/test_model_auto.py
index 619849a5d..aa5459236 100644
--- a/tests/model/test_model_auto.py
+++ b/tests/model/test_model_auto.py
@@ -222,6 +222,27 @@ def test_ADMMSLIM(self):
         }
         quick_test(config_dict)
 
+    def test_SimpleX_with_mean(self):
+        config_dict = {
+            'model': 'SimpleX',
+            'aggregator': 'mean'
+        }
+        quick_test(config_dict)
+
+    def test_SimpleX_with_user_attention(self):
+        config_dict = {
+            'model': 'SimpleX',
+            'aggregator': 'user_attention'
+        }
+        quick_test(config_dict)
+
+    def test_SimpleX_with_self_attention(self):
+        config_dict = {
+            'model': 'SimpleX',
+            'aggregator': 'self_attention'
+        }
+        quick_test(config_dict)
+
     def test_NCEPLRec(self):
         config_dict = {
             'model': 'NCEPLRec',