## 4 Model Description
### Self-Attentive Subset Learning  
As mentioned before, to simulate the decision-making process of set rating, it is reasonable to select a subset of items to reduce the bias between set-level and item-level preferences [5]. Then, the set rating will be predicted based on the chosen subset. To choose subsets, we propose a _self-attentive policy network_ (SaPN). SaPN learns the personalized policy for how users select subsets from sets. Figure 3 illustrates the network structure. Note that we borrow the concept of the _policy network_ form reinforcement learning (RL), which takes the current state as input and outputs the probability distributions of all possible actions [32]. Intrinsically, rather than generating probabilities, SaPN learns to generate the selected subset directly based on the _Gumbel-softmax_ trick [33]. The SaPN consists of three parts: (1) an embedding layer that implicitly models user's preference and item attributes; (2) a self-attention layer that takes all items into consideration while choosing subsets; (3) a policy network layer that samples the chosen subset according to the reparameterization sample. The SaPN is an important part of SaSLM. This section will provide more details layer by layer. **Embedding layer. ** Given a user \(u\) and a set containing several items \(\left\{i_{j}\right\}_{j\in S^{\prime}}\), the embedding layer takes the one-hot representations as input. We assume \(\mathbf{u}\in\mathbb{R}^{n_{u}\times 1}\) is the one-hot vector of user \(u\), where only the element corresponding to \(u\) is one, and other elements are all zero. Similarly, we have the one-hot vectors of items, i.e., \(\mathbf{i}_{j}\in\mathbb{R}^{n_{i}\times 1}\). Through an embedding layer, we can retrieve the embeddings of users and items. The  
Figure 3: An illustrative example of the self-attentive policy network. It takes as input the user \(u\) and items in the set \(S=\left\{i_{1},i_{2},i_{3},i_{4},i_{5}\right\}\). It shows how the indicator variable \(x_{u1}^{s}\) is generated. embedding layer maintains two matrices, user matrix \(P^{\prime}\in\mathbb{R}^{n_{u}\times d}\) and item matrix \(Q^{\prime}\in\mathbb{R}^{n_{i}\times d}\), respectively. The output of the embedding layer is the embeddings of user \(u\) and item \(i_{j}\):  
\[\mathbf{p}_{u}^{\prime}=P^{\prime\top}\mathbf{u},\quad\mathbf{q}_{j}^{\prime}=Q^{\prime \top}\mathbf{i}_{j}.\]  
Note that for rating prediction, we also learn user and item matrices \(P\) and \(Q\). One possible way is to share the embeddings. However, to ensure that the learning of subsets is independent from the prediction of item-level ratings, we learn another set of embeddings, which we distinguish as \(P^{\prime}\) and \(Q^{\prime}\). **Self-attention layer. ** For each item, whether it is to be selected in the subset is determined in turn. However, when choosing the subset, it is important to take all items in the set into consideration. In order to capture correlations among contained items, we apply self-attention. Thus, by transforming item embeddings through self-attentive layer, when determining whether to select an item, other items in the set can be also considered. Aside from the correlations of other items, we also hope that user preferences can be considered when choosing the subset. To be more specific, given the same set, its contained item embeddings should be catered for different users. The intuition is that users have different levels of focus on the same set when they rate. Therefore, we first multiply \(\mathbf{q}_{j}^{\prime}\) with \(\mathbf{p}_{u}^{\prime}\) and then pack the item embeddings into a matrix, denoted by \(Z_{s}\in\mathbb{R}^{|S|\times d}\).