## 3 The ALIEN Architecture
### Model Description
#### 3.4.1 Attributes-Driven Latent Information Extraction Layer (USER \(\times\) CANDIDATE)  
In terms of building a comprehensive user profile, making good use of user and item attributes is an indispensable task. However, issues such as the enormous sparsity and high dimensions of one-hot feature vectors hinder the recommender systems from precisely modeling user's characteristics. In addition, extracting local dependencies and hierarchical structures among fields has been the challenging work. [PERSON] et al. [32] tried to solve these issues by applying a product layer to capturefeature interactions from multi-field categorical data after the embedding layer, but the model has the limitation that it only deals with 1 and 2-order feature interactions, and lacks the ability to model higher-order feature interactions, i.e., the 3-order and higher ones. However, the complexity of n-order inner product grows exponentially with the order of interactions, which results from the exhaustive calculations of inter-fields feature interactions. To solve this issue, we follow [32] and made the following improvements in the attributes-driven latent information extraction layer (USER \(\times\) CANDIDATE):  
* We introduce a compressed interaction network (CIN) [33] to model higher order feature interactions, which solves the high complex and time-consuming issue by compressing the high-order interaction vectors to a fixed value. * Taking both low and high-order feature interactions into consideration, we utilize synthesizer [37] to interpret the contribution of each feature interaction to the candidate item. To our best knowledge, our model is the first one proposed to apply synthesizer in the field of recommender systems. The graph inside the dark-green box in Figure 1 illustrates the work flow of the attributes-driven latent information extraction layer (USER \(\times\) CANDIDATE). In the following paragraphs, we state the functionality and the working procedure of each component in the attributes-driven latent information extraction layer (USER \(\times\) CANDIDATE). _The generation of feature interactions._ Let \(\mathbf{f}^{u,v_{c}}=[\mathbf{u}^{(1)},...,\mathbf{u}^{(m)},\mathbf{v}_{c}^{(1)},...,\mathbf{v}_ {c}^{(n)}]\triangleq[f_{1}^{u,v_{c}},f_{2}^{u,v_{c}},...,f_{m+n}^{u,v_{c}}]\) be the concatenation of \(\mathbf{u}\) and \(\mathbf{v}_{c}\). There are two procedures of generating feature interactions, i.e., the generations of (1) low-order feature interactions, and (2) high-order feature interactions. We define low-order as 1-order and 2-order. The representation of 1-order feature interactions \(\mathbf{\mathcal{F}}_{(1)}^{u,v_{c}}=[f_{1}^{u,v_{c}},f_{2}^{u,v_{c}},...,f_{m+n}^ {u,v_{c}}]\) is the concatenation of products between \(f_{i}^{u,v_{c}}\in\mathbf{\mathcal{F}}_{(1)}^{u,v_{c}}\) and the constant signal \"1.\" For 2-order feature interactions, there are \(\dfrac{(m+n)(m+n-1)}{2}\) 2-order feature interactions. The representation of 2-order feature interactions \(\mathbf{\mathcal{F}}_{(2)}^{u,v_{c}}\) is defined as:  
\[\mathbf{\mathcal{F}}_{(2)}^{u,v_{c}}=[f_{1}^{u,v_{c}}\circ f_{2}^{u,v_{c}},f_{1}^{ u,v_{c}}\circ f_{3}^{u,v_{c}},...,f_{i}^{u,v_{c}}\circ f_{j}^{u,v_{c}},...,f_{m+n-1}^ {u,v_{c}}\circ f_{m+n}^{u,v_{c}}], \tag{2}\]  
where \(\circ\) denotes the Hadamard product, and \(i\in[1,m+n-1],j\in[i+1,m+n]\). In addition to low-order feature interactions, we utilize a compressed interaction network (CIN) approach to generate higher-order feature interactions, whose order is defined to be higher than two in this paper. In CIN, there are multiple layers, each of which has different feature vectors. The \(h\)-th feature vector of the \(l\)-th layer in CIN is:  
\[\mathbf{X}_{h}^{l}=\sum_{i=1}^{H_{l-1}}\sum_{j=1}^{m+n}\mathbf{w}_{ij}^{l,h}(\mathbf{X}_{i }^{l-1}\circ\mathbf{X}_{j}^{0}), \tag{3}\]  
where \(1\leq h\leq H_{l}\) and \(\mathbf{X}_{j}^{0}=f_{j}^{u,v_{c}}\); and \(\mathbf{w}_{ij}^{l,h}\in\mathbb{R}^{H_{l-1}\times(m+n)}\) is the parameter matrix for the \(h\)-th feature vector.