## 3 Recommended Architecture for Industrial Software Service Components
### FM-Based Models for Service Recommendation
#### 4.2.3 Explicit and Implicit Higher-Order Feature Interactions
Figure 2 shows how the embedding layer compresses the input vectors into low-dimensional dense vectors to facilitate network training, the embedding set \(\mathbf{v}_{x}\) is input to the pairwise  
interaction layer, and a set of embedding vectors is mapped into a single vector by a pooling operation, as described by the following formula:  
\[f_{PI}(\mathbf{v}_{x})=\sum_{i=1}^{n}\sum_{j=i+1}^{n}\mathbf{v}_{i}x_{i}\odot x_ {j}\mathbf{v}_{x} \tag{12}\]  
In the model, the pair-wise interaction layer generates a k-dimensional vector to represent the second-order interaction relationships among different features in the embedding domain, conveying information about how these features influence each other. In the xDeepFM and AFM models, the input data for the attention layer and the hidden layer are not the same. The architecture of the EIAFM model is different from both in that it pools the vector outputs from the pair-wise interaction layer and utilizes the processed data as the joint input for both the attention layer and the hidden layer. This aids in the convergence of the model during training. Simultaneously, EIAFM employs sparse features as training data for the FM layer, and the output of the embedding layer serves as the input for the CIN network. Additionally, for the implicit component's output, \(Q_{L}\) is partially mapped to the result based on weights:  
\[f(x)=\mathbf{h}^{T}Q_{L}. \tag{13}\]  
In this context, the vector \(\mathbf{h}\) represents the weights associated with the implicit component prediction layer, and Formula (13) can be generalized as follows:  
\[f(\mathbf{x})=\mathbf{h}^{T}\sigma_{L}(\mathbf{W}_{L}(\cdots\sigma_{1}( \mathbf{W}_{1}f_{PI}(\mathbf{v}_{x})+\mathbf{b}_{1})\cdots)+\mathbf{b}_{L}) \tag{14}\]  
where L represents the count of hidden layers, \(\mathbf{W}_{L}\) represents the weight matrix, \(\mathbf{b}_{L}\) is the bias vector, and \(\sigma_{L}\) denotes the activation function for the \(i\)th layer. The parameters related to the implicit component in the model are represented by \(\Theta_{IC}=\{\mathbf{h},\{\mathbf{W}_{L},\mathbf{b}_{L}\}\}\), and are depicted by the blue link in Figure 2.  
Figure 2: The network architecture of the EIAFM model. In the attention component, the feature interactions are subjected to an attention mechanism in which the sum of the interacting vectors is weighted. The definition of the attention network follows the same structure as that described in Formula (4). The outcome of the pooling layer, which depends on the attention mechanism, is subsequently projected to be proportional to the predicted score. It corresponds to the term identified as the fifth in Formula (9), and the attention model's parameter set is \(\Theta_{AC}=\{\mathbf{p},\mathbf{W},\mathbf{b},\mathbf{h}\}\). For the explicit component, higher-order feature interactions are explicitly learned using the CIN network, which models the interactions at the vector level. The resulting dense embedding vectors are used as inputs to the CIN network; each layer of the CIN network is defined in a manner consistent with Formula (6), and the pooled vectors from each layer are concatenated to form the ultimate CIN output vector. This CIN output vector contributes to the prediction score and corresponds to the third term in Formula (9):  
\[g(\mathbf{x})=\mathbf{p}^{+}=\left[\mathbf{p}^{1},\mathbf{p}^{2},\dots, \mathbf{p}^{T}\right]\in\mathbb{R}\mathbb{R}^{\sum_{i=1}^{T}H_{i}} \tag{15}\]  
where \(T\) signifies the count of hidden layers in CIN, \(\mathbf{p}^{i}\) is the pooling vector of each layer, \(\mathbf{p}^{+}\) is the final output vector of the CIN, and the model parameters of the CIN network are \(\Theta_{EC}=\{\mathbf{W}_{cint},\mathbf{b}\}\). To summarize EIAFM, the overall parameters of the model are \(\Theta=\{\Theta_{LC},\{\mathbf{v}_{i}\}_{i=1}^{n},\Theta_{IC},\Theta_{AC}, \Theta_{EC}\}\), which contain the embedding vector. The problem of recommending industrial software service components can be regarded as a classification task.