## 3 Enabling the Enhancements of Language Features and Domain Knowledge
### Dealing with an Unordered Dependency Library Set  
#### 3.3.1 Processing Input Dependency Library Set  
After obtaining the dependency library representation that integrates certain domain information, EMGSCC focuses on dealing with the unordered set of input to obtain the representation of the project. Ordinary sequences can be encoded into vectors by recurrent neural networks. However, the principle that must be satisfied when the input is a set (which presumes order does not matter) is that swapping the two elements \(x_{i}\) and \(x_{j}\) in the set \(X\) should not change their encoding.  
Adopting a BoW model is an intuitive choice to vectorize each library based on a dictionary of all dependency libraries. In this case, each library can be represented by a one-hot vector of the form [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0]. By adding the corresponding components of all library vectors, we can get the dependency library vector of the project. However, the BoW model faces apparent drawbacks. One is that the dimension of the project representation will be quite high due to the large number of all dependency libraries. The second is that one-hot vectors are incapable of reflecting inherent relationships among libraries. A more sophisticated approach is to use word embeddings to compress each dependency library into a low-dimensional dense vector and map the neighboring libraries to adjacent vectors in space. Since the number of dependency libraries of each project is indeterminate, it is necessary to add and average the dependency vectors of the project according to the corresponding positions. However, this will lead to the unavoidable loss of different criticality between the already worked libraries and the remaining ones.  
In EMGSCC, we borrowed the method proposed by [PERSON] et al. [20] to integrate unordered and variable-sized inputs with the attention mechanism. The main idea of this method is to concatenate the output state of the previous time steps with the attention mechanism over the memory vector composed of inputs. To satisfy the critical property that the order of elements in a set is invariant, we employ the content-based attention for set to obtain the attention weights of each dependency library. An LSTM without inputs or outputs is adopted to continuously update its state by repeatedly reading the input vector \(v_{i}\) integrated by the attention. Afterward, we perform a weighted average of each dependency library vector according to the attention weights, resulting in an unordered input-friendly embedding. As shown in Figure 2, the basic processing steps are as follows:  
1. For each imported dependency library \(d_{i}\) of the project, a fixed-dimensional dependency representation vector \(v_{i}\) is obtained through the LibRepLayer with the identifier, tags, and description as the input.
2. We employ an LSTM to merely calculate the recurrent states: \[h_{t}=LSTM(h_{t-1}^{*})\] (2)
3. After obtaining the input representation vectors \(v_{1},\ldots,v_{n_{d}}\), each of them is considered as a memory vector; the attention weight \(a_{i}\) of each dependency library can be computed by: \[e_{t,i}=v^{e}\text{tanh}(W_{d}v_{i}+U_{d}h_{i})\] (3) \[a_{t,i}=\frac{exp(e_{t,i})}{\sum_{j=1}^{n_{d}}exp(e_{t,j})}\] (4) where \(v^{e}\), \(W_{d}\), and \(U_{d}\) are weight parameters of linear layers.
4. With attention weights, we can obtain the context vector \(c_{t}\). By concatenating the context vectors and hidden state recurrently, the last hidden state \(h_{t}^{*}\) is the target representation that is permutation-invariant to the inputs. \[c_{t}=\sum_{i=1}^{n_{d}}a_{t,i}v_{i}\] (5) \[h_{t}^{*}=[h_{t}\;c_{t}]\] (6)