# Normalization of Web of Science Institution Names Based on Deep Learning
## 3 Institutional Synonym Recognition Model
### Multi-Granularity Feature Embedding Layer  
**Character-level Feature Extraction:** Let \(c\) be the vocabulary of characters and \(d\) be the dimensionality of character embeddings. For each word a, its character sequence is denoted as \((c_{1},c_{2},...,c_{l})\), where \(l\) is the length of word a. The vector matrix representation of word a is denoted as \(C^{a}\in R^{d\times l}\). We use a convolution between \(C^{a}\) and multiple filters (or kernels) \(H\in R^{d\times w}\) of width \(w\). Char-CNN does the following:  
\[x^{a}=max(\sum_{0}^{l-w+1}tanh(\langle C^{a}[i:i+w-1],H\rangle+b)) \tag{1}\]  
where \(C^{a}[i:i+w-1]\) represents the \(i\) to \((i+w-1)\)th column of matrix \(C^{a}\), and \(\langle A,B\rangle\) is the Frobenius inner product. Filters essentially extract n-gram character sequences from words, where the size of the n-gram corresponds to the width of the filter. This represents taking the maximum value, which is used to capture the most important features for a filter. For a word, this study employs a total of m convolutional filters. The structure of Char-CNN is illustrated in Figure 2 below.  
Figure 1: Model architecture.  
**Word Embedding:** In this study, the pre-trained Word2 Vec and TF-IDF are utilized to obtain semantic embeddings for each word, with a length of n. We concatenate the character feature embeddings with the word semantic embeddings and denote the resulting representation as \(y^{a}=\left[y_{1}^{a},...,y_{n+m}^{a}\right]\), \(y\in R^{n+m}\). For \(y^{a}\), this study utilizes a Highway network to adjust the relative contributions of word semantic embeddings and character feature embeddings, thereby obtaining a more effective word representation. The Highway network employs a gating mechanism to control the flow and transformation of information, which is represented by the following equation:  
\[z=t\cdot g(W_{H}y+b_{H})+(1-t)\cdot y \tag{2}\]  
\[t=\sigma(W_{T}y+b_{T}) \tag{3}\]  
Let \(W\) represent the weight matrix and \(b\) denote the bias. The function \(g\) is a non-linear activation function, which can be either ReLU or Tanh. \(g(W_{H}y+b_{H})\) is responsible for modifying the input data, allowing for the network to adaptively choose the extent of the transformation applied to the input data, thereby enhancing the network's expressive power and adaptability. \(t\) represents the transformation gate, which determines the amount of information to be transmitted to the next step or bypassed entirely. It serves as a control mechanism for regulating the flow of information and adjusting the relative contribution of the input data. The structure of it is shown in Figure 3 below.  
**Contextual Embedding:** LSTM (Long Short-Term Memory) is a variant of recurrent neural networks (RNNs) that plays a crucial role in contextual encoding. It is capable of modeling sequential dependencies, storing and transmitting contextual information, handling variable-length sequences, and providing rich representational capacity. Therefore, we employ LSTM to encode the contextual information of entity mentions in organization names. To take into account the position of entities in the context, we utilize two LSTMs to encode both the forward and backward directions and halt after encountering the entity word beyond the context: \(h_{e}=\left\lceil LSTM^{\overrightarrow{t_{e}}},LSTM^{\overleftarrow{t_{e}}}\right\rceil\), where \(t_{e}\) represents the positional index of entity \(e\) in the context and \(h_{e}\in R^{1\times d_{HE}}\), and \(d_{HE}=forward\_hidden\_size+backward\_hidden\_size\).  
Figure 2: The structure of Char-CNN.