## 3 Materials and Methods
### Model Structure
Analogously, \(dt\) stands for a token coming from the definition of the target word and \(dte\) for a vector embedding of the said token. Additionally, \(p\) and \(q\) denote the length of the sentence and the length of the target word's definition measured in the number of tokens, respectively. Subsequently, the mean value of the sentence's token vectors on one side and the mean value of the definition's token vectors on the other are computed within the pooling layer. To the output of this layer, dropout is subsequently applied. On both sides, an output vector is then concatenated with the vector representation of the target word's tokens having undergone the same operations. In case of the cosine-similarity sub-model, an additional third vector representing similarity between the two respective vectors is also concatenated. In order to calculate the gap between these vectors, a multilayer perceptron is applied on the output of the concatenation function. The formulas for the hidden vectors obtained this way in SPV and MIP layers are presented below. * \(\mathit{hv}_{spv}=\Phi(\frac{1}{p}\sum_{i=1}^{p}te_{i},te_{tw})\);
* \(\mathit{hv}_{mip}=\Phi(\frac{1}{q}\sum_{j=1}^{q}\mathit{dte}_{j},te_{tw})\)  
where \(\Phi\) represents concatenation; \(p\) and \(q\) denote length of the sentence and length of the definition, respectively (measured in number of tokens); \(i\) is the index of the sentence's token such that \(i\in\mathbb{Z}\), \(p-1\geq i\geq 1\); and \(j\) is the index of the definition's token such that \(j\in\mathbb{Z}\), \(q-1\geq j\geq 1\). The hidden vector \(\mathit{hv}_{spv}\) represents a vector being the output of the SPV layer while the \(\mathit{hv}_{mip}\) is used to depict a hidden vector that is the output of the MIP layer. As mentioned, for the cosine-similarity sub-model, the similarity vector becomes the third element concatenated in order to obtain the aforementioned hidden vectors. Similarity vectors are obtained as follows:  
* \(similarity_{spv}=\frac{\frac{1}{2}\sum_{i=1}^{p}te_{i}te_{tw}}{max\left(\frac{ 1}{p}\sum_{i=1}^{p}te_{i}\right)\cdot\left\|te_{tw}\right\|_{2}\cdot\epsilon}\);
* \(similarity_{mip}=\frac{\frac{1}{2}\sum_{i=1}^{p}te_{i}te_{tw}}{max\left(\left\|te _{i}\sum_{j=1}^{p}te_{tw}\right\|_{2}\cdot\epsilon}\right)}\)  
where \(similarity_{spv}\) stands for the cosine similarity value measured between the average sentence vector and the target word vector; analogously \(similarity_{mip}\) represents the cosine similarity between the average definition vector and the target word vector. The \(\left\|\|\right\|_{2}\) denotes the Euclidean norm, the \(\cdot\) stands for the dot product between the vectors, and \(\epsilon\) is a parameter of a small value used to avoid division by zero ([[https://pytorch.org/docs/stable/generated/torch.nn.CosineSimilarity.html](https://pytorch.org/docs/stable/generated/torch.nn.CosineSimilarity.html)]([https://pytorch.org/docs/stable/generated/torch.nn.CosineSimilarity.html](https://pytorch.org/docs/stable/generated/torch.nn.CosineSimilarity.html)); last accessed on 21 December 2021). The output of the model is calculated by adding bias to the concatenation function that takes in the two hidden vectors and applies the log-softmax activation function onto the result. Finally, the candidate with the higher probability score is chosen as the predicted label. The process can be represented with the following formula:  
* \(y_{\tau}=\log(\sigma(W^{\top}\Phi(\mathit{hv}_{spv},\mathit{hv}_{mip})+b))\);
* \(\hat{y}=\text{argmax}(y_{\tau})\)  
where \(\hat{y}\) is the label predicted by the model, such that \(\hat{y}\in\{0,1\}\). This prediction is the result of the argmax operation applied on \(y_{\tau}\), which in turn stands for the natural logarithm of the value output by the softmax function denoted with \(\sigma\). Softmax outputs two values that are the probabilities for each class (literal and metaphor) that range from 0 to 1 and sum up to 1. \(W^{\top}\) denotes the weights matrix, \(\Phi\) stands for concatenation, and \(b\) signifies bias.