## 3 Materials and Methods
### Model Structure
In one of the sub-models, MIss WiLDe takes advantage of Sentence-BERT instead of RoBERTa. Figure 2: Input layer using RoBERTa. As single words are often divided into multiple tokens, most of the times the number of input words does not equal the number of output tokens. Context information is used to determine all the tokens that correspond to the target word. Specifically, similarity between the meaning of the sentence and the meaning of the target word is calculated within the SPV block, while similarity between the meaning of the target word's definition and the meaning of the target word itself is calculated within MIP. The input layer for this sub-model is common with the base variant visualized in Figure 2;
* MISs WiLDe_sbert. Since the results published in [41] suggest that using Sentence-BERT should result in acquiring sentence embeddings of better quality than those produced by both [CLS] tokens and averaged token vectors, we have decided to confirm it experimentally. We have therefore replaced RoBERTa with Sentence-BERT as an encoder in one of our 3 sub-models. The input layer using Sentence-BERT is depicted in Figure 3.  
The input to our model consists of the sentence comprising the target word on one side, and the definition of this target word on the other (depending on the part of speech and the availability of the definition in Wiktionary, lemma can be used instead of the definition; cf. Section 3.3 for the details). The conversion of words into tokens is then performed using the improved implementation of Byte-Pair Encoding (BPE), as proposed by [PERSON] et al. in [42] and used by [PERSON] et al. in [10] as well. This can be described as follows:  
* \(TOK(w_{1},w_{2},\cdots tw,\cdots,w_{m-1},w_{m})=t_{1},t_{2},\cdots t_{tw}, \cdots,t_{p-1},t_{p}\);
* \(TOK(dw_{1},dw_{2},\cdots,dw_{n-1},dw_{n})=dt_{1},dt_{2},\cdots,dt_{q-1},dt_{q}\)  
where \(TOK\) stands for the tokenizer, \(w\) represents a single word within the analyzed sentence, with \(tw\) being the target word or, to put it differently, a metaphor candidate; \(m\) in the subscript is the number of words in the input sentence. \(t\) represents an output token, while \(p\) is the number of the output tokens. Depending on a given target word \(tw\), \(t_{tw}\) should be considered an abbreviation for \(t_{tw_{1}},t_{tw_{2}},\cdots,t_{tw_{z-1}},t_{tw_{z}}\), where \(z\) stands for the number of tokens the target word was split into. This can be observed in Figure 2 as well. In the formulas presented in this section, we use the abbreviated forms for simplicity (a single input word is often transformed into multiple tokens: for more details on Byte-Pair Encoding cf. [[https://huggingface.co/docs/transformers/tokenizer_summary](https://huggingface.co/docs/transformers/tokenizer_summary)]([https://huggingface.co/docs/transformers/tokenizer_summary](https://huggingface.co/docs/transformers/tokenizer_summary)); last accessed on 21 December 2021). In the second formula, \(dw\) stands for a component word of the target word's definition and \(dt\) for an output token; \(n\) and \(q\) in the subscripts denote the number of words in the definition and the number of related output tokens, respectively. Afterwards, tokens are transformed into embedding vectors via the encoder layer. Input and output of our two encoders can be illustrated with the following formulas:  
* \(ENC(t_{1},t_{2},\cdots t_{tw},\cdots,t_{p-1},t_{p})=te_{1},te_{2},\cdots te_{tw },\cdots,te_{p-1},te_{p}\);
* \(ENC(dt_{1},dt_{2},\cdots,dt_{q-1},dt_{q})=dte_{1},dte_{2},\cdots,dte_{q-1},dte _{q}\)  
where \(ENC\) stands for the function producing contextualized vector representation for a given input, \(t\) represents a single token within the analyzed sentence, with \(tw\) in the subscript denoting the target word. \(te\) is the vector embedding representation that corresponds to the input token with the same index, while \(te_{tw}\) is the embedding representation of the target word's tokens.