## 3 Methodology
### Long Short-Term Memory Neural Network Model for Inferring Users' Nationality  
#### 3.3.1 Model Description  
In the last few years, remarkable success has been achieved by applying recurrent neural networks (RNNs) to a variety of machine learning problems [34; 35]. Their chain-like structure is very suitable for sequences and lists, leading RNN to be particularly used in applications related to text, audio, and video data processing [36; 37; 38].  
RNNs are composed of a chain of repeating modules of neural networks, processing an input sequence one element at a time. Information flows through the network modules, influencing the output of the subsequent steps of the chain. The repeating RNN module receives two sources of input information about the present (current value of the data sequence) and information about the past (output value of the previous RNN module).  
LSTM is a complex type of RNN, responsible for many outstanding results in the field of speech recognition, language modeling, and translation [39; 40; 41]. Its repeating module is made of four different neural network layers, interacting in a particular way, as shown in Figure 1.  
Unlike standard RNNs, LSTM is characterized by the presence of the cell state \(C\), the vector containing the information used for executing the machine learning task (e.g., prediction or classification). At each step, the cell state is subjected to some interactions with structures called gates, made of a sigmoid neural network layer and a pointwise multiplication operation. Gates act on the inputs they receive, blocking or passing information on the basis of its strength and relevance, therefore optionally modifying (removing or adding) information in the cell state through their own sets of weights, adjusted via a backpropagation learning process.  
Equations (2)-(7) reports the formulas describing the functioning of LSTM. The first gate is called the forget gate layer (2) and defines what information to delete from the cell state. The second gate is named the input gate layer (3) and, before interacting with the cell state, is coupled with a tanh layer (4). They define what new information to store in the cell state: the input gate layer decides which values to update, and the tanh layer determines a vector of new candidate values to be added to the state. The cell state is therefore updated, combining the forgetting action and the updating action: the old cell state \(C_{t-1}\) is filtered by the forget gate layer \(f_{t}\), then the output of the combination between the input gate layer \(i_{t}\) and the tanh layer \(\overline{C}_{t}\) is added (5). The last gate is the output gate layer (6), which defines what parts of the cell state to output. The output is a filtered version of the cell state, resulting from the multiplication between the output gate layer \(o_{t}\) and the tanh of the new cell state \(C_{t}\) (7).  
\[f_{t}=\sigma(W_{f};[h_{t-1},x_{t}]+b_{f}); \tag{2}\]  
\[i_{t}=\sigma(W_{i};[h_{t-1},x_{t}]+b_{i}); \tag{3}\]  
\[\overline{C}_{t}=\tanh(W_{C};[h_{t-1},x_{t}]+b_{C}); \tag{4}\]  
\[C_{t}=f_{t}\star C_{t-1}+i_{t}\star\overline{C}_{t}; \tag{5}\]  
\[o_{t}=\sigma(W_{o};[h_{t-1},x_{t}]+b_{o}); \tag{6}\]  
\[h_{t}=o_{t}\star\tanh(C_{t}). \tag{7}\]  
Figure 1: Structure of the long short-term memory (LSTM) repeating module.  
Since these sequential operations occur at every step in the series, the cell state contains traces not only of the previous state, but also of all those that preceded \(C_{t-1}\).