# VBF Event Classification with Recurrent Neural Networks at ATLAS's LHC Experiment
## 4 Architecture of the Model  
A DeepNN was designed for this binary classification task, a single standard neuron in the last layer is used to obtain one output score. Two hidden layers are based on recurrent cells. A total of 25 long short-term memory (LSTM) cells and a \(tanh\) activation function was used as the core of the architecture. Furthermore, the last outputs of the second RNN layers move into a sigmoid activation function-based single node that allows obtaining a final score. In order to reduce possible overfitting during training, 30% of the connections among inner layers are randomly truncated (this method is called dropout). Finally, the model was trained with Adam optimiser. The network was trained up to 200 epochs, reaching a stable plateau and showing a great goodness of the model with no sign of overtraining. Roughly, two hundred thousand signal events were simulated using Monte Carlo generators [13] and were used for the training. Figure 5 shows a simplified scheme of the architecture employed in this analysis, while Figure 6 shows details of the logic of an LSTM cell. This model was implemented and trained using the Keras package [14] with Theano [15] as backend for the mathematical computation. The architecture as well as the final weights were published through the HEPData of the ATLAS paper [16; 17], and they are available for reproducing the results in whatever theoretical or experimental framework needed. Figure 4: Schematic representation of an event reconstructed at the ATLAS experiment; many jets are present in the event since the LHC is a hadronic collider. The choice of using recurrent-based layers was mainly driven by a physics constraint: since the objects used as input of the network are the jets reconstructed in the event, they constitute a set with a natural variable length size. The recurrent layers represent one of the possible choices to deal with the inputs without any further data manipulation. In addition, the RNN architecture allows exploiting the inputs such that the jets are treated as a sequence of objects. Therefore, all the physical features that are used as inputs are treated as sequences. In this way, any hidden correlation among the sequence of objects is, in principle, caught. The hyperparameters used are the result of a simple optimisation; dedicated scans over the hyperparameters space were conducted to check the impact on the performance and on the stability of the training when the corner regions are explored. A set that prioritised the goodness of the model, quantified as the relative distance between the training and the test data sets and the optimal performance, was selected. In addition, the smoothness of the final score was considered to avoid big changes in the selection efficiencies when the cut is applied for the physics analysis. According to the RNN architectures, the maximum length of the sequences used as inputs can be modified. In some sense, it represents an additional hyperparameter that can be tuned. The performances were tested as a function of this parameter, and the physics considerations were accounted. The fraction of events with more than 5-6 jets is relatively  
Figure 5: Simplified view of the neural network used in this work; RNN layers are the core of this network that is built to have up to N jets as input and only one output node according to the binary classification aim. Figure 6: In\(-\)depth view of an LSTM cell that is the building block of an RNN layer. small, and the additional information coming from those events do not really improve the performances; on the other hand, as soon as the length of the sequence is too short, significant physics information is lost, and the performance breaks down. The optimal choice was found considering the full event topology; indeed, in addition to the two real VBF jets (if reconstructed), the other jets come from the boson decaying hadronically; the maximum number of jets is therefore optimised on this consideration.