# Using LSTM to Identify Help Needs in Primary School Scratch Students
## 2 Materials and Methods
### Help Identification Model Development
#### 2.4.2 Model Construction  
For the construction and training of the model, the dataset in the correlation matrix shown in Figure 10 was used. Since we are looking for the model to predict whether the student needs help or not, we use the variable \"request_help\" as the target variable, and we look for the output of the model to be a boolean value.  
Given that all the student actions are recorded over time and reflect when a student needs help, the model implementation is based on a RNN, making use of LSTM. Since we have many inputs (interactions in Scratch over time), and just one output (whether the student needs help or not), we applied a many-to-one architecture, as shown in Figure 4. In order to identify the architecture model that best fits the problem and the current data, we conducted some experiments with different parameters. In these experiments, we combined the number of LSTM layers (one or two LSTM layers), and none, or a few dropout layers (with a value of 0.5). Taking into account that the model must identify if a student needs help or not in a particular moment, we were faced with a binary problem, hence, we implemented a binary cross entropy loss function with an adam optimizer in each experiment. The different model architectures we tested are shown in Figures 12 and 13.  
The following parameters were used for the training of the different architectures:  
* LSTM units (in each LSTM layer): 256;
* Activation function of the LSTM layers: sigmoid;
* Return sequences: true;
* Dropout layer value (in each dropout layer if applicable): 0.5;
* Dense layer units: 1;
* Dense layer activation function: sigmoid;
* Dense layer dropout function: binary_crossentropy;
* Dense layer optimizer: adam;
* Dense layer metrics: binary_accuracy;
* Training batch size: 1;
* Number of training epochs: 50;
* Percentage of training data: 70%;
* Percentage of validation data: 30%.  
During the user experiences described in Section 2.2, we collected 11,790 interactions, taken from 82 students, who generated 695 help requests. In Table 1 we present the data collected.  
Because each user performs many interactions in a single exercise and day, grouping these interactions by user, date, and exercise, we finally obtained 316 time series. Therefore, we have an average of 2.19 help requests per time series, and 37.31 interactions per time series. Finally, for the training of our model we used 70% of the time series, and 30% for validation, as shown in Table 2.  
Finally, in order to analyze whether gender is a bias in the models, it was decided to train the four architectures shown in Figures 12 and 13 twice. In the first training run the dataset includes the gender of the students, while in the second run it does not.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Interactions** & **Help Requests** & **Students** & **Average Age** & **Girls** & **Boys** \\ \hline
11,790 & 695 & 82 & 13.3 & 30 & 52 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Data collected.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Phase** & **Time Series** & **Help Requests** & **Students** & **Average Age** & **Girls** & **Boys** \\ \hline Training & 221 & 450 & 52 & 13.5 & 16 & 36 \\ Validation & 95 & 245 & 30 & 13 & 14 & 16 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Dataset distribution.  
Figure 13: Architectures with 2 LSTMs layers. (**a**) 2 LSTMs layer with dropouts layer; (**b**) 2 LSTMs layer with no dropout layers.