# Improving Time Study Methods Using Deep Learning-Based Action Segmentation Models
## 5 Methodology
### Action Segmentation Models
The input gate \(\textbf{i}^{(t)}\in\mathbb{R}^{H}\) and the modulation gate \(\widetilde{\textbf{c}}^{(t)}\in\mathbb{R}^{H}\) collectively update the cell state with new information. The cell state update, as defined in Equation (5), combines past information with new insights, ensuring relevant information is retained. The output gate \(\textbf{o}^{(t)}\in\mathbb{R}^{H}\) decides which parts of the cell state contribute to the output at the current timestep \(t\). The hidden state \(\textbf{h}^{(t)}\in\mathbb{R}^{H}\), is then updated to reflect this output. The hyperparameters of biLSTM models were adjusted as shown in Table 1, with their ranges determined based on the literature and preliminary experiments. **1D CNN models:** The convolutional models used in our experiments were based on two types of dilated convolutional blocks, as identified in the research [11, 27]. The use of dilated 1D convolutional layers in the architecture allows for efficient modeling of temporal context. Dilation works on the principle of expanding the kernel by adding gaps between individual elements, thus capturing a longer temporal context without increasing the number of parameters, due to the expansion of the receptive field. The first type of block is the dilated residual block, which begins with a dilated convolutional layer with a kernel size of 3 (\(\widetilde{\textbf{W}}_{l}\in\mathbb{R}^{3\times H\times H}\)). The dilation factor for the layer \(l\) is set to \(2^{l}\), with the number of kernels \(H\) being a hyperparameter adjusted based on experimental results. This approach allows for the capturing of increasingly larger temporal contexts as the model deepens. This block also includes a non-linear transformation using the ReLU activation function, followed by a convolution with a kernel size of 1 (\(\textbf{W}_{l}\in\mathbb{R}^{1\times H\times H}\)) for controlling the model's representational capacity. A dropout layer is added to mitigate overfitting, along with a skip connection to facilitate the learning process. The operations within this block can be mathematically described as follows:  
\[\widetilde{\textbf{H}}_{l}=ReLU\bigg{(}\widetilde{\textbf{W}}_{l}*\textbf{H}_ {l-1}+\widetilde{\textbf{10}}_{l}^{T}\bigg{)} \tag{7}\]  
\[\textbf{H}_{l}=\textbf{H}_{l-1}+\textbf{W}_{l}*\widetilde{\textbf{H}}_{l}+ \textbf{10}_{l}^{T} \tag{8}\]  
In Equations (7) and (8) \(\textbf{H}_{l-1}\in\mathbb{R}^{T\times H}\) and \(\textbf{H}_{l}\in\mathbb{R}^{T\times H}\) are the input and output of dilated residual block \(l\); \(\widetilde{\textbf{b}}_{l}\in\mathbb{R}^{H}\) and \(\textbf{b}_{l}\in\mathbb{R}^{H}\) are bias terms; \(*\) represents the convolution operator; and **1** is a vector of ones of dimension \(T\). The second type of block is the dual dilated block, consisting of two convolutional branches. In one branch, the dilation factor of the convolutional layer increases with the depth of the model, similar to the dilated residual block. In the other branch, it decreases from the largest value to the smallest, according to the expression \(2^{L-l}\), where \(L\) is the depth of the last layer and \(l\) is the depth of the current layer. The goal is to simultaneously capture both short-term and long-term temporal contexts present in the data. Outputs from these two branches are merged and \begin{table}
\begin{tabular}{c c c} \hline \hline
**Label 1** & **Hyperparameters** & **Set of Values Used** \\ \hline M & Number of biLSTM layers & \{1, 2, 3\} \\ BiN & Number of neurons per biLSTM gate & \{4, 8, 16, 32, 64, 128, 256, 512, 1024\} \\ N & Number of additional fully connected layers & \{0, 1\} \\ FN & Number of neurons in additional fully connected layers & \{128, 256, 512\} \\ DR & Dropout rate & \{0, 0.1, 0.2, 0.5\} \\ \hline \hline \end{tabular}  
* Labels of hyperparameters used in Figure 11(a).  
\end{table} 
Table 1: Hyperparameters tuned in experiments with biLSTM architecture. passed through ReLU activation, with a convolution of dimension 1 used to adjust the dimensionality.