# Improving Time Study Methods Using Deep Learning-Based Action Segmentation Models
## 2 Related Work
The HASR framework was developed in [27] to refine predictions from models such as MS-TCN, establishing a hierarchy of features at both segment and video levels and refining the output with additional models like the Gated Recurrent Unit (GRU). [PERSON] et al. [28] addressed the limitations of existing approaches in processing very short activities and proposed an encoder-decoder architecture, with MS-TCN as the encoder and LSTM or biLSTM as the decoder. Transformers have received much attention regarding their capabilities in solving sequence-to-sequence tasks related to natural language processing. This motivated research into their application in action segmentation because of the apparent similarity between these two problems. [PERSON] et al. [29] proposed a Transformer model for action segmentation, adapting the original architecture to video data. They restricted the attention mechanism to local neighborhoods and re-engineered the decoder to learn temporal relations between segments. Paper [30] introduced a similar idea, combining a 1D convolution and attention mechanism in an encoder-decoder architecture. The authors of paper [31] suggested an alternative approach to action segmentation, advocating for direct segment prediction using an autoregressive method akin to natural language translation tasks. Their model was grounded in a modified Transformer architecture. A model entirely based on the Transformer architecture was proposed in paper [32], and its structure resembled a U-net network used in image segmentation, designed to build a feature hierarchy in the encoder and decoder. This study utilized local attention limited to the neighborhood of each frame. In the context of manufacturing, a limited number of papers were found, with notable works being [6; 7; 33; 34; 35; 36; 37] The application of classic machine learning models was predominant in these papers, where models like Hidden Markov Models [6; 34; 35] or Support Vector Machines [7] were employed following the manual extraction of features. [PERSON] et al. [33] applied a deep learning model based on a 2D convolutional neural network and multi-layer perceptron, using manually created features with the Motion History Image algorithm. [PERSON] et al. [36] proposed an encoder-decoder framework for workflow recognition using a 3D CNN, transforming the activations of the last convolutional layer into clip-level representations, which were then fed into an LSTM network with an attention mechanism for enhanced recognition. In paper [37], a system comprising three stages was developed: spatial feature extraction using a Vectors Assembly Graph (VAG) and graph networks from RGB-D video frames; contact force feature extraction via a sliding window technique; and action segmentation through a multi-stage temporal convolution network (MS-TCN) that combines these features. [PERSON] et al. [7] collected data in laboratory conditions, while studies [33; 34; 35] were conducted using a dataset described in [38], which is no longer publicly available. [PERSON] et al. [6] used a dataset from [39], collected with a depth sensor for painting manufactured parts, recording data over a single workday with two different workers. The XIOLIFT dataset was utilized in [36], capturing data from six production lines in an elevator factory over one working month. A newly created dataset of 11 assembly actions performed in laboratory conditions, featuring RGB-D videos and contact force data, was used in paper [37]. The need for further research in this domain is evident, particularly in applying deep learning algorithms to real manufacturing data without manual feature engineering. This gap in research, especially in manufacturing contexts, highlights the potential of deep learning models for the simultaneous recognition and temporal segmentation of human activities, a task still underexplored in various human endeavors, including manufacturing.