# Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning
## 2 Literature Review  
The integration of high-resolution cameras, boosting capability of data storage and the availability of high-speed telecommunication over the past decade, has revealed rich content on the onsite construction operation. Today, cameras and other electronic sensors have been broadly applied for contractors and owners to monitor ongoing construction activities [33]. With the help of algorithms in computer vision, scholars have placed interest in research areas in construction workers' activity mainly including ergonomics posture, safety management and productivity improvement [34; 35]. This section reviews the state-of-the-art techniques for activity recognition and these application domains in the construction industry in which researchers have made great efforts. Thanks to the rapid development of sensor technologies in recent years, automatic HAR has received consistent attention and has been widely explored in many domains. As introduced in the first section, the technologies for HAR can be generally divided into a sensor-based approach and a vision-based approach. The sensor-based approach, based on sensor modality, can be classified into three types: body-worn sensors, environment sensors and object sensors. Body-worn sensors are often attached to the human body to detect the movement of the human body by continuously recording the signal data. Accelerometer, gyroscope and magnetometer are the three most frequently used body-worn sensors. Their applications focus on the activities of daily living (ADL) and sports. Environment sensors measure the medium that interacts between humans and the environment. For instance, pressure sensors, temperature sensors and sound sensors can measure their corresponding environmental parameters. Environment sensors can be utilized by monitoring hand movement and user's daily activities and they have been used in the field of the smart home. Object sensors are installed on objects which are close to human movements in order to infer human activity from the detection of object movement. For example, drinking activity can be detected by installing an accelerometer on a cup. A GPS or RFID module is attached to a worker's helmet to monitor the user's location to infer the worker's activity status [36; 37; 38]. [PERSON] et al. proposed to recognize worker's activities by using wristband Inertial Measurement Units (IMU) and conducted a case study of masonry work [39]. [PERSON], [PERSON] and [PERSON] assessed the reliability of wearable IMU and Electromyography (EMG) for HAR in construction [40]. [PERSON] et al. established an Artificial Neural Networks (ANN) based model for the classification of scaffolder's work by using the data retrieved from IMG and EMG sensors [41]. However, these studies face technical challenges in that the models were trained with at-rest activities and, as a result, merely standardized and repeated movements could be effectively identified and recognized and transitional signals or actions were excluded or could not be correctly distinguished. Our approach takes an image sequence of each activity as the input data, which includes both typical actions and transitional actions to address this limitation. According to the data type, the vision-based approach can be generally divided into RGB data and RGB-D data. RGB data refers to images consisting of red, green and blue color bands in the spectrum and RGB data can be collected through an ordinary camera or video surveillance. RGB-D data is produced by RGB-D cameras which can not only capture the original RGB data but also collect depth information. [PERSON] et al. [37] pointed out that RGB data has the advantage that it is extensively available as well as affordable and contains rich content on the subjects. While compared to RGB data, RGB-D data provides depth information, which can enhance the performance of HAR. The disadvantages of RGB-D data include computation complexity as well as high costs [42].