# Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning
###### Abstract
However, the current approach to workforce assessment remains manual inspection and faces many challenges: first, manual data collection takes extra labor to observe, record and analyze the operation, which is not cost-effective due to high rising labor costs; second, close manual observation may result in an abnormal reflection of workers' performance caused by the Hawthorne Effect that the workers adjust their behaviors or productivity as a result of the awareness of being observed; third, the manual interpretation takes repetitive and random observations and requires a long period of time, and the results heavily rely on the supervisor's own experience [20; 22]. Current workforce assessments require time-consuming and labor-intensive manual observation. To address these limitations, efforts including different machine learning algo  
Figure 1: An example of a workforce assessment. rithms to automize this process have been made by scholars in the past few years [23; 24; 25]. [PERSON] et al. utilized both location information and the worker's body posture for automated activity analysis of the task level. Their method combines the data obtained from UWB for location tracking and the body posture data from a wearable 3-axial accelerometer [26]. However, this approach merely used one single location and body posture as the training sample to infer each activity category. The activity recognition between two activities including intra-class variation and inter-class similarity at the same location would be challenging. [PERSON], [PERSON] and [PERSON] proposed a method based on depth cameras for activity recognition and applied a bag of poses histogram and Hidden Markov Model algorithm to distinguish each activity [22]. However, depth cameras are sensitive to sunlight and a large body of construction activities are located outdoor. Thus, their method is feasible for interior construction operations but it may not be applicable to the external construction environment, such as scaffolding. To automate the workface assessment process, human activity recognition (HAR) performs a core and fundamental procedure in this research. Human activity recognition employs machines to comprehend and categorize a series of human activities from different data sources. Based on the type of data source, HAR can be divided into a sensor-based approach and a vision-based approach. The former exploits data from wearable sensors and other sensors. The sensor is attached to human limbs and other body parts or the surroundings close to the human activity to collect the target's activity continuously. Sensor-based HAR produces one-dimensional data, such as a time-series signal. Accelerometer, gyroscope and magnetometer are three common wearable sensors which detect and recognize the user's activities by analyzing the signal deviation when different activities are performed. Other sensors such as Radio Frequency Identification (RFID), Global Positioning System (GPS) and Ultra Wideband (UWB) also can be integrated into smartphones, wristbands, helmets or safety vests and provides a user's trajectory, absolute or relative locations to infer the user's activity. However, sensor-based HAR suffers from either high cost or sensitivity to the external environment [27; 28; 29]. For instance, a gyroscope is relatively expensive and needs to be attached to at least two key joints for HAR; an accelerometer is sensitive to temperature. In addition, most of these sensors need to be powered by a battery. The latter, vision-based approach, collects human activities with the help of visual devices, such as by video surveillance or a camera, and generates multi-dimensional data including 2D images, 3D images or video frames. Although this approach releases participants from wearing attached sensors and the image processing technique develops, it heavily relies on visual data quality [30; 31; 32]. Elements, including image resolution and illumination conditions, to some extent affect the robustness of recognition.