## 3 Prediction-Based Assembly Support System
### Assembly Assistance System  
The goal of the assembly assistance system (Figure 1) is to support operators to correctly learn the manual assembly process for a customizable tablet, without any human intervention (i.e., trainer). From a hardware point of view, the assembly assistance system is composed of five main components. The aluminum frame has embedded electrical motors that enable the height adjustment of the assembly table. The frame's aluminum profiles enable easy and flexible mounting of other relevant devices on the frame. The Sensytouch ST43 SLIM device is used to display information, to enable user interaction, and to run the training application and the required services (presented below). Because it is embedded in the frame as a tabletop and because it has a protective cover on the touchscreen, all the available surface can be used as working space (assembly and storage area, with the required assembly components laid on it). Its main characteristics are presented in Table 1.  
Figure 1: Assembly assistance system. The sensor for posture and facial expression detection is placed in front of the user. For this, a Microsoft Kinect Azure is utilized. Tobii's Pro 2 glasses are worn for eye tracking and gaze analysis, while a Shimmer sensor is mounted on a finger for galvanic skin response (GSR) readings. The sensor for object and hand movement is mounted on the upper part, above the table and hands. For this, Stereolabs' ZED 3D camera is used. The application's user interface is exemplified in Figure 2, highlighting the main areas for instruction, interaction (i.e., buttons for start, repeat, previous, and next), storage, as well as the location where to store the assembled components after each instruction. From a software perspective, the assembly assistance system is provided with micro-services to enhance user experience. All micro-services communicate via the gRPC framework. Additionally, each micro-service has its own Health Check and Service Discovery mechanisms. These mechanisms allow us to select which services will collaborate at any given time. Next, the micro-services which are relevant from the perspective of assembly prediction and user behaviour are presented. The height adjustment of the assembly station can be done manually, by pressing physical buttons on the station, or from software. The experiment revealed that the height of the subjects is one of the major factors that influence the assembly process. The depth camera streaming service allows the clients to control the depth camera as if they were connected to it. It exposes all the camera capabilities (RGB, depth, point cloud, etc.). The object detection service detects the position of the objects in each image. The object position service combines object detection and depth camera microservices to establish the 3D position of objects relative to the camera. Additionally, it detects if the objects have been inserted correctly in their slots. In case of a wrong step, it \begin{table}
\begin{tabular}{c c} \hline \hline
**Component** & **Characteristics** \\ \hline Display & 43-inch 4K touchscreen \\ CPU & Intel i7-7700 \\ GPU & NVIDIA GeForce GTX 1060 \\ RAM & 16 GB \\ SSD & 250 GB \\ Operating System & Windows 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Characteristics of Sensytouch ST43 SLIM. Figure 2: User interface displayed on the large touchscreen. will prompt the user to undo the action. This feature acts as a safeguard for the prediction service, since it cannot detect the incorrect assemblies, nor should this be its responsibility. In the future, this feature will be extracted in its own service. The emotion detection service relies on face mimics. This service detects the emotion of the user based on a picture of his face. Ongoing work is to be done to estimate emotion based on the data fusion from both mimic and GSR. The human characteristics collector service collects human characteristics such as age and gender. The mood can be identified with the aid of \"emotion detection based on face mimics\" microservice.