# Human Activity Recognition for Assisted Living Based on Scene Understanding
## 6 Design and Implementation
### Thread Experiments
#### 6.2.1 Configuration 1: Single Thread for Acquisition, Object Detection, and Activity Detection  
All components except MQTTConnection run on a single thread. The entire processing pipeline runs sequentially: first the image is acquired, then object detection is performed,  
Figure 14: Asynchronous communication between the HAR module and the SmartCare system. Figure 13: Synchronous communication between the HAR module and the SmartCare system. and finally the HOIs are detected and used to determine the activity performed by the patient. Figure 15 illustrates the single-thread configuration block diagram. An example of a sequential pipeline timeline is presented in Figure 16: the total processing time is the sum of object detection time and the time needed to detect HOIs. The average frame processing time is t1 + t2; therefore, the result for the third frame is available at T + 3 \(\times\) (t1 + t2). t1 and t2 are not real measurements; for Figures 16-18, they were chosen to ideally exemplify the tested configurations. 2.2 Configuration 2: One Thread for Acquisition and Object Detection, Another One for Activity Detection  
In this test scenario, we follow the general diagram from Figure 10: one thread is used for image acquisition and object detection, results go through first queue to the HOI detector which writes the output to the second queue in order to be further processed and extract the patient activity. Figure 17 illustrates the result timeline: In addition to the first iteration, which is always affected by the initialization process and is taken out of the time statistics, each time the HOI detection thread finishes the current frame processing, the queue will feed the object detections for the next frame. The average processing time is, in fact, the inference time for HOI detection (t2), the result for the 3 rd frame is available at T + t1 + 3 \(\times\) t2. Figure 16: Sequential configuration. Figure 15: Sequential configuration: single thread for acquisition, object detection and activity detection. 2.3 Configuration 3: One Thread for Acquisition and Object Detection, Two Threads for Activity Detection  
To the previous scenario we add a second thread for HOI detection: ideally, as exemplified in Figure 18, the run-time for image processing should remain the same even if multiple threads are used. In practice, to run multiple instances of neural networks, there should be multiple GPUs. To satisfy this constraint, multiple logical video cards were created by dividing the physical video memory by the number of threads. Figure 17: Configuration using two threads. Figure 18: Configuration using three threads. The advantage of using a single thread for acquisition, object detection and HOI detection is that the time elapsed between frame acquisition and activity detection is the shortest of the three configurations, representing mostly the inference time of the two neural networks (Figure 16). In the second and third configurations, the time elapsed between frame acquisition and activity detection is longer, and a delay is added due to unavailable threads for HOI detection (Figures 17 and 18). An advantage of using dedicated threads in the following configuration: one for acquisition and object detection and one/two for HOI detection, is that the FPS is higher. Table 7 presents a comparison between different scenarios regarding used threads and queue sizes. Increasing the queue size causes a longer time elapsed between frame acquisition and activity result. In order to have the smallest delay, the queue size is set to be equal to the number of threads used for human-object interaction. Ideally, increasing the number of threads should improve FPS, but, because the memory of the video card is split into multiple virtual GPUs, the processing speed decreases considerably, so that the configuration with two threads for HOI detection is actually slower than the configuration that uses only one thread for this process.