## 5 Assistant System Design  
### Overall Design  
In recent years, researchers have spent much time on recognizing sign language through machines. [PERSON] [31] used a camera to capture real-time video at 30 frames per second and then analyzed dynamic gestures frame by frame. By extracting skin regions as an auxiliary evaluation process, the system converts each frame into an HSV color space and filters out skin-color pixel regions. [PERSON] collected data using four cameras, each angled from the center to the \(-20^{\circ}\), \(-10^{\circ}\), \(10^{\circ}\), and \(20^{\circ}\) positions [32]. Angle changes can better capture the dynamic effect. However, in the actual scene, gestures will be affected by the surrounding environment, especially the background, such as the background and skin color close to the existence of dynamic objects. Some researchers make sensory gloves using flexible acceleration sensors and collect palm-movement data through three-axis acceleration sensors. [PERSON] [33] used a five-dimensional tracker that can track 11 finger features, including the flexion of the thumb, index finger, middle finger, ring finger, and little finger. The use of these flexible sensors is compassionate, and the recognition of motion is quick and accurate. However, because these sensor devices are installed in hand, they do not consider the impact of comfort on hand movement and the deviation brought by it. Therefore, it is necessary to use a device that can efficiently complete the background segmentation and naturally capture the hand joints' position to study gestures. At present, the technology in common use is to recognize sign language by a depth camera, e.g., Kinect and Leap Motion. To recognize sign language by depth camera is to emit infrared light through the infrared light source and receive the infrared light reflected by the environment and the user. By detecting the phase differences, we carry out 3D imaging of the user, and then recognize human contour and joint positioning. Users can define the basic actions as sign \begin{table}
\begin{tabular}{c c c} \hline \hline
**Requirements** & **Transformation** & **Function Design** \\ \hline Variety of interactive & Interactive content & Live captioning \\ lectures & presentation & Multiple definition of words \\  & & Sign language is translated in literal form \\ \hline Class interactions can & Interactive content & Classroom implementation records \\ be recorded & recording & Interactive content is saved in the cloud \\  & & Text interaction facilitates notetaking \\ \hline Teacher–student & & Remind when answering questions \\ interaction & & Sight and hearing draw attention to each other \\  & & Choose answers \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Specific transformation from needs to functional design. language words through the actions read by the depth camera. The depth camera is fixed above the display screen, and a user makes the required action or sign language in front of the display screen. At present, when using in the classroom of teaching college students with hearing impairment, the depth camera is fixed; thus, the range for recognition is limited, which affects the teaching quality. In addition, the current teaching system does not have the function of sending swift messages through simple actions, which is vital; thus, the existing gesture-recognition methods are not suitable in class. Based on these above needs, we propose the following design for a tracking-processing system for gesture recognition and representation, including a main terminal, subterminal, gesture sensor, PTZ, and main controller. 1. The main terminal is a teacher's computer, as an output unit, including a sign-language recognition module, display module, and voice module.