## 4 Proposed System Block Diagram and Novel Features
### Proposed System Block Diagram  
The proposed system provides two-way communication between DnM and NDnM people. The hardware block diagram is shown in Figure 4. Hand gestures from DnM people are acquired using a Leap device, and the data are then sent to the PC application. The leap device captures the hand gesture data as an image, which is then processed to obtain the recognizable gesture. The data then goes through several activities implemented in the software and will be discussed later. The data are finally converted into speech, which is generated through sound card speaker output. NDnM people can listen to the speech and then respond through the speech, which is then acquired using PC microphone input. The acquired audio is then displayed as text on the screen for the DnM person to read.  
Figure 3: Novel features of the system.  
Figure 5 shows the various software building blocks. The gesture data are acquired and processed by the Leap device. The processing is based on MobileNet to detect gesture data. The data are then fed to the PC software application, which is developed in LabVIEW. The data are first processed using a supervised machine-learning algorithm that detects the language. This is done using a dataset created as part of this research work. The LabVIEW application then converts this into text that the NDnM person can read. The last step is to convert the text into speech, which is generated through a PC sound card. This completed the communication from DnM to NDnM. The NDnM person records the speech using a PC sound card microphone input. Speech is acquired, and then language is detected using supervised machine learning. The detected language is then converted into text, which is finally displayed for the DnM person to read.  
Figure 6 shows the machine learning-based implementation through 8 steps. The system acquires hand gesture images using a Leap motion device, which is then processed, and the results are generated. The proposed system collects data that are used to improve system performance. The system also collects user information, which is very useful for improving the machine learning algorithm and implementation. Some parameters used for processing the image are listed in steps 2 and 3.  
In Figure 6, the process starts by providing relevant information to the machine learning algorithm through the input layer as shown by '1'. The processing is completed within the hidden layers and activities are marked by '2' till '7'. Step '2' is the initialization stage where the input image data goes through different tasks as listed. Step '3' is the processing stage where the image data is processed to detect certain features. Sign language dataset is detected in step '4'. The next step is to detect language. Currently there is only one language i.e., English as listed in step '5'. Data is stored in step '6' while the last step  
Figure 4: Hardware Interface Block Diagram—Communication between DnM and NDnM.  
Figure 5: Software Block Diagram—Communication between DnM and NDnM.  
i.e., '7' in the hidden layer is for user data collection. The results are generated through the output layer as marked by step '8'.  
Figure 7 shows a screenshot of the main software application. In this instance, the DnM person generated 5 gestures, i.e., 'H', 'E', 'L', 'L', and 'O', which are detected at a time. The software application created the word using these. Finally, the text is converted into audio and generated for NDnM people to listen to. In response, the NDnM person replied with the word 'GOOD', which is acquired through microphone input and converted from speech to text for the DnM person to read.  
Figure 6: Machine Learning-based Actual and Training System Implementation.  
Figure 7: Software Application Screenshot.