## 5 Performance Evaluation  
For evaluating the performance of the proposed system, two types of evaluation are conducted: measuring the response time for generating the score and measuring its consistency in delivering both the transcribed text and the score. The performance evaluation method sends 100 requests to generate the transcribed text along with the score for each STT engine. Then, the time it takes from tapping the \"Upload\" button until the results appear on the screen is recorded in milliseconds as the response time. The shown transcribed text and score are also recorded to measure its consistency. Specifically for measuring consistency, the same file will be used in all 100 requests.  
For the response time, the average time from 100 requests for each STT is calculated along with the overall response time average from all STT engines. For consistency, each different transcribed text and score result will be counted as 1. Thus a bigger number means less consistency.  
The evaluation environment is carried out in an Android Emulator provided by Android Studio with a 5.7\" screen carrying 1080 \(\times\) 220 pixels, four processor cores, 1536 MB of RAM, 512 MB of internal storage, and Android 10. As for the internet connection, a ground internet via LAN cable provided by KT with download and upload speeds clocked around 824 Mbps and 540 Mbps, respectively, is used at the time of the test.  
The first test is conducted using the Google STT, and out of 100 requests, the average score is measured at 2232 ms or 2.23 s. The second test is carried out using Microsoft  
Figure 10: The Screenshot of the Application. (Note: The Korean at the top of these figures is the selected sentence that users require to speak, while the Korean at the bottom of these figures is the transcribed text from the STT engine).  
Azure STT, and out of 100 requests, the average score is calculated as a faster response time compared to Google STT, which is 1137 ms or 1.13 s. The subsequent measurement is done using IBM Watson. Out of 100 requests, it resulted in the slowest average time, which is 6088 ms or 6.08 s, resulting almost three times slower than Google STT and nearly six times slower compared to Microsoft Azure. The last response time test is done using Naver Clova, and out of 100 requests, the average response time is 5488 ms or 5.48 s. Although not as slow as IBM Watson, it is still considerably slower than Google STT and IBM. Therefore, the overall average response time of every STT combined is 3736 ms or 3.7 s. Because the transcribing process for each STT is done in the cloud using their respective server (every STT engine is a cloud-based service), the speed of their server plays a significant role in evaluating the performance of the mobile applications. Therefore, the response time could fluctuate from time to time, depending on the server's traffic. The response time graph is depicted in Figure 11.  
As for the consistency evaluation, Google STT, Microsoft Azure, and IBM Watson are consistent across 100 iterations scoring only 1, meaning that every request resulted in the exact transcribed text and score. However, for Naver Clova, out of 100 requests, it resulted in eight different results, indicating that Naver Clova is the least consistent among other STT engines tested in this paper. The consistency score is depicted in Figure 12.  
Figure 11: The Response Time Evaluation Graph of Every STT Engine.  
Figure 12: The Consistency Evaluation Graph.