# Fully Open-Source Meeting Minutes Generation Tool
## 5 Evaluations
### Performance Comparison with Other Meeting Minutes Systems  
This experiment evaluates the performance of our proposed system by comparing it with other existing systems. Due to the proprietary nature of these systems, data were manually gathered from free or trial versions. Thus, the execution time, meeting transcription, summary generation, keyword extraction, interface, and customization capabilities were compared to demonstrate the effectiveness of integrating open-source models relative to professionally developed systems. The key metrics in this comparison included _WER_ for transcription accuracy, _ROUGE-N_ scores on the summary quality, and the _cosine similarity_ for keyword extraction relevance. Table 12 shows the comparison results.  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} \\  & & & & & & & & & & \\ \hline Slide 1 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 2 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 3 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 4 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 5 & 0\% & - & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ \hline \hline \end{tabular}
\end{table}
Table 11: _CER_ results from each slide and each topic from resized video using _PaddleOCR_.  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} & \multirow{2}{*}{**Topic**} \\  & & & & & & & & & & \\ \hline Slide 1 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 2 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 3 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 4 & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ Slide 5 & 0\% & - & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% & 0\% \\ \hline \hline \end{tabular}
\end{table}
Table 12: Performance comparison with other existing systems.  
In Table 12, the bolded values represent the highest scores, while the underlined values denote the second-highest scores. These results indicate that the proposed system achieved the lowest _WER_ of 1.88%, outperforming other systems' transcription accuracy. While MeetingBooster had the fastest execution time, it only provided transcription. In contrast, our system handles transcriptions, summaries, and keywords, achieving the second-best time (1:34). For summarization, Piglyph achieved the highest _ROUGE-N_ metrics, with our system closely following with second-highest scores, 41.06% for _ROUGE-1_, 23.76% for _ROUGE-2_, and 32.12% for _ROUGE-L_. Piglyph led with a _cosine similarity_ of 0.521 in keyword extraction, while our system achieved 0.509 keyword relevance. The browser-based design offers easy access, and the system's use of customizable open-source models enables model updates, ensuring adaptability to the latest advancements.  
Although our system does not consistently achieve the highest scores, its performance closely aligns with professionally developed systems. The system performs best with simple presentation formats, and further testing is needed to confirm its effectiveness with more complex layouts, diverse speaker styles, and varied slide designs.