# Design of a Semantic Understanding System for Optical Staff Symbols
## 4 Results
### Experiment and Analysis  
#### 4.4.1 Experiment with LSUS  
In this paper, the type of symbols, the span of the interval, the density of symbols, the density of external notes, and the file size of staff images are defined as staff complexity variables. Table 3 presents statistics for the complexity properties of the test set of 10 track staves and calculates the precision and recall after LSUS. The following analysis is from three perspectives:  
(1) Evaluation metrics  
The average precision of the test set is 0.989, and the recall is 0.972. It is verified that the model has good generalization and robustness to the LSUS of staffs with different complexity. The recall of the model is lower than the precision for all staff images, which shows that the model misses a lot of symbols, especially the external note. For the semantic understanding of line notation, both missed and wrong checks affect the pitch and duration of the corresponding notes, especially the clef and key signatures that determine the pitch and duration of all the notes in a line. Table 4 shows the precision and recall of all the clefs and key signatures in the test set. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Precision** & **Recall** \\ \hline clef & 1.0 00 & 0.993 \\ key signature & 0.992 & 0.990 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Precision and recall of clef and key signatures. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multicolumn{2}{c}{**Staff**} & \multicolumn{5}{c}{**Complexity Variables**} & \multicolumn{3}{c}{**Evaluation**} \\ \hline \multirow{2}{*}{**Name**} & \multirow{2}{*}{**Page**} & \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Span**} & \multirow{2}{*}{**Density**} & \multirow{2}{*}{**Density**} & \multirow{2}{*}{**File Size**} & \multirow{2}{*}{**Precision**} & \multirow{2}{*}{**Recall**} \\  & & & & & **(Symbols)** & & & & \\ \hline Staff 1 & 2 & 16 & 19 & 484 & 78 & 1741 & 0.968 & 0.930 \\ Staff 2 & 5 & 17 & 19 & 679 & 146 & 2232 & 0.996 & 0.988 \\  
(2) Complexity variables  
By analyzing the complexity and evaluation in Table 3, we found that, in the test set of this paper, the main reasons for the error and omission of symbols in the LSUS process are as follows:  
* In Staff 1, many complex note beams along with the high density of symbols result in relatively high rates of error and omission, as shown in Figure 4a;
* Staff 2 has the highest density of symbols, and its recall is relatively low, as shown in Figure 4b;
* Staff 3 has a lower complexity for each item, and its performance evaluation is better;
* The error and omission of notes in Staff 4 are mostly concentrated in the notes with longer note stems, as shown in Figure 4c;
* Staff 5 has a higher complexity for each item and very low image file size (200 kb), and its evaluation is worse than others;
* Staff 6 has a lower image file size (435 kb) and, similar to Staff 1, its notes with common note beams are tedious, as shown in Figure 4d;
* Staff 7 has a lower image file size (835 kb), but its performance evaluation is better due to the lower complexities of other attributes;
* In Staff 9, the error detection notes are those located in the higher positive line on the staff, as shown in Figure 4e. (3) Correlation analysis  
The different complexity of staffs is a factor that affects the accuracy of symbol recognition. Using the Pearson correlation coefficient to calculate the correlation between the precision and recall of each of the complexity variables can eliminate the magnitude of the complexity variables and provide a more direct observation of the correlation between performance evaluation and complexity. The Pearson correlation coefficient is calculated as shown in Equation (8):  
\[\rho_{X,Y}=\frac{Cov(X,Y)}{\sigma(X)*\sigma(Y)} \tag{8}\]  
where, by calculating covariance \(Cov(X,Y)\), the strength of linear correlation between complexity variable \(X\) and the precision and recall \(Y\) is obtained.