# Optical Medieval Music Recognition Using Background Knowledge
## 5 Post-Processing Pipeline  
In this section, the post-processing pipeline is introduced. This pipeline uses the symbols extracted from the probability map. Using the probability map, each pixel in the image is labeled to the symbol class with the highest probability (excluding background). This results in a symbol label map. Next, the system decides whether there is background or a symbol. This is conducted in a separate step where the pixels are suppressed when the background class is above a certain threshold (\(p_{B}\)). The new filtered symbol label map is then used by a connected component analysis [17] to extract the symbols. As mentioned previously, the symbols are segmented using two different thresholds for \(p_{B}\): To extract the baseline symbols, the threshold is set to \(p_{B}=0.5\). To extract additional symbols, the threshold is set to \(p_{B}=0.9\) and filtered afterwards using the baseline symbols such that only the additional \"uncertain\" symbols remain. There are many False Positives (FP) in these uncertain symbols, but also False Negatives (FN) symbols that were not segmented with the lower threshold. These additional symbols are used in the post-processing steps to improve the recognition rate with the usage of background knowledge. In Figure 4, examples of issues the system tries to fix with the post-processing pipeline are shown. Our post-processing steps are described below:  
1. **Correct symbols in wrong layout blocks**: Sometimes FP are recognized inside incorrect layout blocks, such as drop capitals, lyrics, or paragraphs. Such symbols are removed from the baseline symbols and the uncertain symbols if they are inside drop capitals. If symbols appear in text regions, they are also removed if the vertical distance to the nearest staff line is greater than a constant \(d_{SL}\). 2. **Overlapping symbols:** In this post-processing step, symbols that overlap are removed. Since only the centers are recognized by our symbol recognition, the outline for each symbol depending on \(d_{SL}\) is calculated. Due to the fact that notes appear as squares in the original documents, a box around the center with a width and height of \(2\cdot(d_{SL}/4)\cdot 0.8\) can be drawn. Because clefs are bigger and also differ in height and width, this value is modified slightly. For C-Clefs, a box with a width of \(2\cdot(d_{SL}\cdot 0.3)\) and a height of \(2\cdot(d_{SL}\cdot 0.8)\) is drawn. For F-Clefs, a width of \(2\cdot(d_{SL}\cdot 0.4)\) and a height of \(2\cdot(d_{SL}\cdot 0.8)\) is used. If a clef overlaps a symbol, the clef is prioritized and the symbol is removed. If two symbols overlap each other, and they have the same PIS, one of them is removed. 3. **Position in staff of clefs**: In the datasets, all clefs appear to be only on top of a staff line and not in between. If the algorithm has detected that the PIS of a clef is in between two staff lines, then the clef is moved to the closest staff line. 4. **Missing clefs in the beginning of a staff line**: Normally, a staff always begins with a clef. If this is not the case, the system tries to fix it: 1. **Additional symbols**: If no clef is recognized by the algorithm at the beginning, it is checked if a clef has been recognized in the uncertain symbols. If so, this clef is added to the baseline symbols. 2. **Merging symbols**: The C and F-clef consists of two boxes. It can happen that a clef is mistaken for two notes. Therefore, two vertically stacked symbols that appear at the beginning of a staff are replaced by a clef. This is only performed when the position of the two symbols is right at the beginning, which means that the symbols are no more than \(2/3\cdot d_{SL}\) away from the start of the staff lines. 3. **Prior knowledge**: If the above steps did not help, a clef is then inserted at the beginning, based on the clefs of previous staves. This can lead to FP for the segmentation task, even if a clef has to be inserted at this position, because the exact center position can only be guessed.