Source Code for Symbolic Music Alignment Tool
Eita Nakamura
2019/8/13

Contents
1 Licence and Remarks

12 How to Use
2.1 Compile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Run Alignment Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 Score-to-MIDI Alignment . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 MIDI-to-MIDI Alignment . . . . . . . . . . . . . . . . . . . . . . . .

222223 Package Content

34 File Formats
4.1 Spr File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Fmt3x File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 hmm File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Match File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Corresp File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

444455

5 Usage in Detail
5.1 Score-to-MIDI Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 MIDI-to-MIDI Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . .

556

1Licence and Remarks

The content of the source code is licensed under the MIT licence. Please read LICENCE.txt.
If you use this work for academic publications etc., we kindly ask to explicitly cite/mention
the following paper.
• Eita Nakamura, Kazuyoshi Yoshii, and Haruhiro Katayose, “Performance Error Detection and Post-Processing for Fast and Accurate Symbolic Music Alignment,” Proc.
18th International Society for Music Information Retrieval Conference (ISMIR), pp. 347–353, 2017.
Thank you for your understanding.

1

2

How to UseThe usage of the package is explained briefly here. Detailed explanations are given in later
sections.
2.1

CompileTo compile, execute
AAA$./compile.shor run commands equivalent to those in compile.sh with adaptations to the environment.
2.2

Run Alignment Algorithms

There are two main programs: one is for score-to-MIDI alignment (MusicXMLToMIDIAlign.sh)and the other is for MIDI-to-MIDI alignment (MIDIToMIDIAlign.sh). The example files
ex_ref.xml, ex_align1.mid, and ex_align2.mid can be used to test these programs.
2.2.1

Score-to-MIDI Alignment

To use the score-to-MIDI alignment program, execute
AAA$./MusicXMLToMIDIAlign.sh ex_ref ex_align1This produces an output file ex_align1_match.txt (called match file) and other files. Thematch file describes the result of alignment, which looks like:..... (Some comment lines)
0 0 0.852083 C#5 50 80 0 0 0 P1-1-1 0 0
1 0.00625 0.982292 E4 31 80 0 0 0 P1-1-6 0 .....
//Missing 16 P1-2-3
Each line in the first part describes a note in the aligned MIDI file ex_align1.mid. Fromleft to right, columns indicate:
ID (onset time) (offset time) (spelled pitch) (onset velocity)(offset velocity) channel (match status) (score time) (note ID)(error index) (skip index)
Error index is 0 for a correct note, 1 for a note with an erroneous pitch, and 3 for an extra
note. For an extra note, the given note ID is *. The note ID indicates a note in the reference
score MusicXML file ex_ref.xml: Px-y-z means the note is the z th note in the y th bar ofpart x.
Each line in the second part describes missing notes, notes in the score that do notappear in the alignment result. The number and the symbol indicate the score time andthe note ID of the missing note.
2.2.2

MIDI-to-MIDI Alignment

To use the MIDI-to-MIDI alignment program, execute
AAA$./MIDIToMIDIAlign.sh ex_align1 ex_align22

This produces an output file ex_align2_corresp.txt (called Corresp file) and other files.
The Corresp file describes the result of alignment, which looks like:..... (Some comment lines)
0 1.00833 C#5 73 36 0 0 C#5 73 50
1 1.025 A3 57 32 2 0.00625 A3 57 31.....
Each line describes from left to right:(ID) (onset time) (spelled pitch) (integer pitch) (onset velocity)of a note in the aligned MIDI ex_align2.mid followed by the same set of data of a note in
the reference MIDI ex_align1.mid. If ID for the reference MIDI is * it indicates an extra
note and if ID for the aligned MIDI is * it indicates a missing note.

3

Package Content

1. Score-to-MIDI alignment by hidden Markov model (HMM) (ScorePerfmMatcher)
2. Error detection (ErrorDetection)
3. Realignment by merged-output HMM (RealignmentMOHMM)
4. MIDI to Spr converter (midi2pianoroll)
5. MusicXML to Fmt3x converter (MusicXMLToFmt3x)
6. MusicXML to hmm converter (MusicXMLToHMM)
7. Spr to Fmt3x converter (SprToFmt3x)
8. Fmt3x to hmm converter (Fmt3xToHmm)
9. Match to Corresp converter (MatchToCorresp)
The first three programs are the main algorithms for alignment, and the rest are data
converters to supplement the alignment process. For score-to-MIDI alignment, an input
reference score file should be provided as a MusicXML file1 and MIDI files (standard MIDIfiles; SMFs) are used for the aligned signal and the reference signal for MIDI-to-MIDIalignment. The alignment algorithms deal with three internal file formats, Spr, Fmt3x, andhmm formats. The Spr file describes a list representation of a MIDI file, and the Fmt3x andhmm files describe the score information. For MIDI-to-MIDI alignment, the Spr file of thereference MIDI will be converted to Fmt3x and hmm files. The alignment result is describedwith the Match file format. For MIDI-to-MIDI alignment, the Corresp file format is alsoused, which can be produced from a Match file (and the Spr file of the reference MIDI) byMatchToCorresp. Please see Sec. 4 for details on file formats.
1

See https://www.musicxml.com .

34File Formats

The package deals with the following file formats: standard MIDI file (SMF) format, musicXML, Spr, Fmt3x, hmm, Match, Corresp format. A ‘MIDI file’ in this manual always
means an SMF. For musicXML, see https://www.musicxml.com. The rest formats are
original and explained below.
4.1

Spr File

The Spr file describes the note information in the MIDI file. The first lines look like thefollowing:..... (Some comment lines)
0 0.01 0.12 C4 19 80 0
1 0.02 0.15 C5 32 80 0.....
From left to right, columns in each line indicate:
ID (onset time) (offset time) (spelled pitch) (onset velocity)(offset velocity) channel

4.2

Fmt3x File

The fmt3x file describes the score information extracted from the musicXML file. The firstlines go as//TPQN: 24..... (Some comment lines)
4 3 2 1 0 0 chord 12 3 C#4 G4 B4 N.. N.. N.. P1-3-45 P1-3-46 P1-3-47
7 3 1 0 0 0 chord 3 1 C#6 N.. P1-3-26.....
The TPQN indicates the tick per quarter note, which defines the unit of score time. Each linein the main body indicates a score event. From left to right, columns indicate:(onset score time) (bar number) staff voice sub-voice order(event type) duration (number of notes)
followed by (spelled pitch), (note type), and (note ID) for each note. The note ID
indicates the part, bar, and the order of notes in the bar, given in the musicXML file. For
more information, see Fmt3x_v170225.hpp in the code folder.
4.3

hmm File

hmm files are used as reference score information in the alignment algorithm. The described
information is similar to the Fmt3x File.

44.4

Match File

The match file describe the result of alignment. The first lines go as..... (Some comment lines)
0 0.841 1.041 F#3 60 80 0 0 0 P1-1-33 0 0
1 0.864 0.950 G6 83 80 0 0 0 P1-1-1 0 .....
//Missing 330 P1-4-42.....
Each line in the first part describes a note in the aligned MIDI file as in the Spr file. Fromleft to right, columns indicate:
ID (onset time) (offset time) (spelled pitch) (onset velocity)(offset velocity) channel (match status) (score time) (note ID)(error index) (skip index)
Error index is 0 for a correct note, 1 for a note with an erroneous pitch, and 3 for an extra
note. For an extra note, the given note ID is *. Each line in the second part describes
missing notes, notes in the score (fmt3x) file that do not appear in the alignment result.
The number and the symbol indicate the score time and the note ID of the missing note.
4.5

Corresp File

The Corresp file describes the result of MIDI-to-MIDI alignment, which looks like:..... (Some comment lines)
0 1.00833 C#5 73 36 0 0 C#5 73 50
1 1.025 A3 57 32 2 0.00625 A3 57 31.....
Each line describes from left to right:(ID) (onset time) (spelled pitch) (integer pitch) (onset velocity)of a note in the aligned MIDI ex_align2.mid followed by the same set of data of a note in
the reference MIDI ex_align1.mid. If ID for the reference MIDI is * it indicates an extra
note and if ID for the aligned MIDI is * it indicates a missing note.

55.1

Usage in Detail

Score-to-MIDI Alignment

For an input musicXML (in1.xml) and a MIDI file (in2.mid), please enter the following
to run the alignment programs:
AAA$./MusicXMLToMIDIAlign.sh in1 in2
The output file is the final alignment result (in2_match.txt).
If you wish to see the intermediate alignment results, erase the last three lines inMusicXMLToMIDIAlign.sh and run again. This produces the alignment result by the HMM

5

(in2_pre_match.txt), the result after applying the performance error detection toalign_pre_match.txt (in2_err_match.txt), and the results after realignment step(in2_realigned_match.txt), which is equal to in2_match.txt.
5.2

MIDI-to-MIDI Alignment

The MIDI-to-MIDI alignment program finds corresponding notes in two MIDI files. Any
two MIDI files (say, in1.mid and in2.mid) can be used in principle. Internally one of themis used as a reference signal, which will be converted to score files (Fmt3x and hmm files)
and the score-to-MIDI alignment algorithm is applied for this score and the second MIDI
file. Please enter the following to run the alignment programs:AAA$./MIDIToMIDIAlign.sh in1 in2(The first MIDI file in1.mid is used as the reference signal.) The output files are a Match
file in2_match.txt and a Corresp file in2_corresp.txt. The Corresp file can be produced
from the Match file withAAA$./Programs/MatchToCorresp in1 in2_match.txt in1_spr.txt in2_corresp.txt
This command can be used after correcting the match file (e.g. with the accompanying UI).
If you wish to see the intermediate alignment results, erase the last three lines inMusicXMLToMIDIAlign.sh and run again (similarly as in Sec. 5.1).

6