SheetSage consistently messes up covers around the 50 second mark
I have been finding that whenever I do cover songs with YuE2, Sheetsage's transcription gets messed up around 50 seconds in, which causes vocals to be sung early, which then throws off the timing of the rest of the song. Is it possible there is some issue with the way the model is trained, or am I using it incorrectly?
+1, not sure if this is some ComfyUI problem but in the current state it's basically unusable
Thansk for your feedback. Could you share an example song (like a youtube link) that reproduces this? Better if you can also share the ABC output.
Also, which codebase/comfyui workflow did you use to run it?
Best,
Junyan
any instrumental track, default ComfyUI workflow 🤷♂️ btw is this normal that "Ins" is the actual melody and "Vocal" is only chords?
Thanks. Some examples would be helpful.
Known issue: SheetSage2 does not handle songs with constant tempo changes very well. Occasional tempo changes are fine, but free-rhythm passages are currently outside its scope.
"Ins" is the actual melody and "Vocal" is only chords is as expected for songs without any vocal.
Chiptune example: Draconus Main theme
Note the hallucinated pause & two notes z24z4e2e2 in the very beginning.
Sheetsage likes to insert extra beats here, the last time I tried it, I got some weird 4/4 <-> 2/4 <-> 4/4 switch around 40-50 seconds, this time got an extra beat ~2 times later, at 1:40ish. Even more mistakes later, but that might be another question
(oh well, HF does not allow txt attach, IWOT)
X:1
T:
M:4/4
L:1/32
Q:1/4=151
V: Vocal clef=treble name="Vocal Melody" snm="Vocal"
V: Ins clef=treble name="Ins Melody" snm="Inst."
K:Cm
% intro
V: Vocal
z24z4"Cm"z4|"Cm"z32|"Cm"z32|"D#"z32|
V: Ins
z24z4e2e2|e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|
V: Vocal
"D#"z32|"Fm"z32|"G#"z32|"A#"z32|
V: Ins
c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|
V: Vocal
"Fm"z32|"Cm"z32|"Cm"z32|"D#"z32|
V: Ins
c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|
V: Vocal
"D#"z32|"Fm"z32|"G#"z32|"A#"z32|
V: Ins
c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|
V: Vocal
"Fm"z32|"Cm"z32|"Cm"z32|
V: Ins
c4g4B4g4c4g4d4g4|C4C4C4C4C4C4C4C4|C4C4C4C4C4C4C4C4|
% verse
V: Vocal
"Cm"z32|"D#"z16"A#"z16|"Gm"z32|"G#"z16"A#"z16|
V: Ins
c24c4d4|e8g8f8e8|f4c24-c4-|c4z4a4d2e2d8B8|
V: Vocal
"Cm"z32|"D#"z16"A#"z16|"Gm"z32|"G#"z16"A#"z16|
V: Ins
c24c4d4|e8g8a8b8|a4g24-g4-|g4z4c'2b2a2g2f2e2d2c2B2A2C2B,2|
V: Vocal
"Fm"z32|"Gm"z32|"Fm"z32|"G"z32|
V: Ins
F12F4G4A12|G4F12E4D12|F12F4G4A12|G8=B8d8B8|
V: Vocal
"Fm"z32|"Gm"z32|"Fm"z32|"G"z32|
V: Ins
F12F4G4A12|G4F12E4D12|F12F4G4A12|G4=B4d4g4b4d'4b4g4|
V: Vocal
"Cm"z32|"D#"z16"A#"z16|"Gm"z32|"G#"z16"A#"z16|
V: Ins
c24c4d4|e8g8f8e8|f4c24-c4-|c4z4a4d2e2d8B8|
V: Vocal
"Cm"z32|"D#"z16"A#"z16|"Gm"z32|"G#"z16"A#"z16|
V: Ins
c24c4d4|e8g8a8b8|a4g24-g4-|g4z4c'2b2a2g2f2e2d2c2B2A2C2B,2|
V: Vocal
"Fm"z32|"Gm"z32|"Fm"z32|"G"z32|
V: Ins
F12F4G4A12|G4F12E4D12|F12F4G4A12|G8=B8d8B8|
V: Vocal
"Fm"z32|"Gm"z32|"Fm"z32|"G"z32|
V: Ins
F12F4G4A12|G4F12E4D12|F12F4G4A12|G4=B4d4g4b4d'4b4g4|
% interlude
V: Vocal
"Cm"z32|"Cm"z16"A#7"z16|"A#7"z32|"Cm7"z32|
V: Ins
C2B,2C2G,2F,2G,2C2B,2C2G,2F,2G,2C2B,2C2G,2|F,2G,2C2B,2C2G,2F,2G,2B,2A,2B,2F,2E,2F,2B,2A,2|B,2F,2E,2F,2B,2A,2B,2F,2E,2F,2B,2A,2F,2F,2E,2F,2|C2B,2C2G,2F,2G,2C2B,2C2G,2F,2G,2C2B,2C2G,2|
V: Vocal
"Cm7"z16"Fm7"z16|"Fm7"z32|"Cm7"z32|"Cm7"z16"Fm7"z16|
V: Ins
F,2G,2C2B,2C2G,2F,2G,2B,2A,2B,2F,2E,2F,2B,2A,2|B,2F,2E,2F,2B,2A,2B,2F,2E,2F,2B,2A,2B,2F,2E,2F,2|C2B,2C2G,2F,2G,2C2B,2C2G,2F,2G,2C2B,2C2G,2|F,2G,2C2B,2C2G,2F,2G,2B,2A,2B,2F,2E,2F,2B,2A,2|
V: Vocal
"Fm7"z32|"Cm7"z32|"Cm7"z32|"Fm7"z32|
V: Ins
B,2F,2E,2F,2B,2A,2B,2F,2E,2F,2B,2A,2B,2F,2E,2F,2|B,2B,2C2G,2F,2G,2C2B,2C2B,2C2G,2F,2G,2C2B,2|C2B,2C2G,2F,2G,2C2B,2C2B,2C2G,2F,2G,2c2c2|c2c2c2c2c2c2c2c2B2B2B2B2B2B2B2B2|
V: Vocal
"Fm7"z32|
V: Ins
G2G2G2G2G2G2G2G2F2F2F2F2F2F2F2F2|
% verse
V: Vocal
"Cm"z32|"Cm"z16"Gm"z16|"Gm"z32|"Cm"z32|
V: Ins
c4d2e4f6e2d4e6d2c2-|c2d6c2B6c8z2G6-|G32|c4d2e4f6e2d4e6d2c2-|
V: Vocal
"Cm"z16"Gm"z16|"Gm"z32|"Fm"z32|"Fm"z16"Cm"z16|
V: Ins
c2d6c2B6g8z2g2z4|z8g4z4g4g4g4f4|F6G6A6B6c6d2-|d4e6f6e8d2e2c4-|
V: Vocal
"Cm"z32|"Gm"z32|"Gm"z16"G"z16|"G"z32|
V: Ins
c24-c4z4|G24A8-|A4G6d6=B8z2G2A4|z2E2^F2F2=A4A2G2=E2G2=B2B2B2A2F2A2|
V: Vocal
"Cm"z32|"Cm"z16"Gm"z16|"Gm"z32|"Cm"z32|
V: Ins
c4d2e4f6e2d4e6d2c2-|c2d6c2B6c8z2G6-|G24-G2z6|c4d2e4f6e2d4e6d2c2-|
V: Vocal
"Cm"z16"Gm"z16|"Gm"z32|"Fm"z32|"Fm"z16"Cm"z16|
V: Ins
c2d6c2B6g8z2g2z4|z8g4z4f4g4g4f4|F6G6A6B6c6d2-|d4e6f6e8d2e2c4-|
V: Vocal
"Cm"z32|"Gm"z32|"Gm"z16"G"z16|"G"z32|
V: Ins
c24-c4z4|G24A8-|A4G6d6=B8z2G2A4|z2^F2F2F2=A4A2G2G2G2=B2B2B2A2F2A2|
% interlude
V: Vocal
"Cm"z32|"Cm"z16"Gm"z16|"Gm"z32|"Cm"z32|
V: Ins
c'2b2g2^f2=f2e2c2B2G2^F2=F2E2C2G2C2G2|C2G2C2B2C2B2C2B2c'2b2g2^f2=f2e2c2B2|G2^F2=F2E2C2G2C2G2C2G2C2B2C2B2C2B2|c'2b2g2^f2=f2e2c2B2G2^F2=F2E2C2G2C2G2|
V: Vocal
"Cm"z16"Gm"z16|"Gm"z32|"F"z32|"F"z16"Cm"z16|
V: Ins
C2G2C2B2C2B2C2B2c'2b2g2^f2=f2e2c2B2|G2^F2=F2E2C2G2C2G2C2G2C2B2C2B2C2B2|F,2G,2B,2C2E2F2^F2G2B2c2e2=f2b2f2b2f2|b2f2=a2f2a2f2a2f2c'2b2g2^f2=f2e2c2B2|
V: Vocal
"Cm"z32|"Gm"z32|"Gm"z32|"Gm"z32|
V: Ins
G2^F2=F2E2C2G2C2G2C2G2C2B2C2B2C2B2|B,2=A,2G,2F,2E,2F,2^F,2G,2B,2C2E2=F2^F2G2B2c2|e2f2e2c2B2G2^F2=F2c2B2G2c2B2G2c2B2|G2c2B2G2c2B2G2G2G2G2c2B2G2^F2=F2E2|
% verse
V: Vocal
"Cm"z32|"Cm"z16"Gm"z16|"Gm"z32|"Cm"z32|
V: Ins
c4d2e4f6e2d4e6d2c2-|c2d6c2B6c8z2G6-|G24-G2z6|c4d2e4f6e2d4e6d2c2-|
V: Vocal
"Cm"z16"Gm"z16|"Gm"z32|"Fm"z32|"Fm"z16"Cm"z16|
V: Ins
c2d6c2B6g8z2g2z4|z8g4z4g4g4g4f4|F6G6A6B6c6d2-|d4e6f6e8d2e2c4-|
V: Vocal
"Cm"z32|"Gm"z32|"Gm"z16"G"z16|"G"z32|
V: Ins
c24-c4z4|G24A8-|A4G6d6=B8z2G2A4|z2^F2E2F2=A4A2G2=E2G2=B2B2B2A2F2A2|
% interlude
V: Vocal
"Cm"z32|"Cm"z32|
V: Ins
C4C4C4C4C4C4C4C4|C4C4C4C4C4C4C4C4|
% outro
V: Vocal
"Cm"z32|"Cm"z32|"D#"z32|"D#"z32|
V: Ins
e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|
V: Vocal
"Fm"z32|"G#maj7"z32|"A#"z32|"Cm"z32|
V: Ins
e4g4f4g4e4g4d4g4|c4g4B4g4c4g4d4g4|e4g4f4g4e4g4d4g4|g4z24z4|
V: Vocal
"Cm"z32|
V: Ins
Z|