Patent ID: 9640159
Date: 2017-05-02
CPC Classifications: G10H

Claim:
1. A method for synchronizing audio tracks, comprising: obtaining a first audio track of a first track duration, the first audio track representing first audio content recorded over the first track duration, first audio content including harmonic sound, the harmonic sound having a first harmonic and a second harmonic; obtaining a second audio track of a second track duration, the second audio track representing second audio content recorded over the second track duration, second audio content including harmonic sound, the harmonic sound having a third harmonic and a fourth harmonic; obtaining one or more temporal window lengths, the one or more temporal window lengths including a first temporal window length and a second temporal window length, the second temporal window length being different than the first temporal window length; partitioning the first track duration into multiple temporal windows of the first temporal window length; partitioning the first track duration into multiple temporal windows of the second temporal window length; partitioning the second track duration into multiple temporal windows of the first temporal window length; partitioning the second track duration into multiple temporal windows of the second temporal window length; determining a first transformed representation of the first audio track by transforming individual temporal windows of the first audio track having the first temporal window length into frequency space in which energy is represented as a function of frequency; determining a second transformed representation of the first audio track by transforming individual temporal windows of the first audio track having the second temporal window length into frequency space in which energy is represented as a function of frequency; determining a third transformed representation of the second audio track by transforming individual temporal windows of the second audio track having the of the first temporal window length into frequency space in which energy is represented as a function of frequency; determining a fourth transformed representation of the second audio track by transforming individual temporal windows of the second audio track having the of the second temporal window length into frequency space in which energy is represented as a function of frequency; identifying pitches of harmonic sound in the first transformed representation such that pitch of the harmonic sound in the first audio content is determined for individual temporal windows of the first temporal window length; identifying pitches of harmonic sound in the second transformed representation such that pitch of the harmonic sound in the first audio content is determined for individual temporal windows of the second temporal window length; identifying pitches of harmonic sound in the third transformed representation such that pitch of the harmonic sound in the second audio content is determined for individual temporal windows of the first temporal window length; identifying pitches of harmonic sound in the fourth transformed representation such that pitch of the harmonic sound in the second audio content is determined for individual temporal windows of the second temporal window length; determining magnitudes of harmonic energy at harmonics of the harmonic sound in the first transformed representation such that magnitude of energy is determined for the first harmonic and the second harmonic for individual temporal windows of the first temporal window length; determining magnitudes of harmonic energy at harmonics of the harmonic sound in the second transformed representation such that magnitude of energy is determined for the first harmonic and the second harmonic for individual temporal windows of the second temporal window length; determining magnitudes of harmonic energy at harmonics of the harmonic sound in the third transformed representation such that magnitude of energy is determined for the third harmonic and the fourth harmonic for individual temporal windows of the first temporal window length; determining magnitudes of harmonic energy at harmonics of the harmonic sound in the fourth transformed representation such that magnitude of energy is determined for the third harmonic and the fourth harmonic for individual temporal windows of the second temporal window length; comparing the first transformed representation of the first audio track to the third transformed representation of the second audio track to correlate pitch of the harmonic sound and harmonic energy of individual temporal windows in the first transformed representation with pitch of the harmonic sound and harmonic energy of individual temporal windows in the third transformed representation, the correlated pitch and harmonic energy being identified as potentially representing energy in the same sounds; comparing the second transformed representation for at least one individual temporal window of the first audio track to the fourth transformed representation of at least one individual temporal window of the second audio track to correlate pitch of the harmonic sound and harmonic energy in the individual windows of the second transformed representation with pitch of the harmonic sound and harmonic energy of the fourth transformed representation, the second transformed representation of at least one temporal window and the fourth transformed representation of at least one window being selected for the comparison based on the correlation of pitch of the harmonic sound and harmonic energy between the first transformed representation and the third transformed representation; determining, from the correlations of pitch of the harmonic sound and harmonic energy, a temporal alignment estimate between the first audio track and the second audio track, the temporal alignment estimate reflecting an offset in time between commencement of sound in the first audio track and commencement of sound in the second audio track; and synchronizing the first audio track with the second audio track based on the temporal alignment estimate.