Patent ID: 12235896

The figures are not to scale. Instead, the thickness of the layers or regions may be enlarged in the drawings. In general, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts.

DETAILED DESCRIPTION

Descriptors first, second, third, etc. are used herein when identifying multiple elements or components which may be referred to separately. Unless otherwise specified or understood based on their context of use, such descriptors are not intended to impute any meaning of priority, physical order or arrangement in a list, or ordering in time but are merely used as labels for referring to multiple elements or components separately for ease of understanding the disclosed examples. In some examples, the descriptor “first” may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as second or third. In such instances, it should be understood that such descriptors are used merely for ease of referencing multiple elements or components.

Fingerprint or signature-based media monitoring techniques generally utilize one or more inherent characteristics of the monitored media during a monitoring time interval to generate a substantially unique proxy for the media. Such a proxy is referred to as a signature or fingerprint, and can take any form (e.g., a series of digital values, a waveform, etc.) representative of any aspect(s) of the media signal(s) (e.g., the audio and/or video signals forming the media presentation being monitored). A signature can be a series of sub-signatures collected in series over a time interval. The term “fingerprint” and “signature” are used interchangeably herein and are defined herein to mean a proxy for identifying media that is generated from one or more inherent characteristics of the media.

Signature-based media monitoring generally involves determining (e.g., generating and/or collecting) signature(s) representative of a media signal (e.g., an audio signal and/or a video signal) output by a monitored media device and comparing the monitored media signature(s) to one or more reference signatures corresponding to known (e.g., reference) media sources. Various comparison criteria, such as a cross-correlation value, a Hamming distance, etc., can be evaluated to determine whether a monitored signature matches a particular reference signature.

When a match between the monitored signature and one of the reference signatures is found, the monitored media can be identified as corresponding to the particular reference media represented by the reference signature that matched with the monitored media signature. Because attributes, such as an identifier of the media, a presentation time, a broadcast channel, etc., are collected for the reference signature, these attributes can then be associated with the monitored media whose monitored signature matched the reference signature. Example systems for identifying media based on codes and/or signatures are long known and were first disclosed in Thomas, U.S. Pat. No. 5,481,294, which is hereby incorporated by reference in its entirety.

Historically, audio fingerprinting technology has used the loudest parts (e.g., the parts with the most energy, etc.) of an audio signal to create fingerprints in a time segment. In some examples, the loudest parts of an audio signal can be associated with noise (e.g., unwanted audio) and not from the audio of interest. In some examples, fingerprints generated using historic audio fingerprint technology would be generated based on the background noise and not of the audio of interest, which reduces the usefulness of the generated fingerprint. Additionally, fingerprints of music generated using these historic audio fingerprint technologies often are not generated information from all parts of the audio spectrum that can be used for signature matching because the bass spectrum of audio tends to be louder than other frequencies spectra in the audio (e.g., treble ranges, etc.). Some example methods, apparatus, systems, and articles of manufacture to overcome the above-noted deficiencies by generating fingerprints using mean normalization and are disclosed in U.S. patent application Ser. No. 16/453,654, which is hereby incorporated by reference in its entirety

Example methods and apparatus disclosed herein generate a fingerprint from an audio signal using exponential mean normalization. Example methods and apparatus disclosed herein reduce the memory and bandwidth requirements to generate a fingerprint using mean normalization. An example method disclosed herein includes normalizing one or more of the time-frequency bins of the audio signal by an audio characteristic of a first adjacent audio region and an approximation of an exponential mean value corresponding to an average audio characteristic of a second adjacent audio region. As used herein, “a time-frequency bin” is a portion of an audio signal corresponding to a specific frequency bin (e.g., an FFT bin) at a specific time (e.g., three seconds into the audio signal). In some examples disclosed herein, the normalized time-frequency bins are used to generate subfingerprints, which are combined to generate a fingerprint.

Another example method disclosed herein includes matching a generated query fingerprint, sampled at a first sample rate, to fingerprint in a database of reference fingerprints, sampled at a second sample rate. In some examples disclosed herein, the first subfingerprint of the query fingerprint is compared to the subfingerprints of the reference fingerprints in the database to determine candidate reference fingerprints. In some examples disclosed herein, the first sample rate is compared to the second sample rate to determine a sample rate coefficient. In some examples disclosed herein, the subfingerprints of the query fingerprint are compared to the candidate reference fingerprint based on the sample rate coefficient to determine if the query fingerprint matches the candidate reference fingerprint.

FIG.1is an example system100in which the teachings of this disclosure can be implemented. The example system100includes an example audio source102, an example microphone104that captures sound from the audio source102and converts the captured sound into an example audio signal106. An example audio processor108receives the audio signal106and generates an example query fingerprint110, which is transmitted over an example network111to an example fingerprint comparator112. The example fingerprint comparator112matches the example query fingerprint110to fingerprints of an example reference fingerprint database114to generate an example media identification report116.

The example audio source102emits an audible sound. The example audio source can be a speaker (e.g., an electroacoustic transducer, etc.), a live performance, a conversation and/or any other suitable source of audio. The example audio source102can include desired audio (e.g., the audio to be fingerprinted, etc.) and can also include undesired audio (e.g., background noise, etc.). In the illustrated example, the audio source102is a speaker. In other examples, the audio source102can be any other suitable audio source (e.g., a person, etc.).

The example microphone104is a transducer that converts the sound emitted by the audio source102into the audio signal106. In some examples, the microphone104can be a component of a computer, a mobile device (a smartphone, a tablet, etc.), a navigation device, or a wearable device (e.g., a smartwatch, etc.). In some examples, the microphone can include an analog-to digital convertor to digitize the audio signal106. In other examples, the audio processor108can digitize the audio signal106.

The example audio signal106is a digitized representation of the sound emitted by the audio source102. In some examples, the audio signal106can be saved on a computer before being processed by the audio processor108. In some examples, the audio signal106can be transferred over a network to the example audio processor108. Additionally or alternatively, any other suitable method can be used to generate the audio (e.g., digital synthesis, etc.).

The example audio processor108converts the example audio signal106into the example query fingerprint110. In some examples, the audio processor108can convert some or all of the audio signal106into the frequency domain. In some examples, the audio processor108divides the audio signal into time-frequency bins. In some examples, the audio processor108can generate a subfingerprint using an audio characteristic of a first adjacent audio region and an approximation of an average audio characteristic of a second adjacent audio region. In some examples, the audio characteristic is the energy of the audio signal. In other examples, any other suitable audio characteristic can be determined and used to normalize each time-frequency bin (e.g., the entropy of the audio signal, etc.). Additionally or alternatively, any suitable means can be used to generate the query fingerprint110. In some examples, some of or all of the components of the audio processor108can be implemented by a mobile device (e.g., a mobile device associated with the microphone104, etc.). In other examples, the audio processor108can be implemented by any other suitable device(s). An example implementation of the audio processor108is described below in conjunction withFIG.2.

The example query fingerprint110is a condensed digital summary of the audio signal106that can be used to identify and/or verify the audio signal106. For example, the query fingerprint110can be generated by sampling portions of the audio signal106and processing those portions. In some examples, the query fingerprint110is composed of a plurality of subfingerprints, which correspond to distinct samples of the audio signal106. In some examples, the query fingerprint110is associated with a period of time (e.g., six seconds, 48 seconds, etc.) of audio signal106. In some examples, the query fingerprint110can include samples of the highest energy portions of the audio signal106. In some examples, the query fingerprint110can be used to identify the audio signal106(e.g., determine what song is being played, etc.). In some examples, the query fingerprint110can be used to verify the authenticity of the audio. An example illustration of the query fingerprint110is described below in conjunction withFIGS.3A-3C.

The example network111is a network that allows the query fingerprint110to be transmitted to the fingerprint comparator112. For example, the network111is a local area network (LAN), a wide area network (WAN), etc. In some examples, the network111is the Internet. In some examples, the network111is a wired connection. In some examples, the network111is absent. In such examples, the query fingerprint110can be transmitted to fingerprint comparator by any other suitable means (e.g., a physical storage device, etc.). Additionally or alternatively, the audio processor108and the fingerprint comparator112can be implemented by or at the same device (e.g., a server at a central facility of media monitoring entity, etc.).

The example fingerprint comparator112receives and processes the query fingerprint110. For example, the fingerprint comparator112can match the query fingerprint110to one or more reference fingerprint(s) stored in the reference fingerprint database114. In some examples, the fingerprint comparator112can compare the sample rate of the query fingerprint110to the sample rate (e.g., the frequency of subfingerprints, etc.) of the fingerprints stored in reference fingerprint database114. In such examples, the fingerprint comparator112can compare fingerprints sampled at different sample rates based on the comparison of the sample rates by generating a sample rate coefficient. For example, the fingerprint comparator112can determine which subfingerprint(s) of the reference fingerprints to compare with subfingerprints of the query fingerprint110. In some examples, the fingerprint comparator112can determine the query fingerprint matches none of the reference fingerprints stored in the reference fingerprint database114. In such examples, the fingerprint comparator112returns a result indicating the media associated with the reference fingerprint could not be identified.

The reference fingerprint database114stores a plurality of reference fingerprint(s) corresponding to one or more pre-identified pieces of media. In some examples, the reference fingerprint(s) stored of reference fingerprint database114are generated and stored at a first sample rate (e.g., 128 milliseconds (ms), etc.). In other examples, the reference fingerprint(s) can be stored at different sample rates (e.g., 64 ms, 512 ms, etc.). The reference fingerprint database114can be implemented by a volatile memory (e.g., a Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM), etc.) and/or a non-volatile memory (e.g., flash memory). The reference fingerprint database114can additionally or alternatively be implemented by one or more double data rate (DDR) memories, such as DDR, DDR2, DDR3, DDR4, mobile DDR (mDDR), etc. The reference fingerprint database114can additionally or alternatively be implemented by one or more mass storage devices such as hard disk drive(s), compact disk drive(s), digital versatile disk drive(s), solid-state disk drive(s), etc. In the illustrated example ofFIG.1, the reference fingerprint database114is illustrated as a single database. In other examples, the reference fingerprint database114can be implemented by any number and/or type(s) of databases. Furthermore, the reference fingerprint(s) stored in the reference fingerprint database114may be in any data format. (e.g., an 8 bit integer number, a 32 bit floating point number, etc.).

The example media identification report116is a report that identifies the media associated with the audio signal106. For example, the fingerprint comparator112can generate the media identification report116based a match between the query fingerprint110and a reference fingerprint associated with the media. For example, the media identification report116can be sent to the device associated with the microphone104(e.g., a smart phone, etc.) to enable a user of the microphone104to the identify the media. In some examples, the media identification report116can include any other suitable information (e.g., metadata, etc.).

FIG.2is an example implementation of the audio processor108ofFIG.1. The example audio processor108includes an example audio signal interface202, an example audio segmenter204, an example signal transformer206, an example memory manager208, an example audio characteristic determiner210, an example mean calculator212, an example bin normalizer214, an example subfingerprint generator216and an example fingerprint generator218.

The example audio signal interface202receives the digitized audio signal from the microphone104. In some examples, the audio signal interface202can request the digitized audio signal from the microphone104periodically. In other examples, the audio signal interface202can receive the audio signal106from the microphone108as soon as the audio is detected. In some examples when the microphone104is absent, the audio signal interface202can request the digitized audio signal106from a database. In some examples, the audio signal interface202can include an analog-to-digital converter to convert the audio received by the microphone104into the audio signal106.

The example audio segmenter204divides the audio signal106into audio segments (e.g., frames, periods, etc.). For example, the audio segmenter can divide the audio signal106into discrete audio segments corresponding to unique portions of the audio signal106. In some examples, the audio segmenter204determines which portions of the audio signal106correspond to each of the generated audio segments. In some examples, the audio segmenter204can generate segments of any suitable size.

The example signal transformer206transforms portions of the audio signal of the digitized audio signal106into the frequency domain. For example, the signal transformer206performs a fast Fourier transform (FFT) on an audio signal106to transform the audio signal106into the frequency domain. In other examples, the signal transformer206can use any suitable technique to transform the audio signal106(e.g., discrete Fourier transforms, a sliding time window Fourier transform, a wavelet transform, a discrete Hadamard transform, a discrete Walsh Hadamard, a discrete cosine transform, etc.). In some examples, each time-frequency bin has an associated magnitude (e.g., the magnitude of the transformed signal in that time-frequency bin, etc.). In some examples, the signal transformer206can be implemented by one or more band-pass filters (BPFs). In some examples, the output of the example signal transformer206can be represented by a spectrogram. In some examples, the signal transformer206works concurrently with the audio segmenter204. An example output of the signal transformer206is discussed below in conjunction withFIGS.3A and3B.

The example memory manager208manages the content of the memory associated with the audio processor108(e.g., the volatile memory814, the non-volatile memory816, the local memory813, the mass storage828, etc.). For example, the memory manager208can add or remove data from buffers associated with the generation of the query fingerprint110. In some examples, the memory manager208can remove information from a buffer after it is no longer used in the generation of the query fingerprint110. For example, the memory manager208can remove data associated with a group of time-frequency bins (e.g., the energy values associated with each time-frequency bin, etc.) from a buffer after it is no longer needed. In some examples, the memory manager208removes the data associated with an audio region after the mean calculator212has determined the mean associated with that audio region. In some examples, the memory manager208converts the calculate time-frequency bin energy values to the dB (decibel) scale and rounds the values to the nearest half dB (e.g., converts to an 8-bit integer value from a 32-bit floating-point value, etc.).

The example audio characteristic determiner210determines the audio characteristic(s) of a portion of the audio signal106(e.g., an audio region associated with a time-frequency bin, etc.). The audio characteristic determiner210can determine the audio characteristics of a group of time-frequency bins (e.g., the energy of the portion of the audio signal106corresponding to each time-frequency bin in a group of time-frequency bins, the entropy of the portion of the audio signal106corresponding to each time-frequency bin in a group of time-frequency bins, etc.). For example, the audio characteristic determiner210can determine the mean energy (e.g., average power, etc.) of one or more of the audio regions associated with an audio region (e.g., the mean of the magnitudes squared of the transformed signal corresponding to the time-frequency bins in the region, etc.) adjacent to a selected time-frequency bin. In other examples, the audio characteristic determiner210can determine the mean entropy of one or more of the audio regions associated with an audio region (e.g., the mean of the magnitudes of the time-frequency bins in the region, etc.) adjacent to a selected time-frequency bin. In other examples, the audio characteristic determiner210can determine the mean energy and/or mean entropy by any other suitable means. Additionally or alternatively, the audio characteristic determiner210can determine other characteristics of a portion of the audio signal (e.g., the mode energy, the median energy, the mode power, the median energy, the mean energy, the mean amplitude, etc.).

The example mean calculator212determines the exponential mean energy and/or entropy (e.g., a moving average energy, a moving average entropy, etc.) associated with one or more audio regions in time-frequency space. For example, the mean calculator212can determine the mean (e.g., average, median, etc.) energy value and/or entropy value associated with the time-frequency bins in an audio segment. In some examples, the mean calculator212can determine the average energy and/or the average entropy associated with two audio segments based on the mean energy and/or entropy of a subset of audio time-frequency bins of an adjacent audio region (e.g., determined by the audio characteristic determiner210, etc.) and the mean energies of a subset of time-frequency bins in the audio region using equation (1):

M2=(1-α)⁢M1+α⁢S(1)
wherein M2is an approximate mean energy and/or mean entropy of the current audio region, α is an empirically determined weighting coefficient (e.g., between 0 and 1, etc.), M1is the mean energy and/or mean entropy of a previous overlapping audio region and S is the magnitude or magnitude squared of the transformed signal associated with a subset of time-frequency bins in the current audio region. In other examples, the mean calculator212can determine the mean energy and/or mean entropy based on any other suitable means.

The example bin normalizer214normalizes one or more time-frequency bins by an associated audio characteristic of the surrounding audio region. For example, the bin normalizer214can normalize a time-frequency bin by a mean energy of the surrounding audio regions. In some examples, the bin normalizer214can normalize a time-frequency bin by a mean entropy of the surrounding audio regions. In other examples, the bin normalizer214can normalize a time-frequency bin by the exponential mean energy values and/or the exponential mean entropy values determined by the mean calculator212. For example, the bin normalizer214can normalize a time-frequency bin of a previous segment (e.g., temporally behind, etc.) based on the exponential mean values calculated in connection with an audio segment (e.g., temporally in front of the previous segment, etc.). In some examples, the output(s) of the bin normalizer214(e.g., a plurality normalized time-frequency bin, etc.) can be represented as a spectrogram. Example outputs of the bin normalizer214are discussed below in conjunction withFIG.3C.

The example subfingerprint generator216generates subfingerprints associated with an audio sample(s) and/or audio segment. For example, the subfingerprint generator216can generate subfingerprints at a first sample rate. In some examples, the subfingerprint generator216generates a subfingerprint of a sample after the bin normalizer214has normalized the energy value of each time-frequency bin in an audio segment. In some examples, the subfingerprint generator216generates the subfingerprint associated with a sample based on the energy extrema of the normalized time-frequency bins within the sample. In some examples, the subfingerprint generator216selects a group of time-frequency bins (e.g., one bin, five bins, 20 bins, etc.) with the highest normalized energy values in a sample to generate a subfingerprint.

The example fingerprint generator218generates a fingerprint based on the subfingerprints. For example, the fingerprint generator218can generate the query fingerprint110based on the subfingerprints generated by the subfingerprint generator216. For example, the fingerprint generator218can concatenate the subfingerprints associated with each audio segment into the query fingerprint110.

While an example manner of implementing the audio processor108ofFIG.1is illustrated inFIG.2, one or more of the elements, processes, and/or devices illustrated inFIG.2may be combined, divided, re-arranged, omitted, eliminated, and/or implemented in any other way. Further, the example audio signal interface202, the example audio segmenter204, the example signal transform206, the example memory manager208, the example audio characteristic determiner210, the example mean calculator212, the bin normalizer214, the example subfingerprint generator216, the example fingerprint generator218and/or, more generally, the example audio processor108ofFIGS.1and2may be implemented by hardware, software, firmware, and/or any combination of hardware, software, and/or firmware. Thus, for example, any of the example audio signal interface202, the example audio segmenter204, the example signal transform206, the example memory manager208, the example audio characteristic determiner210, the example mean calculator212, the bin normalizer214, the example subfingerprint generator216, the example fingerprint generator218, and/or, more generally, the example audio processor108could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)), and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example audio signal interface202, the example audio segmenter204, the example signal transform206, the example memory manager208, the example audio characteristic determiner210, the example mean calculator212, the example bin normalizer214, the example subfingerprint generator216, the example fingerprint generator218is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc., including the software and/or firmware. Further still, the example audio processor106ofFIGS.1and2may include one or more elements, processes, and/or devices in addition to, or instead of, those illustrated inFIG.2, and/or may include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the phrase “in communication,” including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

FIGS.3A and3Bdepict an example unprocessed spectrogram300associated with the example audio signal106received by the audio signal interface202ofFIG.2. In the illustrated examples ofFIGS.3A and3B, each time-frequency bin of the unprocessed spectrogram300is normalized to generate an example normalized spectrogram301ofFIG.3C. In the illustrated example ofFIG.3A, the example unprocessed spectrogram300includes an example first time-frequency bin302A surrounded by an example first audio region304A and an example second audio region306A. In the illustrated example ofFIG.3B, the example unprocessed spectrogram300includes an example second time-frequency bin302B surrounded by an example third audio region304B and an example fourth audio region306B. The example unprocessed spectrogram300ofFIGS.3A and3Band the normalized spectrogram301each include an example vertical axis308corresponding to frequency and an example horizontal axis310corresponding to time. In other examples, any suitable number of the time-frequency bins of the unprocessed spectrogram300can be normalized to generate the normalized spectrogram301ofFIG.3C.

The example vertical axis308has frequency bin units generated by a fast Fourier Transform (FFT) and has a length of 1024 FFT bins. In other examples, the example vertical axis308can be measured by any other suitable technique(s) of measuring frequency (e.g., Hertz, another transformation algorithm, etc.). In some examples, the vertical axis308encompasses the entire frequency range of the audio signal106. In other examples, the vertical axis308can encompass a portion of the audio signal106.

In the illustrated examples, the example horizontal axis310represents a time period of the unprocessed spectrogram300that has a total length of 11.5 seconds. In the illustrated example, horizontal axis310has sixty-four milliseconds (ms) intervals as units. In other examples, the horizontal axis310can be measured in any other suitable units (e.g., 1 second, etc.). For example, the horizontal axis310encompasses the complete duration of the audio. In other examples, the horizontal axis310can encompass a portion of the duration of the audio signal106. In the illustrated example, each time-frequency bin of the spectrograms300,301has a size of 64 ms by 1 FFT bin.

In the illustrated example ofFIG.3A, the first time-frequency bin302A is associated with an intersection of a frequency bin and a time bin of the unprocessed spectrogram300and a portion of the audio signal106associated with the intersection. The example first audio region304A includes the time-frequency bins within a pre-defined temporal distance (e.g., intervals on the horizontal axis310, etc.) behind the first time-frequency bin302A and the example second audio region306A includes the time-frequency bins within a pre-defined temporal distance (e.g., intervals on the horizontal axis310, etc.) in front of the first time-frequency bin302A. In the illustrated example ofFIG.3A, the audio regions304A,306A have the same vertical length (e.g., intervals on the vertical axis308, etc.) and the same horizontal length (e.g., intervals on the horizontal axis310, etc.). For example, the audio characteristic determiner210can determine the vertical length of the audio regions304A,306A (e.g., the length of the audio regions304A,306A along the vertical axis308, etc.) based by a set number of FFT bins (e.g., 5 bins, 11 bins, etc.). Similarly, the audio characteristic determiner210can determine the horizontal length of the audio regions304A,306A (e.g., the length of the audio regions304A,306A along the horizontal axis310, etc.). In other examples, the first audio region304A and the second audio region306A can have different vertical lengths and/or horizontal lengths. In the illustrated example ofFIG.3A, the audio regions304A,306A are rectangles. Alternatively, the audio regions304A,306A can be any suitable size and shape and can contain any suitable combination of time-frequency bins (e.g., any suitable group of time-frequency bins, etc.) within the unprocessed spectrogram300. In some examples, the audio regions304A,306A can be different sizes and/or shapes.

The example audio characteristic determiner210then determines an audio characteristic of time-frequency bins contained within the audio regions304A,306A (e.g., mean energy, average energy root-square mean energy, mean entropy, average entropy, etc.). Using the determined audio characteristic, the example bin normalizer214ofFIG.2can normalize an associated value of the first time-frequency bin302A (e.g., the energy of the first time-frequency bin302A can be normalized by the mean energy and/or mean entropy of all time-frequency bins within the audio regions304A,306A).

In the illustrated example ofFIG.3B, the second time-frequency bin302B is associated with an intersection of a frequency bin and a time bin of the unprocessed spectrogram300and a portion of the audio signal106associated with the intersection. In the illustrated example ofFIG.3B, the second time-frequency bin302B is in the same frequency band as the first time-frequency bin302A (e.g., the same position on the vertical axis308, etc.). The example third audio region304B includes the time-frequency bins within a pre-defined temporal distance (e.g., intervals on the horizontal axis310, etc.) behind the second time-frequency bin302B and the example fourth audio region306B includes the time-frequency bins within a pre-defined temporal distance (e.g., intervals on the horizontal axis310, etc.) in front of the second time-frequency bin302B. In the illustrated example ofFIG.3B, the audio regions304B,306B have the same vertical length (e.g., intervals on the vertical axis308, etc.) and the same horizontal length (e.g., intervals on the horizontal axis310, etc.). For example, the audio characteristic determiner210can determine the vertical length of the audio regions304B,306B (e.g., the length of the audio regions304B,306B along the vertical axis308, etc.) based by a set number of FFT bins (e.g., 5 bins, 11 bins, etc.). Similarly, the audio characteristic determiner210can determine the horizontal length of the audio regions304B,306B (e.g., the length of the audio regions304B,306B along the horizontal axis310, etc.). In other examples, the third audio region304B and the fourth audio region306B can have different vertical lengths and/or horizontal lengths. In the illustrated example ofFIG.3B, the audio regions304B,306B are rectangles. Alternatively, the audio regions304B,306B can be any suitable size and shape and can contain any suitable combination of time-frequency frequency bins (e.g., any suitable group of time-frequency bins, etc.) within the unprocessed spectrogram300. In some examples, the audio regions304B,306B can be different sizes and/or shapes. In the illustrated example, the audio regions304B,306B have the same size and shape as the audio regions304A,306A. In the other examples, the audio regions304B,306B have a different size and shape as the audio regions304A,306A.

In the illustrated example ofFIG.3B, the portions of the audio signal106associated with the time-frequency bins within the third audio region304B were already analyzed by the signal transformer206and the audio characteristic determiner210(e.g., during the normalization of the first time-frequency bin302A ofFIG.3A, etc.). In some examples, the third audio region304B overlaps with the second audio region306A (e.g., includes some of the same time-frequency bins, etc.). In some examples, the mean calculator212can calculate exponential mean values corresponding to the mean audio characteristic of the time-frequency bins in the third audio region304B and fourth audio region306B. For example, the mean calculator212can calculate the exponential mean corresponding to the audio regions304B,306B based on previous exponential mean values (e.g., the exponential mean values associated with an audio region displayed one time bin from the audio region defined by the audio regions304B,306B, etc.) and the audio characteristic values of the time-frequency bins at the temporal end of the audio region306B (e.g., the left-most, etc.). In such examples, the memory manager208then deletes the data associated with the previous exponential mean values to reduce the memory usage of the audio processor108. Using the determined exponential mean values, the example bin normalizer214ofFIG.2can normalize an associated value of the second time-frequency bin302B (e.g., the magnitude of first time-frequency bin302A can be normalized by the mean energy/mean entropy associated with each time-frequency bin within the audio regions304B,306B) and each other time-frequency bin in the same time bin (e.g., audio segment, frame, etc.).

FIG.3Cdepicts an example of a normalized spectrogram301generated by the audio processor108ofFIG.2by normalizing a plurality of the time-frequency bins of the unprocessed spectrogram300ofFIGS.3A-3B. For example, some or all of the time-frequency bins of the unprocessed spectrogram300can be normalized in a manner similar to how as the first time-frequency bin302A and/or second time-frequency bin302B were normalized. In such examples, the mean calculator212determines the mean value of previous time-frequency bins whenever possible (e.g., whenever the time-frequency bins had previously been used in a calculation, etc.). For example, time-frequency bins associated with temporal earlier portions of the audio signal106(e.g., the first time-frequency bin302A) are not calculated using previous mean values because the time-frequency bins associated with region behind the time-frequency bins have not been processed by the audio processor108(e.g., the time-frequency bins in the audio region304A, etc.). The resulting time-frequency bins ofFIG.3Chave now been normalized by the local mean energy/mean entropy within the local area around the region. As a result, the darker regions are areas that have the most energy in their respective local areas. This allows the fingerprint to incorporate relevant audio features even in areas that are low in energy relative to the usual louder bass frequency area.

FIG.4is an example implementation of the fingerprint comparator112ofFIG.1. The example fingerprint comparator112includes an example network interface402, an example sample rate comparator404, an example subfingerprint selector406, an example query comparator408, and an example media identifier410.

The example network interface402receives the query fingerprint110from the network111. For example, the network interface402can allow the fingerprint comparator112to communicate with the audio processor108and/or another network-enabled device. In some examples, the network interface402can be implemented by a gateway, modem, and/or any other suitable device.

The example sample rate comparator404compares the sample rate of fingerprints. For example, the sample rate comparator404can compare the sample rate (e.g., the subfingerprint rate, etc.) of the query fingerprint110to the sample rate of a reference fingerprint in the reference fingerprint database114. For example, the sample rate comparator404can determine a sample rate coefficient based on the ratio of the sample rate of the query fingerprint110to the sample rate of the reference fingerprint(s). For example, if the sample rate of the reference fingerprint is 64 ms and the sample rate of the query fingerprint110is 128 ms, the sample rate coefficient is 2. For example, if the sample rate of the reference fingerprint is 64 ms and the sample rate of the query fingerprint110is 384 ms, the sample rate coefficient is 6. In other examples, the example sample rate comparator404can determine the sample rate coefficient based on any other suitable means.

The example subfingerprint selector406selects subfingerprints from the reference fingerprint based on the sample rate coefficient. For example, the subfingerprint selector406can select subfingerprints corresponding to each subfingerprint of the query fingerprint. For example, if the sample rate coefficient is 2, the subfingerprint selector406can select every other subfingerprint of the reference fingerprint. For example, if the sample rate coefficient is 6, the subfingerprint selector406can select every sixth subfingerprint of the reference fingerprint(s). In some examples, the subfingerprint selector406selects a subfingerprint from a different reference fingerprint. In such examples, the subfingerprint selector406selects a subfingerprint from a fingerprint associated with an adjacent portion of the fingerprinted audio (e.g., a reference fingerprint associated with the same media at an adjacent time, etc.). The function of the subfingerprint selector406is described in greater detail below in conjunction withFIG.5.

The example query comparator408compares the subfingerprints of the query fingerprints to the corresponding selected subfingerprints of the reference fingerprint(s). For example, the query comparator408can determine if the query fingerprint110matches the reference fingerprint(s). In some examples, the query comparator408compares the first subfingerprint of the query fingerprint110to subfingerprints of the reference fingerprint(s) to determine candidate reference fingerprint(s) for matching. In some examples, the query comparator408can determine if the comparison indicates that query fingerprint110matches the reference fingerprint(s). For example, the query comparator408can compare the fingerprints via hashing, linear matching, etc. In some examples, the query comparator408can generate a similarity value based on the similarity between the query fingerprint and the reference fingerprint(s). In some examples, the query comparator408can determine that multiple reference fingerprint(s) match the query fingerprint110. In such examples, the query fingerprint110can be associated with multiple pieces of media. In some examples, the query comparator408can determine that the query fingerprint matches no reference fingerprint(s).

The example media identifier410generates the media identification report116based on the query comparator408. For example, the media identifier410can identify one or more pieces of media (e.g., song(s), etc.) associated with the audio signal106used to generate the query fingerprint110. In some examples, the media identifier410generates the media identification report116to indicate that the media associated with the query fingerprint110could not be identified. In some examples, the media identifier410can cause the media identification report116to be communicated to an interested-party (e.g., over the network111, etc.).

While an example manner of implementing the example fingerprint comparator112ofFIG.1is illustrated inFIG.4, one or more of the elements, processes, and/or devices illustrated inFIG.4may be combined, divided, re-arranged, omitted, eliminated, and/or implemented in any other way. Further, the example network interface402, the example sample rate comparator404, the example subfingerprint selector406, the example query comparator408, the example media identifier410, and/or, more generally, the example fingerprint comparator112ofFIGS.1and4may be implemented by hardware, software, firmware, and/or any combination of hardware, software, and/or firmware. Thus, for example, any of the example network interface402, the example sample rate comparator404, the example subfingerprint selector406, the example query comparator408, the example media identifier410, and/or, more generally, the example audio processor108could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)), and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example network interface402, the example sample rate comparator404, the example subfingerprint selector406, the example query comparator408, the example media identifier410is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc., including the software and/or firmware. Further still, the example fingerprint comparator112ofFIGS.1and4may include one or more elements, processes, and/or devices in addition to, or instead of, those illustrated inFIG.4, and/or may include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the phrase “in communication,” including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

FIG.5is an example illustration showing an example comparison500of the query fingerprint110and an example first reference fingerprint502by the example fingerprint comparator112. The example query fingerprint110and the example reference fingerprints502,503are generated (e.g., by the audio processor108ofFIGS.1and2, etc.) from the audio signal106. The example query fingerprint110includes an example first query subfingerprint506, an example second query subfingerprint508and an example third query subfingerprint510. The example first reference fingerprint502includes an example first reference subfingerprint512, an example second reference subfingerprint514, an example third reference subfingerprint516and an example fourth reference subfingerprint518. The example second reference fingerprint503includes an example fifth reference subfingerprint520.

In the illustrated example ofFIG.5, the query fingerprint110is generated at a first sample rate that is lower than the second sample rate the reference fingerprint(s) was generated at. That is, the subfingerprints of the query fingerprint110are associated with samples of the audio signal106taken at a first sample rate and the reference fingerprints502,503are associated with samples of the taken at a second sample rate. In the illustrated example ofFIG.5, the query fingerprints110and the reference fingerprints502,503include the same number of subfingerprints. In such examples, the query fingerprint110is associated with a longer portion of the audio signal106than the reference fingerprints502,503because of the comparatively lower sample rate (e.g., the query fingerprint110must encompass a greater portion of the audio signal106to include the same number of subfingerprints, etc.). In the illustrated example ofFIG.5, to compare each subfingerprint (e.g., the subfingerprints506,508,510, etc.) of the query fingerprint110, subfingerprints from multiple reference fingerprints must be selected by the subfingerprint selector406to ensure subfingerprints corresponding to the same portion of the audio signal106are compared.

In the illustrated example ofFIG.5, the sample rate comparator404can compare the sample rate of the query fingerprint110and the reference fingerprints502,503to determine a sample rate coefficient of “2” (e.g. the query fingerprint110has a sample rate of 128 ms and the reference fingerprints502,503have a sample rate of 64 ms, etc.). The subfingerprint selector406can select every other subfingerprint from the reference fingerprint502,503to compare the subfingerprints of the query fingerprint110. For example, the subfingerprint selector406selects the first reference subfingerprint512to compare to the first query subfingerprint506and the third reference subfingerprint516to compare to the second query subfingerprint508. In the illustrated example ofFIG.5, the reference fingerprint502does not contain a sample corresponding to the same portion of the audio signal106as the third query subfingerprint510. In this example, the subfingerprint selector406selects the fifth reference subfingerprint520from the second reference fingerprint503. In the illustrated example ofFIG.5, the subfingerprint selector406does not select the second reference subfingerprint514and the fourth reference subfingerprint518as the reference subfingerprints514,518do not correspond to the portions of the audio signal106fingerprinted by the query fingerprint110.

A flowchart representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the audio processor108ofFIG.2is shown inFIG.6. The machine readable instructions may be an executable program or portion of an executable program for execution by a computer processor such as the processor812shown in the example processor platform800discussed below in connection withFIG.8. The program may be embodied in software stored on a non-transitory computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor812, but the entire program and/or parts thereof could alternatively be executed by a device other than the processor812and/or embodied in firmware or dedicated hardware. Further, although the example programs are described with reference to the flowchart illustrated inFIG.6, many other methods of implementing the example audio processor108may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and/or integrated analog and/or digital circuitry, an FPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

A flowchart representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the fingerprint comparator112ofFIGS.3A-3Cis shown inFIG.7. The machine readable instructions may be an executable program or portion of an executable program for execution by a computer processor such as the processor912shown in the example processor platform900discussed below in connection withFIG.9. The program may be embodied in software stored on a non-transitory computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor912, but the entire program and/or parts thereof could alternatively be executed by a device other than the processor912and/or embodied in firmware or dedicated hardware. Further, although the example programs are described with reference to the flowchart illustrated inFIG.7, many other methods of implementing the example fingerprint comparator112may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and/or integrated analog and/or digital circuitry, an FPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

As mentioned above, the example processes ofFIGS.6and7may be implemented using executable instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory, and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.

“Including” and “comprising” (and all forms and tenses thereof) are used herein to be open ended terms. Thus, whenever a claim employs any form of “include” or “comprise” (e.g., comprises, includes, comprising, including, having, etc.) as a preamble or within a claim recitation of any kind, it is to be understood that additional elements, terms, etc. may be present without falling outside the scope of the corresponding claim or recitation. As used herein, when the phrase “at least” is used as the transition term in, for example, a preamble of a claim, it is open-ended in the same manner as the term “comprising” and “including” are open ended. The term “and/or” when used, for example, in a form such as A, B, and/or C refers to any combination or subset of A, B, C such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, and (7) A with B and with C. As used herein in the context of describing structures, components, items, objects and/or things, the phrase “at least one of A and B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects and/or things, the phrase “at least one of A or B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the performance or execution of processes, instructions, actions, activities and/or steps, the phrase “at least one of A and B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the performance or execution of processes, instructions, actions, activities and/or steps, the phrase “at least one of A or B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

The process ofFIG.6begins at block602. At block602, the audio processor108receives the digitized audio signal106. For example, the audio processor108can receive audio (e.g., emitted by the audio source102ofFIG.1, etc.) captured by the microphone104. In this example, the microphone can include an analog to digital converter to convert the audio into a digitized audio signal106. In other examples, the audio processor108can receive audio stored in a database (e.g., the volatile memory814ofFIG.8, the non-volatile memory816ofFIG.9, the mass storage828ofFIG.8, etc.). In other examples, the digitized audio signal106can transmitted to the audio processor108over a network111. Additionally or alternatively, the audio processor108can receive the audio signal106by any other suitable means.

At block604, the audio segmenter204divides the audio signal106into audio segments. For example, the audio segmenter204can divide the audio signal106into temporal segments corresponding to a length of the audio signal106associated with a sample (e.g., the period of the audio signal106corresponding to a subfingerprint, etc.). In some examples, the audio segmenter204can segment the audio signal106into audio segments into corresponding to the length of a time bin (e.g., a frame, etc.).

At block606, the audio segmenter204selects an audio segment. For example, the audio segmenter204can select a first audio segment (e.g., the audio segment corresponding to the beginning of the audio signal106, etc.). In some examples, the audio segmenter204can select an audio segment temporally immediately adjacent to a previously selected audio segment. In other examples, the audio segmenter204can select an audio segment based on any suitable characteristic.

At block608, the audio segmenter204windows the selected audio segment. For example, the audio segmenter performs a windowing function (e.g., a Hamming function, a Hann function, etc.). In other examples, the audio segmenter204can window the selected audio segment by any other suitable method.

At block610, the signal transformer206transforms the audio signal106corresponding to the selected audio segment to generate time-frequency bins. For example, the signal transformer206can transform the portion of the audio signal106corresponding to the audio segment using a Fast Fourier Transform (FFT). In other examples, the signal transformer206can use any other suitable means of transforming the audio signal106(e.g., discrete Fourier transform, a sliding time window Fourier transform, a wavelet transform, a discrete Hadamard transform, a discrete Walsh Hadamard, a discrete cosine transform, etc.). In some examples, the time-frequency bins generated by the signal transformer206and corresponding to the selected audio segment are associated with the intersection of each frequency bin of the audio signal106and the time bin(s) associated with the audio segment. In some examples, each time-frequency bin generated by the audio segmenter204has an associated magnitude value (e.g., a magnitude of the FFT coefficient of the audio signal106associated with that time-frequency bin, etc.).

At block612, the audio characteristic determiner210determines the audio characteristic of each time-frequency bin. For example, the audio characteristic determiner210can determine the magnitude of each time-frequency bin in the audio segment. In such examples, the audio characteristic determiner210can calculate the energy and/or the entropy associated with each time-frequency bin. In other examples, the audio characteristic determiner210can determine any other suitable audio characteristic(s) (e.g., amplitude, power, etc.).

At block614, the mean calculator212calculates the exponential mean values for each time-frequency bin of the audio segment. For example, the mean calculator212can calculate the exponential mean values for each time-frequency bin using Equation (1). In some examples, if the audio segment is the first audio segment of the audio signal106, the mean calculator212can set the exponential mean value based on the empirically determined weighting coefficient of Equation (1) and the audio characteristic value of the respective time-frequency bin. In other examples, the mean calculator212can determine the exponential mean values for each time-frequency bin of the audio segment based on any other suitable method.

At block616, the memory manager208discards the exponential mean values associated with the previous audio segment(s). For example, the memory manager208can discard the exponential mean values associated with the time-frequency bins of the previous adjacent audio segment after the previous exponential mean value(s) have been used to calculate exponential mean values associated with the current segment.

At block618, the memory manager208determines if the current exponential mean values allow for normalized time-frequency bins of a previous segment to be calculated. For example, the memory manager208can determine if audio characteristic values for time-frequency bins corresponding to the temporal length of the audio region306A are stored in memory (e.g., a time bin displaced 22 bins back from the current time bin, etc.). In such examples, the exponential mean values calculated by the mean calculator212allow the normalized time-frequency bins of the previous segment to be calculated (e.g., the time-frequency bin302A can be normalized after the time-frequency bins of the audio segments corresponding to the audio region306A have been incorporated into the exponential mean value(s), etc.) In some examples, audio segments at the beginning of the audio signal106do not allow the time-frequency bins of one or more previous segment(s) to be normalized (e.g., the exponential mean value(s) of the current audio segment do not correspond to the left-most edge of the audio region306A, etc.). If the memory manager208determines that the current exponential mean values allow the time-frequency bins of a previous segment to be normalized, the process600advances to block620. If the memory manager208determines that the current exponential mean values do not allow the time-frequency bins of a previous segment to be normalized, the process600returns to block606.

At block620, the bin normalizer214normalizes each time-frequency bin of the previous segment(s) based on the current exponential mean value(s) associated with proximate ones of the time-frequency bins of the current audio segment. For example, the bin normalizer214can normalize a time-frequency bin of a previous segment based on the exponential mean values associated with time-frequency bins of the current audio segment within an audio region. For example, the bin normalizer214can normalize the time-frequency bin302A based on the mean exponential values corresponding to the audio region304A,306A (e.g., the current exponential mean value(s) of the time-frequency bins along the left-most edge of the audio region306A, etc.). In some examples, the exponential mean value(s) used to normalize each time-frequency bin are an estimate of the average audio characteristic of a first audio region (e.g., the first audio region304A, the third audio region304B, etc.) and the second audio region (e.g., the second audio region306A, the fourth audio region306B, etc.).

At block622, the subfingerprint generator216computes the subfingerprint(s) associated with the previous audio segment(s). For example, the subfingerprint generator216can generate a subfingerprint based on the normalized values of the time-frequency bins of the previous segment(s) analyzed at block620. In some examples, the subfingerprint generator216generates a subfingerprint by selecting energy and/or entropy extrema (e.g., five extrema, 20 extrema, etc.) in the previous segment(s). In some examples, the subfingerprint generator216does not generate a subfingerprint (e.g., the previous audio segment is not being used to subfingerprint due to down-sampling, etc.).

At block624, the memory manager208discards the audio characteristic values associated with the previous segment(s). For example, the memory manager208can discard the information determined during the execution of block612corresponding to the time-frequency bins of the previous segment(s) analyzed during the execution of blocks620,622. In other examples, the memory manager208can discard any other unneeded information.

At block626, the audio segmenter204can determine if another audio segment is to be selected. In some examples, the audio segmenter204selects another audio segment of the audio signal106until every audio segment has been selected. If the audio segmenter204determines another time-frequency bin is to be selected, the process600returns to block606. If the audio characteristic determiner210determines another time-frequency bin is not to be selected, the process600advances to block628.

At block628, the bin normalizer214normalizes each time-frequency bin of the audio segment(s) including time-frequency bins that have not been normalized. For example, the bin normalizer214can normalize the time-frequency bins of the remaining audio segments of the audio signal106(e.g., the audio segments corresponding to the end of the audio signal106, etc.) using the exponential mean value(s) associated with the time-frequency bins of the last audio segment. For example, the bin normalizer214can normalize each remaining time-frequency bin based on the exponential value(s) associated with adjacent ones of the time-frequency bins of the last audio segment. In other examples, the bin normalizer214can normalize the remaining time-frequency bins based on any other suitable means.

At block630, the subfingerprint generator216computes the subfingerprint(s) associated with the audio segment(s) that do not have associated subfingerprints at a first sample rate. For example, the subfingerprint generator216can generate a subfingerprint based on the normalized values of the time-frequency bins of the audio segments analyzed at block628. In some examples, the subfingerprint generator216generates a subfingerprint by selecting energy and/or entropy extrema (e.g., five extrema, 20 extrema, etc.) in the previous segment(s). In some examples, the subfingerprint generator216do not generate a subfingerprint (e.g., the previous audio segment is not being used to subfingerprint due to down-sampling, etc.).

At block632, the fingerprint generator218generates a fingerprint (e.g., the query fingerprint110ofFIG.1) of the audio signal106based on the generated subfingerprints. For example, the fingerprint generator218can concatenate the generated subfingerprints into the query fingerprint110. In some examples, the fingerprint generator218can utilize any suitable means (e.g., an algorithm, etc.) to generate the query fingerprint110representative of the audio signal106. Once the example query fingerprint(s)110has been generated, the process600ends.

At block634, the memory manager208discards all remaining information from memory. For example, the memory manager208can discard the remaining exponential mean values, audio characteristic values and normalized bin values from memory. In other examples, the memory manager208can discard any or all of the information stored in memory.

The process ofFIG.7begins at block702. At block702, the network interface402receives the query fingerprint110. For example, the network interface402can interface with the network111to receive the query fingerprint110. In some examples, the network interface can directly communicate with the audio processor108to receive the query fingerprint.

At block704, the example query comparator408compares the first subfingerprint of the query fingerprint110with a reference subfingerprint of a candidate reference fingerprint. For example, the example query comparator408can hash the first subfingerprint to identify a matching subfingerprint of a reference fingerprint in the reference fingerprint database114. In other examples, the query comparator408can match a first subfingerprint of the query fingerprint110by any other suitable means.

At block706, the example sample rate comparator404compares the sample rate of the query fingerprint110and the candidate reference fingerprint. For example, the sample rate comparator404can compare the sample rate (e.g., the subfingerprint rate, etc.) of the query fingerprint110to the sample rate (e.g., the subfingerprint rate, etc.) of the candidate reference fingerprint. In some examples, the sample rate comparator404can determine a sample rate coefficient based on the ratio of the sample rate of the query fingerprint110to the sample rate of the reference fingerprint(s). In other examples, the example sample rate comparator404can determine the sample rate coefficient based on any other suitable means.

At block708, the example subfingerprint selector406selects subfingerprints of the reference fingerprint based on the sample rate coefficient. For example, the subfingerprint selector406can select subfingerprints of the candidate reference fingerprint corresponding to each subfingerprint of the query reference fingerprint. For example, if the sample rate coefficient is “2,” the subfingerprint selector406can select every other subfingerprint of the candidate reference fingerprint(s). For example, if the sample rate coefficient is “6,” the subfingerprint selector406can select every sixth subfingerprint of the candidate reference fingerprint(s).

At block710, the example query comparator408compares the subfingerprints of the query fingerprint110with the selected subfingerprints(s) with the selected subfingerprint(s) of the candidate reference fingerprint. For example, the query comparator408can compare the subfingerprints using hash matching, linear matching and/or any other suitable fingerprint matching techniques. In some examples, the query comparator408determines a similarity value associated with the comparison, which indicates the similarity between the query fingerprint110and the candidate reference fingerprint(s).

At block712, the query comparator408determines if the comparison indicates the fingerprints match. For example, the query comparator408can determine if the similarity value satisfies a matching threshold. If the similarity value satisfies the matching threshold, the query comparator408determines the fingerprint matches and the process700advances to block714. If the similarity value does not satisfy the matching threshold, the query comparator408determines the fingerprint matches and the process700returns to block704.

At block714, the media identifier410identifies the media associated with the query fingerprint110based on the candidate reference fingerprint. For example, the media identifier410can generate the media identification report116based on the media (e.g., the song(s), etc.) associated with the candidate reference fingerprint in the reference fingerprint database114. For example, the media identifier410can generate the media identification report116by reporting that no matches were found among the reference fingerprint database114. I In other examples, the media identifier410can identify the media associated with the query fingerprint by any other suitable means. The process700then ends.

FIG.8is a block diagram of an example processor platform800structured to execute the instructions ofFIG.6to implement the audio processor108ofFIG.2. The processor platform800can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a gaming console, a personal video recorder, a set top box, a headset or other wearable device, or any other type of computing device.

The processor platform800of the illustrated example includes a processor812. The processor812of the illustrated example is hardware. For example, the processor812can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor812implements the example audio segmenter204, the example signal transformer206, the example memory manager208, the example audio characteristic determiner210, the example mean calculator212, the example bin normalizer214, the example subfingerprint generator216and the example fingerprint generator218.

The processor812of the illustrated example includes a local memory813(e.g., a cache). The processor812of the illustrated example is in communication with a main memory including a volatile memory814and a non-volatile memory816via a bus818. The volatile memory814may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory (RDRAMR), and/or any other type of random access memory device. The non-volatile memory816may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory814,816is controlled by a memory controller.

The processor platform800of the illustrated example also includes an interface circuit820. The interface circuit820may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near field communication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices822are connected to the interface circuit820. The input device(s)822permit(s) a user to enter data and/or commands into the processor812. The input device(s)822can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), and/or a voice recognition system.

One or more output devices824are also connected to the interface circuit820of the illustrated example. The output devices824can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, a printer, and/or speaker. The interface circuit820of the illustrated example, thus, typically includes a graphics driver card, a graphics driver chip, and/or a graphics driver processor.

The interface circuit820of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and/or a network interface to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network826. The communication can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-site wireless system, a cellular telephone system, etc.

The processor platform800of the illustrated example also includes one or more mass storage devices828for storing software and/or data. Examples of such mass storage devices828include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

The machine executable instructions832to implement the methods ofFIG.6may be stored in the mass storage device828, in the volatile memory814, in the non-volatile memory816, and/or on a removable non-transitory computer readable storage medium such as a CD or DVD.

FIG.9is a block diagram of an example processor platform900structured to execute the instructions ofFIG.7to implement the fingerprint comparator112ofFIG.2. The processor platform900can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a gaming console, a personal video recorder, a set top box, a headset or other wearable device, or any other type of computing device.

The processor platform900of the illustrated example includes a processor912. The processor912of the illustrated example is hardware. For example, the processor912can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor912implements the example network interface402, the example sample rate comparator404, the example subfingerprint selector406, the example query comparator408, and the example media identifier410.

The processor912of the illustrated example includes a local memory913(e.g., a cache). The processor912of the illustrated example is in communication with a main memory including a volatile memory914and a non-volatile memory916via a bus918. The volatile memory914may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory (RDRAM®), and/or any other type of random access memory device. The non-volatile memory916may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory914,916is controlled by a memory controller.

The processor platform900of the illustrated example also includes an interface circuit920. The interface circuit920may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near field communication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices922are connected to the interface circuit920. The input device(s)922permit(s) a user to enter data and/or commands into the processor912. The input device(s)922can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), and/or a voice recognition system.

One or more output devices924are also connected to the interface circuit920of the illustrated example. The output devices924can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, a printer, and/or speaker. The interface circuit920of the illustrated example, thus, typically includes a graphics driver card, a graphics driver chip, and/or a graphics driver processor.

The interface circuit920of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and/or a network interface to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network926. The communication can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-site wireless system, a cellular telephone system, etc.

The processor platform900of the illustrated example also includes one or more mass storage devices928for storing software and/or data. Examples of such mass storage devices928include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

The machine executable instructions932to implement the methods ofFIG.6may be stored in the mass storage device928, in the volatile memory914, in the non-volatile memory916, and/or on a removable non-transitory computer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods and apparatus have been disclosed that allow fingerprints of audio signal to be created that reduces the amount noise captured in the fingerprint. Additionally, by sampling audio from less energetic regions of the audio signal, more robust audio fingerprints are created when compared to previous used audio fingerprinting methods.

Example methods, apparatus, systems, and articles of manufacture to fingerprint an audio signal via exponential normalization are disclosed herein. Further examples and combinations thereof include the following: Example 1 includes an apparatus, comprising an audio segmenter to divide an audio signal into a plurality of audio segments including a first audio segment and a second audio segment, the first audio segment including a first time-frequency bin, the second audio segment including a second time-frequency bin, a mean calculator to determine a first exponential mean value associated with the first time frequency bin based on a first magnitude of the audio signal associated with the first time frequency bin, and a second exponential mean value associated with the second time frequency bin based on a second magnitude of the audio signal associated with the second time frequency bin and the first exponential mean value, a bin normalizer to normalize the first time-frequency bin based on the second exponential mean value, and a fingerprint generator to generate a fingerprint of the audio signal based on the normalized first time-frequency bins.

Example 2 includes the apparatus of example 1, wherein the first time-frequency bin and the second time-frequency bin are in a same frequency band of the audio signal.

Example 3 includes the apparatus of example 1, wherein the bin normalizer is further to normalize the first time-frequency bin based a third exponential mean value associated with a third time-frequency bin, the third time-frequency bin in the second audio segment and proximate to the second time-frequency bin.

Example 4 includes the apparatus of example 1, further including memory, a signal transformer to transform the first audio segment into a frequency domain to thereby generate a first group of time-frequency bins including the first time-frequency bin, transform the second audio segment into the frequency domain to thereby generate a second group of time-frequency bins including the second time-frequency bin, and a memory manager to, after the second exponential mean value is determined, discard the first group of time-frequency bins from the memory.

Example 5 includes the apparatus of example 4, wherein the memory manager is further to, after mean calculator determines the second exponential mean value, discard the first exponential mean value from the memory.

Example 6 includes the apparatus of example 4, wherein each time-frequency bin of the first group of time-frequency bins is a unique combination of (1) a time period of the audio signal and (2) a frequency band of the audio signal.

Example 7 includes the apparatus of example 1, further including a subfingerprint generate to a subfingerprint by selecting energy extrema associated the first audio segment, the fingerprint including the subfingerprint.

Example 8 includes a method comprising, dividing an audio signal into a plurality of audio segments including a first audio segment and a second audio segment, the first audio segment including a first time-frequency bin, the second audio segment including a second time-frequency bin, determining a first exponential mean value associated with the first time frequency bin based on a first magnitude of the audio signal associated with the first time frequency bin, determining a second exponential mean value associated with the second time frequency bin based on a second magnitude of the audio signal associated with the second time frequency bin and the first exponential mean value, normalizing the first time-frequency bin based on the second exponential mean value, and generating a fingerprint of the audio signal based on the normalized first time-frequency bins.

Example 9 includes the method of example 8, wherein the first time-frequency bin and the second time-frequency bin are in a same frequency band of the audio signal.

Example 10 includes the method of example 8, wherein the normalization of the first time-frequency bin is further based a third exponential mean value associated with a third time-frequency bin, the third time-frequency bin in the second audio segment and proximate to the second time-frequency bin.

Example 11 includes the method of example 8, further including transforming the first audio segment into a frequency domain to thereby generate a first group of time-frequency bins including the first time-frequency bin, transforming the second audio segment into the frequency domain to thereby generate a second group of time-frequency bins including the second time-frequency bin, and discarding the first group of time-frequency bins in response to determination of the normalized the first time-frequency bin.

Example 12 includes the method of example 11, wherein each time-frequency bin of the first group of time-frequency bins is a unique combination of (1) a time period of the audio signal and (2) a frequency band of the audio signal.

Example 13 includes the method of example 8, further including discarding the first exponential mean value in response to the determination of the second exponential mean value.

Example 14 includes the method of example 10, further including generating a subfingerprint by selecting energy extrema associated the first audio segment, the fingerprint including the subfingerprint.

Example 15 includes a non-transitory computer readable medium comprising instructions, which when executed, cause a processor to divide an audio signal into a plurality of audio segments including a first audio segment and a second audio segment, the first audio segment including a first time-frequency bin, the second audio segment including a second time-frequency bin, determine a first exponential mean value associated with the first time frequency bin based on a first magnitude of the audio signal associated with the first time frequency bin, determine a second exponential mean value associated with the second time frequency bin based on a second magnitude of the audio signal associated with the second time frequency bin and the first exponential mean value, normalize the first time-frequency bin based on the second exponential mean value, and generate a fingerprint of the audio signal based on the normalized first time-frequency bins.

Example 16 includes the non-transitory computer readable medium of example 15, wherein the first time-frequency bin and the second time-frequency bin are in a same frequency band of the audio signal.

Example 17 includes the non-transitory computer readable medium of example 15, wherein the normalization of the first time-frequency bin is further based a third exponential mean value associated with a third time-frequency bin, the third time-frequency bin in the second audio segment and proximate to the second time-frequency bin.

Example 18 includes the non-transitory computer readable medium of example 15, wherein the instructions further cause the processor to transform the first audio segment into a frequency domain to thereby generate a first group of time-frequency bins including the first time-frequency bin, transform the second audio segment into the frequency domain to thereby generate a second group of time-frequency bins including the second time-frequency bin, and discard the first group of time-frequency bins in response to determination of the normalized the first time-frequency bin.

Example 19 includes the non-transitory computer readable medium of example 15, wherein the instructions further cause the processor to discard the first exponential mean value in response to the determination of the second exponential mean value.

Example 20 includes the non-transitory computer readable medium of example 15, wherein the instructions further cause the processor to generate a subfingerprint by selecting energy extrema associated the first audio segment, the fingerprint including the subfingerprint.

Although certain example methods, apparatus, and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the claims of this patent.