Patent ID: 12217730

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

“Neural beats” may include any audio beat designed to produce or encourage a desired mental state in a user. Desired mental states may include neural entrainment, improved focus, a calmer mood, relaxation, or any other desired mental state. In certain implementations, neural beats may include monaural or binaural beats that combine a lower beat frequency with a higher carrier frequency. In particular, the “beat frequency” may be selected based on a desired mental state (e.g., where different frequencies foster different types of mental states in individuals). In certain implementations, the beat frequency may range from 0.5 to 150 Hz. The “carrier frequency” may be an audio frequency or note selected to carry or audibly reproduce the beat frequency within an audio track. For example, the beat frequency may be at a lower frequency than humans can detect and/or may be at the lower range of human hearing. Therefore, to maximize the effectiveness for the neural beat, a carrier frequency may be selected and the beat frequency may be modulated onto the carrier frequency to form the neural beat. The carrier frequency may range from 207.65 to 392.00 Hz. In various implementations, neural beats may have different numbers of audio channels, such as one audio channel (e.g., monaural beats), two audio channels (e.g., binaural beats), five audio channels, or more.

Not all users enjoy listening to audio tracks that only contain neural beats, and may find them boring or distracting, limiting the effects of neural entrainment. Furthermore, the limited availability of existing audio tracks that include embedded monaural beats may not appeal to all users. Certain systems may automatically generate music that incorporates monaural beats to prevent users from having to listen to the same track multiple times. However, such systems still cannot correct for the possibility that a user will want to listen to a specific track or genre that has not been previously combined with neural beats. Therefore, there exists a need to automatically add neural beats to existing audio tracks such that users may listen to their preferred tracks or music genres while also experiencing the benefits of neural entrainment, relaxation, and/or improved focus provided by neural beats.

One solution to this problem is to analyze the pitch characteristics of a digital audio file over time. In particular, chromagram features may be generated for the digital audio file indicating the strength of different pitch classes over time within the digital audio file. This information may then be used to select a carrier frequency for a neural beat to be added to the digital audio file. For example, dominant pitch classes may be extracted from the chromagram features at various timestamps within the digital audio file and the dominant pitch classes may be used to select carrier frequencies for the neural beat at the various timestamps. In certain instances, the dominant pitch classes may be analyzed with a model (e.g., a hidden Markov model) to select the carrier frequencies to optimize the number of changes in carrier frequency. The neural beat may then be synthesized based on the beat frequency and the selected carrier frequencies and stored for later use. In certain instances, a combined audio track may be generated that combines the digital audio file with the neural beat. In other instances, the neural beat may be stored in association with the digital audio file. Furthermore, in certain instances, the neural beat and/or combined audio track may be generated in real time as a user device streams the digital audio file, such as by a server from which the digital audio file is streamed or by a user device receiving the streamed digital audio file. The neural beat may then be played alongside the digital audio file (e.g., as separate audio files played simultaneously and/or as a single audio file) via the user device.

FIG.1Aillustrates a system100according to an exemplary embodiment of the present disclosure. The system100may be configured to generate and synchronize neural beats for addition to digital audio files. The system100includes a computing device102and a server104. The server104stores digital audio files108,110to which neural beats may be added by the computing device102. For example, the computing device102and the server104may be part of a digital audio streaming platform configured to stream digital audio files106,108,110at a user's request. Furthermore, the computing device102may be configured to add neural beats168,174to digital audio files106,108,110at a user's request. For example, the user may manipulate a preference for adding neural beats168,174to streamed audio files received from the audio streaming platform.

The computing device102may receive a digital audio file106from the server104and may generate a neural beat168and/or an adjusted neural beat174to be added to the digital audio file106. The computing device102may also receive a beat frequency112for the neural beat168,174. The beat frequency112received from a user, such as via a user-configurable beat frequency setting. The neural beat168,174may be a monaural beat, a binaural beat, or may have more audio channels, and the type of neural beat168,174may be selected by a user. Additionally or alternatively, the computing device102may select between a monaural beat and a binaural beat based on the audio device from which the user is streaming digital audio files. For example, if a user is streaming audio from a mono audio device, the computing device102may generate a monaural neural beat and if the user is streaming audio from a stereo audio device (e.g., stereo speakers, stereo headphones), the computing device102may generate a binaural neural beat. In still further implementations, the computing device102may select the number of audio channels based to be the same as the number of audio channels in the digital audio file106.

The computing device102in particular may be configured to generate a neural beat168,174that blends into the digital audio file106. For example, the computing device102may be configured to generate a neural beat168,174that synchronizes with audio pitches within the digital audio file106to avoid noticeable and distracting differences in pitch, which may impede the user's neural entrainment. To do so, the computing device102may extract a plurality of chromagram features116from the digital audio file106. The chromagram features116may include pitch classes124,126and associated intensities136,138at multiple timestamps148,150.

For example,FIG.2depicts chromagram features200according to an exemplary embodiment of the present disclosure. The chromagram features200include the intensities (as defined in the legend202) for multiple pitch classes at multiple timestamps T1-T19. The pitch classes include B, A sharp/B flat, A, G sharp/A flat, G, F sharp/G flat, F, E, D sharp/E flat, D, C sharp/D flat, and C, which represent each of the types of notes that may be reproduced within a digital audio file106. In particular, each pitch class may represent all audible pitches in a song that are separated by a whole number of octaves. For example, the pitch class C may contain middle C, treble C, high C, tenor C, low C, and other octaves of the note C. Other pitch classes may similarly be defined to contain multiple notes at different octaves. In practice, the pitch classes may be defined as a collection of frequency bands. For example, the pitch class C may be defined as 261.626±0.1 Hz (for middle C), 523.251±0.1 Hz (for tenor C), and similarly for the other notes contained within the pitch class. As depicted, certain sharp or flat notes (e.g., A sharp, B flat, G sharp, A flat, F sharp, G flat, D sharp, E flat, C sharp, D flat) are grouped into separate pitch classes from the pitch classes containing natural notes A-G. In additional or alternate implementations, the pitch classes may be defined to contain sharp or flat versions of the notes. Similarly, certain implementations may define the pitch classes differently (e.g., to contain any desired combination of notes). For example, the pitch class for C may contain middle C sharp or middle C flat in an alternative implementation. It should be appreciated by one skilled in the art that the chromagram features200may be calculated according to any of a plurality of conceivable pitch classes, such as an equal temperament tuning (e.g., a 24 tone equal temperament with 24 pitch classes, a 19 tone equal temperament with 19 pitch classes, and/or a 7 tone equal temperament with 7 pitch classes). In practice, a computing device102may calculate more pitch classes than are represented in the chromagram features200and may combine this pitch classes into the desired pitch classes for the chromagram features200. For example, a computing device102may calculate 36 pitch classes that are then combined into the pitch classes depicted for the chromagram features200.

The chromagram features200include intensities for each pitch class at each of the timestamps T1-T19. These intensities change over time (e.g., as the music changes in the digital audio file106). For example, the pitch classes A and D both have high intensities from times T1-T5. From times T6-T10, the pitch class with the highest intensity alternates between C and C sharp/D flat (T8, T12), D (T9-10, T13, T17-18), D and D sharp/E flat (T6, T14), E (T7, T15), E and D sharp/E flat (T11, T19), and F (T16). These intensities may be calculated based on an analysis of the frequency domain of the digital audio file106at each of the timestamps T1-T19. For example, the computing device102may divide the digital audio file106into segments for each of the timestamps T1-T19. The computing device102may then compute a time-frequency representation (e.g., frequency distributions at multiple times) for each of the segments, (e.g., by performing a Fourier transform, a fast Fourier transform (FFT), a Constant-Q transform, a wavelets transform, using a filter bank, and the like). Frequencies in the time-frequency representation may correspond to or be categorized into each of the pitch classes (e.g., according to predefined frequency bands). The intensity for each of the pitch classes may then be calculated based on the intensity of the corresponding frequencies within the time-frequency representation. This process may be repeated multiple times for the segments corresponding to each of the timestamps T1-19. In certain implementations, the timestamps T1-19may occur every 50 milliseconds. In additional or alternative implementations, the timestamps T1-19may occur more frequently (e.g., every 10 milliseconds, every 5 milliseconds, every millisecond) and/or less frequently (e.g., every 0.5 seconds, every 0.25 seconds, 0.1 seconds). In certain implementations, rather than performing a frequency domain analysis of the digital audio file106, the computing device102may perform an analysis in the time domain. For example, a filter bank may be used with one or more filters for each pitch class. An intensity for the resulting, filtered signal at each timestamp may then be used to determine the intensities for the chromagram features200.

Returning toFIG.1A, the computing device102may compute multiple chromagram features116for the digital audio file106. For example, multiple chromagram features116may be calculated to focus on different frequency ranges within the digital audio file106. As one specific example, a first set of chromagram features may be calculated focusing on a lower frequency range within the digital audio file106(e.g., less than C4, or 261.62 Hz) and a second set of chromagram features may be calculated focusing on a higher frequency range (e.g., C1 to C8, or 32.70 Hz to 4186.01 Hz). In such instances, the computing device102may then be configured to combine multiple chromagram features116into a set of primary chromagram features118for the digital audio file106. For example, the computing device102may linearly combine the chromagram features116(e.g., according to predefined weights) to form the primary chromagram features118. The data structure for the primary chromagram features118may be comparable to that of the chromagram features116. For example, in certain implementations, the chromagram features200may represent a set of primary chromagram features118for the digital audio file106. Furthermore, it should be understood that, althoughFIG.2depicts the chromagram features200as a plot of data over time, in practice, the chromagram features116and/or primary chromagram features118may be stored in additional or alternative data structures. For example, the chromagram features116and/or the primary chromagram features118may be stored as an array containing the intensity values for the pitch classes at the timestamps T1-19.

The computing device102may identify dominant pitch classes120based on the primary chromagram features118. In particular, the computing device102may calculate a probability distribution144,146that each of the pitch classes132,134of the dominant pitch class for a particular timestamps156,158. For example,FIG.3illustrates dominant pitch classes300according to an exemplary embodiment of the present disclosure. The dominant pitch classes300include a probability (as defined in the legend302) for each of the pitch classes B, A sharp/B flat, A, G sharp/A flat, G, F sharp/G flat, F, E, D sharp/E flat, D, C sharp/D flat, C at each of the timestamps T1-19. In particular, at times T1-T5, the pitch classes A and D have medium-high probabilities, at times T6and T14, the pitch classes D and D sharp/E flat have medium-high probabilities, at times T7, T11, and T19, the pitch classes E and D sharp/E flat have medium-high probabilities, at times T8and T12, the pitch classes C and C sharp/D flat have medium-high probabilities, at times T9, T10, T13, T17, and T18, has a high probability, at time T15the pitch class E has a high probability, and at time T16, the pitch class F has a high probability. The probabilities may be calculated to reflect a probability that each pitch class represents the dominant pitch class at the given point in time. For example, in certain instances, the probabilities may be calculated by a Hidden Markov Model (HMM). In certain instances, the HMM may be tuned to optimize the number of transitions in dominant pitch class (e.g., to optimize the number of changes in carrier frequency for the neural beat168,174), which a user may find distracting and/or which may adversely affect neural entrainment.

Returning toFIG.1A, the computing device102may then determine carrier frequencies114based on the dominant pitch classes120. The carrier frequencies114may include a single, selected frequency160,162at each timestamp164,166to serve as the carrier frequency at that time within the neural beat168,174. For example,FIG.4depicts carrier frequencies400according to an exemplary embodiment of the present disclosure. The carrier frequencies400include a single selected pitch class at each timestamp T1-19. In particular, the pitch class D is selected as the carrier frequency for timestamps T1-14and T17-19and the pitch class E is selected for timestamps T15-16. The carrier frequencies may be selected to follow the musical harmonies of the digital audio file while also avoiding unnecessary changes in carrier frequency. In particular, the carrier frequency at times T9-T10and T17-20may be selected as pitch class D to align with the dominant pitch class at these times. However, excessive changes in carrier frequency may be distracting to a user, so the selected carrier frequencies may be selected to maintain consistency over time in certain instances, such as when selecting between different pitch classes with similar probabilities or small, brief changes in the dominant pitch class. For example, in the dominant pitch classes300, the pitch classes A and D had similar probabilities at times T1-5. However, the pitch class D may be selected as the carrier frequency from times T1-5to avoid a transition from the pitch class A to the pitch class D at time T6, where the pitch class D is dominant. As another example, at times T7, T11, T19, the pitch classes D sharp/E flat and E both have similar probabilities. However, the pitch class D may be selected as the carrier frequency, even though it does not have the highest probability at these times, to reduce the number of changes in carrier frequency (e.g., because the pitch class D still has a medium probability in the dominant pitch classes300). On the other hand, failing to follow musical harmonies may also adversely affect neural entrainment. Thus, at times T17-T19, the carrier frequency switches from E (at time T16) to D (at times T17-T19) to properly follow the harmonies in the digital audio file.

To select the carrier frequencies400, the computing device102may be configured to balance maximizing the overall probability of selected carrier frequencies while limiting the number of changes in consecutive dominant pitch classes. In certain implementations, the computing device102may perform a Viterbi decoding on the dominant pitch classes300to find the most likely sequence of individual pitch classes at each timestamp that constrains the number of carrier frequency transitions while also ensuring that the carrier frequencies400align musically with the digital audio file106.

Returning toFIG.1A, the computing device may then synthesize the neural beat168based on the beat frequencies112and the carrier frequencies114. In particular, the computing device102may synthesize the neural beat168by modulating the beat frequency112onto the selected carrier frequencies160,162at each of the timestamps164,166. In certain implementations, the timestamps164,166within the carrier frequencies (e.g., timestamps T1-19) may correspond to timestamps for audio data within the digital audio file106. In such instances, the computing device102may synthesize the neural beat168directly based on the carrier frequencies at each of the timestamps164,166.

In certain implementations, the computing device102may further adjust one or more aspects of the neural beat168based on further characteristics of the digital audio file106. For example, the computing device102may adjust a volume of the neural beat168to align with changes in volume for the digital audio file106. In particular, if the neural beat168is relatively quiet compared to the digital audio file106, the benefits of the neural beat may be diminished. Additionally or alternatively, where the neural beat168is loud relative to the digital audio file106, the neural beat168may prove disruptive or distracting for the user, interrupting the benefits provided by the neural beat168. Accordingly, an audio mixer122may be used to adjust the volume of the neural beat168over the course of the digital audio file106.

In particular, the audio mixer122may determine a loudness profile170of the digital audio file106. The loudness profile170may be a representation of how loud the digital audio file106is over time (e.g., throughout the duration of the digital audio file106). The loudness profile170may be computed as a combined intensity (e.g., across audible frequencies) at multiple timestamps within the digital audio file106. The loudness profile170may then be used to generate a volume curve172for the neural beat168. In particular, the loudness profile170may be offset (e.g., according to a maximum desired intensity for the neural beat168) to generate the volume curve172. For example,FIG.5depicts a volume curve500according to an exemplary embodiment of the present disclosure. The volume curve500shows changes in energy (in dB) over the duration of a digital audio file106, where the energy of the audio signals within the digital audio file106may be used as a proxy for volume over time within the digital audio file106. Returning toFIG.1A, the volume curve172may be applied to the neural beat168to generate an adjusted neural beat174. In particular, applying the volume curve172to the neural beat168may include increasing or decreasing the volume (e.g., the intensity) of the neural beat168at different points in time according to the intensities indicated in the volume curve172(e.g., so that the adjusted neural beat174is louder at times of high intensity in the volume curve172and quieter at times of low intensity in the volume curve172).

The neural beat168and/or the adjusted neural beat174may then be stored, transmitted, and/or played back on a user's device. For example, the computing device102may store the neural beat168and/or the adjusted neural beat174in association with the digital audio file106(e.g., in the server104). In certain implementations, the digital audio file106and the neural beat168and/or adjusted neural beat174may be stored separately. In additional or alternative implementations, the computing device102may combine the digital audio file106with the neural beat168and/or adjusted neural beat174to generate a combined audio track that may be stored (e.g., in the server104). As another example, and referring toFIG.1Band the system190, the digital audio file106and the neural beat168and/or adjusted neural beat174may be transmitted to a user device192associated with a user194. The user device162may include a smartphone, tablet computer, wearable computing device, laptop, personal computer, or any other personal computing device. The user device192may also include one or more audio devices for audio playback, such as a speaker, a 3.5 mm audio jack connected to headphones or a speaker, wirelessly-connected headphones, wirelessly-connected speaker(s), or any other device capable of audio playback. The system100may transmit (e.g., stream) the digital audio file106and the neural beat168and/or adjusted neural beat174to the user device192. The user device192may then receive and play back the digital audio file106at the same time as the neural beat168and/or adjusted neural beat174. Additionally or alternatively, the user device192may store the digital audio file106and the neural beat168and/or adjusted neural beat174for future playback. Additionally or alternatively, the computing device102may transmit a combined audio track to the user device192. In still further implementations, the neural beat168and/or the adjusted neural beat174may be generated on the user device192. In such instances, the neural beat168and/or the adjusted neural beat174may be played along with the digital audio file106on the user device192(e.g., as separate audio files, as a combined audio track) and/or may be stored on the user device192for future playback at a later time.

Although not depicted, the computing device102, the server104, and/or the user device192may contain at least one processor and/or memory configured to implement one or more aspects of the computing device102, the server104, and/or the user device192. For example, the memory may store instructions which, when executed by the processor, may cause the processor to perform one or more operational features of the computing device102, the server104, and/or the user device192. The processor may be implemented as one or more central processing units (CPUs), field programmable gate arrays (FPGAs), and/or graphics processing units (GPUs) configured to execute instructions stored on the memory. Additionally, the computing device102, the server104, and/or the user device192may be configured to communicate using a network. For example, the computing device102, the server104, and/or the user device192may communicate with the network using one or more wired network interfaces (e.g., Ethernet interfaces) and/or wireless network interfaces (e.g., Wi-Fi®, Bluetooth®, and/or cellular data interfaces). In certain instances, the network may be implemented as a local network (e.g., a local area network), a virtual private network, L1 and/or a global network (e.g., the Internet).

In certain implementations, the computing device102and the server104may be implemented as a single computing device. For example, the computing device102may store the digital audio files106,108,110(e.g., in a local database). In further implementations, the computing device102and/or the server104may be at least partially implemented by the user device162. In still further implementations, the computing device102, the server104, and/or the user device192may be implemented by multiple computing devices. For example, the computing device102may be implemented as multiple software services executing in a distributed computing environment (e.g., a cloud computing environment). As another example, the user device162may be implemented by multiple personal computing devices (e.g., a smartphone and a wearable computing device such as a smartwatch).

FIG.6illustrates a method600for synthesizing a neural beat according to an exemplary embodiment of the present disclosure. The method600may be implemented on a computer system, such as the systems100,160. For example, the method600may be implemented by the computing device102and/or the user device192. The method600may also be implemented by a set of instructions stored on a computer readable medium that, when executed by a processor, cause the computer system to perform the method600. For example, all or part of the method600may be implemented by a processor and/or a memory of the computing device102and/or the user device192. Although the examples below are described with reference to the flowchart illustrated inFIG.6, many other methods of performing the acts associated withFIG.6may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, one or more of the blocks may be repeated, and some of the blocks described may be optional.

The method600may begin with receiving a digital audio file and a beat frequency for a neural beat to be added to the digital audio file (block602). For example, the computing device102may receive a digital audio file106and a beat frequency112fora neural beat to be added to the digital audio file106. As explained above, the computing device102may receive the digital audio file106from a server104and/or may retrieve the digital audio file106from a local storage. In certain implementations, the digital audio file106may be received according to a user request. For example, a user request may be received from a user device to play back a particular song (e.g., via a music streaming service). The computing device102may receive the beat frequency112from a user (e.g., according to a user request and/or a previously-defined user setting). In certain implementations, the beat frequency112may specify a particular frequency (e.g., 3 Hz) for the neural beat to be added to the digital audio file106. In additional or alternative implementations, the beat frequency112may specify a range of frequencies (e.g., 4-8 Hz) for the neural beat.

A plurality of chromagram features may be extracted from the digital audio file (block604). For example, the computing device102may extract a plurality of chromagram features116,200from the digital audio file106. As explained above, the chromagram features may include intensity information for multiple pitch classes at multiple timestamps within the digital audio file106. In certain implementations, each of the plurality of chromagram features may be extracted according to different parameters applied to the digital audio file106prior to extracting the chromagram features116,200. For example, first chromagram features may be extracted focusing on the lower frequencies of the digital audio file106and second chromagram features may be extracted focusing on higher frequencies of the digital audio file106. As another example, three chromagram features may be extracted from the digital audio file106: first chromagram features focusing on lower frequencies (e.g., less than 200 Hz), second chromagram features focusing on mid-level frequencies (e.g., from 200 Hz-800 Hz), and third chromagram features focusing on higher frequencies (e.g., greater than 800 Hz). In practice, the plurality of chromagram features116,200may be generated by selecting octaves and intensities within the desired frequency ranges for inclusion in the chromagram features116,200after generating the time-frequency representation as discussed above. In other implementations, the plurality of chromagram features may be generated by applying a filter (e.g., a high-pass filter, a low-pass filter, a bandpass filter, and the like) to the digital audio file106prior to extracting the chromagram features116,200(e.g., using an FFT, a constant-Q transform, filter buckets and/or other techniques, as discussed above).

The plurality of chromagram features may be combined to form primary chromagram features of the digital audio file (block606). For example, the computing device102may combine the plurality of chromagram features116,200to form primary chromagram features118of the digital audio file106. In certain implementations, the plurality of chromagram features116,200may be linearly combined to form the primary chromagram features118(e.g., according to previously-defined weights). In additional or alternative implementations, the plurality of chromagram features116,200may be combined according to any other conceivable combination strategy. For example, the plurality of chromagram features116,200may be combined by “stacking” the chromagram features116,200(e.g., so that combining two chromagram features116,200with 12 pitch classes forms primary chromagram features with 24 rows). Generating the primary chromagram features118based on a plurality of chromagram features116may better capture the audio frequency characteristics of the digital audio file106(e.g., by separately focusing on different frequency ranges, such as different octaves, within the digital audio file106). In certain implementations, one or both of blocks604,606may be omitted. For example, in certain implementations, rather than extracting multiple chomagram features and combining them to form the primary chromagram features, a single set of chromagram features may be extracted from the digital audio file106and may be used as the primary chromagram features118.

Dominant pitch classes may be extracted at a plurality of timestamps within the digital audio file (block608). For example, the computing device102may extract dominant pitch classes120,300at a plurality of timestamps156,158within the digital audio file106. The dominant pitch classes120,300may be extracted from the primary chromagram features118using a model, such as a hidden Markov model. In particular, the dominant pitch classes120,300may be extracted as a probability distribution at multiple timestamps T1-19. The timestamps T1-19may be selected based on the timestamps of the primary chromagram features118, as explained above.

A plurality of carrier frequencies may be selected for the neural beat (block610). For example, the computing device102may select a plurality of carrier frequencies114,400for the neural beat168,174. In particular, the plurality of carrier frequencies114,400may include individual carrier frequencies160,162at multiple timestamps164,166, T1-19. The selected carrier frequencies114,400may be selected by a Viterbi process, which may select carrier frequencies such that transitions in carrier frequency at adjacent time periods are optimized according to a transition probability, as explained further herein. In certain implementations, in addition to selecting the plurality of carrier frequencies114,400, a particular beat frequency for the neural beat168may be selected. For example, where the beat frequency112is received as a range of acceptable frequencies, the computing device102may select a beat frequency for the neural beat168from within the acceptable range, as discussed further below.

A synchronized beat may be synthesized for the digital audio file based on the beat frequency and the plurality of carrier frequencies (block612). For example, the computing device102may synthesize a neural beat168for the digital audio file106based on the beat frequency112and the carrier frequencies114. In particular, the neural beat168may be generated by modulating the beat frequency112on two different carrier frequencies160,162at times corresponding to the timestamps164,166, T1-19within the carrier frequencies114,400. In this way, the neural beat168may be synchronized to the changes of musical harmony and/or melody at different time periods within the digital audio file106. In certain implementations, the neural beat168may be synthesized to contain a single audio channel (e.g., as a monaural beat). In additional or alternative implementations, the neural beat168may be synthesized to contain two audio channels (e.g., as a binaural beat with two channels, as a monaural beat with two channels). In still further implementations, the neural beat168may be synthesized to contain more than two audio channels (e.g., three audio channels, four audio channels, five audio channels). In certain implementations, the number of audio channels may be specified by a user or a predetermined setting. In additional or alternative implementations, the number of audio channels may be selected based on the number of audio channels in the digital audio file106(e.g., such that the neural beat168has the same number of audio channels as the digital audio file106).

At least one of the synchronized neural beat and a combined audio track that combines the synchronized neural beat and the digital audio file may be stored (block614). For example, the computing device102may store at least one of the synchronized neural beat168or a combined audio track combining the neural beat168with the digital audio file106. For example, as explained above, the computing device102may store the neural beat168and/or the combined audio track on the server104and/or a local storage within the computing device102. Additionally or alternatively, the computing device102may transmit the neural beat168and/or the combined audio track to a user device for storage and playback (e.g., temporary storage for streaming, long-term storage). In implementations where the computing device102is a user device, the computing device102may store the neural beat168and/or the combined audio track locally for current or future playback. In certain implementations, as explained further above, the computing device102may be further configured to generate an adjusted neural beat174based on the neural be168. In such instances, the computing device102may be configured to store the equalize neural beat174and/or a combined audio track that combines the adjusted neural beat174with the digital audio file106in ways similar to those discussed above.

In this way, the method600enables computing devices to generate neural beats for an arbitrary digital audio file, allowing for increased user selection in the types of music that are used to produce neural entrainment. Furthermore, the computing device is able to do so in real time and may ensure that the neural beat blends with the tonal qualities of the digital audio file and/or the loudness of the digital audio file to minimize user distraction and maximize neural entrainment. Accordingly, the method600ensures that generated neural beats combine constructively with previously-created digital audio files.

FIGS.7A-7Cillustrate methods700,710,720according to an exemplary embodiment of the present disclosure. The methods700,710,720may be performed in combination with at least a portion of the method600. For example, the method700may be performed while implementing blocks608,610of the method600. As another example, the method710may be performed between blocks612and614and/or part of block612of the method600. As a further example, the method720may be performed as part of the block612of the method600. The methods700,710,720may be implemented on a computer system, such as the systems100,190. For example, the methods700,710,720may be implemented by the computing device102and/or the user device192. The methods700,710,720may also be implemented by a set of instructions stored on a computer readable medium that, when executed by a processor, cause the computer system to perform the methods700,710,720. For example, all or part of the methods700,710,720may be implemented by a processor and/or a memory of the computing device102and/or the user device192. Although the examples below are described with reference to the flowchart illustrated inFIGS.7A-7C, many other methods of performing the acts associated withFIG.7A-7Cmay be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, one or more of the blocks may be repeated, and some of the blocks described may be optional.

The method700may be performed to select the plurality of carrier frequencies for the neural beat. The method700may begin generating a probability distribution for pitch classes at a plurality of timestamps (block702). For example, a hidden Markov model may be used to generate a probability distribution for pitch classes (e.g., B, A sharp/B flat, A, G sharp/A flat, G, F sharp/G flat, F, E, D sharp/E flat, D, C sharp/D flat, and C pitch classes) within the digital audio file106at multiple timestamps T1-19within the digital audio file106. The timestamps T1-19may be selected based on timestamps within the primary chromagram features118(e.g., based on the segments of the digital audio file106used to calculate a time-frequency representation for the chromagram features116and/or the primary chromagram features118). The hidden Markov model may be configured by adjusting a transition probability to select when a transition between different carrier frequencies should occur. In particular, a transition probability (e.g., a transition probability of 0.005-0.02) of the hidden Markov model may have been previously received (or may be updated) based on input received from the user, a system administrator, and/or a computing process.

A sequence of dominant pitch classes may then be identified within the probability distribution (block704). For example, the computing device102may identify a sequence of dominant pitch classes within the probability distribution. In particular, the carrier frequencies114,400may contain a series of dominant pitch classes to be used as carrier frequencies for the neural beat168. The sequence of dominant pitch classes may be identified to maximize the combined probability of selected pitch classes within the probability distribution according to a constrained transition probability for changes in selected pitch classes. In particular, the sequence of dominant pitch classes may be selected by a Viterbi process implemented by the computing device102.

In this way, the method700may be performed to select a sequence of carrier frequencies based on the musical harmonies and melodies (e.g., chromagram features) of a received digital audio file. Accordingly, this process enables a neural beat168to be applied to existing digital audio files while also ensuring that changes in carrier frequency do not disrupt or distract users seeking to trigger neural entrainment using the neural beat.

The method710may be performed to adjust the volume of the neural beat168based on the volume of the digital audio file106at different times within the digital audio file106. The method710may begin with generating a loudness profile for the duration of the digital audio file (block712). For example, the computing device102(e.g., the audio mixer122) may generate a loudness profile170for the duration of the digital audio file106. The loudness profile170may be generated based on an intensity (e.g., audio volume) of the digital audio file106at multiple times within the digital audio file106. For example, the loudness profile170may be generated for each data sampling timestamp within the digital audio file106.

A volume curve may be formed based on the loudness profile (block714). For example, the computing device102may form a volume curve172based on the loudness profile170. The volume curve172may be formed as a percentage of the loudness profile170(e.g., 50% of the loudness profile170). Additionally or alternatively, the volume curve172may be formed by normalizing the loudness profile170for a maximum volume desired for the neural beat168). One skilled in the art may similarly recognize one or more additional means of generating a volume curve172based on a loudness profile170for a digital audio file106. All such similar implementations are hereby considered within the scope of the present disclosure.

The volume of the synchronized neural beats may then be adjusted according to the volume curve (block716). For example, the computing device102may adjust the volume of the neural beat168based on the volume curve172to generate an adjusted neural beat174. For example, the neural beat168may be scaled in intensity to match the desired volume reflected in the volume curve172.

In this way, the method710may be performed to adjust the neural beat168. This may reduce the number of intrusive volume mismatches between the neural beat in the digital audio file. For example, where the neural beats is much lower in volume and the digital audio file, a user may not be able to hear the volume of the neural beat, reducing its effectiveness in producing neural entrainment. As another example, where the neural beat is much higher in volume than the digital audio file106, a user may be distracted or disrupted by the difference in volume, interrupting or reducing any neural entrainment produced by the neural beat.

The method720may be used to synchronize the neural beat168with the rhythmic patterns in the digital audio file106. The method720may begin with estimating positions of rhythmic beats within the digital audio file (block722). For example, the computing device102may estimate positions of rhythmic beats within the digital audio file106. Positions for the rhythmic beats within the digital audio file106may be estimated using a machine learning model, such as a pre-trained network configured to detect rhythmic beats within audio files. For example, positions of the rhythmic beats may be estimated using one or more models analogous to those offered by the madmom audio software package, the Essentia audio software package, and the like. In additional or alternative implementations, positions for the rhythmic beats may be estimated using one or more algorithmic techniques.

Timing for the synchronized neural beat may be adjusted based on positions of the rhythmic beats within the digital audio file (box724). For example, the computing device102may adjust timing for the neural beat168based on the positions of the rhythmic beats. For example, the computing device102may adjust the beat frequency112to align with (e.g., to be a multiple of) the tempo of the digital audio file. For example, where the digital audio file106has a tempo of 120 bpm and the beat frequency112is 0.6 Hz (e.g., 100 bpm), the computing device102may adjust the beat frequency112to be an integer multiple of the 120 beats per minute (e.g., 2 Hz) tempo. As a specific example, the computing device102may adjust the beat frequency112to be 0.5 Hz (30 bpm) and/or 1 Hz (60 bpm). In implementations where a user has specified a desired frequency range for the beat frequency112, the beat frequency112may be selected from within the desired frequency range to be an even multiple of the rhythmic frequency and/or as close as possible to a multiple of the rhythmic frequency. In addition, the timing for the synchronized neural beat may be adjusted such that peak values in the neural beat (e.g., peak values at the beat frequency112) occur at the same time as (e.g., align with the timing of) rhythmic beats within the digital audio file106.

In this way, the method720may be used to ensure that the rhythmic beats within the digital audio file and the beat frequency are not out of phase. In particular, when a beat frequency is out of phase with the rhythmic frequency of a digital audio file, interferences between the beat frequency in the digital audio file may negatively impact the sound quality and/or may create distracting or disruptive interference patterns when the digital audio file and a neural beat at the interfering beat frequency are played at the same time. Accordingly, adjusting the beat frequency based on the rhythmic beats within the digital audio file may reduce these interferences, improving the quality of the subsequently-generated neural beat and/or the quality of neural entrainment produced by the neural beat.

FIG.8illustrates an example computer system800that may be utilized to implement one or more of the devices and/or components discussed herein, such as the computing device102. In particular embodiments, one or more computer systems800perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems800provide the functionalities described or illustrated herein. In particular embodiments, software running on one or more computer systems800performs one or more steps of one or more methods described or illustrated herein or provides the functionalities described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems800. Herein, a reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, a reference to a computer system may encompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems800. This disclosure contemplates the computer system800taking any suitable physical form. As example and not by way of limitation, the computer system800may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, the computer system800may include one or more computer systems800; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems800may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example and not by way of limitation, one or more computer systems800may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems800may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

In particular embodiments, computer system800includes a processor806, memory804, storage808, an input/output (I/O) interface810, and a communication interface812. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

In particular embodiments, the processor806includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, the processor806may retrieve (or fetch) the instructions from an internal register, an internal cache, memory804, or storage808; decode and execute the instructions; and then write one or more results to an internal register, internal cache, memory804, or storage808. In particular embodiments, the processor806may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates the processor806including any suitable number of any suitable internal caches, where appropriate. As an example and not by way of limitation, the processor806may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory804or storage808, and the instruction caches may speed up retrieval of those instructions by the processor806. Data in the data caches may be copies of data in memory804or storage808that are to be operated on by computer instructions; the results of previous instructions executed by the processor806that are accessible to subsequent instructions or for writing to memory804or storage808; or any other suitable data. The data caches may speed up read or write operations by the processor806. The TLBs may speed up virtual-address translation for the processor806. In particular embodiments, processor806may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates the processor806including any suitable number of any suitable internal registers, where appropriate. Where appropriate, the processor806may include one or more arithmetic logic units (ALUs), be a multi-core processor, or include one or more processors806. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

In particular embodiments, the memory804includes main memory for storing instructions for the processor806to execute or data for processor806to operate on. As an example, and not by way of limitation, computer system800may load instructions from storage808or another source (such as another computer system800) to the memory804. The processor806may then load the instructions from the memory804to an internal register or internal cache. To execute the instructions, the processor806may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, the processor806may write one or more results (which may be intermediate or final results) to the internal register or internal cache. The processor806may then write one or more of those results to the memory804. In particular embodiments, the processor806executes only instructions in one or more internal registers or internal caches or in memory804(as opposed to storage808or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory804(as opposed to storage808or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple the processor806to the memory804. The bus may include one or more memory buses, as described in further detail below. In particular embodiments, one or more memory management units (MMUs) reside between the processor806and memory804and facilitate accesses to the memory804requested by the processor806. In particular embodiments, the memory804includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory804may include one or more memories804, where appropriate. Although this disclosure describes and illustrates particular memory implementations, this disclosure contemplates any suitable memory implementation.

In particular embodiments, the storage808includes mass storage for data or instructions. As an example and not by way of limitation, the storage808may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The storage808may include removable or non-removable (or fixed) media, where appropriate. The storage808may be internal or external to computer system800, where appropriate. In particular embodiments, the storage808is non-volatile, solid-state memory. In particular embodiments, the storage808includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage808taking any suitable physical form. The storage808may include one or more storage control units facilitating communication between processor806and storage808, where appropriate. Where appropriate, the storage808may include one or more storages808. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

In particular embodiments, the I/O Interface810includes hardware, software, or both, providing one or more interfaces for communication between computer system800and one or more I/O devices. The computer system800may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person (i.e., a user) and computer system800. As an example and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, screen, display panel, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. Where appropriate, the I/O Interface810may include one or more device or software drivers enabling processor806to drive one or more of these I/O devices. The I/O interface810may include one or more I/O interfaces810, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface or combination of I/O interfaces.

In particular embodiments, communication interface812includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system800and one or more other computer systems800or one or more networks814. As an example and not by way of limitation, communication interface812may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or any other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a Wi-Fi network. This disclosure contemplates any suitable network814and any suitable communication interface812for the network814. As an example and not by way of limitation, the network814may include one or more of an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system800may communicate with a wireless PAN (WPAN) (such as, for example, a Bluetooth® WPAN), a Wi-Fi network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or any other suitable wireless network or a combination of two or more of these. Computer system800may include any suitable communication interface812for any of these networks, where appropriate. Communication interface812may include one or more communication interfaces812, where appropriate. Although this disclosure describes and illustrates a particular communication interface implementations, this disclosure contemplates any suitable communication interface implementation.

The computer system802may also include a bus. The bus may include hardware, software, or both and may communicatively couple the components of the computer system800to each other. As an example and not by way of limitation, the bus may include an Accelerated Graphics Port (AGP) or any other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-PIN-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local bus (VLB), or another suitable bus or a combination of two or more of these buses. The bus may include one or more buses, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other types of integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.

All of the disclosed methods and procedures described in this disclosure can be implemented using one or more computer programs or components. These components may be provided as a series of computer instructions on any conventional computer readable medium or machine readable medium, including volatile and non-volatile memory, such as RAM, ROM, flash memory, magnetic or optical disks, optical memory, or other storage media. The instructions may be provided as software or firmware, and may be implemented in whole or in part in hardware components such as ASICs, FPGAs, DSPs, or any other similar devices. The instructions may be configured to be executed by one or more processors, which when executing the series of computer instructions, performs or facilitates the performance of all or part of the disclosed methods and procedures.

It should be understood that various changes and modifications to the examples described here will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present subject matter and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims.