Patent Description:
Applications may want to use a loopback audio stream of the audio data being rendered to a speaker for various processing. For example, applications may want to use a loopback audio stream for echo cancelation or recording audio for the application. When multiple devices are communicating with a computer device, there may be a plurality of loopback streams an application may need to access for the processing and it may become cumbersome for the application to determine which loopback stream to access.

Thus, there is a need in the art for improvements for audio loopback on devices.

<CIT> discloses a processing system comprising an operating system which provides an interface to hardware functions for applications of an application layer. An audio loopback function of the processing system provides audio loopback data to the application layer where the audio loopback data is audio output data for outputting on a hardware output port.

The following presents a simplified summary of one or more implementations of the present disclosure in order to provide a basic understanding of such implementations. This summary is not an extensive overview of all contemplated implementations, and is intended to neither identify key or critical elements of all implementations nor delineate the scope of any or all implementations. Its sole purpose is to present some concepts of one or more implementations of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

One example implementation relates to a computer device. The computer device may include a memory to store data and instructions, at least one processor configured to communicate with the memory, and an operating system configured to receive a plurality of hardware loopback streams with rendered audio data from a plurality of display devices in communication with the computer device, wherein each hardware loopback stream of the plurality of hardware loopback streams corresponds to one display device of the plurality of display devices; combine the rendered audio data from the plurality of hardware loopback streams into a loopback buffer to create aggregated loopback audio data; and provide the loopback buffer with the aggregated loopback audio data to one or more applications executing on the computer device.

Another example implementation relates to a method for aggregating hardware loopback streams of a plurality of display devices in communication with a computer device. The method may include receiving, at an operating system executing on the computer device, a plurality of hardware loopback streams with rendered audio data from the plurality of display devices in communication with the computer device, wherein each hardware loopback stream of the plurality of hardware loopback streams corresponds to one display device of the plurality of display devices. The method may include combining the rendered audio data from the plurality of hardware loopback streams into a loopback buffer to create aggregated loopback audio data. The method may include providing the loopback buffer with the aggregated loopback audio data to one or more applications executing on the computer device.

Another example implementation relates to computer-readable medium storing instructions executable by a computer device. The computer-readable medium may include at least one instruction for causing the computer device to receive a plurality of hardware loopback streams with rendered audio data from a plurality of display devices in communication with the computer device, wherein each hardware loopback stream of the plurality of hardware loopback streams corresponds to one display device of the plurality of display devices. The computer-readable medium may include at least one instruction for causing the computer device to combine the rendered audio data from the plurality of hardware loopback streams into a loopback buffer to create aggregated loopback audio data. The computer-readable medium may include at least one instruction for causing the computer device to provide the loopback buffer with the aggregated loopback audio data to one or more applications executing on the computer device.

Additional advantages and novel features relating to implementations of the present disclosure will be set forth in part in the description that follows, and in part will become more apparent to those skilled in the art upon examination of the following or upon learning by practice thereof.

This disclosure relates to devices and methods for aggregating hardware loopback streams received from a plurality of display devices in communication with a computer device and representing the aggregated hardware loopback streams in a single loopback buffer. As used herein, a hardware loopback stream includes audio data being rendered to the speakers on the display devices for output. The methods and devices may perform additional processing on the audio data before storing the rendered audio data in the hardware loopback streams in the loopback buffer. The additional processing occurs prior to the output of the audio via the speakers. The additional processing may include, but is not limited to, equalization, filtering, pitch shift, and/or any other effect on the audio data. As such, the hardware loopback streams accurately represent the audio coming out of the speakers on the display devices. The methods and devices use a hardware loopback stream instead of copying the output signal sent to the speakers as, for example, currently performed by software loopback streams. Software loopback streams may not accurately represent the audio coming out of the speakers on the display devices since the software loopback streams may capture the audio data before the audio data is rendered to the speakers for output and/or prior to any additional processing that may occur on the audio data prior to output. As such, using the hardware loopback streams by applications may be more efficient than using software loopback streams since the hardware loopback streams may provide a more accurate representation of the sound produced by the speakers on the display devices relative to the software loopback streams.

The plurality of display devices may communicate with the computer device via a network or bus protocol, such as, for example, a universal serial bus (USB) connection. The display devices may have at least one display and corresponding audio inputs and/or audio outputs. The display devices may be any type of display, monitor, visual presentation device, computer device, and/or physical panel capable of presenting information, capturing audio, and/or emitting audio. In addition, each display device may include any number of channels for capturing audio and/or emitting audio. Each speaker and/or microphone of the display devices may correspond to any number of channels. The computer device and/or applications running on the computer device may then access the stored aggregated hardware loopback stream to improve one or more operations or outputs associated with the aggregated hardware loopback stream. For example, in a system that captures hardware loopback data in the form of audio data, the improved operations or outputs may include, but are not limited to, performing echo cancelation and/or obtaining an accurate recording of the audio played by a particular output device (also referred to as a render device).

The display devices may have loopback capabilities within the hardware of the display device that enable the audio data being rendered to the speakers of the display devices to be provided in a hardware loopback stream. The audio data rendered to the speakers may include the audio data for output via the speakers on the display devices. The display devices may include hardware loopback units to receive the audio data currently being rendered to a render device, e.g., a speaker, of the display device. The hardware loopback units may create a hardware loopback stream for the rendered audio data. A hardware loopback stream with the rendered audio data may be stored in the display device loopback buffer. As such, the hardware loopback stream stored on the display device may accurately represent the audio output from the speakers on the display device.

One example may include a person using a microphone on a SKYPE call with a friend while some background music is playing on the speaker. In this case the audio captured from microphone on the SKYPE call may also include the audio from the background music if it is in close proximity to the speaker. Using the hardware loopback stream, the speaker audio (e.g., background music) may be cancelled from the microphone audio so that the friend on the SKYPE call does not hear the background music in the call.

The computer device receives the hardware loopback streams with the audio data from a plurality of display devices. The audio aggregator driver may aggregate the audio data from each of the received hardware loopback streams into a single loopback buffer. The size of the loopback buffer is determined by multiplying a total number of display devices in communication with the computer device by a number of audio channels supported by each display device. Each of the hardware loopback streams from the different display devices may be combined together into a single loopback buffer. The audio aggregator driver may provide the single loopback buffer to one or more applications associated with computer device.

Applications associated with the computer device may want to use the aggregated loopback audio data from the display devices for additional processing, such as, but not limited to, acoustic echo cancellation and/or recording audio coming out of the speakers. At any time applications may access the loopback buffer and may use the same audio output by the speakers on the display device by accessing the corresponding hardware loopback stream containing the audio data. By using hardware loopback, a better representation of the audio data rendered to the display devices speakers may be recorded in the hardware loopback streams, aggregated in the single loopback buffer, and used by the applications.

Referring now to <FIG>, is a system <NUM>, covered by the scope of the appended claims, for aggregating hardware loopback streams from a plurality of display devices <NUM>, <NUM>, <NUM>, <NUM> includes a computer device <NUM> in communication with the display devices via a wired or wireless network <NUM>. In an implementation, display devices <NUM>, <NUM>, <NUM>, <NUM> may communicate with computer device <NUM> via a network or bus protocol, such as, for example, a USB connection. Display devices <NUM>, <NUM>, <NUM>, <NUM> may have at least one display and a corresponding audio input and/or audio output. Display devices <NUM>, <NUM>, <NUM>, <NUM> may be any type of display, monitor, visual presentation device, computer device, and/or physical panel capable of presenting information, capturing audio, and/or emitting audio.

In addition, each display device <NUM>, <NUM>, <NUM>, <NUM>, includes any number of channels for capturing audio and/or emitting audio. For example, display device <NUM> may include speaker <NUM> that supports two channels (e.g., right and left) and may include a microphone <NUM>. Display device <NUM> may include speaker <NUM> that supports two channels (e.g., right and left) and may include a microphone <NUM>. Display device <NUM> may include speaker <NUM> that supports two channels (e.g., right and left) and may include a microphone <NUM>. Display device <NUM> may include speaker <NUM> that supports two channels (e.g., right and left) and may include a microphone <NUM>. Each speaker <NUM>, <NUM>, <NUM>, <NUM> and/or microphone <NUM>, <NUM>, <NUM>, <NUM> of display devices <NUM>, <NUM>, <NUM>, <NUM> may correspond to any number of channels.

Each of the display devices <NUM>, <NUM>, <NUM>, <NUM> may include hardware loopback capabilities so that anytime audio data is rendered for output from one or more speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>, the rendered audio data may be obtained by a hardware loopback unit <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>. Display devices <NUM>, <NUM>, <NUM>, <NUM> may have internal hardware to redirect the final stream of rendered audio data <NUM> to the hardware loopback units <NUM>, <NUM>, <NUM>, <NUM>. The rendered audio data <NUM> for output may include any additional processing performed on the rendered audio data <NUM>. The additional processing may include, but is not limited to, equalization, filtering, pitch shift, base boost, and/or any other effect on the audio. In addition, the rendered audio data <NUM> may include a mix of audio data for a plurality of applications <NUM> actively rendering audio for output. For example, a music application may be playing music on display device <NUM> at the same time a video conference application may have a video call on display device <NUM>. Both applications may be actively rendering audio for output via speaker <NUM> and the rendered audio data <NUM> may include the audio data for both the music application and the video conference application.

The rendered audio data <NUM> may be stored in hardware loopback stream <NUM> in a display device loopback buffer <NUM>. As such, hardware loopback stream <NUM> may accurately represent the audio output from the speaker <NUM> on display device <NUM>. Similarly (but without the rendered audio data illustrated), display device loopback buffer <NUM> may include hardware loopback stream <NUM> with the rendered audio data <NUM> for display device <NUM>. The rendered audio data <NUM> may correspond to the audio output from speaker <NUM>. Also similarly (but without the rendered audio data illustrated), display device loopback buffer <NUM> may include hardware loopback stream <NUM> with the rendered audio data <NUM> from display device <NUM>. The rendered audio data <NUM> may correspond to the audio output from speaker <NUM>. Further, and similarly (but without the rendered audio data illustrated), display device loopback buffer <NUM> may include hardware loopback stream <NUM> with the rendered audio data <NUM> from display device <NUM>. The rendered audio data <NUM> may correspond to the audio output from speaker <NUM>.

Display devices <NUM>, <NUM>, <NUM>, <NUM> may also include capture streams <NUM>, <NUM>, <NUM>, <NUM> with any audio captured by microphones <NUM>, <NUM>, <NUM>, <NUM>. For example, if a person speaks into microphone <NUM> of display device <NUM>, capture stream <NUM> may include the captured audio data from the person speaking. In addition, if background music is playing nearby microphone <NUM> of display device <NUM>, capture stream <NUM> may include the audio data corresponding to the background music.

Computer device <NUM> may include any mobile or fixed computer device, which may be connectable to a network. Computer device <NUM> may be, for example, a computer device such as a desktop or laptop or tablet computer, an internet of things (IOT) device, a cellular telephone, a gaming device, a mixed reality or virtual reality device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA), or a handheld device, or any other computer device having wired and/or wireless connection capability with one or more other devices.

Computer device <NUM> may include an operating system <NUM> may be executed by processor <NUM> and/or memory <NUM>. Memory <NUM> of computer device <NUM> may be configured for storing data and/or computer-executable instructions defining and/or associated with operating system <NUM>, and processor <NUM> may execute such data and/or instructions to instantiate operating system <NUM>. An example of memory <NUM> can include, but is not limited to, a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof. An example of processor <NUM> can include, but is not limited to, any processor specially programmed as described herein, including a controller, microcontroller, application specific integrated circuit (ASIC), field programmable gate array (FPGA), system on chip (SoC), or other programmable logic or state machine.

Operating system <NUM> may include an audio aggregator driver <NUM> that may receive one or more hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> with the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> from one or more display device loopback buffers <NUM>, <NUM>, <NUM>, <NUM>. Hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> may accurately represent the audio output from speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>.

Audio aggregator driver <NUM> may generate a single loopback buffer <NUM> for the rendered audio data <NUM>, <NUM>, <NUM>, <NUM>. Audio aggregator driver <NUM> determines a buffer size <NUM> for the loopback buffer <NUM>. The buffer size <NUM> may be calculated by multiplying the total number of display devices <NUM>, <NUM>, <NUM>, <NUM> by the number of channels supported by each of the display devices <NUM>, <NUM>, <NUM>, <NUM>. In the illustrated example, display devices <NUM>, <NUM>, <NUM>, <NUM> may each support two channels (e.g., speakers <NUM>, <NUM>, <NUM>, <NUM> may support two channels, right and left). As such, the buffer size <NUM> may be eight (e.g., four devices multiplied by two channels).

Audio aggregator driver <NUM> partitions the loopback buffer <NUM> into a plurality of sections, where the number of sections correspond to the buffer size <NUM>. For example, the loopback buffer <NUM> may be partitioned into eight sections, where each individual section may correspond to a single channel supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. A sequencing may be applied to the number of sections so that each section may correspond to a specific display device loopback stream <NUM>, <NUM>, <NUM>, <NUM>. The sequencing may be in response to an order in which display devices <NUM>, <NUM>, <NUM>, <NUM> are connected and/or in communication with computer device <NUM>. The number of sections allocated to a specific display device loopback stream <NUM>, <NUM>, <NUM>, <NUM> may be in response to a number of channels supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, each display device <NUM>, <NUM>, <NUM>, <NUM> may receive two sections in the loopback buffer <NUM> since display devices <NUM>, <NUM>, <NUM>, <NUM> support two channels.

Audio aggregator driver <NUM> combines the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> received from display devices <NUM>, <NUM>, <NUM>, <NUM> into the loopback buffer <NUM>. For example, the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> received from display device loopback streams <NUM>, <NUM>, <NUM>, <NUM> may be placed into different sections of the loopback buffer <NUM> in a specified order resulting in the aggregated loopback audio data <NUM>. As such, audio aggregator driver <NUM> may represent the aggregated loopback audio data <NUM> from the various hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> as one by using a single loopback buffer <NUM>.

Audio aggregator driver <NUM> provides the loopback buffer <NUM> to one or more applications <NUM> associated with computer device <NUM>. Applications <NUM> may want to use the aggregated loopback audio data <NUM> for various processing. One example may include application <NUM> using the aggregated loopback audio data <NUM> to perform echo cancelation on audio output from one or more displays <NUM>, <NUM>, <NUM>, <NUM>. For example, when an individual is using a microphone <NUM> on display device <NUM> on a SKYPE call with a friend while some background music is playing on speaker <NUM>, capture stream <NUM> may include the audio from microphone <NUM> (e.g., the individual speaking) and audio from speaker <NUM> (e.g., background music). Application <NUM> may use the aggregated loopback audio data <NUM> in the loopback buffer <NUM> to cancel out the audio from speaker <NUM> (e.g., the background music) so the friend does not hear the background music in the call. In another example, application <NUM> may want to record the audio emitted by one or more speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>. Application <NUM> may use the aggregated loopback audio data <NUM> and/or a portion of the aggregated loopback audio data <NUM> for the recording. For example, application <NUM> may identify a portion of the aggregated loopback audio data <NUM> that corresponds to display device <NUM> and may extract the aggregated loopback audio data <NUM> and/or a portion of the aggregated loopback audio data <NUM> associated with display device <NUM> (e.g., audio data <NUM>) for the recording.

As such, at any time applications <NUM> may access the loopback buffer <NUM> and may be able to use the same audio output by the speakers <NUM>, <NUM>, <NUM>, <NUM> on any of the display devices <NUM>, <NUM>, <NUM>, <NUM> through the aggregated loopback audio data <NUM>. Moreover, since hardware loopback corresponds more closely to the audio output by the display devices <NUM>, <NUM>, <NUM>, <NUM> than a stored stream of data input to the speakers of the respective display devices, a better representation of the audio output by the display devices <NUM>, <NUM>, <NUM>, <NUM> may be used by the applications <NUM>.

Referring now to <FIG>, an example loopback buffer <NUM>, which is covered by the scope of the appended claims is illustrated. Audio aggregator driver <NUM> partitions loopback buffer <NUM> into a number of sections corresponding to the buffer size <NUM> (<FIG>). For example, when the buffer size <NUM> is equal to eight, the loopback buffer <NUM> may be partitioned into eight sections, where each individual section may correspond to a single channel supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. A subset of the number of sections allocated to a specific loopback stream <NUM>, <NUM>, <NUM>, <NUM> may be in response to a number of channels supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, each display device <NUM>, <NUM>, <NUM>, <NUM> may receive a subset of two sections in the loopback buffer <NUM> because display devices <NUM>, <NUM>, <NUM>, <NUM> support two channels.

A sequencing may be applied to the number of sections so that each section may correspond to a specific hardware loopback stream <NUM>, <NUM>, <NUM>, <NUM>. The sequencing may have an arrangement (or relative position) based on a value of an order in which display devices <NUM>, <NUM>, <NUM>, <NUM> are connected and/or in communication with computer device <NUM>. For example, display device <NUM> may be a first display device and the first two sections of loopback buffer <NUM> may be allocated to hardware loopback stream <NUM>. The rendered audio data <NUM> from display device <NUM> may be stored in the first two sections of loopback buffer <NUM>.

Display device <NUM> may be the second display device and the next two sections of loopback buffer <NUM> may correspond to hardware loopback stream <NUM>. The rendered audio data <NUM> from display device <NUM> may be stored in the next two sections of loopback buffer <NUM>. Display device <NUM> may be the third display device and the following two sections of loopback buffer <NUM> may be associated with hardware loopback stream <NUM>. The rendered audio data <NUM> from display device <NUM> may be stored in the following two sections of loopback buffer <NUM>.

Display device <NUM> may be the fourth display device and the last two sections of loopback buffer <NUM> may correspond to hardware loopback stream <NUM>. As such, the rendered audio data <NUM> from display device <NUM> may be stored in the last two sections of loopback buffer <NUM>.

Loopback buffer <NUM> may include aggregated loopback audio data <NUM> corresponding to different time/sequences <NUM>, <NUM> of when audio may be output from display devices <NUM>, <NUM>, <NUM>, <NUM>. Loopback buffer <NUM> may include an array of eight sections of rendered audio data <NUM>, <NUM>, <NUM>, <NUM> (at each time/sequence). For example, at <NUM> (corresponding to time (ti)), loopback buffer <NUM> may include any rendered audio data <NUM>, <NUM>, <NUM>, <NUM> from one or more display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, at <NUM> (corresponding to time (ti)), loopback buffer <NUM> may include any rendered audio data <NUM>, <NUM>, <NUM>, <NUM> from one or more display devices <NUM>, <NUM>, <NUM>, <NUM>.

The rendered audio data <NUM>, <NUM>, <NUM>, <NUM> may be combined together using a sequencing of the display devices <NUM>, <NUM>, <NUM>, <NUM> having an arrangement (or relative position) based on a value of the order in which display devices <NUM>, <NUM>, <NUM>, <NUM> are connected and/or in communication with computer device <NUM>. The combined rendered audio data <NUM>, <NUM>, <NUM>, <NUM> may result in the aggregated loopback audio data <NUM>. As such, the aggregated loopback audio data <NUM> may include a mix of audio data for one or more applications actively rendering audio for output on display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, display device <NUM> may include a music application playing music via speaker <NUM>. Display device <NUM> may include a media application playing a movie via speaker <NUM> and a conference call application conducing a call via speaker <NUM>. The aggregated loopback audio data <NUM> may include a mix of audio data for the music application, the media application, and the conference call application. The aggregated loopback audio data <NUM> is provided to one or more applications <NUM> (<FIG>) on computer device <NUM> (<FIG>) for use by applications <NUM>.

Referring now to <FIG>, a method <NUM>, which is covered by the scope of the appended claims, is used by computer device <NUM> (<FIG>) for aggregating hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> (<FIG>) of a plurality of display devices <NUM>, <NUM>, <NUM>, <NUM> (<FIG>) in communication with computer device <NUM>. The actions of method <NUM> may be discussed below with reference to the architecture of <FIG>.

At <NUM>, method <NUM> includes receiving a plurality of hardware loopback streams with rendered audio data from a plurality of display devices in communication with the computer device. Display devices <NUM>, <NUM>, <NUM>, <NUM> may have at least one display and a corresponding audio input and/or audio output. Display devices <NUM>, <NUM>, <NUM>, <NUM> may be any type of display, monitor, visual presentation device, computer device, and/or physical panel capable of presenting information, capturing audio, and/or emitting audio.

In addition, each display device <NUM>, <NUM>, <NUM>, <NUM>, may include any number of channels for capturing audio and/or emitting audio. Each speaker <NUM>, <NUM>, <NUM>, <NUM> and/or microphone <NUM>, <NUM>, <NUM>, <NUM> of display devices <NUM>, <NUM>, <NUM>, <NUM> may correspond to any number of channels.

Moreover, each of the display devices <NUM>, <NUM>, <NUM>, <NUM> includes hardware loopback capabilities so that anytime audio output occurs from one or more speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>, the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> for output via speakers <NUM>, <NUM>, <NUM>, <NUM> may be obtained using hardware loopback units <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>. Display devices <NUM>, <NUM>, <NUM>, <NUM> may have internal hardware to redirect the final stream of rendered audio data <NUM>, <NUM>, <NUM>, <NUM> to respective hardware loopback units <NUM>, <NUM>, <NUM>, <NUM>. The rendered audio data <NUM>, <NUM>, <NUM>, <NUM> inludes any additional processing performed on the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> prior to output. The additional processing may include, but is not limited to, equalization, filtering, pitch shift, base boost, and/or any other effect on the audio data. In addition, the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> may include a mix of audio data for a plurality of applications <NUM> actively rendering audio for output on display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, display device <NUM> may include a music application playing music via speaker <NUM> and a game application outputting audio for a game via speaker <NUM> at the same time the music application is playing music. The music application and the game application may be actively rendering audio for output via speaker <NUM> on display device <NUM>. As such, the rendered audio data <NUM> for display device <NUM> may include a mix of audio data for the music application and audio data for the game application.

An audio aggregator driver <NUM> may receive one or more hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> with the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> from one or more display device loopback buffers <NUM>, <NUM>, <NUM>, <NUM>. Hardware loopback stream <NUM>, <NUM>, <NUM>, <NUM> may accurately represent the audio data being rendered to the speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM> for output.

At <NUM>, method <NUM> includes combining the rendered audio data from the plurality of hardware loopback streams into a loopback buffer to create aggregated loopback audio data. Audio aggregator driver <NUM> may generate a single loopback buffer <NUM> for the rendered audio data <NUM>, <NUM>, <NUM>, <NUM>. Audio aggregator driver <NUM> determines a buffer size <NUM> for the loopback buffer <NUM>. The buffer size <NUM> is calculated by multiplying the total number of display devices <NUM>, <NUM>, <NUM>, <NUM> by the number of channels supported by each of the display devices <NUM>, <NUM>, <NUM>, <NUM>. In the illustrated example, display devices <NUM>, <NUM>, <NUM>, <NUM> may each support two channels (e.g., speakers <NUM>, <NUM>, <NUM>, <NUM> may support two channels, right and left). As such, the buffer size <NUM> may be eight (e.g., four devices multiplied by two channels).

Audio aggregator driver <NUM> partitions the loopback buffer <NUM> into a plurality of sections, where the number of sections correspond to the buffer size <NUM>. For example, the loopback buffer <NUM> may be partitioned into eight sections, where each individual section may correspond to a single channel supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. A sequencing may be applied to the number of sections so that a subset of the number of sections may correspond to a specific display device hardware loopback stream <NUM>, <NUM>, <NUM>, <NUM>. The sequencing may be in response to an order in which display devices <NUM>, <NUM>, <NUM>, <NUM> are connected and/or in communication with computer device <NUM>. The number of sections allocated to and/or associated with a specific display device hardware loopback stream <NUM>, <NUM>, <NUM>, 41may be in response to a number of channels supported by display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, each display device <NUM>, <NUM>, <NUM>, <NUM> may receive two sections in the loopback buffer <NUM> since display devices <NUM>, <NUM>, <NUM>, <NUM> support two channels.

Audio aggregator driver <NUM> combines the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> received from display devices <NUM>, <NUM>, <NUM>, <NUM> into the loopback buffer <NUM> to create aggregated loopback audio data <NUM>. For example, the rendered audio data <NUM>, <NUM>, <NUM>, <NUM> received from display device loopback buffers <NUM>, <NUM>, <NUM>, <NUM> may be placed into different sections of the loopback buffer <NUM> using a sequencing of the display devices <NUM>, <NUM>, <NUM>, <NUM> having an arrangement (or relative position) based on a value of an order in which display devices <NUM>, <NUM>, <NUM>, <NUM> are connected and/or in communication with computer device <NUM> resulting in the aggregated loopback audio data <NUM>. In addition, loopback buffer <NUM> may include rendered audio data <NUM>, <NUM>, <NUM>, <NUM> corresponding to different time/sequences of when audio may be output from display devices <NUM>, <NUM>, <NUM>, <NUM>.

The aggregated loopback audio data <NUM> may include a mix of audio data for one or more applications actively rendering audio for output on display devices <NUM>, <NUM>, <NUM>, <NUM>. For example, display devices <NUM>, <NUM>, <NUM>, <NUM> may all include a video conference application where a video conference is currently ongoing on display devices <NUM>, <NUM>, <NUM>, <NUM>. In addition, display device <NUM> may include a music application playing music via speaker <NUM> and display device <NUM> may include a media application playing a movie via speaker <NUM>. The aggregated loopback audio data <NUM> may include a mix of audio data for the video conference application, the music application, and the media application. As such, audio aggregator driver <NUM> may represent the aggregated loopback audio data <NUM> from the various hardware loopback streams <NUM>, <NUM>, <NUM>, <NUM> as one by using a single loopback buffer <NUM>.

At <NUM>, method <NUM> includes providing the aggregated loopback audio data to one or more applications on the computer device. Audio aggregator driver <NUM> provides the aggregated loopback audio data <NUM> to one or more applications <NUM> associated with computer device <NUM>. Applications <NUM> may want to use the aggregated loopback audio data <NUM> for various processing. Applications <NUM> may send a request for the entire loopback buffer <NUM> to use in the processing. For example, application <NUM> may want to record the audio emitted by one or more speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM>. Application <NUM> may request the entire loopback buffer <NUM> and may use the aggregated loopback audio data <NUM> for the recording.

In addition, applications <NUM> may send a request for a portion of the aggregated loopback audio data <NUM> to use in the processing. For example, applications <NUM> may send a request for a portion of the aggregated loopback audio data <NUM> relating to a particular time. Application <NUM> may want to record the audio output by one or more speakers <NUM>, <NUM>, <NUM>, <NUM> on display devices <NUM>, <NUM>, <NUM>, <NUM> during the start of a meeting. Application <NUM> may identify a time period for the start of the meeting and may request a portion of the aggregated loopback audio data <NUM> corresponding to the time period and may use the portion of the aggregated loopback audio data <NUM> for the recording. Another example may include applications <NUM> identifying a particular display device <NUM>, <NUM>, <NUM>, <NUM> and extracting a portion of the aggregated loopback audio data <NUM> from the corresponding hardware loopback stream <NUM>, <NUM>, <NUM>, <NUM> associated with the identified display device <NUM>, <NUM>, <NUM>, <NUM> for use.

Another example may include application <NUM> using the aggregated loopback audio data <NUM> to perform echo cancelation on audio output from one or more displays <NUM>, <NUM>, <NUM>, <NUM>. For example, when an individual is using a microphone <NUM> on display device <NUM> on a SKYPE call with a friend while some background music is playing on speaker <NUM>, capture stream <NUM> may include the audio from microphone <NUM> (e.g., the individual speaking) and audio from speaker <NUM> (e.g., background music). Application <NUM> may use the aggregated loopback audio data <NUM> in the loopback buffer <NUM> to cancel out the audio from speaker <NUM> (e.g., the background music) so the friend does not hear the background music in the call.

At any time applications <NUM> may access the loopback buffer <NUM> and may be able to use the same audio data rendered for output by the speakers <NUM>, <NUM>, <NUM>, <NUM> on any of the display devices <NUM>, <NUM>, <NUM>, <NUM> through the aggregated loopback audio data <NUM>. As such, applications <NUM> may have access to an accurate representation of audio output by the display devices <NUM>, <NUM>, <NUM>, <NUM> for any additional processing and/or use by applications <NUM>.

Referring now to <FIG> a computer <NUM>, which is covered by the scope of the appended claims, and that may be configured as computer device <NUM> in accordance with an implementation includes additional component details as compared to <FIG>. In one example, computer <NUM> includes processor <NUM> for carrying out processing functions associated with one or more of components and functions described herein. Processor <NUM> can include a single or multiple set of processors or multicore processors. Moreover, processor <NUM> can be implemented as an integrated processing system and/or a distributed processing system.

Computer <NUM> further includes memory <NUM>, such as for storing local versions of applications being executed by processor <NUM>. Memory <NUM> can include a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof. Additionally, processor <NUM> may include and execute operating system <NUM> (<FIG>).

Further, computer <NUM> may include a communications component <NUM> that provides for establishing and maintaining communications with one or more parties utilizing hardware, software, and services as described herein. Communications component <NUM> may carry communications between components on computer device <NUM>, as well as between computer device <NUM> and external devices, such as devices located across a communications network and/or devices serially or locally connected to computer device <NUM>. For example, communications component <NUM> may include one or more buses, and may further include transmit chain components and receive chain components associated with a transmitter and receiver, respectively, operable for interfacing with external devices.

Additionally, computer <NUM> may include a data store <NUM>, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with implementations described herein. For example, data store <NUM> may be a data repository for applications <NUM> and/or audio aggregator driver <NUM>.

Computer <NUM> may also include a user interface component <NUM> operable to receive inputs from a user of computer device <NUM> and further operable to generate outputs for presentation to the user. User interface component <NUM> may include one or more input devices, including but not limited to a keyboard, a number pad, a mouse, display (e.g., which may be a touch-sensitive display), a navigation key, a function key, a microphone, a voice recognition component, any other mechanism capable of receiving an input from a user, or any combination thereof. Further, user interface component <NUM> may include one or more output devices, including but not limited to a display, a speaker, a haptic feedback mechanism, a printer, any other mechanism capable of presenting an output to a user, or any combination thereof.

In an implementation, user interface component <NUM> may transmit and/or receive messages corresponding to the operation of applications <NUM> and/or audio aggregator driver <NUM>. In addition, processor <NUM> executes applications <NUM> and/or audio aggregator driver <NUM>, and memory <NUM> or data store <NUM> may store them.

As used in this application, the terms "component," "system" and the like are intended to include a computer-related entity, such as but not limited to hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer device and the computer device can be a component. One or more components can reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets, such as data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems by way of the signal.

Various implementations or features may have been presented in terms of systems that may include a number of devices, components, modules, and the like. It is to be understood and appreciated that the various systems may include additional devices, components, modules, etc. and/or may not include all of the devices, components, modules etc. discussed in connection with the figures. A combination of these approaches may also be used.

The various illustrative logics, logical blocks, and actions of methods described in connection with the embodiments disclosed herein may be implemented or performed with a specially-programmed one of a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor may also be implemented as a combination of computer devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, at least one processor may comprise one or more components operable to perform one or more of the steps and/or actions described above.

Further, the steps and/or actions of a method or algorithm described in connection with the implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor, such that the processor can read information from, and write information to, the storage medium. Further, in some implementations, the processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in a user terminal. Additionally, in some implementations, the steps and/or actions of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a machine readable medium and/or computer readable medium, which may be incorporated into a computer program product.

In one or more implementations, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium. A storage medium may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs usually reproduce data optically with lasers.

Claim 1:
A computer device, comprising:
a memory to store data and instructions;
at least one processor configured to communicate with the memory; and
an operating system configured to communicate with the memory and the at least one processor, wherein the operating system is operable to:
receive a plurality of hardware loopback streams with rendered audio data from a plurality of display devices in communication with the computer device, wherein the rendered audio data includes audio data for a plurality of applications actively rendering the audio data for output on the plurality of display devices, a hardware loopback stream representing the audio data coming out of one or more speakers on a display device including any additional processing performed on the rendered audio occurring prior to the output of the audio via the one or more speakers, wherein each hardware loopback stream of the plurality of hardware loopback streams corresponds to one display device of the plurality of display devices;
determine a buffer size for a loopback buffer by multiplying a total number of the plurality of display devices by a number of channels supported by the plurality of display devices;
partition the loopback buffer into a number of sections equal to the buffer size;
determine a subset of the number of sections for each display device of the plurality of display devices, wherein the subset of the number of sections for each display device correspond to the number of channels supported by each display device of the plurality of display devices;
combine the rendered audio data from the plurality of hardware loopback streams into the loopback buffer to create aggregated loopback audio data, wherein said combining comprises placing the rendered audio data received from each display device in the subset of the number of sections that corresponds to each display device of the plurality of display devices; and
provide the loopback buffer with the aggregated loopback audio data to one or more applications executing on the computer device;
wherein the one or more applications use the aggregated loopback audio data for at least one of echo cancellation or creating a recording.