Time-synchronized hardware controllers and related audio systems and circuitry

Synchronization for audio systems and related systems and circuitry are disclosed. An audio system includes a word select line of a digital audio interface, a serial clock line of the digital audio interface, and hardware circuitry. The hardware circuitry is configured to provide a word select signal to the word select line and a serial clock signal to the serial clock line. The word select signal is configured to indicate channels of a serial data signal provided to a serial data line of the digital audio interface. The hardware circuitry is also configured to synchronize the serial clock signal to a clock reference stream of an audio stream communicated via a network interface.

TECHNICAL FIELD

This disclosure relates generally to time-synchronized hardware controllers.

BACKGROUND

In some digital audio systems synchronization may be important. For example, audio may be synchronized with video in video streaming systems (e.g., so speech audio is synchronized with mouth movements shown in video, without limitation). As another example, digital signals used to generate analog audio signals delivered to different speakers in surround sound systems (e.g., automotive surround sound, professional audio equipment, without limitation) may be synchronized.

DETAILED DESCRIPTION

The illustrations presented herein are not meant to be actual views of any particular method, system, device, or structure, but are merely idealized representations that are employed to describe the examples of the present disclosure. In some instances, similar structures or components in the various drawings may retain the same or similar numbering for the convenience of the reader; however, the similarity in numbering does not necessarily mean that the structures or components are identical in size, composition, configuration, or any other property.

It will be readily understood that the components of the examples as generally described herein and illustrated in the drawings could be arranged and designed in a wide variety of different configurations. Thus, the following description of various examples is not intended to limit the scope of the present disclosure but is merely representative of various examples. While the various aspects of the examples may be presented in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

Those of ordinary skill in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. Some drawings may illustrate signals as a single signal for clarity of presentation and description. It will be understood by a person of ordinary skill in the art that the signal may represent a bus of signals, wherein the bus may have a variety of bit widths and the present disclosure may be implemented on any number of data signals including a single data signal.

The examples may be described in terms of a process that is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operational acts as a sequential process, many of these acts can be performed in another sequence, in parallel, or substantially concurrently. In addition, the order of the acts may be re-arranged. A process may correspond to a method, a thread, a function, a procedure, a subroutine, a subprogram, other structure, or combinations thereof.

As used herein, the term “hardware” refers to implementations including hardware elements and excluding software elements such as executable software or firmware code. For example, hardware may include discrete electrical components (e.g., resistors, capacitors, inductors, logic gates, diodes, transistors, other hardware elements, or combinations thereof).

Inter-IC Sound (I2S) is a standard for a digital serial bus that may be used to interface digital audio devices (e.g., integrated circuits (ICs), without limitation) to each other. According to the Time-Sensitive Network (TSN) International Electronics and Electrical Engineers (IEEE) 802.1 standard, audio listener applications may receive audio streams (e.g., as set forth in the IEEE 1722 or 1733 standards) from the network along with a clock reference stream including a reference clock. The received audio samples are transferred through an I2S interface to an audio device (e.g., a digital signal processor (DSP), an amplifier, without limitation) for playback. The I2S clock is synchronous with the reference clock, and the listener physically recovers this clock from the received stream data. Clock recovery uses a phased locked loop (PLL) to obtain the final audio bit clock (I2S MCK/SCK) by multiplying the recovered reference clock by some constant factor. In some instances, the clock recovery is started in front of the audio transfer for playback, and is run until the PLL is able to clock to the given reference clock. I2S does not permit audio transmission before this point is reached.

Certain problems exist in synchronizing between an I2S clock and data for listener use cases. For example, at the start of I2S streaming, the data and clock may be aligned. Any error during the start of streaming will persist for the entire duration of streaming. Using software solutions, which would be relatively complicated, it is possible to synchronize the I2S clock and the data for listener use cases to within microsecond precision. Hardware solutions disclosed herein, however, provide nanosecond precision in synchronizing the I2S clock and the data for listener use cases.

Certain problems may exist in synchronizing between captured frame synchronization- (FSY-) timestamps and the input stream of audio samples. Capturing the timestamps may take place asynchronously to the I2S stream. If the I2S transmitter is the clock master, then software may not be able to reliably recover the relationship between sample and timestamp.

The inventors of this disclosure appreciate that the use of software solutions to define a starting point of an I2S transmission to enable synchronizing of the I2S clock to the stream reference clock would introduce jitter and cause difficulty when worst-case timing constraints must be guaranteed. Software solutions may be relatively complex to guarantee hard timings and may introduce a range of timing uncertainty (e.g., due to interrupt delays, without limitation) that may be unacceptable or undesirable in some use cases.

In some instances, audio may be streamed over Ethernet networks. Relatively high synchronicity may be desired between multiple audio endpoints (e.g., surround sound systems, without limitation), or to synchronize mouth movements to audio in video streaming. Related streaming protocol standards include IEEE 1722 (Audio Video Transport Protocol (AVTP)) and IEEE 1733 (Real-time Transport Protocol (RTP)) in combination with some timing protocols such as IEEE 1588 (Precision Time Protocol (PTP)) or IEEE 802.1AS (generalized PTP (gPTP)).

Various examples disclosed herein may improve synchronicity of media streaming devices by achieving a small (e.g., the smallest possible, without limitation) timing uncertainty in presentation time of audio listeners and allowing precise (e.g., substantially exact, without limitation) synchronization between sample data and timestamps of audio talkers. Various examples disclosed herein may simplify stream synchronization by offloading complexity from a CPU (e.g., a processing core of a microcontroller, without limitation). Hardware solutions, in contrast to software solutions, may reliably guarantee fixed numbers and lower uncertainty. Software solutions would introduce interrupt latency that is at least partially avoided in various examples of hardware solutions disclosed herein.

Various examples disclosed herein enable synchronization of an I2S clock to a stream reference clock based on a synchronized time base. Implementations of various examples disclosed herein may involve extension to I2S hardware peripherals in MCUs, which enables synchronization of the start of an I2S transmission (clock output enable) to a specific time point. Various examples disclosed herein are directed to hardware mechanisms in contrast to software mechanisms. Accordingly, at least some undesired time shifts, jitter, and poor worst-case-timings associated with software mechanisms may be avoided. Hardware-based mechanisms may guarantee high (e.g., highest possible, without limitation) precision of synchronization between sample data and timestamps of audio talkers. This is the case especially where multiple audio listener nodes play back audio synchronously.

In various examples, an I2S controller that is configured to synchronize its I2S clock to the actual start time is integrated. I2S clocks are derived from an external clock source, which is a multiple of the final audio bit block and is recovered from the audio reference clock.

In various examples, an audio system includes a word select line of a digital audio interface, a serial clock line of the digital audio interface, and hardware circuitry. The hardware circuitry is configured to provide a word select signal to the word select line. The word select signal is configured to indicate channels of a serial data signal provided to a serial data line of the digital audio interface. The hardware circuitry is configured to provide a serial clock signal to the serial clock line and synchronize, at least partially based on a synchronized time base, the serial clock signal to a clock reference stream of an audio stream communicated via a network interface.

In various examples, circuitry for controlling timing of communications includes a serial clock line, an enable gate, and an event generator. The enable gate is electrically connected to the serial clock line. The enable gate is configured to start provision of a serial clock signal to the serial clock line based, at least in part, on an assertion of a synchronization signal. The event generator is configured to assert the synchronization signal responsive to a presentation time stamp from a serial data signal. The presentation time stamp is correlated to a synchronized time base. The enable gate and the event generator are implemented using hardware circuitry.

FIG.1is a block diagram of a transmitter as master I2S system100, according to various examples. The transmitter as master I2S system100includes a transmitter102and a receiver104. The transmitter102and the receiver104are I2S endpoints configured to communicate (e.g., via an Ethernet network, without limitation) using an I2S protocol. The transmitter102is configured to operate as a master node in the transmitter as master I2S system100. As the master node, the transmitter102is configured to transmit a serial clock signal SCK and a word select signal WS to the receiver104. The transmitter102is configured to transmit a serial data signal SD to the receiver104. One or both of the transmitter102and the receiver104may include a time-synchronized hardware controller106according to various examples discussed herein.

The serial data signal SD may include serial data (e.g., serial audio data corresponding to an audio stream, without limitation). The receiver104is configured to use the serial clock signal SCK and the word select signal WS received from the transmitter102to synchronize the serial data of the serial data signal SD.

FIG.2is a block diagram of a receiver as master I2S system200, according to various examples. The receiver as master I2S system200includes a transmitter202and a receiver204similar to the transmitter102and the receiver104ofFIG.1. Similar to the transmitter102and the receiver104ofFIG.1, the transmitter202and the receiver204are I2S endpoints configured to communicate (e.g., via an Ethernet network, without limitation) using an I2S protocol. In contrast to the receiver104of transmitter as master I2S system100ofFIG.1, the receiver204is configured to operate as a master node in the receiver as master I2S system200, as the master node the receiver204is configured to transmit a serial clock signal SCK and a word select signal WS to the receiver204. One or both of the transmitter202and the receiver204may include a time-synchronized hardware controller206according to various examples discussed herein.

Similar to the transmitter102ofFIG.1, the transmitter202is configured to transmit a serial data signal SD to the receiver204. The transmitter102is configured to synchronize the serial data signal SD responsive to the serial clock signal SCK and the word select signal WS received from the receiver204.

FIG.3is a block diagram of a controller as master I2S system300, according to various examples. The controller as master I2S system300includes a transmitter302and a receiver304similar to the transmitter102and the receiver104ofFIG.1. Similar to the transmitter102and the receiver104ofFIG.1and the transmitter202and the receiver204ofFIG.2, the transmitter302and the receiver304are I2S endpoints configured to communicate (e.g., via an Ethernet network, without limitation) using an I2S protocol. In contrast to the transmitter as master I2S system100ofFIG.1and the receiver as master I2S system200ofFIG.2, the controller as master I2S system300includes a controller306configured to operate as a master node in the controller as master I2S system300. As the master node, the controller306is configured to transmit a serial clock signal SCK and a word select signal WS to the transmitter302and the receiver304. Accordingly, the controller306is configured to operate as a stand-alone I2S controller, which works in conjunction with state of the art I2S peripherals (e.g., the transmitter302and the receiver304, without limitation).

Similar to the transmitter102ofFIG.1and the transmitter202ofFIG.2, the transmitter302is configured to transmit a serial data signal SD to the receiver304. The transmitter302and the receiver304are configured to synchronize the serial data signal SD responsive to the serial clock signal SCK and the word select signal WS received from the controller306.

FIG.4is a signal timing diagram of I2S signals400of the transmitter as master I2S system100ofFIG.1, the receiver as master I2S system200ofFIG.2, and the controller as master I2S system300ofFIG.3, according to various examples. The I2S signals400include the serial clock signal SCK, the word select signal WS, and the serial data signal SD. By way of non-limiting example, the serial data signal SD may be transmitted as a two's compliment signal. The serial data signal may include words (e.g., a word N−1 right channel, a word N left channel, a word N+1 right channel, without limitation). Each word may begin with a most significant bit MSB and end with a least significant bit LSB.

The word select signal WS is configured to indicate an audio channel (e.g., a left audio channel and a right audio channel of a stereo audio system, without limitation) that serial data in the serial data signal SD being transmitted corresponds to. As a non-limiting example, the word select signal WS may indicate with a “one” (e.g., a logic level high voltage potential, without limitation) that the serial data transmitted in the serial data signal SD corresponds to a right channel. Also, the word select signal WS may indicate with a “zero” (e.g., a logic level low voltage potential, without limitation) that the serial data transmitted in the serial data signal SD corresponds to a left channel.

The serial clock signal SCK and the word select signal WS enable the I2S endpoints to synchronize the serial clock signal SCK and the serial data signal SD to a reference clock of a media stream with a relatively high level of precision.

FIG.5is a block diagram of an AVTP talker500, according to various examples. The AVTP talker500includes a local oscillator518, an audio video bridging (AVB) transport protocol (AVBTP) timestamp generator (AVBTP timestamp generator504), and an analog to digital (A/D) converter (A/D converter506). The AVTP talker500is configured to generate an IEEE 1722 data stream (1722 stream512) including timestamps508and data510.

The local oscillator518(e.g., a voltage-controlled oscillator, without limitation) is configured to generate a media clock514. The AVBTP timestamp generator504and the A/D converter506are configured to receive the media clock514from the local oscillator518. The A/D converter506is also configured to receive incoming analog data516(e.g., analog audio data, without limitation) and convert the incoming analog data516to data510(digital data) to be included in the 1722 stream512responsive to the incoming analog data516and the media clock514. The AVBTP timestamp generator504is configured to generate the timestamps508for the data510responsive to the media clock514and wall time502(e.g., IEEE 802.1AS network time, without limitation). The 1722 stream512includes the timestamps508and the data510.

The timestamps508indicate values of the wall time502corresponding to bits of the data510. As a non-limiting example, the values of the timestamps508may indicate values of the wall time502at which the values of the data510were sampled by the A/D converter506. By way of non-limiting example, the timestamps508illustrated inFIG.5include wall time values of U.S. Pat. Nos. 7,166,667, 7,333,333, . . . , 8666667, 8833333, and 9000000. Also by way of non-limiting example, the timestamps508may be captured per a certain number of data510samples captured (e.g., one timestamp for every six or eight data510samples, without limitation).

The AVTP talker500may be configured to operate as an I2S transmitter (e.g., the transmitter102ofFIG.1, the transmitter202ofFIG.2, or the transmitter302ofFIG.3, without limitation). The 1722 stream512may be transmitted to an audio device (e.g., an I2S receiver, without limitation) using an I2S protocol. Accordingly, the 1722 stream512may be transmitted from a transmitter (e.g., the transmitter102ofFIG.1, the transmitter202ofFIG.2, or the transmitter302ofFIG.3, without limitation) to a receiver (e.g., the receiver104ofFIG.1, the receiver204ofFIG.2, or the receiver304ofFIG.3, without limitation) as a serial data signal SD (e.g., the SD ofFIG.1,FIG.2,FIG.3, orFIG.4, without limitation) of an I2S interface.

FIG.6is a block diagram of an AVTP listener600, according to various examples. The AVTP listener600may be configured to operate as an I2S receiver (e.g., the receiver104ofFIG.1, the receiver204ofFIG.2, or the receiver304ofFIG.3, without limitation). The AVTP listener600may be configured to receive the 1722 stream512from the AVTP talker500ofFIG.5. The AVTP listener600is configured to extract outgoing analog data606responsive to the 1722 stream512.

The AVTP listener600includes an AVBTP timestamp comparator612, a clock generator610, and a digital to analog (D/A) converter (D/A converter608). The AVBTP timestamp comparator612is configured to compare a wall time602(e.g., an 802.1AS network time synchronized with the wall time502ofFIG.5using, without limitation, a PTP) to the timestamps508of the 1722 stream512. The wall time602of the AVTP listener600may be synchronized with the wall time502of the AVTP talker500ofFIG.5.

The clock generator610is configured to generate a media clock604, which is a recovered version of the media clock514of the AVTP talker500ofFIG.5. The D/A converter608is configured to generate outgoing analog data606responsive to the media clock514and the comparison between the wall time602and the timestamps508from the 1722 stream512. The outgoing analog data606may be synchronized to the media clock604based on the comparison between the wall time602and the timestamps508.

FIG.7is a block diagram of timing reference planes700(e.g., AVTP timing reference planes, without limitation), according to various examples.FIG.7illustrates an ingress time reference plane746, a presentation time reference plane748, timestamp measurement planes744(e.g., gPTP timestamp measurement planes, without limitation), a talker740(e.g., an AVTP talker, without limitation), a listener742(e.g., an AVTP listener, without limitation), and time-sensitive applications704aand704b. The talker740includes a stream packetizer734(e.g., an AVTP stream packetizer, without limitation) and a network interface738a(e.g., an AVTP network interface, without limitation). The stream packetizer734includes a streaming data interface706a, a buffer710a(e.g., including data samples, without limitation), and a control708a. The network interface738aincludes a buffer712a(e.g., including 1722 packets, without limitation), a media access control layer (MAC714a), a physical layer (PHY716a), and a timer718a(e.g., an 802.1AS timer, without limitation).

The listener742includes a network interface738b(e.g., an AVTP network interface, without limitation) and a stream depacketizer736(e.g., an AVTP stream depacketizer, without limitation). The network interface738bincludes a PHY716b, a MAC714b, a timer718b(e.g., an 802.1AS timer, without limitation), and a buffer712b(e.g., including 1722 packets, without limitation). The stream depacketizer736includes a buffer710b(e.g., including data samples, without limitation), a streaming data interface706b, and a control708b.

FIG.7illustrates various latencies (e.g., jack-to-jack latency720, application-to-application latency722, without limitation), timing information (e.g., application times724aand724b, format conversion times726aand726b, a minimum transit time728, an actual transit time730(variable), and a maximum transit time732, without limitation) involved with streaming data from the time-sensitive application704aassociated with the talker740(e.g., the AVTP talker500ofFIG.5) to the time-sensitive application704bassociated with the listener742(e.g., the AVTP listener600ofFIG.6, without limitation) through a cloud702(e.g., an 802.1 AVB/TSN cloud, without limitation).

There may be uncertainty in a maximum timing due to variability in transfer of a packet's ownership. In the ingress time reference plane746, ownership of a fully prepared 1722 packet is transferred to the network interface738afor transmission at a time no later than presentation. In the presentation time reference plane748, the 1722 packet is guaranteed to be written into the buffer no later than the maximum transmit time.

FIG.8illustrates a 1722 stream806, according to various examples. The 1722 stream806includes timestamps802(7166667, 7333333, . . . 866667, 8833333, 9000000) and data804. According to various examples the timestamps802may be synchronized to a serial clock SCK (e.g., identify falling edges of the serial clock SCK that correspond to each of the timestamps802, without limitation). The data804and the serial clock SCK may be provided to an I2S peripheral808.

For a listener user case, it may be challenging to synchronize the serial clock SCK and the data804. As a non-limiting example, at the start of I2S streaming, it may be desirable for the data804and serial clock SCK to be aligned. An error in alignment between the data804and the serial clock SCK at the start of streaming may persist for the entire duration of streaming. In contrast to software solutions, which may only be capable of providing in the range of microsecond precision, hardware solutions disclosed herein may guarantee nanosecond precision without, as a non-limiting example, any real-time requirements for software. An event generator (EG) may trigger a start of a serial clock SCK, which is provided to an I2S peripheral808, to synchronize the timestamps to the serial clock SCK.

For a talker use case, it may be challenging to synchronize captured FSY-timestamps to the input stream of audio samples. As a non-limiting example, capturing the timestamps may take place asynchronously from the I2S streaming. If the I2S transmitter is the clock master, then a software solution may not reliably recover the relationship between audio samples and their corresponding timestamps.

FIG.9Ais a block diagram of an audio system900operating with a CODEC910(e.g., an audio device, without limitation) as a listener and using a standalone I2S controller930, according to various examples. The audio system900includes a microcontroller (microcontroller902), the CODEC910, a phase-locked loop (PLL908), and a network device914. The network device914includes the I2S controller930. The audio system900is configured to operate as a controller as master I2S system such as the controller as master I2S system300ofFIG.3. In the audio system900the microcontroller902may operate as an I2S transmitter (e.g., the transmitter302ofFIG.3, without limitation), the CODEC910may operate as an I2S receiver (e.g., the receiver304ofFIG.3, without limitation), and the I2S controller930may operate as an I2S controller (e.g., the controller306ofFIG.3, without limitation). The I2S controller930is implemented in hardware (in contrast to software). Accordingly, the network device914may be a standalone controller including the hardware circuitry of the I2S controller930.

The microcontroller902includes an I2S interface904and a serial peripheral interface (SPI) master interface (SPI master interface906). The CODEC910includes an I2S interface912electrically connected to the I2S interface904. The audio system900includes a digital audio interface including an SD line934, an SCK line936, and a frame synchronization (FSY, used interchangeably with WS herein) line (FSY line938) electrically connecting the I2S interface912to the I2S interface904. Accordingly, the microcontroller902is in communication with the CODEC910(e.g., an audio device, without limitation) via the serial data (SD) line934. The microcontroller902is configured to transmit a serial digital signal SD to the CODEC910on the SD line934. The serial digital signal SD includes an I2S stream including data and timestamps such as, for example the data510and timestamps508of the 1722 stream512ofFIG.5andFIG.6.

The network device914includes an SPI slave interface916electrically connected to the SPI master interface906via a serial clock line944, a channel select line946, a master out slave in line948, and a master in slave out line950. Accordingly, the network device914may be a standalone controller configured to communicate with the microcontroller902via a peripheral interface as a peripheral to the microcontroller902. The network device914also includes an I2S controller930including a media clock divider920, a serial clock divider922(e.g., 1.5 MHz to 48 kHz, without limitation), and an enable gate924. The serial clock divider922is electrically connected to the serial clock line944and the FSY line938. The serial clock divider922is configured to divide the serial clock signal SCK provided to the serial clock line944to provide the FSY signal (a word select signal) to the FSY line938(a word select line). The network device914further includes an event generator926electrically connected between the SPI slave interface916and the enable gate924. The network device914also includes a timestamp divider932electrically connected to the serial clock divider922, and a time stamper928electrically connected to the SPI slave interface916. The timestamp divider932is configured to divide the frame synchronization signal FSY to a rate at which timestamps are provided by the serial data signal SD.

The network device914includes an event generator918electrically connected between the SPI slave interface916and the PLL908. The event generator918is electrically connected to the PLL908via a reference clock line942. The event generator918and the PLL908may together function as the clock generator610ofFIG.6when the CODEC910is operating as a listener. The PLL908is configured to receive a reference clock signal RCLK from the event generator918and provide a media clock signal MCLK to a media clock divider920of the I2S controller930(e.g., implemented in hardware circuitry, without limitation).

In operation, the event generator918drives the PLL908. More specifically, the microcontroller902may provide timestamps (e.g., the timestamps508ofFIG.5, without limitation) to the event generator918(e.g., via the SPI master interface906, the MOSI line, and the SPI slave interface916, without limitation). For every presentation timestamp the event generator918receives, the event generator918may generate a pulse, resulting in a reference clock signal RCLK that the event generator918provides to the PLL908over the reference clock line942. The PLL908may multiply the reference clock signal RCLK to generate a media clock signal MCLK (e.g., an audio bit clock (Nx), without limitation) on a media clock line940. By way of non-limiting example, the reference clock signal RCLK may pulse at a frequency between substantially 300 Hertz (Hz) and substantially 8 kilohertz (kHz). Also, by way of non-limiting example, the PLL908may multiply the reference clock signal RCLK to a media clock signal MCLK of substantially 1.5 MHz for 16-bit stereo. The PLL908provides the media clock signal MCLK to the CODEC910and the media clock divider920via the media clock line940. The media clock divider920is configured to receive and divide the media clock signal MCLK, and provide the divided media clock signal to the enable gate924.

The enable gate924is configured to provide a serial clock signal SCK to the SCK line936and a frame synchronization signal FSY to the FSY line938. The microcontroller902and the CODEC910are configured to receive the frame synchronization signal FSY (a word select signal) and the serial clock signal SCK provided by the hardware circuitry of the network device914. The enable gate924may be triggered to initiate provision of the serial clock signal SCK and the frame synchronization signal FSY responsive to an assertion (e.g., a pulse, without limitation, without limitation) of a synchronization signal SYNC from the event generator926. The event generator926is configured to assert the synchronization signal SYNC responsive to a presentation timestamp and based at least in part on a synchronized time base (e.g., a wall time, without limitation). Similar to the event generator918, the event generator926may provide a pulse on the synchronization signal SYNC responsive to each presentation timestamp received from the microcontroller902. By way of non-limiting example, the divided media clock signal from the media clock divider920may clock at a frequency suitable for use as the serial clock signal SCK (e.g., 2.5 MHz, without limitation) and the enable gate924may pass the divided media clock signal as the serial clock signal SCK to the SCK line936. The enable gate924provides the serial clock signal SCK to the microcontroller902(via the SCK line936and the I2S interface904) and to the CODEC910(via the SCK line936and the I2S interface912). The enable gate924also provides the serial clock signal SCK to the serial clock divider922, which divides the serial clock signal SCK to generate the frame synchronization signal FSY. The serial clock divider922provides the frame synchronization signal FSY to the microcontroller902(via the FSY line938and the I2S interface904) and to the CODEC910(via the FSY line938and the I2S interface912).

In various examples, audio system900enables start of streaming at a specific time by providing the I2S controller930with a synchronizable start time point, which is triggered by a pulse of the synchronization signal SYNC. Synchronization is accomplished using event generators (event generator926and event generator918).

Streaming (e.g., transmission of data using the serial data signal SD on the SD line934, without limitation) initiates at a dedicated timestamp because the synchronization signal SYNC pulses responsive to a timestamp. Once the clocks (the reference clock signal RCLK, the media clock signal MCLK) are running the data may be streamed on the serial data signal SD. Streaming by the microcontroller902(i.e., provision of data on the serial data signal SD) may be initiated at the same time that the synchronization signal SYNC is pulsed to initiate provision of the serial clock signal SCK and the frame synchronization signal FSY by the enable gate924. As a result, the SPI communications from the microcontroller902to the network device914enable synchronization of the serial clock signal SCK with the streamed data communicated by the serial data signal SD.

Since the serial clock signal SCK runs synchronously, any later synchronization is not required. In various examples, however, the serial clock signal SCK may be monitored (e.g., if it is from an external source, without limitation) to be able to detect issues while the stream is running.

FIG.9Bis a block diagram of the audio system900ofFIG.9Awith the CODEC910operating as a talker, according to various examples. As discussed above with reference toFIG.9A, the I2S controller930may operate as an I2S controller (e.g., the controller306ofFIG.375, without limitation) in a controller as master I2S system (e.g., the controller as master I2S system300ofFIG.3, without limitation). In the audio system900illustrated inFIG.9B, however, the microcontroller902may operate as an I2S receiver (e.g., the receiver304ofFIG.3, without limitation) and the CODEC910may operate as an I2S transmitter (e.g., the transmitter302ofFIG.3, without limitation). Accordingly, the CODEC910may transmit a serial data signal SD (e.g., including the data510and the timestamps508of the 1722 stream512ofFIG.5, without limitation) on the SD line934to the microcontroller902.

As previously discussed, in a talker use case it may be challenging to synchronize captured timestamps (e.g., the timestamps508ofFIG.6, without limitation) to an input stream of audio samples (e.g., the data510ofFIG.5, without limitation). In various examples, I2S controller930enables synchronization of captured time stamps to an input stream of audio samples. The network device914recovers the media clock MCLK, and the I2S controller930provides the serial clock signal SCK and the frame synchronization signal FSY, as discussed with reference toFIG.9A. The I2S controller930(specifically the serial clock divider922) provides the frame synchronization signal FSY to the timestamp divider932, which divides the frame synchronization signal FSY using a timestamp divider932. The timestamp divider932may divide the frame synchronization signal FSY to a rate at which timestamps are provided by the serial data signal SD. A time stamper928may provide the divided frame synchronization signal to the microcontroller902(e.g., using a master in slave out signal MISO on the master in slave out line950, without limitation).

FIG.10is a block diagram of another audio system1000with a CODEC1008(e.g., an audio device, without limitation) operating as a talker and as a master node, according to various examples. The audio system1000includes a microcontroller1002, a CODEC1008, and a network device1012. The microcontroller1002is similar to the microcontroller902ofFIG.9AandFIG.9B. For example, the microcontroller1002includes an I2S interface1004and an SPI master interface1006similar to the I2S interface904and the SPI master interface906discussed above with reference toFIG.9AandFIG.9B. The CODEC1008is similar to the CODEC910ofFIG.9Baccept that the CODEC1008provides a serial data signal SD to the microcontroller1002because the CODEC1008is shown as operating as a talker. The CODEC1008is also operating as an I2S master node, similar to the transmitter102of the transmitter as master I2S system100ofFIG.1. As a result, the CODEC1008provides a master serial clock signal MSCK and a master frame select signal MFSY to the network device1012(e.g., a standalone controller, without limitation). The CODEC1008includes an I2S interface1010similar to the I2S interface912ofFIG.9AandFIG.9B. The microcontroller1002is shown as operating as a receiver, similar to the receiver104ofFIG.1.

The network device1012ofFIG.10, like the network device914ofFIG.9AandFIG.9B, is implemented in hardware (in contrast to software). The network device1012includes an SPI slave interface1014similar to the SPI slave interface916ofFIG.9AandFIG.9Belectrically connected to the SPI master interface1006of the microcontroller1002. The network device1012includes a timestamp divider1024and a timestamp generator1020, which are similar to the time stamper928and the timestamp divider932ofFIG.9AandFIG.9B. The network device1012, however, is different from the network device914ofFIG.9AandFIG.9B. For example, since the network device1012is not operating as the I2S master node (the CODEC1008is operating as the I2S master node), the network device1012does not include circuitry (e.g., the event generator918and the PLL908ofFIG.9AandFIG.9B, without limitation) for recovering a reference clock (see the reference clock RCLK ofFIG.9AandFIG.9B). Also, although the network device1012includes an I2S controller1022, the I2S controller1022is different from the I2S controller930ofFIG.9AandFIG.9B.

The network device1012includes an I2S controller1022including an enable gate EN (enable gate1018) and an AND gate “&” (AND gate1026). The enable gate1018is configured to receive the master serial clock signal MSCK and the master frame synchronization signal MFSY from the CODEC1008(e.g., via the I2S interface1010, without limitation). The enable gate1018is also configured to begin providing the serial clock signal SCK and the frame synchronization signal FSY to the microcontroller1002(e.g., via a serial clock line1030and a frame synchronization line1032, respectively, and the I2S interface1004, without limitation) responsive to a trigger. The network device1012also includes an event generator1016(“EG”), a timestamp generator1020(“TS”), and a timestamp divider1024(“DIV”).

The CODEC1008is configured to provide the serial data signal SD (e.g., including data and timestamps, without limitation) to the microcontroller1002via a serial data line1028. The CODEC910is also configured to provide the master serial clock signal MSCK (e.g., via a master serial clock line1036, without limitation) and the master frame synchronization signal MFSY (e.g., via a master frame synchronization line1034, without limitation, which may also be referred to herein as a “master word select line”) to the enable gate1018of the I2S controller1022. The enable gate1018may be triggered immediately or in response to an output signal of the AND gate1026. Specifically, the event generator1016is configured to assert a synchronization signal SYNC responsive to timestamps provided by the microcontroller1002to the network device1012. The microcontroller1002may receive the data and timestamps from the CODEC1008via the serial data signal SD through the serial data line1028and the I2S interface1004and provide the timestamps to the event generator1016via the SPI master interface1006, a MOSI signal, and the SPI slave interface1014.

The AND gate1026is configured to receive the synchronization signal SYNC and the master frame synchronization signal MFSY. The AND gate1026is configured to trigger the enable gate1018to provide the serial clock signal SCK to the serial clock line1030responsive to an assertion of the synchronization signal SYNC and an assertion of the master frame synchronization signal MFSY received from the CODEC1008. Accordingly, the enable gate1018is configured to start provision of the serial clock signal SCK and the frame synchronization signal FSY responsive to assertions of the synchronization signal SYNC from the event generator1016and the master frame synchronization signal MFSY (a master word select signal).

Since the AND gate1026asserts its output to trigger the enable gate1018when both the synchronization signal SYNC and the master frame synchronization signal MFSY are asserted, the frame synchronization signal FSY and the serial clock signal SCK provided by the enable gate1018are synchronized to the master frame synchronization signal MFSY and the master serial clock signal MSCK provided by the CODEC1008. Once the microcontroller1002receives timestamps via the serial data signal SD from the CODEC1008and provides the timestamps to the event generator1016(e.g., via a MOSI signal from the SPI master interface1006to the SPI slave interface1014, without limitation), triggering the synchronization signal SYNC, the enable gate1018will start providing the serial clock signal SCK and the frame synchronization signal FSY to the microcontroller1002at the next master frame synchronization signal MFSY edge (e.g., falling edge, without limitation).

The audio system1000may solve the problem of the CODEC1008operating as an I2S master mode by enabling of clocks (e.g., the serial clock signal SCK and the frame synchronization signal FSY on the serial clock line1030and the frame synchronization line1032, respectively, without limitation) aligned to the master frame synchronization signal MFSY provided by the CODEC1008to be able to match the FSY timestamp to its related audio sample. The AND gate1026ensures that the enable gate1018does not start providing the serial clock signal SCK and the frame synchronization signal FSY to the microcontroller1002until the first edge (e.g., falling edge, without limitation) of the master frame synchronization signal MFSY provided by the CODEC1008after the enable gate1018confirms (e.g., via an assertion of the synchronization signal, without limitation) that the microcontroller1002has received at least a first timestamp.

Gate latency may be timing critical if the serial data signal SD should not be delayed. The enable gate1018is configured to provide the serial clock signal SCK and the frame synchronization signal FSY to the microcontroller1002via the serial clock line1030and the frame synchronization line1032, respectively. The enable gate1018is also configured to provide the frame synchronization signal FSY to the timestamp divider1024, and the timestamp generator1020is configured to generate timestamps corresponding to the audio data communicated in the serial data signal SD. The network device1012is configured to provide the timestamps to the microcontroller1002via the MISO line.

In various examples where an I2S controller (e.g., the I2S controller1022ofFIG.10, without limitation) is not at the master node, such as in the audio system1000ofFIG.10, the I2S controller receives an external clock. By way of non-limiting example, the I2S controller1022ofFIG.10receives the master serial clock signal MSCK from the CODEC1008. The I2S controller may provide some timestamping capability of the frame synchronization signal FSY, and forward the timestamps (e.g., via direct memory access, or “DMA,” without limitation) so that the timestamps may be stored synchronously with any incoming audio sample data. The audio sample data may also be divided into blocks of samples, where each block corresponds to a single timestamp, to reduce the number of timestamps that require further processing.

FIG.11is a block diagram of an audio system segment1100including a network device1102that may be implemented into a microcontroller1114, according to various examples. The network device1102may be configured to perform some of the operations that the network device914ofFIG.9AandFIG.9Band the network device1012ofFIG.10are configured to perform. Also, the network device1102is implemented in hardware, in contrast to software or firmware similar to the network device914and the network device1012. The audio system segments1100, however, is implemented into the microcontroller1114as a peripheral to a processing core (not shown) of the microcontroller1114, in contrast to the network device914and the network device1012, which are implemented as standalone controllers separate from their respective microcontrollers (microcontroller902ofFIG.9Aand microcontroller1002ofFIG.10, respectively).

The network device1102includes a media clock generator1104(“MCG”), an event generator1106(“EG”), a timestamp generator1108(“TS”), a timestamp divider1110(“DIV”), and an I2S peripheral1112. The I2S peripheral1112is electrically connected to a serial data line1116, a serial clock line1118, and a frame synchronization line1120. The I2S peripheral1112is configured to provide the serial clock signal SCK (e.g., to a codec, without limitation). The I2S peripheral may provide or receive the serial data signal SD and a frame synchronization signal FSY (e.g., to/from the codec, without limitation). The I2S peripheral1112may receive a media clock signal MCLK generated by the media clock generator1104.

With the audio system segment1100operating as a talker the I2S peripheral may receive the SD signal from a codec and provide an FSY′ signal to the timestamp divider1110. The timestamp generator1108may generate timestamps corresponding to data of the serial data signal SD. The I2S peripheral1112may synchronize the timestamps based on the media clock MCLK.

With the network device1102operating as a listener the I2S peripheral1112may receive a synchronization signal SYNC generated by the event generator1106. In response the I2S peripheral1112may provide the serial data signal SD, the serial clock signal SCK, and the frame synchronization signal FSY to the codec.

In various examples, one or more components discussed for audio system segment1100above may be implemented in hardware on the same die with a microcontroller (i.e., separate from a processing core of the microcontroller). An IEEE 1588 wallclock and a PLL may also be implemented on the same die.

The I2S may be directly clocked from the media clock generator1104as an internal clock source. Functionality of the I2S controllers ofFIG.9A,FIG.9B, andFIG.10may be merged into the I2S peripheral1112of the audio system segment1100. For example, the media clock generator1104may directly deliver the media clock signal MCK to the I2S peripheral1112, and the timestamp generator1108may be connectable to an FSY′ output of I2S peripheral1112.

In various other examples, synchronization logic may be integrated directly into I2S periphery of a microcontroller. Such I2S periphery (e.g., an I2S transmitter, an I2S receiver, an I2S controller, without limitation) in master mode (the microcontroller is the clock source) may then start the synchronous stream at a configurable, dedicated time point. This may be handled by a software workaround, bringing in some additional uncertainty and increasing software complexity.

FIG.12is a state flow diagram of the audio system segment1100ofFIG.11, according to various examples. The I2S peripheral1112(FIG.11) is extended by a synchronization state machine. In an initial state INIT1204the I2S peripheral1112(including DMA and event generator1106) is configured to switch to a synchronization state SYNC1206. In the synchronization state SYNC1206outputs are active, but the serial clock signal SCK will not run until the event generator1106triggers a switch to the transmit state TRX1202. In the transmit state TRX1202the serial clock signal SCK and the frame synchronization signal FSY and data transmission are running. Synchronized (gated) frame synchronization signal FSY output results in a case of an external clock source (e.g., frame synchronization signal FSY, without limitation) input.

EXAMPLES

A non-exhaustive, non-limiting list of examples follows. Not each of the examples listed below is explicitly and individually indicated as being combinable with all others of the examples listed below and examples discussed above. It is intended, however, that these examples are combinable with all other examples unless it would be apparent to one of ordinary skill in the art that the examples are not combinable.

Example 1: An audio system, comprising: a word select line of a digital audio interface; a serial clock line of the digital audio interface; and hardware circuitry configured to: provide a word select signal to the word select line, the word select signal configured to indicate channels of a serial data signal provided to a serial data line of the digital audio interface; provide a serial clock signal to the serial clock line; and synchronize the serial clock signal to a clock reference stream of an audio stream communicated via a network interface.

Example 2: The audio system of Example 1, wherein the hardware circuitry includes an enable gate configured to start provision of the serial clock signal responsive to an assertion of a synchronization signal.

Example 3: The audio system of Example 2, wherein the hardware circuitry includes an event generator configured to assert the synchronization signal responsive to a presentation timestamp.

Example 4: The audio system according to any one of Examples 1-3, wherein hardware circuitry is configured to synchronize the serial clock signal at least partially based on a synchronized time base, the synchronized time base including a wallclock time.

Example 5: The audio system according to any one of Examples 1-4, wherein the hardware circuitry includes a serial clock divider electrically connected to the serial clock line and the word select line, the serial clock divider configured to divide the serial clock signal provided to the serial clock line to provide the word select signal to the word select line.

Example 6: The audio system according to any one of Examples 1-5, wherein the hardware circuitry includes a timestamp divider configured to divide the word select signal to a rate at which timestamps are provided by the serial data signal.

Example 7: The audio system according to any one of Examples 1-6, comprising a standalone controller including the hardware circuitry.

Example 8: The audio system of Example 7, comprising: an audio device; and a microcontroller electrically connected to the audio device via the serial data line of the digital audio interface, the microcontroller and the audio device configured to receive the word select signal and the serial clock signal provided by the hardware circuitry.

Example 9: The audio system of Example 8, wherein the standalone controller is configured to communicate with the microcontroller via a peripheral interface as a peripheral to the microcontroller.

Example 10: The audio system according to any one of Examples 8 and 9, further comprising a phase locked loop configured to receive a reference clock signal and provide a media clock signal to a media clock divider of the hardware circuitry.

Example 11: The audio system according to any one of Examples 8 and 9, wherein: the audio device is configured to operate as a master talker node; the standalone controller is configured to receive a master serial clock signal and a master word select signal from the audio device; the hardware circuitry includes an enable gate configured to start provision of the serial clock signal responsive to assertions of a synchronization signal from an event generator and the master word select signal.

Example 12: The audio system according to any one of Examples 1-6, comprising a microcontroller including the hardware circuitry implemented as a peripheral to a processing core of the microcontroller.

Example 13: Circuitry for controlling timing of communications, the circuitry comprising: a serial clock line; an enable gate electrically connected to the serial clock line, the enable gate configured to start provision of a serial clock signal to the serial clock line based, at least in part, on an assertion of a synchronization signal; and an event generator configured to assert the synchronization signal responsive to a presentation time stamp from a serial data signal, the presentation time stamp correlated to a synchronized time base, the enable gate and the event generator implemented using hardware circuitry.

Example 14: The circuitry of Example 13, wherein the hardware circuitry is implemented within a microcontroller as a peripheral to a processing core of the microcontroller.

Example 15: The circuitry of Example 13, wherein the hardware circuitry is implemented within a standalone controller separate from a microcontroller.

Example 16: The circuitry of Example 15, comprising a communication interface configured to electrically connect the standalone controller to the microcontroller, the event generator configured to receive the presentation time stamp from the microcontroller through the communication interface.

Example 17: The circuitry according to any one of Examples 13-16, comprising: a word select line configured to conduct a word select signal; and a timestamp divider electrically connected to the word select line, the timestamp divider configured to divide the word select signal to a rate at which timestamps are provided by the serial data signal.

Example 18: The circuitry according to any one of Examples 13-17, comprising: a master word select line configured to deliver a master word select signal from a master node to the enable gate; and an AND gate configured to receive the synchronization signal and the master enable gate, the AND gate configured to trigger the enable gate to provide the serial clock signal to the serial clock line responsive to the assertion of the synchronization signal and an assertion of the word select signal.

Example 19: The circuitry according to any one of Examples 13-18, wherein the enable gate is configured to provide the serial clock signal to the serial clock line responsive to the assertion of the synchronization signal.

Example 20: The circuitry according to any one of Examples 13, 15-17, and 19, wherein the hardware circuitry is configured to provide the serial clock signal and a word select signal as a standalone controller and as a master node.

Example 21: An audio system, comprising: a word select line of a digital audio interface; a serial clock line of the digital audio interface; and hardware circuitry configured to: provide a word select signal to the word select line, the word select signal configured to indicate channels of a serial data signal provided to a serial data line of the digital audio interface; provide a serial clock signal to the serial clock line; and synchronize, at least partially based on a synchronized time base, the serial clock signal to a clock reference stream of an audio stream communicated via a network interface.

Example 22: The audio system of Example 21, further comprising a standalone controller including the hardware circuitry.

Example 23: The audio system of Example 22, further comprising: an audio device; and a microcontroller electrically connected to the audio device via the serial data line of the digital audio interface, the microcontroller configured to receive the word select signal and the serial clock signal provided by the hardware circuitry.

Example 24: The audio system of Example 23, wherein: the microcontroller is configured to generate the serial data signal responsive to the audio stream received via the network interface; the microcontroller is configured to provide the serial data signal to the audio device via the serial data line; and the hardware circuitry is configured to synchronize a starting point at which the audio stream is to be presented by the audio device.

Example 25: The audio system of Example 23, wherein: the audio device is configured to provide the serial data signal to the microcontroller via the serial data line; and the microcontroller is configured to provide the clock reference stream and the audio stream to the network interface.

Example 26: The audio system according to any one of Examples 23-25, wherein the standalone controller is configured to communicate with the microcontroller via a peripheral interface as a peripheral to the microcontroller.

Example 27: The audio system according to any one of Examples 23-26, wherein the standalone controller includes an enable gate configured to enable the serial clock.

Example 28: The audio system of Example 27, wherein the enable gate is configured to be triggered by an event generator.

Example 29: The audio system according to any one of Examples 23-28, further comprising a phase locked loop configured to receive a reference clock and provide a media clock to a clock divider of the hardware circuitry.

Example 30: The audio system according to any one of Examples 23-29, wherein the audio device is also configured to receive the serial clock signal and the word select signal provided by the hardware circuitry.

Example 31: The audio system of Example 23, wherein: the audio device is configured to provide the serial clock signal and the word select signal to the hardware circuitry; and the hardware circuitry is configured to provide the serial clock signal and the word select signal to the microcontroller by relaying the serial clock signal and the word select signal received from the audio device to the serial clock line and the word select line, respectively.

Example 32: The audio system of Example 21, further comprising a microcontroller including the hardware circuitry as a peripheral controller.

Example 33: The audio system of Example 32, further comprising an audio device electrically connected to the hardware circuitry via the word select line and the serial clock line.

Example 34: The audio system according to any one of Examples 32 and 33, wherein the microcontroller includes a media clock generator, an event generator including a phase locked loop, and a timestamp generator.

Example 35: The audio system according to any one of Examples 21-34, wherein the synchronized clock base comprises a wallclock time.

Example 36: The audio system according to any one of Examples 21-35, wherein the clock reference stream and the audio stream comprise an International Electrical and Electronics (IEEE) 1722 stream.

Example 37: The audio system according to any one of Examples 21-36, wherein the network interface includes a wired network interface.

Example 38: The audio system of Example 37, wherein the wired network interface includes an Ethernet interface.

Example 39: The audio system according to any one of Examples 21-38, wherein the audio system includes an automobile audio system.

Example 40: The audio system according to any one of Examples 21-39, wherein the audio system includes a surround sound system.

CONCLUSION

As used in the present disclosure, the terms “module” or “component” may refer to specific hardware implementations configured to perform the actions of the module or component or software objects or software routines that may be stored on or executed by general purpose hardware (e.g., computer-readable media, processing devices, without limitation) of the computing system. In some examples, the different components, modules, engines, and services described in the present disclosure may be implemented as objects or processes that execute on the computing system (e.g., as separate threads, without limitation). While some of the system and methods described in the present disclosure are generally described as being implemented in software (stored on or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated.

As used in the present disclosure, the term “combination” with reference to a plurality of elements may include a combination of all the elements or any of various different subcombinations of some of the elements. For example, the phrase “A, B, C, D, or combinations thereof” may refer to any one of A, B, C, or D; the combination of each of A, B, C, and D; and any subcombination of A, B, C, or D such as A, B, and C; A, B, and D; A, C, and D; B, C, and D; A and B; A and C; A and D; B and C; B and D; or C and D.

While the present disclosure has been described herein with respect to certain illustrated examples, those of ordinary skill in the art will recognize and appreciate that the present invention is not so limited. Rather, many additions, deletions, and modifications to the illustrated and described examples may be made without departing from the scope of the invention as hereinafter claimed along with their legal equivalents. In addition, features from one example may be combined with features of another example while still being encompassed within the scope of the invention as contemplated by the inventor.