Method and system for wireless communication of audio in wireless networks

A method and system for audio transmission in a wireless communication system which transmits digital video and digital audio in High-Definition Multimedia Interface (HDMI) format. Position information of audio packets within the HDMI frame is obtained. Digital audio information including the position information is transmitted from a data source device to a data sink device via a wireless communication medium. At the data sink device, an HDMI frame is reconstructed by inserting received audio packets into horizontal and vertical blanking periods of the HDMI frame.

FIELD OF THE INVENTION

The present invention relates in general to wireless communication, and in particular, to audio transmission in wireless networks.

BACKGROUND OF THE INVENTION

High-Definition Multimedia Interface (HDMI) is a compact audio/video interface for transmitting uncompressed digital data. In an HDMI frame, audio information can be transmitted in Vertical or Horizontal blanking periods and the positions of the audio information segments in the blanking areas are decided according to the synchronization between audio and video (Lip-sync). However, typically Vertical or Horizontal blanking areas are not transmitted in a wireless system in order to reduce the data rate of a communication stream between a transmitter and a receiver.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention provide a method and system for audio transmission in a wireless communication system which transmits digital video and digital audio in High-Definition Multimedia Interface (HDMI) format.

One embodiment of the invention comprises a process for communicating audio information by obtaining position information of audio packets within a High-Definition Multimedia Interface (HDMI) frame. The process further includes transmitting digital audio information including said position information from a data source device to a data sink device via a wireless communication medium. For wireless transmission, audio and video synchronization is maintained by aggregating audio and video packets together during transmission.

The process may further include, at the data sink device, reconstructing an HDMI frame by inserting received audio packets into horizontal and vertical blanking periods of the HDMI frame.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention provide a method and system for audio transmission in a wireless communication system which transmits digital video and digital audio in High-Definition Multimedia Interface (HDMI) format. The digital video and audio is transmitted from a data source at a wireless transmitter to a data sink at a wireless receiver, over a wireless communication medium (e.g., radio frequency (RF)).

One embodiment of the invention comprises a wireless communication system implementing a communication process for transmitting digital audio information from a data source at a wireless transmitter (e.g., a transmitting wireless station) to a data sink at a wireless receiver (e.g., a receiving wireless station), during blanking periods corresponding to an HDMI frame to maintain time synchronization with video information (i.e., maintaining synchronization between audio and video (Lip-sync)). For wireless transmission, audio and video synchronization is maintained by aggregating audio and video packets together during transmission, according to an embodiment of the invention.

One implementation of the communication process comprises transmission of HDMI audio over a wireless channel (e.g., RF channel) to minimize jitter between digital audio and video information. The position information of each audio packet in the HDMI frame at the transmitter side is included in audio packets, wherein the receiver side reconstructs an HDMI frame containing both video and audio information synchronized to each other. Accordingly, the HDMI audio achieves synchronization with video by carrying the position information of audio packets within the HDMI frame.

Two example schemes for generating position information of audio packets within the HDMI frame according to the present invention are described below. A first scheme (Scheme 1) comprises obtaining accurate position for each audio packet within an HDMI frame, wherein the position information is wirelessly transmitted from the transmitter to the receiver. A second scheme (Scheme 2) comprises obtaining position estimation information for all the audio packets in an aggregated audio packet, wherein the aggregated packet is wirelessly transmitted from the transmitter to the receiver.

In each scheme, the receiver reconstructs an HDMI frame by inserting audio packets into horizontal and vertical (H, V) blanking periods. These schemes are described in more detail below.

In one implementation, said Scheme 1 for generating position information of audio packets within the HDMI frame, according to the present invention, comprises adding vertical and horizontal position information to each audio packet and aggregating multiple audio packets together in an HDMI frame utilizing a packetization process at the transmitter side. The digital data is then wirelessly transmitted from the transmitter to a receiver. From the received data, the receiver reconstructs an HDMI frame that is the same or very similar to the HDMI frame at the transmitter, by placing each audio packet at the indicated vertical and horizontal position within blanking areas.

In one implementation, said Scheme 2 comprises estimating audio packet distribution pattern, aggregating multiple audio packets together, and adding audio packet distribution pattern information to the aggregated audio packet when performing packetization at the transmitter. The digital data is then wirelessly transmitted to the receiver.

From the received data, the receiver reconstructs an HDMI frame that is similar to the HDMI frame at the transmitter, by placing each audio packet at the expected vertical line calculated from the audio packet distribution pattern. The jitter between audio and video is bounded to meet Quality of Service (QoS) requirements by estimating the audio packet distribution pattern.

FIG. 1illustrates a system diagram for a wireless HDMI transmitting station10(TX), implementing wireless communication of HDMI audio, according to an embodiment of the invention. The transmitter10comprises an HDMI source11(e.g., a digital media generating device) and a transmitter communication module12.

The HDMI source11comprises an HDMI Transmitter13that provides digital audio, uncompressed video, and other auxiliary data (e.g., packet header) in HDMI format to the transmitter communication module12. The format includes Transition Minimized Differential Signaling (TMDS) signals.

The communication module12includes a decoder/parser14that decodes the TMDS signals, removing vertical and horizontal areas. The communication module12further includes an audio/video (A/V) packetization controller15that packetizes the audio, uncompressed video and other auxiliary data into packets for transmission on a wireless channel via a wireless transmission module16(e.g., 60 GHz wireless transmission module) via multiple antennas over a wireless channel.

In one implementation, the transmitter communication module12may comprise AV data processing and communication logic12A implementing the functions of the modules14and15, and a Media Access (MAC) layer12B and a physical (PHY) layer12C implementing the functions of the module16. The module12further comprises memory12D and hardware processor12E.

FIG. 2illustrates a system diagram for a wireless HDMI receiving station20, implementing wireless communication of HDMI audio, according to an embodiment of the invention. The receiver20comprises a communication module21and an HDMI sink22(e.g., a digital media player device).

The receiver communication module21includes a wireless receiving module for wirelessly receiving the transmission from the transmitter10over multiple antennas (e.g., over 60 GHz wireless channel). The receiver communication module21further includes an audio/video de-packetization controller24that de-packetizes the information from packets received from the wireless channel.

The receiver communication module21further includes an encoder module25that encodes the de-packetized information into TMDS format, and reconstructs HDMI frames (adding Vertical and Horizontal areas). The encoder module25then passes the TMDS signals to the HDMI sink22for consumption (e.g., HDMI receiver in the HDMI sink).

In one implementation, the receiver communication module21may comprise audio/video (AV) data processing and communication logic21A implementing the functions of the modules24and25, and a MAC layer21B and PHY layer21C implementing the functions of the wireless receiving module23. The module21further comprises memory21D and hardware processor21E.

FIG. 3shows an example HDMI data frame structure30for 720 p digital video, utilized for HDMI communication between the transmitter10and the receiver20, according to an embodiment of the invention. As illustrated, there are 45 lines for vertical blanking and 138 pixels for horizontal blanking at each vertical line which carries active video information. Digital audio or other auxiliary data can be carried in data island periods31within vertical and horizontal blanking areas32,33, respectively.

When an HDMI frame is formed at the HDMI source11(FIG. 1), audio/video synchronization requirements determine placement of audio packets in the blanking areas32,33(FIG. 3). The HDMI frame structure30is designed for synchronous transmission. However, since for wireless transmission (such as 60 GHz wireless communication) only packet-based asynchronous transmission is utilized, the present invention accounts for synchronization between audio and video for wireless transmission of audio and video from the HDMI source11to the HDMI sink22using HDMI frame conversion and packetization.

FIG. 4illustrates an example process40for HDMI frame conversion and packetization at the communication module12of the wireless transmitter10(FIG. 1), according to an embodiment of the invention. The process40comprises process blocks for processing an HDMI frame30arriving as signals from HDMI source11to the communication module12at three TMDS channels, wherein:Block41: Parse the signals at 3 TMDS channels representing an HDMI frame.Block42: Does the HDMI frame include a data island? If yes, proceed to block43, else no audio in the HDMI frame, proceed to block49.Block43: Record (H,V) position of the current data island in the HDMI frame.Block44: Parse a data packet header at TMDS channel0.Block45: Determine if the packet is an audio packet? If yes, proceed to block46, else proceed to block48.Block46: Determine if additional audio packets are to be aggregated? If yes, proceed to block41, else proceed to block47.Block47: Generate an aggregated audio packet (FIGS. 6-7) from the received audio packets for wireless transmission. End.Block48: Generate an Info data packet for wireless transmission (Info data packet typically carries control messages). End.

The transmitter10records the starting position of each data island and parses a data packet header at TMDS channel0from the HDMI source11. If the data packet is an audio packet, the transmitter10considers whether to wait for more incoming audio packets according to an audio/video jitter requirement and the expected size of an aggregated audio packet. If the data packet is an Infoframe packet, typically the transmitter10does not aggregate the Infoframe packets.

In one example, the HDMI frame conversion is performed by the decoder/parser14(FIG. 1), and packetization is performed by the packetization controller15, at the transmitter10. In one example, frame conversion includes decoding TMDS coded data signals and removing the H and V areas from a HDMI frame. In one example, packetization includes separating audio, video and data information, and placing them into packets.

FIG. 5illustrates an example reverse process50for de-packetization and HDMI frame reconstruction at the communication module21of the wireless receiver20(FIG. 2), according to an embodiment of the invention. The process50comprises process blocks for processing an arriving data packet from the transmitter10over the wireless channel, wherein:Block51: Determine if the arriving packet is an aggregated audio packet? If yes, proceed to block52, else proceed to block55.Block52: De-packetize by parsing the aggregated packet header and each audio packet header in the aggregated packet header.Block53: Obtain (H,V) position of the data islands for each audio packet.Block54: Generate HDMI signals at three TMDS channels according to the (H,V) positions, with audio packet header and V/H blanking at channel0, and audio data at channels1and2. End.Block55: Obtain (H,V) position of the data island for the Infoframe packet. Proceed to block54.

In this example, de-packetization is performed by the de-packetization module24(FIG. 2), and HDMI frame reconstruction is performed by the TMDS encoder/transmitter module25, at the receiver20. In one example, depacketization includes extracting information from packets in reverse manner as packetization described above.

For generating aggregated audio packets at the transmitter10according to embodiments of the invention, said Scheme 1 for generating position information of audio packets within the HDMI frame includes carrying an accurate position for each audio packet within the HDMI frame. Referring to an example aggregated audio packet format60inFIG. 6, according to Scheme 1 of the present invention, vertical and horizontal position information (V/H Position field) is added to each audio packet and multiple audio packets are aggregated together during packetization by the packetization controller15at the transmitter10.

The receiver20then reconstructs the same HDMI frame at the transmitter10by placing each audio packet at the indicated vertical and horizontal position within blanking areas of the reconstructed HDMI frame. The V/H Position field in the aggregated packet60contains the vertical and horizontal position information of one audio packet such as n octets (e.g., 12 bits may be used to represent each vertical or horizontal position). The value n in the packet60is an integer such as 3, 4, etc. The Audio Packet Length field in the aggregated packet60indicates the length of an audio packet in octets.

Said Scheme 2 for generating position information of audio packets within the HDMI frame according to an embodiment of the invention, includes carrying the position estimation information for all the audio packets in an aggregated audio packet. Referring to an example aggregated audio packet format70inFIG. 7, according to Scheme 2 of the present invention, 4 octets representing V/H Position and Line distance, are introduced in the header for each audio packet.

To reduce header overhead, the audio packet distribution pattern is estimated. For example, if audio packets at the vertical lines10,12,13,16,18need to be aggregated, it can be estimated that the line distance between two neighboring packets is 2, and inFIG. 7, V/H Position of the first audio packet field is set to 10 and the Line distance of two neighboring audio packets field is set to 2. Multiple audio packets are aggregated together in the aggregated packet70, wherein audio packet distribution pattern information (i.e., Line distance of two neighboring audio packets) is added by the packetization controller15to the aggregated audio packet70during packetization at the transmitter10.

The receiver20reconstructs an HDMI frame similar to the HDMI frame at the transmitter10by placing each audio packet at the expected vertical line calculated from the audio packet distribution pattern. The jitter between audio and video is bounded to meet the QoS requirement by estimating the audio packet distribution pattern. Line distance of two neighboring audio packets field in the aggregated packet70indicates the average number of lines between two audio packets. For example, the receiver side can reconstruct the vertical line numbers of the audio packets at vertical lines10,12,14,16,18, and it needs not be the same as the transmitter side (as line14deviates), with minor differences from the original positions at vertical lines10,12,13,16,18.

An alternative embodiment for the audio/video synchronization according to the present invention comprises a vertical-line based approach which mixes audio and video information as shown by the example aggregated packet80inFIG. 8, during packetization at the transmitter10. The Line type field in aggregated packet80sets to “1” if the first vertical line in the aggregated packet80carries active data, and sets to “0” if the first vertical line is a vertical blanking line. The V position field indicates the vertical line index of the first vertical line in the aggregated packets. The Number of lines field indicates number of vertical lines in the current packet80.

The Line type field sets to “1” if the first vertical line in the aggregated packet80carries active data, and sets to “0” if the first vertical line is a vertical blanking line. The Infoframe exists field indicates whether the Infoframe data are carried in the current packet80. The Audio data exists field indicates whether the audio data is carried in the current packet80. The Video data exists field indicates whether the video data is carried in the current packet80.

The invention provides data rate reduction by avoiding transmission of blanking information while achieving audio/video synchronization performance. The receiver reconstructs the HDMI frame by inserting audio packets into vertical and horizontal blanking periods, while achieving audio/video synchronization. Even though audio and video use different clocks, the invention directly uses position information to avoid conversion at both transmitter side and receiver side.

As is known to those skilled in the art, the aforementioned example architectures described above, according to the present invention, can be implemented in many ways, such as program instructions for execution by a processor, as software modules, microcode, as computer program product on computer readable media, as logic circuits, as application specific integrated circuits, as firmware, as consumer electronic devices, as wireless transmitters, as wireless receiver, etc. Further, embodiments of the invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements.