Source: https://patents.google.com/patent/US7620062B2/en
Timestamp: 2019-04-23 10:30:48+00:00

Document:
A method and apparatus of optimizing transmission (both real time and continuous) of a number of multimedia data packets between a multimedia source device and a multimedia display device is disclosed. In the described embodiment, the multimedia source device and the display device are coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device. The method can be carried out by following at least the following operations. Providing a test pattern by the multimedia source device on the main link, determining a transmission quality factor of the main link based upon the test pattern, and optimizing the transmission of the multimedia data packets based upon the transmission quality factor.
This patent application takes priority under 35 U.S.C. 119(e) to (i) U.S. Provisional Patent Application No. 60/467,804, filed on May 1, 2003 entitled “DIGITAL/ANALOG VIDEO INTERCONNECT AND METHODS OF USE THEREOF” by Kobayashi, (ii) U.S. Provisional Patent Application No.: 60/504,060 filed on Sep. 18, 2003, entitled “DIGITAL/ANALOG VIDEO INTERCONNECT AND METHODS OF USE THEREOF” by Kobayashi, (iii) U.S. Provisional Patent Application No.: 60/474,085 filed on May 28, 2003, entitled “DIGITAL/ANALOG VIDEO INTERCONNECT AND METHODS OF USE THEREOF” by Kobayashi, and (iv) U.S. Provisional Patent Application No.: 60/474,084 filed on May 28, 2003, entitled “SIMPLE ENUMERATION METHOD FOR THE LINK CLOCK RATE AND THE PIXEL/AUDIO CLOCK RATE” by Kobayashi each of which are hereby incorporated by reference herein in their entirety. This application is also related to the following co-pending U.S. Patent applications, which are filed concurrently with this application and each of which are herein incorporated by reference, (i) U.S. patent application Ser. No. 10/726,802, entitled “METHOD OF ADAPTIVELY CONNECTING A VIDEO SOURCE AND A VIDEO DISPLAY” naming Kobayashi as inventor; (ii) U.S. patent application Ser. No. 10/726,438, entitled “METHOD AND APPARATUS FOR EFFICIENT TRANSMISSION OF MULTIMEDIA DATA PACKETS” naming Kobayashi as inventor; (iii) U.S. patent application Ser. No. 10/726,794, entitled “PACKET BASED VIDEO DISPLAY INTERFACE AND METHODS OF USE THEREOF”, naming Kobayashi as inventor; (iv) U.S. patent application Ser. No. 10/727,131, entitled “USING AN AUXILARY CHANNEL FOR VIDEO MONITOR TRAINING”, naming Kobayashi as inventor; (v) U.S. patent application Ser. No. 10/726,350, entitled “TECHNIQUES FOR REDUCING MULTIMEDIA DATA PACKET OVERHEAD”, naming Kobayashi as inventor; (vi) U.S. patent application Ser. No. 10/726,362, entitled “PACKET BASED CLOSED LOOP VIDEO DISPLAY INTERFACE WITH PERIODIC STATUS CHECKS”, naming Kobayashi as inventor; (vii) U.S. patent application Ser. No. 10/726,895, entitled “MINIMIZING BUFFER REQUIREMENTS IN A DIGITAL VIDEO SYSTEM”, naming Kobayashi as inventor; (viii) U.S. patent application Ser. No. 10/726,441, entitled “VIDEO INTERFACE ARRANGED TO PROVIDE PIXEL DATA INDEPENDENT OF A LINK CHARACTER CLOCK”, naming Kobayashi as inventor; and (ix) U.S. patent application Ser. No. 10/726,934, entitled “ENUMERATION METHOD FOR THE LINK CLOCK RATE AND THE PIXEL/AUDIO CLOCK RATE”, naming Kobayashi as inventor. This application is also related to the following co-pending applications: (x) U.S. patent application Ser. No. 10/909,103, entitled “USING PACKET TRANSFER FOR DRIVING LCD PANEL DRIVER ELECTRONICS” filed Jul. 29, 2004, naming Kobayashi as inventor; and (xi) U.S. patent application Ser. No. 10/909,085, entitled “PACKET BASED STREAM TRANSPORT SCHEDULER AND METHODS OF USE THEREOF” filed Jul. 29, 2004, naming Kobayashi as inventor.
Today's display interconnect landscape includes the VGA (analog) and DVI (digital) for desktop display interconnect applications as well as LVDS (digital) for internal connectivity applications within laptops and other all-in-one devices. Graphics IC vendors, display controller IC vendors, monitor manufacturers and PC OEMs as well as desktop PC consumers, to one degree or another, must factor interface choice into their design, product definition, manufacturing, marketing and purchase decisions. For example, if a consumer purchases a PC with an analog VGA interface then the consumer must either purchase an analog monitor or a digital monitor in which the analog video signal provided by the VGA interface has been digitized by way of an inline analog to digital converter (ADC) or an ADC built into the particular monitor.
Therefore, it would be desirable to optimize the transmission of data packets in a packet based digital display in real time based upon particular operating conditions.
According to some embodiments of the invention, a method of optimizing transmission of a number of multimedia data packets between a multimedia source device and a multimedia display device is disclosed. In the described embodiment, the multimedia source device and the display device are coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device. The method can be carried out by following at least the following operations. Providing a test pattern by the multimedia source device on the main link, determining a transmission quality factor of the main link based upon the test pattern, and optimizing the transmission of the multimedia data packets based upon the transmission quality factor.
In another embodiment, an apparatus for optimizing transmission of a number of multimedia data packets between a multimedia source device and a multimedia display device coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device is disclosed. The apparatus includes at least means for providing a test pattern by the multimedia source device on the main link, means for determining a transmission quality factor of the main link based upon the test pattern, and means for optimizing the transmission of the multimedia data packets based upon the transmission quality factor.
In yet another embodiment of the invention, computer program product for optimizing transmission of a number of multimedia data packets between a multimedia source device and a multimedia display device coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device is disclosed. The computer program product includes computer code for providing a test pattern by the multimedia source device on the main link, computer code for determining a transmission quality factor of the main link based upon the test pattern, computer code for optimizing the transmission of the multimedia data packets based upon the transmission quality factor, and computer readable medium for storing the computer code.
FIGS. 2A-2C illustrates a video interface system that is used to connect a video source and a video display unit in accordance with a number of embodiments of the invention.
FIGS. 20-24 illustrate various computer based implementations of the invention.
A method and apparatus suitable for optimizing transmission of a number of multimedia data packets between a multimedia source device and a multimedia display device is disclosed. In the described embodiment, the multimedia source device and the display device are coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device.
A method of optimizing transmission of a number of multimedia data packets between a multimedia source device and a multimedia display device is disclosed. In the described embodiment, the multimedia source device and the display device are coupled by way of a unidirectional main link arranged to carry the multimedia data packets from the multimedia source device and the multimedia display device and a bi-directional auxiliary channel arranged to transfer information between the multimedia source device and the multimedia display device.
As a method, the invention can be carried out by following at least the following operations. Providing a test pattern by the multimedia source device on the main link, determining a transmission quality factor of the main link based upon the test pattern, and optimizing the transmission of the multimedia data packets based upon the transmission quality factor.
For example, FIG. 1 shows a generalized representation of a cross platform packet based digital video display interface 100 in accordance with an embodiment of the invention. The interface 100 connects a transmitter 102 to a receiver 104 by way of a physical link 106 (also referred to as a pipe). In the described embodiment, a number of data streams 108-112 are received at the transmitter 102 that, if necessary, packetizes each into a corresponding number of data packets 114. These data packets are then formed into corresponding data streams each of which are passed by way of an associated virtual pipe 116-120 to the receiver 104. It should be noted that the link rate (i.e., the data packet transfer rate) for each virtual link can be optimized for the particular data stream resulting in the physical link 106 carrying data streams each having an associated link rate (each of which could be different from each other depending upon the particular data stream). The data streams 110-114 can take any number of forms such as video, graphic, audio, etc.
Typically, when the source is a video source, the data streams 110-114 include various video signals that can have any number and type of well-known formats, such as composite video, serial digital, parallel digital, RGB, or consumer digital video. The video signal can be an analog video signal provided the source 102 includes some form of an analog video source such as for example, an analog television, still camera, analog VCR, DVD player, camcorder, laser disk player, TV tuner, set top box (with satellite DSS or cable signal) and the like. The source 102 can also include a digital image source such as for example a digital television (DTV), digital still camera, and the like. The digital video signal can be any number and type of well known digital formats such as, SMPTE 274M-1995 (1920×1080 resolution, progressive or interlaced scan), SMPTE 296M-1997 (1280×720 resolution, progressive scan), as well as standard 480 progressive scan video.
For example, if the data stream 110 is an analog type signal, the an analog to digital converter (not shown) included in or coupled to the transmitter 102 will digitize the analog data which is then packetize by a packetizer that converts the digitized data stream 110 into a number of data packets 114 each of which will be transmitted to the receiver 104 by way of the virtual link 116. The receiver 104 will then reconstitute the data stream 110 by appropriately recombining the data packets 114 into their original format. It should be noted that the link rate is independent of the native stream rates. The only requirement is that the link bandwidth of the physical link 106 be higher than the aggregate bandwidth of data stream(s) to be transmitted . In the described embodiment, the incoming data (such as pixel data in the case of video data) is packed over the respective virtual link based upon a data mapping definition. In this way, the physical link 106 (or any of the constituent virtual links) does not, as does conventional interconnects such as DVI, carry one pixel data per link character clock.
Even for a link whose link rate (which is the serial link bit rate/10 for a link that uses 10-bit character such as 8B/10B characters) may be different from the pixel clock rate, there is a benefit in defining the link rate with these four parameters, A′, B′, C′, and D′: The benefit is the simplicity in regenerating pixel/audio clocks from a link clock. For example, let's say the link rate is set as A′=6, B′=3, C′=7, and D′=0 and the corresponding link rate is 135 MHz. However, suppose the pixel clock rate is set as A=8, B=3, C=6, and D=0 (=108 MHz), then the pixel clock can be generated from link clock as pixel clock rate is equal to the link rate*22/51.
A main link data packet 400 shown in FIG. 4A includes a main link packet header 402 as shown in FIG. 4B that is formed of 16 bits where bits 3-0 are the Stream ID (SID) (indicating that maximum stream count is 16), bit 4 is the Time Stamp (TS) LSB. When bit 4 is equal to 1, this packet header has the least significant 4 bits of Time Stamp value (used only for uncompressed video stream). Bit 5 is a Video frame sequence bit which acts as the least significant bit of the frame counter which toggles from “0” to “1” or from “1” to “0” at the video frame boundary (used only for uncompressed video stream). Bits 7 and 6 are reserved whereas bits 8 through 10 are a 4-bit CRC (CRC) that checks errors for the previous eight bits. Bits 15-12 are Time Stamp/Stream ID Inversion. (TSP/SIDn) which for uncompressed video are used as four bits of 20-bit Time Stamp value.
The single bi-directional auxiliary channel 224 provides a conduit to for various support functions useful for link set up and supporting main link operations as well as to carry auxiliary application data such as USB traffic. For example, with the auxiliary channel 224, a display device can inform the source device of events such as sync loss, dropped packets and the results of training sessions (described below). For example, if a particular training session fails, the transmitter 102 adjusts the main link rate based upon pre-selected or determined results of the failed training session. In this way, the closed loop created by combining an adjustable, high speed main link with a relatively slow and very reliable auxiliary channel allows for robust operation over a variety of link conditions. It should be noted that in some cases (an example of which is shown in FIG. 5B), a logical bi-directional auxiliary channel 520 can be established using a portion 522 of the bandwidth of the main link 222 to transfer data from the source device 202 to the sink device 204 and a uni-directional back channel 524 from the sink device 204 to the source device 202. In some applications, use of this logical bi-directional auxiliary channel may be more desirable than using a half-duplex bi-directional channel described in FIG. 5A.
In the described embodiment, the source device physical layer 1202 includes an electrical sub layer 1202-1 and a logical sub layer 1202-2. The electrical sub layer 1202-1 includes all circuitry for interface initialization/operation such as hot plug/unplug detection circuit, drivers/receivers/termination resistors, parallel-to-serial/serial-to-parallel conversions, and spread-spectrum-capable PLL's . The logical sub layer 1202-2 includes circuitry for, packetizing/de-packetizing, data scrambling/de-scrambling, pattern generation for link training, time-base recovery circuits, and data encoding/decoding such as 8B/10B (as specified in ANSI X3.230-1994, clause 11) that provides 256 link data characters and twelve control characters (an example of which is shown as FIG. 13) for the main link 222 and Manchester II for the auxiliary channel 224 (see FIG. 14).
The invention will now be described in terms of state diagrams shown in FIGS. 18 and 19 described below. Accordingly, FIG. 18 shows the source state diagram described below. At an off state 1802, the system is off such that the source is disabled. If the source is enabled, then the system transitions to a standby state 1804 suitable for power saving and receiver detection. In order to detect whether or not the receiver is present (i.e., hot plug/play), the auxiliary channel is periodically pulsed (such as for 1 us every 10 ms) and a measure of a voltage drop across the termination resistors during the driving is measured. If it is determined that a receiver is present based upon the measured voltage drop, then the system transitions to a detected receiver state 1806 indicating that a receiver has been detected, i.e., a hot plug event has been detected. If, however, there is no receiver detected, then the receiver detection is continued until such time, if ever, a receiver is detected or a timeout has elapsed. It should be noted that in some cases the source device may choose to go to “OFF” state from which no further display detection is attempted.
FIGS. 20-24 show particular implementations of the cross platform display interface.
The methodology of the invention will now be described in terms of a number of flowcharts each describing a particular process for enabling the invention. Specifically, FIGS. 25-29 describe a number of interrelated processes that when used singly or in any combination described aspects of the invention.
CPUs 2910 are also coupled to one or more input/output devices 890 that may include, but are not limited to, devices such as video monitors, track balls, mice, keyboards, microphones, touch-sensitive displays, transducer card readers, magnetic or paper tape readers, tablets, styluses, voice or handwriting recognizers, or other well-known input devices such as, of course, other computers. Finally, CPUs 2910 optionally may be coupled to a computer or telecommunications network, e.g., an Internet network or an intranet network, using a network connection as shown generally at 2995. With such a network connection, it is contemplated that the CPUs 2910 might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using CPUs 2910, may be received from and outputted to the network. The above-described devices and materials will be familiar to those of skill in the computer hardware and software arts.
commencing the optimization immediately subsequent to the detection of the hot plug event.
determining if the transmission rate of the multimedia data packets on the main link is greater than a minimum transmission rate.
determining the bit error rate based upon the reduced main link transmission rate.
4. A method as recited in claim 3, wherein a main link bandwidth is at least equal to an aggregate of the virtual link bandwidths.
5. A method as recited in claim 1, wherein the bi-directional auxiliary channel is formed of a uni-directional back channel configured to carry information from the display device to the source device and a uni-directional forward channel included as part of the main channel for carrying information from the source device to the display device in concert with the back channel.
6. A method as recited in claim 1, wherein the number of multimedia data packets are divided into a number or associated multimedia data packets streams.
8. A method as recited in claim 7 wherein each of the virtual link rates associated with each of the virtual links are optimized.
means for commencing the optimization immediately subsequent to the detection of the hot plug event.
means for determining if the transmission rate of the multimedia data packets on the main link is greater than a minimum transmission rate when the bit error rate is determined to be greater than the predetermined threshold bit error rate.
means for determining the bit error rate based upon the reduced main link transmission rate.
12. An apparatus as recited in claim 11, wherein a main link bandwidth is at least equal to an aggregate of the virtual link bandwidths.
13. An apparatus as recited in claim 9, wherein the bi-directional auxiliary channel is formed of a uni-directional back channel configured to carry information from the display device to the source device and a uni-directional forward channel included as part of the main channel for carrying information from the source device to the display device in concert with the back channel.
14. An apparatus as recited in claim 9, wherein the number of multimedia data packets are divided into a number or associated multimedia data packets streams.
16. An apparatus as recited in claim 15 wherein each of the virtual link rates associated with each of the virtual links are optimized.
computer code for determining if the transmission rate of the multimedia data packets on the main link is greater than a minimum transmission rate when the bit error rate is determined to be greater than the predetermined threshold bit error rate.
computer code for determining the bit error rate based upon the reduced main link transmission rate.
21. Computer program product as recited in claim 20, wherein a main link bandwidth is at least equal to an aggregate of the virtual link bandwidths.
22. Computer program product as recited in claim 18, wherein the bi-directional auxiliary channel is formed of a uni-directional back channel configured to carry information from the display device to the source device and a uni-directional forward channel included as part of the main channel for carrying information from the source device to the display device in concert with the back channel.
23. Computer program product as recited in claim 18, wherein the number of multimedia data packets are divided into a number or associated multimedia data packets streams.
25. Computer program product as recited in claim 24 wherein each of the virtual link rates associated with each of the virtual links are optimized.
Austrian Search and Exam Report dated Feb. 9, 2007 issued in corresponding Singapore Application No. 200401975-8 (GENSP105SG).
Examination Report dated Dec. 7, 2006 from related European Patent Application No. 04252055.1.
Examination Report dated Mar. 15, 2006 from European Patent Application No. 04255610.0.
Examination Report dated Nov. 13, 2006 from related European Patent Application No. 04255609.2.
Examination/Search Report dated Mar. 1, 2006 from related Singapore Patent Application No. 200402057-4.
Notice of Allowance mailed Sep. 24, 2007 from U.S. Appl. No. 10/726,802.
Office Action dated Mar. 31, 2008 from U.S. Application No. 10/726,794.
Office Action mailed Dec. 5, 2006 from U.S. Appl. No. 10/726,802 (GENSP014).
Office Action mailed Jul. 9, 2007 from U.S. Appl. No. 10/726,895.
Supplemental Notice of Allowance mailed Nov. 6, 2007 from U.S. Appl. No. 10/726,802.
U.S. Office Action mailed Aug. 5, 2005 from U.S. Appl. No. 10/726,934.
U.S. Office Action mailed Aug. 9, 2005 from U.S. Appl. No. 10/727,131.

References: Application No. 60
 Application No. 200401975
 Application No. 04252055
 Application No. 04255610
 Application No. 04255609
 Application No. 200402057
 Application No. 10