Patent Description:
HBBTV (Hybrid Broadcast Broadband TV) is a content distribution platform compatible with DVB (Digital Video Broadcasting), which can provide signaling, transmission, and presentation mechanisms for enhancing the interactivity of TV services. With rapid development of HBBTV, the configuration of HBBTV is becoming more and more powerful, and the application is more and more complete. For example, a multimedia playback apparatus supporting the HBBTV standard can be connected to the DVB broadcasting network to receive broadcast video content, and can also be connected to the Internet through a broadband interface, which allows the multimedia playback apparatus to receive audio content from the Internet. From <NPL>, an end-to-end HbbTV-compliant platform for the preparation. (adaptive) delivery and synchronized playout of related hybrid (broadcast and broadband) contents is known. From <CIT> an apparatus for supporting hybrid broadcasting in a digital broadcasting system is known. From <CIT> a synchronization method between media streams from different networks is known.

Since the synchronization accuracy for audio/video synchronization timing for a given terminal required by the HBBTV <NUM>. <NUM> standard is <NUM>, it is challenging to design a multimedia playback apparatus to satisfy the timing requirement.

The present invention is defined by the subject matter of the appended claims.

An embodiment of the present invention provides a multimedia playback apparatus. The multimedia playback apparatus includes a non-volatile memory (<NUM>) and a processor (<NUM>). The non-volatile memory (<NUM>) stores an analyzing program (<NUM>), a first driver (<NUM>), and a second driver (<NUM>). The processor (<NUM>) executes the analyzing program (<NUM>), the first driver (<NUM>), and the second driver (<NUM>) to perform the following steps:
obtaining Motion Picture Expert Group, MPEG, Transport Streams, TS, comprising a video stream (<NUM>) and an audio stream (<NUM>) from a first source terminal (<NUM>) and a second source terminal (<NUM>), respectively; parsing a first packet in the video stream (<NUM>) to obtain a first presentation timestamp, PTS, referred to as video PTS, and parsing a second packet, that corresponds to the first packet, in the audio stream (<NUM>) to obtain a second PTS, referred to as audio PTS, wherein the first source terminal (<NUM>) is a broadcast source, and the second source terminal (<NUM>) is a broadband source; wherein the processor (<NUM>) obtains the video PTS and the audio PTS from a first PTS header of the first packet and a second PTS header of the second packet, respectively, wherein the processor (<NUM>), executing the analyzing program (<NUM>), the first driver (<NUM>), and the second driver (<NUM>) is further adapted to perform the following steps: converting the video PTS on the video PTS timeline (z) into a corresponding video Timed External Media Information, TEMI, on a video TEMI timeline (x) with reference to the video PTS timeline (z), converting the video TEMI into an audio TEMI on an audio TEMI timeline (y), wherein converting the video TEMI into the audio TEMI comprises using additional video and audio TEMI time data and a predefined point of correlation on the video TEMI timeline (x) and the audio TEMI timeline (y), respectively; and converting the audio TEMI into a corresponding audio PTS, C_APTS, on an audio PTS timeline (w); and to further perform the following steps:
modifying the second packet by replacing the audio PTS with the converted PTS, C_APTS, in the second packet to generate a modified second packet; decoding the first packet and the modified second packet by the first driver and the second driver to generate video content and audio content, respectively; and forwarding the first packet to the first driver, and forwarding the modified second packet with the converted PTS, C_APTS, and the video PTS to the second driver; thereby synchronizing the audio content with the video content by the second driver using the converted PTS, C_APTS, and the video PTS.

Another embodiment of the present invention provides a computer-implemented method for synchronizing media streams. The method includes the following steps:
obtaining Motion Picture Expert Group, MPEG, Transport Streams, TS, comprising a video stream (<NUM>) and an audio stream (<NUM>) from a first source terminal (<NUM>) and a second source terminal (<NUM>), respectively; parsing a first packet in the video stream (<NUM>) to obtain a first presentation timestamp, PTS, referred to as video PTS, and parsing a second packet, that corresponds to the first packet, in the audio stream (<NUM>) to obtain a second PTS, referred to as audio PTS, wherein the first source terminal (<NUM>) is a broadcast source, and the second source terminal (<NUM>) is a broadband source; wherein the processor (<NUM>) obtains the video PTS and the audio PTS from a first PTS header of the first packet and a second PTS header of the second packet, respectively, wherein by the processor (<NUM>), executing the analyzing program (<NUM>), the first driver (<NUM>), and the second driver (<NUM>) is further adapted to perform the following steps: converting the video PTS on the video PTS timeline (z) into a corresponding video Timed External Media Information, TEMI, on a video TEMI timeline (x) with reference to the video PTS timeline (z), converting the video TEMI into an audio TEMI on an audio TEMI timeline (y), wherein converting the video TEMI into the audio TEMI comprises using additional video and audio TEMI time data and a predefined point of correlation on the video TEMI timeline (x) and the audio TEMI timeline (y), respectively; and converting the audio TEMI into a corresponding audio PTS, C_APTS, on an audio PTS timeline (w); and to further perform the following steps:
modifying the second packet by replacing the audio PTS with the converted PTS, C_APTS, in the second packet to generate a modified second packet; decoding the first packet and the modified second packet by the first driver and the second driver to generate video content and audio content, respectively; and forwarding the first packet to the first driver, and forwarding the modified second packet with the converted PTS, C_APTS, and the video PTS to the second driver; thereby synchronizing the audio content with the video content by the second driver using the converted PTS, C_APTS, and the video PTS.

The following description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense.

The following description is presented to enable one of ordinary skill in the art to make and use the invention, and is provided in the context of a patent application and its requirements. Various modifications to the embodiments and the generic principles and features described herein will be readily apparent to those skilled in the art.

<FIG> is a diagram of a multimedia playback apparatus in accordance with an embodiment of the invention.

The multimedia playback apparatus <NUM> may be a standalone set-top box (STB) or an STB that is integrated into an electronic device such as a smart TV. As depicted in <FIG>, the multimedia playback apparatus <NUM> may include a processor <NUM>, a volatile memory <NUM>, a non-volatile memory <NUM>, an antenna <NUM>, a broadband interface <NUM>, a video interface <NUM>, and an audio interface <NUM>.

The processor <NUM> may be a central processing unit (CPU), a general-purpose processor, etc., but the invention is not limited thereto. The volatile memory <NUM> may be implemented by a dynamic random access memory (DRAM) or a static random access memory (SRAM), and be used as a workspace of a program and/or driver executed by the processor <NUM>. The non-volatile memory <NUM> may be a hard-disk driver (HDD), a solid-state disk (SSD), a read-only memory (ROM), a flash memory, etc., but the invention is not limited thereto.

The non-volatile memory <NUM> may store an analyzing program <NUM>, a video driver <NUM>, an audio driver <NUM>, a video post-processing program <NUM>, and an audio post-processing program <NUM>. The processor <NUM> may load the analyzing program <NUM>, video driver <NUM>, audio driver <NUM>, the video post-processing program <NUM>, and the audio post-processing program <NUM> from the non-volatile memory <NUM> into the volatile memory <NUM> for execution. For example, the analyzing program <NUM> may parse the input media streams from different sources and transmit the media streams respectively to the video driver <NUM> and audio driver <NUM> for subsequent processing, where details of the analyzing program <NUM> will be discussed later.

The video post-processing program <NUM> may perform video post processing on the decoded video images, such as image scaling, deblocking, sharpening/unsharpening, luminance alterations, blurring/denoising, etc., to generate an output video signal, and the multimedia playback apparatus <NUM> may output the output video signal <NUM> via the video interface <NUM>. The video interface <NUM> may be implemented using a VGA (Video Graphics Array), HDMI (High-Definition Multimedia Interface), DisplayPort interface, but the invention is not limited thereto.

The audio post-processing program <NUM> may perform audio post processing on the decoded audio signal, such as resampling, bandpass filtering, playback equalization, etc., but the invention is not limited thereto, and the multimedia playback apparatus <NUM> may output the output audio signal <NUM> via the audio interface <NUM>. The audio interface <NUM> may be implemented using an analog audio interface (e.g., an RCA, mini-RCA, <NUM>, <NUM> plug, etc.) or a digital audio interface (e.g., a USB, HDMI, DisplayPort, SPDIF coaxial, Toslink interface, etc.), but the invention is not limited thereto. In some embodiments, the audio interface <NUM> and the video interface <NUM> can be integrated as one interface, such as an HDMI or DisplayPort interface.

For example, media streams are composed of Access Units (AUs), which are also referred to as Media Data Units (MDUs), within various MPEG (Motion Picture Expert Group) standards. An AU is the smallest timestamped media unit in a specific media stream. For example, a video AU is an encoded picture, whereas an audio AU is a set of encoded audio samples. At the server side (e.g., the first source terminal <NUM> or second source terminal <NUM>), the AUs of each particular media stream are captured, encoded, and timestamped at particular clock rates. Thereafter, the AUs are packetized for transmission over the network or broadcasting. For example, a video stream may have <NUM> AUs (i.e., <NUM> frames) in one second, whereas an audio stream may have <NUM> AUs (i.e., <NUM> samples) in one second.

At the receiver side (e.g., the multimedia playback apparatus <NUM>), the video driver <NUM> and audio driver <NUM> may perform de-packetization, decoding, post-processing to enable a reconstruction of original temporal dependences between the access units (AUs) within and between the involved media streams. This end-to-end process for ensuring a proper and smooth media play-out is known as media sync.

In an embodiment, the multimedia playback apparatus <NUM> may support the HBBTV <NUM> standard or a later-developed revision, which allows the multimedia playback apparatus <NUM> to receive a first media stream <NUM> from a first source terminal <NUM> via the antenna <NUM>, and receive a second media stream <NUM> from a second source terminal <NUM> via the broadband interface <NUM>.

The first source terminal <NUM> may be as a broadcasting source using terrestrial, satellite, mobile, or cable technologies. The second source terminal <NUM> may be a broadband source from the Internet.

A MPEG Transport Stream may packetize an Elementary Stream (ES) like video, audio or metadata into a Packetized Elementary Stream (PES). Multiple PESs may be multiplexed into a Transport Stream (TS) suitable for storage or transmission. In some embodiments, the first media stream <NUM> and the second media stream <NUM> may be MPEG Transport Streams, such as a video stream and an audio stream, respectively.

<FIG> is a diagram of the data flow for processing the input media streams in accordance with an embodiment of the invention.

Referring to <FIG> and <FIG>, the analyzing program <NUM> may be a transport layer in the OSI reference model, and may receive the first media stream <NUM> and the second media stream <NUM> respectively from the first source terminal <NUM> and the second source terminal <NUM>. The analyzing program <NUM> may parse the first media stream <NUM> to obtain a first PTS for each frame in the first media stream <NUM>, and then forward the first media stream <NUM> to the video driver <NUM> for subsequent processing. In addition, the analyzing program <NUM> may parse the second media stream <NUM> (i.e., audio stream) to obtain a second PTS (e.g., A_PTS) for each AU.

For example, a PTS carried in a broadcast transport stream refers to the presentation timestamp of a particular access unit of media (e.g., a frame of the video). Similarly, for broadband-delivered content, a timestamp associated with an access unit may refer to the time at which that access unit starts to be presented. A PTS timestamp is encoded in units of 90000ths of a second. MPEG TEMI (Timeline External Media Information) timestamps are encoded in units of ticks with a tick rate that is signaled.

The format of a TS packet in the second media stream <NUM> is shown in <FIG>. For example, an MPEG2 TS packet <NUM> may include a <NUM>-byte TS header <NUM> followed by <NUM> bytes shared between the variable-length adaptation field (AF) <NUM> and the TS packet payload <NUM>. In some embodiments, the AF <NUM> may include a TEMI descriptor <NUM> which describes external data and associated timing. In addition, the TS packet payload <NUM> may include a PES header <NUM> and PES payload <NUM>, and the PES header <NUM> may include a PTS field <NUM>, which is a <NUM>-bit size (resolution) encoded at a frequency of <NUM>. If the value of the PTS field if <NUM>, it may indicate a timeline at <NUM> seconds. It should be noted that the TEMI timeline descriptors are not always present on every PES packet that has PTS.

<FIG> is a diagram showing the mapping relationship between a video PTS and a video TEMI in accordance with an embodiment of the invention.

In the first media stream, there are pairs of V_TEMI (i.e., referring to video TEMI) and its corresponding V_PTS (i.e., referring to video PTS), such as {V_TEMI<NUM>, V_PTS<NUM>}, {V_TEMI<NUM>, V_PTS<NUM>}, {V_TEMI<NUM>, V_PTS<NUM>}, and so on. For example, if the frame rate of the first media stream <NUM> is <NUM> frames per second, there are <NUM> PTSs in one second.

As shown in <FIG>, the timeline z starts from V_PTS<NUM>, and the timeline x starts from V_TEMI<NUM>. When the V_PTSi of the currently displayed frame is not located at the same time of any PTS obtained from the first media stream <NUM>, if the pair of {V_TEMI<NUM>, V_PTS<NUM>} is closest to the V_PTSi of the currently displayed frame, the analyzing program <NUM> can convert the V_PTSi of the currently displayed frame into a corresponding V_TEMIi of the currently displayed frame using equation (<NUM>): <MAT>
where the parameter V timescale is a value obtained from the first media stream <NUM>. Accordingly, given that the parameters V_PTS<NUM>, V_PTSi, V_TEMI<NUM>, and V timescale are known, the analyzing program <NUM> can compute the value of V_TEMIi using equation (<NUM>).

<FIG> is a diagram showing the mapping relationship between a video TEMI and an audio TEMI in accordance with an embodiment of the invention.

During decoding of the first media stream <NUM> and the second media stream <NUM>, a time point of correlation (V_TEMI<NUM>, A_TEMI<NUM>), such as the initial time point, can be obtained. For example, the time point V_TEMI<NUM> refers to a known timeline value (e.g., the initial video timeline value) for the video TEMI, and the time point A_TEMI<NUM> refers to another known time value (e.g., the initial audio timeline value) for the audio TEMI, where V_TEMI<NUM> is equal to A_TEMI<NUM>.

As shown in <FIG>, when the video frame and audio frame need to align after n seconds, (V_TEMIi - V_TEMI<NUM>) ticks has passed on timeline x, and (A_TEMIi - A_TEMI<NUM>) ticks has passed on timeline y. Thus, the analyzing program <NUM> may extrapolate the value A_TEMIi on the timeline y using the linear relationship of equation (<NUM>): <MAT>
where the parameters V_timescale and A_timescale are known values obtained from the first media stream <NUM> and the second media stream <NUM>, respectively. Thus, the analyzing program <NUM> may perform timeline alignment on the audio TEMI (A_TEMI) with reference to the timeline x (i.e., video TEMI timeline).

<FIG> is a diagram showing the mapping relationship between an audio TEMI and an audio PTS in accordance with an embodiment of the invention.

As depicted in <FIG>, the timeline y starts from A_TEMI<NUM>, and the timeline w starts from A_PTS<NUM>. Since the analyzing program <NUM> may calculate the audio TEMI (A_TEMIi) from a video PTS (V_PTSi) of the currently displayed frame using equations (<NUM>) and (<NUM>), the analyzing program <NUM> may convert the audio TEMI to the corresponding audio PTS (C_APTSi) using equation (<NUM>): <MAT>
where the parameter A_timescale is a known values obtained from the second media stream <NUM>. Accordingly, given that the parameters A_TEMIi, A_TEMI<NUM>, and A_timescale are known, the analyzing program <NUM> can compute the value of A_PTSi using equation (<NUM>).

The next paragraphs [<NUM>] and [<NUM>] pertain to an embodiment outside the scope of the claimed invention.

Conversely, given that the value of A_PTSi is known, the analyzing program <NUM> can also compute the value of V_PTSi using the linear relationships in equations (<NUM>) to (<NUM>). For example, the analyzing program <NUM> may obtain parameters from HBBTV configurations, such as the starting audio TEMI (A_TEMI<NUM>), starting video TEMI (V_TEMI<NUM>), audio timescale (A_timescale), and video timescale (V_timescale), that are used to calculate a converted audio PTS (C_APTS) with reference to the video PTS timeline. The calculation of the converted audio PTS (C_APTS) is shown in equation (<NUM>): <MAT>
where the parameter A_PTS denotes the PTS obtained from PTS field <NUM> of the PES header <NUM> in the second media stream <NUM>. For example, given that A_TEMI<NUM>=<NUM>, V_TEMI<NUM>=<NUM>, A_timescale = <NUM>, V_timescale = <NUM>, and A_PTS = <NUM>, the analyzing program <NUM> may calculate that the value of C_APTS is <NUM>.

In response to the calculation of the converted audio PTS (C_APTS) being completed, the analyzing program <NUM> may modify the PES packet by replacing the second PTS in the PTS field <NUM> with the converted audio PTS (C_APTS), and then send the modified PES packet to the audio driver <NUM>. Thus, the audio driver <NUM> may perform audio/video synchronization using the converted audio PTS (C_APTS) which is aligned with reference to the video PTS timeline.

Specifically, the audio driver <NUM> may obtain the converted audio PTS (C_APTS) and the video PTS (V_PTS), and may treat the video stream as a master stream in the audio/video synchronization. For example, the audio driver <NUM> may perform audio/video synchronization using the following pseudo codes:
<IMG>.

For example, since a PTS is encoded in units of <NUM>/<NUM> of a second, a PTS value of <NUM> units is approximately equal to <NUM>. In addition, a latency of <NUM> is allowed in the HBBTV <NUM> specification, it indicates that the output audio signal can be behind or ahead of the video signal within an audio/video synchronization tolerance range of <NUM>. Specifically, if the converted audio PTS (C_APTS) is ahead of the video PTS (V_PTS) by more than <NUM> units (e.g., a first amount of units, which is an adjustable parameter) on the video PTS timeline (e.g., timeline z), it indicates that the converted audio PTS (C_APTS) is not within the audio/video synchronization tolerance range of the video PTS (V_PTS). Thus, the audio driver <NUM> may wait for a period of time (e.g., C_ATPS - V_TPS) until the actual audio PTS is within the audio/video synchronization tolerance range of the video PTS (V_PTS).

If the converted audio PTS (C_APTS) is behind the video PTS (V_PTS) by more than <NUM> units (e.g., a second amount of units, which is an adjustable parameter) on the video PTS timeline, it indicates that the converted audio PTS (C_APTS) is not within the audio/video synchronization tolerance range of the video PTS (V_PTS). Thus, the audio driver <NUM> may skip the current audio PES packet and fetch the next audio PES packet from the second media stream <NUM> for subsequent de-packetization, audio decoding and media synchronization.

If the converted audio PTS (C_APTS) is ahead of the video PTS (V_PTS) by less than or equal to the first amount of units on the video PTS timeline or is behind the video PTS (V_PTS) by less than or equal to the second amount of units on the video PTS timeline, it indicates that the converted audio PTS (C_APTS) is within the audio/video synchronization tolerance range of the video PTS (V_PTS). Thus, the audio driver <NUM> can normally play the audio content decoded from the current audio PES packet.

<FIG> is a flow chart of a method for synchronizing media streams in accordance with an embodiment of the invention. Please refer to <FIG> and <FIG>.

Step S510: obtaining a first media stream <NUM> and a second media stream <NUM> from a first source terminal <NUM> and a second source terminal <NUM>, respectively. For example, the first media stream <NUM> and the second media stream <NUM> are respectively a video stream and an audio stream that belong to MPEG (Motion Picture Expert Group) Transport Streams (TS). In addition, the first source terminal <NUM> is a broadcast source, and the second source terminal <NUM> is a broadband source.

Step S520: parsing a first packet in the first media stream to obtain a first presentation timestamp (PTS), and parsing a second packet, that corresponds to the first packet, in the second media stream to obtain a second PTS. For example, the processor <NUM> obtains the first PTS and the second PTS from a first PTS header of the first packet and a second PTS header of the second packet, respectively.

Step S530: converting the second PTS into a third PTS with reference to a first timeline on which the first PTS is located by the analyzing program <NUM>. For example, the processor <NUM> converts the second PTS on a second timeline into a first TEMI (Timed External Media Information) on a third timeline, converts the first TEMI into a second TEMI on a fourth timeline, and converts the second TEMI into the third PTS with reference to the first timeline on which the first PTS is located. More details about the aforementioned conversion operations can be referred to embodiments of <FIG>.

Step S540: modifying the second packet by replacing the second PTS with the third PTS in the second packet to generate a modified second packet by the analyzing program <NUM>. For example, the processor <NUM> forwards the first packet to a first driver (e.g., video driver <NUM>), and forwarding the modified second packet with the third PTS and the first PTS to a second driver (e.g., audio driver <NUM>). As a result, the timeline of the first PTS of the first packet processed by the video driver <NUM> and the timeline of the third PTS of the modified second packet processed by the audio driver <NUM> could align to each other.

Step S550: decoding the first packet and the modified second packet by the first driver and the second driver to generate first media content and second media content, respectively. It should be noted that the decoding latencies of the video driver <NUM> and the audio driver <NUM> may be different. Generally, the latency of decoding a video stream is generally higher than that of decoding an audio stream due to higher complexity of the video decoding.

Step S560: synchronizing the second media content with the first media content by the second driver using the third PTS and the first PTS. For example, in response to the second driver (e.g., audio driver <NUM>) determining that the third PTS is ahead of the first PTS by more than a first predetermined amount of units on the first timeline, the second driver waits for a period of time until an actual PTS of the second media content is within a tolerance range of the first PTS. In response to the second driver determining that the third PTS is behind the first PTS by more than a second predetermined amount of units on the first timeline, the second driver skips the modified second packet and fetches the next second packet from the second media stream. In response to the second driver determining that the third PTS is ahead of the first PTS by less than or equal to the first predetermined amount of units on the first timeline or is behind the first PTS by less than or equal to the second predetermined amount of units on the first timeline, the second driver outputs the second media content. In some embodiments, the first predetermined amount is equal to the second predetermined amount. In some other embodiments, the first predetermined amount is different from the second predetermined amount.

In view of the above, a multimedia playback apparatus and a method for synchronizing media streams, which are capable of performing audio/video synchronization using the audio/video driver. For example, the processor can convert the audio PTS from the audio PES packet to another audio PTS with reference to the video PTS timeline, and modify the audio PES packet by replacing the audio PTS with the converted audio PTS. In other words, the converted audio PTS and the video PTS are on the same video PTS timeline before processed by the audio/video driver. Thereafter, the audio driver can obtain the converted audio PTS from the modified audio PES packet, and perform audio/video synchronization using the converted audio PTS and the video PTS directly, without any seeking PTS step which involved the analyzing program <NUM>, thereby achieving higher synchronization accuracy with reduced delay time.

Embodiments described herein can take the form of an entirely hardware implementation, an entirely software implementation, or an implementation containing both hardware and software elements. Embodiments may be implemented in software, which includes, but is not limited to, application software, firmware, resident software, microcode, etc..

The steps described herein may be implemented using any suitable controller or processor, and software application, which may be stored on any suitable storage location or computer-readable medium. The software application provides instructions that enable the processor to perform the functions described herein.

Furthermore, embodiments may take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

Claim 1:
A multimedia playback apparatus (<NUM>), comprising:
a non-volatile memory (<NUM>), storing an analyzing program (<NUM>), a first driver (<NUM>), and a second driver (<NUM>); and
a processor (<NUM>), executing the analyzing program (<NUM>), the first driver (<NUM>), and the second driver (<NUM>) being adapted to perform the following steps:
obtaining Motion Picture Expert Group, MPEG, Transport Streams, TS, comprising a video stream (<NUM>) and an audio stream (<NUM>) from a first source terminal (<NUM>) and a second source terminal (<NUM>), respectively;
parsing a first packet in the video stream (<NUM>) to obtain a first presentation timestamp, PTS, referred to as video PTS, and parsing a second packet, that corresponds to the first packet, in the audio stream (<NUM>) to obtain a second PTS, referred to as audio PTS,
wherein the first source terminal (<NUM>) is a broadcast source, and the second source terminal (<NUM>) is a broadband source;
wherein the processor (<NUM>) obtains the video PTS and the audio PTS from a first PTS header of the first packet and a second PTS header of the second packet, respectively,
characterized by
the processor (<NUM>), executing the analyzing program (<NUM>), the first driver (<NUM>), and the second driver (<NUM>) being further adapted to perform the following steps:
converting the video PTS on the video PTS timeline (z) into a corresponding video Timed External Media Information, TEMI, on a video TEMI timeline (x) with reference to the video PTS timeline (z),
converting the video TEMI into an audio TEMI on an audio TEMI timeline (y), wherein converting the video TEMI into the audio TEMI comprises using additional video and audio TEMI time data and a predefined point of correlation on the video TEMI timeline (x) and the audio TEMI timeline (y), respectively; and
converting the audio TEMI into a corresponding audio PTS, C_APTS, on an audio PTS timeline (w);
and to further perform the following steps:
modifying the second packet by replacing the audio PTS with the converted PTS, C_APTS, in the second packet to generate a modified second packet;
decoding the first packet and the modified second packet by the first driver and the second driver to generate video content and audio content, respectively; and
synchronizing the audio content with the video content by the second driver using the converted PTS, C_APTS, and the video PTS.