Data processing apparatus and method

Video presentation timing can match sounding timing with high accuracy. A digital broadcast receiver calculates the presentation delay time dt from the presentation time when video based on the compressed video data is to be presented to the time when the video is presented on the presentation unit, and outputs audio based on the compressed audio data when the calculated presentation delay time dt passes from the sounding time when the audio based on the compressed audio data is to be outputted.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese Patent Application JP2004-333322 filed in the Japanese Patent Office on Nov. 17, 2004, the entire contents of which being incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a data processing apparatus and a method for use with the apparatus, and is preferably applied to a decoding device which outputs video and audio by decoding compressed video data and compressed audio data encoded according to, for example, the Moving Picture Experts Group Layer 2 (MPEG2) system, etc.

2. Description of the Related Art

Recently, a digital broadcast receiver has become widespread as a decoding device of the above-mentioned type. The digital broadcast receiver is configured to output video and audio according to a procedure RT1X as shown inFIG. 1when, for example, a user performs a channel switching operation.

That is, the digital broadcast receiver passes control to step SP1when a user performs a channel switching operation, and receives a Network Information Table (NIT). The NIT contains physical information about a transmission line (in this case, an orbit of a satellite, a polarized wave, a frequency for each transponder, etc.). Thus, the digital broadcast receiver can select a transponder corresponding to a request channel requested in the channel switching operation based on the NIT, and as a result receive a transport stream corresponding to the request channel.

Then, the digital broadcast receiver passes control to step SP2, and receives a Program Association Table (PAT). The digital broadcast receiver then passes control to step SP3, and receives a Program Map Table (PMT) corresponding to the request channel based on the PAT.

Then, the digital broadcast receiver passes control to step SP4, and realizes a Program Identification (PID) corresponding to the request channel based on such PMT. In this case, the PID corresponds to the identification information for identifying a packet containing the compressed video data, compressed audio data, Program Clock Reference (PCR) data, etc. corresponding to the request channel. The digital broadcast receiver can obtain the compressed video data, compressed audio data, PCR data, etc. corresponding to the request channel as a stream by separating a packet corresponding to the recognized PID from a received transport stream.

The processes in step SP1to SP4described above are well known as a DEMUX process (refer to Japanese Patent Application JP2003-274304).

The obtained PCR data in the stream includes the system time clock at an encoding device which encodes the compressed video data and the compressed audio data.

Therefore, when the digital broadcast receiver passes control to the next step SP5, it sets the System Time Clock (STC) kept in the digital broadcast receiver to the time clock kept in the PCR data. Thus, the system time clock kept by the digital broadcast receiver can be synchronized with the time clock kept at the encoding device.

Also in the stream, as shown inFIG. 2, the Presentation Time Stamp (PTS) indicating the time when the video is to be presented is associated with each piece of video data “Dpic (0)”, “Dpic (1)”, . . . corresponding to one frame for example. Likewise, in the stream, the sounding time stamp (PTS) indicating the time when the audio is to be outputted is associated with each piece of compressed audio data “Dframe (0)”, “Dframe (1)”, . . . corresponding to one frame for example.

Thus, the digital broadcast receiver passes control to step SP6, for example, when the kept system time clock matches the presentation time stamp “pts (0)” associated with the compressed video data “Dpic (0)”, and presents on the presentation device the video “Vpic (0)” based on the video data obtained by decoding the compressed video data “Dpic (0)”.

In addition, the digital broadcast receiver passes control to step SP7when the kept system time clock matches the sounding time stamp “pts (0)” associated with the compressed audio data “Dframe (0)”, and outputs the audio “Sframe (0)” based on the audio data obtained by decoding the compressed audio data “Dframe (0)” through a speaker, etc.

Thus, the digital broadcast receiver can allow to some extent the video presentation timing indicating the video “Vpic (0)”, . . . to match the sounding time outputting the audio “Sframe (0)”, . . . corresponding to the video “Vpic (0)”.

SUMMARY OF THE INVENTION

The above-mentioned digital broadcast receiver practically generates a vertical synchronizing signal for synchronization in the operation of a scanning line in the vertical direction for use in presenting video on the presentation device, and sequentially presents the video “Vpic (0)”, . . . in synchronization with the generated signal.

Therefore, in the case as shown inFIG. 2, for example, relating to the audio “Sframe (0)”, although the audio “Sframe (0)” can be output at the sounding time “pts (0)” when the audio is to be outputted as scheduled, the video “Vpic (0)” corresponding to the audio “Sframe (0)” is not assigned the vertical synchronizing signal generated at the presentation time “pts (0)” when the video “Vpic (0)” based on the video data is to be presented. Therefore, the video “Vpic (0)” is not allowed to be presented, and the video “Vpic (0)” can be presented when a vertical synchronizing signal is subsequently generated.

As a result, in the digital broadcast receiver, a difference of a maximum of ±½ frame is detected between the sounding timing of the audio “Sframe (0)”, . . . and the video presentation timing of the video “Vpic (0)”, . . . corresponding to the audio “Sframe (0)”, . . . . Therefore, it is hard to recognize the video presentation timing as matching the sounding timing with high accuracy.

A method for solving the above-mentioned problem can be adjusting the vertical synchronizing signal generated in the digital broadcast receiver to match the presentation time stamp before presenting the video “Vpic (0)”, . . . . However, when the adjustment is made, the vertical synchronizing signal causes a disturbance, thereby causing the video “Vpic (0)”, . . . presented a disturbance. As a result, the method is not preferable.

The present invention has been developed to solve the above-mentioned problems, and provides a data processing apparatus and method capable of making video presentation timing matching sounding timing with high accuracy.

To solve the problem, a data processing apparatus according to the present invention includes: a presentation device for presenting video based on video data when a video presentation timing signal is provided from a providing device for providing a video presentation timing signal indicating presentation timing for video at a predetermined cycle time after a presentation time when video based on the video data is to be presented; a calculation device for calculating a presentation delay time from the presentation time when video based on the video data is to be presented to a time when the presentation device presents the video; and a sounding device for outputting audio based on audio data when the calculated presentation delay time passes from a sounding time when audio based on the audio data is to be outputted.

Thus, with the data processing apparatus, the presentation delay time from the presentation time when video based on the video data is to be presented to the presentation of the video by the presentation device is calculated. When the calculated presentation delay time passes from the sounding time when audio based on the audio data is to be outputted, the audio based on the audio data is output, thereby delaying the output of the audio by the delay the presentation of the video, and successfully avoiding the shift between the video presentation timing to present the video and the sounding timing to output the audio.

According to the present invention, the presentation delay time from the presentation time when video based on the video data is to be presented to the presentation of the video by the presentation device is calculated. When the calculated presentation delay time passes from the sounding time when audio based on the audio data is to be outputted, the audio based on the audio data is output, thereby delaying the output of the audio by the delay the presentation of the video, and successfully avoiding the shift between the video presentation timing to present the video and the sounding timing to output the audio. Accordingly, the data processing apparatus capable of matching the video presentation timing with the sounding timing with high accuracy can be realized.

DETAILED DESCRIPTION OF THE EMBODIMENT

An embodiment of the present invention is described below by referring to the attached drawings.

(1) Configuration Of Digital Broadcast Receiver

In a digital broadcast receiver1shown inFIG. 3, a NIT acquisition unit3in a DEMUX processing unit2receives an input stream from a television broadcast wave received through an antenna, etc.

The NIT acquisition unit3acquires a NIT (Network Information Table) from the received input stream. Thus, the digital broadcast receiver1can select a transponder corresponding to the request channel requested in the channel switching operation of a user according to the acquired NIT, and accordingly receives the transport stream corresponding to the request channel.

Then, a PAT acquisition unit4provided in the DEMUX processing unit2acquires a program association table (PAT). Next, a PMT acquisition unit5provided in the DEMUX processing unit2acquires a program map table (PMT) corresponding to the request channel requested in the channel switching operation of the user according to the PAT acquired by the PAT acquisition unit4.

The PMT acquisition unit5recognizes a program identification (PID) corresponding to the request channel based on the acquired PMT, and sets the recognized PID in each of a PCR filter6A, a video filter6B, and an audio filter6C in a filter unit6.

Thus, the PCR filter6A, the video filter6B, and the audio filter6C can respectively extract a PCR packet, a video packet, and an audio packet respectively containing the PCR data, the compressed video data, and the compressed audio data corresponding to the request channel from the received transport stream according to the respectively set PID.

A source oscillation clock generation unit7performs negative feedback control to make a signal output from an STC counter unit10be phase-synchronous with the PCR data in the PCR packet extracted from the PCR filter6A as a reference signal. Thus, the source oscillation clock generation unit7can generate an operation clock signal S1in synchronization with a reference clock signal at the encoding device, and provides the obtained operation clock signal S1for a horizontal/vertical synchronizing signal generation circuit8, a synchronizing signal generation circuit9, and the STC counter unit10.

The horizontal/vertical synchronizing signal generation circuit8generates a vertical synchronizing signal S2for synchronization of the operation of a scanning line in the vertical direction and a horizontal synchronizing signal S3for synchronization of the operation of a scanning line in the horizontal direction based on the operation clock signal S1provided from the source oscillation clock generation unit7. The horizontal/vertical synchronizing signal generation circuit8supplies the generated vertical synchronizing signal S2for a latch circuit11. The horizontal/vertical synchronizing signal generation circuit8provides the operation clock signal S1provided from the source oscillation clock generation unit7, and the generated vertical synchronizing signal S2and horizontal synchronizing signal S3for a presentation process unit12.

Thus, the presentation process unit12provides a digital video signal S4provided from a video decoder unit13for a presentation unit14at the subsequent stage according to the provided operation clock signal S1, vertical synchronizing signal S2, and horizontal synchronizing signal S3. As a result, the video based on the digital video signal S4is presented on the presentation unit14.

The synchronizing signal generation circuit9generates a synchronizing signal S5for sounding according to the operation clock signal S1provided from the source oscillation clock generation unit7. The synchronizing signal generation circuit9provides the generated synchronizing signal S5and the operation clock signal S1provided from the source oscillation clock generation unit7for a sounding process unit15. Thus, the sounding process unit15provides a digital audio signal S6received from an audio decoder unit16for a sounding unit17at the subsequent stage according to the provided operation clock signal S1and synchronizing signal S5. As a result, the audio according to the digital audio signal S6is output through the sounding unit17.

On the other hand, the STC counter unit10for keeping the system time clock (STC) in the digital broadcast receiver1according to the operation clock signal S1from the source oscillation clock generation unit7sets the system time clock kept by the unit to the time clock indicated by the PCR data in the provided PCR packet when the PCR packet is provided from the PCR filter6A. Thus, the system time clock kept in the digital broadcast receiver1can be synchronous with the time clock at the encoding device. Then, the STC counter unit10is configured to provide a system time clock stamp I1indicating the kept system time clock for a latch circuit11and a first comparison circuit18.

A presentation time stamp extraction unit19extracts a presentation time stamp (PTS)12associated with the compressed video data for each frame from the video packet extracted by the video filter6B, and provides it for the first comparison circuit18and an arithmetic circuit20.

The first comparison circuit18compares the system time clock stamp I1sequentially provided from the STC counter unit10with the presentation time stamp I2provided from the presentation time stamp extraction unit19. For example, when the system time clock indicated by the system time clock stamp I1matches the presentation time “pts (0)” indicated in the presentation time stamp I2, the first comparison circuit18provides a decoding process start directive signal S7for starting the decoding process for the video decoder unit13.

When the decoding process start directive signal S7is provided from the first comparison circuit18, the video decoder unit13generates the digital video signal S4by performing the decoding process on one corresponding frame of compressed video data “Dpic (0)” obtained from the video packet extracted by the video filter6B, and provides the signal for the presentation process unit12.

However, in this case, since the presentation process unit12has not received the vertical synchronizing signal S2from the horizontal/vertical synchronizing signal generation circuit8at the system time clock “pts (0)” shown inFIG. 4, the presentation process unit12provides one frame of the digital video signal S4received from the video decoder unit13for the presentation unit14at the time “vt(n+1)” when the S2is provided. Thus, the timing indicating the video “Vpic (0)” based on the digital video signal S4is delayed by a predetermined time dt after the presentation time “pts (0)” when the video “Vpic (0)” is to be presented (hereinafter the predetermined time dt is referred to as a presentation delay time).

The latch circuit11according to the present embodiment latches the system time clock stamp I1indicating the system time clock “vt (n−1)” received from the STC counter unit10with, for example, the timing T1when the vertical synchronizing signal S2is provided in advance from the horizontal/vertical synchronizing signal generation circuit8. Thus, the latch circuit11provides the arithmetic circuit20with the system time clock stamp I1indicating the system time clock “vt(n−1)”.

The arithmetic circuit20calculates the presentation delay time dt by performing the calculation by the following equation (1) when the presentation time stamp extraction unit19provides the presentation time stamp I2indicating the presentation time “pts (0)”.
dt=Vt−{[pts(0)−vt(n−1)]mod Vt}(1)
The equation (1) above is practically explained as follows. That is, after subtracting the time “vt(n−1)” when the vertical synchronizing signal S2is provided before from the presentation time “pts (0)” when the video “Vpic (0)” based on the compressed video data “Dpic (0)” is to be presented, the subtraction result is divided by the cycle time Vt of the vertical synchronizing signal S2. Then, the resultant remainder is subtracted from the cycle time Vt of the vertical synchronizing signal S2. Thus, the presentation delay time dt can be calculated.

The arithmetic circuit20provides an adder21with the presentation delay time stamp I3indicating the obtained presentation delay time dt.

On the other hand, a sounding time stamp extraction unit22extracts a sounding time stamp (PTS) I4associated with each frame of compressed audio data from the audio packet extracted by the audio filter6C, and provides it for the adder21.

Thus, the adder21adds the presentation delay time dt indicated by a presentation delay time stamp I3provided from the arithmetic circuit20to, for example, the sounding time stamp “pts (0)” indicated by the sounding time stamp I4provided from the sounding time stamp extraction unit22, and provides an amended sounding time stamp I5indicating the addition result “vt(n+1)” for a second comparison circuit23.

The second comparison circuit23compares the system time clock stamp I1sequentially provided from the STC counter unit10with the amended sounding time stamp I5provided from the adder21. Then, the second comparison circuit23provides a decoding process start directive signal S8for starting the decoding process for the audio decoder unit16when the system time clock indicated in the system time clock stamp I1matches the sounding time “vt(n+1)” indicated in the amended sounding time stamp I5.

When the decoding process start directive signal S8is described from the second comparison circuit23, the audio decoder unit16generates the digital audio signal S6by performing the decoding process on a corresponding frame of compressed audio data “Dframe (0)” correspondingly obtained from an audio packet extracted by the audio filter6C, and provides the signal for the sounding unit17through the sounding process unit15. Thus, the audio “Sframe (0)” based on the compressed audio data “Dframe (0)” is output to the time “vt(n+1)” delayed by the presentation delay time dt after the time “pts (0)”.

Thus, the digital broadcast receiver1can allow the video presentation timing with which the video “Vpic (0)” is presented to match with high accuracy the sounding timing with which the audio “Sframe (0)” corresponding to the video “Vpic (0)” is output.

Afterwards, the digital broadcast receiver1likewise continues adding the presentation delay time dt indicated by the presentation delay time stamp I3provided from the arithmetic circuit20to the sounding time indicated by a sounding time stamp I4provided from the sounding time stamp extraction unit22. Thus, afterwards, the video presentation timing with which the video “Vpic (1)”, . . . is presented can match with high accuracy the sounding timing with which the corresponding audio “Sframe (1)”, . . . is output.

The video/audio output procedure RT1according to an embodiment of the present invention is described below by referring toFIG. 5with the corresponding component shown inFIG. 1assigned the same reference numeral.

As described above, the digital broadcast receiver1receives a NIT in step SP1, receives a PAT in step SP2, and receives a PMT in step SP3. Then, the digital broadcast receiver1separates a PCR packet, a video packet, and an audio packet from a transport stream using the PID recognized based on the PMT in step SP4. Then, the digital broadcast receiver1sets the system time clock kept by the STC counter unit10to the time clock indicated by the PCR data in the PCR packet in step SP5.

Then, in step SP11, the digital broadcast receiver1latches in the latch circuit11the system time clock stamp I1(“vt(n−1)”) provided from the STC counter unit10with, for example, the timing T1(FIG. 4) of providing the vertical synchronizing signal S2.

Then, in step SP12, the digital broadcast receiver1, determines whether or not the presentation time stamp I2(“pts (0)”) of the compressed video data “Dpic (0)” to be first decoded has been extracted by the presentation time stamp extraction unit19within a predetermined time.

If an affirmative result is obtained in step SP12, it means that the presentation time stamp I2(“pts (0)”) corresponding to the compressed video data “Dpic (0)” has been extracted. At this time, the digital broadcast receiver1passes control to the next step SP13.

In step SP13, the digital broadcast receiver1instructs the arithmetic circuit20to perform calculation by the equation (1) above according to the extracted presentation time stamp I2(“pts (0)”) and the system time clock stamp I1(“vt (n−1)”) latched by the latch circuit11, thereby calculating the presentation delay time dt.

Thus, the digital broadcast receiver1can calculate in advance the presentation delay time dt of the video “Vpic (0)” to be presented in the next step SP15. Thus, the digital broadcast receiver1can delay the audio “Sframe (0)” corresponding to the video “Vpic (0)” by the presentation delay time dt before output in step SP16.

If a negative result is obtained in step SP12, it means that the presentation time stamp I2(“pts (0)”) has not been extracted because the presentation time stamp I2(“pts (0)”) corresponding to the compressed video data “Dpic (0)” has been lost, etc. At this time, the digital broadcast receiver1passes control to step SP14, not to step SP13in which the presentation delay time dt is calculated. In step SP14, the digital broadcast receiver1sets the value of the presentation delay time dt to, for example, “0”, and then control is passed to steps SP15and SP16.

(3) Operation and Effect

With the above-mentioned configuration, the digital broadcast receiver1is configured to present on the presentation unit14the video “Vpic (0)” based on the compressed video data “Dpic (0)” when the horizontal/vertical synchronizing signal generation circuit8for providing the vertical synchronizing signal S2on a predetermined cycle time Vt provides the vertical synchronizing signal S2(time “vt(n+1)”) after the presentation time “pts (0)” when the video “Vpic (0)” based on the compressed video data “Dpic (0)” is to be presented as shown inFIG. 4.

In the present embodiment, the digital broadcast receiver1calculates in advance the presentation delay time dt from the presentation time “pts (0)” when the video “Vpic (0)” based on the compressed video data “Dpic (0)” is to be presented to the time when the video “Vpic (0)” is practically presented on the presentation unit14. The digital broadcast receiver1outputs the audio “Sframe (0)” based on the compressed audio data “Dframe (0)” when the calculated presentation delay time dt passes from the sounding time “pts (0)” when the audio “Sframe (0)” based on the compressed audio data “Dframe (0)” is to be outputted.

Thus, the digital broadcast receiver1can delay the output of the audio “Sframe (0)” by the delay of the presentation of the video “Vpic (0)”, thereby successfully avoiding the shift between the video presentation timing with which the video “Vpic (0)” is presented and the sounding timing with which the audio “Sframe (0)” is output.

With the above-mentioned configuration, the presentation delay time dt from the presentation time when video based on the compressed video data is to be presented to the time when the video is presented on the presentation unit14is calculated, and the audio based on the compressed audio data is output when the calculated presentation delay time dt passes from the sounding time when the audio based on the compressed audio data is to be outputted, thereby possibly delaying the output of the audio by the delay of the presentation of the video. As a result, the shift between the video presentation timing with which the video is presented and the sounding timing with which the audio is output can be avoided. Accordingly, the digital broadcast receiver1capable of matching the video presentation timing with the sounding timing with high accuracy can be realized.

(4) Other Embodiments

In the above-mentioned embodiment, the latch circuit11is allowed to latch the system time clock stamp I1(“vt(n−1)”) with the timing T1shown inFIG. 4. However, the present invention is not limited to this application, and the latch circuit11can latch the system time clock stamp I1with another timing of providing the vertical synchronizing signal S2from the horizontal/vertical synchronizing signal generation circuit8after the system time clock kept by the STC counter unit10is synchronized with the time of the encoding device and before the video based on the compressed video data is presented. Also in this case, the presentation delay time dt can be calculated by the equation (1) described above.

In the embodiment described above, the presentation process unit12and the presentation unit14are applied as presentation devices for presenting the video (“Vpic (0)”, . . . ) based on the video data (“Dpic (0)”, . . . ) when the providing device (8) for providing the video presentation timing signal (S2) indicating the timing with which video is presented at a predetermined cycle time (Vt) provides the video presentation timing signal (S2) after the presentation time (“pts (0)”, . . . ) when the video (“Vpic (0)”, . . . ) based on the video data (“Dpic (0)”, . . . ) is to be presented. However, the present invention is not limited to this application, but various other configurations can also be applied.

Furthermore, in the above-mentioned embodiment, the arithmetic circuit20is applied as a calculation device for calculating the presentation delay time dt from the presentation time (“pts (0)”, . . . ) when the video (“Vpic (0)”, . . . ) based on the video data (“Dpic (0)”, . . . ) is to be presented to the time when the video (“Vpic (0)”, . . . ) is presented on the presentation device. However, the present invention is not limited to this application, but various other configurations can be applied so far as the calculation by the equation (1) described above is used.

Additionally, in the above-mentioned embodiment, the sounding process unit15and the sounding unit17are applied as sounding devices for outputting the audio (“Sframe (0)”, . . . ) based on the audio data (“Dframe (0)”, . . . ) when the calculated presentation delay time dt passes from the sounding time (“pts (0)”, . . . ) when the audio (“Sframe (0)”, . . . ) based on the audio data (“Dframe (0)”, . . . ) is to be outputted. However, the present invention is not limited to the above-mentioned application, but various other configurations can be applied.

Furthermore, the above-mentioned embodiment performs as if it were hardware the process according to the present invention such as the video/audio output procedure RT1using each circuit as shown inFIG. 3. However, the present invention is not limited to this application, but for example, a control unit such as the central processing unit CPU (CPU) according to the program stored in the storage unit such as a hard disk drive, etc. can perform as if it were software the process according to the present invention such as the video/audio output procedure RT1.

The present invention can be applied to a decoding device configured to output video and audio by decoding the compressed video data and compressed audio data encoded by the MPEG2 system.