Video/audio playback apparatus and video/audio playback method

Provided is a video/audio playback apparatus and a video/audio playback method in which on receiving an instruction for switching video operation from the video operation switching circuit, a data reading circuit reads a video/audio stream for playing an audio and a video/audio stream for playing a video from a recording medium, and an audio playback circuit reproduces an audio stream separated from the video/audio stream for playing the audio on the basis of time management information embedded in the audio stream and a predetermined reference time, and a video playback circuit reproduces a video stream separated from the video/audio stream for playing the video on the basis of a reference time different from the reference time, and thereby the video can freely played with the audio being normally played.

FIELD OF THE INVENTION

The present invention relates to an apparatus and a method for reading and reproducing a coded video/audio stream from a random accessible recording medium and, more particularly, to a video/audio synchronous playback technique.

BACKGROUND OF THE INVENTION

In recent years, DVDs in which video and audio are digital-compressed and recorded have become widespread, and are widely used for easily playing various kinds of software such as movie and karaoke. A VTR which has been conventionally utilized played back such as a movie in the recorded order at the most as a function of playing a video. For a DVD, however, a function of playing a desired scene in a moment and the like can be provided. Further, while a program being broadcast can be simply viewed on TV in a prior art, the emergence of digital broadcasting enables viewers to answer quizzes at home by using a remote controller for a television when they are viewing a quiz show, for example. Moreover, when a drama is being viewed, profiles of actors on television can be displayed by merely pressing a button.

For such as the DVD and digital broadcasting, the MPEG (Moving Picture Experts Group) is used as an information compression technique for efficiently storing and transmitting video and audio. In MPEG, a construction of a signal used for multiplexing a plurality of digital signals which are constituents of digital contents such as video and audio into a bitstream is defined as a standard.

In this standard, a PCR (Program Clock Reference) or an SCR (System Clock Reference) which is time information for setting a value of an STC (System Time Clock) as a reference time on the decoder side in the MPEG system, to a value as intended on the encoder side, a PTS (Presentation Time Stamp) which is time information added for each unit of coding video or audio as well as time management information for playback and output, and a DTS (Decoding Time Stamp) which is time management information for decoding are coded into a bitstream, thereby performing synchronous playback with higher precision.

Here, a format of a video/audio stream for MPEG2 will be described with reference toFIG. 15.FIG. 15is a diagram schematically illustrating a construction of the video/audio stream. InFIG. 15, a video/audio stream is constructed as a set of a video pack (V_PCK) and an audio pack (A_PCK). The V_PCK comprises a pack header and a video packet, and an SCR and the like are recorded in the pack header. The video packet further comprises a packet header and a video elementary stream. A stream ID, a DTS, a PTS and the like are recorded in the packet header and the video elementary stream comprises an I picture, a P picture, and a B picture which are bitstreams obtained by compressing digital video signals according to MPEG2. Then, the A_PCK and the V_PCK have the similar construction and there are some cases where the audio elementary stream contains a sub-stream ID according to the audio compression method.

Further, according to a timing rule for realizing synchronous playback using time stamp, or a rule for memory amount defined for sequentially reproducing coded data, the system target decoder (STD) as shown inFIG. 16is modeled.

FIG. 16is a block diagram illustrating an STD according to the MPEG2-PS (Program Stream). An access unit of an audio elementary stream is defined as being instantaneously transmitted from an input buffer to a decoder at a time corresponding to tpn to be decoded, and this tpn is coded into a bitstream as a PTS. On the other hand, for the video elementary stream, the I picture (Intra Picture) and the P picture (Predictive Picture) are decoded prior to B picture (Bidirectionally Predictive Picture), and therefore a decode time does not coincide with a display time. Therefore, data of the I picture and the P picture are defined as being instantaneously transmitted from an input buffer to a decoder at a time corresponding to td1 to be decoded, and the td1 is coded into a bitstream as a DTS, and these data are displayed at a time indicated as the PTS.

Many of the conventional techniques relating to the MPEG is directed to synchronously reproduce the respective stream data into which signals are multiplexed as MPEG streams with higher precision, and as a technique which has an object other than obtaining synchronization, for example, like a playback apparatus disclosed in the Japanese Published Patent Application No. Hei. 8-298645, which discloses a playback apparatus which plays sub-videos a predetermined time period later after it simultaneously plays main videos and audios, and completes the playback of the sub-videos and then simultaneously plays main videos and audios again.

However, while DVD enables various playback functions to be handled, how the contents can be reproduced just depends on the intention of the content producer, and even for the digital broadcasting, the broadcast station or program producer must add additional information to the content data so that the viewers can utilize various services.

For example, it is assumed that a state of a lecture meeting in which materials are projected onto a screen using a liquid crystal projector or an OHP for presentation is photographed and recorded using a video recorder. The viewer who plays the recorded video sometimes wants to view the video which has been played before, such as the materials of the presentation or the posture of the presenter giving the lecture, instead of the video which is played in synchronization with audio, while normally hearing audio of such as the presenter's lecture. Further, in a case where a parent other than a photographer views the video which is obtained by photographing and recording his child's music program held in a kindergarten or a nursery school, the parent sometimes wants to selectively view the video of a portion which is played with his child being centered while normally hearing the audio such as music. Further, also when viewing a live sport such as baseball or soccer on TV, the viewer may sometimes want to view the video of a highlight scene which has been broadcast before again while hearing audio live as it is. When viewing a live news program and the like, the same thing can be said. In either case, since the portions which the respective viewers want to view are various, the viewers sometimes want to carefully view the video other than the video photographed by a photographer or the video provided by the broadcast station with the audio being played as it is, the viewers feel frustrated.

While a novel playback function can be provided to viewers using the data reproduction method disclosed in the above-described patent document, the sub-video is played a predetermined time period later than the main video in this playback function, and the viewers cannot freely make operation for playing the video without feeling stressful.

SUMMARY OF THE INVENTION

The present invention is made to solve the above-described problems and has for its object to provide a video/audio playback apparatus and a video/audio playback method capable of freely changing playback operations for only the video with the audio being normally played.

Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the spirit and scope of the invention will be apparent to those of skill in the art from the detailed description.

According to a 1st aspect of the present invention, there is provided a video/audio playback apparatus which reads a video/audio stream recorded in a random accessible recording medium, and reproduces the video/audio stream on the basis of a reference time and time management information which is embedded in each of a video stream and an audio stream which constitute the video/audio stream, and which comprises: a video operation switching circuit for indicating that a playback operation of the video stream is to be switched; a data reading circuit for reading a video/audio stream for playing an audio and a video/audio stream for playing a video from the recording medium on receiving the instruction for switching the video operation from the video operation switching circuit; a stream buffer for distinguishably storing the video/audio stream for playing audio and the video/audio stream for playing video which are read by the data reading circuit; a separation circuit for separating an audio stream and a video stream from the video/audio stream for playing audio and the video/audio stream for playing video, which are stored in the stream buffer, respectively; a counter for outputting a predetermined clock as a reference time on the basis of the time management information embedded in the video/audio stream for playing audio; an audio playback circuit for reproducing the audio stream separated from the video/audio stream for playing audio on the basis of the time management information embedded in the audio stream and the reference time; and a video playback circuit for reproducing the video stream separated from the video/audio stream for playing video on the basis of a reference time different from that for the audio playback circuit.

Therefore, the video/audio playback apparatus according to the present invention reproduces a video/audio stream for playing audio and a video/audio stream containing a picture frame required for the trick play individually, and reproduces the audio stream on the basis of the time management information embedded in the audio stream and a reference time which is set based on the time management information embedded in the audio stream, thereby enabling the trick play of only the video being performed with the audio being normally played, and improving the operability and convenience for the user.

According to a 2nd aspect of the present invention, in the video/audio playback apparatus of the 1st aspect, the video/audio stream is a stream into which the digital-compressed video and audio are multiplexed according to the MPEG format.

According to a 3rd aspect of the present invention, in the video/audio playback apparatus of the 1st aspect, the video operation switching circuit gives instructions for switching among normal playback for synchronously reproducing an audio stream and a video stream having the same time management information, video trick play for performing trick play of only a video stream, and video asynchronous playback for asynchronously reproducing an audio stream and a video stream each having time management information.

According to a 4th aspect of the present invention, in the video/audio playback apparatus of the 3rd aspect, the video playback circuit performs trick play of the video stream by using the video stream separated from the video/audio stream for playing video on receiving the instruction for the video trick play operation from the video operation switching circuit.

According to a 5th aspect of the present invention, in the video/audio playback apparatus of the 4th aspect, the data reading circuit reads video/audio streams for playing audios with higher priority than video/audio streams for playing videos, and continuously transmits the audio streams to the audio playback circuit so as not to interrupt audios which are to be played by the audio playback circuit when receiving the instruction for the video trick play operation from the video operation switching circuit.

According to a 6th aspect of the present invention, in the video/audio playback apparatus of the 5th aspect, the video playback circuit reproduces the video stream separated from the video/audio stream for playing video using a value obtained by subtracting, from the reference time, a differential value between the time management information embedded in the video stream which was outputted most recently at the video trick play and the reference time as of the time at which the video trick play is stopped, as a reference time for reading the video stream, when receiving the instruction for performing the video asynchronous playback halfway through the video trick play from the video operation switching circuit.

Therefore, in a case where a video asynchronous playback in which the audio stream and the video stream having the same time management information are asynchronously read is indicated in a state where the trick play of only the video stream is being performed, the video/audio playback apparatus according to the present invention calculates a differential value between the reference time as of the time at which the video asynchronous playback is indicated and the time management information embedded in the video stream which is being subjected to the trick play and displayed, and sets a value obtained by subtracting the differential value from the reference time as a reference time used for reading the video stream, and thereby can simultaneously reproduce the audio stream and the video stream having the different time management information from each other, and therefore a state where the trick play of only the video is being performed can be shifted to playing the audio and the video asynchronously by a desired time, thereby improving the operability and the convenience of the user.

According to a 7th aspect of the present invention, in the video/audio playback apparatus of the 6th aspect, the video playback circuit reproduces the video stream separated from the video/audio stream for playing audio on the basis of the time management information embedded in the video stream and the reference time when receiving the instruction for performing the normal playback operation halfway through the video asynchronous playback operation from the video operation switching circuit.

According to an 8th aspect of the present invention, in the video/audio playback apparatus of the 4th aspect, the video playback circuit reproduces the video stream of the video/audio stream for playing audio on the basis of the time management information embedded in the video stream and the reference time when the time management information embedded in the video stream which is being subjected to video trick play coincides with the reference time.

Therefore, in a case where the time management information embedded in the video stream which is being subjected to video trick play and displayed is identical to the time management information embedded in the audio stream being read in a state where the trick play of only the video stream is being performed, the video/audio playback apparatus according to the present invention reads the video stream separated from the video/audio stream for playing the audio on the basis of the reference time, and therefore the audio stream and the video stream having the same time management information can be synchronously read without performing a special operation, thereby improving the operability and convenience for the user.

According to a 9th aspect of the present invention, the video/audio playback apparatus of the 7th aspect or the 8th aspect comprises; a playback time display circuit for visually displaying one of a playback time of the audio which is played by the audio playback circuit and a playback time of the video which is played by the video playback circuit, or both of them.

According to a 10th aspect of the present invention, there is provided a video/audio playback method for reading a video/audio stream recorded in a random accessible recording medium, and reproducing the video/audio stream on the basis of time management information embedded in each of a video stream and an audio stream which constitute the video/audio stream and a predetermined reference time, and which comprises: reading a video/audio stream for playing an audio and a video/audio stream for playing a video from the recording medium; extracting a predetermined clock as a reference time on the basis of a predetermined time management information embedded in the video/audio stream for playing audio; and reproducing an audio stream of the video/audio stream for playing audio on the basis of the time management information embedded in the audio stream and the reference time, and performing trick play of a video stream using the video/audio stream for playing video.

Therefore, the video/audio playback apparatus according to the present invention reproduces a video/audio stream for playing audio and a video/audio stream containing a picture frame required for the trick play individually, and reproduces the audio stream on the basis of the time management information embedded in the audio stream and a reference time which is set based on the time management information embedded in the audio stream, thereby enabling the trick play of only the video being performed with the audio being normally played, and improving the operability and convenience for the user.

According to an 11th aspect of the present invention, the video/audio playback method of the 10th aspect comprises: stopping trick play of the video stream; obtaining a differential value between the time management information embedded in the video stream which was outputted most recently at the trick play of the video stream and the reference time as of the time at which the video trick play is stopped and subtracting the differential value from the reference time; and reproducing the video stream separated from the video/audio stream for playing video with using the subtraction value as a reference time for reproducing video stream

According to a 12th aspect of the present invention, the video/audio playback method of the 10th aspect comprises reproducing the video stream of the video/audio stream for playing audio on the basis of the time management information embedded in the video stream and the reference time when the time management information embedded in the video stream displayed at the trick play of the video stream coincides with the reference time.

According to a 13th aspect of the present invention, the video/audio playback apparatus of the 1st aspect comprises a display video control circuit for displaying a display video which was outputted immediately before an instruction for switching a video operation, on a sub-screen which is other than a main screen onto which the video stream is reproduced, when receiving the instruction for switching the video operation from the video operation switching circuit.

According to a 14th aspect of the present invention, in the video/audio playback-apparatus of the 13th aspect, the display video control circuit displays, when receiving an instruction for switching a video operation from the video operation switching circuit in a state where the videos are displayed on the main screen and the sub-screen, the display video which was outputted onto the main screen immediately before the instruction for switching the video operation is received, on a new sub-screen.

According to a 15th aspect of the present invention, the video/audio playback apparatus of the 14th aspect further comprises a video operation selection circuit for selecting one of the sub-screens, wherein the display video control circuit switches between a video display on the sub-screen selected by the video operation selection circuit and the video display on the main screen and displays the respective videos when receiving an instruction for switching a video operation from the video operation switching circuit, and the video playback circuit reproduces a video stream so as to start playback from the switched display video on the main screen.

Therefore, the video/audio playback apparatus according to the present invention displays a display video being played on a main screen as a paused video on another screen, and starts playback operation from the paused video being displayed on the other screen according to a user's selection, and thereby the video which the user desires to carefully view can be freely displayed and played with the audio being normally played, thereby improving the operability and the convenience for the user.

According to a 16th aspect of the present invention, the video/audio playback method of the 10th aspect comprises performing the trick play of the video stream with the display video from which the trick play is started being displayed on another screen when performing the trick play of the video stream.

Therefore, the video/audio playback apparatus according to the present invention displays a display video being played on a main screen as a paused video on another screen, and starts playback operation from the paused video being displayed on the other screen according to a user's selection, and thereby the video which the user desires to carefully view can be freely displayed and played with the audio being normally played, thereby improving the operability and the convenience for the user.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1is a block diagram illustrating a video/audio playback apparatus according to the embodiment of the present invention.

InFIG. 1, a video operation switching circuit101selects an operation of the video/audio playback apparatus and notifies a system controller102of operation codes corresponding to the respective operations.

The system controller102controls reproduction of a video/audio stream according to a predetermined operation code notified by the video operation switching circuit101, and indicates various control codes to the respective circuits described below and controls states of the other respective circuits.

A recording medium103is a random accessible recording medium for storing video/audio streams, such as a CD, a DVD, an HDD or an SD card, in which the consecutive addresses are allocated to data, thereby enabling the storage position of data in the recording medium to be managed. Then, while in this embodiment the MPEG2-PS (Program Stream) will be exemplified as a video/audio stream, the video/audio stream may be a bitstream into which signals are multiplexed according to the other MPEG formats such as MPEG1 or MPEG2-TS (Transport Stream). Further, while the video/audio streams are sequentially recorded in a recording medium in the below description, the video/audio streams may be distributed at random in a recording medium. Further, the recording medium103may be a memory such as an SDRAM for temporarily storing program data for digital broadcasting.

A data reading circuit104reads the video/audio stream from a predetermined position in the recording medium103, and writes the read video/audio stream into a stream buffer described later. The video/audio stream is read based on read position information which is an address allocated in the recording medium and which indicates from where in the recording medium103the video/audio stream is to be read, and read amount information indicating the amount of the video/audio streams to be read in one access. Then, the read amount information may be read end position information which is an address allocated in the recording medium and which indicates to where the video/audio stream are to be read.

Each of a stream buffer A105and a stream buffer B106is a memory such as an SDRAM for temporarily storing the video/audio streams read from the recording medium103. An address for indicating write position information for the video/audio stream is allocated in each of the stream buffer A105and the stream buffer B106so as to manage the video/audio streams stored in the respective stream buffers. The stream buffer A105and the stream buffer B106may be memories which are physically different from each other, or an area in a single memory may be allocated between the stream buffer A105and the stream buffer B106. Further, while in this embodiment the stream buffer A105and the stream buffer B106circularly and sequentially store video/audio streams as ring buffers, another data storage method may be used.

A demultiplexer107separates each of the video/audio streams held in the stream buffer A105and the stream buffer B106into a video stream and an audio stream and discriminates between the video stream and the audio stream according to the stream ID contained in the packet header and the sub-stream ID contained in the audio elementary stream. Moreover, the demultiplexer107extracts the SCR contained in the pack header of the video/audio stream and the DTS and PTS contained in the packet header thereof.

An SCR extraction circuit108holds the SCR extracted by the demultiplexer107. An STC control circuit109controls an STC counter110, and sets an initial value for the STC counter110. The STC counter110generates a clock which is a reference time for reproducing the video/audio stream (hereinafter, referred to as “STC”).

A video buffer111temporarily stores the video stream separated from the video/audio stream. The video decoder112decodes the video stream, and reproduces and outputs the decoded video stream. A video time management information extraction circuit113holds the DTS and the PTS of the video stream extracted by the demultiplexer107. A video synchronization control circuit114controls an operation timing for the video decoder112on the basis of the DTS and the PTS for the video stream using a clock outputted from the STC counter110as a reference.

An audio buffer116temporarily stores the audio stream separated from the video/audio stream. The audio decoder117decodes the audio stream, and reproduces and outputs the decoded audio stream. An audio time management information extraction circuit118holds the PTS of the audio stream extracted by the demultiplexer107. An audio synchronization control circuit119controls an operation timing for the audio decoder117on the basis of the PTS for the audio stream using a clock outputted from the STC counter110as a reference.

A display video control circuit121holds a video signal which is played back and outputted by the video decoder112, and further superimposes the held video signal on a video signal which is played back and outputted by the video decoder112and outputs and displays the superimposed video signal, according to the instruction from the system controller102.

Next, a configuration and an operation of the video operation switching circuit101will be described with reference toFIGS. 10 and 11.

FIG. 10is a block diagram illustrating a portion of a remote controller in a case where the video operation switching circuit101is realized as the remoter controller. The video operation switching circuit101has a fast-forward button, a re-wind button, a playback button, and an operation switching button. Generally, by pressing the playback button, normal playback in which the video and the audio are synchronized can be performed, and further a trick play operation such as a fast-forward playback and a reverse playback can be performed by pressing the fast-forward button and the re-wind button during the normal playback. Here, by pressing the operation switching button, an operation performed when the fast-forward button or the re-wind button is pressed is an operation of performing the fast-forward playback or the reverse playback of only the video while the audio is kept being normally played. In this embodiment, this trick play operation is referred to as video trick play. Further, when the playback button is pressed halfway through the video trick play, the operation is such that the video is played with the video being delayed or forwarded by a predetermined time period as compared to the audio. In this embodiment, this playback operation is referred to as video asynchronous playback.

Then, buttons for operating normal playback and playback of only the video may be separately provided with no operation switching button being provided. Further, the video operation switching circuit101may have a setting menu on a screen for video display as shown inFIG. 11, and a video playback operation may be selected from the setting menu.

Next, an operation of the video/audio playback apparatus constructed as described above will be described as to a normal playback operation, an operation of performing video trick play shifted from the normal playback, an operation of performing video asynchronous playback shifted from the video trick play, and an operation of performing the normal playback shifted from the video asynchronous playback.

A. An Operation for Normal Playback

Initially, an operation of normal playback will be described.

When a user selects normal playback using the video operation switching circuit101in a state where the video/audio playback apparatus does not perform playback operation, the system controller102identifies video/audio streams as a target to be reproduced in the recording medium103, and calculates the read position and the amount to be read in the recording medium103. The system controller102clears data stored in the stream buffer A105, and initializes the write position in the stream buffer A105, and then notifies the data reading circuit104of the read position, the amount to be read, and the write position in the stream buffer A105for the video/audio stream as the target to be reproduced. Then, a method for identifying the video/audio stream as the target to be reproduced is defined in detail according to the video CD standards or DVD standards, and the method will not be described here.

When receiving the above notification from the system controller102, the data reading circuit104reads the video/audio streams by a prescribed amount to be read from a prescribed read position in the recoding medium103, and writes the video/audio streams into a prescribed write position in the stream buffer A105.

When the prescribed amount of data are stored in the stream buffer A105, the system controller102instructs the demultiplexer107to start decoding.

When receiving the instruction for starting decoding, the demultiplexer107reads the video/audio stream stored in the stream buffer A105, and separates the video/audio stream into a video stream and an audio stream by identifying a stream ID and a sub-stream ID embedded in the stream, and outputs the video stream and the audio stream to the video buffer111and the audio buffer116, respectively. Further, the demultiplexer107extracts the SCR embedded in the video/audio stream and outputs the SCR to the SCR extraction circuit108, and extracts the DTS and the PTS embedded in the video stream and outputs the DTS and the PTS to the video time management information extraction circuit113, and extracts the PTS embedded in the audio stream and outputs the PTS to the audio time management information extraction circuit118.

The STC control circuit109sets a value of the SCR held in the SCR extraction circuit108as an initial value for the STC counter110, and the STC counter110outputs the STC to the demultiplexer107, the video synchronization control circuit114, and the audio synchronization control circuit119.

The video synchronization control circuit114instructs the video decoder112to decode the video stream when the value of the DTS held in the video time management information extraction circuit113coincides with the value of the STC outputted from the STC counter110, and the video decoder112decodes the video stream held in the video buffer111on receiving the instruction. Further, the video synchronization control circuit114instructs the video decoder112to output the video when the value of the PTS held in the video time management information extraction circuit113coincides with the value of the STC outputted from the STC counter110, and the video decoder112outputs the decoded video stream on receiving the instruction.

The audio synchronization control circuit119instructs the audio decoder117to decode the audio stream and output the decoded audio stream when the value of the PTS held in the audio time management information extraction circuit118coincides with the value of the STC outputted from the STC counter110, and the audio decoder117decodes audio stream held in the audio buffer116and outputs the decoded audio stream on receiving the instruction.

By performing the above-described operation, normal playback in which the video and the audio are synchronized is performed for the desired video/audio stream.

B. A Method for Managing Data Storage in Stream Buffer A105

Hereinafter, a method for managing data storage in the stream buffer A105in the video/audio playback apparatus according to the embodiment will be described.

When the data reading circuit104reads a video/audio stream from the recording medium103, and writes the video/audio stream into the stream buffer A105, the system controller102updates the read position in the recording medium103and write position in the stream buffer A105, and updates the state of space areas in the stream buffer A105. When the stream buffer A105has a predetermined amount of space areas, the system controller102calculates the amount of video/audio stream to be read subsequently and notifies the data reading circuit104of the read position in the recording medium103, write position in the stream buffer A105and the amount to be read.

Further, the demultiplexer107reads the video/audio stream from the stream buffer A105at prescribed intervals on the basis of the STC outputted from the STC counter110after the decoding is started, and notifies the system controller102that the video/audio stream is read. The system controller102updates the read position in the recording medium103and the write position in the stream buffer A105on receiving the notification, and updates the state of the space areas in the stream buffer A105. When the stream buffer A105has a predetermined amount of space areas, the system controller102calculates the amount of video/audio stream to be read subsequently, and notifies the data reading circuit104of the read position in the recording medium103, the write position in the stream buffer A105, and the amount to be read.

The above-described operation will be described with reference toFIGS. 2 and 3.FIGS. 2 and 3are schematic diagrams illustrating the internal states of the stream buffer A105. In a state where the stream buffer A105has a spare area S1as shown inFIG. 2(a), when the video/audio stream is written from the write position W1, the subsequent data write position and the space area are updated respectively, and the state of the stream buffer A105is such that the space area is S2and the write position is W2as shown inFIG. 2(b). In such a state, that is, in a state shown inFIG. 3(a), when the demultiplexer107reads data, the space area S2is updated and the state of the stream buffer A105is such that the stream buffer A105has space areas S3and S4as shown inFIG. 3(b).

Thereafter, the system controller102, the demultiplexer107and the data reading circuit104repeat the above-described operations, respectively, thereby managing the data storage in the stream buffer A105.

C. An Operation for Shifting from the Normal Playback Operation to Reverse Playback as a Video Trick Play Operation

Next, an operation performed in a case where a user selects reverse playback as a video trick play operation using the video operation switching circuit101during the normal playback operation will be described.

FIG. 7is a diagram schematically illustrating an asynchronization between the playback times of the video stream and the audio stream which are reproduced by the video playback circuit115and the audio playback circuit120, respectively. The abscissa axis indicates reproduction times for the video/audio stream, and V1and A1indicate reproduction position of the video stream and reproduction position of the audio stream, respectively.

As shown inFIG. 7(a), at the normal playback, the video stream and the audio stream having the same time management information are synchronously reproduced. Here, when reverse playback operation is selected using the video operation switching circuit101, the demultiplexer107stops the outputs of the video stream, and the DTS and the PTS embedded in the video stream. Further, the system controller102performs control so that the video decoder112stops decoding the video stream, the video frame which was outputted most recently remains outputted and displayed, and the video stream stored in the video buffer111is initialized.

The system controller102obtains the PTS of the video frame which was outputted most recently from the video time management information extraction circuit113, calculates the read position and the amount to be read in the recording medium103for the video/audio stream containing I picture immediately before the video frame which was reproduced and outputted most recently on the basis of the PTS, and notifies the data reading circuit104of the read position, the amount to be read, and the write position in the stream buffer B106.

The data reading circuit104reads the video/audio stream containing the I picture immediately before the most recently outputted video frame by a prescribed amount to be read from a prescribed read position in the recording medium103, and writes the video/audio stream into a prescribed write position in the stream buffer B106.

When the data reading circuit104writes the video/audio stream into the stream buffer B106, the system controller102updates the read position in the recording medium103and the write position in the stream buffer B106which are previously calculated.

D. An Update of a Read Position in a Recording Medium

Here, the update of the read position in the recording medium103which is performed by the system controller102will be described with reference toFIGS. 4,5, and6.FIGS. 4,5, and6are schematic diagrams illustrating video/audio streams in the recording medium, andFIGS. 4(a),5(a), and6(a) each shows a state before update of the read position whileFIGS. 4(b),5(b), and6(b) each shows a state in which the read position has been updated.

In updating the read position, in a case where when the video/audio stream is read from the read position R1before the read position is updated, I picture I2has not been completely read as shown inFIG. 4(a), the position immediately after the position R2at which the most recent readout of the video/audio stream was completed is set as a new read position R3as shown inFIG. 4(b).

On the other hand, in a case where the readout of I picture I3is completed in reading video/audio stream from the read position R4before the update of the read position as shown inFIG. 5(a), a record position in the recording medium103from which the whole of I picture I2timewise before the already read I picture I3can be read is set as a new read position R5as shown inFIG. 5(b). At this time, when I picture to be read is determined, some pieces of I pictures may be skipped in consideration of the playback speed for reverse playback. For example, as shown inFIG. 6, in a case where the readout of I picture I4is completed in reading the video/audio stream from the read position R6before the update of the read position, I picture I3may be skipped and a record position in the recording medium103from which the whole of I picture I2in the recording medium103can be read may be set as a new read position R7.

Next, the demultiplexer107reads video/audio stream stored in the stream buffer B106, and identifies the stream ID embedded in the stream, and thereby separates the video stream and outputs the video stream to the video buffer111.

The video decoder112captures video stream containing I picture held in the video buffer111, and decodes and outputs only the I picture. The video decoder112outputs the I picture for a predetermined time period, and then requests the demultiplexer107to output the subsequent video stream.

Then, data stored in the stream buffer B106are managed in a method similar to the above described method for the stream buffer A105. That is, the demultiplexer107notifies the system controller102that it reads the video/audio stream in the stream buffer B106, and the system controller102updates the space areas in the stream buffer B106. When the stream buffer B106has spare areas, the system controller102calculates the amount of video/audio stream to be read subsequently, notifies the data reading circuit of the read position, the amount to be read, and the write position, and instructs the data reading circuit to write the video/audio stream into the stream buffer B106.

The demultiplexer107reads the video/audio stream stored in the stream buffer B106, and identifies the stream ID embedded in the stream, and thereby separates the video stream and outputs the video stream to the video buffer111.

The system controller102manages video/audio stream for playing audio which are stored in the stream buffer A105as well as processes the video stream as described above. The audio stream which is separated from the video/audio stream for playing audio by the demultiplexer107is continuously and normally reproduced on the basis of the STC outputted from the STC counter110and the time management information embedded in the audio stream without being influenced by the processing of the video stream. Then, the system controller102controls the speed at which the demultiplexer107reads the video/audio streams from the stream buffer A105so as to prevent the video/audio streams for playing audio to be held in the stream buffer A105from being absent, and the video/audio stream may be written into the stream buffer A105with higher priority than the video/audio stream is written into the stream buffer B106.

The above-described operation enables the video trick play in which the reverse playback for video stream is performed with the audio stream being normally reproduced. For example, as shown inFIG. 7(b), while the audio stream at A2is normally reproduced, the video stream at V2can be reproduced. Further, when the video trick play further proceeds from the state shown inFIG. 7(b), the video stream at V3can be reproduced while the audio stream at A3is normally reproduced as shown in FIG.7(c).

Then, while the video reverse playback is taken as an example in the above description, in the case of the video fast-forward playback, I picture in the video stream are read in the forward direction, thereby realizing an operation similar to the operation described for the video reverse playback. Further, operations for video slow playback, video frame advance playback, video pause or the like can be performed independently of the normal playback of audio similarly to the video reverse playback.

E. An Operation for Shifting from a State where Video Reverse Playback as Video Trick Play is Performed Independently of the Audio Playback to Video Asynchronous Playback

Next, an operation for shifting from a state where video trick play in which reverse playback of video is performed independently of playback of audio as described above to video asynchronous playback for playing video to be performed will be described.

FIG. 8is a diagram schematically illustrating an asynchronization between the playback times of the video stream and the audio stream which are reproduced by the video playback circuit115and the audio playback circuit120, respectively. As shown inFIG. 8(a), when a user selects a video asynchronous playback operation using the video operation switching circuit101halfway through the video trick play in which reverse playback operation of only video stream is performed, the demultiplexer107stops the outputs of the video stream, and the DTS and the PTS embedded in the video stream. Further, the system controller102performs control so that the video decoder112stops decoding the video stream, the video frame which was outputted most recently remains outputted and displayed, and the video stream stored in the video buffer111is initialized.

The system controller102obtains the PTS of the video frame which was outputted most recently at the reverse playback from the video time management information extraction circuit113, and calculates the read position and the amount to be read in the recording medium103for the video/audio stream containing the PTS. Further, the system controller102clears data stored in the stream buffer B106to initialize the write position in the stream buffer B106, and notifies the data reading circuit104of the read position, the amount to be read, and the write position. Further, the system controller102obtains a reference time STC at the time when the reverse playback operation processing is stopped from the STC counter110, calculates a difference between the STC and the PTS of video frame which was outputted most recently at the reverse playback (ΔSTC), and outputs the difference to the video synchronization control circuit114.

When the data reading circuit104is notified of the read position and the like from the system controller102, the data reading circuit104reads the video/audio stream from a prescribed read position in the recording medium103by a prescribed amount to be read, and writes the video/audio stream into a prescribed write position in the stream buffer B106.

The demultiplexer107reads the video/audio stream stored in the stream buffer B106, and identifies the stream ID embedded in the stream, and thereby separates the video stream and outputs the video stream to the video buffer111, and outputs the DTS and the PTS embedded in the video stream to the video time management information extraction circuit113.

When a value obtained by subtracting the ΔSTC from a value of the STC outputted from the STC counter110coincides with the value of the DTS held in the video time management information extraction circuit113, the video synchronization control circuit114instructs the video decoder112to start the decoding. The video decoder112decodes the video stream in video buffer111on receiving the instruction. Further, when a value obtained by subtracting the ΔSTC from the value of the STC outputted from the STC counter110coincides with the value of the PTS held in the video time management information extraction circuit113, the video synchronization control circuit114instructs the video decoder112to output the video, and the video decoder112outputs the decoded video stream on receiving the instruction.

The system controller102manages video/audio stream for playing audio which are stored in the stream buffer A105as well as processes the video stream as described above. The audio stream which is separated from the video/audio stream for playing audio by the demultiplexer107is continuously and normally played on the basis of the STC outputted from the STC counter110and the time management information embedded in the audio stream without being influenced by the processing of the video stream. Then, the system controller102controls the speed at which the demultiplexer107reads the video/audio streams from the stream buffer A105so as to prevent the video/audio streams for playing audio to be held in the stream buffer A105from being absent, and the video/audio stream may be written into the stream buffer A105with higher priority than the video/audio stream is written into the stream buffer B106.

In the above-described operation, the video trick play in which the reverse playback of only the video stream is performed independently of the normal playback of the audio stream can be shifted to the video asynchronous playback in which the video stream and the audio stream are asynchronously played back by an arbitrary time, thereby performing the video asynchronous playback. That is, it is possible to simultaneously playback the video stream and the audio stream having different time management information from each other. For example, as shown inFIG. 8(b), while the video stream positioned at V4is reproduced, the audio stream positioned at A5can be reproduced. When the playback operation further proceeds from the state shown inFIG. 8(b), the video stream positioned at V5is reproduced and the audio stream positioned at A6can be reproduced with maintaining asynchronization between the playback times of the video stream and the audio stream as shown inFIG. 8(c).

Then, while in this embodiment an operation for shifting from a state where the reverse playback of only video is performed with the audio being normally played to the video playback to be performed is taken as an example, an operation of starting normal playback from a video pause state, or an operation of starting the playback of only the video at an arbitrary position by an operation such as video skip playback or video time search can be similarly realized.

F. An Operation of Shifting from a State of Video Asynchronous Playback to a Normal Playback Operation

Next, an operation for shifting from a state of the video asynchronous playback in which a video is played a predetermined time later than an audio is played as described above to a state of normal playback in which a video is played in synchronization with an audio to be performed again will be described.

FIG. 9is a diagram schematically illustrating an asynchronization between the playback times of the video stream and the audio stream which are reproduced by the video playback circuit115and the audio playback circuit120, respectively. As shown inFIG. 9(a), when a user selects normal playback operation using the video operation switching circuit101halfway through reproducing the video stream and the audio stream asynchronously by an arbitrary time, the demultiplexer107stops the outputs of the video stream read from the stream buffer B106, and the DTS and the PTS embedded in the video stream. Further, the system controller102performs control so that the video decoder112stops decoding the video stream, the video frame which was outputted most recently remains outputted and displayed, and the video stream stored in the video buffer111is initialized.

The system controller102stops reproducing video stream when receiving the instruction for the video playback start operation from the video operation switching circuit101. The demultiplexer121stops outputs of the video stream, and the DTS and the PTS embedded in the video stream. The system controller102performs control so that the video decoder112stops decoding the video stream, the video frame which was outputted most recently remains outputted and displayed, and the video stream stored in the video buffer111is initialized

The system controller102and the data reading circuit104continuously perform writing into the stream buffer A105.

The demultiplexer107outputs the video stream separated from the video/audio stream which is continuously read from the stream buffer A105to the video buffer111again, and outputs the DTS and the PTS embedded in the video stream to the video time management information extraction circuit113.

Thereafter, the normal playback operation in which the audio stream and video stream are synchronized as described above is performed and thereby the normal playback in which a video is played in synchronization with an audio can be shifted and performed from the state of the video asynchronous playback in which a video is played a predetermined time later than an audio is played as shown inFIGS. 9(b) and9(c).

Then, when the playback time of the video overtakes that of the audio in a state where the fast-forward playback or the reverse playback for the video is being performed independently of the playback of the audio, the playback of the video can be started in synchronization with the playback of the audio by the above-described operation.

G. An Operation for Selecting Video Pause as a Video Trick Play Operation to Display Another Screen During Normal Playback Operation

Next, an operation performed in a case where a user selects video pause as a video trick play operation using the video operation switching circuit101to display another screen during the normal playback operation will be described.

FIGS. 17 to 19are diagrams illustrating monitor screens during video pause operation.

When a user selects video pause operation using the video operation switching circuit101, the display video control circuit121holds the video signal which was outputted by the video decoder112most recently, and composites the held video and video signals which are played back and outputted by the video decoder112to output the resultant for display. Thereby, the video displayed when the video pause operation is selected is displayed on a sub-screen S1with the normal playback operation being continued on a main screen M1on the monitor screen as shown inFIG. 17.

Further, as shown inFIG. 17, when a user further selects the video pause operation in a state where the normal playback is performed on the main screen M1with the paused video being displayed on the sub-screen S1in the video pause operation, the display video control circuit121additionally displays the video which was outputted onto the main screen M1immediately before the video pause operation as a new sub-screen S2. Thereby, a plurality of sub-screens S1and S2are displayed together with the main screen M1on the monitor screen as shown inFIG. 18. When the plurality of sub-screens are displayed, the whole monitor screen may be divided into sub-screens S1, S2, S3and S4to be displayed as shown inFIG. 19.

Further, in such as a state shown inFIG. 18, when the user selects the sub-screen S1from the displayed sub-screens S1and S2using a video operation selection circuit not shown and further indicates the video normal playback or the video trick play using the video operation switching circuit101, the displayed image is switched between the main screen M1and the sub-screen S1, and the video normal playback or the video trick play is started on the main screen M1from the paused image which has been displayed on the sub-screen S1which is selected, and the image which has been played on the main screen M1immediately before the displayed image is switched is displayed as paused image on the sub-screen S1.

Then, while the video pause is taken as an example in the above description, another screen can be displayed for the video fast-forward playback, video reverse playback, video slow playback, video frame advance playback and the like in a manner similar to the video pause.

As described above, the video/audio playback apparatus according to this embodiment reproduces a video/audio stream for playing audio and a video/audio stream containing a picture frame required for the trick play individually, and reproduces the audio stream on the basis of the time management information embedded in the audio stream and a reference time which is set based on the time management information embedded in the audio stream, thereby enabling the trick play of only the video being performed with the audio being normally played, and improving the operability and convenience for the user.

Further, in a case where a video asynchronous playback in which the audio stream and the video stream having the same time management information are asynchronously reproduced is indicated in a state where the trick play of only the video stream is being performed, the video/audio playback apparatus according to this embodiment calculates a differential value between the reference time as of the time at which the video asynchronous playback is indicated and the time management information embedded in the video stream which is being subjected to the trick play and displayed, and sets a value obtained by subtracting the differential value from the reference time as a reference time used for reproducing the video stream, and therefore a state where the trick play of only the video is being performed can be shifted to playing the audio and the video asynchronously by a desired time, thereby improving the operability and the convenience of the user.

Moreover, in a case where the time management information embedded in the video stream which is being subjected to video trick play and displayed is identical to the time management information embedded in the audio stream being reproduced in a state where the trick play of only the video stream is being performed, the video/audio playback apparatus according to this embodiment reproduces the video stream separated from the video/audio stream for playing the audio on the basis of the reference time, and therefore the audio stream and the video stream having the same time management information can be synchronously reproduced without performing a special operation, thereby improving the operability and convenience for the user.

Further, in the video/audio playback apparatus according to the present invention, a display video is displayed on another screen, from which the playback is started, and thereby a video which a user desires to carefully view can be freely displayed and played with the audio being normally played, thereby improving the operability and the convenience for the user.

Then, when the user sets the playback time as being displayed on the screen, the playback time of audio can be displayed on a screen such as a monitor as a current playback time by combining the playback time management information with the video signal outputted from the video decoder112, for example, as shown inFIG. 12. Further, both video playback time and audio playback time may be displayed as shown inFIG. 13. Further, when the user sets a setting of “time to be displayed” to “video” in the setting menu shown inFIG. 11, only the video playback time can be displayed as shown inFIG. 14.

Further, while in this embodiment the description is given for the video and audio streams, the similar operation is possible for sub-video stream.

Moreover, while in this embodiment the playback of a recording medium such as a DVD or an HDD is taken as an example, the reproduction of the streaming contents through a digital broadcasting or a network can be performed in the similar operation by temporality storing the streaming contents in a recording medium such as a memory.

The video/audio playback apparatus according to the present invention is useful in that an operation of freely playing a video can be performed with an audio being normally played, and an apparatus for reproducing a coded video/audio stream from a random accessible recoding medium can be provided with a novel playback function.