Source: http://patents.com/us-10057612.html
Timestamp: 2019-06-19 12:48:49
Document Index: 108491373

Matched Legal Cases: ['Application No. 61', 'Application No. 61', 'Application No. 61', 'art 41010', 'art 41020', 'art 41030']

US Patent # 1,005,7612. Method and device for transmitting and receiving broadcast signal for providing trick play service in digital broadcasting system - Patents.com
United States Patent 10,057,612
Hwang , et al. August 21, 2018
The present invention relates to a method and device for transmitting and receiving a broadcast signal for providing trick play in a digital broadcasting system. A broadcast signal transmission method, according to an embodiment of the present invention, comprises the steps of: generating a video stream by encoding video data, wherein the video stream comprises PVR assist information for performing trick play of the video data; generating signaling information; multiplexing the video stream and the signaling information into one broadcast stream; generating a broadcast signal comprising the multiplexed broadcast stream; and transmitting the generated broadcast signal.
Family ID: 53199319
15/039,735
PCT/KR2014/011044
WO2015/080415
US 20160381400 A1 Dec 29, 2016
Current CPC Class: H04N 21/234 (20130101); H04N 21/236 (20130101); H04N 21/2387 (20130101); H04N 21/23614 (20130101); H04N 21/4325 (20130101); H04N 21/6332 (20130101); H04N 21/4147 (20130101); H04N 21/4383 (20130101); H04N 21/6587 (20130101); H04N 19/31 (20141101)
Current International Class: H04N 21/238 (20110101); H04N 21/6587 (20110101); H04N 21/658 (20110101); H04N 21/438 (20110101); H04N 21/4147 (20110101); H04N 21/6332 (20110101); H04N 21/432 (20110101); H04N 21/236 (20110101); H04N 21/234 (20110101); H04N 21/2387 (20110101)
9380280 June 2016 Jang
9860607 January 2018 Hwang
2003/0118243 June 2003 Sezer
2003/0118322 June 2003 Kim
2004/0028386 February 2004 Walls
2014/0119712 May 2014 Jang
2016/0234537 August 2016 Rodriguez
2008-294638 Dec 2008 JP
2009-159625 Jul 2009 JP
10-2013-0129080 Nov 2013 KR
ETSI TS 101 154 v1.9.1 (Sep. 2009), "Digital Video Broadcasting (DVB); Specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream," pp. 1-163. cited by applicant .
Okubo, H.265/HEVC Textbook (Oct. 21, 2013), pp. 192-214. cited by applicant .
DVB Organization: "TS 101 154v1.12.1 HEVC strawman Sep. 11, 2013; List of sections to be modified", XP017842484, Digital Video Broadcasting, Geneva, Switzerland, Sep. 19, 2013. cited by applicant .
Digital Video Broadcasting (DVB): "Specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream", XP014092645, ETSI TS 101 154 V1.11.1, European Telecommunications Standards Institute, Nov. 2012. cited by applicant .
Digital Video Broadcasting (DVB): "Specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream", XP014215858, Draft ETSI TS 101 154 V2.1.1, European Telecommunications Standards Institute, Oct. 2, 2014. cited by applicant.
This application is a National Stage Application of International Application No. PCT/KR2014/011044 filed Nov. 18, 2014, which claims priority from U.S. Provisional Application No. 61/910,416 filed on Dec. 1, 2013, U.S. Provisional Application No. 61/952,140 filed on Mar. 13, 2014, and U.S. Provisional Application No. 61/970,910 filed on Mar. 27, 2014, each of which are hereby incorporated by reference in their entirety.
1. A broadcast signal transmission method comprising: encoding video data to create a video elementary stream, wherein the video elementary stream includes a temporal sub layer for supporting temporal scalability, and wherein the temporal sub layer is identified by a temporal identifier; creating a packetized elementary stream (PES) packet comprising the video elementary stream; creating a transport stream (TS) packet including an adaptation field and a payload, wherein the payload comprises the created PES packet, wherein the adaptation field comprises Personal Video Recorder (PVR) assist information for executing a trick play, wherein the PVR assist information comprises tier number information indicating a tier number obtained by using the temporal identifier of a picture in the temporal sub layer, wherein the PVR assist information comprises maximum temporal identification information indicating a maximum temporal identifier of the video elementary stream comprising the encoded video data, and wherein the maximum temporal identification information is used for providing speeds of the trick play supported from the video elementary stream; creating a broadcast signal comprising the created TS packet; and transmitting the created broadcast signal.
2. The broadcast signal transmission method according to claim 1, wherein the adaptation field comprises a private data byte field, and wherein the private data byte field comprises the PVR assist information.
3. The broadcast signal transmission method according to claim 1, wherein the tier number is 0 when the picture is a Random Access Point (RAP) picture and the tier number is equal to the temporal identifier of the picture plus 1 when the picture is different from the RAP picture, and wherein the temporal identifier is signaled by a nuh_temporal_id_plus1 field in a Network Abstraction Layer (NAL) unit carrying the picture of the temporal sub layer.
4. The broadcast signal transmission method according to claim 1, wherein the PVR assist information comprises additional tier number information indicating an additional tier number, wherein the additional tier number indicates dependencies of pictures which have 0 as a value of the temporal identifier, wherein a higher speed trick play is provided by decoding pictures having 0 as a value of the temporal identifier and having a specific additional tier number, and wherein a lower speed trick play is provided by decoding pictures having 0 as a value of the temporal identifier and pictures having 1 or more as a value of the temporal identifier.
5. The broadcast signal transmission method according to claim 1, wherein the trick play is provided by only parsing a nuh_temporal_id_plus1 field in a Network Abstraction Layer (NAL) unit carrying the picture of the temporal sub layer without parsing the tier number information.
6. The broadcast signal transmission method according to claim 1, wherein the PVR assist information comprises frame number information indicating a number of frames existing between intra frames.
7. A broadcast signal reception device comprising: a tuner for receiving a broadcast signal comprising a transport stream (TS) packet including an adaptation field and a payload, wherein the payload comprises a packetized elementary stream (PES) packet comprising a video elementary stream, wherein the video elementary stream includes a temporal sub layer for supporting temporal scalability, wherein the temporal sub layer is identified by a temporal identifier, wherein the adaptation field comprises Personal Video Recorder (PVR) assist information for executing a trick play, wherein the PVR assist information comprises tier number information indicating a tier number obtained by using the temporal identifier of a picture in the temporal sub layer, wherein the PVR assist information comprises maximum temporal identification information indicating a maximum temporal identifier of the video elementary stream comprising video data, and wherein the maximum temporal identification information is used for providing speeds of the trick play supported from the video elementary stream; a demux for extracting the TS packet from the broadcast signal, the demux extracting the PES packet from the extracted TS packet, the demux extracting the video elementary stream from the extracted PES packet; and a decoder for decoding the extracted video elementary stream using the tier number information and the maximum temporal identification information.
8. The broadcast signal reception device according to claim 7, wherein the adaptation field comprises a private data byte field, and wherein the private data byte field comprises the PVR assist information.
9. The broadcast signal reception device according to claim 7, wherein the tier number is 0 when the picture is a Random Access Point (RAP) picture and the tier number is equal to the temporal identifier of the picture plus 1 when the picture is different from the RAP picture, and wherein the temporal identifier is signaled by a nuh_temporal_id_plus1 field in a Network Abstraction Layer (NAL) unit carrying the picture of the temporal sub layer.
10. The broadcast signal reception device according to claim 7, wherein the PVR assist information comprises additional tier number information indicating an additional tier number, wherein the additional tier number indicates dependencies of pictures which have 0 as a value of the temporal identifier, wherein a higher speed trick play is provided by decoding pictures having 0 as a value of the temporal identifier and having a specific additional tier number, and wherein a lower speed trick play is provided by decoding pictures having 0 as a value of the temporal identifier and pictures having 1 or more as a value of the temporal identifier.
11. The broadcast signal reception device according to claim 7, wherein the trick play is provided by only parsing a nuh_temporal_id_plus1 field in a Network Abstraction Layer (NAL) unit carrying the picture of the temporal sub layer without parsing the tier number information.
12. The broadcast signal reception device according to claim 7, wherein the PVR assist information comprises frame number information indicating a number of frames existing between intra frames.
With the development of digital technology and communication technology, audio/video-based multimedia content has been increasingly distributed and requested in various fields, such as the Internet and personal media, as well as broadcasts and movies. In addition, consumers' demand for realistic media that provide realism and a sense of presence has increased with the popularization of 3DTV/3D movies, which provide a stereoscopic effect through broadcasts and movies. Furthermore, as the size of TVs for homes is increased together with the development of display technology, consumers' demand for realistic content at a high quality of HD or higher level is increasing. As a result, realistic broadcasts through an Ultra High Definition TV (UHDTV) and a 3DTV, which are provided for the post-HDTV market, have attracted considerable attention as next-generation broadcast service. In particular, an Ultra High Definition (UHD) broadcast service has been increasingly discussed.
UHD broadcast content has higher quality than SD or HD broadcast content. For this reason, the amount of data to be transmitted is relatively large. In order to efficiently transmit such high-quality content, therefore, new data compression methods have been developed. In recent years, an HEVC compression method, which is superior to a conventional H.264/AVC compression method, has been commercialized. Consequently, it is necessary to conduct research into a method of utilizing content data encoded using HEVC.
Meanwhile, a trick play means a service providing a function that is capable of reproducing an image after a random time, i.e. enabling random access, as well as an X speed function, such as an X2 speed and an X4 speed. Since there is a difference between a random access point of HEVC and a random access point of H.264, it is necessary to define a new category for the random access point of HEVC. In addition, HEVC provides scalability, and therefore it is necessary to provide a trick play using the same. Furthermore, a conventional CFF media file format specification defines a format for a trick play of H.264/AVC. However, content encoded by HEVC is not defined. Consequently, it is necessary to provide a new format for decoding and a trick play of content encoded by HEVC.
An object of the present invention devised to solve the problem lies on a method and/or device for transmitting and receiving a broadcast signal for a trick play service.
Another object of the present invention devised to solve the problem lies on the definition of a file format suitable for an HEVC stream to more efficiently provide a trick play.
Another object of the present invention devised to solve the problem lies on a method of signaling information regarding a trick play in order to more efficiently provide a trick play in an HEVC stream.
A further object of the present invention devised to solve the problem lies on a method and/or device of providing a trick play that is backward compatible with a broadcasting system based on a conventional encoding method.
The object of the present invention can be achieved by providing a broadcast signal transmission method including encoding video data to create a video stream, wherein the video stream includes PVR assist information for executing a trick play of the video data, creating signaling information, multiplexing the video stream and the signaling information into a single broadcast stream, creating a broadcast signal including the multiplexed broadcast stream, and transmitting the created broadcast signal.
The video stream may include an adaptation field, and the adaptation field may include the PVR assist information.
The adaptation field may include a private data byte field, and the private data byte field may include the PVR assist information.
The video stream may include one or more temporal sublayers, each temporal sublayer indicating a group of pictures, and a NAL unit header including the encoded video data may include temporal identification plus information indicating a value obtained by adding 1 to temporal identification information for identifying the temporal sublayers, the temporal identification plus information being used to identify the temporal sublayers.
The PVR assist information may include tier number information indicating the tier number of pictures included in the video stream, the tier number indicated by the tier number information being used to signal the temporal sublayers.
The tier number of an HEVC RAP picture, among the pictures included in the video stream, may have a value of 0, and the tier number of pictures other than the HEVC RAP picture may have a value obtained by adding 1 to a value indicated by the temporal identification information.
The PVR assist information may include tier next picture information indicating the relative position of the next picture in decoding order among pictures having a tier number equal to a value indicated by the tier number information.
In another aspect of the present invention, provided herein is a broadcast signal reception device including a tuner for receiving a broadcast signal, a demux for demultiplexing the received broadcast signal to extract a video stream, wherein the video stream includes PVR assist information for executing a trick play of video data, and a decoder for decoding the extracted video stream based on the PVR assist information and executing a trick play of the video data.
The decoder may decode only pictures having a specific temporal identification information value to execute a trick play.
The PVR assist information may include tier number information indicating the tier number of pictures included in the video stream, the tier number indicating dependency between pictures having a temporal identification information value of 0, and the decoder may decode only pictures having a specific tier number, among the pictures having a temporal identification information value of 0, to execute a high X speed trick play, and may decode pictures having a temporal identification information value of 0 and pictures having a specific temporal identification information value of more than 0 to execute a low X speed trick play.
The broadcast signal reception device may further include a parser for parsing the temporal identification plus information and transmitting only pictures having a specific temporal identification information value to the decoder, wherein the decoder may decode only the pictures received from the parser to execute a trick play.
The PVR assist information may include information indicating the number of frames, existing between intra frames, which different from the intra frames in a temporal sublayer having a temporal identification information value of 0.
The present invention has the effect of providing a trick play service for an HEVC stream.
The present invention has the effect of providing a trick play service using the structure of an HEVC video stream without modification.
The present invention has the effect of providing a trick play service using a temporal ID-based structure of an HEVC video stream, thereby providing a higher encoding speed without the necessity of signaling a specific picture that will be used in a trick play at an encoding step.
The present invention has the effect of providing a trick play service at a higher X speed.
The present invention has the effect of using a conventional system related to the provision of a trick play service without modification.
FIG. 6 is a view showing a trick play method in a case of a closed GOP according to an embodiment of the present invention (scenario 11).
FIG. 7 is a view showing a trick play method in a case of an open GOP wherein the GOP includes a decodable leading picture according to an embodiment of the present invention (scenario 12).
FIG. 8 is a view showing a trick play method in a case of an open GOP wherein the GOP includes a decodable leading picture and a skipped leading picture according to an embodiment of the present invention (scenario 12).
FIG. 9 is a view showing a trick play method in a case of an open GOP wherein the GOP includes a skipped leading picture according to an embodiment of the present invention (scenario 12).
FIG. 10 is a view showing the configuration of a trick play box for supporting a trick play of an HEVC stream having a max_temporal_id of 0 according to an embodiment of the present invention (scenario 12).
FIG. 11 is a view showing the configuration of a trick play box for supporting a trick play of an HEVC stream having a max_temporal_id of 0 according to another embodiment of the present invention (scenario 12).
FIG. 13 is a view showing the configuration of a trick play box for supporting a trick play of an HEVC stream having a max_temporal_id of 0 in a case in which pic_type does not include contents related to a leading picture according to an embodiment of the present invention (scenario 11).
FIG. 14 is a view showing the configuration of a trick play box for supporting a trick play of an HEVC stream having a max_temporal_id of 0 in a case in which pic_type does not include contents related to a leading picture according to another embodiment of the present invention (scenario 11).
FIG. 18 is a view showing a frame rate changing method in a case in which a temporal sublayer picture type is TSA according to an embodiment of the present invention.
FIG. 19 is a view showing a frame rate changing method in a case in which a temporal sublayer picture type is STSA according to an embodiment of the present invention (scenario 3).
FIG. 26 is a view showing a trick play method based on a method of mapping one temporal id to one tier one to one according to an embodiment of the present invention (scenario Aa).
FIG. 27 is a view showing a trick play method based on a method of mapping one temporal id to one tier one to one according to another embodiment of the present invention (scenario Aa).
FIG. 30 is a view showing a trick play method based on a method of mapping one temporal id to several tiers according to an embodiment of the present invention (scenario Ab).
FIG. 35 is a view showing the configuration of PVR_assist_information having a temporal id frame work added thereto according to an embodiment of the present invention (scenario Baa).
FIG. 36 is a view showing the configuration of PVR_assist_information having a temporal id frame work added thereto according to another embodiment of the present invention (scenario Bab).
FIG. 37 is a view showing the configuration of PVR_assist_information for supporting a trick play using a temporal id according to an embodiment of the present invention (scenario Bb).
FIG. 38 is a view showing a reception device according to an embodiment of the present invention.
FIG. 39 is a view showing comparison between a tier framework and an HEVC temporal sublayer according to an embodiment of the present invention.
FIG. 41 is a view showing a trick play method using an HEVC temporal sublayer according to an embodiment of the present invention.
FIG. 42 is a view showing a broadcast signal transmission method according to an embodiment of the present invention.
FIG. 43 is a view showing the structure of a broadcast signal reception device according to an embodiment of the present invention.
Tier is a term introduced to support a trick play in an AVC stream or an mpeg2 stream and may be included in an adaptation field in a TS packet.
In a case in which a video stream according to an embodiment of the present invention is a stream which does not provide temporal scalability, i.e. in a case in which the video stream includes only a picture having a temporal_id of 0, for a Closed GOP, dependency_level and pic_type defined in a CFF media file format may be signaled to provide a trick play since the video stream does not include a leading picture (scenario 11). On the other hand, for an Open GOP, leading picture type may be included in pic_type and dependency_level may be signaled to provide a trick play (scenario 12).
In this figure, NL0", "NL1", and "NL2" may indicate an inclusion relationship of CFF boxes, "Format Req." may indicate existence or nonexistence of corresponding boxes and the number of the corresponding boxes, "Specification" may indicate a portion of standards defining the corresponding boxes, and "Description" may indicate the description of the corresponding boxes. In a case in which "Format Req." is "+", it may mean that one or more corresponding boxes exist. In a case in which "Format Req." is "0/1", it may mean that a corresponding box exists or does not exist. In a case in which "Format Req." is "1", it may mean that a corresponding box exists. In a case in which "Format Req." is "0", it may mean that a corresponding box does not exist.
A shaded picture type (TSA, STSA) in this figure may indicate an HEVC temporal sublayer picture type which may be used for a trick play in a case in which a stream having a temporal ID of more than 0 is included, i.e. temporal scalability is provided. The shaded picture type may be used scenarios 2 and 3 according to an embodiment of the present invention.
According to an embodiment of the present invention, random access point pictures may include an Instantaneous Decoding Refresh (IDR) picture, a Broken Link Access (BLA) picture, and/or a Clean Random Access (CRA) picture, leading pictures may include a Random Access Decodable Leading (RADL) picture and/or a Random Access Skipped Leading (RASL) picture, and temporal sublayer access pictures may include a Temporal Sublayer Access (TSA) picture and/or a Stepwise Temporal Sublayer Access (STSA) picture.
The Temporal Sublayer Access (TSA) picture may include a case in which the TSA picture is not referred to by the same sublayer and/or a case in which the TSA picture is referred to by the same sublayer.
The Stepwise Temporal Sublayer Access (STSA) picture may include a case in which the STSA picture is not referred to by the same sublayer and/or a case in which the STSA picture is referred to by the same sublayer.
This figure shows a method of executing a trick play in a case in which a maximum temporal_id in a video stream is 0 and in a case of a closed GOP. A GOP is short for a Group Of Pictures and indicates a group of coded pictures for enabling random access. A closed GOP may mean a GIOP which does not include a leading picture and an Open GOP may mean a GIOP which includes a leading picture. The leading picture may indicate a picture in HEVC having a later decoding order but an earlier display order than an Intra Random Access Point (IRAP) (the same concept as a random access point in AVC codec).
According to an embodiment of the present invention, a new box "trikhvc" may be defined. A trikhvc box according to an embodiment of the present invention may include a pic_type field, a temporal_sub_layer_pic_type field, a max_temporal_id field, a temporal_id field, a constraint_trick_play_mode field, and/or a next_temporal_id field in a for loop repeated by a value of sample_count.
The temporal_sub_layer_pic_type field may indicate whether a corresponding picture is a Temporal Sublayer Access (TSA) picture or a Stepwise Temporal Sublayer Access (STSA) picture. In a case in which temporal_sub_layer_pic_type is 1, temporal_sub_layer_pic_type may indicate that a corresponding picture is a TSA picture. In a case in which temporal_sub_layer_pic_type is 2, temporal_sub_layer_pic_type may indicate that a corresponding picture is an STSA picture. In a case in which temporal_sub_layer_pic_type is 3, temporal_sub_layer_pic_type may indicate that a corresponding picture is an unknown picture. HEVC may classify temporal_sub_layer_access_pictures to provide temporal scalability and to adaptively change a frame rate. That is, HEVC may dynamically change a frame rate based on a TSA picture and an STSA picture in a layer not having a temporal_id of 0. There may be a difference between TSA and STSA in terms of how and how much a frame rate can be changed. That is, there may be a difference between TSA and STSA in terms of how many temporal_ids can be skipped at once. For example, on the assumption that, in a case in which temporal_id is 0, a frame rate is 15p, in a case in which temporal_id is 1, a frame rate is 30p, and, in a case in which temporal_id is 2 (max_temporal_id), a frame rate is 60p, TSA may directly access a layer having a temporal_id of 2 from a layer having a temporal_id of 0 and, therefore, a broadcasting system may directly change a service such that a service having a frame rate of 60p is provided while a service having a frame rate of 15p is being provided. On the other hand, only stepwise access is possible for STSA and, therefore, STSA may access a layer having a temporal_id of 1 from a layer having a temporal_id of 0 and then access a layer having a temporal_id of 2. In this case, therefore, the broadcasting system may sequentially provide a service having a frame rate of 15p, a service having a frame rate of 30p, and a service having a frame rate of 60p.
The constraint_trick_play_mode field may indicate providable maximum X speed information. The present invention may signal a providable maximum X speed through an equation of max_trick_play_mode (providable maximum X speed)=2^(max_temporal_id) using a value of max_temporal_id. For example, in a case in which a maximum temporal id is 2, the present invention may maximally provide an X4 speed (2^2). According to an embodiment of the present invention, the present invention may provide a limitation that a value less than the value calculated through the above-described equation is assigned to max_trick_play_mode. For example, in a case in which a value of max_trick_play_mode is 1, it may indicate an X2 speed. In a case in which a value of max_trick_play_mode is 2, it may indicate an X4 speed. In a case in which a value of max_trick_play_mode is 3, it may indicate an X8 speed. In a case in which a value of max_trick_play_mode is 4, it may indicate an X16 speed. Constraint_trick_play_mode may have the same meaning as max_trick_play_mode. According to an embodiment of the present invention, an X speed higher than max_trick_play_mode may not be supported but an X speed equal to max_trick_play_mode may be supported. In addition, according to an embodiment of the present invention, a constraint that a value less than a value of max_trick_play_mode is assigned may be signaled.
The constraint_trick_play_mode field may indicate providable maximum X speed information. The present invention may signal a providable maximum X speed through an equation of max_trick_play_mode (providable maximum X speed)=2^(max_temporal_id) using a value of max_temporal_id. For example, in a case in which a maximum temporal id is 2, the present invention may maximally provide an X4 speed (2^2). According to an embodiment of the present invention, the present invention may provide a limitation that a value less than the value calculated through the above-described equation is assigned to max_trick_play_mode. For example, in a case in which a value of max_trick_play_mode is 1, it may indicate an X2 speed. In a case in which a value of max_trick_play_mode is 2, it may indicate an X4 speed. In a case in which a value of max_trick_play_mode is 3, it may indicate an X8 speed. In a case in which a value of max_trick_play_mode is 4, it may indicate an X16 speed. Constraint_trick_play_mode may have the same meaning as max_trick_play_mode.
In a case in which a temporal sublayer picture type is TSA as shown in this figure, a receiving side may directly display a stream at an X1 speed (normal X speed) during display of only a stream of a layer having a temporal_id of 0, i.e. during execution of a trick play at an X4 speed.
In a case in which a temporal sublayer picture type is TSA as shown in this figure, the receiving side may directly decode and display a picture having a temporal_id of 2 during decoding and displaying of a picture having a temporal_id of 0.
In a case in which a temporal sublayer picture type is STSA as shown in this figure, a receiving side may not directly display a stream at an X1 speed (normal X speed) during display of only a stream of a layer having a temporal_id of 0, i.e. during execution of a trick play at an X4 speed. The receiving side may display a stream at an X2 speed and then display the stream at an X1 speed. In a case in which a temporal sublayer picture type according to an embodiment of the present invention is STSA, therefore, a method of informing of a limitation for a convertible X speed may be needed. That is, it may be necessary to signal next_temporal_id.
In a case in which a temporal sublayer picture type is STSA and it is necessary to display a stream at an X1 speed during displaying of the stream at an X4 speed as shown in this figure, the receiving side may not directly decode and display a picture having a temporal_id of 2 during decoding and displaying of a picture having a temporal_id of 0. The receiving side may decode and display a picture having a temporal_id of 1 and then decode and display a picture having a temporal_id of 2.
The USB memory 23040 may indicate another memory device. A USB memory according to an embodiment of the present invention may store metadata, a URL, and/or a presentation list for a second screen.
A temporal sublayer or a temporal id according to an embodiment of the present invention is a term introduced to support temporal scalability in HEVC and may be signaled by nuh_temporal_id_plus1 of NAL_unit_header.
A tier according to an embodiment of the present invention is a term introduced to support a trick play in an AVC stream or an mpeg2 stream and may be included in an adaptation field in a TS packet.
In order to provide a trick play according to an embodiment of the present invention, a method of mapping one temporal id to one tier one to one may be used (scenario Aa) and a method of mapping one temporal id to several tiers may be used (scenario Ab).
According to an embodiment of the present invention, in scenario As and scenario Ab as described above, a value of temporal id may be mapped to a tier one to one as it is irrespective of the meaning of a conventional tier.
According to another embodiment of the present invention, in scenario Aa and scenario Ab as described above, a value of temporal id may be mapped to a tier one to one and a maximum temporal id may be mapped to tier 6 or 7, which has meaning of a conventional tier as a discardable picture as it is.
According to scenario A as described above, most of a conventional PVR_assist_info descriptor may be used as it is. However, an embodiment of the present invention is based on a HEVC stream and, therefore, it is necessary to modify some fields included in the PVR_assist_info descriptor. For example, it may be newly defined that a PVR_assist_tier_m_cumulative_frames field included in the PVR_assist_info descriptor may deliver a value of the minimum number of frames extractable per 1.28 seconds from tier 1 through a PVR_assist_tier_m field (This field conveys the value of the intended minimum number of extractable frames per 1.28 sec. from tier 1 through "PVR_assist_tier_m").
This figure shows a trick play method based on scenario Aa according to an embodiment of the present invention.
Scenario A according to an embodiment of the present invention may not provide an X8 and higher speed trick play.
This figure shows a result of mapping a value of temporal id to a tier one to one as it is irrespective of the meaning of a conventional tier in scenario Aa according to an embodiment of the present invention.
A nuh_temporal_id plus1 field shown in this figure may be a field included in NAL_unit_header and may indicate a value obtained by adding 1 to a temporal id. For example, in a case in which a value of the nuh_temporal_id plus field is 1, a temporal id may be 0. As shown in this figure, therefore, pictures having a temporal id of 0 may be mapped to tier 1, pictures having a temporal id of 1 may be mapped to tier 2, pictures having a temporal id of 2 may be mapped to tier 3, and pictures having a temporal id of 3 may be mapped to tier 4.
An embodiment of the present invention may map a temporal sublayer and a tier so as to have the same number and may use the above-described mapping information when executing a trick play.
This figure shows a result of mapping a value of temporal id to a tier one to one and mapping a maximum temporal id to tier 6 or 7, which has meaning of a conventional tier as a discardable picture as it is in scenario Aa according to another embodiment of the present invention.
The temporal_sub_layer_dependency_flag field may indicate dependency between temporal sublayers. That is, the temporal_sub_layer_dependency_flag field has a value of 1 in a case in which a lower temporal sublayer picture does not refer to an upper temporal sublayer picture.
The max_temporal_id_plus1 field may indicate a maximum value of a temporal id. A value indicated by the max_temporal_id_plus1 field is a value obtained by adding 1 to a temporal id.
An if (max_temporal_id_plus1>1) condition statement may indicate a case in which max_temporal_id_plus1 is greater than 1, i.e. a stream using temporal scalability. In this case, a trick play may be provided using a temporal id and, therefore, an embodiment of the present invention may map a temporal id to a tier so as to use conventional PVR_assist_information.
The PVR_assist_tier_pic_num field may indicate a tier number of a picture related to PVR_assist_information. A minimum tier number may be 0 and a maximum tier number may be 7. According to an embodiment of the present invention, for HEVC, a tier number of a HEVC RAP picture may be 0 and a tier number of all other pictures different from the HEVC RAP may be a value obtained by adding 1 to a temporal id. According to an embodiment of the present invention, this field may indicate a tier number of a picture included in a video stream and may be named tier number information. The tier number may be used to signal a temporal sublayer.
The PVR_assist_tier_next_pic_in_tier_present_flag field may have a value of 1 in a case in which a PVR_assist_tier_next_pic_in_tier_field exists.
The PVR_assist_extension_present_flag field may have a value of 1 in a case in which any one selected from among the PVR_assist_segmentation_info_present_flag field, the PVR_assist_tier_m_cumulative_frames_present_flag field, and the PVR_assist_tier_n_mmco_present_flag field.
The PVR_assist_temporal_id_info_present_flag field may include whether information related to a temporal id is included.
The PVR_assist_temporal_sub_layer_dependency_flag field may indicate dependency between temporal sublayers. That is, the temporal_sub_layer_dependency_flag field has a value of 1 in a case in which a lower temporal sublayer picture does not refer to an upper temporal sublayer picture.
An if (PVR_assist_max_temporal_id_plus1>1) condition statement may indicate a case in which PVR_assist_max_temporal_id_plus1 is greater than 1, i.e. a stream using temporal scalability. In this case, a trick play may be provided using a temporal id.
A reception device according to an embodiment of the present invention may include a tuner 38010, a demodulator 38020, a trick play execution unit 38030, a system decoder and demux 38040, and/or a video decoder 38050.
The tuner 38010 may receive a broadcast signal transmitted through a broadcasting network, a cable network, and/or an Internet network.
The system decoder and demux 38040 may decode system information and separate a multiplexed broadcast signal per unit stream. The demux may demultiplex a received broadcast signal to extract a video stream.
The video decoder 38050 may decode a video stream. The video decoder may include the system decoder and the trick play execution unit. The video decoder may execute a trick play of a video stream by decoding a video stream selected for a trick play based on the PVR assist information.
A hierarchical structure of an HEVC temporal sublayer according to an embodiment of the present invention is similar to a tier system framework. A temporal id according to an embodiment of the present invention may be matched to a tier number. An HEVC temporal sublayer according to an embodiment of the present invention may support PVR using a method similar to the tier system framework.
An HEVC compliant encoder/decoder according to an embodiment of the present invention may support an HEVC temporal sublayer. An temporal id according to an embodiment of the present invention may exist in a stream encoded by HEVC. In a case in which a stream is encoded by a temporal sublayer structure, any special encoding structure for a trick play may not be needed. Consequently, an HEVC temporal sublayer according to an embodiment of the present invention may reduce an encoding burden in a case in which a trick play is supported.
As shown in this figure, in a tier framework, tier 7 means non-referenced discardable pictures and tier 6 means referenced discardable pictures. However, an HEVC temporal sublayer according to an embodiment of the present invention may not classify referenced pictures and non-referenced pictures. In the tier framework, tier 1 indicates RAP pictures and tier 2 indicates P pictures. In an HEVC temporal sublayer according to an embodiment of the present invention, however, temporal id 0 may indicate all of I pictures including IRAP pictures, P pictures, and B pictures.
According to an embodiment of the present invention, other sublayers may be designated to temporal ids 6 and 7. As compared with tier numbers 6 and 7, therefore, an additional X speed may be supported in a case in which a temporal id is used according to an embodiment of the present invention.
According to an embodiment of the present invention, it may be necessary to classify frames in a base sublayer having a temporal id of 0 so as to support a higher X speed.
According to an embodiment of the present invention, an intra frame may be signaled in a base sublayer having a temporal id of 0 so as to support a higher X speed.
The PVR_assist_temporal_id_plus1 field may indicate a temporal id of a picture. Actually, a temporal id may have a value obtained by subtracting 1 from a value of this field. This field may have a minimum value of 1 and a maximum value of 7. For an HEVC stream, this field may have a value equal to a value of nuh_temporal_id_plus1.
The PVR_assist_temporal_id_info_present_flag field may have a value of 1 in a case in which the PVR_assist_max_temporal_id_plus1 field exists. This field may be provided only in pictures corresponding to RAP pictures.
The PVR_assist_intra_picture_flag field may have a value of 1 in a case in which a current access unit is an intra picture. An embodiment of the present invention may signal the intra picture using this field, and may provide a trick play at a higher X speed by signaling the intra picture.
The PVR_assist_PB_numbers_in_temporalId_zero field may indicate the number of frames that exist between intra frames and are different from the intra frames in a base sublayer having a temporal id of 0. This field may be used to estimate the speed of a trick play.
A first figure part 41010 of this figure shows that pictures corresponding to one GOP are arranged in presentation order. In the first figure part, I may indicate I pictures, B may indicate B pictures, and P may indicate P pictures. In addition, numerical subscripts of the alphabet indicating picture types may indicate the presentation order. In the first figure part, arrows may indicate reference relationship among pictures.
A second figure part 41020 of this figure shows a method of providing a trick play using an HEVC temporal sublayer according to an embodiment of the present invention. One quadrilateral box may indicate one picture. A number in each quadrilateral box may indicate a temporal id. Symbol x shown in each quadrilateral box may indicate a picture decoded and presented during the execution of a trick play. As shown in the second figure part, X2 to X8 speed trick plays may be provided by the HEVC temporal sublayer.
A third figure part 41030 of this figure shows a method of providing a trick play using base sublayer signaling according to an embodiment of the present invention. An embodiment of the present invention may decode and present only intra pictures to provide X12, X24, and X48 speed trick plays.
An embodiment of the present invention may provide a trick play by matching a tier value with a value obtained by adding 1 to temporal id for a picture that is not an RAP picture and designating 0 as a tier value for a picture that is an RAP picture.
An embodiment of the present invention may transmit a broadcast signal through the following procedures. First, an embodiment of the present invention may encode video data to create a video stream (S42010). The video stream may be encoded using an AVC or HEVC codec. The video stream may include PVR assist information for executing a trick play of video data. The PVR assist information may mean information necessary for a receiver or a PVR device to execute a trick play of video data. The PVR assist information has been previously described in detail with reference to FIG. 34. Next, an embodiment of the present invention may create signaling information (S42020). The signaling information may correspond to information, such as PSI, PSIP, and SI. Next, an embodiment of the present invention may multiplex the video stream and the signaling information into a single broadcast stream (S42030). The video stream may mean a video unit stream, a PES stream having a packetized video unit stream, or a transport stream having a TS packet. At the multiplexing step, an embodiment of the present invention may multiplex the video stream together with an audio stream and a program stream into a single transport stream. Next, an embodiment of the present invention may create a broadcast signal including the multiplexed broadcast stream (S42040). The multiplexed broadcast stream may mean a single multiplexed transport stream. An embodiment of the present invention may modulate the multiplexed broadcast stream using various modulation methods to create a broadcast signal. Next, an embodiment of the present invention may transmit the created broadcast signal (S42050). An embodiment of the present invention may transmit the created broadcast signal through at least one selected from among a terrestrial broadcasting network, a cable network, and an Internet network.
According to another embodiment of the present invention, a video stream may include one or more temporal sublayers, and each temporal sublayer may indicate a group of pictures. A NAL unit header including encoded video data according to an embodiment of the present invention may include temporal identification plus information. The temporal identification plus information may indicate a value obtained by adding 1 to a value of temporal identification information and the temporal identification information may include information for identifying a temporal sublayer. Temporal identification plus information according to an embodiment of the present invention may be used to identify a temporal sublayer. The temporal sublayer may be named a temporal sublayer, the temporal identification information may be named a temporal id, and the temporal identification plus information may be named nuh_temporal_id_plus1.
According to another embodiment of the present invention, the PVR assist information may include tier number information indicating the tier number of pictures included in a video stream. The tier number may be used to signal a temporal sublayer. The tier number may indicate dependency between pictures. For HEVC, the tier number may be used to signal a sublayer in a similar manner to the temporal identification information. The tier number information may be named PVR_assist_tier_pic_num. The tier number information has been previously described in detail with reference to FIGS. 34, 35, and 36.
According to another embodiment of the present invention, the tier number of an HEVC picture, among pictures included in a video stream, may have a value of 0, and the tier number of pictures other than the HEVC RAP picture may have a value obtained by adding 1 to a temporal identification information value, which has been previously described in detail with reference to FIGS. 34, 39, and 40.
According to another embodiment of the present invention, the PVR assist information may include tier next picture information indicating the relative position of the next picture in decoding order among pictures having a tier number equal to a value indicated by the tier number information. The tier next picture information has been previously described in detail with reference to FIG. 34.
A broadcast signal reception device 43010 according to an embodiment of the present invention may include a tuner 43020, a demux 43030, and/or a decoder 43040.
The demux may demultiplex the received broadcast signal to extract a video stream. The video stream may include PVR assist information for executing a trick play of the video data.
The decoder may decode the extracted video stream based on the PVR assist information, and may execute a trick play of video data. The decoder may decode only pictures having a specific temporal identification information value to execute a trick play.
PVR assist information according to an embodiment of the present invention may include tier number information indicating the tier number of pictures included in a video stream. The tier number may indicate dependency between pictures having a temporal identification information value of 0. The decoder may decode only pictures having a specific tier number, among the pictures having a temporal identification information value of 0, to execute a high X speed trick play. The decoder may decode pictures having a temporal identification information value of 0 and pictures having a specific temporal identification information value of more than 0 to execute a low X speed trick play.
A broadcast signal reception device according to an embodiment of the present invention may further include a parser for parsing temporal identification plus information and transmitting only pictures having a specific temporal identification information value to the decoder. The decoder may decode only the pictures received from the parser to execute a trick play. The parser may perform the same function as the trick play execution unit shown in FIG. 28.
PVR assist information according to an embodiment of the present invention may include information indicating the number of frames that exist between intra frames and are different from the intra frames in a temporal sublayer having a temporal identification information value of 0.
Components of a broadcast signal reception device according to an embodiment of the present invention shown in this figure having the same names as those of the broadcast signal reception device shown in FIG. 38 may perform the same functions as those of the broadcast signal reception device shown in FIG. 38.
Components of a broadcast signal reception device according to an embodiment of the present invention shown in this figure corresponding to the procedures of the broadcast signal transmission method shown in FIG. 42 may perform functions corresponding to the procedures of the broadcast signal transmission method shown in FIG. 42.
Previous Patent US 10,057,611 | Next Patent US 10,057,613