Patent Description:
In the past, for example, in broadcasting such as DVB (Digital Video Broadcasting), the operation in which subtitle graphics data is converted to bitmap data and then transmitted has been performed (see, for example, Patent Literature <NUM>). In this approach, a "Rendering_guide_segment" indicating the necessity for converting a color gamut and a luminance is transmitted (see, for example, <NPL>).

In the past, when video data and subtitle graphics data are transmitted in different streams, there has not been a noticeable difference between them regarding a color gamut and a luminance. Therefore, at the time of superimposition, synthesizing is performed without special consideration.

For example, in a case where the color gamut of the video data is wide (e.g., in conformity to ITU-R Rec Bt. <NUM>) and the color gamut of the subtitle graphics data is narrow (e.g., sRGB), in order to keep the image quality of the video high, it becomes necessary to adjust the color gamut of the subtitle graphics data to the color gamut of the video data before performing superimposition.

Further, for example, in a case where the video data is produced with a high dynamic range (HDR) and the subtitle graphics data is produced with a dynamic range of a standard level (SDR), in order to keep the image quality of the video high, it becomes necessary to match the subtitle graphics data with the dynamic range region of the video data before performing superimposition.

It is an object of the present technology to alleviate a processing load on a reception side when subtitle graphics data is superimposed on video data.

A concept of the present technology resides in a transmission apparatus including: a video encoder section that generates a video stream including video data; a subtitle encoder section that generates a subtitle stream including bitmap data, the bitmap data being obtained by converting subtitle graphics data; and a transmission section that transmits a container having a predetermined format containing the video stream and the subtitle stream, the subtitle stream including a bitmap conversion table containing conversion information of a color gamut and/or a luminance.

In the present technology, a video stream including video data is generated by a video encoder section. A subtitle stream including bitmap data is generated by a subtitle encoder section, the bitmap data being obtained by converting subtitle graphics data. A container having a predetermined format containing the video stream and the subtitle stream is transmitted by a transmission section.

The subtitle stream is adapted to include a bitmap conversion table containing conversion information of a color gamut and/or a luminance. For example, the subtitle stream may be adapted to include a plurality of bitmap conversion tables each containing the conversion information different from one another. In this case, for example, each of the plurality of bitmap conversion tables may be adapted to additionally contain matching characteristic information of the subtitle graphics data and target video data.

In such a manner, in the present technology, the subtitle stream includes a bitmap conversion table containing conversion information of a color gamut and/or a luminance. Therefore, the reception side can easily obtain subtitle graphics data having characteristics matched with those of target video data of a superimposition destination by just converting bitmap data to subtitle graphics data by using the bitmap conversion table. This enables a processing load on the reception side to be alleviated.

It should be noted that in the present technology, for example, the transmission apparatus may further include an information insertion section that inserts characteristic information of the subtitle graphics data before conversion to the bitmap data into the subtitle stream. Thus, on the reception side, the characteristic of the subtitle graphics data before conversion to the bitmap data can be easily grasped.

Further, another concept of the present technology resides in a reception apparatus including: a reception section that receives a container having a predetermined format containing a video stream and a subtitle stream, the video stream including video data, the subtitle stream including bitmap data obtained by converting subtitle graphics data, the subtitle stream including a bitmap conversion table containing conversion information of a color gamut and/or a luminance; and a control section that controls processing of decoding the video stream to obtain target video data, processing of decoding the subtitle stream to obtain the bitmap data and the bitmap conversion table, processing of converting the bitmap data to the subtitle graphics data by using the bitmap conversion table, and processing of superimposing the subtitle graphics data on the target video data.

In the present technology, a container having a predetermined format containing a video stream and a subtitle stream is received by a reception section, the video stream including video data, the subtitle stream including bitmap data obtained by converting subtitle graphics data. The subtitle stream includes a bitmap conversion table containing conversion information of a color gamut and/or a luminance.

Processing of decoding the video stream to obtain target video data and processing of decoding the subtitle stream to obtain the bitmap data and the bitmap conversion table are performed. Processing of converting the bitmap data to the subtitle graphics data by using the bitmap conversion table and processing of superimposing the subtitle graphics data on the target video data are performed.

For example, the subtitle stream may be adapted to include a plurality of bitmap conversion tables each containing the conversion information different from one another, and in the processing of converting the bitmap data to the subtitle graphics data, out of the plurality of bitmap conversion tables, a bitmap conversion table matched with characteristics of the subtitle graphics data before conversion to the bitmap data and the target video data may be adapted to be used.

In this case, for example, characteristic information of the subtitle graphics data before conversion to the bitmap data may be adapted to be inserted into the subtitle stream, and in the processing of converting the bitmap data to the subtitle graphics data, the characteristic of the subtitle graphics data before conversion to the bitmap data may be adapted to be determined on the basis of the characteristic information inserted into the subtitle stream.

In such a manner, in the present technology, in the processing of converting the bitmap data to the subtitle graphics data, a bitmap conversion table containing conversion information of a color gamut and/or a luminance, which is inserted into the subtitle stream, is used. Therefore, subtitle graphics data having characteristics matched with those of target video data of a superimposition destination can be easily obtained. This enables a processing load to be alleviated.

It should be noted that in the present technology, for example, in the processing of converting the bitmap data to the subtitle graphics data, when the bitmap conversion table is not matched with characteristics of the subtitle graphics data before conversion to the bitmap data and the target video data, the bitmap data may be adapted to be converted to the subtitle graphics data to be matched with the characteristics without using the bitmap conversion table in the processing of the reception apparatus. In this case, subtitle graphics data having characteristics matched with those of target video data of a superimposition destination can be obtained, and thus the lowering of the image quality of a video can be suppressed.

According to the present technology, it is possible to alleviate a processing load on a reception side when subtitle graphics data is superimposed on video data. It should be noted that the effects described in this specification are merely examples and are not limitative, and additional effects may be provided.

Hereinafter, a mode for carrying out the present invention (hereinafter, referred to as "embodiment") will be described. It should be noted that description will be given in the following order.

<FIG> shows a configuration example of a transmission and reception system <NUM> as an embodiment. This transmission and reception system <NUM> includes a transmission apparatus <NUM> and a reception apparatus <NUM>.

The transmission apparatus <NUM> generates an MPEG2 transport stream TS as a container and transmits the transport stream TS through broadcasting waves or network packets. This transport stream TS contains a video stream including video data (image data).

Further, this transport stream TS contains a subtitle stream including bitmap data, which is obtained by converting subtitle graphics data as subtitle data. This subtitle stream is considered as including a bitmap conversion table containing a color gamut and/or a luminance, in this embodiment, conversion information of both of them.

The subtitle stream is considered as including a predetermined number of bitmap conversion tables, i.e., one or two or more bitmap conversion tables. In such a case, each bitmap conversion table is additionally provided with matching characteristic information of subtitle graphics data and target video data. Here, the characteristic(s) is/are a color gamut and/or a dynamic range, in this embodiment, the characteristics of both of them. Thus, on the reception side, it is possible to easily determine which bitmap conversion table is to be used.

Further, in this embodiment, the characteristic information of subtitle graphics data before conversion to bitmap data is inserted into the subtitle stream. Thus, on the reception side, the characteristic of subtitle graphics data before conversion to bitmap data can be easily grasped.

The reception apparatus <NUM> receives the transport stream TS transmitted from the transmission apparatus <NUM>. The reception apparatus <NUM> performs decoding processing on the video stream and obtains target video data. In this case, for the target video data, not only the video data transmitted from the transmission side but also video data obtained by performing conversion processing of a color gamut or a dynamic range thereon as appropriate are conceived.

Further, the reception apparatus <NUM> performs decoding processing on the subtitle stream, obtains bitmap data and a bitmap conversion table, and converts the bitmap data to subtitle graphics data by using the bitmap conversion table. Here, in a case where a plurality of bitmap conversion tables each containing different conversion information are obtained, a bitmap conversion table matched with the characteristics of the subtitle graphics data before conversion to bitmap data and the target video data is used. In this case, the characteristic of the subtitle graphics data before conversion to bitmap data is grasped from, for example, the characteristic information inserted into the subtitle stream.

The reception apparatus <NUM> then superimposes the subtitle graphics data on the target video data, the subtitle graphics data being obtained by converting bitmap data. With the video data on which the subtitle graphics data is superimposed in such a manner, an image with subtitles is displayed on the monitor.

<FIG> shows a configuration example of the transmission apparatus <NUM>. This transmission apparatus <NUM> includes a control section <NUM>, a video encoder <NUM>, a conversion section <NUM>, a subtitle encoder <NUM>, a system encoder <NUM>, and a transmission section <NUM>.

The control section <NUM> includes a CPU (Central Processing Unit) and controls operations of the respective sections of the transmission apparatus <NUM> on the basis of a control program. The video encoder <NUM> performs encoding such as MPEG4-AVC or HEVC on the video data and generates a video stream (PES stream) VS containing the encoded video data.

Here, the video data is SDR (Standard Dynamic Range) or HDR (High Dynamic Range) data and is provided with an SDR or HDR opto-electronic conversion characteristic. The video encoder <NUM> inserts characteristic information of the video data, i.e., meta-information such as information of a color gamut or a dynamic range, into a region of VUI (video usability information) of an SPS NAL unit of an access unit (AU).

The conversion section <NUM> converts the subtitle graphics data to bitmap data. Here, the subtitle graphics data is SDR or HDR data and is provided with an SDR or HDR opto-electronic conversion characteristic. The conversion section <NUM> refers to an HDR or SDR level, which is assumed to be necessary on the reception side, to thus output information of one or two or more bitmap conversion tables together with the bitmap data.

Here, the bitmap conversion table is a bitmap conversion table containing conversion information of a color gamut and/or a luminance. In other words, the bitmap conversion table is not for simply converting the bitmap data into the subtitle graphics data but for converting a color gamut or a luminance together so as to be adjusted to a color gamut or a dynamic range of the target video data that is video data of a superimposition destination.

<FIG> shows a conversion target example other than the conversion from the bitmap data to the subtitle graphics data (Y/CbCr) in the bitmap conversion table. In a case where the subtitle graphics data before conversion to the bitmap data is SDR and the target video data is SDR, the color gamut can be another conversion target. For example, when the color gamut of the subtitle graphics data is BT. <NUM> and the color gamut of the target video data is BT. <NUM>, the color gamut becomes a conversion target.

Further, in a case where the subtitle graphics data before conversion to the bitmap data is SDR and the target video data is HDR, the color gamut and the luminance can be conversion targets. Further, in a case where the subtitle graphics data before conversion to the bitmap data is HDR and the target video data is SDR, the color gamut and the luminance can be conversion targets. It should be noted that in a case where the subtitle graphics data before conversion to the bitmap data is HDR and the target video data is HDR, the color gamut and the dynamic range are common, and thus there is nothing that can be a conversion target.

In this embodiment, the conversion section <NUM> outputs information of the bitmap conversion table used in each of the cases (<NUM>) to (<NUM>) of <FIG>. The case (<NUM>) is a case where the subtitle graphics data before conversion to the bitmap data is SDR and the target video data is SDR (a case where the color gamut is identical). The case (<NUM>) is a case where the subtitle graphics data before conversion to the bitmap data is SDR and the target video data is SDR (a case where the color gamut is different).

The case (<NUM>) is a case where the subtitle graphics data before conversion to the bitmap data is SDR and the target video data is HDR. The case (<NUM>) is a case where the subtitle graphics data before conversion to the bitmap data is HDR and the target video data is SDR. The case (<NUM>) is a case where the subtitle graphics data before conversion to the bitmap data is HDR and the target video data is HDR.

The conversion functions, which are necessary on the reception side, and thus the conversion functions of the bitmap conversion table in the cases (<NUM>) to (<NUM>) will be described in detail with reference to <FIG>. The conversion functions necessary at maximum on the reception side include the following first to seventh conversion functions. The processing in those conversion functions is basically independent on a pixel by pixel basis.

A first conversion function <NUM> is a function of converting bitmap data to subtitle graphics data. A second conversion function <NUM> is a function of converting a domain of the subtitle graphics data, which is converted from the bitmap data, from YCbCr to RGB1. A third conversion function <NUM> is a function of performing electro-optical conversion by applying an electro-optical conversion characteristic so as to change the subtitle graphics data, which is converted from the bitmap data, to a luminance linear space.

A fourth conversion function <NUM> is a function of converting a luminance level in order to eliminate inconvenience due to a difference in dynamic range between the subtitle graphics data converted from the bitmap data and the target video data. A fifth conversion function <NUM> is a function of performing color gamut conversion (RGB1 to RGB2) for adjusting the color gamut of the subtitle graphics data converted from the bitmap data to the color gamut of the target video data.

A sixth conversion function <NUM> is a function of performing opto-electronic conversion by applying an opto-electronic conversion characteristic, which is identical to that of the target video data, to the subtitle graphics data in the luminance linear space. A seventh conversion function <NUM> is a function of converting a domain of the subtitle graphics data from RGB2 to YCbCr.

The case (<NUM>) needs the first conversion function <NUM> only. In this case, the bitmap data is converted to the subtitle graphics data by the first conversion function <NUM>, and this graphics data becomes output graphics data as it is. In this case, each of the subtitle graphics data before the conversion to the bitmap data and the target video data is SDR, and their color gamut are identical, and thus the second conversion function <NUM> to the seventh conversion function <NUM> are bypassed. This conversion processing is exactly identical to the processing performed in the broadcasting as a traditional legacy.

The case (<NUM>) needs the first conversion function <NUM>, the second conversion function <NUM>, the fifth conversion function <NUM>, and the seventh conversion function <NUM>. In this case, the bitmap data is converted to the subtitle graphics data by the first conversion function <NUM>. This subtitle graphics data is converted from a YCbCr domain to an RGB1 domain by the second conversion function <NUM>.

The color gamut of the subtitle graphics data that is converted to the RGB1 domain is converted so as to be adjusted to the color gamut of the target video data by the fifth conversion function <NUM>. For example, the color gamut of BT. <NUM> of the subtitle graphics data is converted so as to be adjusted to BT. <NUM> as the color gamut of the target video data.

The subtitle graphics data whose color gamut has been converted is converted from an RGB2 domain to the YCbCr by the seventh conversion function <NUM>, and this graphics data becomes output graphics data. In this case, each of the subtitle graphics data before the conversion to the bitmap data and the target video data is SDR, and thus the third conversion function <NUM>, the fourth conversion function <NUM>, and the sixth conversion function <NUM> are bypassed.

The case (<NUM>) needs all of the first conversion function <NUM> to the seventh conversion function <NUM>. In this case, the bitmap data is converted to the subtitle graphics data by the first conversion function <NUM>. This subtitle graphics data is converted from the YCbCr domain to the RGB1 domain by the second conversion function <NUM>.

The subtitle graphics data converted to the RGB1 domain is subjected to electro-optical conversion by applying an SDR electro-optical conversion characteristic by the third conversion function <NUM>, to thus be changed to a luminance linear space. The subtitle graphics data changed to the luminance linear space is subjected to luminance level conversion by the fourth conversion function <NUM>. In this case, conversion is performed such that a predetermined SDR luminance level becomes an HDR mapping reference level.

<FIG> shows a state of this luminance level conversion. In <FIG>, the solid line a represents an SDR conversion curve. The solid line b represents an HDR conversion curve. The broken line c represents an SDR data range that is mapped on the HDR conversion curve.

In this case, a luminance value "m" of SDR subtitle graphics data is treated as a mapping reference value that coincides with a luminance value "m" of HDR video data. Assuming that an encoding code value representing the luminance value "m" of the SDR subtitle graphics data is Q% and an encoding code value representing the luminance value "m" of the HDR video data is P%, the SDR subtitle graphics data is converted such that a digital code representing Q% coincides with a digital code representing P%.

In such a manner, [<NUM>··a] of the SDR subtitle graphics data falls within the range of [<NUM>··a'] of the HDR video data. Thus, the luminance of subtitles is prevented from being too bright. It should be noted that, in the figure, in both of the SDR and the HDR, an encoding bit space N is the same. Further, <NUM><P≤<NUM>, and <NUM><Q≤<NUM>.

As described above, the subtitle graphics data, which has been subjected to the luminance level conversion by the fourth conversion function <NUM>, is converted so as to be adjusted to the color gamut of the target video data by the fifth conversion function <NUM>. For example, the color gamut of BT. <NUM> of the subtitle graphics data is converted so as to be adjusted to BT. <NUM> as the color gamut of the target video data.

The subtitle graphics data whose color gamut has been converted is subjected to opto-electronic conversion by applying an HDR opto-electronic conversion characteristic by the sixth conversion function <NUM>. Thus, the subtitle graphics data is provided with the HDR opto-electronic conversion characteristic, like the target video data. This subtitle graphics data is converted from the RGB2 domain to the YCbCr by the seventh conversion function <NUM>, and this graphics data becomes output graphics data.

The case (<NUM>) needs all of the first conversion function <NUM> to the seventh conversion function <NUM> as in the case (<NUM>) described above. In this case, the bitmap data is converted to the subtitle graphics data by the first conversion function <NUM>. This subtitle graphics data is converted from the YCbCr domain to the RGB1 domain by the second conversion function <NUM>.

The subtitle graphics data converted to the RGB1 domain is subjected to electro-optical conversion by applying an HDR electro-optical conversion characteristic by the third conversion function <NUM>, to thus be changed to a luminance linear space. The subtitle graphics data changed to the luminance linear space is subjected to luminance level conversion by the fourth conversion function <NUM>. In this case, conversion is performed such that a predetermined HDR luminance level becomes an SDR mapping reference level.

<FIG> shows a state of this luminance level conversion. In <FIG>, the solid line a represents an SDR conversion curve. The solid line b represents an HDR conversion curve. The broken line c represents a conversion curve aimed at mapping the HDR conversion curve on the SDR.

In this case, a luminance value "m" of HDR subtitle graphics data is treated as a mapping reference value that coincides with a luminance value "m" of SDR video data. Assuming that an encoding code value representing the luminance value "m" of the HDR subtitle graphics data is P% and an encoding code value representing the luminance value "m" of the SDR video data is Q%, the HDR subtitle graphics data is converted with a conversion curve having tone-map characteristics like the broken line c such that a digital code representing P% coincides with a digital code representing Q%.

In such a manner, [<NUM>··b] of the HDR subtitle graphics data becomes [<NUM>··a'] and falls within the range of [<NUM>··a] of the SDR video data without performing clipping. It should be noted that, in the figure, in both of the SDR and the HDR, the encoding bit space N is the same. Further, <NUM><P≤<NUM>, and <NUM><Q≤<NUM>.

The subtitle graphics data whose color gamut has been converted is subjected to opto-electronic conversion by applying an SDR opto-electronic conversion characteristic by the sixth conversion function <NUM>. Thus, the subtitle graphics data is provided with the SDR opto-electronic conversion characteristic, like the target video data. This subtitle graphics data is converted from the RGB2 domain to the YCbCr by the seventh conversion function <NUM>, and this graphics data becomes output graphics data.

The case (<NUM>) needs the first conversion function <NUM> only. In this case, the bitmap data is converted to the subtitle graphics data by the first conversion function <NUM>, and this graphics data becomes output graphics data as it is. In this case, each of the subtitle graphics data before the conversion to the bitmap data and the target video data is HDR, and their color gamut are identical, and thus the second conversion function <NUM> to the seventh conversion function <NUM> are bypassed.

It should be noted that the example of the figure shows a case where the HDR characteristics of the subtitle graphics data before the conversion to the bitmap data and the target video data are the same. In a case where the HDR characteristics of both of the data are different from each other, the luminance level conversion also becomes necessary as in the cases (<NUM>) and (<NUM>). The case where the HDR characteristics of both of the data are different from each other is, for example, a case where the HDR characteristic of the subtitle graphics data is PQ and the HDR characteristic of the video data is HLG.

Referring to <FIG>, the subtitle encoder <NUM> converts display control information and the bitmap data output from the conversion section <NUM> to various segments and generates a subtitle stream SS including PES packets in which those segments are disposed in payloads thereof. The various segments include, in addition to the conventionally well-known segments such as DDS, PCS, RCS, ODS, and EDS, a WC segment and a rendering guide segment (Rendering_guide_segment) that are newly defined.

The WC segment contains information of a predetermined number of bitmap conversion tables output from the conversion section <NUM>. Further, the rendering guide segment contains characteristic information of subtitle graphics data before conversion to bitmap data, i.e., meta-information such as information of the color gamut or the dynamic range.

<FIG> and <FIG> show a structural example (Syntax) of the WC segment, and <FIG> shows contents (Semantics) of main information in that structural example. An <NUM>-bit field of "CLUT_entry_id" represents an ID of an individual CLUT (bitmap conversion table). A <NUM>-bit field of "entry_CLUT_type" represents the type of entry-CLUT, i.e., variation of a display color and corresponds to the range of input bitmap data. For example, "0x1" represents <NUM> bit entry, "0x2" represents <NUM> bit entry, and "0x4" represents <NUM> bit entry.

A <NUM>-bit field of "output_depth" represents a bit depth of each element of YCrCb and T as the output graphics data. For example, "<NUM>" represents <NUM> bits, "<NUM>" represents <NUM> bits, and "<NUM>" represents <NUM> bits.

A <NUM>-bit field of "subtitle_dynamic_range_type" represents information of a dynamic range that is matched as a dynamic range of the subtitle graphics data before the conversion to the bitmap data. For example, "<NUM>" represents "SDR by gamma", "<NUM>" represents "HDR by PQ", and "<NUM>" represents "HDR by HLG". A <NUM>-bit field of "subtitle_color_gamut" represents information of a color gamut that is matched as a color gamut of the subtitle graphics data before the conversion to the bitmap data. For example, "<NUM>" represents "BT. <NUM>", and "<NUM>" represents "BT.

A <NUM>-bit field of the "target_video_dynamic_range_type" represents information of a dynamic range that is matched as a dynamic range of the target video data. For example, "<NUM>" represents "SDR by gamma", "<NUM>" represents "HDR by PQ", and "<NUM>" represents "HDR by HLG". A <NUM>-bit field of "target video color_gamut" represents information of a color gamut that is matched as a color gamut of the target video data. For example, "<NUM>" represents "BT. <NUM>", and "<NUM>" represents "BT.

In this WC segment, the fields of "CLUT entry_id", "entry_CLUT_type", "output_depth", "subtitle_dynamic_range_type", "subtitle_color_gamut", "target_video_dynamic_range_type", and "target video color_gamut" are present in each bitmap conversion table, and the elements of YCrCb and T, which are conversion data corresponding to possible values of the input bitmap data, are present following those fields.

<FIG> shows examples of the values of "CLUT_entry_id", "entry_CLUT_type", and "output_depth", and values of matching characteristic information ("subtitle_dynamic_range_type", "subtitle color_gamut", "target_video_dynamic_range_type", "target video color_gamut") in a list.

In the example shown in the figure, the value of "entry_CLUT_type" is set to "0x4", which represents <NUM> bit entry, i.e., indicates that the range of the input bitmap data is <NUM> bits. Further, in the example shown in the figure, the value of "output_depth" is set to "<NUM>", which indicates that the bit depth of the elements of YCrCb and T as output graphics data is <NUM> bits.

Further, in the example shown in the figure, the matching characteristic information ("subtitle_dynamic_range_type", "subtitle color_gamut", "target_video_dynamic_range_type", "target video color_gamut") of a bitmap conversion table (CLUT) having "CLUT_entry_id" of "<NUM>" is (<NUM>, <NUM>, <NUM>, <NUM>), which represents a bitmap conversion table corresponding to the case (<NUM>) described above.

In other words, this bitmap conversion table (CLUT) can be used in a case where the dynamic range of the subtitle graphics data before the conversion to the bitmap data is "SDR by gamma" and the color gamut is "BT. <NUM>", and also where the dynamic range of the target video data is "SDR by gamma" and the color gamut is "BT.

Further, in the example shown in the figure, the matching characteristic information ("subtitle_dynamic_range_type", "subtitle_color_gamut", "target_video_dynamic_range_type", "target video color_gamut") of a bitmap conversion table (CLUT) having "CLUT_entry_id" of "<NUM>" is (<NUM>, <NUM>, <NUM>, <NUM>), which represents a bitmap conversion table corresponding to the case (<NUM>) described above.

Further, in the example shown in the figure, the matching characteristic information ("subtitle_dynamic range type", "subtitle_color_gamut", "target_video_dynamic_range_type", "target video color_gamut") of a bitmap conversion table (CLUT) having "CLUT_entry_id" of "<NUM>" is (<NUM>, <NUM>, <NUM>, <NUM>), which represents a bitmap conversion table corresponding to the case (<NUM>) described above.

In other words, this bitmap conversion table (CLUT) can be used in a case where the dynamic range of the subtitle graphics data before the conversion to the bitmap data is "SDR by gamma" and the color gamut is "BT. <NUM>", and also where the dynamic range of the target video data is "HDR by HLG" and the color gamut is "BT.

In other words, this bitmap conversion table (CLUT) can be used in a case where the dynamic range of the subtitle graphics data before the conversion to the bitmap data is "HDR by PQ" and the color gamut is "BT. <NUM>", and also where the dynamic range of the target video data is "SDR by gamma" and the color gamut is "BT.

Further, in the example shown in the figure, the matching characteristic information ("subtitle_dynamic_range_type", "subtitle color_gamut", "target_video_dynamic_range_type", "target video color_gamut") of a bitmap conversion table (CLUT) having "CLUT_entry_id" of "<NUM>" is (<NUM>, <NUM>, <NUM>, <NUM>), which represents a bitmap conversion table corresponding to the case (<NUM>) described above (the case where the HDR characteristics are different).

In other words, this bitmap conversion table (CLUT) can be used in a case where the dynamic range of the subtitle graphics data before the conversion to the bitmap data is "HDR by PQ" and the color gamut is "BT. <NUM>", and also where the dynamic range of the target video data is "HDR by HLG" and the color gamut is "BT.

<FIG> shows a structural example (Syntax) of the rendering guide segment, and <FIG> shows contents (Semantics) of main information in that structural example. A <NUM>-bit field of "rendering_conversion_guide" represents a need to adjust the color gamut or dynamic range to the video in the rendering where the subtitle graphics data is superimposed on the target video data. For example, "<NUM>" indicates that conversion is made possible by selecting a matched one from the plurality of bitmap conversion tables, and "<NUM>" represents others.

An <NUM>-bit field of "bitmap data color_gamut information" is information representing the color gamut of the subtitle graphics data before the conversion to the bitmap data and having the semantics defined by the HEVC standard. An <NUM>-bit field of "bitmap_data_dynamic range information" is information representing the dynamic range of the subtitle graphics data before the conversion to the bitmap data and having the semantics defined by the HEVC standard.

Referring back to <FIG>, the system encoder <NUM> generates a transport stream TS containing a video stream VS generated in the video encoder <NUM> and a subtitle stream SS generated in the subtitle encoder <NUM>. The transmission section <NUM> transmits this transport stream TS to the reception apparatus <NUM> through broadcasting waves or network packets.

<FIG> shows a configuration example of the transport stream TS. In this configuration example, "Video PES1" is present, "Video PES1" being a PES packet of the video stream and identified with PID1. Further, in this configuration example, "Subtitle PES2" is present, "Subtitle PES2" being a PES packet of the subtitle stream and identified with PID2.

The PES packet includes a PES header and a PES payload. In the PES packet of the video stream, a video coded stream is inserted into the PES payload. Color gamut identification information (color_primaries) and dynamic range information (transfer_characteristics) of transmitted video data are inserted into a region of VUI of an SPS NAL unit of an access unit. Further, in addition to the conventionally well-known segments such as DDS, PCS, RCS, ODS, and EDS, a WC segment (WCS) and a rendering guide segment (RGS) that are newly defined are inserted into the PES packet of the subtitle stream.

Further, the transport stream TS contains a PMT (Program Map Table) as PSI (Program Specific Information). The PSI is information describing to which program each elementary stream contained in the transport stream belongs. A program loop (Program loop) describing information related to the entire program is present in the PMT.

Further, in the PMT, an elementary stream loop including information related to each elementary stream is present. In this configuration example, a video elementary stream loop (video ES loop) corresponding to the video stream, and a subtitle elementary stream loop (Subtitle ES loop) corresponding to the subtitle stream are present.

In the video elementary stream loop (video ES loop), in order to correspond to the video stream, information such as a stream type and a PID (packet identifier) is disposed, and a descriptor describing information related to that video stream is also disposed. It is assumed that a value of "Stream_type" of the video stream is set to a value representing an HEVC video stream, for example, and that the PID information represents PID1 imparted to a PES packet "video PES1" of the video stream.

In the subtitle elementary stream loop (Subtitle ES loop), in order to correspond to the subtitle stream, information such as a stream type and a PID (packet identifier) is disposed, and a descriptor describing information related to that subtitle stream is also disposed. It is assumed that a value of "Stream_type" of the subtitle stream is set to a value representing a private stream, for example, and that the PID information represents PID2 imparted to a PES packet "Subtitle PES2" of the subtitle stream.

Operations of the transmission apparatus <NUM> shown in <FIG> will be simply described. The video data is supplied to the video encoder <NUM>. This video data is SDR or HDR data and is provided with an SDR or HDR opto-electronic conversion characteristic.

In the video encoder <NUM>, encoding such as MPEG4-AVC or HEVC is performed on the video data, and a video stream (PES stream) VS containing the encoded video data is generated. At that time, meta-information such as information (transfer_function) representing electro-optical conversion characteristics corresponding to the opto-electronic conversion characteristics of the video data or information (color_primaries) representing the color gamut of the video data is inserted into a region of VUI of an SPS NAL unit of an access unit (AU).

Further, the subtitle graphics data is supplied to the conversion section <NUM>. This subtitle graphics data is SDR or HDR data and is provided with an SDR or HDR opto-electronic conversion characteristic. In the conversion section <NUM>, the subtitle graphics data is converted to bitmap data. Together with the bitmap data, information of one or two or more bitmap conversion tables, which is assumed to be necessary on the reception side, is output from the conversion section <NUM>.

Here, the bitmap conversion table is a bitmap conversion table containing conversion information of a color gamut and/or a luminance. In other words, this bitmap conversion table is not for simply converting the bitmap data into the subtitle graphics data but for converting a color gamut or a luminance together so as to be matched with the target video data that is video data of a superimposition destination.

The bitmap data and the bitmap conversion table information, which has been output from the conversion section <NUM>, are supplied to the subtitle encoder <NUM>. In the subtitle encoder <NUM>, the bitmap data and the display control information are converted to various segments, and a subtitle stream SS including PES packets in which those segments are disposed in payloads thereof is generated.

The various segments include, in addition to the conventionally well-known segments such as DDS, PCS, RCS, ODS, and EDS, a WC segment (see <FIG>) and a rendering guide segment (see <FIG>) that are newly defined. The WC segment contains information of the bitmap conversion table. Further, the rendering guide segment contains characteristic information of the subtitle graphics data before the conversion to the bitmap data, i.e., meta-information such as information of the color gamut or the dynamic range.

The video stream VS generated in the video encoder <NUM> is supplied to the system encoder <NUM>. The subtitle stream SS generated in the subtitle encoder <NUM> is supplied to the system encoder <NUM>. In the system encoder <NUM>, the transport stream TS containing the video stream VS and the subtitle stream SS is generated. This transport stream TS is transmitted to the reception apparatus <NUM> by the transmission section <NUM> through broadcasting waves or network packets.

<FIG> shows a configuration example of the reception apparatus <NUM>. The reception apparatus <NUM> includes a control section <NUM>, a reception section <NUM>, a system decoder <NUM>, a video decoder <NUM>, a subtitle decoder <NUM>, a conversion section <NUM>, and a video superimposition section <NUM>. Further, the reception apparatus <NUM> includes a YCbCr/RGB conversion section <NUM>, an electro-optical conversion section <NUM>, a display mapping section <NUM>, and a CE monitor <NUM>.

The control section <NUM> includes a CPU (Central Processing Unit) and controls operations of respective sections of the reception apparatus <NUM> on the basis of a control program. The reception section <NUM> receives the transport stream TS, which is transmitted from the transmission apparatus <NUM> through broadcasting waves or network packets. The system decoder <NUM> extracts the video stream VS and the subtitle stream SS from the transport stream TS.

The video decoder <NUM> performs decoding processing on the video stream VS extracted in the system decoder <NUM> and obtains target video data. In this case, for the target video data, not only the video data transmitted from the transmission side but also video data obtained by performing conversion processing of a color gamut or a dynamic range thereon as appropriate are conceived. Further, the video decoder <NUM> extracts a parameter set and an SEI message inserted into each access unit forming the video stream VS, and transmits them to the control section <NUM>.

The subtitle decoder <NUM> performs decoding processing on the subtitle stream SS extracted in the system decoder <NUM> and obtains the bitmap data and information of a predetermined number of, i.e., one or two or more bitmap conversion tables. Here, the information of a predetermined number of bitmap conversion tables is obtained from the WC segment (see <FIG>) described above. Further, at that time, characteristic information of the subtitle graphics data before the conversion to the bitmap data, i.e., meta-information such as information of a color gamut or a dynamic range is extracted from the rendering guide segment (see <FIG>) and transmitted to the control section <NUM>.

The conversion section <NUM> converts the bitmap data to subtitle graphics data by using a matched bitmap conversion table out of the predetermined number of bitmap conversion tables. In this case, the conversion section <NUM> determines a matched bitmap conversion table on the basis of the matching characteristic information ("subtitle_dynamic_range_type", "subtitle color_gamut", "target_video_dynamic_range_type", "target video color_gamut") (see <FIG>), which is added to each bitmap conversion table.

The conversion section <NUM> considers the following bitmap conversion table as a matched one, in which a dynamic range and a color gamut represented by "subtitle_dynamic_range_type" and "subtitle_color_gamut" coincide with a dynamic range and a color gamut of the subtitle graphics data before the conversion to the bitmap data, and a dynamic range and a color gamut represented by "target_video_dynamic_range_type" and "target video color_gamut" coincide with a dynamic range and a color gamut of the target video data.

In this case, the dynamic range and the color gamut of the subtitle graphics data before the conversion to the bitmap data are grasped from the characteristic information (the information of a color gamut or a dynamic range) of the subtitle graphics data before the conversion to the bitmap data, the characteristic information being extracted from the rendering guide segment (see <FIG>), as described above.

<FIG> shows a selection example of the bitmap conversion table. The example shown in the figure is an example of a case where the WC segment contains information of five bitmap conversion tables (GLUT <NUM> to <NUM>) respectively having "CLUT entry_id" of <NUM> to <NUM>, which are shown in the example of <FIG> described above. Further, the example shown in the figure is an example in a case where a dynamic range and a color gamut of the subtitle graphics data before the conversion to the bitmap data are respectively "SDR by gamma" and "BT. <NUM>", and a dynamic range and a color gamut of the target video data are respectively "HDR by HLG" and "BT. In this case, as shown in the figure, the CLUT <NUM> is selected as a matched bitmap conversion table.

It should be noted that the conversion section <NUM> is assumed to have the first conversion function <NUM> to the seventh conversion function <NUM> shown in <FIG> described above. When there is no matched one in the bitmap conversion tables contained in the WC segment, the conversion section <NUM> generates from the bitmap data the subtitle graphics data having characteristics matched with those of the target video data of a superimposition destination by using a necessary conversion function out of the first conversion function <NUM> to the seventh conversion function <NUM>, without using the bitmap conversion tables.

Referring back to <FIG>, the video superimposition section <NUM> superimposes the subtitle graphics data output from the conversion section <NUM> on the target video data output from the video decoder <NUM>. The YCbCr/RGB conversion section <NUM> converts a video data V1', on which the subtitle graphics data is superimposed, from the YCbCr (luminance/color difference) domain to the RGB domain. In this case, the YCbCr/RGB conversion section <NUM> performs conversion on the basis of the color gamut identification information by using a conversion formula corresponding to the color gamut.

The electro-optical conversion section <NUM> performs electro-optical conversion on the video data V1' converted to the RGB domain by applying an electro-optical conversion characteristic, which corresponds to the opto-electronic conversion characteristic applied to the video data V1', and obtains display video data for displaying images. The display mapping section <NUM> performs display luminance adjustment on the display video data, the display luminance adjustment corresponding to the maximum luminance display capability of the CE monitor <NUM>, or the like. The CE monitor <NUM> displays images on the basis of the display video data on which the display luminance adjustment is performed. The CE monitor <NUM> includes, for example, an LCD (Liquid Crystal Display) or an organic EL display (organic electroluminescence display).

Operations of the reception apparatus <NUM> shown in <FIG> will be simply described. In the reception section <NUM>, the transport stream TS transmitted from the transmission apparatus <NUM> through broadcasting waves or network packets is received. The transport stream TS is supplied to the system decoder <NUM>. In the system decoder <NUM>, a video stream VS and a subtitle stream SS are extracted from the transport stream TS.

The video stream VS extracted in the system decoder <NUM> is supplied to the video decoder <NUM>. In the video decoder <NUM>, the video stream VS is subjected to decoding processing, and target video data is obtained. In this case, for the target video data, not only the video data transmitted from the transmission side but also video data obtained by performing conversion processing of a color gamut or a dynamic range thereon as appropriate are conceived.

The subtitle stream SS extracted in the system decoder <NUM> is supplied to the subtitle decoder <NUM>. In the subtitle decoder <NUM>, decoding processing is performed on the subtitle stream SS, and bitmap data and information of a predetermined number of bitmap conversion tables are obtained. The information of a predetermined number of bitmap conversion tables is obtained from the WC segment (see <FIG>).

Further, at that time, characteristic information of subtitle graphics data before conversion to the bitmap data, i.e., meta-information such as information of a color gamut or a dynamic range is extracted from the rendering guide segment (see <FIG>) and transmitted to the control section <NUM>.

The bitmap data and the information of a predetermined number of bitmap conversion tables, which are obtained in the subtitle decoder <NUM>, are supplied to the conversion section <NUM>. In the conversion section <NUM>, under the control of the control section <NUM>, a matched bitmap conversion table is selected from the predetermined number of bitmap conversion tables. In the conversion section <NUM>, a matched bitmap conversion table is determined on the basis of the matching characteristic information ("subtitle_dynamic range type", "subtitle_color_gamut", "target_video_dynamic_range_type", "target video color_gamut") (see <FIG>), which is added to each bitmap conversion table.

In the conversion section <NUM>, the matched bitmap conversion table is then used to convert the bitmap data to the subtitle graphics data. A color gamut and a dynamic range of the subtitle graphics data thus converted are matched with a color gamut and a dynamic range of the target video data obtained in the video decoder <NUM>.

It should be noted that, in the conversion section <NUM>, when there is no matched one in the transmitted bitmap conversion tables, a necessary conversion function out of the provided first conversion function <NUM> to seventh conversion function <NUM> (see <FIG>) is used through determination of the reception side, and the subtitle graphics data having characteristics matched with those of the target video data of a superimposition destination is generated from the bitmap data.

The target video data obtained in the video decoder <NUM> is supplied to the video superimposition section <NUM>. Further, the subtitle graphics data obtained in the conversion section <NUM> is supplied to the video superimposition section <NUM>. In the video superimposition section <NUM>, the subtitle graphics data is superimposed on the target video data. In this case, the subtitle graphics data is mixed with the video data at a predetermined ratio. Here, the mixing ratio is based on the T-Value.

A video data V1', which is obtained in the video superimposition section <NUM> and on which the subtitle graphics data is superimposed, is converted from the YCbCr (luminance/color difference) domain to the RGB domain in the YCbCr/RGB conversion section <NUM>, and supplied to the electro-optical conversion section <NUM>. In the electro-optical conversion section <NUM>, electro-optical conversion is performed on the video data V1' by applying an electro-optical conversion characteristic, which corresponds to the opto-electronic conversion characteristic applied to the video data V1', and display video data for displaying images is obtained.

The display video data is supplied to the display mapping section <NUM>. In this display mapping section <NUM>, display luminance adjustment is performed on the display video data, the display luminance adjustment corresponding to the maximum luminance display capability of the CE monitor <NUM>, or the like. The display video data on which the display luminance adjustment is thus performed is supplied to the CE monitor <NUM>. On the CE monitor <NUM>, images are displayed on the basis of the display video data.

As described above, in the transmission and reception system <NUM> shown in <FIG>, the subtitle stream transmitted from the transmission side to the reception side includes the bitmap conversion table containing the conversion information of a color gamut and/or a luminance. Therefore, the reception side can easily obtain subtitle graphics data having characteristics matched with those of target video data of a superimposition destination by just converting the bitmap data to the subtitle graphics data by using the bitmap conversion table. This enables a processing load on the reception side to be alleviated.

Further, in the transmission and reception system <NUM> shown in <FIG>, the characteristic information of the subtitle graphics data before the conversion to the bitmap data is inserted into the subtitle stream transmitted from the transmission side to the reception side. Therefore, the reception side can easily grasp the characteristics of the subtitle graphics data before the conversion to the bitmap data.

Claim 1:
A reception apparatus (<NUM>), comprising:
reception circuitry (<NUM>) configured to:
receive a container that includes a video stream and a subtitle stream,
the video stream including video data,
the subtitle stream including bitmap data and conversion information,
the conversion information including at least a bitmap conversion table for converting the bitmap data into subtitle graphics data, and
each of the at least bitmap conversion table including meta-information indicating a color gamut and a dynamic range of the subtitle graphics data; and
processing circuitry (<NUM>; <NUM>; <NUM>; <NUM>; <NUM>); configured to:
decode the video stream to obtain the video data;
decode the subtitle stream to obtain the bitmap data and the conversion information;
convert the bitmap data to obtain the subtitle graphics data applying one of the at least bitmap conversion table indicating a color gamut and a dynamic range of the subtitling data matching with characteristics of the video data; and
superimpose the subtitle graphics data on the video data.