Patent ID: 12256086

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. Note that the description will be given in the following order.1. Reproduction of Partial Subpicture2. Extract Base Track3. Extension of Subpicture ID Sample Group4. Combination of VVC Base Track and VVC Extra Base Track5. Application of V3C File Format6. First Embodiment (File Generation Device)7. Second Embodiment (Reproduction Device)8. Appendix

1. Reproduction of Partial Subpicture

Documents Supporting Technical Content and Technical Terms or the Like

The scope disclosed in the present technology includes not only the content described in the embodiments but also the content described in the following Non-Patent Documents and the like known at the time of filing, the content of other documents referred to in the following Non-Patent Documents, and the like.Non-Patent Document 1: (described above)Non-Patent Document 2: (described above)Non-Patent Document 3: (described above)Non-Patent Document 4: (described above)Non-Patent Document 5: (described above)That is, the content described in the above-describe

Non-Patent Documents, the content of other documents referred to in the above-described Non-Patent Documents, and the like are also grounds for determining the support requirement.

VVC Bitstream>

Conventionally, in versatile video coding (VVC) of a standard specification of an image coding method, each image (picture) is divided into one or more slices, for example, as described in Non-Patent Document 1. Furthermore, each picture can also be divided into one or more subpictures.

FIG.1is a diagram illustrating a main configuration example of a bitstream (also referred to as a VVC bitstream) obtained by encoding image data by VVC. As illustrated inFIG.1, a VVC bitstream10includes a set (a single or a plurality of coded video sequences11) of coded video sequences11obtained by encoding one moving image. Further, the coded video sequence11includes a set of encoded data of pictures12(encoded data of a single or a plurality of pictures12) which are single frames of the moving image. Each picture12may be divided into a plurality of subpictures13.

That is, the subpicture13is a rectangular region obtained by dividing a rectangular picture12. There is no pixel having no encoded data in the picture12. There is no overlap between the subpictures13. There is no pixel that is not included in any subpicture13among the pixels of the picture12.

The subpicture13can be independently decodable. That is, a single or a plurality of subpictures13can be independently decoded. For example, only a part of the encoded data of the subpicture13constituting the picture12may be decoded. In this case, the encoded data of the subpicture13to be decoded is extracted from the VVC bitstream10, and a parameter set and the like are added to generate a new bitstream (the bitstream is reconstructed).

FIG.2illustrates an example of syntax of a sequence parameter set of the VVC bitstream10. The sequence parameter set is a parameter set (metadata that does not change in the coded video sequence11) for each coded video sequence11.FIG.2illustrates syntax of a part of the sequence parameter set.

As illustrated inFIG.2, the sequence parameter set includes “resolution information of a picture” configured by parameters related to the resolution of the picture12. For example, this “resolution information of a picture” includes a parameter

(sps_pic_width_max_in_luma_samples,

sps_pic_height_max_in_luma sample, or the like) indicating the maximum value of the resolution of the picture12in the coded video sequence11.

Furthermore, the sequence parameter set includes “layout information of a subpicture” configured by parameters related to the layout of the subpicture13. For example, the “layout information of the subpicture” includes a parameter (sps_subpic_ctu_top_left_x[i], sps_subpic_ctu_top_left_y[i], or the like) indicating the position of the subpicture13. Furthermore, the “layout information of the subpicture” includes a parameter (sps_subpic_width_minus1[i], sps_subpic_height_minus1[i], or the like) indicating the resolution of the subpicture13. Further, the “layout information of the subpicture” includes a parameter (sps_subpic_treated_as_pic_flag[i], sps_loop_filter_across_subpic_enabled_flag[i], or the like) related to independent decoding of the subpicture13.

Moreover, the sequence parameter set includes a “layout omission flag of a subpicture” configured by information regarding omission of the “layout information of a subpicture”. For example, the “layout omission flag of the subpicture” includes flag information (sps_independent_subpic_flag) indicating whether or not all the subpictures13are independently decodable, flag information (sps_subpic_same_size_flag) indicating whether or not all the subpictures13have the same resolution, and the like. For example, in a case where all the subpictures13can be independently decoded and have the same resolution, regardless of the number of subpictures13, one parameter related to the width and one parameter related to the height of the subpicture13are stored in the sequence parameter set.

Furthermore, the sequence parameter set includes “identification information of a subpicture” configured by parameters related to identification of the subpicture13. For example, the “identification information of the subpicture” includes identification information (sps_subpic_id[i]) of each subpicture13included in the coded video sequence. For example, a number may be sequentially assigned to each subpicture in the sequence (for example, integer values increasing by “+1” are allocated, as in 0, 1, 2, . . . ), and the number may be used as the identification information of the subpicture13. Note that there are three types of “identification information of a subpicture,” that is, a case where the identification information of the subpicture is omitted, a case where the identification information of the subpicture is stored in the sequence parameter set, and a case where the identification information of the subpicture is stored in the picture parameter set.

In a case where some subpictures13are extracted from such a VVC bitstream10and decoded, since the number of subpictures included in the coded video sequence11changes, “identification information of a subpicture” and “layout information of a subpicture” can change. That is, in this case, it is necessary to update the sequence parameter set of the VVC bitstream10(coded video sequence11) and generate the sequence parameter set corresponding to the bitstream of the extracted subpicture.

Furthermore,FIGS.3and4illustrate examples of syntax of a picture parameter set of the VVC bitstream10inFIG.1. The picture parameter set is a parameter set (metadata that does not change in picture12) for each picture12. InFIGS.3and4, syntax of a part of the picture parameter set is illustrated.

As illustrated inFIG.3, the picture parameter set includes “resolution information of a picture” configured by parameters related to the resolution of the picture12. For example, the “resolution information of a picture” includes a parameter

(pps_pic_width_in_luma_samples,

pps_pic_heigh_in_luma_samples, or the like) indicating the resolution of the picture12.

Furthermore, the picture parameter set includes “identification information of a subpicture” configured by parameters related to identification of the subpicture13. For example, the “identification information of the subpicture” includes identification information pps_subpic_id[i]) of each subpicture13included in the picture12. For example, a number may be sequentially assigned to each subpicture13in the picture12(for example, integer values increasing by “+1” are allocated, as in 0, 1, 2, . . . ), and the number may be used as the identification information of the subpicture13.

Moreover, the picture parameter set includes tile information configured by parameters related to the configuration of tiles in the picture12. If tiles having the same width or height continue to the edge of the screen, this tile information may be omitted.

Furthermore, as illustrated inFIG.4, the picture parameter set includes slice information (slice information) configured by parameters related to a configuration of a slice in a picture. In a case where all the subpictures13are configured by one slice, this slice information may be omitted.

In a case where some subpictures13are extracted from such the VVC bitstream10and decoded, since the configuration of the subpictures in the picture changes, the above-described information of the picture parameter set may change. That is, in this case, it is necessary to update the picture parameter set of the VVC bitstream10(picture12) and generate the picture parameter set corresponding to the bitstream of the extracted subpicture.

VVC File Format

Meanwhile, as a method of storing a file of a picture encoded by VVC, for example, as described in Non-Patent Document 2 to Non-Patent Document 4, a VVC file format using an international organization for standardization base media file format (ISOBMFF) which is a file container specification of the international standard technology for video compression moving picture experts group-4 (MPEG-4) is being developed.

FIG.5is a diagram illustrating a configuration example of a track of the VVC file format. As illustrated inFIG.5, the VVC track20is a track for storing the VVC bitstream, and includes a moov box and an mdat box. The moov box stores metadata and the like related to the track, and the mdat box stores the VVC bitstream.

In a case where a plurality of subpictures is configured in a picture, in this VVC file format, for example, the subpictures in the picture can be stored in a plurality of tracks, or can be collectively stored in one track.

For example, in a case where a subpicture is divided into a plurality of tracks and stored, independent decoding of some subpictures can be realized by selecting a track to be decoded at the time of decoding. In that case, the information (metadata) regarding the merge of the selected track is stored in a track different from the track storing the bitstream of the subpicture. This track is also referred to as a VVC base track. Furthermore, a track storing the bitstream of the subpicture is also referred to as a VVC subpicture track.

FIG.6illustrates an example of syntax of a VVC subpicture order entry (VvcSubpicOrderEntry) stored in the VVC base track in a case where a subpicture is divided into a plurality of tracks and stored. As illustrated inFIG.6, in the VVC base track, VisualSampleGroupEntry is extended to generate a VVC subpicture order entry (VvcSubpicOrderEntry( ) extends VisualSampleGroupEntry(‘spor’)). The VVC subpicture order entry stores information defining a subpicture order sample group which is a group of subpictures to be reconstructed. The subpicture order sample group can include information such as “the number of subpictures” indicating the number of subpictures to be merged, “the decoding order of the subpictures” indicating the decoding order of the subpictures to be merged, and “rewriting information of a subpicture ID” which is information related to rewriting of identification information of the subpictures. For example, subp_track_ref_idx indicates a track or a track group including a subpicture to be merged. Furthermore, subpic_id_info_flag indicates the presence or absence of “rewriting information of the subpicture ID”.

Furthermore,FIG.7illustrates an example of syntax of the VVC subpicture ID entry (VvcSubpicIDEntry) stored in a VVC subpicture track in a case where a subpicture is divided into a plurality of tracks and stored. As illustrated inFIG.7, in the VVC subpicture track, VisualSampleGroupEntry is extended to generate the VVC subpicture ID entry (VvcSubpicIDEntry( ) extends VisualSampleGroupEntry(‘spid’)). The VVVC subpicture ID entry stores information defining a subpicture ID sample group which is a group of subpicture identification information (subpic_id) included in the sample of the VVC subpicture track. The subpicture ID sample group indicates identification information (subpic_id) of the subpictures included in the track in decoding order.

As described above, the subpictures in the picture can be collectively stored in one track. The VVC track in the example ofFIG.5stores a bitstream of a picture including six subpictures as a sample23in the mdat box. That is, the sample23includes the bitstreams24(subpicture1data to subpicture6data) of six subpictures.

In this case, a tile region loop entry (TlieRegionGroupEntry(trif))21and a null unit map entry (NALUMapEntry(nalm))22are formed in the moov box of the VVC track20.

The tile region loop entry21stores information defining a tile region sample group which is a sample group indicating a tile region of a picture. The null unit map entry22stores information defining a null unit map sample group (NAL unit map sample group) which is a sample group for classifying the null unit (NAL unit) included in the sample23for each tile region.

In the tile region loop entry21, a groupID is stored as the identifier. That is, the groupID is identification information of the tile region. For example, in a case where four tile regions are formed in a picture as illustrated in A ofFIG.8, different groupIDs are assigned to the respective regions (groupID=1 to groupID=4). The tile region loop entry21also stores information specifying (the position and size of) the tile region. In VVC, one tile region may contain one or more subpictures. That is, as illustrated in B of

FIG.8, one tile region may be configured by one subpicture or may be configured by a plurality of subpictures.

FIG.9illustrates an example of syntax of the tile region loop entry21. As illustrated inFIG.9, the tile region loop entry21stores an identifier (groupID) of the tile region loop entry21, a parameter (horizontal offset, vertical offset) indicating a position of a tile region, a parameter (region width, region height) indicating a size of a tile region, a parameter (dependency tile count, dependencyTileGroupID) indicating a dependency relation between tile regions, and the like.

The null unit map entry22stores information indicating a correspondence between a null unit and a group ID. Since the groupID is given to the tile region as described above, the null unit corresponds to any groupID as illustrated in A ofFIG.10. A correspondence between the null unit and the group ID is indicated in the null unit map entry22. An example of syntax of the null unit map entry22is illustrated in B ofFIG.10. As illustrated inFIG.10, information (NALU start number) indicating a null unit corresponding to each group ID is illustrated.

Use Case: Omnidirectional Video

FIG.11is a diagram illustrating a use case of applying a subpicture. For example, as illustrated in A ofFIG.11, a video of each side of the omnidirectional video content (of Cubemap) including videos in six directions of up, down, left, right, front, and back may be used as a subpicture. In the case of such omnidirectional video content, there are various cases such as a case where only one of six surfaces is decoded, a case where only two surfaces are decoded, . . . , and a case where all six surfaces are decoded.

Therefore, as illustrated in B ofFIG.11, videos of all surfaces are stored in one VVC track, and some subsamples can be appropriately extracted, so that each case described above can be handled.

Use Case: V3C Bitstream

Furthermore, there is video-based point cloud compression (V-PCC) in which a point cloud, which is a set of points having position information and attribute information (color, reflection, or the like) at the same time in a three-dimensional space, is segmented to form a region, an attribute image (color information or the like) and a geometry image (constituted by depth information) are generated by plane-projecting the point cloud for each region, an ocular map image and atlas information (information for reconstructing the point cloud from a patch) are generated, and three images are encoded by a moving image codec. Non-Patent Document 5 discloses a technique of storing a V3C bitstream in ISOBMFF for the purpose of improving the efficiency of reproduction processing and network distribution of a bitstream (referred to as a V3C bitstream) encoded by such V-PCC from a local storage.

The video based point cloud data (V3C) file format (V3C File Format) is a file format using an ISO base media file storing V-PCC. The V3C file format is capable of storing point cloud data in a plurality of tracks (V3C track, occ, geo, att), for example, as illustrated in A ofFIG.12. In this case, metadata for reconstructing the point cloud is included in the V3C track. Moreover, the V3C track is a track (video component track) including a video component of each of the occupancy map, the geometry, and the attribute of the V-PCC.

Furthermore, the V3C file format can separate tracks for each partial area, for example, as illustrated in B ofFIG.12, in order to enable partial access of point cloud data (independently decoding a partial area of a point cloud). In that case, the following track configuration is possible. The V3C track is a V3C atlas tile track for a partial region. Each V3C atlas tile track is a respective video component track of an occupancy map, geometry, and an attribute included in its atlas tile. Each video component track may, for example, store and independently decode a VVC bitstream. In the case of VVC bitstreams, the VVC file format is applied to each video component track.

Even in the case of the content of such the V3C file format, in order to realize partial access, setting each partial area as a subpicture, storing all the partial areas in one VVC track, and enabling some subpictures to be appropriately extracted is conceivable.

Reconstruction of Parameter Set

As described above, in the VVC file format, a plurality of subpictures can be stored in one track, and some of the subpictures can be extracted and decoded. However, in this case, one bitstream is reconstructed using the extracted subpicture, and the bitstream is decoded. Since the configuration of the subpicture is different between the bitstream before reconstruction and the bitstream after reconstruction, the parameter set (for example, the sequence parameter set or the picture parameter set) cannot be used as it is. Therefore, it is necessary to derive the parameter set corresponding to the bitstream after the reconstruction, for example, by updating the parameter set before the reconstruction.

That is, it is necessary to perform complicated work such as analyzing the bitstream before reconstruction and the parameter set thereof, updating the parameter set, and deriving the parameter set corresponding to the bitstream after reconstruction, which may increase the load of the reproduction processing. Note that, the reproduction processing includes at least processing of decoding a bitstream of content stored in a file and processing related to the decoding (for example, processing of reconstructing a bitstream to be decoded, or the like). Furthermore, processing of generating a display image using data obtained by decoding may be included in the reproduction processing, or other processing may be included.

For example, in a case where a file of content having a plurality of subpictures is distributed from a server, and a client device generates a display image by decoding only some subpictures stored in the file, there is a possibility of a load on the client device increasing. Generally, the processing capability of the client device is lower than that of the server. Therefore, there is a possibility of the processing being delayed or failing due to such an increase in load. In other words, in order to prevent a processing delay or failure from occurring, it is necessary to increase the processing capacity of the client device, which may increase the cost.

Note that, as described above, in the case of the method of storing each subpicture in a plurality of tracks, it is possible to store parameter sets necessary for bitstream reconstruction using the VVC base track. However, in the VVC base track, a desired subpicture cannot be extracted from a plurality of subpictures existing in the track.

2. Extract Base Track

Method 1

Therefore, as illustrated in the uppermost row of the table inFIG.13, the VVC extract base track having reconstruction information such as a parameter set necessary for reconstructing a bitstream from some subpictures is provided and stored in a file (Method 1). Then, in a case where some subpictures are reproduced, a reconstructed bitstream is generated on the basis of the information stored in the VVC extract base track.

FIG.14is a diagram illustrating a main configuration example of a file to which the present technology is applied. A file100illustrated inFIG.14is a file conforming to a VVC file format for storing a VVC bitstream in ISOBMFF. As illustrated inFIG.14, the file100has a VVC bitstream track110and a VVC extract base track120.

The VVC bitstream track110is a track storing the VVC bitstream, similarly to the VVC track20described above. The VVC bitstream track110may store a VVC bitstream having a plurality of subpictures. In the case of this example, in the mdat box of the VVC bitstream track110, a bitstream of a picture including six subpictures is stored as a sample115. That is, the sample115includes a bitstream116(subpicture1data to subpicture6data) of six subpictures.

Furthermore, a tile region loop entry

(TlieRegionGroupEntry(trif))111and a null unit map entry (NALUMapEntry(nalm))112are stored in the moov box of the VVC bitstream track110. Similarly to the tile region loop entry21described above, information defining a tile region sample group is stored in the tile region loop entry111. Similarly to the null unit map entry22described above, the null unit map entry112stores information indicating a correspondence between a null unit and a group ID.

Moreover, a sample entry (SampleEntry(vvc1))113is stored in the moov box of the VVC bitstream track110. The sample entry113stores a VVC configuration box (VvcConfigurationBox)114.

The VVC extract base track120stores reconstruction information necessary for reconstructing a bitstream from some subpictures. The reproduction device can extract the bitstream116of a desired subpicture from the VVC bitstream track110on the basis of the reconstruction information stored in the VVC extract base track120, and reconstruct the extracted bitstream of the subpicture.

For example, the reproduction device first determines a region to be reproduced on the basis of the information stored in the tile region loop entry111of the VVC bitstream track110. Therefore, the groupID indicating the region to be reproduced is determined. Next, the reproduction device searches the VVC extract base track120including the area to be reproduced.

Then, the reproduction device updates (rewrites) the parameter set (such as the sequence parameter set and the picture parameter set) on the basis of the information stored in the searched VVC extract base track120. Then, the reproduction device extracts the null unit of the region to be reproduced from the sample115using the null unit map sample group defined in the null unit map entry112of the VVC bitstream track110, and reconstructs the sample using the information stored in the VVC extract base track120.

As described above, the reproduction device can easily reconstruct the bitstream without analyzing the VVC bitstream stored in the VVC bitstream track110. Therefore, an increase in the load of the reproduction processing can be suppressed.

Note that, in the case of the example ofFIG.14, the file100includes, as the VVC extract base track120, a VVC extract base track121storing reconstruction information for reproduction of one subpicture (1-sided), a VVC extract base track122that stores reconstruction information for reproduction of two subpictures (2-sides), a VVC extract base track123storing reconstruction information for reproduction of three subpictures (3-sided), a VVC extract base track124storing reconstruction information for reproduction of four subpictures (4-sided), and a VVC extract base track125storing reconstruction information for reproduction of five subpictures (5-sided). In this manner, the VVC extract base track may be generated for each number of subpictures to be reconstructed.

Method 1-1

Note that, as illustrated in the second row from the top of the table inFIG.13, a track used for reproduction and extraction information stored in the track and specifying a subpicture to be reproduced may be stored in the VVC extract base track (Method 1-1). Then, in a case where some subpictures are reproduced, a reconstructed bitstream may be generated on the basis of the extraction information stored in the VVC extract base track.

For example, the VVC extract base track120may store, as the reconstruction information, subpicture specifying information that is information for specifying a subpicture to be extracted in the VVC bitstream track110.

For example, in an information processing method, a file storing a bitstream track storing a bitstream of content having a subpicture in a frame and an extract base track storing subpicture specifying information that is information for specifying a subpicture to be extracted in the bitstream track is generated.

For example, an information processing apparatus includes a file generation unit that generates a file storing a bitstream track storing a bitstream of content having a subpicture in a frame, and an extract base track storing subpicture specifying information that is information for specifying a subpicture to be extracted in the bitstream track.

By doing so, a file including the extract base track that stores the subpicture specifying information is generated. Therefore, the reproduction device can extract a desired subpicture from the bitstream track on the basis of the extract base track and reconstruct the bitstream. That is, the reproduction device can easily reconstruct the bitstream without analyzing the bitstream. Therefore, an increase in the load of the reproduction processing can be suppressed.

Furthermore, for example, in an information processing method, a subpicture is extracted from a bitstream track and a bitstream is generated, on a basis of the extract base track of a file storing a bitstream track storing a bitstream of content having a subpicture in a frame, and an extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

For example, an information processing apparatus includes a bitstream generation unit that extracts a subpicture from a bitstream track and generates a bitstream on the basis of an extract base track of a file storing a bitstream track storing a bitstream of content having the subpicture in a frame and an extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

By doing so, it is possible to extract a desired subpicture from the bitstream track on the basis of the extract base track including the subpicture specifying information and to reconstruct the bitstream. That is, it can easily reconstruct the bitstream without analyzing the bitstream. Therefore, an increase in the load of the reproduction processing can be suppressed.

Note that the subpicture specifying information may include tile region identification information that is identification information of a tile region including the subpicture. For example, the groupID as described with reference toFIG.8may be used as the subpicture specifying information. By doing so, the reproduction device can more easily specify the tile region (subpicture) to be reproduced.

Method 1-1-1

Note that, as illustrated in the third row from the top of the table inFIG.13, a sample group may be defined, and extraction information that allows the client to select one from a plurality of sets of subpictures may be stored in the sample group (Method 1-1-1). Then, in a case where some subpictures are reproduced, a set of subpictures to be extracted may be selected on the basis of the sample group.

For example, a visual sample group entry (VisualSampleGroupEntry(spex)) may be extended to generate a VVC subpicture extraction entry (VvcSubpictureExtractionEntry( ), and information defining a subpicture extraction sample group that is a sample group of subpictures to be extracted (to be reproduced) may be stored in the VVC subpicture extraction entry.

The subpicture extraction sample group can include, for example, information indicating groupID information (also referred to as a region set) of the tile region loop entry of the track of the candidate of the subpicture to be extracted. Note that, in a case where the tile region is directly designated, it is only required to store a region set including one tile region. Furthermore, the subpicture extraction sample group can include information (subpic_id rewriting information) related to rewriting (updating) of the subpicture identification information that is the identification information of the subpicture. As described above, by applying the region set, the reproduction device can more easily extract a plurality of tile regions. The reproduction device (client) selects and extracts one groupID from the region set of the subpicture extraction sample group.

FIG.15illustrates an example of syntax of the VVC subpicture extraction entry (VvcSubpictureExtractionEntry( ) in this case. In the case of the example ofFIG.15, the following parameters can be stored in the VVC subpicture extraction entry. num_region_set indicates the number of region sets. num_region_in_set indicates the number of tile regions included in the region set. groupID indicates a groupID indicating a tile region included in the region set. num_regions indicates the number of tile regions to be extracted. region_idx is designated using region_set_id or groupID which is identification information of a region set.

For example, the extract base track may set one set of subpictures extracted into one bitstream, and store the subpicture specifying information for each set.

Furthermore, the extract base track may further store set identification information which is identification information of the set.

By setting the region set as described above, it is possible to store the extraction information of a plurality of patterns in one VVC extract base track. That is, a plurality of combinations of subpictures to be extracted can be defined in one VVC extract base track. Therefore, the reproduction device can extract more various combinations of subpictures on the basis of such a VVC extract base track. That is, the extraction pattern (the number of combinations) of the subpicture can be increased without increasing the number of tracks.

Therefore, redundant tracks can be collected, and a reduction in encoding efficiency can be suppressed.

Note that, in the syntax example illustrated inFIG.15, information after the line of “if (subpic_id_info_flag) {” is information regarding rewriting of the subpicture identification information. That is, information for updating the parameter sets such as the sequence parameter set and the picture parameter set so as to correspond to the reconstructed bitstream is stored in the VVC subpicture extraction entry. More specifically, information for updating the subpicture identification information included in the parameter set to correspond to the reconstructed bitstream is stored in the VVC subpicture extraction entry.

That is, the extract base track may further store parameter set update information that is information for updating the parameter set to correspond to the bitstream of the extracted subpicture.

For example, the parameter set update information may include subpicture update information that is information for updating subpicture identification information that is identification information of a subpicture to correspond to a bitstream of an extracted subpicture.

By doing so, the reproduction device can more easily make the parameter set correspond to the reconstructed bitstream on the basis of the information stored in the extract base track. For example, the reproduction device can more easily make the subpicture identification information included in the parameter correspond to the reconstructed bitstream on the basis of the information stored in the extract base track.

3. Extension of Subpicture ID Sample Group

Method 1-2

As illustrated in the fourth row from the top of the table inFIG.13, the subpicture ID sample group may be extended, and mapping information indicating a correspondence between a subpicture and a tile region may be stored (Method 1-2). For example, as such mapping information, information indicating a tile region including a subpicture may be stored in the subpicture ID sample group.

For example, in the case of the subpicture ID sample group illustrated inFIG.7, a relationship between a subpicture and a tile region (in which tile region a subpicture is included) is unknown. As described with reference toFIG.8, since tile regions and subpictures do not always correspond one-to-one, there may be a case where a subpicture necessary for a desired tile region (groupID) cannot be specified on the basis of the subpicture extraction sample group illustrated inFIG.15.

Therefore, in the subpicture ID sample group, the tile region (groupID) including the subpicture (subpic_id) is indicated.

FIG.16illustrates an example of syntax of VvcSubpicIDEntry( ) storing information defining the subpicture ID sample group in that case. As illustrated inFIG.16, next to the subpicture identification information (subpic_id), the identification information (groupID) of the tile region including the subpicture corresponding to the subpicture identification information (subpic_id) is stored in VvcSubpicIDEntry( ). Therefore, the correspondence between the subpicture (subpic_id) and the tile region (groupID) is clearly indicated.

That is, the bitstream track may further store, for each subpicture stored in the bitstream track, the subpicture identification information and the tile region identification information of the tile region including the subpicture.

By doing so, the reproduction device can more easily specify the subpicture (subpic_id) included in the desired tile region on the basis of the information stored in the subpicture ID sample group. Therefore, the reproduction device can more easily rewrite (update) the subpicture identification information (subpic_id) on the basis of the information of the subpicture extraction sample group.

Method 1-2-1>

As illustrated in the fifth row from the top of the table inFIG.13, storage of mapping information indicating the correspondence between the subpicture and the tile region described above may be omitted (Method 1-2-1).

FIG.17illustrates an example of syntax of

VvcSubpicIDEntry( ) storing information defining the subpicture ID sample group in that case. As illustrated inFIG.17, in this case, all_subpic_id_is_same as groupID_flag is stored in VvcSubpic_IDEntry( ) all_subpic_id_is_same_as groupID_flag is flag information indicating whether or not all subpictures (subpic_id) and tile regions (groupIDs) in a picture match (correspond on a one-to-one basis). In a case where a value of all_subpic_id_is_same as groupID flag is true (for example, 1), it indicates that all subpictures (subpic_id) in a picture match a tile region (groupID). In addition, in a case where a value of all_subpic_id is same as groupID_flag_is false (for example, 0), it indicates that at least some subpictures and tile regions do not correspond on a one-to-one basis (one tile region contains a plurality of subpictures). Then, it is defined that the groupID is stored only in a case where a value of all_subpic_id is same as groupID_flag is false (for example, 0). In other words, in a case where all the subpictures (subpic_d) in the picture and the tile region (groupID) match (correspond on a one-to-one basis), the storage of the groupID is omitted.

That is, the bitstream track may store only the subpicture identification information for the subpicture in which the subpicture identification information matches the tile region identification information.

In a case where all the subpictures (subpic_id) and the tile region (groupID) in the picture correspond to each other on a one-to-one basis, the groupID and the subpic_id are easily associated with each other on a one-to-one basis even if there is no mapping information. Therefore, by doing so, it is possible to suppress an increase in an unnecessary code amount. Therefore, a reduction in encoding efficiency can be suppressed.

4. Combination of VVC Base Track and VVC Extra Base Track

Method 2

As shown in the sixth row from the top of the table inFIG.13, subpictures may be extracted from a plurality of tracks using both the VVC base track and the VVC extract base track (Method 2). That is, the VVC extract base track supports only extraction from one track, and in a case where extraction is performed from a plurality of tracks, the VVC base track may be used in combination.

For example, as illustrated in A ofFIG.18, some subpictures are extracted from two pictures, and the extracted subpictures are merged into one picture. In this case, as illustrated in B ofFIG.18, the information of the VVC extract base track may be applied when a subpicture is extracted from a picture, and the information of the VVC base track may be applied when the extracted subpictures are merged.

That is, the file storing the bitstream track and the extract base track may further include a merging base track storing information regarding merging of a plurality of subpictures extracted from different bitstream tracks on the basis of the extract base track.

Any of the VVC extract base tracks described in <2. Extract base track> and <3. Extension of Subpicture ID sample group> can be applied to this VVC extract base track.

Furthermore, an example of syntax of a VVC subpicture order entry (VvcSubpicOrderEntry( ) extends VisualSampleGroupEntry (‘spor’)) of the VVC base track is illustrated inFIG.19.

As illustrated inFIG.19, in the case of this example, extract track exist flag is stored in the VVC subpicture order entry. The extract track exist flag is flag information indicating whether or not the VVC extract base track is included in the track to be referred to. For example, in a case where the extract_track_exist_flag is true (for example, 1), it indicates that the VVC extract base track is included in the track to be referred to. Further, in a case where the extract_track_exist_flag is false (for example, 0), it indicates that the VVC extract base track is not included in the track to be referred to.

By storing such extract track exist flag, it is possible to easily identify whether or not the VVC extract base track is included in the track to be referred to. That is, the VVC extract base track and the VVC base track as described above can be used together more easily. Furthermore, the VVC extract base track can be included in the tracks that can be merged in the VVC base track. Furthermore, it is possible to easily determine whether the VVC extract base track is included in the VVC base track.

5. Application of V3C File Format

Method 3

As shown in the bottom row of the table ofFIG.13, the present technology may be applied to the V3C file format (Method 3).

As described with reference toFIG.12, in the V3C file format, the V-PCC bitstream can be stored in the ISOBMFF file. The present technology may be applied to such a V3C file format, and partial access may be realized using the subpicture.

That is, the content may be a point cloud representing a three-dimensional object as a set of points, and the subpicture may be configured by a partial region that can be independently decoded of the point cloud.

For example, as illustrated in A ofFIG.20, partial access of the V-PCC bitstream may be realized using the VVC extract base track. In a case of the example shown in A ofFIG.20, there is a V3C atlas tile track. In such a case, as illustrated in a dotted frame in A ofFIG.20, it is only required to provide the VVC extract base track for each of the occupancy map, the geometry, and the attribute.

The VVC extract base track of each example described in any of <2. Extract base track>, <3. Extension of Subpicture ID sample group>, and <4. Combination of VVC Base Track and VVC Extra Base Track> can be applied to this VVC extract base track. In addition, a plurality of these examples may be applied in combination. That is, the extract base track storing the subpicture specifying information, which is information for specifying the subpicture to be extracted, is stored in the file.

At the time of reproduction, it is only required to extract and decode necessary subpictures using the VVC extract base track for each of the occupancy map, the geometry, and the attribute.

Furthermore, as illustrated in B ofFIG.20, the V3C file format may be a multi-track configuration in which each of the occupancy map, geometry, and attributes are stored in different tracks from one another in which there is no V3C atlas track.

However, the track storing the occupancy map is a VVC track including a plurality of subpictures. The same applies to the geometry and the attribute. In a case where the partial reproduction is performed, it can be realized by extracting and reproducing a specific subpicture.

6. First Embodiment

File Generation Device

(Each method of) the present technology described above can be applied to any device.FIG.21is a block diagram illustrating an example of a configuration of a file generation device which is an aspect of an information processing apparatus to which the present technology is applied. The file generation device300illustrated inFIG.21is a device that stores, in ISOBMFF, a VVC bitstream obtained by encoding video content including subpictures by VVC.

The file generation device300applies the above-described present technology, stores a plurality of subpictures in one track, and stores a VVC bitstream in a file so that some subpictures can be independently reproduced. That is, a file generation device300stores a bitstream track storing a bitstream of content having a subpicture in a frame and an extract base track storing subpicture specifying information which is information for specifying a subpicture to be extracted in the bitstream track in a file.

Note that, inFIG.21, main processing units, data flows, and the like are illustrated, and those illustrated inFIG.21are not necessarily all. That is, in the file generation device300, there may be a processing unit not illustrated as a block inFIG.21, or there may be processing or a data flow not illustrated as an arrow or the like inFIG.21.

As illustrated inFIG.21, the file generation device300includes a control unit301and a file generation processing unit302. The control unit301controls the file generation processing unit302. The file generation processing unit302is controlled by the control unit301and performs processing related to file generation. For example, the file generation processing unit302acquires data of content having a subpicture in a picture, encodes the data, and generates a VVC bitstream. The file generation processing unit302further stores the generated VVC bitstream in the file of ISOBMFF, and outputs the file to the outside of the file generation device300.

The file generation processing unit302includes an input unit311, a preprocessing unit312, an encoding unit313, a file generation unit314, a recording unit315, and an output unit316.

The input unit311acquires data of content having a subpicture in a picture, and supplies the data to the preprocessing unit312. The preprocessing unit312extracts information necessary for file generation from the data of the content. The preprocessing unit312supplies the extracted information to the file generation unit314. In addition, the preprocessing unit312supplies the data of the content to the encoding unit313.

The encoding unit313encodes the data of the content supplied from the preprocessing unit312using the VVC method to generate a VVC bitstream. The encoding unit313supplies the generated VVC bitstream to the file generation unit314.

The file generation unit314stores the VVC bitstream supplied from the encoding unit313in the file of ISOBMFF. At that time, the file generation unit314appropriately stores the information supplied from the preprocessing unit312in the file.

Furthermore, the file generation unit314applies the present technology described in any one of <2. Extract base track>, <3. Extension of Subpicture ID sample group>, <4. Combination of VVC Base Track and VVC Extra Base Track>, and <5. Application of V3C File Format> or a combination thereof, and generates a file that stores a VVC bitstream.

That is, the file generation unit314generates a file storing a bitstream track storing a bitstream of content having a subpicture in a frame, and an extract base track storing subpicture specifying information that is information for specifying a subpicture to be extracted in the bitstream track.

The file generation unit314supplies the generated file to the recording unit315. The recording unit315includes an arbitrary recording medium such as a hard disk or a semiconductor memory, for example, and records the file supplied from the file generation unit314in the recording medium. Furthermore, the recording unit315reads a file recorded in the recording medium in accordance with a request from the control unit301or the output unit316or at a predetermined timing, and supplies the file to the output unit316.

The output unit316acquires the file supplied from the recording unit315, and outputs the file to the outside of the file generation device300(for example, a distribution server, a reproduction device, or the like).

With such a configuration, the file generation device300can store, in the extract base track, reconstruction information necessary for reconstructing a bitstream from some subpictures. Therefore, the reproduction device that reproduces the file can extract the bitstream of the desired subpicture from the bitstream track on the basis of the reconstruction information stored in such the extract base track and reconstruct the bitstream of the extracted subpicture. Therefore, an increase in the load of the reproduction processing can be suppressed.

Flow of File Generation Processing

An example of a flow of file generation processing executed by the file generation unit314inFIG.21will be described with reference to a flowchart inFIG.22.

When the file generation processing is started, the file generation unit314of the file generation device300generates a VVC bitstream track storing a VVC bitstream in step S301.

In step S302, the file generation unit314determines a combination of subpictures to be extracted.

In step S303, the file generation unit314generates, for each combination of subpictures determined in step S302, reconstruction information necessary for reconstructing a bitstream from some subpictures. That is, the file generation unit314generates, as the reconstruction information, subpicture specifying information that is information for specifying a subpicture to be extracted in the bitstream track.

In step S304, the file generation unit314stores the reconstruction information generated in step S303in the VVC extract base track and generates a file. That is, the file generation unit314generates a file storing a bitstream track storing a bitstream of content having a subpicture in a frame, and an extract base track storing subpicture specifying information that is information for specifying a subpicture to be extracted in the bitstream track.

When the file is generated, the file generation processing ends.

As described above, by executing each processing, the file generation device300can generate the extract base track storing the reconstruction information necessary for reconstructing the bitstream from some subpictures, and generate the file storing the extract base track. Therefore, the reproduction device that reproduces the file can extract the bitstream of the desired subpicture from the bitstream track on the basis of the reconstruction information stored in such the extract base track and reconstruct the bitstream of the extracted subpicture. Therefore, an increase in the load of the reproduction processing can be suppressed.

7. Second Embodiment

Client Device

FIG.23is a block diagram illustrating an example of a configuration of a reproduction device which is an aspect of an information processing apparatus to which the present technology is applied. A client device400illustrated inFIG.23is a reproduction device that decodes a VVC bitstream stored in a file of a VVC file format, and generates and displays a display image of the generated moving image content. For example, the client device400decodes the VVC bitstream stored in the file generated by the file generation device300, and generates and displays a display image of the generated moving image content. At that time, the client device400applies the above-described present technology, extracts some subpictures from the file, reconstructs the bitstream, and decodes the reconstructed bitstream.

Note that, inFIG.23, main processing units, data flows, and the like are illustrated, and those illustrated inFIG.23are not necessarily all. That is, in the client device400, there may be a processing unit not illustrated as a block inFIG.23, or there may be processing or a data flow not illustrated as an arrow or the like inFIG.23.

As illustrated inFIG.23, the client device400has a control unit401and a reproduction processing unit402. The control unit401performs processing related to control of the reproduction processing unit402. The reproduction processing unit402performs processing related to reproduction of video content stored in a file. For example, the reproduction processing unit402is controlled by the control unit401and acquires a file from a distribution server (not illustrated) or the like.

This file is a file conforming to the VVC file format generated by applying the present technology. For example, the file is a file generated by the file generation device300. That is, the file is for storing a bitstream track storing a bitstream of content having a subpicture in a frame and an extract base track storing subpicture specifying information which is information for specifying a subpicture to be extracted in the bitstream track.

The reproduction processing unit402executes reproduction processing on the acquired file, decodes a bitstream of video content stored in the file, and generates and displays a display image of the video content.

The reproduction processing unit402includes a file acquisition unit411, a file processing unit412, a decoding unit413, a display information generation unit414, a display unit415, a measurement unit416, and a display control unit417.

The file acquisition unit411acquires a file in the VVC file format supplied from the outside of the client device400, for example, the distribution server, the file generation device300, or the like. As described above, in this file, a bitstream track storing a bitstream of content having a subpicture in a frame and an extract base track storing subpicture specifying information which is information for specifying a subpicture to be extracted in the bitstream track are stored. The file acquisition unit411supplies the acquired file to the file processing unit412.

The file processing unit412acquires the file supplied from the file acquisition unit411. The file processing unit412acquires the measurement result supplied from the measurement unit416. The file processing unit412acquires the control information supplied from the display control unit417.

Using these pieces of information, the file processing unit412extracts some subpictures from the file, and generates a bitstream of the extracted subpictures. At that time, the file processing unit412applies the present technology described in any one of <2. Extract base track>, <3. Extension of Subpicture ID sample group>, <4. Combination of VVC Base Track and VVC Extra Base Track>, and <5. Application of V3C File Format> or a combination thereof, and executes processing on this file.

That is, the file processing unit412extracts a subpicture from the bitstream track stored in the file on the basis of the extract base track stored in the file, and generates a bitstream of the extracted subpicture.

The file processing unit412supplies the generated bitstream to the decoding unit413. The decoding unit413decodes the supplied bitstream, and generates data of video content of the extracted subpicture. The decoding unit413supplies data of the generated video content to the display information generation unit414.

The display information generation unit414acquires the data of the video content supplied from the decoding unit413. Furthermore, the display information generation unit414acquires the control information supplied from the display control unit417. Then, the display information generation unit414generates the display image and the like from the acquired data of the video content according to the control information. The display information generation unit414supplies the generated display image and the like to the display unit415. The display unit415includes a display device, and displays the supplied display image using the display device. The measurement unit416measures arbitrary information and supplies the measurement result to the file processing unit412. The display control unit417controls display by supplying control information to the file processing unit412and the display information generation unit414.

With such a configuration, the client device400can extract a bitstream of a part of subpictures as a file on the basis of the extract base track that stores reconstruction information necessary for reconstructing a bitstream from the part of subpictures. Therefore, the client device400can more easily reconstruct the bitstream of the extracted subpicture. Therefore, an increase in the load of the reproduction processing can be suppressed.

Flow of Reproduction Processing

An example of a flow of the reproduction processing executed by the client device400inFIG.23will be described with reference to the flowchart inFIG.24.

When the reproduction processing is started, the file processing unit412of the client device400acquires a file in the VVC file format in step S401.

In step S402, the file processing unit412determines some tile regions to be reproduced on the basis of the information stored in the VVC bitstream track stored in the file acquired in step S401.

In step S403, the file processing unit412searches and acquires the VVC extract base track including the tile region determined in step S402from the file.

In step S404, the file processing unit412extracts subpictures corresponding to some tile regions to be reproduced from the file on the basis of the reconstruction information of the VVC extract base track acquired by the processing in step S403, and reconstructs a bitstream thereof.

That is, the file processing unit412extracts a desired subpicture from a bitstream track that is stored in the file and that stores a bitstream of content having a subpicture in a frame, on the basis of an extract base track storing subpicture specifying information that is stored in the file and that is information for specifying a subpicture to be extracted, and generates a bitstream.

In step S405, the decoding unit413decodes the bitstream reconstructed in step SS04, and obtains data of the video image content of the desired tile region. The display information generation unit414generates the display image. The display unit415displays the display image.

When the processing of step S405ends, the reproduction processing ends.

As described above, by executing each processing, the client device400can extract a desired subpicture from the bitstream track on the basis of the extract base track including the subpicture specifying information and to reconstruct the bitstream. That is, it can easily reconstruct the bitstream without analyzing the bitstream. Therefore, an increase in the load of the reproduction processing can be suppressed.

8. Appendix

Computer

The series of processes described above can be executed by hardware or by software. In a case where the series of processing is executed by software, a program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like, for example.

FIG.25is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

In a computer900illustrated inFIG.25, a central processing unit (CPU)901, a read only memory (ROM)902, and a random access memory (RAM)903are mutually connected via a bus904.

An input/output interface910is also connected to the bus904. An input unit911, an output unit912, a storage unit913, a communication unit914, and a drive915are connected to the input/output interface910.

The input unit911includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like. The output unit912includes, for example, a display, a speaker, an output terminal, and the like. The storage unit913includes, for example, a hard disk, a RAM disk, a nonvolatile memory, and the like. The communication unit914includes, for example, a network interface. The drive915drives a removable medium921such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, for example, the CPU901loads a program stored in the storage unit913into the RAM903via the input/output interface910and the bus904and executes the program, whereby the above-described series of processing is performed. The RAM903also appropriately stores data and the like necessary for the CPU901to execute various processes.

The program executed by the computer can be applied by being recorded in the removable medium921as a package medium or the like, for example. In this case, the program can be installed in the storage unit913via the input/output interface910by attaching the removable medium921to the drive915.

Furthermore, this program can be also provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit914and installed in the storage unit913.

In addition, this program can be installed in the ROM902or the storage unit913in advance.

Object to which Present Technology is Applicable

The present technology can be applied to an arbitrary image encoding/decoding method.

Furthermore, the present technology can be applied to an arbitrary configuration. For example, the present technology can be applied to various electronic devices.

Furthermore, for example, the present technology can also be implemented as a partial configuration of a device, such as a processor (for example, a video processor) as a system large scale integration (LSI) or the like, a module (for example, a video module) using a plurality of processors or the like, a unit (for example, a video unit) using a plurality of modules or the like, or a set (for example, a video set) obtained by further adding other functions to a unit.

Furthermore, for example, the present technology can also be applied to a network system including a plurality of apparatuses. For example, the present technology may be implemented as cloud computing shared and processed in cooperation by a plurality of apparatuses via a network. For example, the present technology may be implemented in a cloud service that provides a service related to an image (moving image) to an arbitrary terminal such as a computer, an audio visual (AV) device, a portable information processing terminal, or an Internet of Things (IoT) device.

Note that, in the present specification, a system means a set of a plurality of configuration elements (apparatuses, modules (parts), or the like), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of apparatuses housed in separate housings and connected via a network and one apparatus in which a plurality of modules is housed in one housing are both systems.

Field and Application to Which Present Technology is Applicable

The system, the apparatus, the processing unit, and the like to which the present technology is applied can be used in arbitrary fields such as traffic, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factory, home appliance, weather, and natural monitoring. Furthermore, the application thereof is also arbitrary.

For example, the present technology can be applied to a system or a device provided for providing content for appreciation or the like. Furthermore, for example, the present technology can also be applied to systems and devices provided for traffic, such as traffic condition supervision and automatic driving control. Moreover, for example, the present technology can also be applied to a system or a device provided for security. Furthermore, for example, the present technology can be applied to systems or devices used for automatic control of machines and the like. Moreover, for example, the present technology can also be applied to systems and devices used for agriculture and livestock industry. Furthermore, the present technology can also be applied to a system and a device that monitor natural states such as volcanoes, forests, and oceans, wildlife, and the like. Moreover, for example, the present technology can also be applied to systems and devices used for sports.

8. Others

Note that, in the present specification, the “flag” is information for identifying a plurality of states, and includes not only information used for identifying two states of true (1) and false (0) but also information capable of identifying three or more states. Therefore, the value that can be taken by the “flag” may be, for example, a binary of 1/0 or a ternary or more. That is, the number of bits constituting this “flag” is arbitrary, and may be one bit or a plurality of bits. Furthermore, since the identification information (including the flag) is assumed to include not only the identification information in the bitstream but also the difference information of the identification information with respect to certain reference information in the bitstream, in the present specification, the “flag” and the “identification information” include not only the information but also the difference information with respect to the reference information.

Furthermore, various types of information (metadata or the like) related to the encoded data (bitstream) may be transmitted or recorded in any form as long as the information is associated with the encoded data. Here, the term “associate” means, for example, that one data can be used (linked) when the other data is processed. That is, the data associated with each other may be collected as one data or may be individual data. For example, information associated with encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image). Furthermore, for example, the information associated with the encoded data (image) may be recorded in a recording medium (or another recording area of the same recording medium) different from the encoded data (image). Note that this “association” may be a part of data instead of the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part in a frame.

Note that, in the present specification, terms such as “combine,” “multiplex,” “add,” “integrate,” “include,” “store,” “put in,” “inlet” and “insert” mean to combine a plurality of items into one, for example, to combine encoded data and metadata into one data, and mean one method of the above-described “associate”.

Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modification examples can be made without departing from the gist of the present technology.

For example, a configuration described as one apparatus (or processing unit) may be divided and configured as a plurality of apparatuses (or processing units). Conversely, configurations described above as a plurality of apparatuses (or processing units) may be collectively configured as one apparatus (or processing unit). Furthermore, a configuration other than the above-described configuration may be added to the configuration of each apparatus (or each processing unit). Moreover, as long as the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain apparatus (or processing unit) may be included in the configuration of another apparatus (or another processing unit).

Furthermore, for example, the above-described program may be executed in an arbitrary apparatus. In that case, it is only required that the apparatus has a necessary function (functional block or the like) and can obtain necessary information.

Furthermore, for example, each step of one flowchart may be executed by one apparatus, or may be shared and executed by a plurality of apparatuses.

Moreover, in a case where a plurality of processes is included in one step, the plurality of processing elements may be executed by one apparatus, or may be shared and executed by a plurality of apparatuses. In other words, a plurality of processing elements included in one step can also be executed as processing elements of a plurality of steps. Conversely, the processing described as a plurality of steps can be collectively executed as one step.

Furthermore, for example, in the program executed by the computer, processing of steps describing the program may be executed in time series in the order described in the present specification, or may be executed in parallel or individually at necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-described order. Moreover, the processing of steps describing this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

Furthermore, for example, a plurality of techniques related to the present technology can be implemented independently as a single body as long as there is no contradiction. Of course, a plurality of arbitrary present technologies can be implemented in combination. For example, some or all of the present technology described in any of the embodiments can be implemented in combination with some or all of the present technology described in other embodiments. Furthermore, some or all of the above-described arbitrary present technology can be implemented in combination with other technologies not described above.

Note that the present technology can also adopt the following configurations.

(1) An information processing apparatus including:a file generation unit configured to generate a file storinga bitstream track storing a bitstream of content having a subpicture in a frame, andan extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

(2) The information processing apparatus according to (1), in whichthe subpicture specifying information includes tile region identification information that is identification information of a tile region including the subpicture.

(3) The information processing apparatus according to (2), in whichthe extract base track sets the subpictures extracted into one bitstream as one set, and stores the subpicture specifying information for each set.

(4) The information processing apparatus according to (3), in whichthe extract base track further stores set identification information that is identification information of the set.

(5) The information processing apparatus according to any one of (2) to (4), in whichthe bitstream track further stores, for each of the subpictures stored in the bitstream track, subpicture identification information that is identification information of the subpicture, and the tile region identification information of a tile region including the subpicture.

(6) The information processing apparatus according to (5), in whichthe bitstream track stores only the subpicture identification information for the subpicture in which the subpicture identification information matches the tile region identification information.

(7) The information processing apparatus according to any one of (1) to (6), in whichthe extract base track further stores parameter set update information that is information for updating a parameter set to correspond to a bitstream of the extracted subpicture.

(8) The information processing apparatus according to (7), in whichthe parameter set update information includes subpicture update information that is information for updating subpicture identification information that is identification information of the subpicture to correspond to a bitstream of the extracted subpicture.

(9) The information processing apparatus according to any one of (1) to (8), in whichthe file further includesa merging base track storing information regarding merging of a plurality of the subpictures extracted from the bitstream tracks different from each other on the basis of the extract base track.

(10) The information processing apparatus according to any one of (1) to (9), in whichthe content is a point cloud representing an object having a three-dimensional shape as a set of points, and the subpicture is configured by a partial region in which the point cloud can be independently decoded.

(11) An information processing method including:generating a file storinga bitstream track storing a bitstream of content having a subpicture in a frame, andan extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

(31) An information processing apparatus including:a bitstream generation unit configured to extract a subpicture from a bitstream track and generate a bitstream on the basis of the extract base track of a file storinga bitstream track storing a bitstream of content having a subpicture in a frame, andan extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

(32) The information processing apparatus according to (31), in whichthe subpicture specifying information includes tile region identification information that is identification information of a tile region including the subpicture.

(33) The information processing apparatus according to (32), in whichthe extract base track sets the subpictures extracted into one bitstream as one set, and stores the subpicture specifying information for each set.

(34) The information processing apparatus according to (33), in whichthe extract base track further stores set identification information that is identification information of the set.

(35) The information processing apparatus according to any one of (32) to (34), in whichthe bitstream track further stores, for each of the subpictures stored in the bitstream track, subpicture identification information that is identification information of the subpicture, and the tile region identification information of a tile region including the subpicture.

(36) The information processing apparatus according to (35), in whichthe bitstream track stores only the subpicture identification information for the subpicture in which the subpicture identification information matches the tile region identification information.

(37) The information processing apparatus according to any one of (31) to (36), in whichthe extract base track further stores parameter set update information that is information for updating a parameter set to correspond to a bitstream of the extracted subpicture.

(38) The information processing apparatus according to (37), in whichthe parameter set update information includes subpicture update information that is information for updating subpicture identification information that is identification information of the subpicture to correspond to a bitstream of the extracted subpicture.

(39) The information processing apparatus according to any one of (31) to (38), in whichthe file further includes:a merging base track storing information regarding merging of a plurality of the subpictures extracted from the bitstream tracks different from each other on the basis of the extract base track.

(40) The information processing apparatus according to any one of (31) to (39), in whichthe content is a point cloud representing an object having a three-dimensional shape as a set of points, andthe subpicture is configured by a partial region in which the point cloud can be independently decoded.

(41) An information processing method including:extracting a subpicture from a bitstream track and generating a bitstream on the basis of the extract base track of a file storinga bitstream track storing a bitstream of content having a subpicture in a frame, andan extract base track storing subpicture specifying information that is information for specifying the subpicture to be extracted in the bitstream track.

REFERENCE SIGNS LIST

300File generation device301Control unit302File generation processing unit311Input unit312Preprocessing unit313Encoding unit314File generation unit315Recording unit316Output unit400Client device401Control unit402Reproduction processing unit411File acquisition unit412File processing unit413Decoding unit414Display information generation unit415Display unit416Measurement unit417Display control unit