Patent Description:
Video encoding uses compression tools to encode two-dimensional video frames into a compressed bitstream representation that is more efficient for storing or transporting over a network. Traditional video coding techniques that use two-dimensional video frames for encoding sometimes are inefficient for representation of visual information of a three-dimensional visual scene.

Document <NPL>) provides an overview of geometry based coding of point clouds.

Document "<NPL>) describes single track and multi-track encapsulation of G-PCC data in an ISO-BMFF format.

Document <NPL>) describes including level of detail (LoD) information in a stream.

This patent document describes, among other things, techniques for encoding or decoding a bitstream that includes three-dimensional visual media representation.

In one example aspect, a method of processing three-dimensional content is disclosed. The method includes parsing a level of detail (LoD) information of a bitstream containing three-dimensional (3D) content that is represented as one geometry sub-bitstream and one or more attribute sub-bitstreams, generating, based on the LoD information, decoded information by decoding at least a portion of the geometry sub-bitstream and the one or more attribute sub-bitstreams corresponding to a desired level of detail, and reconstructing, using the decoded information, a three-dimensional scene corresponding at least to the desired level of detail. The bitstream conforms to a format organized according to multiple levels of details of the 3D content.

In another example aspect, a method encoding three-dimensional content is disclosed. The method includes encoding a three-dimensional (3D) content into a bitstream comprising a geometry sub-bitstream and one or more attribute sub-bitstreams organized according to one or more level of details (LoD), and including, in the bitstream, an LoD information indicative of a correspondence between the one or more LoDs and the geometry sub-bitstream and the one or more attribute bitstreams.

In another example aspect, an apparatus for implementing one or more of the above-described methods is disclosed. The apparatus may include a processor configured to implement the described encoding or decoding methods.

In yet another example aspect, a computer-program storage medium is disclosed. The computer-program storage medium includes code stored thereon. The code, when executed by a processor, causes the processor to implement a described method.

These, and other, aspects are described in the present document.

A point cloud is defined as a multi-set of points where a point is described by its 3D position with (x, y, z) coordinates and (optionally) a set of attributes. Typically, each point in a cloud has the same number of attributes attached to it. G-PCC (Geometry-based Point Cloud Compression) represents an efficient compression method of sparse dynamically varying point clouds such as those used in vehicular light detection and ranging (LiDAR) or three-dimensional (3D) mapping, as well as dense static point clouds used in art, science, cultural heritage, and industrial applications. G-PCC may include decomposing the 3D space into a hierarchical structure of cubes and encoding each point as an index of the cube it belongs to.

A G-PCC bitstream (or simply called a bitstream) may be composed of parameter sets (e.g., a sequence parameter set, a geometry parameter set, an attribute parameter set), geometry slices, or attribute slices. In a G-PCC bitstream, a slice is defined as a set of points that can be encoded or decoded independently. For attribute coding, an efficient method named lifting scalability is supported, that enables to partially decode G-PCC bitstream for constructing a point cloud with a desired level of detail (LoD). The LoD may refer to, for example, a resolution of the content. In some cases, the desired LoD may dependent on a spatial portion of the 3D content. In some cases, the desired LoD may depend on temporal properties, e.g., frame rate, of the 3D content.

This patent document describes a structuring and grouping mechanism to indicate the association of G-PCC components with different levels of detail in a bitstream As further described in this patent document, the described techniques can be used to facilitate partial access and delivery of point cloud data based on a desired LoD.

The spatial scalability is important functionality for G-PCC. It is especially useful when the source point cloud is dense even in the local area as the Level of Detail (or, the octree depth) should be large enough to represent the original quality. With the spatial scalability, a viewer can access a lower resolution point cloud as a thumbnail with less decoder complexity and/or with less bandwidth.

When the spatial scalability is needed, it is desirable to decode lower geometry and the corresponding attribute bitstream in a harmonized way. As specified in the latest G-PCC specification, when attribute data is encoded by LoD with a Lifting Transform with lifting scalability enabled, the attribute decoding process allows a pruned octree decode result for the input geometry points. The geometry decoder could decode geometry data unit until the octree depth corresponding to the desired LoD and then stops decoding. After input of the pruned geometry points, the attribute decoder decodes only a part of attribute data unit that corresponds to input geometry points and stops decoding. Consequently, a point cloud with the desired LoD is constructed from partially decoded geometry and attribute data units.

The G-PCC bitstream may consist of a sequence of type-length-value (TLV) structures that each represents a single coded syntax structure, e.g., geometry payload, attribute payload, a certain type of parameter sets. There may be two types of encapsulation for G-PCC bitstream using the International Standards Organization Base Media File Format (ISOBMFF): Single-track encapsulation and multiple-track encapsulation.

When the G-PCC bitstream is stored in a single track, each G-PCC sample corresponds to a single point cloud frame and may include one or more TLV encapsulation structures which belong to the same presentation time. Each TLV encapsulation structure contains a single type of G-PCC payload, e.g., a geometry slice or an attribute slice. Each G-PCC sub-sample may contain only one G-PCC TLV encapsulation structure.

When the G-PCC bitstream is carried in multiple tracks, each geometry or attribute sub-stream (or sub-bitstream) is mapped to an individual track. There are two types of G-PCC component tracks: a geometry track and an attribute track. The geometry track carries a geometry sub-stream (or sub-bitstream) and the attribute track carries a single type of the attribute sub-stream (or sub-bitstream). Each sample in a track contains one TLV encapsulation structure carrying a single G-PCC component data, not both of geometry and attribute data or multiplexing of different attribute data.

For future improvements in availability, performance and efficiency of G-PCC data delivering services across various networks and to customers using a broad range of decoding and viewing devices, it will be beneficial to identify levels of details of G-PCC data at system level. Such techniques described in the present documents will allow encoder embodiments to generate a well-structured bitstream that can be easily parsed by a decoder to select only data that is needed for reconstruction of an encoded 3D scene according to a desired LoD, e.g., as specified by a viewer or by another app such as a vehicle navigation application.

Section headings are used in the present document only to improve readability and do not limit scope of the disclosed embodiments and techniques in each section to only that section. Certain features are described using the example of the H. <NUM>/AVC (advanced video coding) and H. <NUM>/HEVC (high efficiency video coding) and MPEG (moving pictured experts group) standards. However, applicability of the disclosed techniques is not limited to only H. <NUM>/AVC or H. <NUM>/HEVC systems.

In the present document, various syntax elements are disclosed in different sections for point cloud data processing. However, it is understood that a syntax element with same name will have a same format and syntax as used in different sections, unless otherwise noted. Furthermore, the different syntax elements and structures described under different section headings may be combined together in various embodiments. In addition, while the specific structures are described as implementation examples, it will be understood that the order of various entries of syntax structures may be changed, unless otherwise noted in the present document.

In general, embodiments based on the disclosed technique may be used for video data processing. In some embodiments, omnidirectional video data is stored in a file based on an International Organization for Standardization (ISO) basic media file format. Among them, the ISO basic media file format such as the restricted scheme information box, the track reference box, and the track group box can refer to the ISO/IEC JTC1/SC29/WG11 Moving Picture Experts Group (MPEG) MPEG-<NUM>. Part <NUM> ISO Base Media File Format (ISOBMFF) to operate.

All the data in the ISO basic file format is installed in a box. The ISO basic file format represented by an MPEG <NUM> (MP4) file is composed of several boxes, each of which has a type and a length and can be regarded as a data object. A box can contain another box called a container box. An MP4 file will first have only one "ftyp" type of box, as a markup of the file format and contain some information about the file. There will be and only one "MOOV" type of box (Movie Box), which is a container box whose subbox contains metadata information for the media. The media data of the MP4 file is included in the "mdat" type of media box (Media Data Box), which is also a container box, which may or may not be available (when the media data refers to other files), the structure of the media data is composed of metadata.

A timed metadata track is a mechanism in the ISO Base Media File Format (ISOBMFF) that establishes timed metadata associated with a particular sample. Timed metadata is less coupled to media data and is usually "descriptive.

In the present document, several technical solutions are provided to allow representation of levels of details of a point cloud data, such as the G-PCC data of MPEG, into a format that is compatible with the traditional 2D video formats such as the MP4 or the ISOBMFF format. One advantageous aspect of the proposed solutions is to be able to reuse traditional 2D video techniques and syntax for implementation of the new functionality.

For G-PCC point cloud data that supports spatial scalability, the decoder can decode part of the 3D point cloud data belonging to one or more levels of details. The method of delivering, decoding, and reconstructing 3D point cloud data may include the following steps:.

In this embodiment, a scalable G-PCC bitstream is represented by a single track in a file. Each level of G-PCC data is signaled by sub-sample structure.

<FIG> shows an example of syntax used by a G-PCC track, showing a sample entry portion and a sample portion (multiple sample portions are possible). The sample portion includes geometry data units of multiple levels of details level0 to level L, and attribute data units corresponding to level <NUM> to level L.

The codec_specific_parameters in sub-sample box information is further extended to indicate information of levels of details for partial geometry data unit and a set of attribute data unit that corresponds to the specific LoD.

As shown in <FIG>, three alternative sub sample structures are listed as follows.

The codec_specific_parameters field of the SubsampleInformationBox is defined as below:
<IMG>
<IMG>.

PayloadType indicates the tlv_type of the TLV encapsulation structure or a part of that contained in the sub-sample.

NOTE: When PayloadType equals to <NUM> (geometry data unit), the boundary of one TLV structure can be identified as the boundary of a set of continuous sub-samples with the same PayloadType.

lifting_scalability_enabled_flag equal to <NUM> when attribute data in the sub-sample or the attribute data associated with geometry data in the sub-sample is encoded by the LoD with Lifting Transform with lifting scalability enabled. Otherwise lifting_scalability_enabled_flag equal to <NUM>.

is_geometry_data_unit_header equal to <NUM> when the sub-sample only contains geometry data unit header. Is_geometry_data_unit_header equal to <NUM> when the sub-sample contains part of geometry data unit data which belongs to the same LoD layer.

is_attribute _data_unit_header equal to <NUM> when the sub-sample only contains attribute data unit header. Is_attribute_data_unit_header equal to <NUM> when the sub-sample contains part of attractive data unit data which belongs to the same LoD layer.

lod indicates the maximum value of level of detail of the sub-sample, when the TLV encapsulation structure containing geometry payload or attribute payload is decoded from its first sub-sample to this sub-sample.

AttrIdx indicates the ash_attr_sps_attr_idx of the TLV encapsulation structure containing attribute payload in the sub-sample.

geometry_data_unit_header_exist equal to <NUM> when the sub-sample contains geometry data unit header, otherwise equal to <NUM>. geometry_data_unit_header_exist shall only be equal to <NUM> when the value of LoD is the smallest in the sample.

attribute_data_unit_header_exist equal to <NUM> when the sub-sample contains attribute data unit header, otherwise equal to <NUM>. attribute_data_unit_header_exist shall only be equal to <NUM> when the value of LoD is the smallest in the sample.

The codec_specific_parameters field of the SubsampleInformationBox is defined as below:
<IMG>.

GPCC_sub_type indicates the type of G-PCC data within a sub-sample. Table <NUM> describes the list of supported data type. Note that GPCC_sub_type <NUM>, <NUM>, <NUM> and <NUM> shall only be used when attribute data in the sub-sample or the attribute data associated with geometry data in the sub-sample is encoded by the LoD with Lifting Transform with lifting scalability enabled.

As shown in <FIG>, one implementation of this embodiment is that G-PCC geometry data and attribute data is carried in a single track and each sample in this track contains one or more G-PCC components with the complete levels of details. The division of each level is signaled in sub-sample structure.

In that case, GPCC tracks uses a Volumetric VisualSampleEntry with a sample entry type of 'gpea' or 'gpe1'. The sample entry of type 'gpe1' is further extended to indicate spatial scalability functionality of G-PCC data. When the decoder is to decode and render a point cloud with its LoD equals to L, then the sub-sample with LoD value less than L is obtained.

<FIG> shows single-track encapsulation of G-PCC data using sub-sample structure.

Under the 'gpea' sample entry, all levels of details of G-PCC geometry data and attribute data are stored in a single track, and information of each level is signaled in sub-sample.

GPCCConfigurationBox specifies the G-PCC decoder configuration information for geometry-based point cloud content. The setupUnit array shall include G-PCC TLV encapsulation structures that are constant for the stream referred to by the sample entry in which the decoder configuration record is present.

lod indicates the maximum value of level of detail of geometry data and attribute data in the track.

Under the 'gpe1' sample entry, information of levels of details of G-PCC geometry data and attribute data shall be signaled in sub-sample when the lifting_scalability_enabled_flag is equal to <NUM>. <IMG>
<IMG>.

As shown in <FIG>, another implementation of this embodiment is that G-PCC geometry data and attribute data is carried in separate tracks and each sample in a single track contains a single type of G-PCC components with the complete levels of details. The division of each level is signaled in sub-sample structure.

In that case, GPCC tracks may use VolumetricVisualSampleEntry with a sample entry type of 'gpcl' or 'gpc1'. The sample entry of type 'gpc1' is further extended to indicate spatial scalability functionality of G-PCC data. When the decoder is to decode and render a point cloud with its LoD equals to L, then the sub-sample with LoD value from <NUM> to L is obtained.

<FIG> shows an example of multi-track encapsulation of G-PCC data using sub-sample structure.

Under the 'gpca' sample entry, all levels of details of G-PCC geometry data are stored in a single track, and information of each level is signaled in a sub-sample. Furthermore, each type of attribute data is stored in separate tracks, the storage of LoD information is similar to that of geometry data.

GPCCComponentTypeBox indicates the type of G-PCC component carried in this track.

lod indicates the maximum value of level of detail of G-PCC component in the track.

Under the 'gpc1' sample entry, information of levels of details of G-PCC geometry data and attribute data may be signaled in sub-sample when the lifting_scalability_enabled_flag is equal to <NUM>. <IMG>
<IMG>.

In this embodiment, a scalable G-PCC bitstream is represented by one or more tracks in a file. Each track represents a level of details of the scalable stream. In this case, the decoding process of higher levels of G-PCC data requires data with lower LoD values. Thus, different tracks may logically share data using corresponding Extractors.

In this embodiment, a set of one or more tracks that, when taken together, contain the complete set of encoded point cloud data. Let the lowest operating point be the one of all the operating points represented by level of details that has the least value of LoD. Tracks with higher level of details may be linked to lower tracks by means of a track reference of type 'scal' (scalable). The complete encoded information can be retained when the tracks included in the complete track group are retained.

In ISO/IEC <NUM>-<NUM>, extractors enable compact formation of tracks that extract, by reference, network abstraction layer (NAL) unit data from other tracks. Similarly, an extractor in a G-PCC bitstream is defined to enable a compact formation of tracks that extract, by reference, geometry data unit or attribute data unit from other tracks. When an extractor is processed by a file reader that acquires it, the extractor is logically replaced by the bytes it references. An extractor may contain one or more constructors for extracting data from another track that is linked to the track in which the extractor resides by means of a track reference of type 'scal'. Extractor extends the syntax structure type of TLV as follows.

constructor_type specifies the number of constructor that follows.

track_ref_index specifies the index of the track reference of type 'scal' to use to find the track from which to extract data. The sample in that track from which data is extracted is temporally aligned or nearest preceding in the media decoding timeline, i.e. using the time-to-sample table only, adjusted by an offset specified by sample_offset with the sample containing the Extractor. The first track reference has the index value <NUM>; the value <NUM> is reserved.

sample_offset gives the relative index of the sample in the linked track that shall be used as the source of information. Sample <NUM> (zero) is the sample with the same, or the closest preceding, decoding time compared to the decoding time of the sample containing the extractor; sample <NUM> (one) is the next sample, sample -<NUM> (minus <NUM>) is the previous sample, and so on.

data_offset: The offset of the first byte within the reference sample to copy. If the extraction starts with the first byte of data in that sample, the offset takes the value <NUM>.

data_length: The number of bytes to copy. When data_offset + data_length is greater than the size of the sample, the bytes from the byte pointed to by data_offset until the end of the sample, inclusive, are copied.

As shown in <FIG>, a scalable G-PCC bitstream is represented by one or more tracks in a file. Each track represents one level of the scalable stream. In that case, when G-PCC geometry data and attribute data is carried in a single track, each sample in this track contains one or more G-PCC components with the same value of LoD.

TrackGroupTypeBox with track_group_type equal to 'cptg' indicates that this track is part of the same scalable G-PCC bitstream. The complete encoded information can be retained when the tracks included in the "complete subset" are retained.

<FIG> shows an example of a single-track encapsulation of G-PCC data with extractor.

GPCC tracks may use VolumetricVisualSampleEntry with a sample entry type of 'gpe1' or 'gpeg' or 'gpes'. Under the 'gpes' sample entry, all parameter sets (as defined in ISO/IEC <NUM>-<NUM> [GPCC]) may be in the setupUnit array.

Tracks contribute to the same point cloud data have the same value of track_group_id for track_group_type 'cptg', and the track_group_id of tracks from one point cloud data differs from the track_group_id of tracks from any other point cloud data.

lod_num indicates the maximum value of level of detail of a complete set of point cloud.

entry_id indicates the track_id of the entry point of a level, that is, a track with geometry data.

lod indicates the value of level of detail of the track with track_id equal to entry_id.

As shown in <FIG>, a scalable G-PCC bitstream is represented by one or more tracks in a file. Each track represents one operating point of the scalable stream. In this case, when G-PCC geometry data and attribute data are carried in multiple tracks, each geometry or attribute sub-stream is mapped to an individual track with the same value of LoD. The geometry track carries a geometry sub-stream and the attribute track carries a single type of the attribute sub-stream.

<FIG> shows an example of multi-track encapsulation of G-PCC data with an extractor.

GPCC tracks may use VolumetricVisualSampleEntry with a sample entry type of 'gpe1' or 'gpeg' or 'gpcs'. Under the 'gpcs' sample entry, all parameter sets (as defined in ISO/IEC <NUM>-<NUM> [GPCC]) may be in the setupUnit array.

A scalable G-PCC bitstream is represented by multiple tracks in a file. Each level of G-PCC data is signaled by sub-sample structure.

In this embodiment, the sub-sample information is used to indicate information of levels of details for partial geometry data unit and a set of attribute data unit that corresponds to the specific LoD.

As shown in <FIG>, one implementation of this embodiment is that G-PCC geometry data and attribute data is carried in a single track. Each sample in this track contains one or more G-PCC components with LoD value from <NUM> to the maximum LoD of this track. G-PCC data in each track may be decodable independently.

In this case, GPCC tracks may use VolumetricVisualSampleEntry with a sample entry type of 'gpei'.

<FIG> shows an example of a single-track encapsulation of G-PCC data with redundant levels.

Under the 'gpei' sample entry, each level of details of G-PCC geometry data and attribute data are stored in a single track.

In this case, GPCC tracks shall use VolumetricVisualSampleEntry with a sample entry type of 'gpci'. When the decoder is to decode and render a point cloud with its LoD equals to L, then the sub-sample with LoD value less than L is obtained.

<FIG> shows an example of multi-track encapsulation of G-PCC data with redundant levels.

Under the 'gpci' sample entry, all levels of details of G-PCC geometry data are stored in a single track, and information of each level is signaled in sub-sample. And for each type of attribute data stored in separate tracks, the storage of LoD information is similar to that of geometry data. <IMG>
<IMG>.

The single-track mode in DASH enables streaming of G-PCC ISOBMFF files where geometry stream and attribute stream are stored as a single-track. The single-track mode in DASH should be represented as one AdaptationSet with one or more Representations.

Geometry or attribute stream may be represented in the MPD file as a separate AdaptationSet, and an AdaptationSet with geometry stream serves as the entry point of the G-PCC stream.

If a G-PCC stream has multiple levels of details, each level may be signaled using a separate AdaptationSet, and the LoD related information is signaled by GPCCLoDInfo Descriptor.

To identify the spatial scalability and LoD information of the point cloud, a GPCCLoDInfo descriptor may be used. Using this descriptor indicates current stream support spatial scalability.

At most one GPCCLoDInfo descriptor may be present at the adaptation set level in the geometry AdaptationSet and each attribute AdaptationSet of the point cloud for the multi-track mode, or in the AdaptationSet of the point cloud for the single-track mode.

The GPCCLoDInfo descriptor may include elements and attributes as specified in Table <NUM>.

<FIG> is a diagram illustrating a first example device containing at least an example encoder for generating bitstreams that represent 3D content using one or more of the formats described in the present document. The encoder may include a video encoder or picture encoder.

Acquisition unit <NUM> captures a 3D scene, including a video or a picture. Acquisition unit <NUM> may be equipped with one or more cameras for shooting a video or a picture of nature scene. Optionally, acquisition unit <NUM> may be implemented with a camera to get depth video or depth picture. Optionally, acquisition unit <NUM> may include a component of an infrared camera. Optionally, acquisition unit <NUM> may be configured with a remote sensing camera. Acquisition unit <NUM> may also be an apparatus or a device of generating a video or a picture by scanning an object using radiation.

Optionally, acquisition unit <NUM> may perform pre-processing on video or picture, for example, automatic white balance, automatic focusing, automatic exposure, backlight compensation, sharpening, denoising, stitching, up-sampling/down sampling, frame-rate conversion, virtual view synthesis, and etc..

Acquisition unit <NUM> may also receive a video or picture from another device or processing unit. For example, acquisition unit <NUM> can be a component unit in a transcoder. The transcoder feeds one or more decoded (or partial decoded) pictures to acquisition unit <NUM>. Another example is that acquisition unit <NUM> get a video or picture from another device via a data link to that device.

Note that acquisition unit <NUM> may be used to capture other media information besides video and picture, for example, audio signal. Acquisition unit <NUM> may also receive artificial information, for example, character, text, computer-generated video or picture, and etc..

Encoder <NUM> is an implementation of the example encoder. Input of encoder <NUM> is the video or picture outputted by acquisition unit <NUM>. Encoder <NUM> encodes the video or picture and outputs generated a 3D (e.g., G-PCC) bitstream.

Storage/Sending unit <NUM> receives the video or picture bitstream from encoder <NUM> and performs system layer processing on the bitstream. For example, storage/sending unit <NUM> encapsulates the bitstream according to transport standard and media file format, for example, e.g. MPEG-<NUM> TS, ISOBMFF, DASH, MMT, and etc. Storage/Sending unit <NUM> stores the transport stream or media file obtained after encapsulation in memory or disk of the first example device, or sends the transport stream or media file via wireline or wireless networks.

Note that besides the video or picture bitstream from encoder <NUM>, input of storage/sending unit <NUM> may also include audio, text, image, texture, graphic, and etc. Storage/sending unit <NUM> generates a transport or media file by encapsulating such different types of media bitstreams.

The first example device described in this embodiment can be a device capable of generating or processing a video (or picture) bitstream in applications of video communication, for example, mobile phone, computer, media server, portable mobile terminal, digital camera, broadcasting device, CDN (content distribution network) device, surveillance camera, video conference device, and etc..

<FIG> is a diagram illustrating a second example device that may decode a bitstream having a format as disclosed in the present document to reconstruct a 3D scene. The example device may include at least a video decoder or a picture decoder.

Receiving unit <NUM> receives video or picture or G-PCC bitstream by obtaining bitstream from wireline or wireless network, by reading memory or disk in an electronic device, or by fetching data from other device via a data link.

Input of receiving unit <NUM> may also include transport stream or media file containing video or picture bitstream. Receiving unit <NUM> extracts video or picture bitstream from transport stream or media file according to specification of transport or media file format.

Receiving unit <NUM> outputs and passes video or picture bitstream to decoder <NUM>. Note that besides video or picture bitstream, output of receiving unit <NUM> may also include audio bitstream, character, text, image, graphic and etc. Receiving unit <NUM> passes the output to corresponding processing units in the second example device. For example, receiving unit <NUM> passes the output audio bitstream to audio decoder in this device.

Decoder <NUM> is an implementation of the example decoder. Input of encoder <NUM> is the video or picture bitstream outputted by receiving unit <NUM>. Decoder <NUM> decodes the video or picture bitstream and outputs decoded video or picture.

Rendering unit <NUM> receives the decoded video or picture from decoder <NUM>. Rendering unit <NUM> presents the decoded video or picture to viewer. Rendering unit <NUM> may be a component of the second example device, for example, a screen. Rendering unit <NUM> may also be a separate device from the second example device with a data link to the second example device, for example, projector, monitor, TV set, and etc. Optionally, rendering <NUM> performs post-processing on the decoded video or picture before presenting it to viewer, for example, automatic white balance, automatic focusing, automatic exposure, backlight compensation, sharpening, denoising, stitching, up-sampling/down sampling, frame-rate conversion, virtual view synthesis, and etc..

Note that besides decoded video or picture, input of rendering unit <NUM> can be other media data from one or more units of the second example device, for example, audio, character, text, image, graphic, and etc. Input of rendering unit <NUM> may also include artificial data, for example, lines and marks drawn by a local teacher on slides for attracting attention in remote education application. Rendering unit <NUM> composes the different types of media together and then presented the composition to viewer.

The second example device described in this embodiment can be a device capable of decoding or processing a video (or picture) bitstream in applications of video communication, for example, mobile phone, computer, set-top box, TV set, HMD, monitor, media server, portable mobile terminal, digital camera, broadcasting device, CDN (content distribution network) device, surveillance, video conference device, and etc..

<FIG> is a diagram illustrating an electronic system containing the first example device in <FIG> and the second example device in <FIG>.

Service device <NUM> is the first example device in <FIG>.

Storage medium / transport networks <NUM> may include internal memory resource of a device or electronic system, external memory resource that is accessible via a data link, data transmission network consisting of wireline and/or wireless networks. Storage medium / transport networks <NUM> provides storage resource or data transmission network for storage/sending unit <NUM> in service device <NUM>.

Destination device <NUM> is the second example device in <FIG>. Receiving unit <NUM> in destination device <NUM> receives a video or picture bitstream, a transport stream containing video or picture bitstream or a media file containing video or picture bitstream from storage medium / transport networks <NUM>.

The electronic system described in this embodiment can be a device or system capable of generating, storing or transporting, and decoding a video (or picture) bitstream in applications of video communication, for example, mobile phone, computer, IPTV systems, OTT systems, multimedia systems on Internet, digital TV broadcasting system, video surveillance system, potable mobile terminal, digital camera, video conference systems, and etc..

<FIG> shows an example apparatus <NUM> that may be used to implement encoder-side or decoder-side techniques described in the present document. The apparatus <NUM> includes a processor <NUM> that may be configured to perform the encoder-side or decoder-side techniques or both. The apparatus <NUM> may also include a memory (not shown) for storing processor-executable instructions and for storing the video bitstream and/or display data. The apparatus <NUM> may include video processing circuitry (not shown), such as transform circuits, arithmetic coding/decoding circuits, look-up table based data coding techniques and so on. The video processing circuitry may be partly included in the processor and/or partly in other dedicated circuitry such as graphics processors, field programmable gate arrays (FPGAs) and so on. Other hardware details such as peripherals used for capturing or rendering 3D content are also omitted from <FIG> for brevity.

The volumetric visual media data encoding or decoding apparatus may be implemented as a part of a computer, a user device such as a laptop, a tablet or a gaming device.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment.

Claim 1:
A method of processing three-dimensional content, comprising:
parsing a level of detail, LoD, information of a bitstream containing three-dimensional, 3D, content that is represented as one geometry sub-bitstream and one or more attribute sub-bitstreams, wherein the parsing the LoD information comprises identifying a first syntax structure in the bitstream that includes multiple levels of details;
generating, based on the LoD information, decoded information by decoding at least a portion of the geometry sub-bitstream and the one or more attribute sub-bitstreams corresponding to a desired level of detail;
reconstructing, using the decoded information, a three-dimensional scene corresponding at least to the desired level of detail,
wherein the bitstream conforms to a format organized according to multiple levels of details of the 3D content; and
using a sample entry type field in the bitstream for determining whether the bitstream supports a spatial scalability functionality and for the identifying the first syntax structure, wherein the first syntax structure of the bitstream with multiple levels of details comprises:
a first structure in which a complete set of levels of the bitstream is carried in one track with sub-sample structure;
a second structure that includes an extractor with each level of the bitstream in the one track; and
a third structure that includes one or more levels of the bitstream in the one track with redundant data from lower levels.