Patent Description:
A point cloud is a collection of points in a <NUM>-dimensional space. The points may correspond to points on objects within the <NUM>-dimensional space. Thus, a point cloud may be used to represent the physical content of the <NUM>-dimensional space. Point clouds may have utility in a wide variety of situations. For example, point clouds may be used in the context of autonomous vehicles for representing the positions of objects on a roadway. In another example, point clouds may be used in the context of representing the physical content of an environment for purposes of positioning virtual objects in an augmented reality (AR) or mixed reality (MR) application. Point cloud compression is a process for encoding and decoding point clouds. Encoding point clouds may reduce the amount of data required for storage and transmission of point clouds. The article titled "<NPL>), describes Geometry-based Point Cloud Compression and future enhancements thereto.

Further embodiment are provided by the description. In general, this disclosure describes techniques for coding nodes of a point cloud, such as for the Geometry Point Cloud Compression (G-PCC) standard currently being developed. However, the example techniques are not limited to the G-PCC standard. In some examples of G-PCC, coordinates of position of nodes (also referred to as points) of a point cloud may converted into a (r, ϕ, i) domain in which a position of a node is represented by three parameters, a radius r, an azimuth ϕ, and a laser index i. When using an angular mode for predictive geometry coding in G-PCC, a G-PCC coder may perform prediction in the (r, ϕ, i) domain. For instance, the G-PCC coder may determine a predicted position of a node and add the predicted position of the node to primary residual data to determine a reconstructed position of the node. Accordingly, in at least some examples, the primary residuals may be coded in the (r, ϕ, i) domain. Due to the errors in rounding (e.g., for coordinate conversion), coding in r, ϕ, i may be lossy. In some examples, this loss may by reduced or eliminated by coding a second set of residuals, which may be in a Cartesian domain. However, some implementations of G-PCC may require a lot of context coded bins to signal the primary and secondary residuals, which may be undesirably computationally intensive.

<FIG> is a block diagram illustrating an example encoding and decoding system <NUM> that may perform the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and/or decoding) point cloud data, i.e., to support point cloud compression. In general, point cloud data includes any data for processing a point cloud. The coding may be effective in compressing and/or decompressing point cloud data.

As shown in <FIG>, system <NUM> includes a source device <NUM> and a destination device <NUM>. Source device <NUM> provides encoded point cloud data to be decoded by a destination device <NUM>. Particularly, in the example of <FIG>, source device <NUM> provides the point cloud data to destination device <NUM> via a computer-readable medium <NUM>. Source device <NUM> and destination device <NUM> may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, terrestrial or marine vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, or the like. In some cases, source device <NUM> and destination device <NUM> may be equipped for wireless communication.

In the example of <FIG>, source device <NUM> includes a data source <NUM>, a memory <NUM>, a G-PCC encoder <NUM>, and an output interface <NUM>. Destination device <NUM> includes an input interface <NUM>, a G-PCC decoder <NUM>, a memory <NUM>, and a data consumer <NUM>. In accordance with this disclosure, G-PCC encoder <NUM> of source device <NUM> and G-PCC decoder <NUM> of destination device <NUM> may be configured to apply the techniques of this disclosure related to predictive geometry coding. Thus, source device <NUM> represents an example of an encoding device, while destination device <NUM> represents an example of a decoding device. In other examples, source device <NUM> and destination device <NUM> may include other components or arrangements. For example, source device <NUM> may receive data (e.g., point cloud data) from an internal or external source. Likewise, destination device <NUM> may interface with an external data consumer, rather than include a data consumer in the same device.

System <NUM> as shown in <FIG> is merely one example. In general, other digital encoding and/or decoding devices may perform the techniques of this disclosure related to predictive geometry coding. Source device <NUM> and destination device <NUM> are merely examples of such devices in which source device <NUM> generates coded data for transmission to destination device <NUM>. This disclosure refers to a "coding" device as a device that performs coding (encoding and/or decoding) of data. Thus, G-PCC encoder <NUM> and G-PCC decoder <NUM> represent examples of coding devices, in particular, an encoder and a decoder, respectively. In some examples, source device <NUM> and destination device <NUM> may operate in a substantially symmetrical manner such that each of source device <NUM> and destination device <NUM> includes encoding and decoding components. Hence, system <NUM> may support one-way or two-way transmission between source device <NUM> and destination device <NUM>, e.g., for streaming, playback, broadcasting, telephony, navigation, and other applications.

In general, data source <NUM> represents a source of data (i.e., raw, unencoded point cloud data) and may provide a sequential series of "frames") of the data to G-PCC encoder <NUM>, which encodes data for the frames. Data source <NUM> of source device <NUM> may include a point cloud capture device, such as any of a variety of cameras or sensors, e.g., a 3D scanner or a light detection and ranging (LIDAR) device, one or more video cameras, an archive containing previously captured data, and/or a data feed interface to receive data from a data content provider. Alternatively or additionally, point cloud data may be computer-generated from scanner, camera, sensor or other data. For example, data source <NUM> may generate computer graphics-based data as the source data, or produce a combination of live data, archived data, and computer-generated data. In each case, G-PCC encoder <NUM> encodes the captured, pre-captured, or computer-generated data. G-PCC encoder <NUM> may rearrange the frames from the received order (sometimes referred to as "display order") into a coding order for coding. G-PCC encoder <NUM> may generate one or more bitstreams including encoded data. Source device <NUM> may then output the encoded data via output interface <NUM> onto computer-readable medium <NUM> for reception and/or retrieval by, e.g., input interface <NUM> of destination device <NUM>.

Memory <NUM> of source device <NUM> and memory <NUM> of destination device <NUM> may represent general purpose memories. In some examples, memory <NUM> and memory <NUM> may store raw data, e.g., raw data from data source <NUM> and raw, decoded data from G-PCC decoder <NUM>. Additionally or alternatively, memory <NUM> and memory <NUM> may store software instructions executable by, e.g., G-PCC encoder <NUM> and G-PCC decoder <NUM>, respectively. Although memory <NUM> and memory <NUM> are shown separately from G-PCC encoder <NUM> and G-PCC decoder <NUM> in this example, it should be understood that G-PCC encoder <NUM> and G-PCC decoder <NUM> may also include internal memories for functionally similar or equivalent purposes. Furthermore, memory <NUM> and memory <NUM> may store encoded data, e.g., output from G-PCC encoder <NUM> and input to G-PCC decoder <NUM>. In some examples, portions of memory <NUM> and memory <NUM> may be allocated as one or more buffers, e.g., to store raw, decoded, and/or encoded data. For instance, memory <NUM> and memory <NUM> may store data representing a point cloud.

Computer-readable medium <NUM> may represent any type of medium or device capable of transporting the encoded data from source device <NUM> to destination device <NUM>. In one example, computer-readable medium <NUM> represents a communication medium to enable source device <NUM> to transmit encoded data directly to destination device <NUM> in real-time, e.g., via a radio frequency network or computer-based network. Output interface <NUM> may modulate a transmission signal including the encoded data, and input interface <NUM> may demodulate the received transmission signal, according to a communication standard, such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from source device <NUM> to destination device <NUM>.

In some examples, source device <NUM> may output encoded data from output interface <NUM> to storage device <NUM>. Similarly, destination device <NUM> may access encoded data from storage device <NUM> via input interface <NUM>. Storage device <NUM> may include any of a variety of distributed or locally accessed data storage media such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded data.

In some examples, source device <NUM> may output encoded data to file server <NUM> or another intermediate storage device that may store the encoded data generated by source device <NUM>. Destination device <NUM> may access stored data from file server <NUM> via streaming or download. File server <NUM> may be any type of server device capable of storing encoded data and transmitting that encoded data to the destination device <NUM>. File server <NUM> may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device <NUM> may access encoded data from file server <NUM> through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both that is suitable for accessing encoded data stored on file server <NUM>. File server <NUM> and input interface <NUM> may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

Output interface <NUM> and input interface <NUM> may represent wireless transmitters/receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components that operate according to any of a variety of IEEE <NUM> standards, or other physical components. In examples where output interface <NUM> and input interface <NUM> comprise wireless components, output interface <NUM> and input interface <NUM> may be configured to transfer data, such as encoded data, according to a cellular communication standard, such as <NUM>, <NUM>-LTE (Long-Term Evolution), LTE Advanced, <NUM>, or the like. In some examples where output interface <NUM> comprises a wireless transmitter, output interface <NUM> and input interface <NUM> may be configured to transfer data, such as encoded data, according to other wireless standards, such as an IEEE <NUM> specification, an IEEE <NUM> specification (e.g., ZigBee™), a Bluetooth™ standard, or the like. In some examples, source device <NUM> and/or destination device <NUM> may include respective system-on-a-chip (SoC) devices. For example, source device <NUM> may include an SoC device to perform the functionality attributed to G-PCC encoder <NUM> and/or output interface <NUM>, and destination device <NUM> may include an SoC device to perform the functionality attributed to G-PCC decoder <NUM> and/or input interface <NUM>.

The techniques of this disclosure may be applied to encoding and decoding in support of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping, or other applications.

Input interface <NUM> of destination device <NUM> receives an encoded bitstream from computer-readable medium <NUM> (e.g., a communication medium, storage device <NUM>, file server <NUM>, or the like). The encoded bitstream may include signaling information defined by G-PCC encoder <NUM>, which is also used by G-PCC decoder <NUM>, such as syntax elements having values that describe characteristics and/or processing of coded units (e.g., slices, pictures, groups of pictures, sequences, or the like). Data consumer <NUM> uses the decoded data. For example, data consumer <NUM> may use the decoded data to determine the locations of physical objects. In some examples, data consumer <NUM> may comprise a display to present imagery based on a point cloud.

G-PCC encoder <NUM> and G-PCC decoder <NUM> each may be implemented as any of a variety of suitable encoder and/or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof. Each of G-PCC encoder <NUM> and G-PCC decoder <NUM> may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device. A device including G-PCC encoder <NUM> and/or G-PCC decoder <NUM> may comprise one or more integrated circuits, microprocessors, and/or other types of devices.

G-PCC encoder <NUM> and G-PCC decoder <NUM> may operate according to a coding standard, such as video point cloud compression (V-PCC) standard or a geometry point cloud compression (G-PCC) standard. This disclosure may generally refer to coding (e.g., encoding and decoding) of pictures to include the process of encoding or decoding data. An encoded bitstream generally includes a series of values for syntax elements representative of coding decisions (e.g., coding modes).

This disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values for syntax elements and/or other data used to decode encoded data. That is, G-PCC encoder <NUM> may signal values for syntax elements in the bitstream. In general, signaling refers to generating a value in the bitstream. As noted above, source device <NUM> may transport the bitstream to destination device <NUM> substantially in real time, or not in real time, such as might occur when storing syntax elements to storage device <NUM> for later retrieval by destination device <NUM>.

ISO/IEC MPEG (JTC <NUM>/SC <NUM>/WG <NUM>) is studying the potential need for standardization of point cloud coding technology with a compression capability that significantly exceeds that of the current approaches and will target to create the standard. The group is working together on this exploration activity in a collaborative effort known as the <NUM>-Dimensional Graphics Team (3DG) to evaluate compression technology designs proposed by their experts in this area.

Point cloud compression activities are categorized in two different approaches. The first approach is "Video point cloud compression" (V-PCC), which segments the 3D object, and project the segments in multiple 2D planes (which are represented as "patches" in the 2D frame), which are further coded by a legacy 2D video codec such as a High Efficiency Video Coding (HEVC) (ITU-T H. <NUM>) codec. The second approach is "Geometry-based point cloud compression" (G-PCC), which directly compresses 3D geometry i.e., position of a set of points in 3D space, and associated attribute values (for each point associated with the 3D geometry). G-PCC addresses the compression of point clouds in both Category <NUM> (static point clouds) and Category <NUM> (dynamically acquired point clouds). A recent draft of the G-PCC standard is available in G-PCC DIS, ISO/IEC JTC1/SC29/WG11 w19328, Brussels, Belgium, January <NUM>, and a description of the codec is available in G-PCC Codec Description v8, ISO/IEC JTC1/SC29/WG11 w19525, Brussels, Belgium, January <NUM>.

A point cloud contains a set of points in a 3D space, and may have attributes associated with the point. The attributes may be color information such as R, G, B or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds may be captured by a variety of cameras or sensors such as LIDAR sensors and 3D scanners and may also be computer-generated. Point cloud data are used in a variety of applications including, but not limited to, construction (modeling), graphics (3D models for visualizing and animation), and the automotive industry (LIDAR sensors used to help in navigation).

The 3D space occupied by a point cloud data may be enclosed by a virtual bounding box. The position of the points in the bounding box may be represented by a certain precision; therefore, the positions of one or more points may be quantized based on the precision. At the smallest level, the bounding box is split into voxels which are the smallest unit of space represented by a unit cube. A voxel in the bounding box may be associated with zero, one, or more than one point. The bounding box may be split into multiple cube/cuboid regions, which may be called tiles. Each tile may be coded into one or more slices. The partitioning of the bounding box into slices and tiles may be based on number of points in each partition, or based on other considerations (e.g., a particular region may be coded as tiles). The slice regions may be further partitioned using splitting decisions similar to those in video codecs.

<FIG> provides an overview of G-PCC encoder <NUM>. <FIG> provides an overview of G-PCC decoder <NUM>. The modules shown are logical, and do not necessarily correspond one-to-one to implemented code in the reference implementation of G-PCC codec, i.e., TMC13 test model software studied by ISO/IEC MPEG (JTC <NUM>/SC <NUM>/WG <NUM>).

In both G-PCC encoder <NUM> and G-PCC decoder <NUM>, point cloud positions are coded first. Attribute coding depends on the decoded geometry. In <FIG> and <FIG>, the gray-shaded modules are options typically used for Category <NUM> data. Diagonal-crosshatched modules are options typically used for Category <NUM> data. All the other modules are common between Categories <NUM> and <NUM>.

For Category <NUM> data, the compressed geometry is typically represented as an octree from the root all the way down to a leaf level of individual voxels. For Category <NUM> data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root down to a leaf level of blocks larger than voxels) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both Category <NUM> and <NUM> data share the octree coding mechanism, while Category <NUM> data may in addition approximate the voxels within each leaf with a surface model. The surface model used is a triangulation comprising <NUM>-<NUM> triangles per block, resulting in a triangle soup. The Category <NUM> geometry codec is therefore known as the Trisoup geometry codec, while the Category <NUM> geometry codec is known as the Octree geometry codec.

At each node of an octree, an occupancy is signaled (when not inferred) for one or more of its child nodes (up to eight nodes). Multiple neighborhoods are specified including (a) nodes that share a face with a current octree node, (b) nodes that share a face, edge or a vertex with the current octree node, etc. Within each neighborhood, the occupancy of a node and/or its children may be used to predict the occupancy of the current node or its children. For points that are sparsely populated in certain nodes of the octree, the codec also supports a direct coding mode where the 3D position of the point is encoded directly. A flag may be signaled to indicate that a direct mode is signaled. At the lowest level, the number of points associated with the octree node/leaf node may also be coded.

Once the geometry is coded, the attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed/decoded geometry point, an attribute value may be derived that is representative of the reconstructed point.

There are three attribute coding methods in G-PCC: Region Adaptive Hierarchical Transform (RAHT) coding, interpolation-based hierarchical nearest-neighbour prediction (Predicting Transform), and interpolation-based hierarchical nearest-neighbour prediction with an update/lifting step (Lifting Transform). RAHT and Lifting are typically used for Category <NUM> data, while Predicting is typically used for Category <NUM> data. However, either method may be used for any data, and, just like with the geometry codecs in G-PCC, the attribute coding method used to code the point cloud is specified in the bitstream.

The coding of the attributes may be conducted in a level-of-detail (LOD), where with each level of detail a finer representation of the point cloud attribute may be obtained. Each level of detail may be specified based on distance metric from the neighboring nodes or based on a sampling distance.

At G-PCC encoder <NUM>, the residuals obtained as the output of the coding methods for the attributes are quantized. The quantized residuals may be coded using context adaptive arithmetic coding.

In the example of <FIG>, G-PCC encoder <NUM> may include a coordinate transform unit <NUM>, a color transform unit <NUM>, a voxelization unit <NUM>, an attribute transfer unit <NUM>, an octree analysis unit <NUM>, a surface approximation analysis unit <NUM>, an arithmetic encoding unit <NUM>, a geometry reconstruction unit (GRU) <NUM>, an RAHT unit <NUM>, a LOD generation unit <NUM>, a lifting unit <NUM>, a coefficient quantization unit <NUM>, and an arithmetic encoding unit <NUM>.

As shown in the example of <FIG>, G-PCC encoder <NUM> may receive a set of positions and a set of attributes. The positions may include coordinates of points in a point cloud. The attributes may include information about points in the point cloud, such as colors associated with points in the point cloud.

Coordinate transform unit <NUM> may apply a transform to the coordinates of the points to transform the coordinates from an initial domain to a transform domain. This disclosure may refer to the transformed coordinates as transform coordinates. Color transform unit <NUM> may apply a transform to transform color information of the attributes to a different domain. For example, color transform unit <NUM> may transform color information from an RGB color space to a YCbCr color space.

Furthermore, in the example of <FIG>, voxelization unit <NUM> may voxelize the transform coordinates. Voxelization of the transform coordinates may include quantization and removing some points of the point cloud. In other words, multiple points of the point cloud may be subsumed within a single "voxel," which may thereafter be treated in some respects as one point. Furthermore, octree analysis unit <NUM> may generate an octree based on the voxelized transform coordinates. Additionally, in the example of <FIG>, surface approximation analysis unit <NUM> may analyze the points to potentially determine a surface representation of sets of the points. Arithmetic encoding unit <NUM> may entropy encode syntax elements representing the information of the octree and/or surfaces determined by surface approximation analysis unit <NUM>. G-PCC encoder <NUM> may output these syntax elements in a geometry bitstream.

Geometry reconstruction unit <NUM> may reconstruct transform coordinates of points in the point cloud based on the octree, data indicating the surfaces determined by surface approximation analysis unit <NUM>, and/or other information. The number of transform coordinates reconstructed by geometry reconstruction unit <NUM> may be different from the original number of points of the point cloud because of voxelization and surface approximation. This disclosure may refer to the resulting points as reconstructed points. Attribute transfer unit <NUM> may transfer attributes of the original points of the point cloud to reconstructed points of the point cloud.

Furthermore, RAHT unit <NUM> may apply RAHT coding to the attributes of the reconstructed points. Alternatively or additionally, LOD generation unit <NUM> and lifting unit <NUM> may apply LOD processing and lifting, respectively, to the attributes of the reconstructed points. RAHT unit <NUM> and lifting unit <NUM> may generate coefficients based on the attributes. Coefficient quantization unit <NUM> may quantize the coefficients generated by RAHT unit <NUM> or lifting unit <NUM>. Arithmetic encoding unit <NUM> may apply arithmetic coding to syntax elements representing the quantized coefficients. G-PCC encoder <NUM> may output these syntax elements in an attribute bitstream.

In the example of <FIG>, G-PCC decoder <NUM> may include a geometry arithmetic decoding unit <NUM>, an attribute arithmetic decoding unit <NUM>, an octree synthesis unit <NUM>, an inverse quantization unit <NUM>, a surface approximation synthesis unit <NUM>, a geometry reconstruction unit <NUM>, a RAHT unit <NUM>, a LoD generation unit <NUM>, an inverse lifting unit <NUM>, an inverse transform coordinate unit <NUM>, and an inverse transform color unit <NUM>.

G-PCC decoder <NUM> may obtain a geometry bitstream and an attribute bitstream. Geometry arithmetic decoding unit <NUM> of decoder <NUM> may apply arithmetic decoding (e.g., Context-Adaptive Binary Arithmetic Coding (CABAC) or other type of arithmetic decoding) to syntax elements in the geometry bitstream. Similarly, attribute arithmetic decoding unit <NUM> may apply arithmetic decoding to syntax elements in the attribute bitstream.

Octree synthesis unit <NUM> may synthesize an octree based on syntax elements parsed from the geometry bitstream. In instances where surface approximation is used in the geometry bitstream, surface approximation synthesis unit <NUM> may determine a surface model based on syntax elements parsed from the geometry bitstream and based on the octree.

Furthermore, geometry reconstruction unit <NUM> may perform a reconstruction to determine coordinates of points in a point cloud. Inverse transform coordinate unit <NUM> may apply an inverse transform to the reconstructed coordinates to convert the reconstructed coordinates (positions) of the points in the point cloud from a transform domain back into an initial domain.

Additionally, in the example of <FIG>, inverse quantization unit <NUM> may inverse quantize attribute values. The attribute values may be based on syntax elements obtained from the attribute bitstream (e.g., including syntax elements decoded by attribute arithmetic decoding unit <NUM>).

Depending on how the attribute values are encoded, RAHT unit <NUM> may perform RAHT coding to determine, based on the inverse quantized attribute values, color values for points of the point cloud. Alternatively, LOD generation unit <NUM> and inverse lifting unit <NUM> may determine color values for points of the point cloud using a level of detail-based technique.

Furthermore, in the example of <FIG>, inverse transform color unit <NUM> may apply an inverse color transform to the color values. The inverse color transform may be an inverse of a color transform applied by color transform unit <NUM> of encoder <NUM>. For example, color transform unit <NUM> may transform color information from an RGB color space to a YCbCr color space. Accordingly, inverse color transform unit <NUM> may transform color information from the YCbCr color space to the RGB color space.

The various units of <FIG> and <FIG> are illustrated to assist with understanding the operations performed by encoder <NUM> and decoder <NUM>. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide particular functionality, and are preset on the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks, and provide flexible functionality in the operations that can be performed. For instance, programmable circuits may execute software or firmware that cause the programmable circuits to operate in the manner defined by instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations that the fixed-function circuits perform are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

Predictive geometry coding was introduced as an alternative to the octree geometry coding, where the nodes are arranged in a tree structure (which defines the prediction structure), and various prediction strategies are used to predict the coordinates of each node in the tree with respect to its predictors. <FIG> is a conceptual diagram illustrating an example of a prediction tree, a directed graph where the arrow points to the prediction direction. The horizontally shaded node is the root vertex and has no predictors; the grid shaded nodes have two children; the diagonally shaded node has <NUM> children, the non-shaded nodes have one children and the vertically shaded nodes are leaf nodes and these have no children. Every node has only one parent node.

Four prediction strategies may be are specified for each node based on its parent (p0), grand-parent (p1) and great-grand-parent (p2). The prediction strategies include, no prediction, delta prediction (p0), linear prediction (<NUM>*p0 - p1), and parallelogram prediction (<NUM>*p0 + p1 - p2).

The encoder (e.g., G-PCC encoder <NUM>) may employ any algorithm to generate the prediction tree; the algorithm used may be determined based on the application/use case and several strategies may be used. The encoder may encode, for each node, the residual coordinate values in the bitstream starting from the root node in a depth-first manner. Predictive geometry coding may be particularly useful for Category <NUM> (e.g., LIDAR-acquired) point cloud data e.g., for low-latency applications.

Angular mode may be used in predictive geometry coding, where the characteristics of LIDAR sensors may be utilized in coding the prediction tree more efficiently. The coordinates of the positions are converted to the (r, ϕ, i) (radius, azimuth, and laser index) and a prediction is performed in this domain (the residuals are coded in r, ϕ, i domain). Due to the errors in rounding, coding in r, ϕ, i is not lossless and hence a second set of residuals may be coded which correspond to the Cartesian coordinates. A description of the encoding and decoding strategies used for angular mode for predictive geometry coding is reproduced below. The description is based on <FIG>, which are conceptual diagrams of a spinning LIDAR acquisition model.

The method focuses on point clouds acquired using a spinning LIDAR model. Here, the LIDAR has N lasers (e.g., N=<NUM>, <NUM>, <NUM>) spinning around the Z axis according to an azimuth angle ϕ (see <FIG>). Each laser may have different elevation θ(i)i=<NUM>. N and height ς(i)i=<NUM>. Supposing that the laser i hits a point M, with cartesian integer coordinates (x, y, z), defined according to the coordinate system described in <FIG>.

The position of M is modelled with three parameters (r, ϕ, i), which are computed as follows:.

More precisely, a G-PCC coder may use the quantized version of (r, ϕ, i), denoted (r̃, ϕ̃, i), where the three integers r̃, ϕ̃ and i are computed as follows:.

To avoid reconstruction mismatches due to the use of floating-point operations, the values of <IMG>(i)i=<NUM>. N and tan(θ(i))i=<NUM>. N may be pre-computed and quantized as follows: <MAT> <MAT> where.

The reconstructed cartesian coordinates are obtained as follows:.

where app_cos(. ) and app_sin(. ) are approximation of cos(. ) and sin(. The calculations could be using a fixed-point representation, a look-up table and linear interpolation.

Note that (x̂, ŷ, ẑ) may be different from (x, y, z) due to various reasons:.

Let (rx, ry, rz) be the reconstruction residuals defined as follows: <MAT> <MAT> <MAT>.

In this method, G-PCC encoder <NUM> may proceed as follows:.

G-PCC decoder <NUM> may proceed as follows:.

Lossy compression could be achieved by applying quantization to the reconstruction residuals (rx, ry, rz) or by dropping points.

The quantized reconstruction residuals are computed as follows:.

Where (qx, ox), (qy, oy)and (qz, oz) are quantization parameters controlling the precision of r̃x, r̃y and r̃z, respectively.

In some examples, G-PCC encoder <NUM> and/or G-PCC decoder <NUM> may use Trellis quantization to further improve the RD (rate-distortion) performance results. The quantization parameters may change at sequence/frame/slice/block level to achieve region adaptive quality and for rate control purposes.

What follows is an example predictive geometry coding syntax, semantics, syntax binarization, and context table:.

When nodeIdx % PtnQpInterval is equal to <NUM>, the node QP for the next PtnQpInterval nodes in decoding order is determined as follows:
When geom_scaling_enabled_flag is equal to <NUM>: <MAT>.

Otherwise, PtnQp[ nodeIdx ] is set equal to <NUM>. ptn_point_cnt_gt1_flag and ptn_point_cnt_minus2 together specify the number of points represented by the current predictive tree node. When not present, the values of ptn_point_cnt_gt1_flag and ptn_point_cnt_minus2 are both inferred to be <NUM>. The number of points represented by the current predictive tree node is derived as follows: <MAT>.

ptn_child_cnt[ nodeIdx ] is the number of direct child nodes of the current predictive tree node present in the geometry predictive tree. ptn_pred_mode[ nodeIdx ] is a mode used to predict the position associated with the current node. ptn_phi_mult_eq0_flag, ptn_phi_mult_sign_flag, ptn_phi_mult_eq1_flag, ptn_phi_mult_abs_minus2, and ptn_phi_mult_abs_minus17 together specify a multiplicative factor used in delta angular prediction. ptn_phi_mult_eq0_flag, when present, specifies whether the factor is equal to zero. ptn_phi_mult_eq1_flag, when present, specifies whether the factor's magnitude is equal to one. ptn_phi_mult_sign_flag equal to <NUM> indicates that the factor's sign is positive. ptn_phi_mult_sign_flag equal to <NUM> indicates that the factor's sign is negative. Any of ptn_phi_mult_sign_flag, ptn_phi_mult_abs_minus2, or ptn_phi_mult_abs_minus17 that are not present are inferred to be <NUM>. Any of ptn_phi_mult_eq0_flag, or ptn_phi_mult_eq1_flag that are not present are inferred to be <NUM>. The phi factor for the current tree node is derived as follows: <MAT>.

ptn_residual_eq0_flag[ k ], ptn_residual_sign_flag[ k ], ptn_residual_abs_log2[ k ], and ptn_residual_abs_remaining[ k ] together specify the first prediction residual of the k-th geometry position component. ptn_residual_eq0_flag[ k ] specifies whether the residual component is equal to zero. ptn_residual_sign_flag[ k ] equal to <NUM> indicates that the sign of the residual component is positive. ptn_residual_sign_flag[ k ] equal to <NUM> indicates that the sign of the residual component is negative. Any of ptn_residual_sign_flag[ k ], ptn_residual_abs_log2[ k ], or ptn_residual_abs_remaining[ k ] that are not present are inferred to be <NUM>. The first prediction residual associated with the current tree node is derived as follows: <MAT>.

ptn_sec_residual_eq0_flag[ k ], ptn_sec_residual_eq1_flag[ k ], ptn_sec_residual_sign_flag[ k ], ptn_sec_residual_abs_minus2[ k ], and ptn_sec_residual_abs_minus17[ k ] together specify the secondary residual of the k-th geometry position component. ptn_sec_residual_eq0_flag[ k ] specifies whether the residual component is equal to zero. ptn_sec_residual_eq1_flag[ k ], when present, specifies whether the residual component magnitude is equal to one. ptn_sec_residual_sign_flag[ k ] equal to <NUM> indicates that the sign of the residual component is positive. ptn_sec_residual_sign_flag[ k ] equal to <NUM> indicates that the sign of the residual component is negative. Any of ptn_src_residual_sign_flag[ k], ptn_sec_residual_abs_minus2[ k ], or ptn_sec_residual_abs_minus17[ k ] that are not present are inferred to be <NUM>. Any of ptn_sec_residual_eq0_flag[ k], or ptn_sec_residual_eq1_flag[ k ] that are not present are inferred to be <NUM>. The second prediction residual associated with the current tree node is derived as follows: <MAT>.

The above implementation of predictive geometry coding may present one or more disadvantages. As one example, the above implementation of predictive geometry coding with angular coding mode may requires a lot of context coded bins, as the above implementation requires coding of both primary (r, ϕ, i) and secondary (rx, ry, rz) residuals, which is computationally intensive. As another example, for the signaling of predictor index and number of child signaling (both cases have total <NUM> candidates), the above implementation of predictive geometry coding uses <NUM> bit fixed length coding, which may not be optimal as the candidates are not equi-probable. As another example, for the primary residual in the above implementation of predictive geometry coding, the signaling associated with laser index (i) may be redundant, such as when the point count is captured using a single laser. As another example, in the above implementation of predictive geometry coding, zero-predictor (or "no prediction" described above) is very inefficient and it may rarely be used for the prediction. Moreover, for the "r" part, as it is non-negative, and the as prediction is always zero, the sign of the residual for "r" component can always be inferred, and hence the corresponding signaling is redundant.

In accordance with one or more techniques of this disclosure, an encoder (e.g., G-PCC encoder <NUM>) and/or a decoder (e.g., G-PCC decoder <NUM>) may overcome the aforementioned disadvantages. As a first example, the encoder and/or the decoder may perform a reduction of context and context coded bins for secondary residual and phi multiplier. As a second example, the encoder and/or the decoder may perform variable length coding for number of child and predictor index signalling. As a third example, the encoder and/or decoder may remove the signalling of laser index when content is captured by single laser. As a fourth example, the encoder and/or decoder may modify zero prediction by utilizing the azimuth and laser index of parent neighbor, and sign inference.

Some details examples of the techniques of this disclosure follow:
As discussed above, in accordance with the first example, the encoder and/or the decoder may perform a reduction of context and context coded bins for secondary residual and phi multiplier. For instance, the secondary residual coding may be simplified by removing ptn_sec_residual_abs_minus17[ k ] syntax element and associated <NUM>*<NUM> = <NUM> contexts. In some examples, the binarization and contexts for ptn_sec_residual_abs_minus2[ k] may be modified. Some example modifications are show below with additions in <ADD>. </ADD> tags and removals in <REMOVE>. </REMOVE> tags.

The second prediction residual associated with the current tree node is derived as follows: <MAT>.

Alternatively, the G_PCC coder may perform the signalling using a certain number of prefix contexts (e.g., a contexts) and suffix contexts (e.g., b contexts) for exponential Golomb coding. The use of prefix contexts and suffix context may be effective (e.g., provide coding gains) when there is a considerable energy in secondary residual components. In one example: a = <NUM>, b = <NUM>.

Secondly, the syntax associated with the phi multiplier is simplified by removing <NUM> contexts and modifying the signaling, details are as follows:.

The phi factor for the current tree node is derived as follows: <MAT>.

As discussed above, in accordance with the second example, the encoder and/or the decoder may perform variable length coding for number of child and predictor index signalling. Currently, predictor mode and Number of Child signaling information both signaled in <NUM> bits fixed length coding with <NUM> contexts with the following mapping:.

However, to account for the un-equiprobable symbol statistics, it is proposed to signal with a variable length binarization, for example, truncated unary binarization. It may reduce the overall number of bins associated with the signaling. As the statistics of predMode may be different depending on whether angular mode is applied, different mapping may be used depending on whether angular mode is enabled or not which is already signaled in the corresponding geometry parameter set.

As discussed above, in accordance with the third example, the encoder and/or decoder may remove the signalling (e.g., not encode and not decode) of laser index when content is captured by single laser. When num_lasers_minus1 is equal to <NUM>, laser index is the same for all the point in the point cloud. So, associated residual signaling for laser index may be redundant (i.e., the residual of third component is not needed to be signaled). The changes in the syntax is shown below considering the third component corresponds to laser index. (It may change if the axis conversion is present).

If the axis conversion is present, the component corresponds to laser index is not to be signaled.

Moreover, when num_lasers_minus1 is <NUM> (there are only two lasers), the residual can be either zero, -<NUM> or +<NUM>, however, here for the nonzero case, the sign and subsequent information can be inferred and does not need to be signaled. For example, let say, we have two lasers, with laserIdx <NUM> and <NUM>, then if the predicted laserIdx is <NUM>, and residual is nonzero, then it can be inferred that the current laserIdx = <NUM>. So, in that case, just signaling ptn_residual_eq0_flag[<NUM>] is sufficient. Accordingly, the following changes in the syntax may be implemented:.

As discussed above, in accordance with the fourth example, the encoder and/or decoder may modify zero prediction by utilizing the azimuth and laser index of parent neighbor, and sign inference.

Currently, zero prediction in the angular domain corresponds to all components equal to zero. However, the current implementation of zero prediction may be inefficient, and thusly is rarely used.

In accordance with one or more techniques of this disclosure, an encoder or decoder may inherit the azimuth and laser index from the parentNode (ancestors), if parentNode is available. So, in a way, similar to delta prediction, except radius is set to zero (or a minimum radius value).

The following changes in the position prediction process is needed (section <NUM>. <NUM>) [w19522]:
###################################################################### #######
When predMode is equal to <NUM>, the predicted point position is <REMOVE><NUM></REMOVE>: <MAT> When predMode is equal to <NUM>, the predicted point position is the position associated with the first ancestor. <MAT> When predMode is equal to <NUM>, the predicted point position is a linear combination of the positions associated with the first two ancestors <MAT> Otherwise, predMode is equal to <NUM>, the predicted point position is a linear combination of the positions associated with all three ancestors <MAT> ###################################################################### #######.

Secondly, when zero predictor is used, as the predicted radius is zero, the corresponding residual is inferred to be positive, accordingly, the corresponding sign do not need to be signaled, but to be inferred. The same also holds when angular mode is disabled.

Alternatively, the minimum value of radius (the minimum radius among all points) can be signaled in the slice header, which can be used as the radius for zero predictor instead of <NUM>.

To improve the zero predictor, phi multiplier can be applied to all predictor, not just delta predictor. The following changes in the syntax are:.

<FIG> is a conceptual diagram illustrating an example range-finding system <NUM> that may be used with one or more techniques of this disclosure. In the example of <FIG>, range-finding system <NUM> includes an illuminator <NUM> and a sensor <NUM>. Illuminator <NUM> may emit light <NUM>. In some examples, illuminator <NUM> may emit light <NUM> as one or more laser beams. Light <NUM> may be in one or more wavelengths, such as an infrared wavelength or a visible light wavelength. In other examples, light <NUM> is not coherent, laser light. When light <NUM> encounters an object, such as object <NUM>, light <NUM> creates returning light <NUM>. Returning light <NUM> may include backscattered and/or reflected light. Returning light <NUM> may pass through a lens <NUM> that directs returning light <NUM> to create an image <NUM> of object <NUM> on sensor <NUM>. Sensor <NUM> generates signals <NUM> based on image <NUM>. Image <NUM> may comprise a set of points (e.g., as represented by dots in image <NUM> of <FIG>).

In some examples, illuminator <NUM> and sensor <NUM> may be mounted on a spinning structure so that illuminator <NUM> and sensor <NUM> capture a <NUM>-degree view of an environment. In other examples, range-finding system <NUM> may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable illuminator <NUM> and sensor <NUM> to detect ranges of objects within a specific range (e.g., up to <NUM>-degrees). Although the example of <FIG> only shows a single illuminator <NUM> and sensor <NUM>, range-finding system <NUM> may include multiple sets of illuminators and sensors.

In some examples, illuminator <NUM> generates a structured light pattern. In such examples, range-finding system <NUM> may include multiple sensors <NUM> upon which respective images of the structured light pattern are formed. Range-finding system <NUM> may use disparities between the images of the structured light pattern to determine a distance to an object <NUM> from which the structured light pattern backscatters. Structured light-based range-finding systems may have a high level of accuracy (e.g., accuracy in the sub-millimeter range), when object <NUM> is relatively close to sensor <NUM> (e.g., <NUM> meters to <NUM> meters). This high level of accuracy may be useful in facial recognition applications, such as unlocking mobile devices (e.g., mobile phones, tablet computers, etc.) and for security applications.

In some examples, range-finding system <NUM> is a time of flight (ToF)-based system. In some examples where range-finding system <NUM> is a ToF-based system, illuminator <NUM> generates pulses of light. In other words, illuminator <NUM> may modulate the amplitude of emitted light <NUM>. In such examples, sensor <NUM> detects returning light <NUM> from the pulses of light <NUM> generated by illuminator <NUM>. Range-finding system <NUM> may then determine a distance to object <NUM> from which light <NUM> backscatters based on a delay between when light <NUM> was emitted and detected and the known speed of light in air). In some examples, rather than (or in addition to) modulating the amplitude of the emitted light <NUM>, illuminator <NUM> may modulate the phase of the emitted light <NUM>. In such examples, sensor <NUM> may detect the phase of returning light <NUM> from object <NUM> and determine distances to points on object <NUM> using the speed of light and based on time differences between when illuminator <NUM> generated light <NUM> at a specific phase and when sensor <NUM> detected returning light <NUM> at the specific phase.

In other examples, a point cloud may be generated without using illuminator <NUM>. For instance, in some examples, sensors <NUM> of range-finding system <NUM> may include two or more optical cameras. In such examples, range-finding system <NUM> may use the optical cameras to capture stereo images of the environment, including object <NUM>. Range-finding system <NUM> may include a point cloud generator <NUM> that may calculate the disparities between locations in the stereo images. Range-finding system <NUM> may then use the disparities to determine distances to the locations shown in the stereo images. From these distances, point cloud generator <NUM> may generate a point cloud.

Sensors <NUM> may also detect other attributes of object <NUM>, such as color and reflectance information. In the example of <FIG>, a point cloud generator <NUM> may generate a point cloud based on signals <NUM> generated by sensor <NUM>. Range-finding system <NUM> and/or point cloud generator <NUM> may form part of data source <NUM> (<FIG>). Hence, a point cloud generated by range-finding system <NUM> may be encoded and/or decoded according to any of the techniques of this disclosure.

<FIG> is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used. In the example of <FIG>, a vehicle <NUM> includes a range-finding system <NUM>. Range-finding system <NUM> may be implemented in the manner discussed with respect to <FIG>. Although not shown in the example of <FIG>, vehicle <NUM> may also include a data source, such as data source <NUM> (<FIG>), and a G-PCC encoder, such as G-PCC encoder <NUM> (<FIG>). In the example of <FIG>, range-finding system <NUM> emits laser beams <NUM> that reflect off pedestrians <NUM> or other objects in a roadway. The data source of vehicle <NUM> may generate a point cloud based on signals generated by range-finding system <NUM>. The G-PCC encoder of vehicle <NUM> may encode the point cloud to generate bitstreams <NUM>, such as geometry bitstream (<FIG>) and attribute bitstream (<FIG>). Bitstreams <NUM> may include many fewer bits than the unencoded point cloud obtained by the G-PCC encoder.

An output interface of vehicle <NUM> (e.g., output interface <NUM> (<FIG>) may transmit bitstreams <NUM> to one or more other devices. Bitstreams <NUM> may include many fewer bits than the unencoded point cloud obtained by the G-PCC encoder. Thus, vehicle <NUM> may be able to transmit bitstreams <NUM> to other devices more quickly than the unencoded point cloud data. Additionally, bitstreams <NUM> may require less data storage capacity.

In the example of <FIG>, vehicle <NUM> may transmit bitstreams <NUM> to another vehicle <NUM>. Vehicle <NUM> may include a G-PCC decoder, such as G-PCC decoder <NUM> (<FIG>). The G-PCC decoder of vehicle <NUM> may decode bitstreams <NUM> to reconstruct the point cloud. Vehicle <NUM> may use the reconstructed point cloud for various purposes. For instance, vehicle <NUM> may determine based on the reconstructed point cloud that pedestrians <NUM> are in the roadway ahead of vehicle <NUM> and therefore start slowing down, e.g., even before a driver of vehicle <NUM> realizes that pedestrians <NUM> are in the roadway. Thus, in some examples, vehicle <NUM> may perform an autonomous navigation operation based on the reconstructed point cloud.

Additionally or alternatively, vehicle <NUM> may transmit bitstreams <NUM> to a server system <NUM>. Server system <NUM> may use bitstreams <NUM> for various purposes. For example, server system <NUM> may store bitstreams <NUM> for subsequent reconstruction of the point clouds. In this example, server system <NUM> may use the point clouds along with other data (e.g., vehicle telemetry data generated by vehicle <NUM>) to train an autonomous driving system. In other example, server system <NUM> may store bitstreams <NUM> for subsequent reconstruction for forensic crash investigations (e.g., if vehicle <NUM> collides with pedestrians <NUM>).

<FIG> is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. Extended reality (XR) is a term used to cover a range of technologies that includes augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example of <FIG>, a user <NUM> is located in a first location <NUM>. User <NUM> wears an XR headset <NUM>. As an alternative to XR headset <NUM>, user <NUM> may use a mobile device (e.g., mobile phone, tablet computer, etc.). XR headset <NUM> includes a depth detection sensor, such as a range-finding system, that detects positions of points on objects <NUM> at location <NUM>. A data source of XR headset <NUM> may use the signals generated by the depth detection sensor to generate a point cloud representation of objects <NUM> at location <NUM>. XR headset <NUM> may include a G-PCC encoder (e.g., G-PCC encoder <NUM> of <FIG>) that is configured to encode the point cloud to generate bitstreams <NUM>.

XR headset <NUM> may transmit bitstreams <NUM> (e.g., via a network such as the Internet) to an XR headset <NUM> worn by a user <NUM> at a second location <NUM>. XR headset <NUM> may decode bitstreams <NUM> to reconstruct the point cloud. XR headset <NUM> may use the point cloud to generate an XR visualization (e.g., an AR, MR, VR visualization) representing objects <NUM> at location <NUM>. Thus, in some examples, such as when XR headset <NUM> generates an VR visualization, user <NUM> may have a 3D immersive experience of location <NUM>. In some examples, XR headset <NUM> may determine a position of a virtual object based on the reconstructed point cloud. For instance, XR headset <NUM> may determine, based on the reconstructed point cloud, that an environment (e.g., location <NUM>) includes a flat surface and then determine that a virtual object (e.g., a cartoon character) is to be positioned on the flat surface. XR headset <NUM> may generate an XR visualization in which the virtual object is at the determined position. For instance, XR headset <NUM> may show the cartoon character sitting on the flat surface.

<FIG> is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. In the example of <FIG>, a mobile device <NUM>, such as a mobile phone or tablet computer, includes a range-finding system, such as a LIDAR system, that detects positions of points on objects <NUM> in an environment of mobile device <NUM>. A data source of mobile device <NUM> may use the signals generated by the depth detection sensor to generate a point cloud representation of objects <NUM>. Mobile device <NUM> may include a G-PCC encoder (e.g., G-PCC encoder <NUM> of <FIG>) that is configured to encode the point cloud to generate bitstreams <NUM>. In the example of <FIG>, mobile device <NUM> may transmit bitstreams to a remote device <NUM>, such as a server system or other mobile device. Remote device <NUM> may decode bitstreams <NUM> to reconstruct the point cloud. Remote device <NUM> may use the point cloud for various purposes. For example, remote device <NUM> may use the point cloud to generate a map of environment of mobile device <NUM>. For instance, remote device <NUM> may generate a map of an interior of a building based on the reconstructed point cloud. In another example, remote device <NUM> may generate imagery (e.g., computer graphics) based on the point cloud. For instance, remote device <NUM> may use points of the point cloud as vertices of polygons and use color attributes of the points as the basis for shading the polygons. In some examples, remote device <NUM> may use the reconstructed point cloud for facial recognition or other security applications.

<FIG> show examples of this process at bin n. In example <NUM> of <FIG>, the range at bin n includes the RangeMPS and RangeLPS given by the probability of the LPS (pσ) given a certain context state (σ). Example <NUM> shows the update of the range at bin n+<NUM> when the value of bin n is equal to the MPS. In this example, the low stays the same, but the value of the range at bin n+<NUM> is reduced to the value of RangeMPS at bin n. Example <NUM> of <FIG> shows the update of the range at bin n+<NUM> when the value of bin n is not equal to the MPS (i.e., equal to the LPS). In this example, the low is moved to the lower range value of RangeLPS at bin n. In addition, the value of the range at bin n+<NUM> is reduced to the value of RangeLPS at bin n.

In some examples, the range may be expressed with <NUM> bits and the low with <NUM> bits. There is a renormalization process to maintain the range and low values at sufficient precision. The renormalization occurs whenever the range is less than <NUM>. Therefore, the range is always equal or larger than <NUM> after renormalization. Depending on the values of range and low, the BAC outputs to the bitstream, a '<NUM>,' or a '<NUM>,' or updates an internal variable (called BO: bits-outstanding) to keep for future outputs. <FIG> shows examples of BAC output depending on the range. For example, a '<NUM>' is output to the bitstream when the range and low are above a certain threshold (e.g., <NUM>). A '<NUM>' is output to the bitstream when the range and low are below a certain threshold (e.g., <NUM>). Nothing is output to the bitstream when the range and lower are between certain thresholds. Instead, the BO value is incremented and the next bin is encoded.

As discussed above, arithmetic coding methods may be used to provide high compression efficiency. This is achieved by first transforming the non-binary syntax elements into a binary representation (e.g., <NUM>, <NUM>) using a process called binarization. The resulting transformed entries are called as bins or bin-strings. These bins or bin strings are then fed into the arithmetic coding process. <FIG> illustrates an example context adaptive binary arithmetic coding (CABAC) encoding stage. The example CABAC encoding stage may be implemented in a G-PCC encoder, such as by arithmetic encoding unit <NUM> and/or arithmetic encoding unit <NUM> of G-PCC encoder <NUM> of <FIG>.

In some examples of G-PCC, context-adaptive binary arithmetic coding (CABAC) may be used for generating the bins through the binarization process. For each coded bin value, an appropriate context model is selected. These context models are used for encoding each bin value into output bits based on the bin probability values. CABAC engine bypasses context-modeling and bin encoding when the bin is equally probable to be <NUM> or <NUM>. This is the bypass coding stage discussed below. Otherwise, an appropriate context model is specified as the bin values are encoded and models based on the probability of bin-values. Contexts are adapted as the encoder encodes more bins. Lastly, the context-coded bin values or raw bitstreams are transmitted or otherwise provided to the decoder.

<FIG> is a block diagram of an example arithmetic encoding unit <NUM> that may be configured to perform CABAC in accordance with the techniques of this disclosure. A syntax element <NUM> is input into the arithmetic encoding unit <NUM>. If the syntax element is already a binary-value syntax element (e.g., a flag or other syntax element that only has a value of <NUM> and <NUM>), the step of binarization may be skipped. If the syntax element is a non-binary valued syntax element (e.g., a syntax element that may have values other than <NUM> or <NUM>), the non-binary valued syntax element is binarized by binarizer <NUM>. Binarizer <NUM> performs a mapping of the non-binary valued syntax element into a sequence of binary decisions. These binary decisions are often called "bins. " For example, for transform coefficient levels, the value of the level may be broken down into successive bins, each bin indicating whether or not the absolute value of coefficient level is greater than some value. For example, bin <NUM> (sometimes called a significance flag) indicates if the absolute value of the transform coefficient level is greater than <NUM> or not. Bin <NUM> indicates if the absolute value of the transform coefficient level is greater than <NUM> or not, and so on. A unique mapping may be developed for each non-binary valued syntax element.

Each bin produced by binarizer <NUM> is fed to the binary arithmetic coding side of arithmetic encoding unit <NUM>. That is, for a predetermined set of non-binary valued syntax elements, each bin type (e.g., bin <NUM>) is coded before the next bin type (e.g., bin <NUM>). Coding may be performed in either regular mode or bypass mode. In bypass mode, bypass encoding engine <NUM> performs arithmetic coding using a fixed probability model, for example, using Golomb-Rice or exponential Golomb coding. Bypass mode is generally used for more predictable syntax elements.

Coding in regular mode involves performing CABAC. Regular mode CABAC is for coding bin values where the probability of a value of a bin is predictable given the values of previously coded bins. The probability of a bin being an LPS is determined by context modeler <NUM>. Context modeler <NUM> outputs the bin value and the probability state for the context (e.g., the probability state σ, including the value of the LPS and the probability of the LPS occurring). The context may be an initial context for a series of bins, or may be determined based on the coded values of previously coded bins. The identify of a context may be expressed and/or be determined based on a value of a variable ctxInc (context increment, such as the value of the ctxInc representing an increment to apply to a previous context). As described above, context modeler <NUM> may update the state based on whether or not the received bin was the MPS or the LPS. After the context and probability state σ is determined by context modeler <NUM>, regular encoding engine <NUM> performs BAC on the bin value.

<FIG> is a block diagram of an example arithmetic decoding unit <NUM> that may be configured to perform CABAC in accordance with the techniques of this disclosure. The arithmetic decoding unit <NUM> of <FIG> performs CABAC in an inverse manner as that of arithmetic encoding unit <NUM> described in <FIG>. Coded bits from bitstream <NUM> are input into arithmetic decoding unit <NUM>. The coded bits are fed to either context modeler <NUM> or bypass decoding engine <NUM> based on whether they were entropy coded using regular mode or bypass mode. If the coded bits were coded in bypass mode, bypass decoding engine will use Golomb-Rice or exponential Golomb decoding, for example, to retrieve the binary-valued syntax elements or bins of non-binary syntax elements.

If the coded bits were coded in regular mode, context modeler <NUM> may determine a probability model for the coded bits and regular decoding engine <NUM> may decode the coded bits to produce bins of non-binary valued syntax elements (or the syntax elements themselves if binary-valued). After the context and probability state σ is determined by context modeler <NUM>, regular decoding engine <NUM> performs BAC to decode the bin value. In other words, regular decoding engine <NUM> may determine a probability state of a context, and decode a bin value based on previously coded bins and a current range. After decoding the bin, context modeler <NUM> may update the probability state of the context based on the window size and the value of the decoded bin.

<FIG> is a flowchart illustrating an example method for encoding a current predictive tree node, in accordance with one or more techniques of this disclosure. The current predictive tree node (PTN) may be included in a point cloud. Although described with respect to G-PCC encoder <NUM> (<FIG> and <FIG>), it should be understood that other devices may be configured to perform a method similar to that of <FIG>. For instance, a G-PCC decoder, such as G-PCC decoder <NUM> (<FIG> and <FIG>) may perform a complimentary method to that of <FIG> (e.g., decode as opposed to encode).

G-PCC encoder <NUM> may obtain a value of a secondary residual for geometry coding a current predictive tree node (PTN) of the point cloud (<NUM>). As discussed above, when using an angular mode for predictive geometry coding in G-PCC, G-PCC encoder <NUM> may perform prediction in the (r, ϕ, i) domain. Due to the errors in rounding, coding in r, ϕ, i may be lossy. In some examples, this loss may by reduced or eliminated by coding a second set of residuals (referred to as secondary residuals), which may be in a Cartesian domain. For instance, the current PTN may include three secondary residuals (rx, ry, rz), one or more of which may be encoded using the technique of <FIG>.

G-PCC encoder <NUM> may encode the value of the secondary residual. To encode the value of the secondary residual, G-PCC encoder <NUM> may encode, using a first set of context-adaptive binary arithmetic coding (CABAC) contexts, prefix bins of a syntax element having a value that specifies an absolute value of the value of the secondary residual minus <NUM> (<NUM>); and encode, using a second set of CABAC contexts that is different than the first set of contexts, suffix bins of the syntax element (<NUM>). For instance, arithmetic encoding unit <NUM> of G-PCC encoder <NUM> may encode prefix bins of a ptn_sec_residual_abs_minus2 syntax element using contexts with context indices (ctxIdx) between <NUM> and a-<NUM> and encode suffix bins of the ptn_sec_residual_abs_minus2[ k ] syntax element using contexts with context indices (ctxIdx) between a and a+b-<NUM>. In some examples, a may be <NUM> and b may be <NUM>.

In some examples, in addition to the syntax element having a value that specifies an absolute value of the value of the secondary residual minus <NUM>, G-PCC encoder <NUM> may encode one or more other syntax elements that specify the value of the secondary residual. As one example, G-PCC encoder <NUM> may encode a syntax element having a value that specifies whether the value of the secondary residual is equal to zero (e.g., ptn_sec_residual_eq0_flag). As another example, where the value of the secondary residual is not equal to zero, G-PCC encoder <NUM> may encode a syntax element having a value that specifies a sign of the value of the secondary residual (e.g., ptn_sec_residual_sign_flag), and a syntax element having a value that specifies whether the value of the secondary residual is greater than one (e.g., ptn_sec_residual_eq1_flag).

However, as discussed above an in accordance with one or more techniques of this disclosure, G-PCC encoder <NUM> may avoid encoding a syntax element that specifies the absolute value of the secondary residual minus <NUM> (e.g., ptn_sec_residual_abs_minus17) even where the absolute value of the secondary residual is greater than <NUM>. By not encoding the syntax element that specifies the absolute value of the secondary residual minus <NUM> and by context coding the prefix and suffix of the syntax element that specifies the absolute value of the secondary residual minus <NUM>, G-PCC encoder <NUM> may reduce the number of contexts and/or context coded bins used to signal secondary residuals. As such, the techniques of this disclosure may reduce computational complexity of point cloud coding.

The techniques of this disclosure may be applicable to signalling beyond secondary residuals. For instance, G-PCC encoder <NUM> may obtain a value of a phi multiplier for geometry coding the current predictive tree node of the point cloud (<NUM>) and encode the value of the phi multiplier by at least encoding a syntax element having a value that specifies an absolute value of the value of the phi multiplier minus <NUM> (e.g., ptn_phi_mult_abs_minus9) (<NUM>). However, similar to the secondary residual, G-PCC encoder <NUM> may avoid encoding a syntax element that specifies the absolute value of the phi multiplier minus <NUM> (e.g., even where the absolute value of the phi multiplier is greater than <NUM>).

As discussed above, in some examples, a G-PCC encoder may signal a number of direct child nodes of PTNs. For instance, G-PCC encoder <NUM> may encode a syntax element that represents a number of direct child nodes of the current predictive tree node that are present in a geometry predictive tree that represents the point cloud (e.g., Ptn_child_cnt). In some examples, G-PCC encoder <NUM> may utilize fixed length coding for encoding the syntax element (e.g., binarizing). For instance, G-PCC encoder <NUM> may utilize the following table to encode the syntax element that represents the number of direct child nodes.

However, in some examples, utilizing fixed length coding may be undesirable. For instance, a probability distribution of the number of child nodes may result in more PTNs have one child node than having <NUM>, <NUM>, or <NUM> child nodes. In accordance with one or more techniques of this disclosure, G-PCC encoder <NUM> may encode, using variable length coding, the syntax element that represents a number of direct child nodes of the current predictive tree node that are present in a geometry predictive tree that represents the point cloud (e.g., Ptn_child_cnt) (<NUM>). For instance, G-PCC encoder <NUM> may utilize the following table to encode the syntax element that represents the number of direct child nodes.

As can be seen in the above table, variable length coding the syntax element that represents the number of direct child nodes may include utilizing a shorter codeword where the number of direct child nodes is <NUM> than where the number of direct child nodes is <NUM> (e.g., using the code word "<NUM>" where the number of child nodes is <NUM> vs using the code word "<NUM>" where the number of direct child nodes is <NUM>).

<FIG> is a flowchart illustrating an example method for decoding a current predictive tree node, in accordance with one or more techniques of this disclosure. The current predictive tree node (PTN) may be included in a point cloud. Although described with respect to G-PCC decoder <NUM> (<FIG> and <FIG>), it should be understood that other devices may be configured to perform a method similar to that of <FIG>. For instance, a G-PCC encoder, such as G-PCC encoder <NUM> (<FIG> and <FIG>) may perform a complimentary method to that of <FIG> (e.g., encode as opposed to decode, such as in a reconstruction loop performed by GRU <NUM>).

G-PCC decoder <NUM> may select, from a plurality of predefined prediction modes, a prediction mode for performing predictive geometry coding of a position of a current predictive tree node of the point cloud. As discussed above, the plurality of prediction modes may include at least: a zero prediction mode, and a delta prediction mode. In some examples, G-PCC decoder <NUM> may select the prediction mode based on a value of a syntax element. For instance, geometry arithmetic decoder unit <NUM>) may decode a ptn_pred_mode syntax element having a value that specifies which prediction mode is to be selected.

Responsive to selecting the zero prediction mode (<NUM>), G-PCC decoder <NUM> may perform zero prediction to determine the position of the current PTN. To perform zero prediction, G-PCC decoder <NUM> may determine a radius, an azimuth, and a laser index of a parent node of the current predictive tree node (<NUM>). For instance, G-PCC decoder <NUM> may obtain the radius, azimuth, and laser index of the parent node from memory (e.g., as previously determined by G-PCC decoder <NUM>).

G-PCC decoder <NUM> may infer an azimuth and a laser index of a predicted position of the current predictive tree node as the azimuth and the laser index of the parent node (<NUM>). For instance, G-PCC decoder <NUM> may copy the azimuth and laser index of the parent node to as the azimuth and laser index of the predicted position of the current node.

G-PCC decoder <NUM> may infer a radius of the predicted position to be a minimum radius value (<NUM>). For instance, G-PCC decoder <NUM> may always set the radius of the predicted position to be the minimum radius value. In some examples, the minimum radius value may always be zero. In some examples, the minimum radius value may be greater than zero. For instance, G-PCC decoder <NUM> may decode a syntax element that specifies the minimum radius value (e.g., from a slice header).

To complete performance of zero prediction, G-PCC decoder <NUM> may determine, based on the predicted position of the current predictive tree node, the position of the current predictive tree node (<NUM>). For instance, G-PCC decoder <NUM> may obtain a residual radius value that represents a difference between a value of the radius of the predicted position of the current predictive tree node and a value of the radius of the position of the current predictive tree node. In some examples, such as where the selected mode is the zero prediction mode, G-PCC decoder <NUM> may infer that a sign of the residual radius value is positive (e.g., and avoid having to signal whether the sign is positive or negative). G-PCC decoder <NUM> may add the residual radius value to the minimum radius value (e.g., the radius value of the predicted position of the current PTN) to obtain the value of the radius of the position of the current PTN. G-PCC decoder <NUM> may similarly obtain and add residuals for other components (e.g., azimuth and laser index).

In some examples, G-PCC decoder <NUM> may modify decoding based on a quantity of lasers used to generate the point cloud. For instance, G-PCC decoder <NUM> may determine a quantity of lasers used to capture light detection and ranging (LIDAR) data that represents the point cloud (<NUM>); and responsive to determining that the quantity of lasers is one, inferring laser indices for all nodes in the point cloud to be a same value (e.g., a laser index value of <NUM>) (<NUM>). In this way, G-PCC decoder <NUM> may avoid having to signal and/or predict laser indices. As such, the techniques of this disclosure may reduce the number of bits needed to represent point clouds and/or reduce point cloud coding complexity.

Examples in the various aspects of this disclosure may be used individually or in any combination.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the terms "processor" and "processing circuitry," as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein.

Claim 1:
A method of decoding a point cloud, the method comprising:
selecting (<NUM>), from a plurality of predefined prediction modes, a prediction mode for performing predictive geometry coding of a position of a current predictive tree node of the point cloud, wherein the plurality of prediction modes includes at least:
a zero prediction mode, and a delta prediction mode;
determining (<NUM>) a radius of a parent node of the current predictive tree node, an azimuth of the parent node of the current predictive tree node, and a laser index of the parent node of the current predictive tree node; and
responsive to selecting the zero prediction mode:
inferring (<NUM>) an azimuth and a laser index of a predicted position of the current predictive tree node as the azimuth and the laser index of the parent node;
inferring (<NUM>) a radius of the predicted position to be a minimum radius value, wherein the minimum radius value is different than the radius of the parent node; and
determining (<NUM>), based on the predicted position of the current predictive tree node, the position of the current predictive tree node.