Patent Description:
Advanced three-dimensional (3D) representations of the world are enabling more immersive forms of interaction and communication. They also allow machines to understand, interpret, and navigate our world. Point clouds have been widely used as a 3D representation of the world. Several use cases associated with point cloud data have been identified, and some corresponding requirements for point cloud representation and compression have been developed.

A point cloud may be a set of points in a 3D space, each with associated attributes, e.g. color, material properties, etc. Point clouds can be used to reconstruct an object or a scene as a composition of such points. They can be captured using multiple cameras and depth sensors in various setups and may be made up of thousands up to billions of points in order to realistically represent reconstructed scenes.

Compression technologies are needed to reduce the amount of data required to represent a point cloud. As such, technologies may be needed for lossy compression of point clouds for use in real-time communications and six Degrees of Freedom (DoF) virtual reality. In addition, technology is sought for lossless point cloud compression in the context of dynamic mapping for autonomous driving and cultural heritage applications, etc. MPEG has started working on a standard to address compression of geometry and attributes such as colors and reflectance, scalable/progressive coding, coding of sequences of point clouds captured over time, and random access to subsets of the point cloud.

In an embodiment, method of decoding a video stream encoded using video point cloud coding, the method is performed by at least one processor and includes obtaining a geometry-reconstructed point cloud; for a first iteration, computing a first set of reference points respectively for a plurality of 2x2x2 cells comprising a first plurality of points, wherein each reference point in the first set of reference points is computed as a centroid of points in one 2x2x2 cell; applying a first interpolation filter to the first plurality of points based on the first set of reference points to obtain a second plurality of points; for a second iteration, computing a second set of reference points respectively for the plurality of the 2x2x2 cells comprising the second plurality of points, wherein each reference point in the second set of reference points is computed as a centroid of points in one 2x2x2 cell; applying a second interpolation filter to the second plurality of points based on the second set of reference points to obtain a third plurality of points; wherein a strength of the first filter is higher than a strength of the second filter; obtaining a smoothed geometry-reconstructed point cloud based on the third plurality of points; and reconstructing a dynamic point cloud using the smoothed geometry-reconstructed point cloud; the method further comprising: comparing a number of points of each 2x2x2 cell to a threshold value, wherein, based on the number being lower than a threshold value, the centroid of the points of the 2x2x2 cell is determined using a vector median of the points of the 2x2x2 cell, and wherein, based on the number being higher than the threshold value, the centroid of the points of the 2x2x2 cell is determined using an average filter.

In an embodiment, an apparatus for decoding a video stream encoded using video point cloud coding includes at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code configured to cause the at least one processor to obtain a geometry-reconstructed point cloud; for a first iteration, compute a first set of reference points respectively for a plurality of 2x2x2 cells comprising a first plurality of points, wherein each reference point in the first set of reference points is computed as a centroid of points in one 2x2x2 cell; apply a first interpolation filter to the first plurality of points based on the first set of reference points to obtain a second plurality of points; for a second iteration, compute a second set of reference points respectively for the plurality of the 2x2x2 cells comprising the second plurality of points, wherein each reference point in the second set of reference points is computed as a centroid of points in one 2x2x2 cell; apply a second interpolation filter to the second plurality of points based on the second set of reference points to obtain a third plurality of points; obtain a smoothed geometry-reconstructed point cloud based on the third plurality of points; and reconstruct a dynamic point cloud using the smoothed geometry-reconstructed point cloud, wherein a strength of the first filter is higher than a strength of the second filter; wherein the program code is further configured to: compare a number of the points of each 2x2x2 cell to a threshold value, wherein, based on the number being lower than a threshold value, the centroid of the points of the 2x2x2 cell is determined using a vector median of the points of the 2x2x2 cell, and wherein, based on the number being higher than the threshold value, the centroid of the points of the 2x2x2 cell is determined using an average filter.

In an embodiment, a non-transitory computer-readable medium stores computer instructions decoding a video stream encoded using video point cloud coding that, when executed by at least one processor, cause the at least one processor to: obtain a geometry-reconstructed point cloud; for a first iteration, compute a first set of reference points respectively for a plurality of 2x2x2 cells comprising a first plurality of points, wherein each reference point in the first set of reference points is computed as a centroid of points in one 2x2x2 cell; apply a first interpolation filter to the first plurality of points based on the first set of reference points to obtain a second plurality of points; for a second iteration, compute a second set of reference points respectively for the plurality of the 2x2x2 cells comprising the second plurality of points, wherein each reference point in the second set of reference points is computed as a centroid of points in one 2x2x2 cell; apply a second interpolation filter to the second plurality of points based on the second set of reference points to obtain a third plurality of points; obtain a smoothed geometry-reconstructed point cloud based on the third plurality of points; and reconstruct a dynamic point cloud using the smoothed geometry-reconstructed point cloud, wherein a strength of the first filter is higher than a strength of the second filter; wherein the computer instructions further cause the at least one processor to: compare a number of the points of each 2x2x2 cell to a threshold value, wherein, based on the number being lower than a threshold value, the centroid of the points of the 2x2x2 cell is determined using a vector median of the points of the 2x2x2 cell, and wherein, based on the number being higher than the threshold value, the centroid of the points of the 2x2x2 cell is determined using an average filter.

A consideration behind video-based point cloud compression (V-PCC) is to leverage existing video codecs to compress the geometry, occupancy, and texture of a dynamic point cloud as three separate video sequences. The extra metadata needed to interpret the three video sequences may be compressed separately. A small portion of the overall bitstream is the metadata, which could be encoded/decoded efficiently using software implementation. The bulk of the information may be handled by the video codec.

Embodiments of the present disclosure relate to an annealing iterative geometry smoothing to avoid over-smoothing in an iterative smoothing framework. Embodiments of the present disclosure relate to using a combination of average and median statistics to derive the reference points aiming to reduce the computational complexity of using the pure median. The concept of geometry smoothing for point clouds has been addressed in the standardisation contribution "<NPL>.

With reference to <FIG>, an embodiment of the present disclosure for implementing encoding and decoding structures of the present disclosure are described. The encoding and decoding structures of the present disclosure may implement aspects of V-PCC described above.

<FIG> illustrates a simplified block diagram of a communication system <NUM> according to an embodiment of the present disclosure. The system <NUM> may include at least two terminals <NUM>, <NUM> interconnected via a network <NUM>. For unidirectional transmission of data, a first terminal <NUM> may code video data at a local location for transmission to the other terminal <NUM> via the network <NUM>. The second terminal <NUM> may receive the coded video data of the other terminal from the network <NUM>, decode the coded data and display the recovered video data.

<FIG> illustrates a second pair of terminals <NUM>, <NUM> provided to support bidirectional transmission of coded video that may occur, for example, during videoconferencing. For bidirectional transmission of data, each terminal <NUM>, <NUM> may code video data captured at a local location for transmission to the other terminal via the network <NUM>. Each terminal <NUM>, <NUM> also may receive the coded video data transmitted by the other terminal, may decode the coded data and may display the recovered video data at a local display device.

In <FIG>, the terminals <NUM>-<NUM> may be, for example, servers, personal computers, and smart phones, and/or any other type of terminal. For example, the terminals (<NUM>-<NUM>) may be laptop computers, tablet computers, media players and/or dedicated video conferencing equipment. The network <NUM> represents any number of networks that convey coded video data among the terminals <NUM>-<NUM> including, for example, wireline and/or wireless communication networks. The communication network <NUM> may exchange data in circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For the purposes of the present discussion, the architecture and topology of the network <NUM> may be immaterial to the operation of the present disclosure unless explained herein below.

<FIG> illustrates, as an example of an application for the disclosed subject matter, a placement of a video encoder and decoder in a streaming environment. The disclosed subject matter can be used with other video enabled applications, including, for example, video conferencing, digital TV, storing of compressed video on digital media including CD, DVD, memory stick and the like, and so on.

As illustrated in <FIG>, a streaming system <NUM> may include a capture subsystem <NUM> that includes a video source <NUM> and an encoder <NUM>. The streaming system <NUM> may further include at least one streaming server <NUM> and/or at least one streaming client <NUM>.

The video source <NUM> can create, for example, a stream <NUM> that includes a 3D point cloud corresponding to a 3D video. The video source <NUM> may include, for example, 3D sensors (e.g. depth sensors) or 3D imaging technology (e.g. digital camera(s)), and a computing device that is configured to generate the 3D point cloud using the data received from the 3D sensors or the 3D imaging technology. The sample stream <NUM>, which may have a high data volume when compared to encoded video bitstreams, can be processed by the encoder <NUM> coupled to the video source <NUM>. The encoder <NUM> can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoder <NUM> may also generate an encoded video bitstream <NUM>. The encoded video bitstream <NUM>, which may have e a lower data volume when compared to the uncompressed stream <NUM>, can be stored on a streaming server <NUM> for future use. One or more streaming clients <NUM> can access the streaming server <NUM> to retrieve video bit streams <NUM> that may be copies of the encoded video bitstream <NUM>.

The streaming clients <NUM> can include a video decoder <NUM> and a display <NUM>. The video decoder <NUM> can, for example, decode video bitstream <NUM>, which is an incoming copy of the encoded video bitstream <NUM>, and create an outgoing video sample stream <NUM> that can be rendered on the display <NUM> or another rendering device (not depicted). In some streaming systems, the video bitstreams <NUM>, <NUM> can be encoded according to certain video coding/compression standards. Examples of such standards include, but are not limited to, ITU-T Recommendation H. <NUM>, Versatile Video Coding (VVC), and MPEG/V-PCC.

With reference to <FIG>, some aspects of V-PCC that may be performed by embodiments of the present disclosure are described below.

<FIG> illustrates an example functional block diagram of a video encoder <NUM> according to an embodiment of the present disclosure.

As illustrated in <FIG>, the video encoder <NUM> may receive a point cloud frame(s) <NUM>, and generate a geometry image <NUM>, a texture image <NUM>, and an occupancy map <NUM> based on the point cloud frame <NUM>. The video encoder <NUM> may compress the geometry image <NUM> into a compressed geometry image <NUM>, the texture image <NUM> into a compressed texture image <NUM>, and the occupancy map <NUM> into a compressed occupancy map <NUM>. A multiplexer <NUM> of the video encoder <NUM> may form a compressed bitstream <NUM> that includes the compressed geometry image <NUM>, the compressed texture image <NUM>, and the compressed occupancy map <NUM>.

More specifically, in embodiments, the video encoder <NUM> may include a patch generation module <NUM> that segments the point cloud frame <NUM> into patches. Patches are useful entities of V-PCC. The patch generation process includes decomposing the point cloud frame <NUM> into a minimum number of patches with smooth boundaries, while also minimizing the reconstruction error. Encoders of the present disclosure may implement various methods to generate such a decomposition.

The video encoder <NUM> may include a patch packing module <NUM> that performs a packing process. The packing process includes mapping the extracted patches onto a 2D grid while minimizing the unused space and guaranteeing that every M×M (e.g., 16x16) block of the grid is associated with a unique patch. Efficient patch packing directly impacts the compression efficiency either by minimizing the unused space or ensuring temporal consistency. The patch packing module <NUM> may generate the occupancy map <NUM>.

The video encoder <NUM> may include a geometry image generation module <NUM> and a texture image generation module <NUM>. In order to better handle the case of multiple points being projected to the same sample, each patch may be projected onto two images, referred to as layers. For example, the geometry image generation module <NUM> and the texture image generation module <NUM> may exploit the 3D to 2D mapping computed during the packing process of the patch packing module <NUM> to store the geometry and texture of the point cloud as images (a. The generated images/layers may be stored as a video frame(s) and compressed using a video codec (e.g. HM video codec) according to configurations provided as parameters.

In embodiments, the geometry image generation module <NUM> generates the geometry image <NUM> and the texture image generation module <NUM> generates the texture image <NUM>, based on the input point cloud frame <NUM> and the occupancy map <NUM>. An example of the geometry image <NUM> is illustrated in <FIG> and an example of the texture image <NUM> is illustrated in <FIG>. In an embodiment, the geometry image <NUM> may be represented by a monochromatic frame of WxH in YUV420-8bit format. In an embodiment, the occupancy map <NUM> image consists of a binary map that indicates for each cell of the grid whether it belongs to the empty space or to the point cloud. To generate the texture image <NUM>, the texture image generation module <NUM> may exploit the reconstructed/smoothed geometry <NUM> in order to compute the colors to be associated with the re-sampled points.

The video encoder <NUM> may also include an image padding module <NUM> and an image padding module <NUM> for padding the geometry image <NUM> and the texture image <NUM>, respectively, to form a padded geometry image <NUM> and a padded texture image <NUM>. The image padding (a. background filling) simply fills unused space of the images with redundant information. A good background filling is a one that minimally increases the bit rate while does not introduce significant coding distortion around the patch boundaries. The image padding module <NUM> and the image padding module <NUM> may use the occupancy map <NUM> to form the padded geometry image <NUM> and the padded texture image <NUM>, respectively. In an embodiment, the video encoder <NUM> may include a group dilation module <NUM> to form the padded texture image <NUM>.

The video encoder <NUM> may include a video compression module <NUM> and a video compression module <NUM> for compressing the padded geometry image <NUM> and the padded texture image <NUM> into the compressed geometry image <NUM> and the compressed texture image <NUM>, respectively.

The video encoder <NUM> may include an entropy compression module <NUM> for lossless encoding <NUM> of the occupancy map <NUM> and a video compression module <NUM> for lossy encoding <NUM> of the occupancy map <NUM>.

In embodiments, the video encoder <NUM> may include a smoothing module <NUM> for generating smoothed geometry <NUM> by using a reconstructed geometry image <NUM>, provided by the video compression module <NUM>, and patch info <NUM>. The smoothing procedure of the smoothing module <NUM> may aim at alleviating potential discontinuities that may arise at the patch boundaries due to compression artifacts. The smoothed geometry <NUM> may be used by the texture image generation module <NUM> to generate the texture image <NUM>.

The video encoder <NUM> may also include an auxiliary patch information compression module <NUM> for forming compressed auxiliary patch information <NUM> that is provided in the compressed bitstream <NUM> by the multiplexer <NUM>.

<FIG> illustrates an example functional block diagram of a video decoder <NUM> according to an embodiment of the present disclosure.

As illustrated in <FIG>, the video decoder <NUM> may receive the coded bitstream <NUM> from the video encoder <NUM> to obtain the compressed texture image <NUM>, the compressed geometry image <NUM>, the compressed occupancy map <NUM>, and the compressed auxiliary patch information <NUM>. The video decoder <NUM> may decode the compressed texture image <NUM>, the compressed geometry image <NUM>, the compressed occupancy map <NUM>, and the compressed auxiliary patch information <NUM> to obtain a decompressed texture image <NUM>, a decompressed geometry image <NUM>, a decompressed occupancy map <NUM>, and decompressed auxiliary patch information <NUM>, respectively. Following, the video decoder <NUM> may generate a reconstructed point cloud <NUM> based on the decompressed texture image <NUM>, the decompressed geometry image <NUM>, the decompressed occupancy map <NUM>, and the decompressed auxiliary patch information <NUM>.

In embodiments, the video decoder <NUM> may include a demultiplexer <NUM> that separates the compressed texture image <NUM>, the compressed geometry image <NUM>, the compressed occupancy map <NUM>, and the compressed auxiliary patch information <NUM> of the compressed bitstream <NUM> received.

The video decoder <NUM> may include a video decompression module <NUM>, a video decompression module <NUM>, an occupancy map decompression module <NUM>, and an auxiliary patch information decompression module <NUM> that decode the compressed texture image <NUM>, the compressed geometry image <NUM>, the compressed occupancy map <NUM>, and the compressed auxiliary patch information <NUM>, respectively.

The video decoder <NUM> may include a geometry reconstruction module <NUM> that obtains reconstructed (three dimensional) geometry <NUM> based on the decompressed geometry image <NUM>, the decompressed occupancy map <NUM>, and the decompressed auxiliary patch information <NUM>.

The video decoder <NUM> may include a smoothing module <NUM> that smooths the reconstructed geometry <NUM> to obtain smoothed geometry <NUM>. The smoothing procedure may aim at alleviating potential discontinuities that may arise at the patch boundaries due to compression artifacts.

The video decoder <NUM> may include a texture reconstruction module <NUM> for obtaining reconstructed texture <NUM> based on the decompressed texture image <NUM> and the smoothed geometry <NUM>.

The video decoder <NUM> may include a color smoothing module <NUM> that smooths the color of the reconstructed texture <NUM> to obtain a reconstructed point cloud <NUM>. Non-neighboring patches in 3D space are often packed next to each other in 2D videos. This implies that pixel values from non-neighboring patches might be mixed up by the block-based video codec. The color smoothing of the color smoothing module <NUM> may aim to reduce the visible artifacts that appear at patch boundaries.

After compression of geometry video, the point cloud may be reconstructed using the compressed geometry video and the corresponding occupancy map, which may also be referred to as a geometry-reconstructed cloud. Colors may be then transferred to the resulting geometry-reconstructed cloud. Geometry smoothing may be applied on the geometry-reconstructed cloud before color-transfer.

A goal of geometry smoothing may be to recover the geometry distorted at patch boundaries which is due to geometry compression as well as conversion of a high-resolution occupancy map to a lower-resolution one. A better smoothing strategy could lead to a better geometry-reconstructed cloud which in turn leads to a higher-quality compressed texture-video.

Embodiments of the present disclosure relate to an annealing strategy to decay the strength of smoothing when the number of iterations increase in order to avoid over-smoothing when multiple smoothing iterations are performed. This approach may be independent of the type of the interpolation filter adopted, for example a trilinear filter. Also, to derive the reference points, embodiments of the present disclosure may use a combination of median and average statistics to reduce the computational complexity compared to the case where the pure median is used.

According to an embodiment, a process of geometry smoothing of V-PCC may include two steps:.

According to an embodiment, in an iterative smoothing framework, each of the above steps or both could be done differently in each iteration. Embodiments of the present disclosure relate to an annealing iterative smoothing approach to avoid over-smoothing incurred by step <NUM>, where the strength of the interpolation filter is reduced in each iteration.

Embodiments of the present disclosure also relate to using a combination of median and average statistics to derive the reference points in step <NUM> aiming to reduce the computational complexity of computing the pure median.

In an embodiment, the strength of the interpolation filter may be decayed in an annealing fashion to avoid over-smoothing. Assume the current point p is moved to p~ by the trilinear filter (i.e., the interpolation filter) as shown in Equation <NUM> below: <MAT>.

For the second iteration, a partial movement may be used to avoid over-smoothing: <MAT>.

Where w is a filter weight. For example, w=<NUM> may be used.

The proposed partial movement may be used regardless of the type of filter and, and may be used for filter other than trilinear filter. In an embodiment, if the number of iterations is larger than <NUM>, an annealing strategy may be adopted such that the value of w is decreased in each iteration. Any usual annealing approach could be used including the ones used in gradient descent optimization techniques. Examples are illustrated in Equations <NUM> and <NUM> below: <MAT> <MAT>.

Wherein p represents a location of a reference point, p~ represents a location of the reference point after an interpolation filter is applied, α0 represents a first constant, β0 represents a second constant, and k represents an iteration number. For example, k=<NUM> may be used for the first iteration, k=<NUM> may be used for the first iteration, and k=<NUM> may be used for the first iteration.

In one embodiment, he computational complexity of the iterative smoothing may be reduced if a highly complex reference point computation is utilized, for example if vector median is used in place of average. In that case, a threshold may be chosen for the number of points of each cell, N. This threshold may be denoted by T. The complex reference computation may be applied if N ≤ T, and the average filter may be applied otherwise. The computational complexity is increased slightly compared to the average filter if T is set to a number around <NUM>.

In another embodiment, duplicate points are removed from a cell before computing its corresponding reference point. This means that, as an example, the centroid (average) is computed as shown in Equation <NUM> below: <MAT>
where Q represents the set of current cell unique points and |Q| denotes the cardinality of set Q. and pi represents a point having an index i.

An example of a metadata syntax for signaling the embodiments discussed above is shown in Table <NUM> below:.

In Table <NUM> above, iterative_smoothing_present_flag may indicate whether iterative smoothing is used or not.

number_of_iterations may indicate the number of times the smoothing process is applied sequentially.

reference_point_type_index_per_iteration[i] may indicate the index of the type of the reference point used in the i-th iteration. Examples include average and vector median. The value of reference_point_type_index_per_iteration[i] may be in the range [<NUM>, number_of_available_reference_point_types - <NUM>].

interpolation_filter_type_index_per_iteration[i] may indicate the index of the type of the interpoloation (smoothing) filter used in the i-th iteration. The value of interpolation_filter_type_index_per_iteration[i] may be in the range [<NUM>, number_of_available_interpolation_filter_types - <NUM>]. Examples include trilinear and weighted trilinear.

An example decoding process may take as inputs Indexes of types of reference points in each iteration, indexes of types of interpolation filters in each iteration, and parameters of the decaying weights. A decoder, for example decoder <NUM>, may sequentially apply the smoothing filters so that the input at the i-th iteration filter is the output of the (i - <NUM>)-th filter. Reference points at iteration i may be computed according to the value of reference_point_type_index_per_iteration[i]. Interpolation may then be done at iteration i according to the value of interpolation_filter_type_index_per_iteration[i]. Results of the interpolation may be weighted according to the values of weight_annealing_strategy[i], weight_annealing_param_alpha0[i] and weight_annealing_param_beta0[i].

<FIG> is a flowchart of a method <NUM> of decoding a video stream encoded using video point cloud coding which may include annealing iterative geometry smoothing, according to embodiments. In some implementations, one or more process blocks of <FIG> may be performed by decoder <NUM>. In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including the encoder <NUM>, such as the encoder <NUM>.

As shown in <FIG>, in operation <NUM>, the method <NUM> may include obtaining a geometry-reconstructed point cloud.

In operation <NUM>, the method <NUM> may include dividing the geometry-reconstructed point cloud into a plurality of cells, wherein a cell of the plurality of cells includes a first plurality of points.

In operation <NUM>, the method <NUM> may include obtaining, from among the first plurality of points, a first reference point including a centroid of the first plurality of points.

In operation <NUM>, the method <NUM> may include generating a second plurality of points by applying a first filter to the first plurality of points based on the first reference point.

In operation <NUM>, the method <NUM> may include obtaining, from among the second plurality of points, a second reference point including a centroid of the second plurality of points.

In operation <NUM>, the method <NUM> may include generating a third plurality of points by applying a second filter to the second plurality of points based on the second reference point, wherein a strength of the first filter is higher than a strength of the second filter.

In operation <NUM>, the method <NUM> may include obtaining a smoothed geometry-reconstructed point cloud based on the third plurality of points.

In operation <NUM>, the method <NUM> may include reconstructing a dynamic point cloud using the smoothed geometry-reconstructed point cloud.

In an embodiment, the first filter includes a first trilinear filter, and the second filter includes a second trilinear filter.

In an embodiment, the method may further include obtaining, from among the third plurality of points, a third reference point comprising a centroid of the third plurality of points; generating a fourth plurality of points by applying a third filter to the third plurality of points based on the third reference point; generating the smoothed geometry-reconstructed point cloud based on the fourth plurality of points.

In an embodiment, the method may further include comparing a number of the first plurality of points to a threshold value. Based on the number being lower than a threshold value, the centroid of the first plurality of points may be determined using a vector median of the first plurality of points. Based on the number being higher than the threshold value, the centroid of the first plurality of points may be determined using an average filter.

In an embodiment, duplicate points may be removed from the first plurality of points before the first reference point is determined.

Although <FIG> shows example blocks of the method <NUM>, in some implementations, the method <NUM> may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in <FIG> Additionally, or alternatively, two or more of the blocks of the method <NUM> may be performed in parallel.

<FIG> is a diagram of an apparatus <NUM> for decoding a video stream encoded using video point cloud coding which may include annealing iterative geometry smoothing, according to embodiments. As shown in <FIG>, the apparatus <NUM> includes first obtaining code <NUM>, dividing code <NUM>, second obtaining code <NUM>, first generating code <NUM>, third obtaining code <NUM>, second generating code <NUM>, fourth obtaining code <NUM>, and reconstructing code <NUM>.

The first obtaining code <NUM> may be configured to cause the at least one processor to obtain a geometry-reconstructed point cloud.

The dividing code <NUM> may be configured to cause the at least one processor to divide the geometry-reconstructed point cloud into a plurality of cells, wherein a cell of the plurality of cells comprises a first plurality of points.

The second obtaining code <NUM> may be configured to cause the at least one processor to obtain, from among the first plurality of points, a first reference point comprising a centroid of the first plurality of points.

The first generating code <NUM> may be configured to cause the at least one processor to generate a second plurality of points by applying a first filter to the first plurality of points based on the first reference point.

The third obtaining code <NUM> may be configured to cause the at least one processor to obtain, from among the second plurality of points, a second reference point comprising a centroid of the second plurality of points.

The second generating code <NUM> may be configured to cause the at least one processor to generate a third plurality of points by applying a second filter to the second plurality of points based on the second reference point, wherein a strength of the first filter is higher than a strength of the second filter.

The fourth obtaining code <NUM> may be configured to cause the at least one processor to obtain a smoothed geometry-reconstructed point cloud based on the third plurality of points.

The reconstructing code <NUM> may be configured to cause the at least one processor to reconstruct a dynamic point cloud using the smoothed geometry-reconstructed point cloud.

The techniques, described above, can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, <FIG> shows a computer system <NUM> suitable for implementing certain embodiments of the disclosure.

The components shown in <FIG> for computer system <NUM> are examples and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the non-limiting embodiment of a computer system <NUM>.

Input human interface devices may include one or more of (only one of each depicted): keyboard <NUM>, mouse <NUM>, trackpad <NUM>, touch screen <NUM>, data-glove, joystick <NUM>, microphone <NUM>, scanner <NUM>, camera <NUM>.

Computer system <NUM> may also include certain human interface output devices. Such human interface output devices may include tactile output devices (for example tactile feedback by the touch-screen <NUM>, data glove, or joystick <NUM>, but there can also be tactile feedback devices that do not serve as input devices). For example, such devices may be audio output devices (such as: speakers <NUM>, headphones (not depicted)), visual output devices (such as screens <NUM> to include CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch-screen input capability, each with or without tactile feedback capability-some of which may be capable to output two dimensional visual output or more than three dimensional output through means such as stereographic output; virtual-reality glasses (not depicted), holographic displays and smoke tanks (not depicted)), and printers (not depicted).

Computer system <NUM> can also include human accessible storage devices and their associated media such as optical media including CD/DVD ROM/RW <NUM> with CD/DVD or the like media <NUM>, thumb-drive <NUM>, removable hard drive or solid state drive <NUM>, legacy magnetic media such as tape and floppy disc (not depicted), specialized ROM/ASIC/PLD based devices such as security dongles (not depicted), and the like.

Computer system <NUM> can also include interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses <NUM> (such as, for example USB ports of the computer system <NUM>; others are commonly integrated into the core of the computer system <NUM> by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system <NUM> can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), unidirectional send-only (for example CANbus to certain CANbus devices), or bi-directional, for example to other computer systems using local or wide area digital networks. Such communication can include communication to a cloud computing environment <NUM>. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

Aforementioned human interface devices, human-accessible storage devices, and network interfaces <NUM> can be attached to a core <NUM> of the computer system <NUM>.

The core <NUM> can include one or more Central Processing Units (CPU) <NUM>, Graphics Processing Units (GPU) <NUM>, specialized programmable processing units in the form of Field Programmable Gate Areas (FPGA) <NUM>, hardware accelerators for certain tasks <NUM>, and so forth. These devices, along with Read-only memory (ROM) <NUM>, Randomaccess memory <NUM>, internal mass storage such as internal non-user accessible hard drives, SSDs, and the like <NUM>, may be connected through a system bus <NUM>. In some computer systems, the system bus <NUM> can be accessible in the form of one or more physical plugs to enable extensions by additional CPUs, GPU, and the like. The peripheral devices can be attached either directly to the core's system bus <NUM>, or through a peripheral bus <NUM>. A graphics adapter <NUM> may be included in the core <NUM>.

CPUs <NUM>, GPUs <NUM>, FPGAs <NUM>, and accelerators <NUM> can execute certain instructions that, in combination, can make up the aforementioned computer code. That computer code can be stored in ROM <NUM> or RAM <NUM>. Transitional data can be also be stored in RAM <NUM>, whereas permanent data can be stored for example, in the internal mass storage <NUM>. Fast storage and retrieve to any of the memory devices can be enabled through the use of cache memory, that can be closely associated with one or more CPU <NUM>, GPU <NUM>, mass storage <NUM>, ROM <NUM>, RAM <NUM>, and the like.

Claim 1:
A method of decoding a video stream encoded using video point cloud coding, the method being performed by at least one processor and comprising:
obtaining (<NUM>) a geometry-reconstructed point cloud;
for a first iteration,
computing a first set of reference points respectively for a plurality of 2x2x2 cells comprising a first plurality of points, wherein each reference point in the first set of reference points is computed as a centroid of points in one 2x2x2 cell;
applying a first interpolation filter to the first plurality of points based on the first set of reference points to obtain a second plurality of points;
for a second iteration,
computing a second set of reference points respectively for the plurality of the 2x2x2 cells comprising the second plurality of points, wherein each reference point in the second set of reference points is computed as a centroid of points in one 2x2x2 cell;
applying a second interpolation filter to the second plurality of points based on the second set of reference points to obtain a third plurality of points;
obtaining (<NUM>) a smoothed geometry-reconstructed point cloud based on the third plurality of points; and
reconstructing (<NUM>) a dynamic point cloud using the smoothed geometry-reconstructed point cloud,
wherein a strength of the first interpolation filter is higher than a strength of the second interpolation filter;
the method further comprising:
comparing a number of points of each 2x2x2 cell to a threshold value,
wherein, if said number is lower or equal than a threshold value, the centroid of the points of the 2x2x2 cell is determined using a vector median of the points of the 2x2x2 cell, and
wherein, if said number is higher than the threshold value, the centroid of the points of the 2x2x2 cell is determined using an average filter.