2D Gaussian Splatting for Geometrically Accurate Radiance Fields
BINBIN HUANG, ShanghaiTech University, China
ZEHAO YU, University of Tübingen Tübingen AI Center, Germany
ANPEI CHEN, University of Tübingen Tübingen AI Center, Germany
ANDREAS GEIGER, University of Tübingen Tübingen AI Center, Germany
SHENGHUA GAO, ShanghaiTech University, China

arXiv:2403.17888v2 [cs.CV] 9 Jun 2024

https://surfsplatting.github.io
Disk (color)

Radiance field

Disk (normal)

Surface normal(a) 2D disks as surface elementsMesh(b) 2D Gaussian splatting(c) Meshing

Fig. 1. Our method, 2DGS, (a) optimizes a set of 2D oriented disks to represent and reconstruct a complex real-world scene from multi-view RGB images. These
optimized 2D disks are tightly aligned to the surfaces. (b) With 2D Gaussian splatting, we allow real-time rendering of high quality novel view images withview consistent normals and depth maps. (c) Finally, our method provides detailed and noise-free triangle mesh reconstruction from the optimized 2D disks.
3D Gaussian Splatting (3DGS) has recently revolutionized radiance fieldreconstruction, achieving high quality novel view synthesis and fast rendering speed. However, 3DGS fails to accurately represent surfaces due to themulti-view inconsistent nature of 3D Gaussians. We present 2D Gaussian
Splatting (2DGS), a novel approach to model and reconstruct geometricallyaccurate radiance fields from multi-view images. Our key idea is to collapse
the 3D volume into a set of 2D oriented planar Gaussian disks. Unlike 3D
Gaussians, 2D Gaussians provide view-consistent geometry while modelingsurfaces intrinsically. To accurately recover thin surfaces and achieve stable
Authors’ Contact Information: Binbin Huang, huangbb@shanghaitech.edu.cn, ShanghaiTech University, Shanghai, China; Zehao Yu, zehao.yu@uni-tuebingen.de, University of Tübingen and Tübingen AI Center, Tübingen, Germany; Anpei Chen,
anpei.chen@uni-tuebingen.de, University of Tübingen and Tübingen AI Center, Tübingen, Germany; Andreas Geiger, a.geiger@uni-tuebingen.de, University of Tübingen and
Tübingen AI Center, Tübingen, Germany; Shenghua Gao, gaoshh@shanghaitech.edu.cn,
ShanghaiTech University, Shanghai, China.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA© 2024 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0525-0/24/07
https://doi.org/10.1145/3641519.3657428

optimization, we introduce a perspective-accurate 2D splatting process utilizing ray-splat intersection and rasterization. Additionally, we incorporate
depth distortion and normal consistency terms to further enhance the qualityof the reconstructions. We demonstrate that our differentiable renderer allows for noise-free and detailed geometry reconstruction while maintaining
competitive appearance quality, fast training speed, and real-time rendering.
CCS Concepts: • Computing methodologies → Reconstruction; Rendering; Machine learning approaches.
Additional Key Words and Phrases: Novel View Synthesis, Radiance Fields,
Surface Splatting, Surface Reconstruction
ACM Reference Format:
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao.
2024. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields.
In Special Interest Group on Computer Graphics and Interactive Techniques
Conference Conference Papers ’24 (SIGGRAPH Conference Papers ’24), July
27-August 1, 2024, Denver, CO, USA. ACM, New York, NY, USA, 13 pages.
https://doi.org/10.1145/3641519.3657428

1

INTRODUCTION

Photorealistic novel view synthesis (NVS) and accurate geometry
reconstruction stand as pivotal long-term objectives in computer
graphics and vision. Recently, 3D Gaussian Splatting (3DGS) [Kerbl
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

2 •Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

et al. 2023] has emerged as an appealing alternative to implicit [Barron et al. 2022a; Mildenhall et al. 2020] and feature grid-based representations [Barron et al. 2023; Müller et al. 2022] in NVS, due to
its real-time photorealistic NVS results at high resolutions. Rapidly
evolving, 3DGS has been quickly extended with respect to multiple
domains, including anti-aliasing rendering [Yu et al. 2024], material
modeling [Jiang et al. 2023; Shi et al. 2023], dynamic scene reconstruction [Yan et al. 2023], and animatable avatar creation [Qianet al. 2023; Zielonka et al. 2023]. Nevertheless, it falls short in capturing intricate geometry since the volumetric 3D Gaussian, whichmodels the complete angular radiance, conflicts with the thin natureof surfaces.
On the other hand, earlier works [Pfister et al. 2000; Zwicker et al.
2001a,b] have shown surfels (surface elements) to be an effective
representation of complex geometry. Surfels approximate the object
surface locally with shape and shade attributes and can be derived
from known geometry. They are widely used in SLAM [Whelan et al.
2016] and other robotics tasks [Schöps et al. 2019] as an efficient geometry representation. Subsequent advancements [Yifan et al. 2019]
have incorporated surfels into a differentiable framework. However,these methods typically require ground truth (GT) geometry, depth
sensor data, or operate under constrained scenarios with known
lighting.
Inspired by these works, we propose 2D Gaussian Splatting for
3D scene reconstruction and novel view synthesis that combines
the benefits of both worlds, while overcoming their limitations. Unlike 3DGS, our approach represents a 3D scene with 2D Gaussianprimitives, each defining an oriented elliptical disk. The significant
advantage of 2D Gaussian over its 3D counterpart lies in the accurate geometry representation during rendering. Specifically, 3DGS
evaluates a Gaussian’s value at the intersection between a pixel ray
and a 3D Gaussian [Keselman and Hebert 2022, 2023], which leads
to inconsistency depth when rendered from different viewpoints.
In contrast, our method utilizes explicit ray-splat intersection, resulting in a perspective correct splatting, as illustrated in Figure 2,
which in turn significantly improves reconstruction quality. Furthermore, the inherent surface normals in 2D Gaussian primitives
enable direct surface regularization through normal constraints. In
contrast with surfels-based models [Pfister et al. 2000; Yifan et al.
2019; Zwicker et al. 2001a], our 2D Gaussians can be recovered fromunknown geometry with gradient-based optimization.
While our 2D Gaussian approach excels in geometric modeling, optimizing solely with photometric losses can lead to noisy
reconstructions, due to the inherently unconstrained nature of 3D
reconstruction tasks, as noted in [Barron et al. 2022b; Yu et al. 2022b;
Zhang et al. 2020]. To enhance reconstructions and achieve smoothersurfaces, we introduce two regularization terms: depth distortionand normal consistency. The depth distortion term concentrates 2D
primitives distributed within a tight range along the ray, addressing the rendering process’s limitation where the distance between
Gaussians is ignored. The normal consistency term minimizes discrepancies between the rendered normal map and the gradient of
the rendered depth, ensuring alignment between the geometries
defined by depth and normals. Employing these regularizations
in combination with our 2D Gaussian model enables us to extract
highly accurate surface meshes, as demonstrated in Figure 1.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

Fig. 2. Comparison of 3DGS and 2DGS. 3DGS utilizes different intersection planes for value evaluation when viewing from different viewpoints,
resulting in inconsistency. Our 2DGS provides multi-view consistent value
evaluations.

In summary, we make the following contributions:• We present a highly efficient differentiable 2D Gaussian renderer, enabling perspective-correct splatting by leveraging
2D surface modeling, ray-splat intersection, and volumetricintegration.
• We introduce two regularization losses for improved andnoise-free surface reconstruction.
• Our approach achieves state-of-the-art geometry reconstruction and NVS results compared to other explicit representations.

2 RELATED WORK
2.1 Novel view synthesis
Significant advancements have been achieved in NVS, particularly
since the introduction of Neural Radiance Fields (NeRF) [Mildenhall et al. 2021]. NeRF employs a multi-layer perceptron (MLP) torepresent geometry and view-dependent appearance, optimized via
volume rendering to deliver exceptional rendering quality. PostNeRF developments have further enhanced its capabilities. For instance, Mip-NeRF [Barron et al. 2021] and subsequent works [Barron
et al. 2022a, 2023; Hu et al. 2023] tackle NeRF’s aliasing issues. Additionally, the rendering efficiency of NeRF has seen substantial
improvements through techniques such as distillation [Reiser et al.
2021; Yu et al. 2021] and baking [Chen et al. 2023a; Hedman et al.
2021; Reiser et al. 2023; Yariv et al. 2023]. Moreover, the training
and representational power of NeRF have been enhanced using
feature-grid based scene representations [Chen et al. 2022, 2023c;
Fridovich-Keil et al. 2022; Liu et al. 2020; Müller et al. 2022; Sun et al.
2022a].
Recently, 3D Gaussian Splatting (3DGS) [Kerbl et al. 2023] hasemerged, demonstrating impressive real-time NVS results. Thismethod has been quickly extended to multiple domains [Xie et al.
2023; Yan et al. 2023; Yu et al. 2024; Zielonka et al. 2023]. In this work,
we propose to “flatten” 3D Gaussians to 2D Gaussian primitives tobetter align their shape with the object surface. Combined with
two novel regularization losses, our approach reconstructs surfaces
more accurately than 3DGS while preserving its high-quality andreal-time rendering capabilities.

2.2

3D reconstruction

3D Reconstruction from multi-view images has been a long-standing
goal in computer vision. Multi-view stereo based methods [Schönberger et al. 2016; Yao et al. 2018; Yu and Gao 2020] rely on a modular

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

pipeline that involves feature matching, depth prediction, and fusion. In contrast, recent neural approaches [Niemeyer et al. 2020;
Yariv et al. 2020] represent surface implicitly via an MLP [Mescheder
et al. 2019; Park et al. 2019] , extracting surfaces post-training via
the Marching Cube algorithm. Further advancements [Oechsle et al.
2021; Wang et al. 2021; Yariv et al. 2021] integrated implicit surfaces
with volume rendering, achieving detailed surface reconstructions
from RGB images. These methods have been extended to large-scalereconstructions via additional regularization [Li et al. 2023; Yu et al.
2022a,b], and efficient reconstruction for objects [Wang et al. 2023].
Despite these impressive developments, efficient large-scale scenereconstruction remains a challenge. For instance, Neuralangelo [Liet al. 2023] requires 128 GPU hours for reconstructing a single scenefrom the Tanks and Temples Dataset [Knapitsch et al. 2017]. In thiswork, we introduce 2D Gaussian splatting, a method that significantly accelerates the reconstruction process. It achieves similar orslightly better results compared to previous implicit neural surface
representations, while being an order of magnitude faster.

2.3

Differentiable Point-based Graphics

Differentiable point-based rendering [Aliev et al. 2020; Insafutdinov
and Dosovitskiy 2018; Rückert et al. 2022; Wiles et al. 2020; Yifanet al. 2019] has been explored extensively due to its efficiency andflexibility in representing intricate structures. Notably, NPBG [Aliev
et al. 2020] rasterizes point cloud features onto an image plane,
subsequently utilizing a convolutional neural network for RGB
image prediction. DSS [Yifan et al. 2019] focuses on optimizing oriented point clouds from multi-view images under known lighting
conditions. Pulsar [Lassner and Zollhofer 2021] introduces a tilebased acceleration structure for more efficient rasterization. Morerecently, 3DGS [Kerbl et al. 2023] optimizes anisotropic 3D Gaussian primitives, demonstrating real-time photorealistic NVS results.
Despite these advances, using point-based representations from unconstrained multi-view images remains challenging. In this paper,
we demonstrate detailed surface reconstruction using 2D Gaussianprimitives. We also highlight the critical role of additional regularization losses in optimization, showcasing their significant impact
on the quality of the reconstruction.

2.4

Concurrent works

Since 3DGS [Kerbl et al. 2023] was introduced, it has been rapidly
adapted across multiple domains. We now review the closest workin inverse rendering. These work [Gao et al. 2023; Jiang et al. 2023;
Liang et al. 2023; Shi et al. 2023] extend 3DGS by modeling normals
as additional attributes of 3D Gaussian primitives. Our approach,in contrast, inherently defines normals by representing the tangent
space of the 3D surface using 2D Gaussian primitives, aligning themmore closely with the underlying geometry. Additionally, the aforementioned works predominantly focus on estimating the material
properties of the scene and evaluating their results for relighting
tasks. Notably, none of these works specifically target surface reconstruction, the primary focus of our work.
We also highlight the distinctions between our method and concurrent works SuGaR [Guédon and Lepetit 2023] and NeuSG [Chenet al. 2023b]. Unlike SuGaR, which approximates 2D Gaussians with• 3

3D Gaussians, our method directly employs 2D Gaussians, simplifying the process and enhancing the resulting geometry without
additional mesh refinement. NeuSG optimizes 3D Gaussian primitives and an implicit SDF network jointly and extracts the surface
from the SDF network, while our approach leverages 2D Gaussianprimitives for surface approximation, offering a faster and conceptually simpler solution.

3

3D GAUSSIAN SPLATTING

Kerbl et al. [Kerbl et al. 2023] propose to represent 3D scenes with
3D Gaussian primitives and render images using differentiable volume splatting. Specifically, 3DGS explicitly parameterizes Gaussianprimitives via 3D covariance matrix 𝚺 and their location p𝑘 :1
G(p) = exp(− (p − p𝑘 ) ⊤ 𝚺 −1 (p − p𝑘 ))2(1)

where the covariance matrix 𝚺 = RSS⊤ R⊤ is factorized into a scaling matrix S and a rotation matrix R. To render an image, the 3D
Gaussian is transformed into the camera coordinates with worldto-camera transform matrix W and projected to image plane via a
local affine transformation J [Zwicker et al. 2001a]:′

𝚺 = JW𝚺W⊤ J⊤(2)′

By skipping the third row and column of 𝚺 , we obtain a 2D Gaussian
G 2𝐷 with covariance matrix 𝚺2𝐷 . Next, 3DGS [Kerbl et al. 2023]
employs volumetric alpha blending to integrate alpha-weighted
appearance from front to back:c(x) =𝐾∑︁

c𝑘 𝛼𝑘 G𝑘2𝐷 (x)

𝑘=1

𝑘Ö
−1

(1 − 𝛼 𝑗 G𝑗2𝐷 (x))

(3)

𝑗=1

where 𝑘 is the index of the Gaussian primitives, 𝛼𝑘 denotes the alphavalues and c𝑘 is the view-dependent appearance. The attributes of
3D Gaussian primitives are optimized using a photometric loss.
Challenges in Surface Reconstruction. Reconstructing surfaces using 3D Gaussian modeling and splatting faces several challenges.
First, the volumetric radiance representation of 3D Gaussians conflicts with the thin nature of surfaces. Second, 3DGS does not natively model surface normals, essential for high-quality surface
reconstruction. Third, the rasterization process in 3DGS lacks multiview consistency, leading to varied 2D intersection planes for different viewpoints [Keselman and Hebert 2023], as illustrated in
Figure 2 (a). Additionally, using an affine matrix for transforming a
3D Gaussian into ray space only yields accurate projections near thecenter, compromising on perspective accuracy around surrounding
regions [Zwicker et al. 2004]. Therefore, it often results in noisy
reconstructions, as shown in Figure 5.

4

2D GAUSSIAN SPLATTING

To accurately reconstruct geometry while maintaining high-quality
novel view synthesis, we present differentiable 2D Gaussian splatting (2DGS).
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

4•

Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

Tangent frame (u,v)

4.2

Image frame (x,y)

𝑠! 𝐭 !
𝑠" 𝐭 "𝑠 𝐭
𝐩! # #

2D Gaussian Splat
in object space𝑠" 𝐭 "

2D Gaussian Splat
in image space

Fig. 3. Illustration of 2D Gaussian Splatting. 2D Gaussian Splats are elliptical disks characterized by a center point p𝑘 , tangential vectors t𝑢 andt𝑣 , and two scaling factors (𝑠𝑢 and 𝑠 𝑣 ) control the variance. Their ellipticalprojections are sampled through the ray-splat intersection ( Section 4.2) andaccumulated via alpha-blending in image space. 2DGS reconstructs surface
attributes such as colors, depths, and normals through gradient descent.

4.1

Modeling

Unlike 3DGS [Kerbl et al. 2023], which models the entire angularradiance in a blob, we simplify the 3-dimensional modeling by adopting “flat” 2D Gaussians embedded in 3D space. With 2D Gaussianmodeling, the primitive distributes densities within a planar disk,
defining the normal as the direction of the steepest change of density. This feature enables better alignment with thin surfaces. Whileprevious methods [Kopanas et al. 2021; Yifan et al. 2019] also utilize
2D Gaussians for geometry reconstruction, they require a dense
point cloud or ground-truth normals as input. By contrast, we simultaneously reconstruct the appearance and geometry given only
a sparse calibration point cloud and photometric supervision.
As illustrated in Figure 3, our 2D splat is characterized by its
central point p𝑘 , two principal tangential vectors t𝑢 and t𝑣 , anda scaling vector S = (𝑠𝑢 , 𝑠 𝑣 ) that controls the variances of the 2D
Gaussian. Notice that the primitive normal is defined by two orthogonal tangential vectors t𝑤 = t𝑢 × t𝑣 . We can arrange the orientation
into a 3 × 3 rotation matrix R = [t𝑢 , t𝑣 , t𝑤 ] and the scaling factors
into a 3 × 3 diagonal matrix S whose last entry is zero.
A 2D Gaussian is therefore defined in a local tangent plane inworld space, which is parameterized:
𝑃 (𝑢, 𝑣) = p𝑘 + 𝑠𝑢 t𝑢 𝑢 + 𝑠 𝑣 t𝑣 𝑣 = H(𝑢, 𝑣, 1, 1) T

 

𝑠 t
𝑠 𝑣 t𝑣 0 p𝑘
RS p𝑘
where H = 𝑢 𝑢
=
0
0
0 1
0
1

(4)
(5)

where H ∈ 4 × 4 is a homogeneous transformation matrix representing the geometry of the 2D Gaussian. For the point u = (𝑢, 𝑣) in
𝑢𝑣 space, its 2D Gaussian value can then be evaluated by standard
Gaussian 2
𝑢 + 𝑣2G(u) = exp −(6)
2
The center p𝑘 , scaling (𝑠𝑢 , 𝑠 𝑣 ), and the rotation (t𝑢 , t𝑣 ) are learnable
parameters. Following 3DGS [Kerbl et al. 2023], each 2D Gaussianprimitive has opacity 𝛼 and view-dependent appearance 𝑐 parameterized with spherical harmonics.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

Splatting

One common strategy for rendering 2D Gaussians is to project the
2D Gaussian primitives onto the image space using the affine approximation of the perspective projection [Zwicker et al. 2001a,b].
However, as noted in [Zwicker et al. 2004], this projection is only
accurate at the center of the Gaussian and has increasing approximation error with increased distance to the center. To address thisissue, Zwicker et al. proposed a formulation based on homogeneous
coordinates. Specifically, projecting the 2D splat onto an image plane
can be described by a general 2D-to-2D mapping in homogeneous
coordinates. Let W ∈ 4 × 4 be the transformation matrix from worldspace to screen space. The screen space points are hence obtainedbyx = (𝑥𝑧, 𝑦𝑧, 𝑧, 1) T = W𝑃 (𝑢, 𝑣) = WH(𝑢, 𝑣, 1, 1) T

(7)

where x represents a homogeneous ray emitted from the camera andpassing through pixel (𝑥, 𝑦) and intersecting the splat at depth 𝑧. To
rasterize a 2D Gaussian, Zwicker et al. proposed to project its conic
into the screen space with an implicit method using M = (WH) −1 .
However, the inverse transformation introduces numerical instability especially when the splat degenerates into a line segment(i.e., if it is viewed from the side). To address this issue, previous
surface splatting rendering methods discard such ill-conditioned
transformations using a pre-defined threshold. However, such ascheme poses challenges within a differentiable rendering framework, as thresholding can lead to unstable optimization. To address
this problem, we utilize an explicit ray-splat intersection inspiredby [Sigg et al. 2006].
Ray-splat Intersection. We efficiently locate the ray-splat intersections by finding the intersection of three non-parallel planes, amethod initially designed for specialized hardware [Weyrich et al.
2007]. Given an image coordinate x = (𝑥, 𝑦), we parameterize theray of a pixel as the intersection of two orthogonal planes: thex-plane and the y-plane. Specifically, the x-plane is defined by anormal vector (−1, 0, 0) and an offset 𝑥. Therefore, the x-plane canbe represented as a 4D homogeneous plane h𝑥 = (−1, 0, 0, 𝑥) T . Similarly, the y-plane is h𝑦 = (0, −1, 0, 𝑦) T . Thus, the ray x = (𝑥, 𝑦) is
determined by the intersection of the 𝑥-plane and the 𝑦-planes.
Next, we transform both planes into the local coordinates ofthe 2D Gaussian primitives, the 𝑢𝑣-coordinate system. Note thattransforming points on a plane using a transformation matrix M
is equivalent to transforming the homogeneous plane parameters
using the inverse transpose M −T . Therefore, applying M = (WH) −1
is equivalent to (WH) T , eliminating explicit matrix inversion andyielding:
h𝑢 = (WH) T h𝑥

h𝑣 = (WH) T h𝑦

(8)

As introduced in Section 4.1, points on the 2D Gaussian plane arerepresented as (𝑢, 𝑣, 1, 1). At the same time, the intersection pointshould fall on the transformed 𝑥-plane and 𝑦-plane. Thus,
h𝑢 · (𝑢, 𝑣, 1, 1) T = h𝑣 · (𝑢, 𝑣, 1, 1) T = 0(9)This leads to an efficient solution for the intersection point u(x):h2 h4 − h𝑢4 h2𝑣
𝑢 (x) = 𝑢1 2𝑣
h𝑢 h𝑣 − h𝑢2 h1𝑣

h4 h1 − h𝑢1 h4𝑣
𝑣 (x) = 𝑢1 2𝑣
h𝑢 h𝑣 − h𝑢2 h1𝑣(10)

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

where h𝑢𝑖 , h𝑖𝑣 are the 𝑖-th parameter of the 4D homogeneous planeparameters. We obtain the depth 𝑧 of the intersected points via Eq. 7
and evaluate the Gaussian value with Eq 6.
Degenerate Solutions. When a 2D Gaussian is observed from aslanted viewpoint, it degenerates to a line in screen space. Therefore,
it might be missed during rasterization. To deal with these cases andstabilize optimization, we employ the object-space low-pass filter
introduced in [Botsch et al. 2005]:
n
x−c o
)
(11)
Ĝ(x) = max G(u(x)), G(𝜎where u(x) is given by (10) and c is the projection of center p𝑘 .
Intuitively, Ĝ(x) is lower-bounded by a fixed screen-space Gaussian
low-pass√ filter with center c and radius 𝜎. In our experiments, weset 𝜎 = 2/2 to ensure sufficient pixels are used during rendering.
Rasterization. We follow a similar rasterization process as in
3DGS [Kerbl et al. 2023]. First, a screen space bounding box is computed for each Gaussian primitive. Then, 2D Gaussians are sorted
based on the depth of their center and organized into tiles based on
their bounding boxes. Finally, volumetric alpha blending is used tointegrate alpha-weighted appearance from front to back:c(x) =∑︁c𝑖 𝛼𝑖 Ĝ𝑖 (u(x))

𝑖=1

𝑖 −1
Ö

(1 − 𝛼 𝑗 Ĝ𝑗 (u(x)))

(12)

𝑗=1

The iterative process is terminated when the accumulated opacity
reaches saturation.

5

TRAINING

Our 2D Gaussian method, while effective in geometric modeling, canresult in noisy reconstructions when optimized only with photometric losses, a challenge inherent to 3D reconstruction tasks [Barron
et al. 2022b; Yu et al. 2022b; Zhang et al. 2020]. To mitigate thisissue and improve the geometry reconstruction, we introduce tworegularization terms: depth distortion and normal consistency.
Depth Distortion. Different from NeRF, 3DGS’s volume rendering
doesn’t consider the distance between intersected Gaussian primitives. Therefore, spreading out Gaussians might result in a similarcolor and depth rendering. This is different from surface rendering,where rays intersect the first visible surface exactly once. To mitigate this issue, we take inspiration from Mip-NeRF360 [Barron et al.
2022a] and propose a depth distortion loss to concentrate the weight
distribution along the rays by minimizing the distance between the
ray-splat intersections:
L𝑑 =

∑︁

𝜔𝑖 𝜔 𝑗 |𝑧𝑖 − 𝑧 𝑗 |

(13)

𝑖,𝑗

Î −1
where 𝜔𝑖 = 𝛼𝑖 Ĝ𝑖 (u(x)) 𝑖𝑗=1
(1 − 𝛼 𝑗 Ĝ𝑗 (u(x))) is the blending
weight of the 𝑖−th intersection and 𝑧𝑖 is the depth of the intersection
points. Unlike the distortion loss in Mip-NeRF360, where 𝑧𝑖 is the
distance between sampled points and is not optimized, our approachdirectly encourages the concentration of the splats by adjusting theintersection depth 𝑧𝑖 . Note that we implement this regularization
term efficiently with CUDA in a manner similar to [Sun et al. 2022b].

• 5

Normal Consistency. As our representation is based on 2D Gaussian surface elements, we must ensure that all 2D splats are locallyaligned with the actual surfaces. In the context of volume rendering
where multiple semi-transparent surfels may exist along the ray,we consider the actual surface at the median point of intersectionp𝑠 , where the accumulated opacity reaches 0.5. We then align the
splats’ normal with the gradients of the depth maps as follows:∑︁L𝑛 =𝜔𝑖 (1 − n𝑖T N)(14)
𝑖where 𝑖 indexes over intersected splats along the ray, 𝜔 denotes
the blending weight of the intersection point, n𝑖 represents thenormal of the splat that is oriented towards the camera, and N is
the normal estimated by the gradient of the depth map. Specifically,
N is computed with finite differences from nearby depth points asfollows:∇𝑥 p𝑠 × ∇𝑦 p𝑠
N(𝑥, 𝑦) =(15)|∇𝑥 p𝑠 × ∇𝑦 p𝑠 |By aligning the splat normal with the estimated surface normal, we
ensure that 2D splats locally approximate the actual object surface.
Final Loss. Finally, we optimize our model from an initial sparse
point cloud using a set of posed images. We minimize the following
loss function:
L = L𝑐 + 𝛼 L𝑑 + 𝛽L𝑛(16)where L𝑐 is an RGB reconstruction loss combining L1 with the
D-SSIM term from [Kerbl et al. 2023], while L𝑑 and L𝑛 are regularization terms. We set 𝛼 = 1000 for bounded scenes, 𝛼 = 100 for
unbounded scenes, and 𝛽 = 0.05 for all scenes.

6

EXPERIMENTS

We now present evaluations of our 2D Gaussian Splatting reconstruction method, including appearance and geometry comparison
with previous state-of-the-art implicit and explicit approaches. We
then analyze the contribution of the proposed components.

6.1

Implementation

We implement our 2D Gaussian Splatting with custom CUDA kernels, building upon the framework of 3DGS [Kerbl et al. 2023]. We
extend the renderer to output depth distortion maps, depth maps
and normal maps for regularizations (See detailed computations inAppendices A and B of the supplemental material). During training, we increase the number of 2D Gaussian primitives following
the adaptive control strategy in 3DGS. Since our method does notdirectly rely on the gradient of the projected 2D center, we hence
project the gradient of 3D center p𝑘 onto the screen space as an
approximation. Similarly, we employ a gradient threshold of 0.0002
and remove splats with opacity lower than 0.05 every 3000 step. We
conduct all the experiments on a single GTX RTX3090 GPU.
Mesh Extraction. To extract meshes from reconstructed 2D splats,
we render depth maps of the training views using the depth valueof the splats projected to the pixels and utilize truncated signed
distance fusion (TSDF) to fuse the reconstruction depth maps, using
Open3D [Zhou et al. 2018]. We set the voxel size to 0.004 and thetruncated threshold to 0.02 during TSDF fusion. We also extend the
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

6 •Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

Ground truth

Ours (color)

Ours (normal)

Ours

3DGS

SuGaR

Fig. 4. Visual comparisons (test-set view) between our method, 3DGS [Kerbl et al. 2023], and SuGaR [Guédon and Lepetit 2023] using scenes from an real-world
dataset [Barron et al. 2022b]. Our method excels at synthesizing geometrically accurate radiance fields and surface reconstruction, outperforming 3DGS and
SuGaR in capturing sharp edges and intricate details.

explicit implicit

Table 1. Quantitative comparison on the DTU Dataset [Jensen et al. 2014]. Our 2DGS achieves the highest reconstruction accuracy among other methods andprovides 100× speed up compared to the SDF based baselines.

NeRF [Mildenhall et al. 2021]
VolSDF [Yariv et al. 2021]
NeuS [Wang et al. 2021]
3DGS [Kerbl et al. 2023]
SuGaR [Guédon and Lepetit 2023]
2DGS-15k (Ours)
2DGS-30k (Ours)

24
1.90
1.14
1.00
2.14
1.47
0.48
0.48

37
1.60
1.26
1.37
1.53
1.33
0.92
0.91

40
1.85
0.81
0.93
2.08
1.13
0.42
0.39

55
0.58
0.49
0.43
1.68
0.61
0.40
0.39

63
2.28
1.25
1.10
3.49
2.25
1.04
1.01

65
1.27
0.70
0.65
2.21
1.71
0.83
0.83

Table 2. Quantitative results on the Tanks and Temples Dataset [Knapitsch
et al. 2017]. We report the F1 score and training time.

Barn
Caterpillar
Courthouse
Ignatius
Meetingroom
Truck
Mean
Time

NeuS
0.29
0.29
0.17
0.83
0.24
0.45
0.38
>24h

Geo-Neus
0.33
0.26
0.12
0.72
0.20
0.45
0.35
>24h

Neurlangelo
0.70
0.36
0.28
0.89
0.32
0.48
0.50
>24h

SuGaR
0.14
0.16
0.08
0.33
0.15
0.26
0.19
>1h

3DGS
0.13
0.08
0.09
0.04
0.01
0.19
0.09
14.3 m

Ours
0.41
0.23
0.16
0.51
0.17
0.45
0.32
15.5 moriginal 3DGS to render depth and employ the same technique forsurface reconstruction for a fair comparison.

6.2

Comparison

Dataset. We evaluate the performance of our method on various datasets, including DTU [Jensen et al. 2014], Tanks and Temples [Knapitsch et al. 2017], and Mip-NeRF360 [Barron et al. 2022a].
The DTU dataset comprises 15 scenes, each with 49 or 69 images of
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

69
1.47
0.72
0.57
1.43
1.15
0.83
0.81

83
1.67
1.29
1.48
2.07
1.63
1.36
1.36

97
2.05
1.18
1.09
2.22
1.62
1.27
1.27

105
1.07
0.70
0.83
1.75
1.07
0.76
0.76

106
0.88
0.66
0.52
1.79
0.79
0.72
0.70

110
2.53
1.08
1.20
2.55
2.45
1.63
1.40

114
1.06
0.42
0.35
1.53
0.98
0.40
0.40

118
1.15
0.61
0.49
1.52
0.88
0.76
0.76

122
0.96
0.55
0.54
1.50
0.79
0.60
0.52

Mean
1.49
0.86
0.84
1.96
1.33
0.83
0.80

Time
> 12h
>12h
>12h
11.2 m∼ 1h
5.5 m
10.9 mTable 3. Performance comparison between 2DGS (ours), 3DGS and SuGaR
on the DTU dataset [Jensen et al. 2014]. We report the averaged chamferdistance, PSNR (training-set view), reconstruction time, and model size.

3DGS [Kerbl et al. 2023]
SuGaR [Guédon and Lepetit 2023]
2DGS-15k (Ours)
2DGS-30k (Ours)

CD ↓ PSNR ↑ Time ↓ MB (Storage) ↓1.96
35.76 11.2 m
113
1.33
34.57
∼1 h
1247
0.83
33.42
5.5 m
52
0.80
34.52
10.9 m52

resolution 1600 × 1200. We use Colmap [Schönberger and Frahm2016] to generate a sparse point cloud for each scene and downsample the images into a resolution of 800 × 600 for efficiency. We
use the same training process for 3DGS [Kerbl et al. 2023] andSuGaR [Guédon and Lepetit 2023] for a comparison.
Geometry Reconstruction. In Table 1 and Table 3, we compare
our geometry reconstruction to SOTA implicit (i.e., NeRF [Mildenhall et al. 2020], VolSDF [Yariv et al. 2021], and NeuS [Wang et al.
2021]), explicit (i.e., 3DGS [Kerbl et al. 2023] and concurrent work
SuGaR [Guédon and Lepetit 2023]) methods on Chamfer distance2D Gaussian Splatting for Geometrically Accurate Radiance Fields• 7

Table 5. Quantitative studies for the regularization terms and mesh extraction methods on the DTU dataset.

A. w/o normal consistency
B. w/o depth distortion
C. w / expected depth
D. w / SPSR
E. Full Model

Accuracy ↓ Completion ↓ Average ↓
1.35
1.13
1.24
0.89
0.87
0.88
0.88
1.01
0.94
1.25
0.89
1.07
0.79
0.86
0.83

Fig. 5. Qualitative comparison on the DTU benchmark [Jensen et al. 2014].
Our 2DGS produces detailed and noise-free surfaces.
Table 4. Quantitative results on Mip-NeRF 360 [Barron et al. 2022a] dataset.
All scores of the baseline methods are directly taken from their papers
whenever available. We report the performance of 3DGS, SuGaR and ours
using 30𝑘 iterations.

NeRF
Deep Blending
Instant NGP
MERF
BakedSDF
MipNeRF360

Outdoor Scene
PSNR ↑ SSIM ↑ LIPPS ↓
21.46
0.458
0.515
21.54
0.524
0.364
22.90
0.566
0.371
23.19
0.616
0.343
22.47
0.585
0.349
24.47
0.691
0.283

Indoor scene
PSNR ↑ SSIM ↑ LIPPS ↓
26.84
0.790
0.370
26.40
0.844
0.261
29.15
0.880
0.216
27.80
0.855
0.271
27.06
0.836
0.258
31.72
0.917
0.180

Mobile-NeRF
SuGaR
3DGS
2DGS (Ours)

21.95
22.93
24.64
24.34

29.43
30.41
30.40

0.470
0.629
0.731
0.717

0.470
0.356
0.234
0.246

0.906
0.920
0.916

0.225
0.189
0.195

and training time using the DTU dataset. Our method outperforms
all compared methods in terms of Chamfer distance. Moreover, as
shown in Table 2, 2DGS achieves competitive results with SDF models (i.e., NeuS [Wang et al. 2021] and Geo-Neus [Fu et al. 2022])
on the TnT dataset, and significantly better reconstruction than
explicit reconstruction methods (i.e., 3DGS and SuGaR). Notably,
our model demonstrates exceptional efficiency, offering a reconstruction speed that is approximately 100 times faster comparedto implicit reconstruction methods and more than 3 times fasterthan the concurrent work SuGaR. Our approach can also achieve
qualitatively better reconstructions with more appearance and geometry details and fewer outliers, as shown in Figure 5. Moreover,SDF-based reconstruction methods require predefining the sphericalsize for initialization, which plays a critical role in the success of
SDF reconstruction. By contrast, our method leverages radiancefield based geometry modeling and is less sensitive to initialization.
We include the full geometry reconstruction results for both DTU
and TnT in Figure 9 and Figure 10.
Appearance Reconstruction. Our method represents 3D scenes
as radiance fields, providing high-quality novel view synthesis. In
this section, we compare our novel view renderings using the MipNeRF360 dataset against baseline approaches, as shown in Table 4
and Figure 4. Note that, since the ground truth geometry is notavailable in the Mip-NeRF360 dataset and we hence focus on quantitative comparison. Remarkably, our method consistently achieves

Input(A) w/o. NC(B) w/o. DD

Full Model

Fig. 6. Qualitative studies for the regularization effects. From left to right –input image, surface normals without normal consistency, without depth
distortion, and our full model. Disabling the normal consistency loss leads
to noisy surface orientations; conversely, omitting depth distortion regularization results in blurred surface normals. The complete model, employing
both regularizations, successfully captures sharp and flat features.

competitive NVS results across state-of-the-art techniques whileproviding geometrically accurate surface reconstruction. We include
the appearance rendering results in Figure 11.

6.3

Ablations

In this section, we isolate the design choices and measure their effecton reconstruction quality, including regularization terms and meshextraction. We conduct experiments on the DTU dataset [Jensen et al.
2014] with 15𝑘 iterations and report the reconstruction accuracy,completeness and average reconstruction quality. The quantitative
effect of the choices is reported in Table 5. Additional baseline comparisons can be found in Appendix C of the supplemental material.
Regularization. We first examine the effects of the proposed normal consistency and depth distortion regularization terms. Ourmodel (Table 5 E) provides the best performance when applying
both regularization terms. We observe that disabling the normal
consistency (Table 5 A) can lead to incorrect orientation, as shownin Figure 6 A. Additionally, the absence of depth distortion (Table 5B) results in a noisy surface, as shown in Figure 6 B.
Mesh Extraction. We now analyze our choice for mesh extraction.
Our full model (Table 5 E) utilizes TSDF fusion for mesh extraction
with median depth. One alternative option is to use the expected
depth instead of the median depth. However, it yields worse reconstructions as it is more sensitive to outliers, as shown in Table 5
C. Further, our approach surpasses screened Poisson surface reconstruction (SPSR)(Table 5 D) [Kazhdan and Hoppe 2013] using 2D
Gaussians’ center and normal as inputs, due to SPSR’s inability toincorporate the opacity and the size of 2D Gaussian primitives.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

8 •

7

Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

CONCLUSION

We presented 2D Gaussian splatting, a novel approach for geometrically accurate radiance field reconstruction. We utilized 2D Gaussian primitives for 3D scene representation, facilitating accurate andview consistent geometry modeling and rendering. We proposed
two regularization techniques to further enhance the reconstructed
geometry. Extensive experiments on several challenging datasets
verify the effectiveness and efficiency of our method.
Limitations. While our method successfully delivers accurate appearance and geometry reconstruction for a wide range of objects
and scenes, we also discuss its limitations: First, we assume surfaces
with full opacity and extract meshes from multi-view depth maps.
This can pose challenges in accurately handling semi-transparent
surfaces, such as glass, due to their complex light transmission properties, as shown in Figure 12. Secondly, our current densification
strategy favors texture-rich over geometry-rich areas, occasionallyleading to less accurate representations of fine geometric structures.
A more effective densification strategy could mitigate this issue.
Finally, our regularization often involves a trade-off between image
quality and geometry, and can potentially lead to over-smoothingin certain regions.

ACKNOWLEDGMENTS
BH and SG are supported by NSFC #62172279, #61932020, Program of
Shanghai Academic Research Leader. ZY, AC and AG are supportedby the ERC Starting Grant LEGO-3D (850533) and DFG EXC number
2064/1 - project number 390727645.

REFERENCES
Kara-Ali Aliev, Artem Sevastopolsky, Maria Kolos, Dmitry Ulyanov, and Victor Lempitsky. 2020. Neural point-based graphics. In Computer Vision–ECCV 2020: 16th
European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16.
Springer, 696–712.
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo MartinBrualla, and Pratul P. Srinivasan. 2021. Mip-NeRF: A Multiscale Representation for
Anti-Aliasing Neural Radiance Fields. ICCV (2021).
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman.
2022a. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedingsof the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5470–5479.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman.
2022b. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. CVPR (2022).
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman.
2023. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. ICCV (2023).
Mario Botsch, Alexander Hornung, Matthias Zwicker, and Leif Kobbelt. 2005. Highquality surface splatting on today’s GPUs. In Proceedings Eurographics/IEEE VGTC
Symposium Point-Based Graphics, 2005. IEEE, 17–141.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. 2022. TensoRF:
Tensorial Radiance Fields. In European Conference on Computer Vision (ECCV).
Hanlin Chen, Chen Li, and Gim Hee Lee. 2023b. NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance. arXiv preprint arXiv:2312.00846(2023).
Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi. 2023a.
Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field
rendering on mobile architectures. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition. 16569–16578.
Zhang Chen, Zhong Li, Liangchen Song, Lele Chen, Jingyi Yu, Junsong Yuan, and Yi
Xu. 2023c. NeuRBF: A Neural Fields Representation with Adaptive Radial Basis
Functions. In Proceedings of the IEEE/CVF International Conference on Computer
Vision. 4182–4194.
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, andAngjoo Kanazawa. 2022. Plenoxels: Radiance Fields without Neural Networks. In
CVPR.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

Qiancheng Fu, Qingshan Xu, Yew-Soon Ong, and Wenbing Tao. 2022. Geo-Neus:
Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction. Advances in Neural Information Processing Systems (NeurIPS) (2022).
Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. 2023. Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition
and Ray Tracing. arXiv:2311.16043 (2023).
Antoine Guédon and Vincent Lepetit. 2023. SuGaR: Surface-Aligned Gaussian Splattingfor Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering. arXiv
preprint arXiv:2311.12775 (2023).
Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan T Barron, and Paul Debevec. 2021. Baking neural radiance fields for real-time view synthesis. In Proceedingsof the IEEE/CVF International Conference on Computer Vision. 5875–5884.
Wenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen
Ma. 2023. Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural
Radiance Fields. In ICCV.
Eldar Insafutdinov and Alexey Dosovitskiy. 2018. Unsupervised learning of shape andpose with differentiable point clouds. Advances in neural information processing
systems 31 (2018).
Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs. 2014.
Large scale multi-view stereopsis evaluation. In Proceedings of the IEEE conferenceon computer vision and pattern recognition. 406–413.
Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, andYuexin Ma. 2023. GaussianShader: 3D Gaussian Splatting with Shading Functions
for Reflective Surfaces. arXiv preprint arXiv:2311.17977 (2023).
Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction.
ACM Transactions on Graphics (ToG) 32, 3 (2013), 1–13.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 2023.
3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on
Graphics 42, 4 (July 2023). https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/Leonid Keselman and Martial Hebert. 2022. Approximate differentiable rendering withalgebraic surfaces. In European Conference on Computer Vision. Springer, 596–614.
Leonid Keselman and Martial Hebert. 2023. Flexible techniques for differentiable
rendering with 3d gaussians. arXiv preprint arXiv:2308.14737 (2023).
Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. 2017. Tanks andTemples: Benchmarking Large-Scale Scene Reconstruction. ACM Transactions on
Graphics 36, 4 (2017).
Georgios Kopanas, Julien Philip, Thomas Leimkühler, and George Drettakis. 2021. PointBased Neural Rendering with Per-View Optimization. In Computer Graphics Forum,Vol. 40. Wiley Online Library, 29–43.
Christoph Lassner and Michael Zollhofer. 2021. Pulsar: Efficient sphere-based neuralrendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition. 1440–1449.
Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H Taylor, Mathias Unberath, MingYu Liu, and Chen-Hsuan Lin. 2023. Neuralangelo: High-Fidelity Neural Surface
Reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition(CVPR).
Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. 2023. GS-IR: 3D Gaussian
Splatting for Inverse Rendering. arXiv preprint arXiv:2311.16473 (2023).
Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. 2020.
Neural Sparse Voxel Fields. NeurIPS (2020).
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. 2024. Dynamic
3D Gaussians: Tracking by Persistent Dynamic View Synthesis. In 3DV.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas
Geiger. 2019. Occupancy Networks: Learning 3D Reconstruction in Function Space.
In Conference on Computer Vision and Pattern Recognition (CVPR).
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields
for View Synthesis. In ECCV.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields
for view synthesis. Commun. ACM 65, 1 (2021), 99–106.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant
Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans.
Graph. 41, 4, Article 102 (July 2022), 15 pages.
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. 2020. Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D
Supervision. In Conference on Computer Vision and Pattern Recognition (CVPR).
Michael Oechsle, Songyou Peng, and Andreas Geiger. 2021. UNISURF: Unifying Neural
Implicit Surfaces and Radiance Fields for Multi-View Reconstruction. In International
Conference on Computer Vision (ICCV).
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape
Representation. In The IEEE Conference on Computer Vision and Pattern Recognition(CVPR).
Hanspeter Pfister, Matthias Zwicker, Jeroen Van Baar, and Markus Gross. 2000. Surfels:
Surface elements as rendering primitives. In Proceedings of the 27th annual conference2D Gaussian Splatting for Geometrically Accurate Radiance Fields

on Computer graphics and interactive techniques. 335–342.
Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain,
and Matthias Nießner. 2023. GaussianAvatars: Photorealistic Head Avatars with
Rigged 3D Gaussians. arXiv preprint arXiv:2312.02069 (2023).
Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. 2021. KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs. In International
Conference on Computer Vision (ICCV).
Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas
Geiger, Jon Barron, and Peter Hedman. 2023. Merf: Memory-efficient radiance fields
for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics(TOG) 42, 4 (2023), 1–12.
Darius Rückert, Linus Franke, and Marc Stamminger. 2022. Adop: Approximate differentiable one-pixel point rendering. ACM Transactions on Graphics (ToG) 41, 4(2022), 1–14.
Johannes Lutz Schönberger and Jan-Michael Frahm. 2016. Structure-from-Motion
Revisited. In Conference on Computer Vision and Pattern Recognition (CVPR).
Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm.
2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In European
Conference on Computer Vision (ECCV).
Thomas Schöps, Torsten Sattler, and Marc Pollefeys. 2019. Surfelmeshing: Online
surfel-based mesh reconstruction. IEEE transactions on pattern analysis and machine
intelligence 42, 10 (2019), 2494–2507.
Yahao Shi, Yanmin Wu, Chenming Wu, Xing Liu, Chen Zhao, Haocheng Feng, Jingtuo
Liu, Liangjun Zhang, Jian Zhang, Bin Zhou, Errui Ding, and Jingdong Wang. 2023.
GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization. Arxiv(2023). arXiv:2312.05133
Christian Sigg, Tim Weyrich, Mario Botsch, and Markus H Gross. 2006. GPU-based
ray-casting of quadratic surfaces.. In PBG@ SIGGRAPH. 59–65.
Cheng Sun, Min Sun, and Hwann-Tzong Chen. 2022a. Direct Voxel Grid Optimization:
Super-fast Convergence for Radiance Fields Reconstruction. In CVPR.
Cheng Sun, Min Sun, and Hwann-Tzong Chen. 2022b. Improved Direct Voxel Grid
Optimization for Radiance Fields Reconstruction. arxiv cs.GR 2206.05085 (2022).
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping
Wang. 2021. NeuS: Learning Neural Implicit Surfaces by Volume Rendering forMulti-view Reconstruction. Advances in Neural Information Processing Systems 34(2021), 27171–27183.
Yiming Wang, Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, andLingjie Liu. 2023. NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view
Reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer
Vision (ICCV).
Tim Weyrich, Simon Heinzle, Timo Aila, Daniel B Fasnacht, Stephan Oetiker, Mario
Botsch, Cyril Flaig, Simon Mall, Kaspar Rohrer, Norbert Felber, et al. 2007. A
hardware architecture for surface splatting. ACM Transactions on Graphics (TOG)
26, 3 (2007), 90–es.
Thomas Whelan, Renato F Salas-Moreno, Ben Glocker, Andrew J Davison, and Stefan
Leutenegger. 2016. ElasticFusion: Real-time dense SLAM and light source estimation.
The International Journal of Robotics Research 35, 14 (2016), 1697–1716.
Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. 2020. SynSin:
End-to-end View Synthesis from a Single Image. In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR).
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu
Jiang. 2023. PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics. arXiv preprint arXiv:2311.12198 (2023).
Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. 2023. Street Gaussians for Modeling
Dynamic Urban Scenes. (2023).
Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. 2018. MVSNet: DepthInference for Unstructured Multi-view Stereo. European Conference on Computer
Vision (ECCV) (2018).
Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. 2021. Volume rendering of neural
implicit surfaces. Advances in Neural Information Processing Systems 34 (2021),
4805–4815.
Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P. Srinivasan, Richard
Szeliski, Jonathan T. Barron, and Ben Mildenhall. 2023. BakedSDF: Meshing Neural
SDFs for Real-Time View Synthesis. arXiv (2023).
Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, andYaron Lipman. 2020. Multiview Neural Surface Reconstruction by Disentangling
Geometry and Appearance. Advances in Neural Information Processing Systems 33(2020).
Wang Yifan, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine-Hornung.
2019. Differentiable surface splatting for point-based geometry processing. ACM
Transactions on Graphics (TOG) 38, 6 (2019), 1–14.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. 2021.
PlenOctrees for Real-time Rendering of Neural Radiance Fields. In ICCV.

• 9

Zehao Yu, Anpei Chen, Bozidar Antic, Songyou Peng, Apratim Bhattacharyya, Michael
Niemeyer, Siyu Tang, Torsten Sattler, and Andreas Geiger. 2022a. SDFStudio: A Unified Framework for Surface Reconstruction. https://github.com/autonomousvision/
sdfstudio
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. 2024. MipSplatting: Alias-free 3D Gaussian Splatting. Conference on Computer Vision and
Pattern Recognition (CVPR) (2024).
Zehao Yu and Shenghua Gao. 2020. Fast-MVSNet: Sparse-to-Dense Multi-View Stereo
With Learned Propagation and Gauss-Newton Refinement. In Conference on Computer Vision and Pattern Recognition (CVPR).
Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger.
2022b. MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface
Reconstruction. Advances in Neural Information Processing Systems (NeurIPS) (2022).
Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. 2020. NeRF++: Analyzing
and Improving Neural Radiance Fields. arXiv:2010.07492 (2020).
Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. 2018. Open3D: A Modern Library for
3D Data Processing. arXiv:1801.09847 (2018).
Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollhöfer, Justus Thies, and Javier Romero. 2023. Drivable 3D Gaussian Avatars. (2023).
arXiv:2311.08581 [cs.CV]
Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. 2001a. EWA
volume splatting. In Proceedings Visualization, 2001. VIS’01. IEEE, 29–538.
Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. 2001b. Surface
splatting. In Proceedings of the 28th annual conference on Computer graphics andinteractive techniques. 371–378.
Matthias Zwicker, Jussi Rasanen, Mario Botsch, Carsten Dachsbacher, and Mark Pauly.
2004. Perspective accurate splatting. In Proceedings-Graphics Interface. 247–254.

A

DETAILS OF DEPTH DISTORTION

While Barron et al. [Barron et al. 2022b] calculates the distortion
loss with samples on the ray, we operate Gaussian primitives, where
the intersected depth may not be ordered. To this end, we adopt
an L2 loss and transform the intersected depth 𝑧 to NDC space todown-weight distant Gaussian primitives, 𝑚 = NDC(𝑧), with nearand far plane empirically set to 0.2 and 1000. We implemented our
depth distortion loss based on [Sun et al. 2022b], also powered bytile-based rendering. Here we show that the nested algorithm canbe implemented in a single forward pass:

L=

𝑁
−1 ∑︁
𝑖 −1
∑︁

𝜔𝑖 𝜔 𝑗 (𝑚𝑖 − 𝑚 𝑗 ) 2

𝑖=0 𝑗=0
𝑖 −1
𝑖 −1
𝑖 −1
∑︁
∑︁
© ∑︁
ª
𝜔𝑖 ­𝑚𝑖2𝜔𝑗 +𝜔 𝑗 𝑚 2𝑗 − 2𝑚𝑖𝜔 𝑗𝑚 𝑗 ®
𝑖=0
𝑗=0
𝑗=0
𝑗=0
«
¬
𝑁
−1


∑︁
=
𝜔𝑖 𝑚𝑖2𝐴𝑖 −1 + 𝐷𝑖2−1 − 2𝑚𝑖 𝐷𝑖 −1 ,

=

𝑁
−1
∑︁

(17)

𝑖=0

Í𝑖
2 Í𝑖
2
𝑗=0 𝜔 𝑗 , 𝐷𝑖 =  𝑗=0 𝜔 𝑗 𝑚 𝑗 and 𝐷𝑖 = 𝑗=0 𝜔 𝑗 𝑚 𝑗 .
Specifically, we let 𝑒𝑖 = 𝑚𝑖2𝐴𝑖 −1 + 𝐷𝑖2−1 − 2𝑚𝑖 𝐷𝑖 −1 so that theÍdistortion loss can be “rendered” as L𝑖 = 𝑖𝑗=0 𝜔 𝑗 𝑒 𝑗 . Here, L𝑖 mea-

where 𝐴𝑖 =Í𝑖sures the depth distortion up to the 𝑖-th Gaussian. During marching
Gaussian front-to-back, we simultaneously accumulate 𝐴𝑖 , 𝐷𝑖 and𝐷𝑖2 , preparing for the next distortion computation L𝑖+1 . Similarly,
the gradient of the depth distortion can be back-propagated to theprimitives back-to-front. Different from implicit methods where𝑚 are the pre-defined sampled depth and non-differentiable, we
additionally back-propagate the gradient through the intersection
𝑚, encouraging the Gaussians to move tightly together directly.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

10•

Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

Color

Table 6. Additional baselines on DTU dataset. All the models are trainedwith 30𝑘 iterations.

Depth

SuGaR
SuGaR + TSDF
3DGS + SPSR (center)
3DGS + TSDF (mean)
2DGS + SPSR (center)
2DGS (affine approx) + TSDF (mean)
2DGS (our rasterizer) + TSDF (mean)
2DGS (our rasterizer) + TSDF (median)

Ground truth

Ours

3DGS

Fig. 7. Visualization of a plane tiled by 2D Gaussians. Affine approximation [Zwicker et al. 2001b] adopted in 3DGS [Kerbl et al. 2023] causes
perspective distortion and inaccurate depth, violating normal consistency.

B

DEPTH CALCULATIONS

Mean depth: There are two optional depth computations used forour meshing process. The mean (expected) depth is calculated byweighting the intersected depth:∑︁∑︁𝑧 mean =𝜔𝑖 𝑧𝑖 /( 𝜔𝑖 + 𝜖)(18)𝑖improved overall completion metrics compared to results obtainedusing TSDF.
Finally, we conduct ablation experiments on our 2DGS. Notably,
2DGS demonstrates enhanced performance by iteratively integrating components such as TSDF, perspective-correct rasterization,and median depth. For the affine approximation baseline, we utilize
3DGS’s rasterization method by configuring one scale of the 3D
Gaussian to 1𝑒 −6 . While affine approximation already yields promising results, integrating the proposed ray-splat intersection scheme
results in more accurate depth map generation under perspective
projection, as depicted in Figure 7, thus enhancing depth fusion
performance.

𝑖where 𝜔𝑖 = 𝑇𝑖 𝛼𝑖 Ĝ𝑖 (u(x) is the weight contribution of the 𝑖-th
Î −1Gaussian and 𝑇𝑖 = 𝑖𝑗=1
(1 − 𝛼 𝑗 Ĝ𝑗 (u(x))) measures its visibility.
It is important to normalize the depth with the accumulated alphaÍ
𝐴 = 𝑖 𝜔𝑖 to ensure that a 2D Gaussian can be rendered as a planar
2D disk in the depth visualization.
Median depth: We compute the median depth as the largest “visible” depth, considering 𝑇𝑖 = 0.5 as the pivot for surface and freespace:
𝑧 median = max{𝑧𝑖 |𝑇𝑖 > 0.5}.
(19)
We find our median depth computation is more robust to [Luiten
et al. 2024]. When a ray’s accumulated alpha does not reach 0.5,
while Luiten et al. sets a default value of 15, our computation selects
the last Gaussian, which is more accurate and suitable for training.

C

Accuracy ↓ Completion ↓ Average ↓
1.48
1.17
1.33
2.47
1.90
2.18
2.05
1.25
1.65
1.93
1.99
1.96
1.25
0.89
1.07
0.96
1.20
1.08
0.79
0.98
0.88
0.78
0.83
0.80

D

ADDITIONAL RESULTS

Our 2D Gaussian Splatting method achieves comparable performance even without the need for regularizations, as Table 7 shows.
We have included a detailed breakdown of per-scene metrics for the
MipNeRF360 dataset [Barron et al. 2022b] in Table 9. Additionally,
we have provided a comparison of our rendered depth maps withthose from 3DGS and MipNeRF360 in Figure 8.
Table 7. PSNR scores for Synthetic NeRF dataset. Our model achieve comparable performance without using regularizations.

Plenoxels
INGP-Base
Mip-NeRF
3DGS
Ours

Mic
33.26
36.22
36.51
35.36
35.09

Chair
33.98
35.00
35.14
35.83
35.05

Ship
29.62
31.10
30.41
30.80
30.60

Materials
29.14
29.78
30.71
30.00
29.74

Lego
34.10
36.39
35.70
35.78
35.10

Drums
25.35
26.02
25.48
26.15
26.05

Ficus
31.83
33.51
33.29
34.87
35.57

Hotdog
36.81
37.40
37.48
37.72
37.36

Mean
31.76
33.18
33.09
33.32
33.07

Truck
24.40
25.01
26.85

Mean
24.16
25.33
25.56

ADDITIONAL BASELINESIn this section, we present additional baselines to ablate the impactof our design choices, as summarized in Table 6. Furthermore, we
integrate our meshing approach into the comparison against thesebaselines for a comprehensive analysis. SuGaR extracts a mesh from
depth points utilizing SPSR (Screen Poisson Surface Reconstruction)
during the coarse stage, followed by refinement using a mesh renderer. To assess the effect of this meshing strategy, we substituted
SPSR with TSDF using the depth maps, followed by an identicalrefinement stage. However, we found that the depth map generated
from their flat Gaussian intersection is sparse and discontinuous. As
a result, the adaptation of TSDF with its discontinuous depth map
yields inferior results. For 3DGS, we leverage SPSR for mesh generation. Because the 3D Gaussian lacks a surface normal, we treat its
normal as a trainable parameter [Gao et al. 2023; Liang et al. 2023]
distilled from the depth map, employing the normal consistency
regularization. We then utilize all center points for SPSR, resulting in
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

Table 8. PSNR scores for TnT dataset.
SuGaR
3DGS
Ours

Barn
28.63
27.99
28.79

Caterpillar
23.27
24.82
24.23

Courthouse Ignatius
23.33
20.72
23.33
23.95
23.51
23.82

Meetingroom
25.47
26.89
26.15

Table 9. PSNR↑, SSIM↑, LIPPS↓ scores for MipNeRF360 dataset.

SugaR
3DGS
Ours
SuGaR
3DGS
Ours
SuGaR
3DGS
Ours

bicycle
23.34
25.24
24.87
0.634
0.771
0.752
0.354
0.205
0.218

flowers
19.54
21.52
21.15
0.499
0.605
0.588
0.407
0.336
0.346

garden
25.40
27.41
26.95
0.762
0.868
0.852
0.240
0.103
0.115

stump
25.07
26.55
26.47
0.705
0.775
0.765
0.325
0.210
0.222

treehill
21.30
22.49
22.27
0.546
0.638
0.627
0.452
0.317
0.329

room
29.97
30.63
31.06
0.904
0.914
0.912
0.259
0.220
0.223

counter
27.56
28.70
28.55
0.885
0.905
0.900
0.244
0.204
0.208

kitchen
29.41
30.32
30.50
0.902
0.922
0.919
0.178
0.129
0.133

bonsai
30.77
31.98
31.52
0.933
0.938
0.933
0.220
0.205
0.214

mean
25.82
27.20
27.03
0.752
0.815
0.805
0.298
0.214
0.223

2D Gaussian Splatting for Geometrically Accurate Radiance Fields•

11(a) Ground-truth(c) 3DGS, normals from depth gradient(e) Our model (2DGS), normals from depth gradient(b) MipNeRF360 [Barron et al. 2022b], SSIM=0.813(d) 3DGS [Kerbl et al. 2023], SSIM=0.834(f) Our model (2DGS), SSIM=0.845

Fig. 8. We visualize the depth maps generated by MipNeRF360 [Barron et al. 2022b], 3DGS [Kerbl et al. 2023], and our method. The depth maps for 3DGS (d)and 2DGS (f) are rendered using Eq. 18 and visualized following MipNeRF360. To highlight the surface smoothness, we further visualize the normal estimatedfrom depth gradient using Eq. 15 for both 3DGS (c) and ours (e). While MipNeRF360 is capable of producing plausibly smooth depth maps, its sampling
process may result in the loss of detailed structures. Both 3DGS and 2DGS excel at modeling thin structures; however, as illustrated in (c) and (e), the depth
map of 3DGS exhibits significant noise. In contrast, our approach generates sampled depth points with normals consistent with the rendered normal map(refer to Figure 1b), thereby enhancing depth fusion during the meshing process.

SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

12•

Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao

2DGS

scan24

scan37

scan40

scan55

scan63scan65

scan69scan83scan97

scan105

scan106

scan110

scan114

scan118

scan122

scan24

scan37

scan40

scan55

scan63scan65

scan69scan83scan97

scan105

scan106

scan110

scan114

scan118

scan122

3DGS

Fig. 9. Comparison of surface reconstruction using our 2DGS and 3DGS [Kerbl et al. 2023]. Meshes are extracted by applying TSDF to the depth maps.
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

•

13

Barn
Caterpillar

Truck

MeetingRoom

Ignatius
Fig. 10. Qualitative studies for the Tanks and Temples dataset [Knapitsch et al. 2017].

Fig. 11. Appearance rendering results from reconstructed 2D Gaussian disks, including DTU, TnT, and Mip-NeRF360 datasets.

(A) Semi-transparent(B) High light

Fig. 12. Illustration of limitations: Our 2DGS struggles with the accurate reconstruction of semi-transparent surfaces, for example, the glass shown in example(A). Moreover, our method tends to create holes in areas with high light intensity, as shown in (B).
SIGGRAPH Conference Papers ’24, July 27-August 1, 2024, Denver, CO, USA.