Layered Weighted Blended Order-Independent Transparency
TH
Martin
TU
A BSTRACT
Our approach improves the accuracy of weighted blended order- independent transparency, while remaining efﬁcient and easy to implement.
We extend the original algorithm to a layer-based ap- proach, where the content of each layer is blended independently before compositing them globally.
Hereby, we achieve a partial ordering but avoid explicit sorting of all elements.
To ensure smooth transitions across layers, we introduce a new weighting function.
Additionally, we propose several optimizations and demonstrate the method’s effectiveness on various challenging scenes in terms of geometric- and depth complexity.
We achieve an error reduction more than an order of magnitude on average compared to weighted blended order-independent transparency for our test scenes.
1 I NTRODUCTION
Real-time rendering of semi-transparent objects is challenging for a rasterizer.
The classic approach is costly, as it sorts primitives or objects by depth and alpha-blends them back-to-front or front- to-back, which heavily depends on the scene’s depth complexity.
Further, intersecting primitives or objects are difﬁcult to handle.
Order-independent approaches use depth-sorted fragments per pixel via A-Buffers [5] or depth peeling and its variants [4,9].
Peeling costs can be reduced by a constant factor rendering several layers per pass [12] or by applying bin depth peeling [11] which introduces potential artifacts due to collisions within a bin.
These depth peeling solutions do not scale well with depth complexity, reducing the framerate in scenes containing many layers of transparency, e.g. hair or smoke.
Current fast techniques approximate the correct solution.
Easy-to- implement and resource-efﬁcient are the blended order-independent transparency (OIT) operators [14].
They modify the classic alpha blending to make it order-independent at the cost of correctness.
Unfortunately, they cannot always represent depth cues adequately, especially in scenes with many rather opaque surfaces (e.g. foliage) or different colors (e.g. colored smoke).
To address the blending accuracy, we extend the OIT algo- rithm [14] via a layering approach.
We divide the view frustum into multiple depth intervals on which the weighted blended OIT algorithm is applied separately.
Then the intermediate values are blended front-to-back to obtain the ﬁnal result.
This leads to more robust depth cues and improves color precision.
In the following section, we recap the existing order-independent blending operators (Sect. 2), introduce our algorithm (Sect. 3), including two variants and their implementation details (Sect. 4) before discussing results (Sect. 5) and concluding (Sect. 6).
2 R ELATED W ORK
The seminal Porter-Duff algorithm blends two semi-transparent colors C
a and C b with opacity values α a and α b respectively: C dst =
( α a C a +( 1 − α a ) α b C b ) /
( α a +( 1 − α a ) α b ) .
Extending it to more than two surfaces requires ordering the surfaces, which is costly for modern GPU rasterizers.
A plethora of methods has been developed to achieve or approximate transparency on a pixel basis, we refer to [17] for a good overview and focus on the most relevant.
Meshkin et al.
introduced the ”weighted sum” operator [15], which is basically an alpha blending ignoring order-dependent parts of the expanded expression, expressed as: C dst = (cid:0)
∑ ni = 1 C i α i (cid:1)
+
C bg (cid:0) 1 − ∑ ni =
α
i (cid:1) , where C bg denotes the background color, n the number of semi-transparent fragments in front of it.
This operator works well with surfaces of small coverage but becomes increasingly inaccurate for α -values ≥ 0 .
Blending surfaces with large coverage values leads to colors outside of the span of the interpolants, i.e., causes over-darkening or over-brightening of the ﬁnal color.
Replacing all occurrences of surface colors and coverage with their respective coverage-weighted averages, no longer ignores parts of the alpha-blending equation, while maintaining order
indepen- dence:
[Equation]
This removes the overdarkening issue of Meshkin’s operator [15], but two other issues remain.
Firstly, because every surface receives the same average coverage, invisible surfaces (zero coverage) can erroneously affect others [14].
Secondly, there is still no depth information incorporated, so it is impossible to tell if a surface is behind or in front of another surface.
McGuire and Bavoil [14] extended this approach by a pre-pass to determine the coverage α net = 1 − ∏ ni = 1 ( 1 − α i ) to compute
C dst
[Equation]
avg α
[Equation]
+
[Equation]
bg ( 1 − α net ) .
They also incorporated depth by weighting contributions of fragments w.r.t their distance from the camera (Eq. 1).
The weights used come from a monotonic decreas- ing function w ( z , α ) , which has to be carefully tuned for each scene in order to provide good results (Eq.
(7)–(10) in [14]).
[Equation]
Replacing C avg with C dwavg leads to the ﬁnal weighted blended order-independent transparency operator (WBOIT).
[Equation]
The algorithm can have issues with rather opaque surfaces, particu- larly for dense clusters of transparent geometry with large distances between.
Figure 1 shows a stack of squares rendered using the WBOIT technique.
The material is amost fully opaque, i.e α ≈ 1 . 0.
Hence, the frontmost, red square should occlude the others but the close proximity leads to a wrong result using the suggested depthweighting functions [14].
To achieve better results, the ratio w i w i + 1 for two subsequent layers has to be very large.
Rescaling the weighting function helps with accuracy but unfortunately one looses the ability to accurately combine surfaces farther away by doing so.
[Figure]
Figure 2 illustrates our basic algorithm.
The view frustum is divided into a ﬁxed number of depth intervals (”bins”).
After applying
WBOIT to every bin, the per-bin results are blended front-to-back using the under blending operator as described in [4].
Explicit sorting is avoided entirely and, instead, the bins enforce a partial order.
After describing this basic solution (Sec. 3.1), we will increase its robustness, make it temporally coherent, and improve its accuracy (Sec. 3.2–3.4).
To further motivate our approach let us assume non-identical depth of different fragments per pixel.
Let z ∆ be the minimum depth distance between two fragments of the same pixel, and let z min , z max be the depth of the closest and furthest bin boundary respectively.
If the number of bins n fulﬁlls n > ( z max − z min ) /
z ∆ the result of our algorithm is equal to the reference solution.
If two fragment have the same depth value, our algorithm still produces correct results, due to the same depth weights computed by the WBOIT algorithm.
We show an empirical evaluation of the inﬂuence the number of bins have on the overall error in Sec. 5.
[Figure]
3.1 Naive layering
For the naive approach, we apply OIT per bin k (cf.
Eq.
(1) and ((2)), with F k denoting the list of fragment indices in the k -th of B bins:
[Equation]
The individual results are then combined using under blending [4] to discard fragments when the accumulated visibility of the background approaches zero.
Given the opaque background C bg , the ﬁnal pixel color is given by C f
=
A dst B C bg + c dst B , where
[Equation]
This naive approach is accurate if every primitive falls into a single bin (Figure 2).
However, when bins cut through primitives layer artifacts can become visible (Fig. 3 left).
3.2 Smooth layer transitions
To reduce the visibility of layer boundaries (Fig. 3 right) we dis- tribute the contribution of each rendered fragment to multiple bins using a bin weighting function.
Its peak is at the bin’s center and it decreases smoothly with increasing distance.
A tent function seems natural but produces visible artifacts due to being only Ccontinuous.
We chose a bell-shaped function instead:
[Equation]
where z bstart k being the depth of the k -th bins near-plane k , z bend k the depth of the respective far-plane and z the depth value the function is evaluated for.
Its peak is at the center of a bin with a value of 1 .
and it decreases smoothly to 0 .
5 at the bin’s boundaries with σ = 0 .
Instead of considering only the fragments in bin k , we now iterate over all fragments F of a pixel and weigh their color contribution with the bin weight function b k ( z ) :
[Equation]
Additionally we must weigh the fragments’ contribution to the cov- erage in the bin: α net
[Equation]
=
[Equation]
3.3 Normalization
The multiplication of coverage α i with the bin weight b
k
( z i ) in equation (3) introduces some error in the total coverage per bin α net k .
If there are only a few layers of transparent geometry near a bin boundary, the ﬁnal color of that bin tends to get slightly too dark, because of premultiplied colors usage.
It leads to visible layer artifacts, especially at shallow viewing angles (Fig.
3 right).
For correct results the total coverage should be equal to α net from the WBOIT algorithm.
To mitigate this problem, we employ a pre-pass, similar to [14], to compute the correct net coverage of the background.
We further include a normalization factor during blending of the bin’s results.
Under blending then simply becomes another weighted sum:
[Equation]
Note that, due to the normalization, the resulting color is not in premultiplied format anymore.
Therefore, we multiply color
C dst B with coverage α net before blending the result with the scene background.
3.4 Shaping the layers
Using depth maps can reduce artifacts for semi-transparent layers with great success [7,18].
In another pre-pass, we render the depth of the ﬁrst layer of transparent geometry.
Using these depth values, we can offset the bins at each pixel, which increases precision and eliminates the artifacts for the ﬁrst layer (Fig. 4).
The bin boundaries can still cut through the geometry of the remaining layers but these artifacts are typically strongly reduced.
Furthermore, the ﬁnal color is invariant w.r.t view-space depth since layers and depth weighting function are not moving directly with the camera anymore.
Hereby, one issue of the original WBOIT algorithm is ﬁxed.
A dependency on the viewing angle, which has an impact on the depth distance between layers remains (Fig. 5 and 6) but are less noticeable.
Furthermore, if all primitives share the same orientation, all layering artifacts are removed (Fig. 7).
[Figure]
We additionally shift the origin of the weighting function towards the camera to prevent unreasonable depth weights for the ﬁrst layer,
Fig. 8.
This option allows to trade off between accuracy in highly opaque surfaces and accuracy in fairly transparent surfaces using the amount of shift as a single parameter.
[Figure]
4 S TORING AND D ISTRIBUTING F RAGMENTS
To implement our rendering method, we will discuss two implemen- tations to store the rendered fragments: per-pixel linked lists and layered rendering.
The ﬁrst will collect all fragments that will vir- tually be associated with a bin in a postprocess, the second enables us to collect the content directly in bins.
The linked lists method is advantageous in scenes with high geometric but lower depth complexity, while layered rendering is beneﬁcial for scenes with high depth complexity but lower geometric complexity.
Per-Pixel Linked Lists This implementation uses per-pixel linked lists to store the rendered fragments before blending them using our approach from Sec. 3.
While advanced schemes for per- pixel linked lists exist that are cache efﬁcient and robust regarding capacity issues, we chose for an easy-to-implement approach storing color, depth and an index to the next fragment [2].
This solution works well when the manually-set capacity (usually a multiple of the target resolution) is not exceeded.
If the buffer’s capacity is reached, remaining fragments are discarded.
Consequently, complex scenes might require a high memory consumption and can lead to potential buffer overﬂow artifacts.
To produce the ﬁnal image, we apply the layer shaping (Sec. 3.4) implicitly.
The ﬁnal rendering pass without geometry loops over all collected fragments f per pixel, normalizes their position, ﬁnds each respective bin b f , applies the corresponding blending weights w f , and accumulates the result in an array of bin values v [ b f ] .
Finally, the values in v are blended to determine the ﬁnal transparent color that is blended with the background.
Layered Rendering One can render into multiple layers of a 2D-array texture or 3D texture simultaneously with layered rendering.
Each layer represents a bin and the primitives are redirected to the bins based on their spanned depth interval from within the
GPU’s geometry shader.
In practice, geometry-shader instancing allows us to invoke the shader a user-deﬁned number of times per primitive.
As 32 instances are guaranteed on current hardware [10], we render batches of 32 bins per pass, adding additional passes if more than 32 bins are requested, and blend the result batch by batch.
To produce the ﬁnal image, the span of each triangle is detected and if the instance intersects the depth interval of the corresponding bin, we output the triangle via layered rendering.
We apply the weights corresponding to the bin and the fragments position in the fragment shader.
In a ﬁnal render pass without geometry, we blend the different layers together.
Unfortunately, when layer shaping is used, the span of a triangle would need to be determined via the depth map.
NBuffers [6] and
Mipmaps can be used to compute an estimate by extracting the min/max value over a bounding square but this adds an overhead to the method.
In practice, we emit the triangle to all layers and resolve the span in the fragment shader.
For a small amount of layers, one can also use multiple render targets (current GPUs typically support eight) to avoid geometric duplications.
5 E VALUATION
We evaluated our approach on seven scenes and compared to the original WBOIT algorithm
[14] and depth peeling [9] as a reference solution (Fig. 12).
The scenes all feature different depth- and geo- metric complexity and are summarized in Table 1.
The distribution of non-opaque fragments is highly non-uniform in image space and depth (Fig. 9).
This challenges our algorithm, as a uniform distribution would be best for performance.
Table 2 shows the tested algorithms and abbreviations used in the diagrams.
All measurements were conducted on an AMD Ryzen 7 3700X CPU with an 8GB AMD Radeon RX 5700XT at a 1280 × 720-pixel resolution
[Table]
scenes.
GC: Geometric Complexity (num. non-opaque triangles), DC: Depth Complexity (avg./max.
num.
non-opaque fragments per pix and bin), BG: Background.
The number of bins per scene is listed in Fig.
[Figure]
We measured the average rendering time for transparent geometry and blending with the opaque background (Fig. 11).
Mean squared error with respect to the depth-peeling reference is reported globally and per-pixel in a heatmap (Fig. 12).
For qualitative comparisons, we chose the highest number of layers/bins, while maintaining real- time performance ( > 25 fps) on our system with all optimizations enabled.
We also analyze the performance and mean squared error with respect to the number of bins (Fig. 10).
5.1 Results
We provide performance measurements (Fig. 9–11) and visual com- parisons (Fig. 12).
Our baseline algorithms perform as expected
[Figure]
with WBOIT, a very simple two-pass algorithm, being the fastest technique in all cases at the cost of reduced quality and DP being the slowest with highest quality.
Bin count: Fig.
10 plots the effect of the bin count vs. MSE (top row) and rendering time (bottom row) exemplarily for two test scenes (the results for all test scenes are available in the supplemen- tal material).
We see for most scenes an exponential reduction in the MSE and a clear convergence towards the reference solution which reinforces our theoretical considerations from Sec. 3.
Small irregularities in convergence can appear as neighboring fragments in depth might or might not fall into the same bin, depending on the number of bins.
The only exception is the grey smoke scene as the number of bins becomes irrelevant with high depth complexity but low color variance.
Looking at the bin count vs. frametime we can see a few inter- esting patterns in the data.
In geometrically complex scenes, such as ”power plant”, the linked list approach works extremely well and the limiting factor here is the rendering itself.
On the other hand the layered rendering approach struggles to deliver good performance in these scenes as multiple rendering passes are required.
The sudden ”jumps” in the performance are due to the limit on the number of rendering targets that can be rendered during one pass.
Scenes with a low geometric complexity, such as ”grey/colored smoke” and ”burning wood”, show a clear linear increase in rendering time with respect to the number of bins which is desired as it shows that no bigger additional overhead is produced by our algorithm.
Squares: Very low geometric and depth complexity.
Our linked-list versions outperform layered rendering due to the overhead of the geometry-shader stage.
Increasing the bin count from 32 to 64 leads to a signiﬁcant increase in frame times because two 32-batches of geometry shader invocations are now needed instead of just one.
Color accuracy using our approach is improved reducing the MSE compared to WBOIT by a factor of 46 × .
[Figure]
f r a m e t i m e i n m
[Figure]
[Table]
Here mostly becomes
Since all
[Table] [Table]
[Table] [Table]
optimal
WBOIT because all particles share a uniform color, which hides blending ar- tifacts.
Still, color accuracy improves by a factor of 1 .
× compared to WBOIT.
Burning wood: Medium geometric and depth complexity.
We included this one to test rendering of foliage, which combines very opaque and very transparent objects.
WBOIT attributes an insufﬁ- cient contribution to the nearby needles with respect to the orange particles, creating the impression of needles disappearing.
This leads to clearly visible artifacts around the stem.
Layered rendering produces more accurate images improving the results by a factor of 1 . 3 × compared to WBOIT.
Engine: Medium geometric and depth complexity.
This scene is typical for CAD design, where surfaces are often semi-transparent.
Layered rendering suffers when using a too low number of bins (here 16).
WBOIT cannot cope well with the different materials encountered, resulting in a higher overall error but also strong error spikes in several positions.
The linked lists work well in this scene reducing the MSE compared to WBOIT by a factor of 4 . 4 × .
5.2 Discussion
Our layering approach enhances color accuracy and can improve depth cues.
When applying layer shaping, artifacts are eliminated to a large extent and it partially solves the problem of color varia- tions when the distance to the camera changes.
However, a slight dependence on the viewing angle is introduced, which can be visible for very simple and consistent shapes, such as the squares.
Organic
[Figure]
Scene: squares
[Figure] [Figure]
shapes, like the Stanford Dragon, are less susceptible to this effect and most layering artifacts are hard to perceive.
The linked-list approach works well for complex transparent ge- ometry but runs into memory issues for a high depth complexity.
Using layered rendering introduces an overhead for complex
ge- ometry but delivers good performance for high depth complexity because only a portion of the fragments must be held in memory at any given time.
High geometric and depth complexity remains a challenge (Table 3).
Still, our algorithm handles several scenes that are problematic for existing algorithms.
For example, it renders particle systems well, which rely heavily on sorting, as well as de- tailed transparent objects, which would typically result in a serious reduction of quality.
[Table]
6 C ONCLUSIONS
We introduced a layered solution of weighted blended order- independent transparency that improves color accuracy.
We achieved smooth layer transitions by introducing a bin weighting function and shaping the bin decomposition based on the ﬁrst layer of semitransparent surfaces.
Our approach is easy to implement and builds upon two variants; per-pixel linked lists and layered rendering.
It avoids sorting of fragments but enforces a partial order implicitly, the latter enabling results closer to a reference solution.
Our algorithm shows good performance for challenging cases, such as particles with high depth but relatively low geometric complexity, or scenes with only a few layers of transparency but rather complex geometry and therefore closes a gap in existing OIT algorithms.
In the future, we want to test techniques such as K-Buffers [3] or stochastic transparency [8] to bound the memory requirements and optimize the list structures, e.g., by coupling them to screen tiling for a reduced memory consumption, increased throughput and improved cache friendliness.
A CKNOWLEDGMENTS