China Unity Tech Week 2020

This article contains AI-generated translation.

 Author
 Published

## Introduction

近日，首届中国Unity线上技术大会顺利开幕。和往届相似，大会邀请了行业内许多专家聚首一堂，给观众们分享关于游戏制作的经验和技术。
Recently, the inaugural China Unity Online Tech Conference successfully kicked off. Similar to previous editions, the conference invited many industry experts to come together and share their experiences and technical knowledge about game development with the audience.

在昨晚的游戏分会场上，米哈游的技术总监弋振中以「从手机走向走主机」为主题，分享了《原神》在PS4的渲染技术。「原神」相信你们都很熟悉了，它是一款开放世界游戏，目前登陆了PC、iOS、Android、PS4四个平台。
During the game session held last night, miHoYo's Technical Director, Yi Zhenzhong, presented on the topic of "From Mobile to Console," sharing the rendering techniques of *Genshin Impact* on PlayStation 4. *Genshin Impact* is a well-known open-world game that is currently available on four platforms: PC, iOS, Android, and PS4.

弋振中曾在美国和中国的游戏公司工作多年，有十余年的主机游戏开发经验。回国前，他在西雅图的微软Xbox任职，19年初回国加入了“米哈游”，组建了自己的主机开发团队。
Yi Zhenzhong has extensive experience in both American and Chinese game companies, with over a decade of console game development experience. Prior to returning to China, he worked at Microsoft Xbox in Seattle. In early 2019, he joined miHoYo and established his own console development team.

在项目经验方面，他曾负责《正当防卫3》《正当防卫4》《腐烂国度2》等主机游戏的开发。
In terms of project experience, he has been involved in the development of console games such as *Just Cause 3*, *Just Cause 4*, and *State of Decay 2*.

以下是手游那点事整理的部分演讲实录，为提升阅读体验，内容有删减和调整：
The following is a partial transcript of the speech compiled by Mobile Game Affairs. To enhance the reading experience, some content has been edited and adjusted:

## Lecture

大家好，我叫弋振中，今天我的分享主要分为三个部分。
Hello everyone, my name is Yi Zhenzhong. Today, my presentation will be divided into three parts.

> 1. 《原神》主机平台的一些基本情况；
> An overview of the basic situation of *Genshin Impact* on the console platform;
>
> 2. 按照我们开发时改造渲染管线的思路，选择部分的技术点进行解析；
> Analysis of selected technical points based on our approach to modifying the rendering pipeline during development;
>
> 3. 个人的一些开发体会。
> Some personal development insights.
>
>

首先，Unity是一款灵活度很高的游戏引擎，代码风格简洁，可以让我们更方便地定制化开发《原神》的渲染管线。在这里，我对Unity中国的技术支持表示感谢。
Firstly, Unity is a highly flexible game engine with clean code style, which allows us to customize the rendering pipeline of *Genshin Impact* more conveniently. I would like to express my gratitude to Unity China for their technical support.

《原神》的主机版本主要是针对PS4，而我们开发过程中超过一半的精力都用在如何利用主机硬件架构的开发和优化上面，对此我们也积累了相当多的经验，也有不错的技术实现。今天分享的技术主要涉及渲染管线部分。
The console version of *Genshin Impact* mainly targets the PS4 platform. During our development process, over half of our efforts were devoted to utilizing and optimizing the hardware architecture of the console. As a result, we have accumulated a significant amount of experience and achieved some impressive technical implementations. Today's presentation will mainly focus on the rendering pipeline.

### Overview of *Genshin Impact* on Consoles and an Introduction to the Rendering Pipeline

我们在Unity上对《原神》进行了深度的开发，手机平台和主机平台分别采用了两种不同的渲染管线。但是，游戏的基调都是基于PBR的风格化渲染。
We have undertaken in-depth development of *Genshin Impact* on the Unity platform, employing two different rendering pipelines for the mobile and console platforms. However, the game's overall style is based on PBR (Physically Based Rendering) techniques with stylized rendering.

游戏基于PBR的渲染模式，是想让整个大环境的光影效果保持统一。
The game adopts a PBR (physically-based) rendering mode to ensure consistent lighting and shadow effects throughout the entire environment.

因为我们的光影系统是实时计算的，有24小时循环，也有动态的天气系统。而PBR可以确保在不同的光照条件下，不会脱离我们预期的渲染效果。
Since our lighting and shadow system is calculated in real-time and includes a 24-hour cycle with dynamic weather, PBR ensures that the rendering effects remain as expected under different lighting conditions.

作为风格化的游戏，也需要根据美术的需求修改不同的材质。
As a stylized game, we also need to modify different materials based on the artistic requirements.

《原神》在PS4上，我们把1440P作为渲染分辨率，最后输出到1080p上面，这样得到的图像会更加清晰和锐利。
On the PS4 platform, we set 1440p as the rendering resolution, which is then downscaled to 1080p for the final output. This results in sharper and more defined images.

《原神》的风格化渲染是很独特的，尤其是光影效果，大家可以看到「脏、黑、死、焦、噪」这些词都频繁地出现了美术和程序的沟通当中。
The stylized rendering in *Genshin Impact* is quite unique, especially when it comes to lighting and shadow effects. You will often hear words like "dirty, dark, dead, burnt, noisy" frequently mentioned in the communication between the art and programming teams.

### Technical Challenges and Analytical Approaches

《原神》的主机版开发可以说是从零开始。
The development of the console version of *Genshin Impact* can be said to start from scratch.

一开始的主机团队就我一个人，光杆司令。为了全球同步上市，留给我们的开发时间大概只有一年半，还伴随着大量PS4独有的问题需要解决，工作量和工作难度都非常大。
Initially, I was the only member of the console team, leading the way. In order to have a simultaneous global release, we had a development time of approximately one and a half years, accompanied by a multitude of PS4-specific issues that needed to be addressed. The workload and difficulty of the task were both significant.

因为开发的时间很紧，所以我们在选择技术改造点的时候，遵循下面几个原则：
Due to the tight development schedule, we followed the following principles when selecting the areas for technical improvements:

> 1. 拒绝堆砌“feature”，排除开发周期长，需要过多前期研究工作的功能；
> Rejecting the accumulation of "features" that would prolong the development cycle and require excessive preliminary research.
>
> 2. 尽量选择成熟实用的技术，因为没有时间试错；
> Choosing mature and practical technologies whenever possible, as there was no time for trial and error.
>
> 3. 选择对画质帮助更大的地方去提升，希望新功能之间相互互动，让画面更加系统化，得到的画质提升也会有1+1>2的效果。
> Focusing on areas that would have a greater impact on visual quality, aiming for interactive effects between new features to create a more systematic visual experience, where the enhancement of visual quality would exceed the sum of its parts.
>
>

下面就是我们做一些技术点的解析，以及提供一些优化思路。
Now, let's analyze some of the technical points we worked on and provide some optimization ideas.

场景光影方面，第一个是关于方向光的阴影。
Regarding the lighting and shadows in the game environment, the first point is about directional light shadows.

《原神》的大量游戏时间是在室外，室外方向光的阴影质量非常重要。一方面，近处的阴影细节需要更加细腻，才显得画面更干净；另一方面，阴影覆盖范围需要足够大，因为游戏的可视距离非常远，有800米的范围。
A significant portion of *Genshin Impact* gameplay takes place outdoors, where the quality of directional light shadows is crucial. On one hand, the nearby shadow details need to be delicate to create a cleaner visual presentation. On the other hand, the shadow coverage needs to be extensive since the game has a long visible distance, up to 800 meters.

大家可以看下这张贴图。即使在远处墙壁上一小片的绿植产生的阴影，在放大之后都能够看到树叶的轮廓，而且非常稳定。
Take a look at this texture. Even a small patch of vegetation casting shadows on a distant wall can display the outline of leaves when zoomed in, and the shadows remain stable.

我们阴影的技术比较常规，使用了Cascaded shadow map（一种游戏实时阴影技术）加上基于Poisson disc（泊松分布算法）的soft shadow（软阴影），我们游戏没有使用通常的4级cascades（一种数据优化器），而是用了8级，这属于“大力出奇迹”的方式。
Our shadow technology follows a more conventional approach. We utilize Cascaded Shadow Maps, a real-time shadow rendering technique, combined with soft shadows based on the Poisson disc algorithm. Instead of the typical 4-level cascades optimization, we employ 8-level cascades, which can be seen as going above and beyond.

虽然这样带来了更好的阴影效果，也带来了更多的性能开销。更多的drawcall（CPU命令GPU进行的图片渲染操作）和cascades，也会加重GPU和CPU的负担。
While this provides better shadow effects, it also comes with increased performance overhead. The higher number of draw calls (CPU commands for GPU rendering) and cascades puts additional strain on both the GPU and CPU.

质量上去了之后，我们就要想办法解决性能问题。
Once we achieved the desired quality, we needed to address the performance issues.

首先在CPU端，我们做了一个shadow cache，8级cascades前4级我们每帧都更新，后面4级是采用轮流更新的方式，确保每8帧所有的cascades都能至少更新一次。每一帧的话，我们只更新5级cascades。
Firstly, on the CPU side, we implemented a shadow cache. We update the first 4 cascades every frame, while the remaining 4 cascades are updated in a round-robin fashion, ensuring that all cascades are updated at least once every 8 frames. Each frame, we only update the 5th-level cascade.

主要的工作量其实在GPU端。8级cascades下，screen space shadow map（屏幕空间深度阴影贴图）的开销长期是大于2毫秒的，在某些情况之下能够超过2.5毫秒。GPU比4级cascades的情况下，也爆涨了0.5到0.8毫秒。
The majority of the workload lies on the GPU side. Under the 8-level cascades, the cost of screen space shadow map (SSSM) rendering is consistently above 2 milliseconds and, in certain cases, can exceed 2.5 milliseconds. Compared to the 4-level cascades, the GPU workload has increased by 0.5 to 0.8 milliseconds.

因为软阴影采用的是泊松分布的采样，这一整套的操作都是很重量级的。但实际上不需要对每个像素都要做这么多操作。
Due to the adoption of Poisson disc sampling for soft shadows, the entire process is computationally intensive. However, it is unnecessary to perform these operations for every pixel.

所以我们优化思路是尽量只在必要的地方做软阴影计算，我们会生成一张Mask贴图，在贴图里面标出阴影、半影和非影片区。
So our optimization approach is to perform soft shadow calculations only where necessary. We generate a mask texture that identifies shadow, penumbra, and non-shadow regions.

阴影区和非阴影区只需要直接返回0和1就好了，只有在半影区才会去计算软阴影，通过这种方式，我们的GPU开销大致减少了30%左右。甚至比采用4级cascades还要再快一些。
For shadowed and non-shadowed regions, we simply return 0 and 1, respectively. Soft shadow calculations are only performed in the penumbra region. Through this approach, we have reduced GPU overhead by approximately 30%. It is even faster than using 4-level cascades.

（图里面被红色标注的区域，是半影区，需要做软阴影处理。其他的区域，就是在阴影区域或者是非阴影区域，直接返回0和1）
(The areas highlighted in red in the image are the penumbra regions where soft shadow processing is required. The other regions correspond to shadowed or non-shadowed areas, and we directly return 0 and 1, respectively.)

上面这张Mask贴图的分辨率是屏幕分辨率的1/4×1/4，也就是说一个Mask值对应的是一个4×4的block。然后我们对4×4的block里面的每一个像素，来判断它是不是在阴影中，最后汇总成一个阴影、半影和非阴影的三个状态，保存到Mask贴图里。
The resolution of the mask texture shown above is 1/4 × 1/4 of the screen resolution. This means that each mask value corresponds to a 4x4 block. We examine each pixel within the 4x4 block to determine whether it is in shadow, and then we aggregate the results to obtain the three states: shadow, penumbra, and non-shadow. These states are saved in the mask texture.

这样我们能够得到一个准确的半影信息，但是它不够快，所以我们做了进一步的优化，只选择4×4这个block里面很少的几个像素，来判断是不是在阴影当中。
While this approach provides accurate penumbra information, it is not fast enough. Therefore, we further optimize it by selecting only a few pixels within the 4x4 block to determine if they are in shadow.

这样会容易出现一些误差，所以我们把这样计算得到了Mask贴图做了模糊处理，让半影的区域稍微扩散出去。整个Mask贴图的生成，包括模糊处理大概的开销是在0.3毫秒左右。大家可以看一下对比图。
This may introduce some errors, so we apply a blur effect to the generated mask texture to slightly spread the penumbra region. The overall cost of generating the mask texture, including the blur processing, is approximately 0.3 milliseconds. You can see the comparison in the images.

（肉眼几乎看不出来区别）
(The difference is barely noticeable to the naked eye.)

这样优化完之后，我们的GPU开销时间大概稳定在1.3到1.7毫秒。
After implementing these optimizations, our GPU processing time stabilized at around 1.3 to 1.7 milliseconds.

把阴影搞好以后，下面我们来看看AO（Ambient Occlusion环境光遮蔽）。
Now, let's take a look at Ambient Occlusion (AO).

大家先考虑一种情况，人物和场景的物体都已经处在山或建筑物的阴影当中，这个时候人物和物体的投影跟山和建筑的投影是融为一体的，画面缺乏对比，人和物体就会显得浮空。
Consider a situation where both characters and objects in the scene are already in the shadow of mountains or buildings. In this case, the shadows of the characters and objects blend with the shadows of the mountains and buildings, resulting in a lack of contrast and making the characters and objects appear floating.

为了解决这个问题，我们在游戏里面采用了多种的AO技术，针对不同的情景生成不同的AO。
To address this issue, we employ various AO techniques in the game to generate different types of AO for different scenarios.

#### Conventional HBAO

大家可以看一下这是HBAO开关的对比图，效果还是很明显的。
You can see the comparison image with the HBAO toggle on, and the effect is quite noticeable.

#### AO Volume for Static Objects

下面这个是AO Volume的开关情况，大家可以重点看一下我们在红圈里面的区域，椅子对地面产生了柔和的投影。
The following image shows the AO Volume toggle, and I would like you to pay attention to the area highlighted in the red circle. You can see that the chair is casting a subtle and soft shadow on the ground.

因为技术原理和性能的限制，HBAO是没办法针对类似桌子或者椅子产生大面积AO，AO Volume这个时候就体现很好的补充。
Due to technical principles and performance limitations, HBAO cannot generate extensive AO for objects like tables or chairs. This is where AO Volume serves as a valuable complement.

要实现AO volume，首先我们是在离线的时候对需要产生AO volume的物体做一个遮挡信息的计算。这个计算是在物体的本地空间（Local space）去做的，生成的遮挡信息我们保存下来，在运行的时候注入到volume texture中去使用。
To implement AO Volume, we first calculate occlusion information for objects that require AO Volume during an offline process. This calculation is done in the object's local space, and the generated occlusion information is saved. During runtime, we inject this information into a volume texture for utilization.

#### Capsule AO for Dynamic Objects

下面是关于Capsule AO的对比图，大家可以重点看一下被我们红色的圈给圈出来的区域。大家可以看到相邻在屏风和地面，能够产生出能够反映体形和人影的投影。而且如果在游戏中大家去观察的话，随之相邻动作的改变，阴影的形状也会随之产生变化。
The following image shows a comparison of Capsule AO, and I would like you to pay attention to the area highlighted by the red circle. You can see that adjacent to the folding screen and the ground, it generates shadows that reflect body shape and human figures. Additionally, if you observe in the game, you will notice that the shape of the shadows changes along with adjacent movements.

Capsule AO的做法就是用一些胶囊体包裹住人物的四肢和躯干，这些胶囊体和角色的骨骼动画绑定进行同步更新。然后这些胶囊体会被用来做遮挡计算，计算的时候我们把它分为无方向的环境遮挡计算，以及带方向的遮挡信息计算。
The approach of Capsule AO involves encapsulating the limbs and torso of the character with capsules. These capsules are synchronized and updated with the character's skeletal animation. They are then used for occlusion calculations, which include both non-directional ambient occlusion and directional occlusion information.

带方向的遮挡信息计算采用的方向是主光源方向和法线进行混合之后的得到的虚拟遮挡方向。通过这种方式，角色可以同时在周围的墙和地面等投出多个阴影。
The directional occlusion information calculation combines the main light source direction with the normals to obtain a virtual occlusion direction. This allows the character to cast multiple shadows on surrounding walls and surfaces simultaneously.

#### Optimization Techniques for AO

《原神》的AO都是在1/2×1/2分辨率的RT（Render Texture）上去做计算。我们对AO还做了一个模糊处理（blur）。然后再Upsample一个全分辨率的贴图上面去。
In *Genshin Impact*, all AO calculations are performed on a 1/2×1/2 resolution Render Texture (RT). We also apply a blur to the AO and then upsample it onto a full-resolution texture.

所有的模糊处理和Upsample pass（上采样），我们都用了一个Bilateral filter（双边滤波算法），确保不会有无效的AO渗透到周围的区域。
For all the blur and upsample passes, we use a Bilateral filter algorithm to ensure that no invalid AO seeps into the surrounding areas. This helps maintain the integrity of the AO without unwanted artifacts.

模糊处理和Upsample加起来一共有三个pass，这就意味着AO需要被读取和写入多次。所以我们采用的优化方式是将所有的计算都放到一个compute pass里面去做。
The blur and upsample process consists of three passes, which means that AO needs to be read from and written to multiple times. To optimize this, we perform all calculations in a single compute pass.

然后通过LDS来保存blur的中间值，通过同时输出四个像素的方式，来重用相邻像素的计算。最终我们还可以通过async compute pipe（游戏引擎中的异步计算）把性能开销进一步降低。
We use LDS (Local Data Share) to store intermediate values during the blur process. By simultaneously outputting four pixels, we can reuse calculations from neighboring pixels. Additionally, we can further reduce the performance overhead by utilizing async compute pipe, which leverages asynchronous computing in the game engine.

#### About Local Light

我们在游戏里面采用了Clustered deferred lighting（集群延迟和前向着色），支持是视野内同时出现最多1024盏灯。大概的做法是我们将屏幕分成64×64像素的tile，然后每一个tile在深度的方向上面继续分为16级的clusters。
In the game, we employ Clustered Deferred Lighting, which supports up to 1024 lights simultaneously within the field of view. The approach involves dividing the screen into tiles of 64×64 pixels and further subdividing each tile into 16 clusters along the depth direction.

（游戏内许多灯源表现）
(Many light sources in the game are represented in the following ways)

多个不同的Local Light，它们的照明范围是交错存在的，然后角色也投下多个不同的阴影朝不同的方向，画面就显得细节很丰富。
Multiple different Local Lights are present, and their lighting ranges are staggered. The characters also cast multiple shadows in different directions, creating a visually rich scene with abundant details.

（通过Local Light的阴影提升画面）
Enhancing the visuals through Local Light shadows

我们的Local Light 阴影系统支持接近100盏灯的实时阴影。阴影的分辨率是根据优先级和距离进行动态调整，最终的阴影是通过烘焙的静态场景阴影和实时生成的动态场景阴影结合得到的。
Our Local Light shadow system supports real-time shadows for nearly 100 lights. The shadow resolution is dynamically adjusted based on priority and distance. The final shadows are a combination of baked static scene shadows and dynamically generated dynamic scene shadows.

然后我们需要一个好的算法来对于烘焙的shadow texture做一个压缩。这个压缩需要在精度损失足够低的同时，还要保持压缩率足够高，同时我们的解压开销要非常小才行。
We need an efficient algorithm to compress the baked shadow textures. This compression needs to achieve a high compression ratio while maintaining a low level of precision loss. Additionally, the decompression cost should be minimal.

我们开发的这个系统是在离线制作的时候，对于shadow texture做一个压缩，尽量地去保持精度，运行的时候解压的速度也非常快，用compute shader（一种Unity的技术）去解压的情况，1K×1K的shadow texture，我们解压只需要0.05毫秒。
The system we developed involves compressing the shadow texture during offline production, aiming to maintain accuracy while ensuring fast decompression during runtime. We utilize compute shaders, a technology in Unity, to perform the decompression. For a 1K×1K shadow texture, the decompression process only takes 0.05 milliseconds.

接下来我介绍一下算法思路。
Now, let me explain the algorithm's approach.

首先我们对于shadow texture按照一个2×2的block来进行编码，每4个深度值，我们用32bit来保存。如果想要降低精度损失，可以选择高精度压缩，这种情况之下每个block的大小变成64bit。
Firstly, we encode the shadow texture using a 2×2 block structure, where we save 32 bits for every 4 depth values. To reduce precision loss, you can opt for high-precision compression, which increases the block size to 64 bits.

编码的方式有两种，一种是基于深度平面方程的方式，或者是通过压缩的浮点数方式。编码完成之后，还要进一步通过一个quad tree来合并编码以后的数据，进一步提高压缩率。quad tree是每个tile要保存一个，而每个tile又包含了16×16个block。
There are two encoding methods available: one based on depth plane equations and the other using compressed floating-point numbers. After encoding, we further improve the compression rate by merging the encoded data using a quadtree. Each tile contains a quadtree, and each tile consists of 16×16 blocks.

（三个图从左到右分别是，没有压缩的深度贴图，中间是我们的平面方程编码的视图，最右边是我们quad tree 0到4级的深度视图。）
(The three images from left to right depict the uncompressed depth map, the view of our plane equation-based encoding, and the depth view of our quadtree at levels 0 to 4.)

大家可以看一个对比，默认精度压缩，能够看到红圈里面有一些瑕疵。这个实际上是我们找到的可以说是最差的一个情况。
You can observe a comparison, where the default precision compression reveals some flaws within the red circle. This particular case can be considered the worst-case scenario we have encountered.

这是高精度压缩，基本上看不出任何瑕疵来。
This is high-precision compression, and practically no flaws can be discerned.

#### Adding Volumetric Fog

体积雾是可以接受Local Light的照明影响，在灯的影响范围内形成一圈光晕，可以极大地提升画面的体积感。
Volumetric fog accepts the illumination influence of Local Lights and creates a halo within the range of the light, greatly enhancing the sense of volume in the visuals.

（近处的灯笼周围会有一圈泛光。包括画面远处的建筑物，因为笼罩在灯光下会使得周围的体积雾也被照亮，而显得有一丝的朦胧）
(There will be a halo around the lanterns in close proximity. Even the buildings in the distance will be illuminated by the surrounding volumetric fog under the influence of the light, creating a subtle haze.)

下面这个图是一个更有意思的情况。如果我们给Local Light加一个projection texture，也就是我们通过这个贴图来控制Local Light光照的形状，就像右边这样，体积雾也会产生相应的变化。
The following image depicts a more interesting scenario. If we add a projection texture to the Local Light, which allows us to control the shape of the light using the texture, like on the right side, the volumetric fog will also undergo corresponding changes.

为了让体积雾更加稳定，画面更加细腻，我们给体积雾添加了Temporal filter（ 时间滤波器），进行了多帧的混合。整体的GPU开销，也控制的不错，在PS4 Pro下面大致在1毫秒甚至更少。
To make the volumetric fog more stable and the visuals more refined, we added a Temporal filter, which involves blending multiple frames of the fog. We have managed to control the overall GPU cost quite well, with it being around 1 millisecond or even less on the PS4 Pro.

实现思路：
Here's an overview of the implementation approach:

#### About God Rays

（游戏中GodRay的表现）
(Performance of God Rays in the game)

对于方向光进行遮挡就可以产生God Ray的效果，我们的做法是有一个单独的pass来生成God Ray，然后是在1/2×1/2分辨率下面。
The God Ray effect is achieved by occluding directional light. Our approach involves a separate pass to generate the God Rays, which is done at a 1/2×1/2 resolution.

God Ray也是通过Ray marching的方式去生成的，我们会去采样shadow map，但是最多会采样5级的cascades。God Ray生成完之后，我们会提供美术一些可以调整的参数，然后将God Ray的结果叠加到体积雾上面去。
God Rays are also generated using ray marching, and we sample the shadow map, with a maximum of 5 cascade levels. After generating the God Rays, we provide artists with adjustable parameters, and then overlay the results onto the volumetric fog.

#### Example of Combining God Rays With Volumetric Fog

这个就是一个很好的技术和美术磨合的例子，体积雾直接生成的God Ray，在游戏里面实际效果其实不能够让美术满意，原因有两点：
This is a good example of the technical and artistic collaboration. The God Rays generated directly from the volumetric fog may not satisfy the artistic requirements for two reasons:

> ①分辨率不够，因为体积雾的分辨率是靠Voxel，而我们的Voxel是不会划分的特别精细；
> Insufficient resolution: The resolution of the volumetric fog relies on voxels, and our voxel division is not extremely fine.
>
> ②God Ray强度是完全依赖于体积雾的浓度，浓度一旦提高了，画面就会显得不通透，太脏。
> God Ray intensity depends solely on the density of the volumetric fog. If the density is increased, the scene becomes less transparent and appears too dirty.
>
>

所以我们是采用了单独的pass去生成God Ray，这样可以得到更锐利、更清晰的效果，美术调整也更灵活。
To address these issues, we use a separate pass to generate the God Rays. This allows us to achieve sharper and clearer results and provides more flexibility for artistic adjustments.

#### About the Imaged-Based Lighting System

（图中左边的是Reflection probe（反射探针），右边是Ambient probe）
(On the left side of the image is the Reflection Probe, and on the right side is the Ambient Probe.)

我们先看一下左边的Reflection probe，是用来给场景提供反射信息的。因为游戏的光影不断变化，我们是不能够简单地为反射探针烘焙一张环境贴图作为反射信息之用。所以对于每一个Reflection probe，我们是烘焙了一个mini GBuffer。
Let's first take a look at the Reflection Probe on the left, which is used to provide reflection information for the scene. Since the lighting and shadows in the game are constantly changing, we cannot simply bake an environment map as the reflection information for the Reflection Probe. Instead, for each Reflection Probe, we bake a mini GBuffer.

然后在运行的时候，我们会去更新场景的Reflection probe的cubemap。
During runtime, we update the Reflection Probe's cubemap with the scene's reflection information.

整个过程大概分为三步：
The entire process can be divided into three steps:

第一步是Relight。
The first step is to Relight.

第二步Convolve和Compress。
The second step is to Convolve and [the third step is to] Compress.

最后这个贴图需要再通过一个Compute Shader的做法，压缩成BC6H的格式，然后送到渲染管线里面去使用。
Finally, this texture needs to be compressed into the BC6H format using a Compute Shader, and then it is sent to the rendering pipeline for use.

下面是我们的Ambient probe也是实时生成的。
Below is our real-time generated Ambient probe.

这样，我们这个系统就基本完成了。但实际上还有很多可以改进的地方，比如说在Relight完成得到的环境贴图是漏光的，本来应该处于阴影当中的地面也会变得非常明亮。
In this way, our system is basically completed. However, there are still many areas that can be improved. For example, the environment texture obtained from Relight has light leakage, and surfaces that should be in shadow appear too bright.

我们的做法是，我们把24小时的shadow都烘焙下来，就是隔一段时间我们烘焙一下，把shadow转化成一个shadow SH保存起来。在运行的时候简单通过当前的时间对shadow SH进行插值，用来压暗Relight以后的结果。
Our approach is to bake the shadows for a full 24-hour cycle, periodically converting the shadows into a shadow SH representation. During runtime, we simply interpolate the shadow SH based on the current time to darken the results obtained from Relight.

这样得到的效果是出乎意料的好，而且我们需要保存的数据非常的少。
The resulting effect is surprisingly good, and we only need to store a small amount of data.

同样的方式，我们还可以把Local Light的信息也保存下来，作为Local Light的SH在Relight的时候也加上去，这样可以得到非常好、非常廉价的一个Local Light 反弹的效果。
Using the same approach, we can also save the information from Local Lights as Local Light SH, which can be added during Relight to achieve a very good and cost-effective Local Light bounce effect.

接下来我们要处理的是个室内室外光照环境不一致带来的问题。
Next, we need to address the issue of inconsistent lighting environments between indoors and outdoors.

我们是把Reflection probe分成室内、室外两种，然后美术通过摆放一个室内环境用的网格（interior mesh）来标记受室内光影响的像素。Ambient probe也会相应地为室内、室外生成不同的环境光。
We divide the Reflection probe into two types: indoor and outdoor. Artists place an interior mesh to mark the pixels affected by indoor lighting. The Ambient probe also generates different ambient lighting for indoor and outdoor areas.

看下对比，如果不区分的话，室内跟室外一样都会受天光的影响而变得很蓝，然后做了室内（interior）标记，室内的像素就能够正确地反应出室内的光照条件，会显得更黄一些。
Take a look at the comparison. Without differentiation, both indoor and outdoor areas would be affected by skylight and appear blue. However, with the interior marking, pixels inside the room correctly reflect the indoor lighting conditions and appear slightly more yellow.

下面这个就是室内环境用的网格（interior mesh）生成的Mask标记图，红色区域就是室内的区域，大家可以看一下对照关系。
The image below shows the Mask generated by the interior mesh for the indoor environment. The red area represents the interior region. Take a look at the corresponding relationship.

除了通过Reflection probe得到的反射，我们还有Screen space reflection来提供实时的反射信息。SSR在PS4 Pro上面的GPU开销大概是在1.5毫秒左右，我们对SSR也加了一个Temporal filter，通过当前帧的SSR信息和历史信息混合起来，来提高SSR计算结果的稳定性，让画面也更平滑一些。
In addition to the reflections obtained through the Reflection probe, we also have Screen Space Reflection (SSR) to provide real-time reflection information. The GPU cost of SSR on PS4 Pro is approximately 1.5 milliseconds. We have added a Temporal filter to SSR, blending the SSR information from the current frame with the historical information to improve the stability of SSR calculations and make the visuals smoother.

为了得到更多的反射信息，我们为SSR生成了Hi-Z的buffer，我们可以让每条射线通过Hi-Z最多能够跟踪的距离达到整个屏幕。
To obtain more reflection information, we generate a Hi-Z buffer for SSR. This allows each ray to trace up to the maximum distance covered by the Hi-Z buffer across the entire screen.

（SSR效果开关对比图，在比较光滑的地板上效果尤其明显。）
(Comparison of SSR effect toggle, with a particularly noticeable effect on smooth floors.)

#### About HDR Display

这里面的HDR Display包含了两个方面。
The HDR display in this context encompasses two aspects.

在这里我们称使用了ST2084和Rec.2020色彩空间的渲染管线为HDR管线，而使用的Rec.709色彩空间的非HDR管线，我们把他们叫做SDR管线。
Here, we refer to the rendering pipeline that uses the ST2084 and Rec.2020 color space as the HDR pipeline, while the pipeline that uses the Rec.709 color space as the non-HDR pipeline is referred to as the SDR pipeline.

下面我主要讲一下，为了让这个技术放到《原神》里面去，做了哪些调整。
Now, I will mainly discuss the adjustments made to incorporate this technology into *Genshin Impact*.

这张图是《原神》 的SDR和HDR的管线对比图，和SDR管线相比，HDR管线没有了tone mapping，color grading变成了HDR的color grading。而代替tone mapping的是RRT+ODT（reference rendering transform + output display transform）的组合，这就是很多人熟悉的ACES调色。
This image is a comparison between the SDR and HDR pipelines in *Genshin Impact*. In the HDR pipeline, tone mapping is eliminated, and color grading is replaced with HDR color grading. Instead of tone mapping, we use a combination of RRT+ODT (Reference Rendering Transform + Output Display Transform), which is a familiar approach known as ACES color grading.

另外，UI在HDR下面，也是单独画到一张RT的。然后再跟场景做一个合并，这是因为UI的亮度处理方式跟场景是不太一样的。
Additionally, in HDR mode, the UI is rendered separately on its own render target (RT) and then merged with the scene. This is because the brightness processing of the UI differs from that of the scene.

前面管线图的RRT+ODT部分是灰掉的，虽然这个是一个主流的调色方式，但是我们并没有采用它，因为和我们的游戏风格不太搭。
The RRT+ODT section of the pipeline diagram is grayed out because although it is a mainstream color grading approach, we did not adopt it as it did not align well with our game's art style.

为了保证在低亮度范围内的画面和SDR版本的游戏一致，我们将HDR的渲染画面和tone mapping处理之后的画面做了一个基于亮度的混合，然后在亮度不高的地方，就尽量保证了filmic tonemapping的关于toe部分的处理。
To ensure consistency between the HDR rendering and the SDR version of the game in low brightness ranges, we blend the HDR rendering with the tone-mapped image based on brightness. In areas with lower brightness, we aim to maintain the filmic tonemapping, particularly in the toe region.

但是这里面存在一个问题，就是1886的gamma是2.4，但是SDR的OETF是一个分段函数，大致上可以看作是gamma 2.2，这就出现PPT上写的问题。OETF处理完的颜色，经过EOTF，它得到的并不是它本身，就产生了一个误差。
However, there is a problem here. The gamma of the 1886 OETF is 2.4, but the SDR OETF is a segmented function that can be approximated as gamma 2.2. This leads to the issue mentioned in the presentation. After OETF processing, the color, when passed through the EOTF, does not yield the original color, resulting in an error.

在HDR下面，因为我们OETF和EOTF是被很好的定义了的情况，所以他们是互逆的，于是这个颜色在经过这两个处理之后能够得到原来的颜色，所以在HDR下面没有这么一个问题。
In HDR mode, since our OETF and EOTF are well-defined and inversely related, the color can be restored to its original value after passing through these two processes. Thus, this issue does not occur in HDR mode.

但是大家已经习惯了在HDR下面有这么一个误差的画面，所以为了模拟这个误差，我们是在HDR管线里面添加了OOTF，把差异给补上去了。
However, players have become accustomed to seeing this error in HDR mode, so to simulate this error, we added an OOTF in the HDR pipeline to compensate for the difference.

做完这些，是不是HDR就彻底没问题了呢？并不是。这还有一个大坑，叫做Hue Shift。
After implementing these adjustments, does HDR have no further issues? Not quite. There is another challenge known as Hue Shift.

举个例子，大家可以看一下游戏里面火焰制作的示意图，我们通过一个灰度图，就是下面像馒头一样的东西，加一个噪声贴图，然后进行扰动，得到了火焰扰动的纹理，然后我们用橘色去染色。
As an example, consider the schematic of flame creation in the game. We use a grayscale map, represented by the bun-like structure below, and apply a noise texture to create flame disturbance. Then, we color it orange.

最后我们把火焰的整体亮度往上提，在SDR下面因为有tone mapping的原因，是有一条曲线的，会让亮度增加逐渐变慢，于是橘色的R通道跟G通道的差异本来是很大，但是随着亮度的增加，tone mapping曲线介入了，R通道的增长就变慢，G通道就逐渐赶了上来，于是就产生了Hue Shift，画面渐渐开始发黄，于是就得到了后面看到的SDR下面火焰的效果。
Finally, we increase the overall brightness of the flames. In SDR mode, due to tone mapping, there is a curve that gradually slows down the increase in brightness. As a result, the difference between the orange channel (R) and the green channel (G) is initially significant. However, as the brightness increases, the tone mapping curve comes into play, causing the growth of the red channel to slow down, and the green channel catches up. This creates a Hue Shift, and the scene gradually becomes yellowish, resulting in the flame effect seen in SDR mode.

问题来了，因为在HDR下面，是没有tone mapping的，Hue Shift是不会发生的。
Now, the issue arises that in HDR mode, there is no tone mapping, and therefore, Hue Shift does not occur.

那怎么去修改呢？一种常用的方法，也是很多大作都用的方法，叫做黑体辐射。而《原神》使用的是另一种方法，在shader里面去模拟了Hue Shift，并且把模拟放到了color grading pass里面去，合并到Look-Up-Table的计算中。
How can we address this? One commonly used method, employed by many major titles, is known as blackbody radiation. However, in *Genshin Impact*, we use a different approach. We simulate the Hue Shift within the shader and incorporate this simulation into the color grading pass, merging it with the Look-Up Table (LUT) calculation.

这样的好处是，我们不仅仅是让火焰特效在HDR下面的效果跟SDR几乎一致，而且因为通过引入tone mapping的方式模拟了Hue Shift，所以我们前面提到的在非HDR的亮度部分的画面一致性的问题，也被顺手解决掉了，不需要跟tone mapping处理之后的画面做混合。
The advantage of this approach is that it not only allows the flame effects to appear almost identical in HDR and SDR modes, but also addresses the issue of visual consistency in the non-HDR brightness range. By simulating the Hue Shift through the introduction of tone mapping, we no longer need to blend the image with the tone-mapped version to maintain consistency.

而且这个操作，也是在生成Look-Up-Table的时候去做的，所以它的性能增加是可以忽略不计的。
Furthermore, this operation is performed during the generation of the Look-Up Table (LUT), so the performance impact is negligible.

### Personal Experience and Insights

《原神》的主机版开发是第一次尝试，时间、资源和人才都很缺乏。通过一年多的开发，我们积累了很多的经验，尤其是如何把写实的渲染技术跟风格化游戏结合的经验，非常的宝贵！
The development of *Genshin Impact* for consoles was a first-time attempt, and we faced challenges in terms of time, resources, and talent shortage. Over the course of more than a year of development, we accumulated valuable experience, especially in combining realistic rendering techniques with stylized gameplay.

随着新的主机平台的到来，我们又面临了一大波的技术升级。不过和在美国的时候相比，国内主机开发的从业人员太少了，希望大家能够多多交流，和我们探讨一下技术，交流一下开发经验。
With the arrival of new console platforms, we are now facing a wave of technological upgrades. However, compared to the situation in the United States, there is a shortage of professionals engaged in console development in China. We hope that everyone can engage in more communication, discuss technology, and exchange development experiences with us.

我这次的分享就结束了，感谢大家！
That concludes my presentation. Thank you, everyone!

## Q&A

Q：第一个是关于这些AO贴图是整合到一起进行计算的吗？
Are these AO maps calculated together?

A：我们有三种不同的AO技术，每一个AO技术都是独立去做计算的。计算完的结果，我们都是混合到同一个RT上面去，这个RT就是我刚刚提到的是一个半分辨率的RT。
We have three different AO techniques, and each technique is calculated independently. After the calculations are completed, we blend the results onto the same render target (RT), which is the half-resolution RT I mentioned earlier.

这些东西混合完之后，我们会统一地去做Bilateral的Blur加一个 Bilateral的Upsampling，最后把它们输出到一个全分辨率的贴图上面去。分别计算，统一整合，这可以算是一起计算吗？
Once these elements are blended, we apply bilateral blur and bilateral upsampling uniformly. Finally, we output them to a full-resolution texture. Although they are calculated separately and then integrated, can we consider this as calculating them together?

Q：Local Light是Unity还是自己开发的？
Is Local Light developed in Unity or your own engine?

A：这个是我们自己开发的。我们一开始是先做了一个tile based deferred lighting，然后发现因为我们游戏的光照布局方式，可能cluster deferred lighting 更适合我们的，于是我们做了这么一套东西。
It is developed by our own team. Initially, we implemented a tile-based deferred lighting system. However, based on the lighting layout in our game, we found that cluster deferred lighting was more suitable for our needs. Therefore, we developed this system accordingly.