Youxi Putao 2019

This article contains AI-generated translation.

 Author
 Published

米哈游，在业届素有“一家被游戏耽误的动画公司”之称。在Unite 2018大会中，米哈游技术总监贺甲就曾分享过《崩坏3》：在Unity中实现高品质的卡通渲染。
miHoYo, often referred to as "a game company distracted by animation," is known in the industry for their achievements in creating high-quality cartoon rendering using Unity. During the Unite 2018 conference, miHoYo's Technical Director, He Jia, shared insights into how they achieved high-quality cartoon rendering in *Honkai Impact 3rd*.

本文将由米哈游技术总监贺甲分享米哈游最新动画短片《八重樱 · 桃源恋歌》中的制作技术。
In this article, miHoYo's Technical Director, He Jia, will share the production techniques used in their latest animated short film, "Yae Sakura: Paradise Love Song."

## "Yae Sakura: Paradise Love Song"

米哈游使用Unity实时渲染的MMD次世代动画短片《八重樱 · 桃源恋歌》，今年2月在B站发布以来，获得了260多万的播放量。
miHoYo created the MMD (MikuMikuDance) next-gen animated short film "Yae Sakura: Paradise Love Song" using Unity's real-time rendering. Since its release on Bilibili in February, the film has garnered over 2.6 million views.

《八重樱 · 桃源恋歌》这部短片，对于玩家来说绝对是一场美的享受，而对于从事Unity开发者来看，这部短片中的模型角色，动画渲染的技术绝对顶尖。
"Yae Sakura: Paradise Love Song" is undoubtedly a visual feast for viewers and showcases cutting-edge modeling, character animation, and rendering techniques for Unity developers.

友情提示：观看及下载高清晰版本请访问B站，搜索“av42250009“或“八重樱”。
Friendly reminder: For the high-definition version, please visit Bilibili and search for "av42250009" or "Yae Sakura."

## Interview Guest

贺甲拥有中国科学技术大学硕士学位，曾担任英伟达半导体科技的研发工程师，现任米哈游技术总监和美术指导，并专注PBR、NPR卡通渲染、过程动画及可交互物理等实时渲染领域。他致力于使用Unity做出高品质的卡通CG渲染。
He Jia holds a master's degree from the University of Science and Technology of China and previously served as a research and development engineer at NVIDIA Semiconductor Technology. Currently, he is the Technical Director and Art Director at miHoYo, focusing on real-time rendering areas such as PBR (Physically-Based Rendering), NPR (Non-Photorealistic Rendering), procedural animation, and interactive physics. He is dedicated to achieving high-quality cartoon CG rendering using Unity.

## Technical Interview

**米哈游创作《八重樱 · 桃源恋歌》短片的初衷和背景是什么？
What was the original intention and background behind miHoYo's creation of the short film "Yae Sakura: Paradise Love Song"?**

《八重樱 · 桃源恋歌》短片参与人员共20人，其中包含程序8人，其余为美术，但实际项目仅6名美术全职参与，其余人员则为兼职。2018年6月开始制作，2019年1月完成，历时半年。
The short film "Yae Sakura: Paradise Love Song" involved a total of 20 participants, including 8 programmers and the rest being artists. However, only 6 artists were full-time participants, while the others were part-time contributors. The production started in June 2018 and was completed in January 2019, taking six months in total.

我们主要是希望通过结合最新的技术来探索新的渲染风格和表现的可能性，以及积累Unity引擎实时渲染制作CG动画方面的经验。
The main goal was to explore new rendering styles and possibilities for expression by combining the latest technologies, as well as to accumulate experience in creating CG animations using the Unity engine's real-time rendering capabilities.

大家对于《八重樱 · 桃源恋歌》的认可，同时也验证了米哈游对于技术研发和美术结合的把控能力。
The recognition received for "Yae Sakura: Paradise Love Song" also validated miHoYo's ability to combine technological development with artistic expertise.

**为什么选择Unity进行实时渲染创作这部短片？
Why did you choose Unity for real-time rendering in this short film?**

我们对Unity整套渲染流程做了全面的定制化处理：渲染管线、材质系统、特效表现及整套后处理流程围绕最终表现完全重写，这些高度定制化的内容在Unity上是相对易于实现的。
We fully customized the entire rendering process within Unity, including the rendering pipeline, material system, special effects, and post-processing. These highly customized elements were relatively easy to implement within Unity.

使用Unity 2018，我们基于Unity默认的HDRP高清晰模版进行定制化开发，并制作了专门的Shader。
Using Unity 2018, we based our development on Unity's default HDRP (High Definition Render Pipeline) template and created dedicated shaders.

使用实时渲染引制作动画短片无论从渲染开销还是效果调整迭代成本是极低的。渲染效率自然不必多说，从迭代效率的角度而言，我们可以直观的以所见即所得的形式去调整镜头，灯光，材质等效果。
Using real-time rendering for creating animated short films was cost-effective in terms of rendering overhead and effect adjustments. Rendering efficiency goes without saying, and from an iteration perspective, we could intuitively adjust camera angles, lighting, material effects, and more.

我们使用了Unity Asset Store资源商店的插件AVPro Capture，只需2个小时就可以输出一个4K 60fps 5分钟规格的视频进行最终效果迭代，这相对于传统流程来说都是不太可能做到的。
We utilized the Unity Asset Store plugin "AVPro Capture," which allowed us to output a 4K 60fps 5-minute video for final effect iterations in just 2 hours. This would have been nearly impossible with traditional workflows.

我们的制作团队也是一个比较年轻的团队，正是因为上述优势使得这部短片的制作成为可能。
Our production team is relatively young, and it's because of these advantages that making this short film became possible.

**Unity中HDRP高清晰渲染管线在这部短片中具体体现在哪些方面？
How does the HDRP in Unity specifically manifest in this short film?**

在《八重樱 · 桃源恋歌》之前，我们曾使用Unity的默认渲染管线渲染过另外二部短片《琪亚娜·极乐净土》和《芽衣·染上你的颜色》。
Before "Yae Sakura: Paradise Love Song," we used Unity's default render pipeline to render two other short films: "Kiana: Blissful Land" and "Mei: Dye Your Color."

在开发过程中，我们遇到了很多在默认管线下开发的局限性。这些局限性主要是由于默认渲染管线的高度封装，使得我们无法自由地控制许多效果的渲染。
During the development process, we encountered many limitations when using the default pipeline. These limitations were mainly due to the high level of encapsulation in the default render pipeline, which prevented us from having full control over the rendering of many effects.

首先，遇到的问题是无法精确控制光源。
Firstly, we faced issues with precise light source control.

为了实现高质量的角色阴影，我们需要为角色指定某个光源，并在渲染时判断当前光源是否为角色主光源，是则使用高质量阴影贴图，否则使用内置的默认阴影贴图。
To achieve high-quality character shadows, we needed to specify a particular light source for the characters and determine during rendering whether the current light source was the main light source for the characters. If it was, we used high-quality shadow maps; otherwise, we used the built-in default shadow maps.

Unity的默认管线无法让我们方便地遍历整个光源列表，并且无法在Shader中得知光源与场景中实际光源的对应关系，这使得我们无法在多光源下灵活地兼顾角色自阴影和默认阴影。
The default Unity pipeline did not allow us to easily iterate through the entire list of light sources and obtain the correspondence between the light sources and the actual light sources in the scene within the Shader. This made it difficult to flexibly balance character self-shadows and default shadows under multiple light sources.

在新的HDRP高清晰渲染管线中，所有的光源列表可以在一个Pass中遍历完成，同时可以依靠光源ID判定是否为角色主光源，从根本上解决了我们的问题。
In the new HDRP, all the light sources can be iterated in a single pass, and we can rely on the light source ID to determine whether it is the main light source for the characters, fundamentally solving our problem.

其次，在Unity的默认管线下，我们无法灵活控制很多渲染。
Secondly, under Unity's default pipeline, we couldn't flexibly control many renderings.

例如：Camera depth texture、Camera depth normal texture、Camera motion vector texture等，这些Screen space textures通常是由Unity使用Replacement shader来渲染的，对用户来说几乎是完全黑盒的。
For example: Camera depth texture, Camera depth normal texture, Camera motion vector texture, and so on. These screen space textures are usually rendered by Unity using a Replacement shader, which is essentially a black box for users.

为了实现风格化的卡通渲染，我们需要高度定制化这些纹理的渲染。
To achieve stylized cartoon rendering, we needed highly customized rendering of these textures.

角色渲染需要使用额外的Pass来实现描边效果，而Replacement shader无法支持多ass渲染，这使得很多屏幕特效在描边处会有问题，例如DOF、TAA、SSAO等。
Character rendering requires the use of additional passes to achieve the outline effect, but the Replacement shader does not support multi-pass rendering. This can cause issues with many screen effects at the outlines, such as Depth of Field (DOF), Temporal Anti-Aliasing (TAA), Screen Space Ambient Occlusion (SSAO), and others.

尤其是，描边区域对TAA的质量有很大影响，为了为描边区域渲染正确的Motion vector，我们需要拿到前一帧Skinned mesh的normal、tangent等数据。如果使用了ABC格式的网格或者对偶四元数，我们甚至需要更多的数据。
Especially for TAA, the quality is greatly affected by the outline areas. To render the correct Motion Vector for the outline region, we need access to the previous frame's Skinned Mesh data, including normals and tangents. If using ABC format meshes or dual quaternions, even more data is required.

我们无法在Unity的默认管线下方便地拿到这些数据。但是在HDRP高清晰渲染管线下，我们可以通过自定义的Lightmode和Shader来完全控制这些纹理的渲染，极大地方便了我们的开发。
We cannot easily access this data under Unity's default pipeline. However, in the HDRP, we can have full control over the rendering of these textures through custom Lightmodes and Shaders, which greatly facilitates our development.

此外，HDRP高清晰渲染管线让我们可以在场景中使用大量实时光源，Hybrid Deferred & Forward rendering大大增加了渲染的灵活性。得益于Tile & cluster lighting，即便是需要使用forward rendering的物体，也可以高效地使用大量实时光进行渲染。
Furthermore, the HDRP allows us to use a large number of real-time light sources in the scene. The Hybrid Deferred & Forward rendering greatly increases rendering flexibility. Thanks to Tile & Cluster Lighting, even objects that require forward rendering can efficiently use a large number of real-time lights for rendering.

SSR和SSAO等一些屏幕后处理效果，在HDRP高清晰渲染管线下可以同时正确地应用于Deferred和Forward rendering上。而在Unity的默认渲染管线中，这些效果对Forward rendering都很不友好。
Some screen post-processing effects like Screen Space Reflections (SSR) and SSAO can be correctly applied to both Deferred and Forward rendering in the HDRP. In Unity's default render pipeline, these effects are not very friendly to Forward rendering.

灵活的渲染管线可以让我们高度定制许多风格化效果，例如风格化的Lighting、雾效、Decal system等。
A flexible rendering pipeline enables us to highly customize many stylized effects, such as stylized lighting, fog effects, and Decal system.

总体而言，原先Unity默认的渲染管线由于高度封装性给渲染开发造成了很多障碍，使得很多效果的高效实现变得束手束脚，甚至无法实现。
Overall, the highly encapsulated nature of Unity's default render pipeline posed many obstacles to rendering development, making it difficult or even impossible to efficiently achieve many effects.

新的HDRP高清晰渲染管线使用了更加先进和通用的渲染架构，给了开发人员极大的自由度和开放性，这对于实现高品质的游戏或影视渲染都是至关重要的。
The new HDRP uses a more advanced and versatile rendering architecture, providing developers with great freedom and openness, which is crucial for achieving high-quality game or cinematic rendering.

尽管目前HDRP仍有很多底层渲染是黑盒的，很多渲染API也仍未开放，功能也尚未完全稳定，但我们相信对于有能力的开发团队来说，HDRP高清晰渲染管线绝对是未来Unity下高品质渲染的最佳选择。
Although there are still many aspects of HDRP that are black-boxed and many rendering APIs remain closed, and the functionality is not fully stable yet, we believe that for capable development teams, the HDRP is undoubtedly the best choice for high-quality rendering in Unity's future.

**这么漂亮的布料解算是怎么实现的？？
How was such beautiful cloth simulation achieved??**

布料部分我们使用Qcloth做解算，通过Alembic格式导入Unity。
For the cloth simulation, we used Qcloth and imported it into Unity in Alembic format.

Abc动画文件的大小和模型面数有很大关系，我们把布料主体控制在2W面左右，基本可以保留褶皱模拟所需要的网格精度。要保证表面的平滑则需后续在渲染的时候做曲面细分，这样4分钟30fps的动画数据大概只有1G大小，是一个相对适中的大小尺寸。
The size of the Abc animation file is closely related to the number of model faces. We kept the cloth body at around 20,000 faces, which retained the mesh precision needed for the wrinkle simulation. To ensure surface smoothness, we performed surface subdivision during rendering. This way, the 4-minute, 30fps animation data was approximately 1GB in size, which is a relatively moderate size.

导入Unity引擎后，我们使用自定义实现的基于GPU的Catmull-Clark Subdivsion以渲染出绝对平滑的布料表面，整套流程从表现方面非常令人满意。
After importing into the Unity engine, we utilized a custom GPU-based Catmull-Clark Subdivision implementation to render an absolutely smooth cloth surface. The entire process was very satisfying in terms of visual representation.

由于是离线烘培的数据，对于效果迭代会有一定的数据交换成本。为了进一步提升迭代效率，我们还初步完成了Position Based的GPU Cloth模拟。
Since it was offline-baked data, there was some data exchange cost during effect iteration. To further improve iteration efficiency, we also implemented a preliminary GPU Cloth simulation based on Position-Based methods.

在模拟4万个粒子的情况下已经可以实时跑60fps且表现出自然的褶皱。这将极大减少效果迭代所需要的时间，基于GPU实时模拟的方案将在之后的制作流程中开始应用。
With the simulation of 40,000 particles, we achieved real-time performance at 60fps while naturally displaying wrinkles. This will significantly reduce the time required for effect iteration, and the GPU-based real-time simulation approach will be applied in the subsequent production process.

**能否介绍一下头发的实现细节？
Could you introduce some details about the implementation of the hair?**

基础的Diffuse shading使用了Multi-layer toon ramp实现丰富的阴影及色彩的渐变和细节。高光计算则使用了各向异性材质，并使用两层高低频高光叠加来获得更丰富的层次和变化。
The basic Diffuse shading uses a Multi-layer toon ramp to achieve rich shading, color gradients, and details. Anisotropic materials are used for specular calculations, and two layers of high and low-frequency highlights are combined to achieve richer levels and variations.

为了体现更为风格化的高光形状，我们还使用参数化曲线结合Jitter noise来定义头发高光的形状，这些参数都可以实时调整，方便最终调出满意的效果。
To achieve a more stylized highlight shape, we also use parameterized curves combined with Jitter noise to define the shape of the hair highlights. These parameters can be adjusted in real-time, making it convenient to achieve the desired effect.

卡通造型的头发如果要实时计算光照对模型制作方面的要求也比较高，要按照严格的硬表面来制作要求才能保证高光的顺滑。
For cartoon-style hair, real-time lighting calculations also require strict hard surface modeling to ensure smooth highlights.

**眼睛看上去那么有神是如何实现的？使用了特殊的Shader吗？
How was the vivid look of the eyes achieved? Was a special shader used?**

眼睛是卡通渲染核心的表现点，我们一方面希望眼睛可以从不同角度体现质感，另一方面又要保证卡通插画的感觉，因此专门为眼睛定制了特殊的Shader，主要模拟了眼球折射和光束焦散效果，再配合手绘的纹理材质，最终达到了期望的效果。
The eyes are a crucial aspect of cartoon rendering. On one hand, we wanted the eyes to convey texture from different angles, and on the other hand, we needed to maintain the feel of a cartoon illustration. Therefore, we specifically customized a special shader for the eyes, mainly simulating eye refraction and beam dispersion effects. Combined with hand-painted texture materials, we achieved the desired effect.

**如何看待实时渲染技术对于未来动画影视行业的意义？
How do you view the significance of real-time rendering technology for the future of the animation and film industry?**

毫无疑问实时渲染技术对于未来影视动画是一个趋势，已经逐步开始有厂商使用实时引擎去做动画。对于整体行业制作方面转型以及大规模使用还有很长的路要走，但是它的优势已经逐步开始显露。
Undoubtedly, real-time rendering technology is a trend for the future of animation and film. Some studios have already started using real-time engines for animation. While there is still a long way to go for overall industry adoption and large-scale use, its advantages are already becoming evident.

一方面可以从渲染效率和迭代成本上极大加速现有制作流程，另一方面也使得小型团队甚至个人做出高品质的影视及动画作品成为可能。
On one hand, it greatly accelerates existing production processes in terms of rendering efficiency and iteration costs. On the other hand, it makes it possible for small teams or even individuals to create high-quality film and animation works.

新兴技术的出现，例如：基于机器学习的Procedural animation，GPU实时Raytracing等也都将加速这一进程。实时渲染技术对动画影视行业的影响将经历一个从量变到质变的过程。
The emergence of new technologies, such as Procedural Animation based on machine learning and GPU real-time ray tracing, will also expedite this process. The impact of real-time rendering technology on the animation and film industry will undergo a transformation from incremental to substantial.

**米哈游还会有计划基于Unity创作实时渲染的动画项目吗？
Does miHoYo have plans to create real-time rendering animation projects based on Unity?**

我们还在进一步完善提高整套流程以及渲染品质来推动引擎在动画制作方面的应用，敬请期待后续作品。
We are still further refining and improving the entire production process and rendering quality to promote the engine's application in animation production. Please look forward to our future works.