Artificial Intelligence Institute 2019

This article contains AI-generated translation.

 Author

上午去听了那个"人工智能遇上逆熵科技"的讲座，就给大家稍微总结一下内容吧，或许能看出一些米哈游未来的计划
I went to the lecture on "Artificial Intelligence Meets Anti-Entropy Technology" in the morning. Let me give you a brief summary of the content. Perhaps we can discern some of miHoYo's future plans.

> 人工智能前沿应用系列讲座
> Lecture Series on Frontiers of Artificial Intelligence
>
> 人工智能“遇见”逆熵科技
> Artificial Intelligence "Meets" Anti-Entropy Technology
>
> 时间：10月29日(周二) 10:00-12:00
> Date: October 29th (Tuesday) 10:00-12:00
>
> 地点：闵行交大东下院303
> Location: Room 303, East Lower Building, Minhang Campus, Shanghai Jiao Tong University
>
> 主持人：杨小康教授，人工智能研究院常务副院长
> Host: Professor Yang Xiaokang, Executive Vice Dean of the Institute of Artificial Intelligence
>
> 刘伟，米哈游创始人，现任公司总裁，毕业于上海交通大学，曾获“上海市优秀中国特色社会主义事业建设者”、“上海市青年五四奖章”等荣誉称号。
> Liu Wei, Founder of miHoYo and current President of the company. He graduated from Shanghai Jiao Tong University and has received honors such as "Outstanding Builder of Socialism with Chinese Characteristics in Shanghai" and "Shanghai Youth May Fourth Medal."
>
> 近年来人工智能的发展极大的丰富了人们解决问题的手段。米哈游逆熵科技研究院致力于运用机器智能与数据驱动，推动游戏制作、运营相关的图形图像音频处理、动画编辑、数据分析等技术的进步。
> In recent years, the development of artificial intelligence has greatly enriched the means for problem-solving. miHoYo's Anti-Entropy Research Institute is dedicated to using machine intelligence and data-driven approaches to advance technologies in graphics, image and audio processing, animation editing, data analysis, and related areas for game production and operations.
>
> 主讲嘉宾：刘伟
> Guest speaker: Liu Wei
>
> 主办方：上海交通大学人工智能研究院
> Organizer: Artificial Intelligence Institute, Shanghai Jiao Tong University
>
>

> 人工智能前沿应用论坛
> Forum on Cutting-Edge Applications of Artificial Intelligence
>
> 人工智能“遇见”逆熵科技
> Artificial Intelligence "Meets" Anti-Entropy Technology
>
> 刘伟
> Liu Wei
>
> 上海米哈游网络科技股份有限公司
> Shanghai miHoYo Network Technology Co., Ltd.
>
> 2019年10月29日
> October 29, 2019
>
>

首先是大伟哥被聘为我校创业导师（哪个学院我忘了，不过估计是个虚职，类似于荣誉称号吧）
Firstly, Brother Wei was appointed as an entrepreneurship mentor at our school (I forgot which college, but it's probably an honorary position, similar to a title of honor).

讲座讲了许多专业内容，没有接触过人工智能的同学可能不太能听懂，我就简单的给大家概括一下mihoyo目前的研究方向吧
The lecture covered many technical topics, which might be challenging for students unfamiliar with artificial intelligence. Let me briefly summarize miHoYo's current research directions for everyone.

之所以叫逆熵科技，是因为mihoyo创立了一个逆熵研究院，大概也可以叫做米哈游人工智能研究院吧
The reason it's called "Anti-Entropy Technology" is that miHoYo established an Anti-Entropy Research Institute, which could also be referred to as miHoYo Artificial Intelligence Research Institute.

和我之前想的差不多，mhy现在正在进行的研究的目标是为了提高工业化生产中的生产效率，解放生产力。主要做了以下几个方面的研究：
It's similar to what I thought before. miHoYo's current research aims to improve production efficiency and unleash productivity in industrialized manufacturing. They have primarily conducted research in the following areas:

>
> 1. 场景建模 "Scene Modeling"
> 2. 纹理迁移 "Texture Transfer"
> 3. 布料建模 "Fabric Modeling"
> 4. 布料动画 "Fabric Animation"
> 5. 人脸捕捉 "Facial Capture"
> 6. 声音合成 "Sound Synthesis"
> 7. 动作风格迁移 "Motion Style Transfer"
>
>
>

1、场景建模
在建造大型场景，如草原、森林、大型建筑物等时，每一个模型都由建模师手动建造是不现实的。以树木为例，传统的方法是由建模师建造一些树木模型，在此基础上对这些模型的一些特征（如：树杈的位置、数量等）进行随机处理（借用高斯函数...）从而获得大量不重复的模型。然而这样的方法得到的模型在大规模应用时还是用规律的、重复的。
1. Scene Modeling:
When constructing large-scale scenes such as grasslands, forests, or massive buildings, it's impractical for every model to be manually created by modelers. For instance, with trees, the traditional approach involves modelers creating several tree models and then applying random processes (using Gaussian functions, for example) to modify some features of these models (e.g., the position and quantity of branches) to obtain a variety of non-repetitive models. However, such methods still result in patterns and repetitions when extensively applied.

逆熵研究院现在在做的是借助深度学习的方法从积累的模型数据集的基础上学习各种模型的有用特征，从而自动生成各种风格的大规模场景建筑模型。
The Anti-Entropy Research Institute is currently utilizing deep learning techniques to learn useful features of various models from an accumulated dataset of model data, enabling the automatic generation of large-scale scene and architectural models in different styles.

说一句，我所说的都是我听了之后总结的，不保证准确，毕竟我只是一个本科生，很多地方我也不一定懂，有说错的地方概不负责
Just a note, everything I mentioned is based on my understanding from what I heard, and I cannot guarantee complete accuracy. After all, I am just an undergraduate student, and there might be areas where I lack understanding. I take no responsibility for any inaccuracies.

2、纹理迁移
与渲染相关的最重要的部分就是材质和光照，逆熵研究院在做的是：将给定图片上的纹理迁移到3D建模的表面，其好处在于图片作为数据集，数据量很大（图片好找/画）。同时纹理也不再局限于表面，当物体破损后，新露出的表面也拥有相似的纹理
2. Texture Transfer:
One of the most critical aspects related to rendering is materials and lighting. The Entropy Reversal Research Institute is working on transferring textures from given images to the surfaces of 3D models. The advantage of this approach is that images can be used as a dataset, providing a large amount of data (images are easy to find or create). Additionally, textures are not limited to the surface; when an object is damaged, the newly exposed surface also possesses similar textures.

3、布料建模
这部分走神了，没仔细听，不过大概和场景建模相似。
3. Fabric Modeling:
I zoned out during this part and didn't listen carefully, but it's probably similar to scene modeling.

4、布料动画
由动补或k帧得到的人物运动信息是没有穿上布料的，要在后期为人物加上衣服布料，并为其配上与人物运动匹配的动画（运动）。
Fabric Animation:
The motion information obtained from motion capture or keyframes for characters does not include clothing or fabric. In the post-production phase, clothing and fabric need to be added to the characters, and animations that match the character's movements need to be created.

传统方法是由动画师密集的k，每几帧手动调整一次，中间帧自动平滑过渡，这样可以得到任意我们想要的结果，但工作量巨大。
The traditional approach involves intensive keyframing, where animators adjust the animation manually every few frames, with automatic smoothing in between, to achieve the desired results. However, this method requires a huge amount of work.

另外的方法就是由物理引擎根据物理规则自动计算布料的运动，但这样的方法生成的动画是严格符合物理规则的，很多时候并不是我们想要的结果，且需要调整许多参数。
Another method involves using a physics engine to automatically calculate the fabric's movement based on physical rules. However, animations generated using this method strictly adhere to the physical rules, which may not always produce the desired results and often require adjusting many parameters.

逆熵研究院做的是通过增强学习的方法训练模型，从而让动画师根据喜好，调整个别关键帧（很少），通过模型（model）自动解读动画师的想法，生成完整的动画。对生成的结果不满意的地方也可以随时改动，其他部分也会随调整自动调整。
The Anti-Entropy Research Institute is using reinforcement learning techniques to train a model. This allows animators to adjust specific keyframes (very few) based on their preferences. The model then automatically interprets the animator's intentions and generates the complete animation. If there are any areas of dissatisfaction with the generated results, they can be easily modified, and other parts of the animation will automatically adjust accordingly.

5、有标记/无标记的人脸捕捉
标记就是在脸部标记关键点，根据关键点的运动复原捕捉结果，无标记就是不标记，达到相同的目的。这部分大家应该挺熟悉的，2D/3D动捕通常都会涉及面部捕捉。不过逆熵做的要更加精细，要捕捉到细微的表情之类的。vtuber/乔碧萝专用（误
5. Marker-based/Markerless Facial Capture:

Marker-based facial capture involves marking key points on the face and then reconstructing the capture results based on the movement of these key points. On the other hand, markerless facial capture achieves the same goal without using markers. This part should be quite familiar to everyone, as 2D/3D motion capture typically involves facial capture. However, what Anti-Entropy is doing is more precise, capturing subtle facial expressions and the like. VTuber/Qiao Biluo exclusive.

6、声音合成
通过深度学习，学习声优的声音（包括音色、情感等），做到输入文字自动得到该声优的配音结果，以避免档期不足等情况，同时还能将声优的声音塑造成IP，永久保留。甚至可以出售声音版权（邪恶的商业味道）。这部分和我一年多以前的想法几乎一样，可惜没有动手去实现（实际是太难了）
6. Voice Synthesis:
Through deep learning, the system learns the voice of a voice actor, including their tone, emotions, and more. This allows for automatic voice acting results when inputting text, which helps to avoid scheduling issues and also preserves the voice actor's voice as an intellectual property (IP) permanently. It even enables the possibility of selling voice copyrights (with a somewhat questionable commercial flavor). This part is almost identical to an idea I had over a year ago, but unfortunately, I didn't pursue it (probably because it's quite challenging).

（动补+人脸捕捉+声音合成×虚拟歌姬，米哈游虚拟偶像成了）
(Facial capture + Motion capture + Voice synthesis = Virtual Idol; miHoYo's virtual idols have become a reality.)

7、动作风格迁移
没仔细听，大概就是人物在不同状态下的动作风格不同（如饥饿、寒冷、正常状态），如何自动的将这些风格迁移到他们的动作上面（行走，跑步等）。（图二我自己找到，大概能帮助理解） "7. Motion Style Transfer:
I didn't listen carefully, but it's probably about how a character's motion style differs in different states (e.g., hunger, cold, normal state) and how to automatically transfer these styles to their actions like walking or running. (The two images I found might help with understanding.)"

大致就这么多，剩下的就是大伟哥讲了讲创业经历，逆熵研究院的招聘/合作需求之类的。
（图: 大伟哥当着校领导面放极乐净土）
That's roughly it. The rest of the lecture was Brother Wei talking about his entrepreneurial experience and the recruitment/collaboration requirements of the Anti-Entropy Research Institute. (Image: Brother Wei playing "Blissful Land" in front of school leaders.)

最后讲一讲我的想法，我认为米哈游走的道路是正确的道路。大概是理工科背景的原因，我始终坚信"科学技术是第一生产力"这句话的正确性。关心工业和科技的同学应该或多或少从各处听到过第四次工业革命，我也觉得我们已经在第四次工业革命的前夜了，人工智能将会成为这场革命的主角。每一次工业革命都会带来生产力的飞跃，这样的革命通常也会伴随着新旧势力的交替，霸权的易主。mihoyo能顺应时代的趋势，主动拥抱技术，这是他们的优势。相比体量太小，没有能力参与科研的小公司和体量太大患得患失的巨头，mihoyo或许真的未来可期。但不管怎么说，能坚持高投入的研发，这条道路是绝对正确的。只要mihoyo坚持投入研发，我就会一直支持他的
Finally, let me share my thoughts. I believe miHoYo is on the right path, perhaps due to my background in science and engineering. I've always firmly believed in the correctness of the phrase "science and technology are the primary productive forces." Students who are concerned about industry and technology may have heard about the Fourth Industrial Revolution from various sources. I also believe that we are on the eve of the Fourth Industrial Revolution, where artificial intelligence will play a central role. Each industrial revolution brings a leap in productivity and often accompanies the shift of power between old and new forces, the change of hegemony. miHoYo's ability to adapt to the trends of the times and proactively embrace technology is their advantage. Compared to smaller companies with limited research capabilities and giants that are hesitant due to their massive size, miHoYo's future seems promising. But regardless, investing heavily in research and development is undoubtedly the right path. As long as miHoYo keeps investing in research and development, I will continue to support them.

## Notes

1. ↑ The video in the image is 《八重樱 · 桃源恋歌》"Yae Sakura: Paradise Love Song."