PifuHD: 3D avatar from 1 image (well... almost)
Tags: 3d, ar, machinelearning, webdev
Published: 2020-07-07 08:08:00

Recently, [GDG DevParty Russia](https://gdg-devparty.ru/) was held at which, in addition to talk on [MagicString](https://youtu.be/UB2XK7Cf-Ms?list=PLGlZ_ld11os-nnB5CG_p6brIUWMGXU5Tr) library, there was a report by [Vasilika Klimova](https://twitter.com/Lik04ka) on the possibilities of WebGL in the browser “Let's dance?”. In it, she talked about how to create a 3D avatar, and then animate it directly in the browser. The problem is that the avatar is created using a huge device called *"Shapify Booth"*. Cool if you have access and some extra money for this kind of entertainment, but what if not? :)


Thanks to the achievements of Machine Learning, we have an alternative. Facebook Research published a work called [PifuHD](https://shunsukesaito.github.io/PIFuHD/). Using it, we can create a 3D avatar from single image 😱
To do this, find 1 full-body-length photograph of yourself.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/j5p7co213vdnh2gcvutj.jpg)

Next, open Google Colab ([via the link](https://colab.research.google.com/drive/11z58bl3meSzo6kFqkahMa35G5jmh2Wgt)) (better to do this in Google Chrome) and follow the instructions. If you get confused, watch the video.

{% youtube LWDGR5v3-3o %}

Literally in 5-10 minutes you will have your own 3D avatar from the picture.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0fzmjarbb0cb10d7jg2p.png)

But that is not all! Stitching 2 photos into one (front & back), thereby preparing the texture for our avatar

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/06awr1wi5l5mmonxwf9s.png)

and watching a couple of videos about an easy way to texture a 3D model in [Blender](https://www.blender.org)

{% youtube kERzhJKYvGo %}
{% youtube WFN8zn_7PZk %}

We can create a textured model.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/x23p79yffwb1rjkldb51.png)

Having exported all this in FBX format and creating an archive in which the fbx file and texture file will lie, we can import this archive into [Mixamo](https://www.mixamo.com)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xqt4ci2rbd7d7tnkgn3r.png)

In which we can choose the appropriate animation to poison it in [Adobe Aero](https://www.adobe.com/ru/products/aero.html) where we are already substituting our model in a suitable environment to get just such an animation 😅

{% youtube -9D0AQ0wTEs %}

Well, after a bit of training, I’ll finally deal with my inner self.

{% youtube 6B9r0nFIzps %}

Of course, the quality of the models leaves much to be desired and sometimes even an obvious picture gives a huge number of artifacts. PifuHD is especially bad with hands.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qzzi4ftiv3kxy2wzc7zl.png)

And texturing/animating will take you more than 10 minutes, especially for the first time (it took me 6 hours), but with a little training you can make a lot of cool animations and maybe even a simple game :)

*What a time to be alive! (c)*

Subscribe to my twitter https://twitter.com/obenjiro and Telegram chat TensorflowJS to always be up to date with the latest ML https://t.me/tensorflowjs_en