🤷‍♂️ How I trained a neural network to turn you into an Archer cartoon  
Tags: javascript, python, showdev, machinelearning
Published: 2020-05-14 14:23:41

---
title: 🤷‍♂️ How I trained a neural network to turn you into an Archer cartoon  
published: true
description: A quick outline of how I trained a Generative Adversarial Network to generate archer style cartoons from regular portrait photographs.
tags: javascript, python, showdev, machinelearning
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/t2gqkwzq3dfaqibd0qhr.gif
---

A few weeks back I decided I wanted a cartoon version of my profile picture. 

Sadly, as I'm not much of an artist, drawing it myself was out of the question. So I set about on what seemed the only other logical course of action...

I trained a neural network to do it for me! 

---

{% twitter 1260864003149979648 %}

---

Let's get this out of the way - if you'd like a cartoon version of *your* profile picture, here's how:

1. **Tweet at me ([@harrison_g_reid](https://twitter.com/harrison_g_reid)) and include "cartoonify me" somewhere in the text of the tweet.**

1. **Within a few minutes, my (hopefully?) dependable twitter bot will reply with a cartoon-ified version of your profile picture.**

*NOTE: Since this was posted, the twitter bot has been disabled because 💸, but I've put it online so you can try it out [here](https://keen-yonath-ed0e96.netlify.app/cartoonify/)*

I should warn you, the results are...mixed. But it's funnier when it does a terrible job anyway, so 🤷‍♂️. Here's a GIF demo:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/6d4kso9w07dwxprjqwa8.gif)

Read on if you want to learn how I built it...

---

## Finding a Model (CartoonGAN):

The first thing I did after deciding that this would be a fun project was google around to see if there were any existing libraries I could use. 

As usual, the open source world did not disappoint! I soon came across a tensorflow implementation of the [CartoonGAN](http://openaccess.thecvf.com/content_cvpr_2018/papers/Chen_CartoonGAN_Generative_Adversarial_CVPR_2018_paper.pdf) generative adversarial neural network.


{% github mnicnc404/CartoonGan-tensorflow no-readme %}


The Github repo has some really cool examples of converting images and gifs into anime style cartoons - I recommend checking it out. 

But this wasn't quite the style I was after. I wanted something a little more comic-book style - heavy black lines & flat colors. I wanted it to look like Archer!


![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/79r5n6ltx6e6rm5m6r0v.jpg)


Fortunately, the repo contains some pretty detailed instructions on how to train the network on your own training data. 

So, I set about gathering **a lot** of images.

---

## Gathering Data:

To train CartoonGAN, I would need two sets of images:

1. A large set of *real life* images of human faces.

2. An equally large set of cartoon faces (from Archer)

It was **relatively** easy to find a good dataset of human faces. I found the [VGGFace2](http://www.robots.ox.ac.uk/~vgg/data/vgg_face2/) face dataset, which is an **enormous** dataset, and far exceeded my needs.

Of course, there's no dataset of faces from Archer available, so I'd need to create my own. 

Since I was aiming for a dataset of about 3500 images, there was no way I could realistically do this manually.

It took a little creativity, but I managed to *mostly* automate this. It basically ended up as a four stage process.

1. Using [ffmpeg](https://ffmpeg.org/), extract a frame for every 4 seconds of video, for every episode of the first season of Archer. (If you're interested, the ffmpeg command to do this for a single video is: `ffmpeg -i video.mov -r 0.25 video-images/%04d.png`.)

2. Detect the location of all the faces in every frame using [facedetect](https://www.thregr.org/~wavexx/software/facedetect/#download). Yes, this works surprisingly well on cartoon faces!

3. Crop images for each located face using [Jimp](https://github.com/oliver-moran/jimp).

4. Manually check the extracted images, and remove all weird things that had incorrectly been identified as faces.

The end result was a set of ~3700 images. Just about every face from the first season of archer:

![A series of cartoon faces extracted from the show Archer](https://dev-to-uploads.s3.amazonaws.com/i/aldh62mpszx017dlemnk.jpeg)

Now we're talking.

## Training the Model:

This was the easy part - it basically involved cloning the the CartoonGAN repo mentioned above, copying the images to the correct directory and running the python script as per the instructions in the repo. 

It was a real workout for my computer though - it took *several days* running the training in the background to make it through 30 epochs of training.

Here's a gif of the training progress over the first few epochs.

![A gif with 8 faces showing progress in training of the neural network](https://dev-to-uploads.s3.amazonaws.com/i/frr4prcckmmc7i7dxdfs.gif)

---

## Running it on Node:

If you're a JavaScript developer and you haven't tried [TensorFlow.js](https://www.tensorflow.org/js) yet, get amongst it. You don't really need to know all that much about Machine Learning to make use of existing models, and build some cool stuff.

In any case, the Node API's for TensorFlow.js let you directly load the format of model output by the CartoonGAN training process (SavedModel format).

Viola! A cartoon generating neural network running on Node.

---

***If you're interested in how I've deployed the model as a twitter bot, stay tuned! I'll provide a walkthrough in a future post.***

*Note: The code for this isn't yet available on my Github, but will be made available soon.*