Facial recognition + good faces bot = best faces 🏆
Tags: rust, showdev, javascript, ai
Published: 2022-05-30 20:48:36

[@good_faces_bot](https://twitter.com/good_faces_bot) is probably the best Twitter account there is. It's pretty self-explanatory: just a lot of good faces posted regularly. I wanted to sift through the endless stream to find the best faces and this was the result: [bryce.io/best-faces](http://bryce.io/best-faces).

## Get the faces 🐦

This part doesn't need much explaining, everyone with basic coding experience knows how to and probably has fetched from Twitter's API. Borrowing from the timelines [quick-start guide](https://developer.twitter.com/en/docs/twitter-api/tweets/timelines/quick-start) I got an API token and downloaded all the tweets for the account. 

After that, it sorts them and filters for tweets above a certain threshold of likes (1000). Then the last thing to do is to download the image associated with each tweet with a slight staggering to avoid getting rate-limited and it's ready to be analyzed!

## Enter: [rustface](https://github.com/atomashpolskiy/rustface) 🔬

Put simply, this is a Rust-y wrapper around the [SeetaFace Engine](https://github.com/seetaface/SeetaFaceEngine), an excellent open-source tool for real-time facial recognition. It works surprisingly well given applications like this generally need massive amounts of processing power. Can read more about it [here](https://github.com/seetaface/SeetaFaceEngine/tree/master/FaceDetection).

The included example was very helpful; I modified it to loop over the list of images and made it so instead of drawing a rectangle over the face it outputs a cropped image of the face. With some minor error-handling it was making it through all of the images in no time! 

Unfortunately it only recognizes a face in about 1/3 of all the images currently (and some of the "faces" aren't actually faces, like the first one on the page 😂). With some tweaking this could be improved though.

## Putting it all together 🪢

Now that the Rust part is complete, all that's left is to display the faces somehow. I used Svelte to build a simple site since I like using it for small projects like this.

The trickiest part here was that although the Rust step generated a bunch of images, there isn't an easy way to connect them back to each tweet. I used `fs.exists` to loop over the output directory and generate a bunch of imports & exports. It's hacky but ended up working well! But beware 👻 you shouldn't be generating raw javascript like this unless there are no other options, here's a snippet:

```js
const hasOne = await fsExists(generatePath(tweet));

if (hasOne) {
    facePaths.push(generatePath(tweet));
}

facePaths.forEach(facePath => {
  output += `import ${faceFileName} from "../${facePath}";\n`;
});

output += `export const facesFor${tweet.id} = [${faceFileNames.join(',')}];\n`;

await fs.appendFile(outputFile, output);
```

Anyways once that was done I could loop over the list of tweets from the first step and display them in Svelte, referencing the `facesForX` import directly:

```js
{#each tweets as tweet}
    {#if faces[`facesFor${tweet.id}`].length > 0}
        {#each faces[`facesFor${tweet.id}`] as face, i}
            <Face face={face} />
        {/each}
    {/if}
{/each}
```

Note that it checks if `length > 0`: since it isn't able to detect a face in every image, unfortunately some get left behind. It's sad too since a lot of the really good faces are the stranger looking ones.

Then I added a simple hover state to show the original image that the face was detected in:

```js
<script>
    export let tweet;
    export let face;

    export let show = () => showOriginal = true;
    export let hide = () => showOriginal = false;
    
    let showOriginal = false;
</script>

<a href={tweet.tweet.text}>
    <img src={showOriginal ? tweet.media.url : face} on:mouseenter={show} on:mouseleave={hide} />
</a>
```

That's pulled directly from the Svelte component too! No wrappers or boilerplate, part of the reason why it's so satisfying to use for development.

---

Et voilà! The last kicker is that since all of these are just scripts, they can run sequentially in a GitHub Action! Although the facial recognition part takes a while (~20m) the simpler ones can fetch/generate the necessary data and run entirely in CI. There are some optimizations to be done here around uploading/downloading artifacts but in the meantime I'm happy it all works remotely.

Check out the code here:

{% github https://github.com/brycedorn/best-faces %}