How to generate unique artworks for less than 1$ ?
Tags: machinelearning, aws, api, programming
Published: 2022-12-08 18:58:46

## Some context

It all started as a side project. I wanted to create my own version of a NST algorithm. NST stands for Neural Style Transfer. It is a kind of Deep Learning algorithm that will apply the style of one image to another one.

Simply stated, it will take 2 inputs (a content image and a style image). And it will output a new image with the content of the content image and the style of the style image. A new artwork is born! You can find some examples on my Instagram account => https://www.instagram.com/atomicwombat_/

New York
![New York view](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tv009cd7626vxydijiwj.jpg)

Paris
![Paris view](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/weuge3swfi1gl6by95em.jpg)

How is that even possible? Well, Machine Learning made huge progress in the recent years. You can re-use what computer vision algorithms learned and apply it to a new field. 🤯

If you want to create your own NST, simply google it. There are a lot of free resources out there that will show you how to do it. You do not even need some Machine Learning knowledge, simple programming knowledge is enough.

Of course, if you want to tweak it, customize it, improve its speed, etc., you will need some ML knowledge. But for a first version, uncle Google will provide you with everything.

Before :
![Burj Khalifa before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vhnpxx0dseloa835twtt.jpg)

After :
![Burj Khalifa after](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h5qsdiwqt4fxamo2j4o2.jpg)

Before :
![Hong-Kong before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5v8bcmpjyn70uc1ix5sa.jpg)

After :
![Hong-Kong after](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h2jyq8vk9pkgsajteni8.jpg)

But trust me, the tricky part is deploying it. When I was done with my algorithm, I wondered : "Why not deploy that as an API? It would be super cool!". And this is where troubles started. 😅

I wanted to deploy a heavy NST algorithm in a secure and cost-effective way. I wanted it to be fast, reliable, scalable and easy to maintain. There is no easy guide or tutorial for this.

I made a lot of mistakes, but I learned a lot as well. And that is what I am going to share with you here. Ready? Let's go! 🥳

## The journey 

### The beginning 🛣

So, you have a nice NST algorithm. What is next? First, as any ML algorithm, you need to save it. I used tensorflow, but any ML framework offers you the possibility to save an algorithm. It will create a folder and store everything there (the model, the weights, etc.).

I have some AWS experience, so I quickly thought about AWS Lambda. It is super cheap and scalable. But there is a problem: the deployment package size. 😨

When you deploy a lambda, you must package your code in a zip archive. That archive must not exceed 250 MB unzipped. When you know that Tensorflow weighs more than 300 MB (depending on your OS), you know that it is not going to work.

If you are familiar with AWS Lambda, you may know about Lambda Layers. It is a way to divide your deployment package between your code and its dependencies. Well tried, but the 250 MB limit includes the whole lambda (code+dependencies).

So, I was back to square one. But there is another way to deploy on Lambda. You can use Docker containers. You build your Docker Image, you push it to AWS ECR, and then you can use it to create a Lambda function. 💡

2 traps here (I fell into those traps 🤦‍♂️). First, the AWS Lambda API is expected to interact in a certain way with your image. So, in your Dockerfile, you must use a AWS base image. It is free and you can find some examples in their documentation. Without it - it is called the RIC for Runtime Interface Client -, AWS will never spin your lambda.

Second trap, you must carrefully choose your architecture. AWS offers you two options : X86 and ARM64. I was working on a MAC M1, so the image I built was created for ARM64 and I did not notice the default value being X86 when deploying my Lambda.

To fix this, you can add a flag to your docker build command to build your image for the desired architecture. But you cannot change the lambda architecture. You must create a new Lambda function.

When you have done all this, you finally have a running Lambda function, congrats! You can simply add API Gateway as a trigger. And you can now make HTTP requests to your API! 🤩

And you will realize that it is not working. 😅 Well, technically, it is working but not like expected. The first time you call your Lambda, it will "coldstart" (prepare the server basically). It will take a few seconds (especially as Tensorflow is heavy). But API Gateway has a timeout of 30 seconds. No matter what, you cannot change that timeout.

So, after your first call, you will simply receive a timeout error. All those efforts to get that error... If you are like any regular dev (including me), you will start feeling the impostor syndrom kicking in. 

Once the lambda is "warm" (the server is ready), you will get a response. But if you try to call it again, you will get a 500 internal server error. What happened? Well, Tensorflow is greedy and it has not enough memory. So, you might continue to use your API with super small images (250px) or you can try to find a better solution... 

### The real solution 👨‍💻

After a few weeks on the problem, I was really frustrated that my API was not working as expected. So I deep-dived in AWS to find a solution. That is why AWS is so powerful. It does not just give you powerful tools, it lets you combine them.

You need to create an EFS resource, which is a file system. Then, you can create a EC2 instance and connect it to your EFS. At that moment, you just have a linux machine connected to a file system, no rocket science. 🚀

Now, you can install all the dependencies you need, including tensorflow. Just create a dedicated directory and install them there. You can also create a dedicated directory for your NST model.

Once you have everything installed on the file system, do not forget to terminate your EC2 instance. You will not need it anymore, and AWS may charge your for it (depending on whether you are elligible to the free-tier or not). 💰

Now, all you have to do is to create an Access Point in your EFS, link that to your Lambda function and overwrite the PYTHONPATH env variable (to point to the correct directory).

The deployment package of your lambda only requires your code now. It is a super small zip archive. All the dependencies are stored in a dedicated file system (in EFS).

But we still have the API Gateway timeout error, right? One workaround is to use Lambda Function urls. It does not offer some of the API Gateway advantages (api keys management, usage plans, etc.). But Lambda function urls provide you with a clear url to trigger your lambda. And the timeout is the one defined on the lambda level (up to 15 minutes).

Now, you have a secure, cost-effective, fast, reliable, scalable and easy-to-maintain API that can generate new artworks for you! 😎


## Conclusion 🙏

It has been quite a journey. It took me a few months from the original idea to the deployed API. If you are interested in this API, but you do not want to build it yourself, feel free to send me an email at alexandrebardiaux0@gmail.com. I can give you a free access to the demo version. 🙂

Of course, there are a lot of details that I did not mention in this article (VPC configurations, Health check endpoint, AWS helpers, etc.). This article is already long enough. But if you are interested into that subject, send me a message. I will be more than happy to discuss it with you (and it may motivate me to write a part II to this article).