I sold my startup because of bugs: I wish I had this serverless repository!
Tags: webdev, javascript, programming, opensource
Published: 2024-04-04 11:00:03

In 2021, I built a startup called Linvo, a LinkedIn automation tool. It generated good money but also had a high burn rate.

I can tell you that I lost most of my money as a young entrepreneur (entirely bootstrapped) 😆

The problem with my system is that it kept on failing; I am not the best programmer in the world, but let me elaborate.

![Revenue](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4l95m758p7m9fg4914dc.png)

---

## Puppeteer sucks your memory

Linvo went into a few iterations during its lifetime. I used the tech that I know: React and NestJS.

Basically, any automated cloud scraping goes into making tasks and then solving them.

It looks something like this:

![Workers and scraping](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9n4d78oe0cz6vxl1y34n.png)

- Cron runs every 10 minutes, looks for new stuff to do, and sends it to the queue.
- Workers take tasks from the queue and process them.
- Save everything to the DB.

This is a very naive solution that was running on multiple EC2 (AWS) and Redis.

In theory, it should work - Yes, you would need to scale the workers horizontally, but that was not my problem.

I started to experience something else - **memory leaks.**

As you probably know, [Puppeteer](https://pptr.dev/) - a chromium automated browser, is heavy (same as Chrome on your computer), and I was running lots of them - they sucked both my memory - causing the servers to stop working and took the internet bandwidth.

![Sucks](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5fsl47a1yzumo3qexa8h.gif)

---

## Needed a solution

Refactoring sounds like - yeah, let's do that, but my Intercom support was filled with support tickets (I already had many users), and I knew I had to create some crazy refactoring.

I had an idea for a solution - let's use AWS lambda.

- Lambda start
- Do the task
- Die (most of the time)

It doesn't suck your internet.
It will die and not cause a memory leak.

It's a bit more expensive because Lambdas were not meant to run indefinitely—that would remove the whole concept, but I was desperate. (Well, the good thing is that you don't need to think about scale.)

Here is the new infrastructure:

![New Infra](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sr304vf0z81f0bqfxnup.png)

- Event bridge cron running every 1 minute (that's the minimum)
- Lambda that creates six events for the queue (making it like the cron runs every 10 seconds)
- SQS queue (bye bye, Redis)
- Lambdas shooting out with auto-scaling (yay?)

---

## Vendor Locking

As you probably understand from the context - now I am all in with AWS, but I didn't mind - only how I would test it on my computer.

Back in the day, I used [Sam](https://github.com/aws/aws-sam-cli). It's almost the same as [Serverless](https://www.serverless.com/) to create the entire environment quickly.

And yes, they do bring some local testing. They can take your lambdas and turn them into endpoints! Yay!

But I was using Event Bridge, SQS, and Lambdas. That was just not possible.

I have encountered [localstack](https://github.com/localstack/localstack). It was supposed to help me build an entire AWS stack on my computer, but many things were not supported back then, such as Lambda layers. And I was overwhelmed in using it.

So I started to create my "own" infrastructure where I deployed one "test" SQS on AWS and ran local code with it (a function that calls a function), and it was - bad.

I did not work well in production; I felt like the word was squeezing up on me back in the day.

I just wanted to get rid of this startup; I even offered it to people for free, taking the customers and rid myself of that problem.

Long story short, I managed to sell the system for a low price, but lately, I discovered something interesting.

---

## Winglang - Typescript like dialect for the cloud

I guess my pain was real, and even though I would be happier to find this back in the day - I am glad it's alive today.

[Winglang](https://github.com/winglang/wing) helps you to build the same cloud infrastructure as I used, but the nice thing is:

- It doesn't require you to use any configuration files. You basically define the entire infrastructure with code (you can check their entire preflight and inflight documentation)
- You can run it locally - thank you, god, 🙏🏻🙏🏻🙏🏻🙏🏻 with all the infrastructure included.
- You can deploy it to any cloud provider, and you don't have to use AWS. They can literally generate any Terraform or cloud formation for easy deployment.

I talked to the founder of Winglang, Elad (builder of the famous CDK), and he said something that made me twitch.

> You can literally work on your projects when you are on an airplane and write tests for it 🤯

I can't give them enough stars myself - but do me a favor and star their repository and of course, use it for your next project.

https://github.com/winglang/wing

![Thank you](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/giofk3zupgxjgnc57zw5.gif)

---

## Final words

I am angry at Winglang. I wish they had done it sooner. Now they show up. When it's all about AI?

I started a new YouTube channel mostly about Tech, open-source, and marketing :)

If that's something that interests you, feel free to subscribe to it here:
https://www.youtube.com/@nevo-david?sub_confirmation=1