This is how my AWS Deep Racer model finished best in 11 seconds
Tags: deepracer, aws, machinelearning, ai
Published: 2022-07-09 10:47:17

My new company flew me to Sydney 🛩️ to attend a 2-day team building event which included half of the day putting our **AWS Deep Racer** model into action by competing in an actual physical race.

Despite the workload, I spent time outside working hours to develop our team's model. Since I am a very competitive person (yes, I trash talk my kids when I win a Mario Kart game 🏎️ over them - teaches them resilience 👀), the motivation to win was multi-fold: bragging rights, winning cool prizes (we each got Echo dot and AWS notebooks!) and most importantly, my inclination to learn "real" machine learning (I used to be part of a team "selling" AI/ML but it's all just big data 🤷).

The awesome AWS team had a couple of workshops with everyone on how to prepare your models (creating accounts, training, simulation, racing with the company-specific community race). Non-technical or newbies don't need to be scared as the AWS console for Deep Racer is quite a user-friendly UI. We only had one track to train on which is the **_re:Invent 2018_** so it was easier to tweak your models.

I trained multiple models (and iterations) after deep-diving into _github_ repositories and browsing through multiple online articles
![AWS Deep Race models](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mpjb02q8qotajkvqfyft.png)
Each models were trained between an hour and three hours based on different reward functions. One of the things I learned is that longer training would not mean a better time (as it might pick up some bad habits too as you can see in the graph below where it shows a bit of downward trend at the end)
![Training logs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ovwseoyqgrfajwthgry8.png)
We were allowed to submit any models we have (as many times as we want) and enter into the simulation virtual race. Overall, my entry came 3rd with a time of **34.724** seconds having no lap going **"off-track"**
![Race results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fg8xwenrqzv24hze8yu5.png)
You can see above that I tried to iterate my **superpumped** model (sorry, I just watched the Amazon Prime series so that's the first thing that came to my mind, although I'm not a fan of Uber's toxic culture) by tweaking the values in the hyper parameters and reward functions but the **version 2** did the best time (not bad considering there were 20 entries). Each team then had to download the _physical car model_ then submit to the AWS team the day before the actual physical race.

During "race" day 🏁 (thanks AWS team for preparing the track and all the set-ups), we had to drive our models on the track one team at a time for 3 minutes each (so the car can go as many laps as possible and the best lap will be recorded). We were the last team to drive the vehicle (a.k.a. RC model with a camera) and as they say, _"saved the best for last"_. Our car clocked **11.04** seconds, while the runner-up was 8 seconds slower (yep, big gap which validated the hard work!). AWS team also said the fastest ever was 7 seconds so our result was not bad for a first timer 😎. The other thing to note is that there is a **_"speed function"_** introduced while driving the car. The AWS team initially set it to **41%** and you would need to adjust it accordingly while the car is moving (e.g. turn up the speed when it's a straight path while try to  decrease it when it's curving). There were a couple of times where the car went off-track so I realised, I had to slow it down.
![Actual race results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u4ybzqx4xyvt5xu79v89.png)

---

I know you might have skipped the babble above so for those who are just here to check how I did it, click the repository link below containing my reward function:
[my awesome aws deeprace model](https://github.com/poponuts/aws-deepracer-model)  ⬅️ ⬅️ ⬅️
... with training configuration below:
![Training config](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gpclydybwfb5ft1qxzkt.png)
I ran it for 100 minutes (1 hour and 20 minutes) because when I iterated and changed the duration to 3 hours, it just went ballistic and did no good.
 
The model focused on accuracy instead of speed where it tries to stay at the centre as much as possible. It rewards the model the closer it is while it penalises if it veers away.

---

By no means I am taking full credit for this as I checked out a number of articles and tried out a dozen of models (hence, used up all my training credits). I used the _**Example 4**_ from the reference below with only changing the Loss type from _Mean Square_ to _Huber_ (by accident... but it worked out well in the end):
https://www.linkedin.com/pulse/samples-reward-functions-aws-deepracer-bahman-javadi?trk=public_profile_article_view