Machine learning in Scratch?? 🐱💡
Tags: machinelearning, scratch, computerscience, algorithms
Published: 2021-04-28 17:28:20

> Wait, really?

Yes, absolutely! Scratch has all the necessary tools for a working linear regression with gradient descent. With a few nifty tricks, we can even visualize how the algorithm works!

# What is Scratch, though?

Scratch is a project of the Scratch Foundation, in collaboration with the Lifelong Kindergarten Group at the MIT Media Lab. It is available for free at [https://scratch.mit.edu](https://scratch.mit.edu). Scratch is a tool to teach programming by offering a GUI where instructions are combined via drag&drop. Sprites and backgrounds allow for a user interface with animations and user input. One can create all kinds of programs, like simple interactive movies, entire games, or, well, machine learning algorithms.

![Some example code in Scratch](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63xnofrcrbimy1emworp.png)

## Machine learning in Scratch? Why would you even do this?

Because it's possible. I wanted to explore the boundaries of Scratch and figure out new ways to solve certain problems. It made me understand the algorithm better and think outside the box. Besides: I've got a little history in implementing those things in languages or tools you wouldn't expect:

{% post https://dev.to/thormeier/algorithm-explained-linear-regression-using-gradient-descent-with-php-1ic0 %}

**This post also explains how linear regression works and how to implement it.**

# Ok, let's implement linear regression in Scratch then!

Awesome! First, I need some data. For linear regression, that's a bunch of XY-coordinates. Since Scratch doesn't have nested lists, I keep them in separate lists. A third list keeps track of the error values, so we can inspect or display it later.

![The lists in Scratch's side bar](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l3wygl2o8gv864hku4cn.png)

I'm generating the data now, because why not. I could simply generate random data, but some kind of correlation would be nice. [UVM has a good formula for that](https://www.uvm.edu/~statdhtx/StatPages/More_Stuff/Gener_Correl_Numbers.html). First, {% katex inline %}x{% endkatex %} and {% katex inline %}y{% endkatex %} are set randomly. Then, a new {% katex inline %}y{% endkatex %} is calculated with this formula, with {% katex inline %}r{% endkatex %} as the "rate of correlation":

{% katex %}
y = x*r+y*\sqrt{\smash[b]{1-r^2}}
{% endkatex %}

I create a new sprite with a single red dot in the center and tell it to calculate it's own coordinates as soon as it's created. I can then clone this point sprite to create new as many data points as I want:

![The code for the data point sprite](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sbmad9trkatospruvbqo.png)

The nice thing about this: The data is already visualized. This comes more or less out of the box.

I now introduce the variable `m` and set it to 100, to know how many data points I need. Next, I create a "main" sprite that's empty, basically the main entry point of my program. Think of it like Java's `public static void main(String[] args)`. There I reset all the variables, set the parameters I need for the linear regression/gradient descent and clone the data point `m` times:

![The main code to kick off the algorithm](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wo83zxzbokoeouzxx016.png)

And that's the result so far:

![Data points, neatly distributed with some correlation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fq7xxemqmpno9ey3s0i9.png)

Next, I create a sprite called "error point" to plot the error rate. The "error point" is similar to the data point sprite. Also, since the "errors" list is already public, I introduce two more variables: `maxIter` for the total number of iterations of gradient descent and `currentIter` to know which iteration I'm currently at. For each iteration, I create a clone of the "error point" and tell it to adjust itself, to get a nice plot of the error rate:

![Error point code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zxzlgsukl3tirrzrq59b.png)

Next, I introduce a sprite called "line". This is where the main logic will happen. The line can basically adjust itself, rotate and move, whatever its internals tell it to do.

Linear regression basically tells me the coefficients {%katex inline%}c_0{%endkatex%} and {%katex inline%}c_1{%endkatex%} of a linear function. I can use those to calculate a single point (i.e. {%katex inline%}(x, c_0 + c_1 * x){%endkatex%}). To calculate the angle {%katex inline%}\alpha{%endkatex%} of the hypotenuse, I can use this formula:

{%katex%}
  \tan \alpha = \frac{side}{adjacent side}
{%endkatex%}

Therefore:

{%katex%}
  \alpha = \arctan \frac{side}{adjacent side} 
{%endkatex%}

When I assume {%katex inline%}x{%endkatex%} as 1 and neglect {%katex inline%}c_0{%endkatex%} (this is only moving the line, not rotating it), I can calculate the angle of the line as this:

{%katex%}
  \alpha = \arctan c_0
{%endkatex%}

This is then part of the iteration of gradient descent in the "line" sprite:

![Line sprite, basic reset code, and the loop for gradient descent](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ujhwdxu4mreimtog4nh4.png)

So far so good. I've got data points, I've got an error plot, I've got a line that I can move around. Now for the fun part: The linear regression with gradient descent itself.

I've worked with two lists for x's and y's since Scratch doesn't allow for nested lists. Minor inconvenience, but nothing that'll stop me.

So, first, I'll give the variables used in the loop some default values:

![Default values in gradient descent loop](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/411bwl3ah5jjz6jxx53n.png)

Next up: Looping through the data points to calculate the different between predicted Y and actual Y:

![Looping through all data points](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j60xb7xt89rdcmip2h43.png)

Next, I use the sums `descentSumC0` and `descentSumC1` to calculate new versions of `c0` and `c1` and set them simultaneously:

![Setting of C0 and C1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s73rnktl6jm37ab29t8k.png)

After this block ran through, the line is adjusted and a new calculation is done. Until the maximum number of iterations is reached.

# And this works?

Yup:

![Animation of Scratch executing linear regression with gradient descent](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0ugav8uk848b15kr7ynv.gif)

For the full code and to try it yourself, here's the Scratch project: [scratch.mit.edu/projects/520553339](https://scratch.mit.edu/projects/520553339)

---

*I hope you enjoyed reading this article as much as I enjoyed writing it! If so, leave a* ❤️ *or a* 🦄*! I write tech articles in my free time and like to drink coffee every once in a while.*

*If you want to support my efforts,* [*buy me a coffee* ☕](http://buymeacoffee.com/pthormeier) *or* [*follow me on Twitter* 🐦](https://twitter.com/pthormeier)*!* *You can also support me directly via [Paypal](https://www.paypal.me/pthormeier)!*

[![Buy me a coffee button](https://dev-to-uploads.s3.amazonaws.com/i/x55hp6jopwyy161d8e2u.png)](http://buymeacoffee.com/pthormeier)