Machine Learning in Deno: Single Layer Perceptrons
Tags: machinelearning, typescript, deno, javascript
Published: 2024-02-25 12:03:29

Ever wished you could solve your regression and classification problems in JavaScript/TypeScript without having to rely on Neural Network libraries like Tensorflow.js?

Presenting [La Classy](https://github.com/retraigo/la-classy), a single-layer perceptron library for Deno.

Classy provides a bunch of solvers (gradient descent, OLS, etc.), loss functions, activation functions, learning rate schedulers, and optimizers for gradient descent such as Adam and RMSProp. You can try various combinations of these depending on the problem.

Here's a quick example of how to classify the popular Iris dataset using Classy in Deno:

## Import required libraries
First, we import the following libraries. You'll see use for them in later sections:

### deno.land
```ts
import { parse } from "https://deno.land/std@0.217.0/csv/mod.ts";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "https://deno.land/x/vectorizer@v0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "https://deno.land/x/classylala@v1.1.0/mod.ts";
```
You can also load them from JSR.

### JSR
```ts
import { parse } from "jsr:@std/csv";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "jsr:@lala/appraisal@0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "jsr:@lala/classy@1.1.0";
```

## Load the Dataset
First we load our dataset. You can download the `iris.csv` file from [here](https://github.com/retraigo/deno-ml/blob/main/datasets/iris.csv).

```ts
const data = parse(Deno.readTextFileSync("iris.csv"));
```

We then skip the first row of our dataset (the header).

```ts
data.shift();
```

## Prepare input variables

Now that we have our data ready, we can separate the predictor and target variables from our dataset.

The first four columns of our dataset are the predictor variables and the fifth column indicates the iris species, our target variable.
```ts
const x = data.map((fl, i) => fl.slice(0, 4).map(Number));
```

Classy requires inputs to follow the following type:

```ts
type MaybeMatrix = {
    data: Float64Array,
    shape: [number, number]
}
```

However, our data is currently in the form of a 2D array. We can use the `Matrix` class from `@lala/appraisal` since it satisfies the type.

```ts
const X = new Matrix(x, {shape: [data.length], dType: "f64"})
```

In the meantime, let's also prepare the classes. We take just the fourth column from our dataset.

```ts
const y_pre = data.map((fl) => fl[4]);
```

Our `y_pre` will now be a 1D array with strings that denote the classes of each sample. We need to perform one-hot encoding on this data to use it for machine learning.

For one-hot encoding, we can use `CategoricalEncoder` from `@lala/appraisal`. 

```ts
const encoder = new CategoricalEncoder()
encoder.fit(y_pre)

const y = encoder.transform(y_pre, "f64")
```

Now that we have both the predictors and the targets, let's take a small portion of the dataset out for testing accuracy.

```ts
const [[x_train, y_train], [x_test, y_test]] = useSplit(
  { ratio: [7, 3], shuffle: true },
  X,
  y
);
x_train.slice(0, 10)
```


We can now begin training our model.

## Initialize the Model

In this example, we are using Stochastic Average Gradients with RMSProp as our optimizer.

```ts
const solver = new SagSolver({
  loss: crossEntropy(),
  activation: softmaxActivation(),
  optimizer: rmsPropOptimizer(4, 3),
});
```

- `crossEntropy` is the standard loss function for multiclass problems.

- `softmax` is an activation function used for multiclass problems.

> _Try using `hinge` as the loss function with `linearActivation` later._

With our solver ready, we can begin training the model.

## Train the model

We can train our model using the `train()` method.

```ts
solver.train(x_train, y_train, {
  learning_rate: 0.01,
  epochs: 300,
  n_batches: 20,
  patience: 10
});
```

Let's break down each argument:

- The first two arguments are the training input and output variables.

- The `learning_rate` hyperparameter determines how large each step should be when updating parameters during training. If you are not using `adamOptimizer` or `rmsPropOptimizer`, set this to a very small value.

- The `epochs` determine how many iterations the model should train for. This is the maximum number of iterations even if it doesn't converge.

- `n_batches` is the number of minibatches. A large number will result in longer epochs.

- `patience` determines how many epochs the model should continue learning if the loss starts increasing. If there is no improvement in loss within `10` epochs, the model will roll back `10` epochs and stop training.

## Testing Metrics

Now that our model is trained, we can test its accuracy on the data we kept aside for testing.

```ts
const res = solver.predict(x_test)
```

We now have a `Matrix` with joint probabilities of classes for each sample. We can use the `CategoricalEncoder` again to convert them into one-hot representations.

```ts
// this method mutates the variable
CategoricalEncoder.fromSoftmax(res)
```

We can now convert the categorical variables into class labels.

```ts
// Predicted Values
const y_pred = encoder.untransform(res)
// Actual Values
const y_act = encoder.untransform(y_test)
```

The `ClassificationReport` class accepts predicted and actual values as parameters to create a classification report and provide metrics.

```ts
const report = new ClassificationReport(y_act, y_pred)

console.log(report)
```

At the time of writing this article, I gave the code a run and this was my classification report.

```
Class	Precision	F1Score	Recall	Support
Iris-setosa	1	1	1	15
Iris-versicolor	1	1	1	13
Iris-virginica	0.85	0.92	1	17
Accuracy			0.98	45
```

You can try different combinations of loss functions and activation functions, try different hyperparameters, and try to solve different supervised learning problems using `Classy`.

Check out more examples at [deno-ml](https://github.com/retraigo/deno-ml)!

I will make another article with a regression problem later. Have a great day and good luck on making your machine learn!