Demystifying machine learning via Bluetooth with Arduino
Tags: arduino, javascript, bluetooth, tfjs
Published: 2022-10-28 15:42:47

---
title: Demystifying machine learning via Bluetooth with Arduino
published: true
description: Learn how a machine learning model can be transferred via bluetooth from the browser to an Arduino
tags: arduino, javascript, bluetooth, tfjs
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ux1476t7yf94a777jht8.png
canonical_url: https://charliegerard.dev/blog/demystifying-machine-learning-bluetooth-arduino
---

The first time I experimented with [TensorFlow.js](https://www.tensorflow.org/js) for micro-controllers, I got really excited about the fact that a machine learning model was transferred via bluetooth to my [Arduino](https://www.arduino.cc/). In just a few seconds gesture control was enabled on a website! My excitement quickly turned into curiosity; how does it actually work? 

To understand it better, I spent time diving into the open-source script [tf4micro-motion-kit.js](https://github.com/googlecreativelab/tf4micro-motion-kit/blob/main/web/dist/tf4micro-motion-kit.js) that is used as part of the project [Tiny Motion Trainer](https://experiments.withgoogle.com/tiny-motion-trainer/view/). In this post, I am going to explain how a machine learning model can be transferred via bluetooth from the browser to an Arduino. 

If you’re interested in how this technology can be used to create gesture-controlled web applications, check out the [previous blog post](https://dev.to/stripe/gesture-based-payments-2ipd) I wrote about this.

## Bluetooth and GATT

A core concept to understand for the rest of this blog post is that Bluetooth Low Energy (BLE) devices transfer data back and forth using what is called **services** and **characteristics**, which are concepts that are part of GATT (Generic ATTribute Profile). GATT relies on the ATT (Attribute Protocol) data protocol, which uses 16-bit or 128-bit UUIDs (Universally Unique Identifiers) to store services and characteristics.

You can think of services and characteristics as variables that will hold data about the functionalities of your device, that can be read or written to from another application or device. 

For example, a service you can find on many BLE devices is called the Battery Service. It contains multiple characteristics, for example the battery level and the battery power state characteristic. When you buy an off-the-shelf device, these services and characteristics are already set for you and you might be able to do some “read” or “write” requests to them.

On the Arduino however, you can create your own services and characteristics. When it comes to being able to upload a machine learning model to the board, you might want a generic service with characteristics such as the transfer state, the total length of the file, the settings with which your model was trained, and more.

What do these services and characteristics look like? As mentioned above, they are either 16-bit or 128-bit UUIDs so a service ID could be: 

```
"81c30e5c-0000-4f7d-a886-de3e90749161"
```

In this example, you have 32 hexadecimal values. A hexadecimal value is represented as 2^4 so 4 bits that can hold a value of either 0 or 1. As a result, 32 x 4 = 128 bits.

When these services and characteristics are created in the program running on the Arduino, they can then be read from the browser using these UUIDs.

Have a look at [the Arduino upload program](https://github.com/googlecreativelab/tf4micro-motion-kit/blob/main/arduino/tf4micro-motion-kit/tf4micro-motion-kit.ino) if you want to see how all the services and characteristics are created.

Now, let’s move on to how this works in practice.

## Connecting to the board

First of all, the board has to be somehow connected to the browser to be able to send or receive data. In this project, this connection is established via bluetooth. Using the [Web Bluetooth API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Bluetooth_API), you need to start by using the `requestDevice` method on the `navigator.bluetooth` object. You can pass in some filters if you’d like so the browser will show you only the device(s) you’re interested in when initiating the connection.

For example, in the `tf4micro-motion-kit.js script`, the main service UUID that is created in the program running on the Arduino is used as a filter to identify the device.

```js
const SERVICE_UUID = "81c30e5c-0000-4f7d-a886-de3e90749161";
const device = await navigator.bluetooth.requestDevice({
     filters: [{ services: [SERVICE_UUID] }],
 });
```

The code above returns a promise and the next step is to connect to the device:

```js
const server = await device.gatt.connect();
```

If this step is successful, you can start to store the reference of these services and characteristics in variables so you can use them later on. For example, storing a reference to the main service is done with the following line:

```js
const service = await server.getPrimaryService(SERVICE_UUID);
```

And you can define different characteristics with the code below, for example, the file length characteristic using the same ID as the one defined in the [program uploaded to the Arduino](https://github.com/googlecreativelab/tf4micro-motion-kit/blob/main/arduino/tf4micro-motion-kit/ble_file_transfer.cpp#L43):

```js
const FILE_LENGTH_UUID = "bf88b656-3001-4a61-86e0-769c741026c0";
const fileLengthCharacteristic = await service.getCharacteristic(FILE_LENGTH_UUID);
```

The characteristic above is used to refer to the file size of the machine learning model in bytes so, when it is transferred to the Arduino, you can check when the entire file has been transferred.

After the `connect()` method has been called and you store all the references to the services and characteristics you’re interested in, the board is connected to the browser and you can move on to the next step.

## Loading the machine learning model

The model is stored in your application folder as a `.tflite` file. First, it needs to be loaded using either the `fetch` API or an XMLHTTPRequest. Second, the response needs to be served as an array buffer.

```js
const loadFile = (modelUrl) => {
 return new Promise((resolve, reject) => {
   const oReq = new XMLHttpRequest();
   oReq.open("GET", modelUrl, true);
   oReq.responseType = "arraybuffer";
 
   oReq.onload = function () {
     const arrayBuffer = oReq.response;
     if (arrayBuffer) {
       resolve(arrayBuffer);
     } else {
       reject(new Error("Failed fetching arrayBuffer"));
     }
   };
 
   oReq.onerror = reject;
   oReq.send(null);
 });
};
```

Once the file is loaded, the next step prepares the file to be transferred.

## Splitting the machine learning model

The [Arduino Nano 33 BLE Sense](https://store-usa.arduino.cc/products/arduino-nano-33-ble-sense) is designed around a [Nordic Semiconductor](https://www.nordicsemi.com/) chip which has a Maximum Transmission Unit (MTU) of 23 bytes, which represents the largest packet size that can be sent at a time. According to [this resource](https://infocenter.nordicsemi.com/index.jsp?topic=%2Fsds_s140%2FSDS%2Fs1xx%2Fble_data_throughput%2Fble_data_throughput.html), the maximum data throughput for this size is 128 kbps so you need to split the machine learning model into packets of this size to be able to transfer it over to the Arduino.

You also need to make sure that the total size of the file isn’t too large for the microcontroller to handle.

To do this, you start by reading the characteristic that holds the maximum file size value from the Arduino.

```js
let maximumLengthValue = await fileMaximumLengthCharacteristic.readValue();
let maximumLengthArray = new Uint32Array(maximumLengthValue.buffer);
let maximumLength = maximumLengthArray[0];
// The fileBuffer variable is the model buffer you loaded in the code sample above.
 if (fileBuffer.byteLength > maximumLength) { 
   console.log(
     `File length is too long: ${fileContents.byteLength} bytes but maximum is ${maximumLength}.`
   );
   return;
 }
```

If the file isn’t too long, you can start splitting it and transferring it. The code below details the different steps.

```js
// 1. Start by writing the length value of the model to the file length characteristic on the Arduino.
let fileLengthArray = Int32Array.of(fileContents.byteLength);
await fileLengthCharacteristic.writeValue(fileLengthArray);
 
// 2. Pass the file buffer and the starting index to the function
sendFileBlock(fileContents, 0)

async function sendFileBlock(fileContents, bytesAlreadySent) {
// 3. Calculate the remaining bytes.
 let bytesRemaining = fileContents.byteLength - bytesAlreadySent;
 const MAX_BLOCK_LENGTH = 128;
 const blockLength = Math.min(bytesRemaining, MAX_BLOCK_LENGTH);
 
// 4. Transform the content into a Uint8array
 const blockView = new Uint8Array(fileContents, bytesAlreadySent, blockLength);
 
// 5. Write the value to the characteristic on the Arduino
 return fileBlockCharacteristic
   .writeValue(blockView)
   .then((_) => {
// 6. Remove the length of the block already sent from the size of bytes remaining
     bytesRemaining -= blockLength;
// 7. If the file has not finished transferring, repeat
     if (bytesRemaining > 0 && isFileTransferInProgress) {
       console.log(`File block written - ${bytesRemaining} bytes remaining`);
       bytesAlreadySent += blockLength;
       return sendFileBlock(fileContents, bytesAlreadySent);
     }
   })
   .catch((error) => {
     console.log(error);
     console.log(
       `File block write error with ${bytesRemaining} bytes remaining, see console`
     );
   });
}
```

Once this code runs, it should transfer the machine learning model, packet by packet, until there are no bytes remaining. Even though this works, you can add an additional step to ensure that the file received is the same as the file transferred.

## Checksum

A checksum can be used to detect errors that may have been introduced when the file was transferred between the browser and the Arduino. The `tf4micro-motion-kit.js` library uses a [CRC32](https://crc32.online/) function to verify data integrity.

It calculates a 32-bit [cyclic redundancy checksum](https://en.wikipedia.org/wiki/Cyclic_redundancy_check#CRCs_and_data_integrity) for the data transferred and repeats this calculation on the receiving side. If the two values don’t match, the data has somehow been corrupted and an error message can indicate that the transfer has failed.

I’m not going to dive into how the CRC32 function is implemented but you can view the source in [the repository](https://github.com/googlecreativelab/tf4micro-motion-kit/blob/main/web/dist/tf4micro-motion-kit.js#L264).

## How the Arduino runs the model

Once the Arduino has received the complete file and the checksum has verified the integrity of the data, the code on the device gets live input data from the built-in sensors using the `IMU.readAcceleration` method, transforms that data into tensors, and feeds it to the model to predict gestures. This code is written in C++ and is part of the program you need to upload to the Arduino so I won’t go into details, but you can have a look at the code in [this repository](https://github.com/googlecreativelab/tf4micro-motion-kit/tree/main/arduino/tf4micro-motion-kit).

## Conclusion

Transferring a machine learning model from the browser to an Arduino via bluetooth is done in 5 steps including connecting the board to the browser using the Web Bluetooth API, splitting the model into chunks, transferring them one by one to the device, running it with live data from the Arduino’s built-in sensors, and notifying the frontend once a gesture has been detected.

If you want to dive deeper into the entire code written for the Arduino sketch, you can find it in [this repository](https://github.com/googlecreativelab/tf4micro-motion-kit/releases/tag/v005) and if you want to look at the code written to create the model, the [Tiny Motion Trainer project is also open-source](https://github.com/googlecreativelab/tiny-motion-trainer)!

If you’re building anything interesting with this technology, let us know!

In the meantime, you can stay up to date with Stripe developer updates on the following platforms:
📣 Follow [@StripeDev](https://twitter.com/stripedev) and [our team](https://twitter.com/i/lists/1459720002567868418) on [Twitter](https://twitter.com/StripeDev)
📺 Subscribe to our [Youtube channel](https://www.youtube.com/StripeDevelopers)
💬 Join the official [Discord server](https://discord.com/invite/RuJnSBXrQn)
📧 Sign up for the [Dev Digest](https://go.stripe.global/dev-digest)

## About the author

![Charlie's profile picture. She has long brown hair and is wearing glasses. She is standing in front of a white wall with purple lights.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ww26qf6h3lwh0aiiegvr.png)

[Charlie Gerard](https://twitter.com/@devdevcharlie) is a Developer Advocate at Stripe, a [creative technologist](https://charliegerard.dev/) and [Google Developer Expert](https://developers.google.com/community/experts). She loves researching and [experimenting with technologies](https://charliegerard.dev/). When she’s not coding, she enjoys spending time outdoors, trying new beers and reading.