TinyML: Deploying TensorFlow models to Android
Tags: machinelearning, deeplearning, tinyml, tensorflow
Published: 2021-08-24 17:21:30

## What is TinyML?

Tiny machine learning (TinyML) is a field that focuses on running machine learning (mostly deep learning) algorithms directly on edge devices such as microcontrollers and mobile devices. The algorithms have to be highly optimized to be able to run on such systems since most of them are low powered.

## Wait, what do you mean by 'edge devices'?

An edge device is the device which makes use of the **final** output of machine learning algorithms, for instance, a camera that displays the result of image recognition, or a smartphone that plays speech synthesized from text. Most practitioners run machine learning models on more powerful devices, then send the output to edge devices, but this is starting to change with the advent of TinyML.

## Why TinyML?

The need to run machine learning directly on edge devices and the convenience that comes with this has made TinyML become one of the fastest growing fields in deep learning.

## How does one go about deploying ML to edge devices?

1. Train a machine learning model on a more powerful environment such as a cloud virtual machine or a faster computer.

2. Optimize the model, say, by reducing the number of parameters, or by using low precision data types such as 16 bit floats. This will make the model smaller and the inference faster and more power efficient at the cost of accuracy, which is a compromise you'll have to take.

3. Run the model 'on the edge'!

## TensorFlow Lite Quick Start

TensorFlow Lite is TensorFlow's take on TinyML.

### Converting a saved model from TensorFlow to TensorFlow Lite

```python
import tensorflow as tf
model=tf.keras.models.load_model("/path_to_model.h5")
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("/tflite_model.tflite", "wb").write(tflite_model)
```

As you can see, it only takes a few lines of code 😊.

### Running a TensorFlow Lite model in TensorFlow Lite's Python Interpreter

```python
import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="/tflite_model.tflite")	#initialize interpreter with model
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

inputs = []  #list of input tensors
for index,item in enumerate(inputs):
    interpreter.set_tensor(input_details[index]['index'], item)
    
interpreter.invoke()	#run model

outputs = [] #output tensors
num_outputs = len(output_details) #number of output tensors
for index in range(num_outputs):
	outputs.append(interpreter.get_tensor(output_details[index]['index']))

```

### Running a TensorFlow Lite model in an Android application

#### 1. Create a new Android Studio Project

#### 2. Import the model into Android Studio

Copy the .tflite model to app/src/main/assets/ - create the assets folder if it does not exist.

#### 3. Import TensorFlow Lite into your project

Add the following dependency to your app-level build.gradle

`implementation 'org.tensorflow:tensorflow-lite:+'`

#### 4. Load Model

Load the .tflite model you placed in your assets folder as a MappedByteBuffer.

```java
private MappedByteBuffer loadModelFile(Context c, String MODEL_FILE) throws IOException {
    AssetFileDescriptor fileDescriptor = c.getAssets().openFd(MODEL_FILE);
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

model = loadModelFile(this, 'name_of_model.tflite')
```

#### 5. Initialize Interpreter

```java
try {
    interpreter = new Interpreter(model);
}
catch (IOException e) {
    e.printStackTrace();
}
```

#### 6. Run Model

```Java
Object[] inputs = {input1, input2, ...}	//the objects in inputs{} are jagged arrays - what in TensorFlow would be considered tensors

Map<Integer, Object> outputs = new HashMap<>();	//same for outputs
outputs.put(0, output1);	//add outputs to the map

interpreter.runForMultipleInputsOutputs(inputs, outputs);	//get inference from interpreter
```

And there you go.

## Sample project

[Here is source code](https://github.com/badass-techie/IAmNotReal) for a [GAN](https://en.wikipedia.org/wiki/Generative_adversarial_network) deployed to an Android app with TensorFlow Lite. [Here is the android app](https://play.google.com/store/apps/details?id=com.apptasticmobile.iamnotreal) for you to play with.