Adding a TFLite model to your android app
Tags: tflite, android, machinelearning, keras
Published: 2022-12-28 14:24:31

# Adding a TFLite model to your android app
Hello there. Perhaps you are wondering how to straight-forwardly add some Machine Learning into your app. The [documentation](https://www.tensorflow.org/lite/examples/on_device_training/overview) currently talks about on-device training. Furthermore, you are likely to run into data type incompatibility errors when trying to run some inferences. In this tutorial, we are going to see how we can add an already-trained model into your app and get predictions from it. It will perform a heart attack disease probability detection. Let's dig in.

## Prerequisites
To follow through, you need hands-on knowledge in the following areas:
- Python Programming
- Machine Learning. Basic deep-learning knowledge is a plus.
- Kotlin programming
- Android development

You will also need the following tools installed.
- [TensorFlow](https://www.tensorflow.org/install). We will use this for creating the model.
- We will be using [this](https://www.kaggle.com/datasets/rashikrahmanpritom/heart-attack-analysis-prediction-dataset) Kaggle dataset. Download and extract it. Our interest will be the **`heart.csv`** file.
- [Pandas](https://pandas.pydata.org/docs/getting_started/install.html). This will be used to help in the easy manipulation of the dataset.
- [Scikit-learn](http://scikit-learn.org/stable/install.html). It will be for preprocessing the data.
- You also need [Android Studio](https://developer.android.com/studio) and [Jupyter Notebook](https://jupyter.org/install) installed. If you don't have Jupyter Notebook, you can use the normal Python scripts. But for the article, Jupyter Notebook will be used.

## What we will be doing
To achieve the goals, we will follow these steps:
1. Create a TensorFlow model.
2. Convert it to TFLite.
3. Create an Android app and install the dependencies needed.
4. Import the converted TFLite model.
5. Add the code to access the model and run the inferences.

## Creating the TensorFlow model
Be sure to start your Jupyter Notebook server before this.

In your working folder, create a folder called **`datasets`**. In the created folder, add the **`heart.csv`** file you extracted earlier in it. When done, move back to the root of your folder.

In a new cell, import the modules we need using the code below. (In the next steps, you will be adding the codes in new cells).

```python
from sklearn.model_selection
import train_test_split
import tensorflow as tf
import pandas as pd
```

These were discussed in the prerequisites section. We will be using the Keras(`tf.keras`) option provided by TensorFlow.

We then access the dataset and read it using pandas using this code.

```python
file_path = 'datasets/heart.csv'
heart_data = pd.read_csv(file_path)
```

Create a pipeline that will impute it and then scale it.

```python
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

proc_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy="median")),
    ('std_scaler', StandardScaler()),
])
```

Remove the output column.

```python
heart_data_no_output = heart_data.drop("output", axis=1)
```

Run the pipeline.

```python
heart_data_imputed = proc_pipeline.fit_transform(heart_data_no_output)
```

Split it into train data, test data, and validation data while maintaining a random state using a seed of 42.

```python
X_train_full, X_test, y_train_full, y_test = train_test_split(heart_data_imputed, heart_data.output, random_state=42)

X_train, X_valid, y_train, y_valid = train_test_split(X_train_full, y_train_full, random_state=42)
```
Create a class called `Model`.

```python
class Model(tf.Module):
    def __init__(self):
        input = tf.keras.layers.Input(shape=X_train.shape[1:])
        hidden1 = tf.keras.layers.Dense(50, activation="sigmoid")(input)
        hidden2 = tf.keras.layers.Dense(50, activation="sigmoid")(hidden1)
        output = tf.keras.layers.Dense(1)(hidden2)
        self.model = tf.keras.models.Model(inputs=[input], outputs=[output])
        opt = tf.keras.optimizers.SGD(learning_rate=0.01)
        self.model.compile(loss="mse", optimizer=opt, metrics=['accuracy'])
        self.model.fit(X_train, y_train, epochs=20,
                       validation_data=(X_valid, y_valid))

    @tf.function(input_signature=[
        tf.TensorSpec([None, 13], tf.float32),
    ])
    def predictor(self, x):
        predictions = self.model(x)
        return {
            "prediction": predictions
        }
```

We create the model(deep learning model) in the constructor. The model contains 4 layers:
- One input(`input`) layer takes in the number of train features as the shape parameter. That will be 13, equating to 13 neurons.
- Two hidden layers(`hidden1` and `hidden2`) each with 50 neurons, sigmoid activation functions, and each taking the previous layer as the input. This follows the [Keras Functional API](https://keras.io/guides/functional_api) architecture. We used the `sigmoid` activation function since we are trying to get the probability.
- The output layer(`output`) had only 1 neuron since we are outputting the highest probability, which is only one thing.

We use a Stochastic Gradient Descent Optimizer with a learning rate of 0.01. This optimizer will be used when compiling the model. The Mean Squared Error(`mse`) and `accuracy` are used for the loss and performance measuring metrics.

In fitting the model, we use 20 epochs while passing in the necessary data.

After the constructor, we have a TensorFlow function called `predictor`. We pass in the shape of the data(input) and the data type of the input in the function's signature.

```python
    @tf.function(input_signature=[
        tf.TensorSpec([None, 13], tf.float32),
    ])
```

The input will be passed in as an argument signified by the parameter, **`x`**. We use the `self.model(x)` statement. This returns data containing the model's prediction. We will use a map with a key, `prediction`, to tap it.

```python
    def predictor(self, x):
        predictions = self.model(x)
        return {
            "prediction": predictions
        }
```

## Convert the model to TFlite
We will first initialize the model.

```python
tcardio_model = Model()
```

We then save it in a directory called **`our_model`**. Here is where we will pass in the signatuure of our Tensorflow function. We give it the signature key, `predictor`. This will be used by our TFLite model to identify the function.


```python
SAVED_MODEL_DIR = "saved_model"

tf.saved_model.save(
    tcardio_model,
    SAVED_MODEL_DIR,
    signatures={
        'predictor':
            tcardio_model.predictor.get_concrete_function(),
    })
```

The next code is where will create the TFLite model.

```python
import os

converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_DIR)

converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS, 
    tf.lite.OpsSet.SELECT_TF_OPS 
]

converter.experimental_enable_resource_variables = True
tflite_model = converter.convert()

model_file_path = os.path.join('tcardio_model.tflite')
with open(model_file_path, 'wb') as model_file:
    model_file.write(tflite_model)
```

It starts by importing the `os` module since we are accessing the file system.

The next line simply instructs the Python interpreter to create a variable for a TFLite converter from the saved model.

In the array(`converter.target_spec.supported_ops`), we add options for enabling TFLite operations.

And finally, we add the TFLite model to the root path.

Run all cells. If everything completes successfully, you 
 will see a file called `tcardio_model.tflite` in your working directory.

## Creating the Android app and install the dependencies needed
Follow the process of creating an android app. You can choose the **Empty Activity** option. But we won't modify the UI in any way.

Open the module-level `build.gradle` file and add the TFLite dependencies. Take note of the versions. There is a newly released version that isn't compatible with what we are going to do next. Stay tuned for the newly updated article.

```kotlin
implementation 'org.tensorflow:tensorflow-lite:2.9.0'
implementation 'org.tensorflow:tensorflow-lite-gpu:2.10.0'
implementation 'org.tensorflow:tensorflow-lite-support:0.4.2'
```

Sync the Gradle.

## Import the converted TFLite model.
After the build is done, create a new folder called **`assets`**. Then copy the TFLite model generated into the directory.

In the `MainActivity` class, we will add the code necessary for accessing the model ad running the inferences. This is achieved in the next step.

## Add the code to access the model and run the inferences
Open the `MainActivity.kt` file. In the file create a `companion` object and add the model's path.

```kotlin
    companion object {

        private const val MODEL_PATH = "tcardio_model.tflite"
    }
```

Then declare a class-level variable for the TFLite model using a lazy delegation. We pass in the TFLite file(in `ByteBuffer` form) in the `Interpreter` class.

```kotlin
    private val tflite by lazy {
        Interpreter(
            FileUtil.loadMappedFile(this, MODEL_PATH))
    }

```

Due to the compatibility issues between Kotlin's(Java) Float and TensorFlow Float(`float32` or `float64`) datatypes, we need a helper function to convert the Kotlin Float(s) to float buffers before sending them to the TFLite model. We will use the method below.

```kotlin
    fun floatArrayToBuffer(floatArray: FloatArray): FloatBuffer? {
        val byteBuffer: ByteBuffer = ByteBuffer
            .allocateDirect(floatArray.size * 4)

        byteBuffer.order(ByteOrder.nativeOrder())

        val floatBuffer: FloatBuffer = byteBuffer.asFloatBuffer()

        floatBuffer.put(floatArray) 
        floatBuffer.position(0)
        return floatBuffer
    }
```

It first allocates a capacity of 4 since Java Floats are 4 bytes in size. 

> NOTE: We are referring to Java because Kotlin is based on Java and still uses some of Java's classes.

We then set the buffer allocation order to be the same as that of the [underlying JVM machine](https://developer.android.com/reference/java/nio/ByteOrder#nativeOrder()). We then use the `asFloatBuffer()` method to convert the byte buffer to a float buffer. The method finishes by adding the float array we will pass into the float buffer we just created and sets the float buffer's start position to 0.

For performing the prediction, we will need a method called `doInference()`.

```kotlin
    fun doInference():  Map<String, FloatBuffer?> {
        val input = floatArrayOf(63.0F, 1.0F, 3.0F, 145.0F , 233.0F , 1.0F, 0.0F, 150.0F, 0.0F, 2.3F, 0.0F, 0.0F, 1.0F)

        val inF = floatArrayToBuffer(input)

        var outs = floatArrayOf(0.0F)

        var outF = floatArrayToBuffer(outs)

        val inputs: Map<String, FloatBuffer?> = mapOf("x" to inF)

        var outputs: Map<String, FloatBuffer?> = mutableMapOf("prediction" to outF)

        tflite.runSignature(inputs, outputs, "predictor")
        return outputs
    }
```

The method sets our input array to the `floatArrayToBuffer` method we have just finished creating. We do the same for the output variable. We need this as it is what TFLite will map the output to.

The maps are used for matching the inputs and the outputs according to what was set in the TensorFlow model's function. Remember we had the input as `x` and the output as `prediction`.

We use the TFLite's `runSignature` method to get the prediction. The input, the output variable, and the signature of the Tensorflow function are passed in as parameters.

To get the output, we will add some statements in the `onCreate` method. These are the statements.

```kotlin
        try {
            var pred = doInference()

            var cb = pred.get("prediction")

            Log.i("PREDICTION: ", cb?.get(0).toString())
        }catch(e: Exception){
            Log.e("ERR: ", e.toString())
        }
```

W use a `try-catch` block to get any exception that may occur. Since we said the TensorFlow function returns a map, we will use the `Map.get()` method while passing in the key to retrieve the corresponding output. In this case, it will be a float buffer. The prediction value is normally returned as the first value of the float buffer returned. So we used the method `get(0)` while passing in the position.

On checking the logcat, you will see such an output.

```kotlin
PREDICTION:             com.bigdolphin.predtest    I  0.78253055
```

That means the model gave a 78.25% chance of a heart attack. This is in agreement with the sigmoid curve. Above 0.5 is 1 and below that is 0. The input data is from the first row of the dataset whose output is 1.

## Conclusion
That is it. We have seen how to use a TFLite model in our apps. We did so by first creating a model from scratch, converting it, and then feeding it to the Android platform where we run an inference. I hope you gained some insights. In case of any questions, inaccuracies, or contributions, please feel free to leave a comment or email me at contact@terrence-aluda.com.

Have a good one.