TensorFlow Lite Android Support Library: Simplify ML On Android | by Shubham Panchal | Towards Data Science

Everyone loves TensorFlow and even more when you can run a TF model on Android directly. We all use TensorFlow Lite on Android and we have a couple of CodeLabs on it too. Using the Interpreter class on Android, we are currently running our .tflite models in apps.
But we have to do a lot before that, right? If we’re performing an image classification task, you’ll probably get a Bitmap or an Image object from the Camera library and then we transform it into a float[][][] or a byte[] . Then we load our model from the assets folder as a MappedByteBuffer . After calling interpreter.run() , we get the class probabilities, on which we perform the argmax() operation and then finally get a label from the labels.txt file.
This is the traditional approach which we developers follow and there’s no other way round.
The TensorFlow team has released the TensorFlow Lite Android Support Library to solve the tedious tasks of preprocessing. The GitHub page gives an intuition of their aim,
Mobile application developers typically interact with typed objects such as bitmaps or primitives such as integers. However, the TensorFlow Lite Interpreter that runs the on-device machine learning model uses tensors in the form of ByteBuffer, which can be difficult to debug and manipulate. The TensorFlow Lite Android Support Library is designed to help process the input and output of TensorFlow Lite models, and make the TensorFlow Lite interpreter easier to use.
First, we need to get this right in our Android project. Remembering build.gradle file? Right! We’ll add these dependencies in our app-level build.gradle file,
The first step in running a TFLite model is to create some array object which can store the inputs for our model as well the outputs which the model will produce. To make our lives easier and less struggling with float[] objects, TF Support Library includes a TensorBuffer class that takes in the shape of the desired array and its data type.
Note: As of 1st pril 2020, only DataType.FLOAT32 and DataType.UINT8 are supported.
You can even create a TensorBuffer object from an existing TensorBuffer object by modifying its data type,
val newImage = TensorImage.createFrom( image , DataType.FLOAT32 )
If you’re working with object detection, image classification or other images -related models, you need to work on Bitmap and resize it or normalize it. We have three ops for this namely, ResizeOp , ResizeWithCropOrPadOp and Rot900p .
First, we define our preprocessing pipeline using the ImageProcessor class.
Question: What are BILINEAR and NEAREST_NEIGHBOR methods?
Answer: Read this.
Next, create a TensorImage object and process the image.
Normalization of image arrays is necessary for almost all models to be it image classification models or regression models. For processing tensors, we have a TensorProcessor . Along with NormalizeOp we have CastOp , QuantizeOp and DequantizeOp .
Question: What is Normalization?
Answer: The process of converting an actual range of values into a standard range of values, typically -1 to +1 or 0 to 1. For example, suppose the natural range of a certain feature is 800 to 6,000. Through subtraction and division, you can normalize those values into the range -1 to +1.
Also, we have the freedom to build custom ops by implementing TensorOperator class, as shown below.
We can easily load our .tflite model using the FileUtil.loadMappedFile() method. Similarly, we can load the labels from a InputStream or from the assets folder.
And then perform inference using Interpreter.run() ,
I hope you liked the new TensorFlow Lite Android Support Library. This was a quick review of what’s inside but try exploring it yourself too. Thanks for reading!