Point, Shoot, and Detect: Object Detection with PyTorch Mobile
Tags: ai, python, android, opensource
Published: 2023-11-19 20:39:10


In this article we will set up a Mobile Application for Object Detection. We will use PyTorch Mobile to optimize and run the YOLOv5 on an Android app.

## AI mobile app development Challenges 
Developing AI applications in mobile is difficult due to slow loading speeds and scalability problems. They heavily rely on cloud infrastructure, In addition, certain AI frameworks or designs can't be used on mobile devices, limiting their access and usefulness.

Larger AI systems need cloud connections, making them costly. Developers need to make algorithms compatible with mobile devices. We should be able to execute our AI locally on the phone instead of using the cloud as a Service. This will contribute to making AI more accessible and efficient in mobile applications. In this article, we will introduce you to PyTorch Mobile and setup a basic app for testing out this framework.

### PyTorch 
[PyTorch](https://pytorch.org/) is a machine learning framework based on the Torch library. People use PyTorch's tools to make neural networks and machine learning algorithms. PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on a CUDA-capable NVIDIA GPUs.

### About PyTorch mobile 

Running machine learning algorithms on mobile devices for prediction is crucial in today's tech landscape. The move to edge computing is not about speed. It's also about protecting user privacy and enabling new interactive experiences.
[PyTorch Mobile](https://pytorch.org/mobile/home/) helps connect training and deployment in the PyTorch environment. This platform streamlines the journey from model inference in mobile applications. PyTorch Mobile is in beta and gaining popularity for widespread production use. This software has APIs that will make mobile based machine learning better in the future.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kiog6u6xisda6ivqja6r.png)

Next, we will focus on trying out an Android app to test PyTorch Mobile capabilities.


## Setting up the Demo App

### YOLOv5 Vision AI
[YOLO](https://github.com/ultralytics/yolov5) is one of the popular open-source vision AI. YOLOv5 is a great object detection algorithm that uses Deep Convolution Neural Networks(DCNN). This helps to predict object classes instantaneously.YOLOv5 can detect people, vehicles, and animals in various environments. Its full form is "You only look once". This indicates that it takes the input photo and spits it into grids. Then it assigns bounding boxes and probabilities to each grid. The current v5 works on a complex architecture called the EfficientDet network, which improves its capability to detect objects smoothly.

These are the steps we take to setup YOLO for mobile:

### Optimizing the YOLO for the Android App

Clone repo and install requirements.txt in a `Python>=3.8.0` environment 

```sh
git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt
```

Create an optimized model for mobile

```sh
python export.py --weights yolov5s.pt --include torchscript --optimize
```

You will get a `yolov5s.torchscript` which is a mobile-optimized version of the model. Then we move it to the app directory and import it at `ObjectDetection/app/src/main/assets/yolov5s.torchscript`.

### Setting up the Android App

We will use an app from the PyTorch example repository to demonstrate PyTorch Mobile.

This is the repository [here](https://github.com/pytorch/android-demo-app/tree/master/ObjectDetection)


For setup download the latest Android Studio from [here](https://developer.android.com/studio)

Setup the Android Studio SDK from the SDK tools menu:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b0w6qq6y7irp36f6zy18.png)

Connect to a phone using wireless debugging.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z4825ox7jf3vboxc2fjd.png)

Run the gradle script with the `--stacktrace` option to show more debug information.

```sh
./gradlew installDebug --stacktrace
```

This will create a prompt on your phone to install the app.

## Results 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wbhhtrzbld8trseb3da3.png)

## Conclusion

This concludes our exploration of PyTorch Mobile. We got to know how to run algorithms in a mobile Android app and set up basic object detection. Hope this article was useful for you.

[![Hexmos](https://journal.hexmos.com/content/images/2023/08/hexmoslogo_2_1-2.png)](https://hexmos.com)


[![Twitter](https://img.shields.io/badge/Twitter-%231DA1F2.png)](https://twitter.com/HexmosTech)