Detecting Objects with a Tello Drone 
Tags: ibmcloud, node, machinelearning, npm
Published: 2019-09-25 17:15:25

Have you ever wanted to detect objects using live video streaming from a drone? If so, with this tutorial you now can! 

{% twitter 1174800354560630790 %}

# **Let's Get Started** 

In this tutorial, we will be using [Node-RED](https://nodered.org/about/). Designed and built by IBM, **Node-RED** is a free open source logic engine that allows programmers of any level to interconnect IoT, cloud-based systems, web services, databases, API’s and more!


### **Step 1:** Get started by installing Node-RED locally 

You can check out [this tutorial](https://nodered.org/docs/getting-started/local) on how to install Node-RED on your local computer , or follow the commands below : 

``` sudo npm install -g --unsafe-perm node-red ``` 

This should install Node-RED, once installed run this command : 

```node-red ``` 

This should point you to a Node-RED URL `http://127.0.0.1:1880/` that you can put in your browser to see the Node-RED editor 

*Note: In later steps, we will be adjusting the settings.js file that comes when you install Node-RED* 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/n5u9sjpb164qh9mvjhj1.png)

*Node-RED editor in your browser* 

### **Step 2:** Install [FFMpeg](https://ffmpeg.org/) on your local computer 

**[FFMpeg](https://www.hongkiat.com/blog/ffmpeg-guide/)** is a very powerful command-line tool that is used for performing various conversion operations on audio and video files. This tool will be the bread and butter for video streaming! It is free to use and is available for Windows, Linux and Mac operating systems

If you have a Mac, the best way to install it is by using homebrew.

Run: `brew install ffmpeg` to install FFmpeg locally to your computer. 

This will be critical to see streams that are flowing in from your local computer! 

### **Step 3:** Install the [FFMpeg Node](https://github.com/bourdakos1/node-red-contrib-ffmpeg ) in Node-RED 

There are two ways you can install this node 
1. Do `npm install node-red-contrib-ffmpeg` in the same directory that node-RED (from step 1) was installed 

2. Open your node-RED browser and find `node-red-contrib-ffmpeg` in manage pallet 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/ibu5vo6o2qecbbmk0ie6.png)

*Open hamburger menu on right hand side > manage palette* 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/3l5otykekic0zx17v7t1.png)

*Click install and you should see the ffmpeg node in your palette* 


### **Step 4:** Train a Model 

Follow [this tutorial](https://github.com/cloud-annotations/training/) to train an object detection model . 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/puooxnym5h89okwdj2wy.png)

Once you finish this tutorial you should end up with a `model_web` folder. 

### **Step 5:** Copy `model_web` into Node-RED and adjust settings
Once you have a `model_web` directory with your trained object model you will need to copy that into the directory that you installed Node-RED 

We will then adjust the settings.js file in your node-RED director 

Open `settings.js` and search for and uncomment out the following 

```httpAdminRoot: '/editor',
    httpStatic: '<path to model web directory >',```
I set the endpoint of `httpAdminRoot` to editor, so now when you run `node-red` command to see your node-RED editor you will now go to this url `http://127.0.0.1:1880/editor`  

`httpStatic` will include the location of your model_web folder  

### **Step 6:** Configure your [Tello Drone](https://www.ryzerobotics.com/tello) 

Before you can use your Tello Drone you MUST activate it in the official Tello Drone app. Once your drone is activated, you can connect to it's WiFi Network TELLO-XXXXXX and send it commands via UDP 

### **Step 7:** Import Flow into Node-RED 

Start Node-RED locally and Import the following flow : 

{% gist https://gist.github.com/pmmistry/7d28532871ada1179232e7f5e4efb769 %}

To import this flow , copy the json in the GIST above to your clipboard. 

On your Node-RED editor go to the hamburger menu on top right-hand side > Import > Clipboard 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/4estveen8mfbc6jxbbpi.png)

Paste the json that you copied from your clipboard and click Import 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/sg3a8fdsyjkv4s6v4f2s.png)

You should see the following flow on your editor :

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/u1je8nhbio2t4219g8we.png)

### **Step 8:** Adjust Nodes and Stream from Tello! 

Double click on the FFmpeg node and make sure it is configured for Tello Drone. Also by default, the URL will be set to `/stream` . *Note, this can be changed to whatever url endpoint you want* 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/zr84q1z8gl96ki0f2eu9.png)

*In this example our stream will be accessible at ws://<host>:<port>/stream* 

**Using the stream** 
To render the video stream in the browser, we use a library called [JSMpeg](https://github.com/phoboslab/jsmpeg).

*If you changed the stream url endpoint to something else , make sure you configure this in the template node on line 58. In the default we have it set as /stream * 

```const url = `ws://${window.location.hostname}:${window.location.port}/<url_endpoint>```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/tffe3jfpe1khumc00x5i.png)


**Turn Tello Wifi On** 
Make sure your Tello drone is charged and you have turned it on. You should be able to connect to your Tello's wifi. 

Once connected click Command and then Stream on 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/iyg4muh1b1w4yet5s0bv.png)

Go to `http://127.0.0.1:1880/dashboard` and you should see video streaming from your tello! If you have trained an object detection model, you should also see your tello detect objects! In my case I trained a model to distinguish between thumbs up and thumbs down 

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/b2tvoogxldigd4hr2km7.png)

*If you want to stop stream make sure to click on the stopstream inject button in the node-RED editor* 

### The End 

That's all folks :). Thank you so much for reading! 

Check out [this repo](https://github.com/bourdakos1/node-red-contrib-ffmpeg) if you're interested in video streaming without the object detection! 

Feel free to reach out to me with any questions in the comments below, or connect with me on twitter [@poojamakes](https://twitter.com/poojamakes)

If you found this tutorial fun and helpful, it would mean a lot to me if you gave it some <3 and shared it to help others! Thank you again!