Bees Health detection using Azure Custom Vision Service
Tags: azure, machinelearning, computervision, deeplearning
Published: 2021-03-31 15:52:38

**Every third bite of food relies on pollination by bees.** Honey beehive losses are quite prevalent due to the diseased bees. 

While many indications of hive strength and health are visible on the inside of the hive, frequent check-ups on the hive are time-consuming and disruptive to the bees' workflow and hive in general. By investigating the bees that leave the hive, we can gain a more complete understanding of the hive itself. For example, an unhealthy hive infected with varroa mites will have bees with deformed wings or mites on their backs. These characteristics can be observed without opening the hive. 

We aim to  **detect healthy and various types of diseased bees such as bees having ant problems and varro and hive beetles** with the help of **Azure Custom Computer Vision Services**. 

This post also demonstrates how easy is to build world-class computer vision problems with minimal effort using Azure Custom Vision services.      

# Data

> Data is taken from [Honey Bee Images Full Set](https://www.kaggle.com/jenny18/honey-bee-annotated-images). We have taken a subset of the data provided in the dataset to demonstrate the power of Azure Cognitive Services   

I have made a small dataset from the parent dataset in Kaggle mentioned above for quick experimentation. If you want to follow along, please have a look at [Bees Little Data Set ](https://www.kaggle.com/ambarish/beeslittledataset). The data has around `100 images of each of the classes healthy, ant problems, and varro and hive beetles`    

The steps to model and predict for this problem are as follows:

1. Create a Custom Vision resource   
2. Create a Custom Vision AI project   
3. Add Images to the project   
4. Train on the images and create the model  
5. Publish the model and expose the endpoint for use by other clients  
6. Use the exposed endpoint and predict using new images   



# Create a Custom Vision Resource

Create a Custom Vision Resource in the Azure Portal. Please make sure to use the Free Tier F0 so that you do not incur any costs.

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qzyy6613t33e3ycdrbxe.png)



![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0f4agdkk75gnaywhu4oy.png)


# Create a Custom Vision AI Project   

Navigate to https://www.customvision.ai/projects to create a custom vision project.

We created a project with
* Name - bees
* Project Type - Classification. Since we are classifying whether the bee is healthy, having ant problems, or having varro and hive mites   
* Classification Type - Multiclass. There are 2 choices here, Multiclass and Multilabel. We choose **Multiclass** since the image is associated with only one class ( healthy, ant problems, varro, and hive mites ). A single image is **not associated with multiple classes**. If a single image was associated with multiple classes, then we had to choose the Classification type as Multilabel.


![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ugaw3mihpkl7c12w8yn.png)

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5mpdkhn3ryfyoalozrgr.png)    

# Add Images     


![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/na7va9gjmcn301molhy1.png)


## Healthy Images

We upload the Healthy images and also tag them.

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qe0gd1l3u3ptycxlqqyz.png)

## Varro Hive beetles

We upload the Varro Hive beetles images and also tag them.

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k9lpe804zdpbycronm5i.png)    

We upload the ant problem images and also tag them.



# Train the images    

We train the images by clicking the **Train** button in the portal   

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ifcjqupu1jh5jld1053x.png)

## Quick Training  

We select Quick Training for the training of the images     

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9oqv48w67dahbfkr9ntp.png)

## Training Progress

Training on the dataset takes time. Once completed, we can see the metrics for classification such as Precision, Recall, and Accuracy            

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r93sbvxxv1stn2vqxlm4.png)

## Performance

We observe that the Precision, Recall, and Accuracy are very high. 

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m9942kf1q53iuochajzd.png)
 
# Quick Test

We choose an image of a bee having ant problems and see whether the model can predict correctly. We see that the model provides the highest probability to the ant problem class, therefore it successfully predicts     

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3ek0nxtter7dmwwbx4nq.png)


# Publish

We can now Publish the model so that we can use the endpoint of the model for the prediction of unseen images.   

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/78li3i76owh7rczv2z86.png)

## Publish Model

We publish the model with the name as **bee** and the prediction resource is **bee-Prediction**  

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sc7hzr74m3r3lhj0mq6d.png)


# Project details

We display the Azure Cognitive project which has the `project id, the published endpoint`. This will be used for predicting the unseen test images.   


![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7lahflkrxfquytvda8w2.png)

# Azure Cognitive  Resources

We show the 2 Azure Cognitive Resources, the training resource as well as the prediction resource. 
![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bofvznpasr6sdpqj5gds.png)      

# Code to predict   

The complete code is present in GitHub in

{% github ambarishg/beeshealthdetection %}

## Create the Node application

Navigate to a directory where the Node application is to be created. 

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ee13oft3bxn372x1lorr.png)

Run the following command to create the `package.json` file.   

```
npm init
```
![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hbyf6ame8gtwys8y6njn.png)
 

We install the Custom Vision npm packages. We run the following commands in `Azure Powershell` to install these packages   

## Install the Client library

```
npm install @azure/cognitiveservices-customvision-training
npm install @azure/cognitiveservices-customvision-prediction
```
![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/okcljzsx4mb0y2vera3h.png)
 
## Import the Node js libraries 

Create the file `index.js` and put the following lines in `index.js`  

```javascript
const util = require('util');
const fs = require('fs');
const TrainingApi = require("@azure/cognitiveservices-customvision-training");
const PredictionApi = require("@azure/cognitiveservices-customvision-prediction");
const msRest = require("@azure/ms-rest-js");
```
## Intialize with  Azure endpoint and keys  

```javascript
const sampleDataRoot = "f:/bees/";
const predictionKey = "<prediction key>";
const endPoint = "https://bees.cognitiveservices.azure.com/"
const publishIterationName = "bees"
const projectid = "<your-project-id>" 
```


## Authenticate with prediction keys   

```javascript
const predictor_credentials = new msRest.ApiKeyCredentials({ inHeader: { "Prediction-key": predictionKey } });
const predictor = new PredictionApi.PredictionAPIClient(predictor_credentials, endPoint);
```

## Predict with unseen data   

We predict with an image that has `varro and hive beetles`. We would see in the following section whether the prediction is correct or not.   

```javascript
const testFile = fs.readFileSync('f:/bees/Test/040_314.png');

(async () => {
const results = await predictor.classifyImage(projectid,
 publishIterationName, testFile);

// Show results
console.log("Results:");
results.predictions.forEach(predictedResult => {
    console.log(`\t ${predictedResult.tagName}: ${(predictedResult.probability * 100.0).toFixed(2)}%`);
});
})()
```

# Results

We get the results and observe that the model predicts correctly the image `varro and hive beetles`. The varro and hive beetles class has the highest probability around 75%.

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/06kigio93rg9ld8vy3ow.png)
 
# References  
1. Nodejs and Azure Custom Vision - https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/quickstarts/image-classification?tabs=visual-studio&pivots=programming-language-javascript