Computer vision API - Using Microsoft Azure Cognitive services
Tags: tutorial, azure, machinelearning, node
Published: 2022-05-13 14:56:48

## Cognitive services
Cognitive Services are a set of machine learning algorithms that Microsoft has developed to solve problems in the field of Artificial Intelligence (AI). The goal of Cognitive Services is to democratise AI by packaging it into discrete components that are easy for developers to use in their own apps.

I recently created an Application - `Azura` with same method.
{%embed https://github.com/Seek4samurai/Azura %}

If you're familiar with Computer vision, you must know how it works. This is a technique in which we train a machine's vision to recognise real world objects and similar things, which could either be some objects or even living things like human face or recognising animals.

Microsoft Azure provides with some free to use cognitive service APIs to create such computer vision powered applications.

## Getting started
**Creating Azure resource**

Select Computer vision from resource and then create a resource.
![Azure Home](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/otr5z9vxli4b6zz7mdvs.png)

After you've created a resource.

![Azure resource](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/33ovrcjg6ic2o3koo986.png)

**Using API client**
Once you did all before steps correctly, you can get started with your workspace.

**Server setup**
Get started with creating a server using, we are using nodejs `npm init -y`. Once you've initialised, you've to install following packages and libraries.
```
{
  "name": "azura-backend",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "nodemon ./src/index.js",
    "start": "node ./src/index.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@azure/cognitiveservices-computervision": "^8.1.0",
    "cors": "^2.8.5",
    "dotenv": "^16.0.0",
    "express": "^4.17.2"
  },
  "devDependencies": {
    "nodemon": "^2.0.15"
  }
}
```
Here, we are using Express for creating server. And to use the Azure-cognitive services we install 
`npm i @azure/cognitiveservices-computervision`

Create a `src` folder and `index.js` file to start a server and handle basic routes in it.
```
const express = require("express");
const dotenv = require("dotenv");
const cors = require("cors");

dotenv.config();

const imageController = require("./controller");

const app = express();
app.use(cors({
  origin: "*"
}));
app.use(express.json());

// Routes
app.use("/", imageController);

const PORT = process.env.PORT || 5000;

app.listen(PORT, () => {
  console.log(`App running or port ${PORT}`);
});
```

Once this is done, create `controller.js` file, where we will use computer vision client for our application.
```
const express = require("express");
const ComputerVisionClient =
  require("@azure/cognitiveservices-computervision").ComputerVisionClient;
const ApiKeyCredentials = require("@azure/ms-rest-js").ApiKeyCredentials;

const router = express.Router();

router.post("/describe", async (req, res) => {
  const KEY = process.env.KEY;
  const ENDPOINT = process.env.ENDPOINT;

  // Create a new Client
  const computerVisionClient = new ComputerVisionClient(
    new ApiKeyCredentials({ inHeader: { "Ocp-Apim-Subscription-Key": KEY } }),
    ENDPOINT
  );

  if (!req.body.imageUrl) {
    return res.send("Image url is not set! Please provide an image!");
  }
});

module.exports = router;
```
Remember you've to create `.env` file in your local workspace and paste your API keys and endpoint, and to use them I'm using `dotenv` package (hope that is understandable). We've initialised the client and when we hit the post request to redirect to `/describe`, it should hit our client. You can try using postman to check this API call.
And in the last line we are just checking if the request is empty, then simply return that empty url message.

After all this we can go ahead and create a `try-catch` block and use the 
```
  try {
    // Describe and Image Url
    const descUrl = req.body.imageUrl;
    const caption = (await computerVisionClient.describeImage(descUrl))
      .captions[0];

    res.send(
      `This maybe ${caption.text} (confidence ${caption.confidence.toFixed(2)})`
    );
  } catch (error) {
    console.log(error);
    res.send(error.message)
  }
```
Here, we are getting the `req.body.imageUrl` from our frontend and using that URL for our client. And it will return and send response back to frontend.

## Frontend overview
Since frontend is not the point of focus in this tutorial, we can take a quick overview of it.
We take input from user and that URL is sent to our backend. I'm using `Axios` for that purpose.
```
const res = await axios.post(
      "https://YourURL/describe",
      {
        imageUrl,
      }
    );
```
In place of YourURL, paste your server's URL.

You can check to print the response or log it in console. This will accept image's URL and return it's description, what the image is about.

_Thank you for reading._