Enriching SharePoint Image Libraries with Azure Cognitive Services
Tags: azure, cognitiveservices, office365, serverless
Published: 2019-07-29 23:48:52

---
title: Enriching SharePoint Image Libraries with Azure Cognitive Services
published: true
tags: Azure,Cognitive Services,Office 365,Serverless
canonical_url: https://blog.siliconvalve.com/2019/03/08/enriching-sharepoint-image-libraries-with-azure-cognitive-services/
---

The ability to locate content in SharePoint via search has been continually improving over the years, seeing big jumps in impact with the release of features such as [Delve](https://support.office.com/en-ie/article/what-is-office-delve-1315665a-c6af-4409-a28d-49f8916878ca) and [OCR in OneDrive for Business](https://techcommunity.microsoft.com/t5/Microsoft-Search-Blog/Search-for-words-in-your-images-in-Office-365/ba-p/135703).

Indexing file content from Document Libraries has improved no end, but one item I see as a gap is extracting inferred meta-data out of images stored in SharePoint.

In this post, and the associated screencast, I’ll take a look at how we can utilise several of the capabilities of [Azure Cognitive Services](https://azure.microsoft.com/en-au/services/cognitive-services/) and [Azure Logic Apps](https://docs.microsoft.com/en-au/azure/logic-apps/) to quickly add value to image data that you upload to SharePoint.

### Baseline Services

In order to use this solution you will need:

- A SharePoint Online (or SharePoint on-premises) Document Library to which you can authenticate a [Logic App SharePoint Connector](https://docs.microsoft.com/en-us/azure/connectors/connectors-create-api-sharepoint).
- Images against which you can [train and test a Custom Vision classifier.](https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/getting-started-build-a-classifier) The [Open Images Dataset](https://storage.googleapis.com/openimages/web/index.html) is a good starting point if you want source images by category.
- An Azure Subscription (Trial or Paid) – sign-up for your free trial at [http://azure.com/free](http://azure.com/free). This demo will run happily within a Trial account.
- A trained [Azure Cognitive Services Custom Vision](https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/) classifier. For the purpose of my demonstration I used just the free tier for Cognitive Services which should work just as well for you.

### Building the Logic App

The best thing about Logic Apps is not only are they serverless, but they are almost entirely “codeless” as well. Rather than me describe in detail the steps, I’ve included the following items to help you get going.

#### 1. The designer view of the Logic App

Click on the image to view an enlarged version.

[![Logic App](https://siliconvalve.files.wordpress.com/2019/03/vision-logic-app.png?w=387&h=1024 "Logic App")](https://siliconvalve.files.wordpress.com/2019/03/vision-logic-app.png)

#### 2. The Logic App exported to GitHub

Grab the code from here: [https://github.com/sjwaight/SharePointCognitiveServices](https://github.com/sjwaight/SharePointCognitiveServices).

Note: this doesn’t come with the Connections which you will need to manually recreate using your own SharePoint and Cognitive Services API details.

In the image below you can see how to retrieve the GUID for a List in SharePoint Online – get to this page by clicking on “Library settings” from the settings menu (the cog) that is near your username in the Office 365 header.

![SharePoint List GUID](https://siliconvalve.files.wordpress.com/2019/03/sharepoint-list-guid.png?w=1100&h=370)

#### 3. A Screencast showing how this all hangs together

<iframe type="text/html" width="1100" height="619" src="https://www.youtube.com/embed/30Y9NV0Pmgw?version=3&amp;rel=1&amp;fs=1&amp;autohide=2&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent" allowfullscreen="true" style="border:0;"></iframe>

Note that in the screencast there’s a slight difference to my published Logic App above… see if you can find it !😉

Well, there we have it – how you can (fairly) easily add more value to the image data you store in SharePoint. Note that you could apply this approach to documents stored in Google Drive or Dropbox, as well as using many of other categories of Azure’s Cognitive Services to enrich data you already have!

One final item before I wrap up – if you want to learn more about Azure Cognitive Services then you can do it for free and without the need for an Azure Subscription. Microsoft Learn has a range of modules you can complete that covers this area. [Access them online using just a browser](https://docs.microsoft.com/en-us/learn/browse/?roles=developer&products=azure-cognitive-services).

Happy Days ! 😎