Enriching a Knowledge Graph using Connectors and Functions
Tags: yext, typescript, webdev
Published: 2022-05-23 22:21:27

I was recently developing a React web app designed by one of the product designers on my team. She created a clean looking home screen design that featured some image carousels where each image appears over top of a light orange background. After I wrote the necessary components, I realized they didn’t look the way I wanted.

![white background](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8d1uahkztbic64lhm9f4.png)

Each image returned by the [Answers API](https://hitchhikers.yext.com/docs/liveapis/answersapi/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors) had a white background when I needed them to have a transparent background. I knew that Cloudinary offered an image transformation API that I could use to remove the white background, but I needed to transform every image for over 1000 entities in my [Knowledge Graph](https://hitchhikers.yext.com/modules/io102-what-is-knowledge-graph/01-what-is-knowledge-graph/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors).  In this article, I will show how I transformed every image in my Knowledge Graph data with the Cloudinary Image Transformation API, a [function plugin](https://hitchhikers.yext.com/guides/local-function-development-with-deno/06-deploy-to-production/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors), and the [Data Connector framework](https://hitchhikers.yext.com/modules/kg139-data-connectors/01-data-connectors-overview/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors).                                                       

## Writing My Function Plugin

All of the images in my Knowledge Graph were JPEGs which do not support transparent backgrounds. I needed to use Cloudinary [chained transformations](https://cloudinary.com/documentation/image_transformations#chained_transformations) to first [convert my images to PNGs](https://cloudinary.com/documentation/transformation_reference#f_format) and then [remove the white background](https://cloudinary.com/documentation/transformation_reference#e_bgremoval). I also chose to use [Eager Transformations](https://cloudinary.com/documentation/transformations_on_upload) to receive the transformed asset immediately after storing it in my Cloudinary account. 

After creating my [plugin folder structure](https://hitchhikers.yext.com/guides/local-function-development-with-deno/02-create-plugin-folder-structure/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors), added `uploadImageTransform` to `mod.ts` to make the transformation request and return the new URL.

```tsx
// Chained, eager tranformation 
const eager = "f_png,e_bgremoval";

const uploadImageAndTransform = async (
  file: string,
  public_id: string,
  signature: string,
  timestamp: number
): Promise<string> => {
  console.log(`Uploading ${public_id} to Cloudinary and transforming ${file}`);

  try {
    const response = await axiod.post(
      "https://api.cloudinary.com/v1_1/yext/image/upload",
      {
        file,
        public_id,
        api_key: CLOUDINARY_API_KEY,
        eager,
        signature,
        timestamp,
      },
      {
        headers: {
          "Content-Type": "application/x-www-form-urlencoded",
        },
      }
    );

    return response.data.eager[0].url;
  } catch (error) {
    if (error.response) {
      console.error(`${error.response.status}: ${error.response.data}`);
    }
    throw error;
  }
};
```

Cloudinary’s transformation API requires a [digital signature](https://cloudinary.com/documentation/signatures) to provide authentication for each request. Since I am not using a Cloudinary SDK, I had to write my my own function for generating a signature using the [web crypto API that is included with Deno](https://medium.com/deno-the-complete-reference/web-crypto-apis-in-deno-6ba9cb232503). 

```tsx
const generateCloudinarySignature = async (params: string) => {
  const data = new TextEncoder().encode(params);
  const digest = await crypto.subtle.digest("sha-1", data.buffer);
  return encodeToString(new Uint8Array(digest));
};
```

I then wrote the `removeBackground` function for my Data Connector to call. The function accepts a string that is expected to be delineated with a ‘|’. The `public_id` is intended to be the Knowledge Graph `entityId` and `url` is the original url of the photo that needs transformed.

```tsx
export const removeBackground = async (input: string) => {
  const timestamp = Math.floor(Date.now() / 1000);
  const [public_id, url] = input.split("|");

  const signature = await generateCloudinarySignature(
    `eager=${eager}&public_id=${public_id}&timestamp=${timestamp}${CLOUDINARY_API_SECRET}`
  );

  if (public_id && url) {
    const imageUrl = await uploadImageAndTransform(
      url,
      public_id,
      signature,
      timestamp
    );

    return imageUrl;
  }
};
```

I wrote a function in `test.ts` to make sure the transformation worked for one of the images in my Knowledge Graph.

```tsx
const testInput =
  "beer_60|https://a.mktgcdn.com/p-sandbox/rrK5bxcoVAgkGckDnA7GlhyC1VOpV6eEf4KjjlFumQs/400x600.jpg";

Deno.test("test remove background", async () => {
  const newUrl = await removeBackground(testInput);
  console.log(newUrl);
});
```

## Applying the Function Plugin to a Data Connector

After uploading my function plugin to my Yext account, I created a new Data Connector with “Pull from API” as the source to pull the entities from my own account using the [Knowledge Graph Entities: Get API](https://hitchhikers.yext.com/docs/knowledgeapis/knowledgegraph/entities/entities/#operation/getEntity/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors). I also needed to create an app in the Developer Console to get an API Key to allow my Data Connector to make requests to the Entities API.

I autogenerated my selectors and then applied a Merge Columns Transformation to combine `entityId` and `primaryPhoto.image.url` into a new column called Transformed Image.

![merge columns](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/efwo7a8a74gf7ttr7bb4.png)

I took the new column and applied my `removeBackground` function to it.

![apply function](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/70ey6zhmd4ocw1tgedu8.png)

Finally, I mapped the `meta.id` back to entityId to let the Connector know that I am editing existing entities and replacing the Primary Photo Image URL with a new URL.

![map fields](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7cszx6vs7161l3y7q9cq.png)

I ran my newly created Connector and it completed in less than 5 minutes. When I refreshed my app home screen, the white backgrounds were gone.

![transparent background](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5954o3co4k1hqhnyrd10.png)

## Conclusion

I had the option of making a request to Cloudinary from my React application each time I wanted to fetch the transformed image. However, I am already fetching the entities from my Knowledge Graph via the Answers API. Making the additional request to Cloudinary would eat into my month [bandwidth allotment](https://support.cloudinary.com/hc/en-us/articles/203125631-How-does-Cloudinary-count-my-plan-s-quotas-and-what-does-every-quota-mean-). If this were a production application, I would go over my free account allotment pretty quickly. This way, the resources are only fetched when I make the initial request for each image on the Connector run. 

This example could be extended to apply more complex image transformations. I could also upload a separate function to call a different API for further Knowledge Graph enhancements. 

You can see the complete code for the function [here](https://github.com/YextSolutions/transform-image/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors) and read more about Data Connectors [here](https://hitchhikers.yext.com/modules/kg139-data-connectors/01-data-connectors-overview/?utm_source=draftdev&utm_campaign=hitchhikers&utm_countent=enrich-kg-functions-and-connectors).