Localizing Image Text Overlays using Cloudinary + 8base
Tags: javascript, serverless, aws, graphql
Published: 2019-09-07 16:34:38

Imagine that you're a social media intern. Please, bear with me! You're a social media intern and your manager, instead of including you in on strategy meetings, hands you a laundry list of image-related tasks. It includes:

1. Crop *this*.
2. Brighten *that*.
3. Overlay "New arrivals!" on the Twitter header image.
4. Resize the Facebook share card.
5. Get me coffee.
6. Translate "New arrivals!" to Russian, German, and Swahili.

...You get the point 

Now, you're a smart cookie. There's no way you want to spend your day wasting away on the computer having to **MANUALLY** manipulate all these images. So, you set out to find a better way. 

##⏸ Story over, for now

That scenario is no made-up story. It's real! Every day, whether at work or for personal projects, millions of images get created, edited, updated, hosted, taken down, lost, and so on. Services that help manage the chaos or streamline the process can be **incredibly** helpful.

The other day, a friend shared with me [Cloudinary's URL API](https://cloudinary.com/documentation/solution_overview?query=URL%20API&c_query=Account%20and%20API%20setup%20%E2%80%BA%20URLs%20and%20endpoints#urls_and_endpoints). Immediately, I saw it as being an answer to so many image-related inefficiencies suffered by companies and people. Very quickly, I'll mention a *why* and a *how*.

### Why 
From a single image, dozens of tweaked versions might need to get created (faded, text-overlay, black-white, etc.). Each of those versions takes time to create, update, and organize.

### How
Cloudinary's URL API takes a dynamic parameter that applies transformations to the image upon retrieval. Think of it like on-demand Photoshop!

Personally, this got me excited. Notably, the *Text Overlay* transformation. After spending a little time playing with it, I wanted to see if it could be extended to incorporate localization (translation) of image text.

A working demo came out of the exercise. You can play with it [here](http://cloudycam.8base.demo.s3-website-us-east-1.amazonaws.com/), or keep reading and learn how it works!

![Cloudinary URL API Anatomy](https://thepracticaldev.s3.amazonaws.com/i/1nnhvf3ruoy51906doj4.png)

First off, let's take a quick look at the anatomy of the URL API. A large section of parameters exists between the `upload/` and `/horse.jpg` sections. These are a list of image transformations that get executed when the image is requested. Pretty cool! Right? The documentation is [right here](https://cloudinary.com/documentation/solution_overview?query=URL%20API&c_query=Account%20and%20API%20setup%20%E2%80%BA%20URLs%20and%20endpoints#urls_and_endpoints) if you'd like to dive deeper.

```text
https://res.cloudinary.com/demo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:I%20am%20a%20unicorn!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/horse.jpg
```

Now, the image you see below gets rendered using the link above. Moreover, - this is the crucial part - if you change the transformation, a brand new image gets returned!

![Cool unicorn](https://thepracticaldev.s3.amazonaws.com/i/sj9fusjqry3q6wbcizg1.png)

The `l_text:Times_100_bold_italic:I%20am%20a%20unicorn!` is easily visible when inspecting the URL. While we can't add a custom transformation tags (that is, on Cloudinary's side), we do have the ability to apply transformations to the URL. Meaning that, in the case of localizing our image overlays, we can coerce the URL before requesting the image.

A serverless GraphQL resolver function can get deployed to an [8base workspace](https://8base.com) to accomplish this. It can handle the parsing of the URL and translation. There are many ways to deploy a serverless function. However, [8base](https://8base.com) made it super simple and straight forward.

As a quick specification, let's design the function to behave as follows.

1. If a `local_[2-char-lang-code]` tag precedes the text, translate the text, and update the URL.
2. If a local code does not precede the text, return the original URL.

## Enough talk, let's do it

##### 1. Create a new 8base project 
*If you have an existing projected, you can always add a new function to it.*

```sh
# Install the CLI globally
npm install -g 8base-cli

# Initialize a new project with a GraphQL resolver called "localizer."
8base init my-project --functions="resolver:localizer"
```

These commands create a new project with all the files and code we need to start invoking our GraphQL resolver function. We'll need to make a few changes though before it's translating our Cloudinary URL's 😉

##### 2. Update the resolver's `graphql.schema`
Open up the file at `src/resolvers/localizer/schema.graphql`. We need to define our query operation and response. In this case, we'll be returning an object with the updated `url` after having received the `cloudinaryUrl`. Update the file with the following snippet.

```javascript
type LocalizeResult {
  url: String!
}

extend type Query {
  localize(cloudinaryUrl: String!): LocalizeResult
}
```

##### 3. Update the mock for `invoke-local`
Update `src/resolvers/localizer/mocks/request.json` so that the function can get invoked locally with data. The mock file generated has the same schema as what gets passed to the function in production.

```json
{
  "data": {
    "cloudinaryUrl": "https://res.cloudinary.com/cdemo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:local_es:Breaking%20news:%208base%20solves%20all%20your%20image%20related%20needs!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/dosh1/img-0.jpg"
  },
  "headers": {
    "x-header-1": "header value"
  },
  "body": "{\"cloudinaryUrl\":\"https://res.cloudinary.com/cdemo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:local_es:Breaking%20news:%208base%20solves%20all%20your%20image%20related%20needs!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/dosh1/img-0.jpg\"}"
}
```

##### 4. The function
We're going to need a translation engine. I chose AWS Translate, which offers 2-million free characters per month. Let's add the required library and config to the project.

```sh
# Install AWS SDK
npm install --save aws-sdk
```

Update `src/resolvers/localizer/handler.ts`
```javascript
const AWS  = require('aws-sdk');

AWS.config.update({
    region: 'us-east-1', 
    credentials: {
      accessKeyId: process.env.AWS_IAM_SECRET_KEY, 
      secretAccessKey: process.env.AWS_IAM_ACCESS_KEY 
    }
});

const translate = new AWS.Translate({ apiVersion: '2017-07-01' });

/* Other code ... */
```

When developing locally, you'll need to set your AWS credentials as environment variables or static values. The example you see above is what works when the function gets deployed to 8base. Here's the documentation on accessing [8base environment variables](https://docs.8base.com/development-tools/dev-env/runtime_environment#environment-variables).

Since we're using TypeScript, the function response needs a `type`. This type **must** match the structure and name of that added to the `graphql.schema` file. For our scenario, prepend the following to the function body.

```javascript
type LocalizeResult = {
  data: {
    url: string
  }
};
```

The function body is pretty self-explanatory. Instead of describing it *here* and then showing it *there*, please read the inline comments for clarification on what's happening.

```javascript
export default async (event: any, ctx: any) : Promise<LocalizeResult> => {
  /**
   * Regex Statement for matching our custom local_tag and preceeding text
   */
  const REG_EX = /(local_[a-z]{2})\:(.*?)([,\/])/g
  /**
   * Pull the given cloudinary url from our function arguments 
   */
  let url = event.data.cloudinaryUrl
  /**
   * Execute our Regex statement returning a match object
   */
  const matchObj = REG_EX.exec(url);
  /**
   * If a local tag is matched, we're in business! If not,
   * we're simply returning the passed url.
   */
  if (matchObj) {
    /**
     * Pull out the matched local and text values from
     * the matchObj array.
     */
    let local = matchObj[1], text  = matchObj[2];

    try {
      /**
       * Make the request to AWS Translate after decoding the given text
       * and slicing the last two characters from the local tag (e.g. local_es)
       */
      let request = translate.translateText({
        TargetLanguageCode: local.slice(-2),
        SourceLanguageCode: 'auto',
        Text: decodeURI(text)
      }).promise();

      let data = await request;
      /**
       * The ACTUAL cloudinary url will break if it has our custom tag. Plus, we
       * need to update the text with the translation! So, let's replace the previously
       * matched locale and text with our tranlsated text, that needs to be escaped.
       */
      url = url.replace(`${local}:${text}`, data.TranslatedText.replace(/[.,%\`\s]/g,'%20'))
    } 
    catch (err) {
      console.log(err, err.stack);
    }    
  } 
  /**
   * Return the final result.
   */
  return {
    data: {
      url
    }
  }
};
```

##### 5. Run it!
Done! Let's prove it by invoking our function locally. The returned URL's text section translates to the locale specified language! Copy the link and throw it in a browser to see the magic.

```sh
8base invoke-local localize -p src/resolvers/localize/mocks/request.json
invoking...

Result:
{
  "data": {
    "localize": {
      "url": "https://res.cloudinary.com/demo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:¡Soy%20un%20unicornio%20genial!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/horse.jpg"
    }
  }
}

```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/z21h9bknxszlihrwn6uk.png)

##🏁 Wrap up
Sorry, we're going back to storytime. Remember back when you were a social media intern? Well, you ended up finding and using Cloudinary for all your on-the-fly image transformation and 8base for lightening fast serverless deployment of serverless GraphQL functions.

Excited by the chance to become "Employee of the Month", you approach your boss and share with him the big news by saying:

>"I was able to apply dynamic URL transformations to our images using a URL API and extend its functionality to support real-time translations of text overlay!"

Seemingly confused, your manager looks at your hands and responds:

>"You forgot my coffee?"

*[Cloudinary](https://cloudinary.com) and [8base](https://8base.com) both do A LOT more than what is in this post. I highly recommend you check them out!*