Lokalise custom processors: Ruby on Rails
Published: 2022-08-25 14:41:00

In this article, you will learn about a new Lokalise feature called **Custom processor**. The Custom processor app enables you to intercept translation keys uploaded and downloaded on Lokalise, then analyze or transform those as needed.

Today I'll show you how to create a Custom processor using Ruby on Rails framework, set it up on Lokalise, and test it.

_If you'd like to know how to build a Custom processor with Node and Fastify, check out the corresponding tutorial. You can find even more code samples in our DevHub._

## What is a Custom processor?

The Custom processor app enables you to utilize your own server or a third-party service to fetch translation keys imported to and exported from Lokalise, analyze, format, transform, or even remove those as needed.

To better understand the idea, let's see how a regular upload is performed on Lokalise:

1. You choose one or more translation files to upload.
2. Lokalise detects the translation file languages automatically, but you can adjust those as needed.
3. You adjust any additional uploading options and click Upload.
4. Lokalise parses the uploaded data, and extracts keys and translation values.
5. Extracted data is displayed in the Lokalise TMS translation editor.

However, if you have a Custom processor set up, after performing step 4 Lokalise will automatically send the parsed data to your processor. There you can perform any additional actions, such as analyzing the uploaded data, transforming your translations, adding special formatting, removing unnecessary special characters or certain banned words, restructuring translation keys, and so on. The only requirement is that the data returned by your processor must preserve the initial structure.

Now let's briefly cover the download process:

1. You choose file formats to download.
2. You adjust any additional options as needed and click Build and download.
3. Lokalise collects the chosen keys and translations.
4. Lokalise generates translation files based on the download options and zips those into an archive.

So, a Custom processor will intercept the data collected in step 3 and once again it can perform any additional modifications before these data are zipped into an archive.

As you can see, the idea is quite simple, however with this feature you can automate repetitive manual work.

### Custom processor use cases

There are various use cases:

- Translate the uploaded content using your own or  third-party service.
- Perform placeholder replacement.
- Clean up the imported translations by removing unnecessary special characters.
- Restructure or rename the exported translation keys.
- Perform text analysis for your localization workflow needs.
- Apply special formatting to the imported or exported translations.
- Remove or replace banned or undesired words.

Now, let's see how to code a custom processor.

## Creating a Custom processor

In this section, you'll learn how to create a script performing pre-processing and post-processing of your translation data.

- Pre-processing happens when you are uploading translation files. We are going to use a third-party API called funtranslations.com to translate regular English texts into pirate speech.
- Post-processing is performed when you are downloading your translation files back. We'll create a script to remove a banned phrase from all our translations.

### Preparing the app

To get started, let's create a new Rails app as usual:

```
rails new CustomProcessors
```

I will be using Rails 7 but all the code samples should be relevant for Rails 5 and 6 as well.

Create a new controller in the `app/controllers/processors_controller.rb` file:

```ruby
require 'uri'
require 'net/http'
require 'openssl'
require 'json'

class ProcessorsController < ApplicationController
  skip_before_action :verify_authenticity_token, only: %i[preprocess postprocess]
end
```

First, we import all the necessary modules that will be used to perform the API request.

Next, we also have to skip authenticity token verification for preprocessing and postprocessing because these actions will be called from another resource.

Let's also add two routes in the `config/routes.rb` file:

```ruby
post '/preprocess', to: 'processors#preprocess'
post '/postprocess', to: 'processors#postprocess'
```

That's it!

### Performing pre-processing

Now let's discover how to perform data pre-processing. I would like to find all English translations and turn them into funny pirate speech using funtranslations.com. To get started, let's iterate over translation keys:

```ruby
def preprocess
  payload = params

  payload[:collection][:keys].each do |key|
  end
end
```

Next, we have to modify translations for every key, but only if the language ID is equal to 640 (which is English), and respond with a JSON containing the uploaded data:

```ruby
def preprocess
  payload = params

  payload[:collection][:keys].each do |key|
    key[:translations].map! do |trans|
      if trans[:languageId] == 640
        trans[:translation] = translate(trans[:translation])
      end
      trans
    end
  end

  render json: payload
end
```

Let's also add the `translate` private method:

```ruby
private

def translate(initial_text)
  url = URI("https://api.funtranslations.com/translate/pirate.json")
  url.query = URI.encode_www_form({text: initial_text})

  http = Net::HTTP.new(url.host, url.port)

  http.use_ssl = true

  request = Net::HTTP::Get.new(url)

  request["Accept"] = 'application/json'

  response = http.request(request)

  JSON.parse(response.read_body)['contents']['translated']
end
```

There's nothing complex really: we generate a proper URL, add our initial text to the query, send the request, and then get the translated content. Great job!

### Performing post-processing

Post-processing is performed in a very similar way. Let's create a new action which is going to remove a banned phrase from all translations. Specifically, pirates fear the Flying Dutchman so let's get rid of it:

```ruby
def postprocess
  payload = params

  payload[:collection][:keys].each do |key|
    key[:translations].map! do |trans|
      trans[:translation].gsub!(/flying\sdutchman/i, '')
      trans
    end
  end

  render json: payload
end
```

So, we are simply using `gsub!` to remove all occurrences of the given phrase. Once again, make sure to respond with a JSON containing all translations data.

## Setting up a Custom processor

Once you have created a custom processor, it's time to enable it! Therefore, perform the following steps:

- Proceed to [lokalise.com](http://lokalise.com/signup), log in to the system, and open your translation project.
- Proceed to **Apps**, find **Custom processor** in the list, and click on it.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g5y0g8bcibj3blt2hgsd.png)

- Then click **Install** and configure the app by entering a pre-process and a post-process URL.
- If needed, choose a file format to run this processor for.
- When you are ready, click **Save changes**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/602f4mr8aem6pkcx5g2f.jpg)

That's it, great job!

## Testing it out

Now that everything is ready, you can test your new processor. To do that, proceed to the **Upload** page, choose an English translation file, and click **Upload**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5g807mhyng1lp6s91kfr.jpg)

Return to the project editor and observe the results:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a8w85ktxvqev7nbxh2jt.jpg)

To test the post-processing feature, simply add the "Flying Dutchman" phrase to any translation, proceed to the Download page, and click Build and download. You should not see the banned phrase in the resulting translations.

```yaml
en:
  password: "Say th' secret phrase and enter. It's  over there",
  welcome: "Welcome, me bucko!"
```

Congratulations!

## Conclusion

And this concludes our tutorial. As you can see, the Custom processor is a very powerful feature that can be used to build custom workflows, so be sure to check it out. You can find even more code samples in our DevHub. However, if you have any additional questions, please don't hesitate to [drop us a line](https://forms.gle/u5CoQpbe3qdcTPL29).

Thank you for sticking with me today, and happy coding!