Using Meta-Programming For Internationalization and Localization in Nim
Tags: i18n, l10n, nim, metaprogramming
Published: 2023-03-03 15:23:24

![Nim Programming Language](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ka3r7fa3lqqqjxeckatq.png)

## TL;DR

Check this out: <https://github.com/heinthanth/ni18n>

## What is Nim

The Nim Programming Language is a **Efficient**, **Expressive**, **Elegant** programming language that is transpiled to different backend ( C, C++, ObjC, JavaScript, etc. ). Nim compiler has **Nim VM**, embedded in it, which allow us to execute code, generate code at compile-time.

## The Problem

Managing translations for internationalizing or localizing an app was never easy. For example, We have to worry about:

* **missing translation** for certain locales
* having **typo** in translation name ( key )
* **invalid substitution** or interpolation, etc.

These are some problems or difficulties I found while developing internationalized web apps and things like that.

## [ni18n](https://github.com/heinthanth/ni18n) to the rescue

What if we can validate our translations and their usage at **compile-time**? Here's my approach to solve those problems.

### DSL

I created a custom DSL to write translations. So that I can traverse that DSL to validate translations and generate code for it.

```nim
type
    Locale = enum
        English
        Chinese
        Myanmar

i18nInit Locale, true:
    hello:
        # translations can be string literal
        English = "Hello, $name!"
        Chinese = "你好, $name!"
        Myanmar = "မင်္ဂလာပါ၊ $name ရေ။"
```

With this DSL, I'm forcing myself to:

* **define** a `enum` of supported locales
* write translation for **every** locale
* every translation **signature / method must be the same** for all locale for a given translation name ( key )

### Code Generation

Nim has an awesome feature called `macro`. With this, you can write Nim AST and generate Nim code at compile-time.

Let's say, we want to generate a function that return a string like this:

```nim
proc someFn(): string {.inline.} =
    return "HELLO, WORLD!"
```

Then, we can write Nim AST like this:

```nim
nnkProcDef.newTree(
    ident("someFn")           # function name
    newEmptyNode()            # not interested in here
    newEmptyNode()            # no generic parameter
    nnkFormalParams.newTree(  # normal parameter
        ident("string")       # return type
    ),
    nnkPragma.newTree(        # pragma
        ident("inline")
    ),
    newEmptyNode(),           # reserved slot ( not interested in here )
    newStmtList(              # function body      
        nnkReturnStmt.newTree(
            newLit("HELLO, WORLD!")
        )
    )
)
```

So, with the same idea, we can convert a piece of the following DSL:

```nim
hello:
    English = "Hello"
```

into Nim function by emitting Nim AST like this:

```nim
nnkProcDef.newTree(
    newIdentNode("hello_English"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
        newIdentNode("string"),
        nnkIdentDefs.newTree(
            newIdentNode("args"),
            nnkBracketExpr.newTree(
                newIdentNode("varargs"),
                newIdentNode("string")
            ),
            nnkBracket.newTree()
        )
    ),
    nnkPragma.newTree(
        newIdentNode("inline")
    ),
    newEmptyNode(),
    nnkStmtList.newTree(
        nnkReturnStmt.newTree(
            nnkCall.newTree(
                newIdentNode("format"),
                newLit("Hello"),
                newIdentNode("args")
            )
        )
    )
)
```

Now, we got the following Nim function generated at compile-time.

```nim
proc hello_English(args: varargs[string] = []): string {.inline.} =
    return format("Hello", args)
```

The same strategy goes for Chinese. We have another function generated like this:

```nim
proc hello_Chinese(args: varargs[string] = []): string {.inline.} =
    return format("你好", args)
```

What next? Well, we have to generate a **lookup function** that **matches runtime locale value** and call correct **locale-suffixed** function ( like `hello_Chinese`, `hello_English`, etc. ).

```nim
proc hello(locale: Locale, args: varargs[string] = []): string {.inline.} =
   case locale
   of English:
       return hello_English(args)
   of Chinese:
       return hello_Chinese(args) 
```

Cool, right?

### Validation

To check if we're **missing certain locale**, we can **track the number** of generated function like this:

```nim
# convert Locale enum into HashSet like {"English", "Chinese", "Myanmar"}
var missingLocales = allowedEnums.toHashSet()

for locale in generatedFns.keys:
    # then, remove locale that has generated function from the set
    missingLocales.excl(locale)

# so, locales that are left in missingLocales don't have translation
if missingLocales.len() > 0:
    # now, we know that some locale are missing
```

So, we can tell compiler to stop compiling and show error message like this:

```nim
let missing = missingLocales.toSeq().map(l => escape(l)).join(", ")
error("missing $# translations for $#" % [missing, escape(identToDotNotation(curIdent))], namePair)
```

So, If I write some code like below:

```nim
type
    Locale = enum
        English
        Chinese
        Myanmar

i18nInit Locale, true:
    hello:
        # translations can be string literal
        English = "Hello, $name!"
        Chinese = "你好, $name!"
```

Then, Nim compiler will complain me about translation of `hello` is missing for `Myanmar` locale.

For validating function signature, we can do something like this:

```nim
proc sameSignatureProc(a, b: NimNode): bool {.inline, compileTime.} =
    ## Checks if two proc definitions have the same signature.
    expectKind(a, nnkProcDef)
    expectKind(b, nnkProcDef)
    result = true

    # see above comment for ProcDef structure
    # to check if two procs have the same signature,
    # we need to check nnkGenericParams, nnkFormalParams and nnkPragma.
    # But generic can't be used in Lambda and not supported in ni18n,
    # so we will check just nnkFormalParams and nnkPragma
    if a[3] != b[3] or a[4] != b[4]: return false

proc sameSignatureProcs(fns: varargs[NimNode]): bool {.inline, compileTime.} =
    for i in 1 ..< fns.len(): (if not sameSignatureProc(fns[0], fns[i]): return false)
    result = true

sameSignatureProcs(generatedFns.values.toSeq())
```

So, if I write code like below:

```nim
type
    Locale = enum
        English
        Chinese
        Myanmar

i18nInit Locale, true:
    hello:
        English = proc(name: string): string =
            return "Hello, " & name & "!"
        Chinese = proc(name: bool): string =
            return "Hello, " & name & "!"
        Myanmar = proc(name: int): string =
            return "Hello, " & name & "!"
```

Then, Nim compiler will complain me about procedure ( function ) signature mismatch across locales.

## Conclusion

Phewwww, we're done ... sounds complex, right? But that's pretty simple. I'd recommend you to read this article along with `ni18n` source code and Nim macro docs. It will help you understand a bit more.

The main idea is to:

1. define a DSL
2. traverse a DSL
3. generate a function if we find `=` assign operator
4. if we find `:` call operator, we will recursively traverse the RHS of `:` operator ( go back to step 2 )
5. Keep track of generated functions from step 3.
6. Validate the number and signature of generated functions

That's it! Hope you learn something new or did I blow your mind?

Thanks for reading till the end. I appreciate your curiosity!
Don't forget to check `ni18n` at <https://github.com/heinthanth/ni18n> and please consider giving a star if that's useful for you!