I like things to be easy
Tags: raku, rakulang, json
Published: 2022-04-14 12:15:28

---
title: I like things to be easy
published: true
description: A possibly useful Raku class for handling JSON data
tags: raku, rakulang, json
//cover_image: https://direct_url_to_image.jpg
---
## The Situation
You've got a *large* JSON file, in my case 21MB, that takes a while to parse. You want to have a web service running that can load this file into memory and server data from it. Nice and simple right? Sure the initial data read might take a few seconds but after that the service can just trundle along.

Here's a simple example using [Cro](https://cro.services/).

```raku
use JSON::Fast;
use Cro::HTTP::Router;
use Cro::HTTP::Server;

constant DATAFILE = 'data.json';
my $data = from-json($data)

my $application = route {
    get -> 'count' {
        content 'text/plain', "{$data.keys.elems}\n";
    }
    get -> 'keys' {
        content 'text/plain', "{$data.keys.join(",")}\n";
    }
    get -> $uid {
        not-found unless $data{$uid}:exists;
        content 'text/plain', "{$data{$uid}.keys.join(",")}\n";
    }
    get -> $uid, $cid {
        not-found unless ($data{$uid}:exists) && ($data{$uid}{$cid}:exists);
        content 'text/plain', "{$data{$uid}{$cid}}\n";
    }
}

my Cro::Service $server = Cro::HTTP::Server.new:
    :host<localhost>, :port<5000>, :$application;

$server.start;
react whenever signal(SIGINT) {
    $server.stop;
    exit;
}

```

(Most of this is cribbed from the Cro docs to be honest.)

`$data` is read at the start and then shared between our threads. (Note it's a two level data structure but that's not that important for the example). The data is immutable and everything is fine.

## The wrinkle

Every once in a while the data file gets updated. When this happens we have a bunch of rules we need to apply :

* it's OK for a request to server stale data for a bit
* request should return in a timely fashion (so no 3 second waits to read our data file allowed)
* the server should always serve response (so we can't just reboot the server).

Now we could fire up a new server and swap them in a proxying system, that wouldn't be too hard. But what if we could hot swap the *data* and keep the server running?

## A solution

It might not be *the* solution, feel free to comment with others but here's what I came up with.

```raku
unit class JSONDataWatcher;

use JSON::Fast;

subset DataFilePath of Str:D where *.IO:e && *.IO.f;

has DataFilePath $!datafile;
has $.data;
has Lock $!lock;

method !update-data {
    my $read;
    try {
        $read = from-json( $!datafile.IO.slurp );
    }
    $!lock.protect( {$!data = $read} ) unless $!;
}

submethod BUILD( DataFilePath:D :$!datafile ) {
    $!lock = Lock.new();
    self!update-data();
    start react {
        whenever $!datafile.IO.watch() {
           self!update-data();
        }
    }
}
```

Here was have a JSONDataWatcher class. You give it a file path and it does a couple of things :

* Parses the JSON file and puts it in it's $.data value
* Sets up a watcher on the file path if the file changes it tries to reload and update the data.
  * If the data parsing fails (which if the file hasn't finished being written could well happen) we ignore it and keep using the old data
  * If it parses OK we lock the data attribute while we update it. This will give us a small blip when doing reads but I figure it's safer.

Then we can update our server so :

```raku
use JSONDataWatcher;
use Cro::HTTP::Router;
use Cro::HTTP::Server;

constant DATAFILE = 'data.json';
my $w = JSONDataWatcher( datafile => DATAFILE );

my $application = route {
    get -> 'count' {
        content 'text/plain', "{$w.data.keys.elems}\n";
    }
    get -> 'keys' {
        content 'text/plain', "{$w.data.keys.join(",")}\n";
    }
    get -> $uid {
        not-found unless $w.data{$uid}:exists;
        content 'text/plain', "{$w.data{$uid}.keys.join(",")}\n";
    }
    get -> $uid, $cid {
        not-found unless ($w.data{$uid}:exists) && ($data{$uid}{$cid}:exists);
        content 'text/plain', "{$w.data{$uid}{$cid}}\n";
    }
}

my Cro::Service $server = Cro::HTTP::Server.new:
    :host<localhost>, :port<5000>, :$application;

$server.start;
react whenever signal(SIGINT) {
    $server.stop;
    exit;
}
```

## Notes

This is all a bit rough and ready, there's some error checking I'd add. But I thought it would be interesting to show what you can do with [Raku](https://raku.org/). Note that apart from the data parsing everything in the `JSONDataWatcher` class is core to the language.

Enjoy and stay safe.