How I used DynamoDB as a long-term cache layer for AppSync
By: Benoît Bouré
Tags: Dynamodb, GraphQL, Aws Dynamodb, Appsync, Aws Appsync

Let me show you how I implemented this with a simplified example. In this demo, we will build a GraphQL API that fetches Wikipedia articles. Because we don't want to spam Wikipedia's servers and because articles don't change that much very often, they should be cached for a month before we have to hit Wikipedia again and get the updated versions.

To build that, we will use the Serverless Framework and the AppSync plugin. I will not go into details on how the plugin works. For more information, please refer to the documentation on the repository or this series of articles.

I will explain the most important parts only, but you can find the full code of this example on GitHub.

Let's start with the serverless.yml.

mappingTemplates

This is where we define our resolver. As I explained earlier this is going to be a PIPELINE resolver with three consecutive functions: fetchFromCache, fetchWikipedia and saveToCache.

dataSources

Here we define our two data sources:

an HTTP endpoint which points to the Wikipedia API in English

a DynamoDB table

functionConfigurations

And here, we declare the 3 pipeline functions that we use in the data source we created earlier.

WikipediaTable resource

Finally, we declare our DynamoDB table. It will have a HASH key, which will be the title of the article. We also set a TimeToLiveSpecification on the expires_at attribute.

DynamoDB Time to Live (TTL) is a feature that allows us to define a per-record timestamp when the record is no longer needed. When the timestamp is reached, the record is deleted. We will use that in order to auto-expire the cache.

Now, we need to define our mapping templates. There are a few of them. Let's go through them in the order they will be executed.

Let's begin with the "before" pipeline request mapping template.

## Query.wikipedia.request.vtl

$util.qr($ctx.stash.put("title", $ctx.args.title))

{}

Here, we simply put the title argument coming from the request into the stash (see the schema definition). We will use it later in the pipeline. We also return an empty Map (because mapping templates cannot be empty).

At this point, the first function in the pipeline will be called: fetchFromCache

## fetchFromCache.request.vtl

{

"version": "2018-05-29",

"operation": "GetItem",

"key": {

"title": $util.dynamodb.toStringJson("${ctx.stash.title}")

}

}

Here, we execute a GetItem operation on our DynamoDB table using the title of the article as the key. Let's see what is in the response template:

#if($ctx.error)

$util.error($ctx.error.message, $ctx.error.type)

#end #if($ctx.result)

$util.qr($ctx.stash.put("result", $ctx.result.content))

#end

{}

First, check for any error, and stop the process if we find any. Then, if we have a result (it means that we have a hit!), we stick it into the stash. You will find out why later.

Now, at this point, if we have a hit, we want to stop the execution and return the value to the user. It turns out that AppSync has a neat solution for that: the return directive.

The #return directive comes handy if you need to return prematurely from any mapping template. #return is analogous to the return keyword in programming languages, as it will return from the closest scoped block of logic. What this means is using #return inside a resolver mapping template will return from the resolver. Additionally, using #return from a function mapping template will return from the function and will continue the execution to either the next function in the pipeline or the resolver response mapping template.

The important part to notice here is that, when used in a pipeline function, #return will continue to the next function in the pipeline. But we don't want the next function to be executed, right? Well, it turns out that if you call #return in a request mapping, it will skip to the next function without executing the current one.

This is why we previously kept the result into the stash. We will use it in the following two functions' request mappings to determine if there was a hit, and skip to the next one directly, in cascade. See our fetchWikipedia request template:

## fetchWikipedia.request.vtl

## Bypass this function if result is present in the stash

#if($ctx.stash.result)

#return($ctx.stash.result)

#end

{

"version": "2018-05-29",

"method": "GET",

"params": {

"query": {

"action": "query",

"format": "json",

"prop": "extracts",

"exintro": "true",

"titles": "${ctx.stash.title}",

"explaintext": "true",

"exsentences": 10

}

},

"resourcePath": "/w/api.php"

}

If we have a result in the stash, we call #return prematurely and continue to the next function. Otherwise, the function would get executed, the API endpoint would be called and the response template too, where we extract the data we need. Here it is:

## fetchWikipedia.response.vtl

#if($ctx.result.statusCode == 200)

#set($body = $utils.parseJson($ctx.result.body))

#foreach ($page in $body.query.pages.entrySet())

#if ($page.value.title == $ctx.args.title)

#return($page.value.extract)

#end

#end

$utils.error("Article not found", "NotFound")

#else

$utils.error($ctx.result.statusCode, "Error")

#end

All right, we are almost there. There is just one last function to define.

Remember, if we return early within any function, the return directive will skip to the next function. So, here again, we need to check if we have a result in the stash and return early one more time. Otherwise, this is where we save the result from the previous function into DynamoDB. We also set an expiry timestamp for 30 days in the future:

## saveToCache.request.vtl

## Bypass this function if result is present in the stash

#if($ctx.stash.result)

#return($ctx.stash.result)

#end

#set($expires_at = $util.time.nowEpochSeconds() + 3600 * 24 * 30)

{

"version" : "2018-05-29",

"operation" : "PutItem",

"key" : {

"title" : $util.dynamodb.toStringJson(${ctx.stash.title})

},

"attributeValues": {

"expires_at": $util.dynamodb.toNumberJson($expires_at),

"content": $util.dynamodb.toStringJson($ctx.prev.result)

}

}

And we return the result.

## saveToCache.response.vtl

#if($ctx.error)

$util.error($ctx.error.message, $ctx.error.type)

#end

$utils.toJson($ctx.prev.result)

Finally, our "after" pipeline just forwards the result to the resolver

## Query.wikipedia.response.vtl

$util.toJson($ctx.result)

And we are done!