Super-powered newsletter content with Pocket and Netlify Lambda
Tags: jamstack, serverless, javascript, productivity
Published: 2019-06-08 07:32:41

---
title: Super-powered newsletter content with Pocket and Netlify Lambda
published: true
tags: jamstack, serverless, JavaScript, productivity
canonical_url: https://codewithhugo.com/pocket-newsletter-netlify-lambda/
---


> An example Netlify Lambda to fetch all “newsletter” posts from [Pocket](https://getpocket.com).

[Pocket is an application and web service for managing a reading list of articles from the Internet](https://en.wikipedia.org/wiki/Pocket_(service)). It's quite widely used and tightly integrated into the Firefox browser. I find that I use it extensively to save articles (usually about development).

For the [Enterprise Node.js and JavaScript newsletter](https://buttondown.email/hugo) I have a “From the Web” section, which is populated from links from the web. Since most links that end up there are at one point or another stored on Pocket, I built a lambda that fetches posts tagged with “newsletter” from Pocket. I then consume that lambda from a [Hugo](https://gohugo.io) newsletter file generator.

See the lambda code at [src/lambda/newsletter.js](https://github.com/HugoDF/pocket-newsletter-lambda/blob/master/src/lambda/newsletter.js) in the repository [github.com/HugoDF/pocket-newsletter-lambda](https://github.com/HugoDF/pocket-newsletter-lambda).

Run it at [https://pocket-newsletter-lambda.netlify.com/](https://pocket-newsletter-lambda.netlify.com/), or even [deploy your own on Netlify]((https://app.netlify.com/start/deploy?repository=https://github.com/HugoDF/pocket-newsletter-lambda)).

The application is styled using TailwindCSS, you can see a starter project for that at [github.com/HugoDF/netlify-lambda-tailwind-static-starter](https://github.com/HugoDF/netlify-lambda-tailwind-static-starter).

<!--more-->

## Pocket Fetching logic

The bulk of the Pocket-specific logic is the `fetchBookmarks` function, it does the following:

- fetch from Pocket API using consumer key and access token
  - passes `state: 'all'` in order to get both archived and unarchived posts
  - uses `tag: 'newsletter'` to fetch only posts tagged with `newsletter`
  - `detailType: 'complete'` means the API returns more complete data
- convert the response to a flat list of `{ title, url, excerpts, authors }` (all of those fields are strings)
- return the list

See the code (full source at [github.com/HugoDF/pocket-newsletter-lambda](https://github.com/HugoDF/pocket-newsletter-lambda))
```js
async function fetchBookmarks(consumerKey, accessToken) {
  const res = await axios.post('https://getpocket.com/v3/get', {
    consumer_key: consumerKey,
    access_token: accessToken,
    tag: 'newsletter',
    state: 'all',
    detailType: 'complete'
  });
  const {list} = res.data;
  // List is a key-value timestamp->entry map
  const entries = Object.values(list);
  return entries.map(
    ({
      given_title,
      given_url,
      resolved_url,
      resolved_title,
      excerpt,
      authors,
      rest
    }) => ({
      ...rest,
      title: given_title || resolved_title,
      url: given_url || resolved_url,
      excerpt,
      authors: authors
        ? Object.values(authors)
            .map(({name}) => name)
            .filter(Boolean)
            .join(',')
        : ''
    })
  );
}
```

## Netlify Lambda request validation and body-parsing

### HTTP verb and payload presence validation 

The lambda only supports POST with a body, hence:
```js
if (event.httpMethod !== 'POST') {
  return {
    statusCode: 404,
    body: 'Not Found'
  };
}

if (!event.body) {
  return {
    statusCode: 400,
    body: 'Bad Request'
  };
}
```

### Parsing POST request bodies from form submissions and AJAX/JSON requests

We support both URL-encoded form POST requests (done eg. when JS is disabled on the demo page) and JSON requests.

The body arrives either base64 encoded (if using a URL-encoded form body request) or not. This is denoted by the `isBase64Encoded` flag on the `event`.

Parsing a base64-encoded string in Node is done using `Buffer.from(event.body, 'base64').toString('utf-8)`.

To convert the body from URL-encoded form into an object, the following function is used, which works for POSTs with simple fields.
```js
function parseUrlEncoded(urlEncodedString) {
  const keyValuePairs = urlEncodedString.split('&');
  return keyValuePairs.reduce((acc, kvPairString) => {
    const [k, v] = kvPairString.split('=');
    acc[k] = v;
    return acc;
  }, {});
}
```

Here's the functionality in the lambda:

```js
const {
    pocket_consumer_key: pocketConsumerKey,
    pocket_access_token: pocketAccessToken
  } = event.isBase64Encoded
    ? parseUrlEncoded(Buffer.from(event.body, 'base64').toString('utf-8'))
    : JSON.parse(event.body);
```

If the consumer key or access token are missing we send a 400:
```js
if (!pocketConsumerKey || !pocketAccessToken) {
  return {
    statusCode: 400,
    body: 'Bad Request'
  };
}
```

### Sending appropriate responses on failure

We attempt to `fetchBookmarks`, this functionality has been broken down in [“Pocket Fetching logic”](#pocket-fetching-logic).

When the Pocket API fails on a request error we want to send back that response's information to the client. If the failure can't be identified, 500. When `fetchBookmarks` succeeds send a 200 with data.

Thankfully when axios fails, it has a `response` property on the error. This means that our “Proxy back Pocket API errors” use-case and the other 2 cases are easily fulfilled with:
```js
try {
  const bookmarks = await fetchBookmarks(pocketConsumerKey, pocketAccessToken);

  return {
    statusCode: 200,
    body: JSON.stringify(bookmarks)
  };
} catch(e) {
  if (e.response) {
    return {
      statusCode: e.response.statusCode,
      body: `Error while connecting to Pocket API: ${e.response.statusText}`
    }
  }
  return {
    statusCode: 500,
    body: e.message
  }
}
```

## Sample response

See [github.com/HugoDF/pocket-newsletter-lambda#sample-response](https://github.com/HugoDF/pocket-newsletter-lambda#sample-response) or try out the application yourself at [pocket-newsletter-lambda.netlify.com/](https://pocket-newsletter-lambda.netlify.com/).

## A Pocket-driven newsletter in the real-world

On [codewithhugo.com](https://codewithhugo.com), the lambda doesn't read the access token and consumer key from the request. Instead it's a GET endpoints that reads the token and key from environment variables.

It's actually simpler to do that. We set `POCKET_CONSUMER_KEY` and `POCKET_ACCESS_TOKEN` in the Netlify build configuration. Then update the lambda to the following:
```js
const {POCKET_CONSUMER_KEY, POCKET_ACCESS_TOKEN} = process.env;

// keep fetchBookmarks as is

export async function handler(event) {
  if (event.httpMethod !== 'GET') {
    return {
      statusCode: 404,
      body: 'Not Found'
    };
  }

  const bookmarks = await fetchBookmarks(POCKET_CONSUMER_KEY, POCKET_ACCESS_TOKEN);
  return {
    statusCode: 200,
    body: JSON.stringify(bookmarks)
  };
}
```

[codewithhugo.com](https://codewithhugo.com) runs on [Hugo](https://gohugo.io). To generate a new newsletter file, I leverage [Hugo custom archetypes](https://gohugo.io/content-management/archetypes/) (ie. a content type that has a template to generate it).

The archetype looks something like the following:
```go
{{- $title := replace (replaceRE `[0-9]{4}-[0-9]{2}-[0-9]{2}-` "" .Name) "-" " " | title -}}
---
title: {{ $title }} - Code with Hugo
---

{{- $newsletterBookmarks := getJSON "https://codewithhugo.com/.netlify/functions/newsletter" }}
{{ range $newsletterBookmarks }}
[{{ .title }}]({{.url}}) by {{ .authors }}: {{ .excerpt }}
{{ end }}
```

Once the newsletter has been generated and edited, it gets added to [buttondown.email](https://buttondown.email), you can see the result of this approach at the [Enterprise Node.js and JavaScript Archives](https://buttondown.email/hugo/archive).

[<svg xmlns="http://www.w3.org/2000/svg" style="height:12px;width:auto;position:relative;vertical-align:middle;top:-2px;fill:white" viewbox="0 0 32 32"><title>unsplash-logo</title>
<path d="M10 9V0h12v9H10zm12 5h10v18H0V14h10v9h12v-9z"></path></svg>Mr Cup / Fabien Barral](https://unsplash.com/@iammrcup?utm_medium=referral&utm_campaign=photographer-credit&utm_content=creditBadge "Download free do whatever you want high-resolution photos from Mr Cup / Fabien Barral")