Serverless Logging
Tags: aws, logging, serverless, lambda
Published: 2021-07-30 07:52:44

---
title: Serverless Logging
published: true
description: Blazing fast, dynamic logging designed exclusively for serverless.
tags: aws, logging, serverless, lambda
cover_image: https://www.sensedeep.com/images/senselogs-logo.png
canonical_url: https://www.sensedeep.com/blog/posts/senselogs/serverless-logging.html
---

SenseLogs is a simple, flexible, dynamic, blazing fast log library designed exclusively for serverless apps using NodeJS.

While there are many other good logging libraries that claim to be flexible and fast, they were not designed for serverless. They are thus bigger and slower than necessary and don't meet the unique logging needs of serverless.

Serverless apps have special requirements like minimizing cold-start time and being exceptionally efficient due short execution lifespans of most Lambda invocations. Furthermore, the ephemeral nature of serverless makes it difficult to remotely control the volume and type of emitted log messages without redeploying code.

SenseLogs was designed to work in this environment and offers a fast, dynamic logging library exclusively for serverless. SenseLogs offers a flexible API to simply log data with rich context in JSON and then dynamically control which messages are emitted at run-time without redeploying your functions.

## SenseLogs Overview

Here are some of the key features of SenseLogs:

* Extremely fast initialization time to shorten cold-starts.
* Up to 7 times faster than the nearest competitor logger.
* Flexible log channels and filters.
* Dynamic log control to change log filters without redeploying.
* Log sampling to emit increased logs for a percentage of requests.
* Emits logs in JSON with rich context.
* For local debugging, emits in human readable formats.
* Inheriting child log instances for per-module logging.
* Stack capture for uncaught exceptions.
* Easily emit CloudWatch custom metrics using EMF.
* Integrates with SenseDeep developer studio.
* No dependencies.
* Clean, readable small code base (<500 lines).
* Full TypeScript support.


## Quick Tour

Install the library using npm or yarn.

    npm i senselogs

Import the SenseLogs library. If you are not using ES modules or TypeScript, use `require` to import the library.

```javascript
import SenseLogs from 'senselogs'
```

Create and configure a logger.

```javascript
const log = new SenseLogs()
```

Then log with a message:

```javascript
log.info('Simple messsage')
log.error(`Request error for user: ${user.email}`)
```

You can also supply addition data to log via a map of additional context information.

```javascript
log.error('Another log error message with context', {
    requestId: 1234,
    userId: '58b23f29-3f84-43ff-a767-18d83500dbd3'
})
```

This will emit

```javascript
{
    "message": "Another log error message with context",
    "requestId": 1234
    "userId": "58b23f29-3f84-43ff-a767-18d83500dbd3"
}
```

## Log Channels

To implement [observability](https://en.wikipedia.org/wiki/Observability) for your serverless apps, you need to proactively embed logging that captures rich and complete request state. This logging needs to impose little to no overhead and ideally, should be remotely manageable to activate without needing to redeploy code.

SenseLogs achieves this via log channels and log filters. SenseLogs organizes log messages via channels which are names given to classify log message types. You can then select log messages to emit by filtering on channel names.

SenseLogs provides standard channels like: debug, error, fatal, warn and info.

```javascript
log.error('Bad things happen sometimes')
log.error(new Error('Invalid request state'), {requestId})

log.debug('The queue was empty')
log.trace('Database request', {request})
```

You can extend upon this basic set of channels and use your own custom channels via the `emit` API. For example:

```javascript
log.emit('custom-channel', 'My custom channel')
```


## Message Filtering

SenseLogs selects log messages to emit depending on whether the log channel is enabled in the log filter. The log filter is a set of channel names that are enabled for output.

The default log filter will emit messages for the fatal, error, metrics, info and warn, channels. The data, debug and trace channels will be hidden by default.

You can change the filter set at any time to add or remove filters. For example, to enable the output of log messages for the data and debug channels.

```javascript
log.addFilter(['data', 'debug'])
```


## Context Information

It is highly desirable to emit detailed request information in each logging call such as X-Ray IDs, API Gateway request IDs and other critical context information. It is cumbersome to encode this information explicitly in each logging call. It is much better to define once and have it inherited by each logging call.

SenseLogs supports this by providing additional log information via contexts. These can be defined and updated at any point and will be included in the logged data by each logging call.

```javascript
log.addContext({ userId })

log.info('Basic message', {
    event: "Login event",
})
```

This will emit the logged message with the additional contexts:

```javascript
{
    "message": "Basic Message",
    "event": "Login event",
    "userId": "58b23f29-3f84-43ff-a767-18d83500dbd3"
}
```


## Dynamic Logging Control

SenseLogs manages log filtering via environment variables that determine the log filter sets. If you change these environment variables, the next time your Lambda function is invoked, it will be loaded with the new environment variable values. In this manner, you can dynamically and immediately control your logging output without modifying code or redeploying.

SenseLogs keeps three log filter sets:

* The default filter via the LOG_FILTER environment variable.
* The override filter via LOG_OVERRIDE.
* The sample filter via LOG_SAMPLE.

The `default` set defines the base set of log channels that are enabled for output. The `override` set is added to the default set for a limited time duration. The `sample` set is added to the default set for a percentage of log requests.

You can change these environment values via the AWS console or via the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/lambda/update-function-configuration.html).

![Console](https://www.sensedeep.com/images/blog/lambda-env-edit-console.png)

Better still, the [SenseDeep Serverless Studio](https://www.sensedeep.com) provides an integrated, high-level way to manage these filter settings.

![SenseDeep](https://www.sensedeep.com/images/sensedeep/lambda-edit-brief.png)


## CloudWatch Metrics and EMF

AWS CloudWatch can receive custom metrics via log data according to the [Embedded Metric Format](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html) (EMF).

SenseLogs makes it easy to emit your application metrics via the SenseLogs `metrics` log channel. For example:

```javascript
log.metrics('Acme/Rockets', {Launches: 1})
```

Once emitted, you can view and track your metrics in the AWS console or in the SenseLogs dashboard.


## Benchmarks

Because SenseLogs was designed exclusively for serverless, it does not carry unnecessary enterprise baggage and is blazing fast for serverless logging tasks.

Here are the results of benchmarks against the self-claimed fastest logger [Pino](https://github.com/pinojs/pino).

SenseLogs 6.5 times faster than the best alternative.

| Logger | Time | Code Size |
| -------- | :--: | ----------- |
| SenseLogs | 477 ms | 478 lines (commented) |
| Pino | 3,269 ms | 1281 lines (comments stripped) |


## More?

SenseLogs is provided open source (MIT license) from [GitHub SenseLogs](https://github.com/sensedeep/senselogs) or [NPM SenseLogs](https://www.npmjs.com/package/senselogs).

You can read more in the detailed documentation including the full API at:

* [GitHub SenseLogs](https://github.com/sensedeep/senselogs) or [NPM SenseLogs](https://www.npmjs.com/package/senselogs).
* [SenseLogs Discussions](https://github.com/sensedeep/senselogs/discussions)

With [SenseDeep](https://www.sensedeep.com), you can easily manage your SenseLogs configuration and view log data to quickly debug your serverless apps.

### Contact

You can contact me (Michael O'Brien) on Twitter at: [@mobstream](https://twitter.com/mobstream), or [email](mob-pub-18@sensedeep.com) and read my [Blog](https://www.sensedeep.com/blog).

### Links

* [SenseDeep Web Site](https://www.sensedeep.com/)
* [SenseDeep App](https://app.sensedeep.com/)