A User Journey: Setting Up the Node Beeline on Lambda
Tags: node, serverless, honeycomb, observability
Published: 2020-11-10 22:50:53

<span style="font-weight: 400;">Nic Wise at</span><a href="https://tend.nz/" target="_blank" rel="noopener noreferrer"> <span style="font-weight: 400;">Tend Health</span></a><span style="font-weight: 400;"> recently wrote a series of blog posts exploring how they moved away from logs and metrics, toward adopting observability with Honeycomb. In that series, he shares lessons learned as they got their NodeJS app instrumented in an AWS environment making use of CloudFront, API Gateway, Lambda, and a few other services.</span>

<span style="font-weight: 400;"><img class="alignnone size-full wp-image-7909" src="https://www.honeycomb.io/wp-content/uploads/2020/10/unnamed-2.png" alt="Tend Health application architecture digraph" width="655" height="423" /></span>

<span style="font-weight: 400;">Tend is a New Zealand-based healthcare platform launching in 2020. With the opportunity of a near-greenfield project to work with, Nic shared his experiences learning Honeycomb that you might find useful.</span>
<h2><span style="font-weight: 400;">Getting started</span></h2>
<span style="font-weight: 400;">Nic used the</span><a href="https://docs.honeycomb.io/getting-data-in/javascript/beeline-nodejs/" target="_blank" rel="noopener noreferrer"> <span style="font-weight: 400;">Node Beeline integration</span></a><span style="font-weight: 400;"> and set up some middleware to get data into Honeycomb from dev, but encountered some snags with his lambda code waiting on API calls to Honeycomb’s ingest service. In order to solve that, he needed to better understand Honeycomb’s data model. He very clearly phrases his take on using events:</span>
<blockquote><span style="font-weight: 400;">You can almost think of an event as a log line - something happened and I want to know about it - but it's also a data structure, so you can add any other context you want to it.</span></blockquote>
<span style="font-weight: 400;">Once Nic updated his mental model of working with structured events, “around 90% of the metrics [he’d previously] created had no purpose anymore.” He changed his existing log lines and metrics to dump that data into the Beeline’s spans as context, deleting a bunch of code in the process.</span>
<h2><span style="font-weight: 400;">Working with traces and AWS Lambda</span></h2>
<span style="font-weight: 400;">Nic also encountered some missing spans because of how his codebase was using `async`/`await`:</span>

<img class="alignnone size-full wp-image-7908" src="https://www.honeycomb.io/wp-content/uploads/2020/10/unnamed-1.png" alt="Honeycomb trace on a graphql-api service with a missing span beneath the root span" width="1000" height="206" />

<span style="font-weight: 400;">It can be tricky to debug missing parts of a trace, but I can promise when you solve it you'll have learned that part of the code like the back of your hand.</span>

<span style="font-weight: 400;">That’s the thing about instrumentation and observability tooling: the point isn’t for the tools to think for you, it’s to help you better reason about your code and your systems. Auto-instrumentation is great for getting started but the most important parts of your code are unique to your app because they’re unique to your business. That means you’ll benefit the most from some adding custom instrumentation on top of auto-instrumentation, and it’s why Honeycomb makes the most sense as a developer tool.</span>

<span style="font-weight: 400;">Nic got advice from </span><a href="https://www.honeycomb.io/blog/spread-the-love-appreciating-our-pollinators-community/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Honeycomb’s Pollinators community Slack</span></a><span style="font-weight: 400;"> on how to fix the issue with Lambda blocking on API calls. This involved dropping down into `Libhoney`, the event handler library used by the Beeline SDK, and redirecting his events to emit to stdout. From there, Cloudwatch Logs and </span><a href="https://github.com/honeycombio/agentless-integrations-for-aws" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Honeycomb’s agent Lambda integration</span></a><span style="font-weight: 400;"> send his data along to Honeycomb asynchronously.</span>

<span style="font-weight: 400;">Once he got that working, Nic encountered a scenario where he was seeing separate traces for an asynchronous call to the Twilio API, which he wanted to combine into a single trace alongside the caller. By updating his code to pass along the trace ID and parent span ID, he was able to combine them:</span>

<img class="alignnone size-full wp-image-7907" src="https://www.honeycomb.io/wp-content/uploads/2020/10/unnamed.png" alt="Honeycomb trace showing an asynchronous call from an SMS onboarding service out to an SMS sender service. The onboarding service call takes 28 milliseconds, and about half a second later the sender service call takes about 1.2 seconds." width="1005" height="175" />

<span style="font-weight: 400;">Nic’s scenario is common among teams using Lambda, but his post clearly explores that with a straightforward example.</span>

<span style="font-weight: 400;">Read Nic’s entire series plus his follow-up post to get the full story along with code samples:</span>
<ul>
 	<li style="font-weight: 400;"><a href="https://fastchicken.co.nz/2020/10/04/honeycomb-1-the-beginning/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Honeycomb 1 - The Beginning</span></a></li>
 	<li style="font-weight: 400;"><a href="https://fastchicken.co.nz/2020/10/04/honeycomb-2-what-are-all-these-new-terms/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Honeycomb 2 - What are all these new terms?</span></a></li>
 	<li style="font-weight: 400;"><a href="https://fastchicken.co.nz/2020/10/04/honeycomb-3-breaking-the-api-dependency/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Honeycomb 3 - Breaking the API dependency</span></a></li>
 	<li style="font-weight: 400;"><a href="https://fastchicken.co.nz/2020/10/12/propagating-the-honeycomb-context-between-lambda-calls/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Propagating the Honeycomb context between Lambda calls</span></a><span style="font-weight: 400;"> </span></li>
</ul>
<span style="font-weight: 400;">Learn more about how metrics work in Honeycomb and how you can eliminate the majority of them by using structured events: download <a href="https://www.honeycomb.io/getting-started-with-honeycomb-metrics/" target="_blank" rel="noopener noreferrer">Getting Started with Honeycomb Metrics</a>.</span>

Join the swarm. Get started with [Honeycomb for free](https://ui.honeycomb.io/signup?&utm_source=Devto&utm_Devto=blog&utm_campaign=referral&utm_keyword=%7Bkeyword%7D&utm_content=a-user-journey-nic-wise).