Lambda-less outbound HTTP requests on AWS serverless
Tags: aws, serverless, cdk
Published: 2022-01-01 04:14:45

---
title: Lambda-less outbound HTTP requests on AWS serverless
published: true
description: Making outbound HTTP requests can be tricky. Let us leverage AWS services to do that hard work.
tags: aws, serverless, cdk
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/698hfcffhppouq75l3hh.jpeg
---
I would argue that almost all applications make some sort of network requests. Be it to fetch a collection of items or perform a mutation of a resource.

Network as means of transporting data is not exactly 100% reliable. Because of this, engineers employ various strategies to ensure data delivery. The most common of which might be retrying the request itself.

Would it not be nice for us not to write code for all that logic ourselves? Luckily one might leverage various AWS services to do just that – have the service itself worry about retries and data delivery.

Let us explore this topic by looking at a few AWS services and how they enable us to make network requests via configuration rather than code.

> You can find all the code samples used in this blog post in [this GitHub repository](https://github.com/WojciechMatuszewski/lambdaless-outbound-http-requests).

## Using _Amazon EventBridge_ API destinations

The _Amazon EventBridge_ service took the serverless world by storm from its inception. Managed event bus with rich filtering capabilities was a great addition to the vast (some might argue too vast) family of AWS services.

On March 4, 2021, the _Amazon EventBridge_ feature set expanded to [include _API destinations_](https://aws.amazon.com/about-aws/whats-new/2021/03/amazon-eventbridge-introduces-support-for-api-destinations/). This feature allows for a configuration-only way of making HTTP requests!

![EventBridge API destinations](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v1d16nco37kdy4bi2ps3.jpeg)

Other great developers wrote about this topic, so I will not be re-hashing their excellent work. For a much broader overview of this feature, look no further than [this superb blog post](https://medium.com/lego-engineering/amazon-eventbridge-api-destinations-demystified-part-i-23fa70d9a04d) (check out part 2 as well!).

If you want to get your hands dirty, I've published an [example _AWS SAM_ app](https://github.com/WojciechMatuszewski/eb-destinations-example) that utilizes the _API destinations_ and _Amazon Cognito_.

## Using _AWS Step Functions_ and _Amazon API Gateway_

It seems like one cannot write a blog post about low-code architectures on AWS without mentioning _AWS Step Functions_. No wonder! This service lets you integrate with other AWS services by writing declarative, _AWS Step Functions_ specific, [ASL code](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html).

In our case, the most relevant integration is the integration with _Amazon API Gateway_ and the underlying configuration of the _Amazon API Gateway_ service. If we configure those pieces correctly, we will be able to make outbound HTTP requests from the _AWS Step Functions_ state machine with configuration-only code.

![AWS Step Functions Amazon API Gateway integration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qfdaypdhb0nvkvc3h7d2.jpeg)

### Using _Amazon API Gateway_ REST API and _AWS CDK_

Let us walk through the steps required to deploy such integration using _AWS CDK_ as an IaC tool. I will be using an excellent [https://webhook.site/](https://webhook.site/) to simulate an external HTTP endpoint.

The **first step** is to create the _Amazon API Gateway_ REST API.

```ts
import { aws_apigateway } from "aws-cdk-lib";

// class definition ...

const api = new aws_apigateway.RestApi(this, "api");
```

---

The **second step** is to create a _method resource_ along with the integration. The following code snippet creates a `GET` _Amazon API Gateway_ endpoint, which, when invoked, will perform a `POST` request with a static payload.

```ts
import { aws_apigateway } from "aws-cdk-lib";

// class and API definition ...

api.root.addMethod(
  "GET",
  new aws_apigateway.HttpIntegration("YOUR_API_ENDPOINT", {
    httpMethod: "POST",
    options: {
      requestTemplates: {
        "application/json": `{
            "static": "body_payload"
          }`
      },
      requestParameters: {
        "integration.request.header.Content-Type": "'application/json'"
      },
      integrationResponses: [{ statusCode: "200" }]
    },
    // Otherwise the mapping template is ignored.
    // The headers are mapped.
    proxy: false
  }),
  {
    methodResponses: [{ statusCode: "200" }]
  }
);
```

Let us unpack some of the configuration options.

- We need to specify the `integrationResponses` and `methodResponses` parameters due to the `proxy: false` setting.
  Without the `proxy:false` setting, the _Amazon API Gateway_ would ignore what we have specified in the `requestTemplates`.

- The `statusCode` in the `integrationResponse` and `methodResponse` has to match. Otherwise, an error will be thrown at **method invoke time and NOT deployment time**.

- Keep in mind that this is an example. You **should add more `statusCode` mappings for 4xx and 5xx HTTP response cases**.

We have only scratched the surface of the _Amazon API Gateway_ (REST API) 's possibilities regarding configuration. For a deep dive, refer to [this excellent blog post](https://www.alexdebrie.com/posts/api-gateway-elements/) by [Alex DeBrie](https://twitter.com/alexbdebrie).

---

The **third step** is to create an _AWS Step Functions_ state machine with a _Task step_ which will invoke the endpoint we have just created. What follows is a minimal example of doing so.

```ts
import { aws_stepfunctions, aws_stepfunctions_tasks } from "aws-cdk-lib";

// Your api definition...

const stateMachineDefinition =
  new aws_stepfunctions_tasks.CallApiGatewayRestApiEndpoint(
    this,
    "callAPIGWTask",
    {
      api,
      method: aws_stepfunctions_tasks.HttpMethod.GET,
      stageName: "prod"
    }
  );

new aws_stepfunctions.StateMachine(this, "stateMachine", {
  definition: stateMachineDefinition
});
```

When the _state machine_ runs, the _AWS Step Functions_ service will invoke our _Amazon API Gateway_ endpoint. From there, _Amazon API Gateway_ will invoke the external API endpoint according to the configuration we have specified in the previous step.

#### Gotcha with `queryParameters` property

There is a particular way the values in the `queryParameters` of the `CallApiGatewayRestApiEndpoint` construct have to be formatted. I've spent a considerable amount of time figuring this out, so you might find it helpful as well.

Let us assume your external API requires you to provide a `token` _query parameter_. The `token` is fetched dynamically from the _AWS Systems Manager Parameter Store_ in the _state machine_. We need to do two things to forward the `token` from the _state machine_ to the external HTTP endpoint.

**The first step** is to amend the existing definition of the `callAPIGWTask` (the example assumes that the result of fetching the token is present under `APIKey` path).

```diff
const stateMachineDefinition =
  new aws_stepfunctions_tasks.CallApiGatewayRestApiEndpoint(
    this,
    "callAPIGWTask",
    {
      api,
      method: aws_stepfunctions_tasks.HttpMethod.GET,
      stageName: "prod",
+     queryParameters: aws_stepfunctions.TaskInput.fromObject({
+        "token.$": "States.StringToJson(States.Format('[\"{}\"]', $.APIKey))"
+      })
    }
  );
```

The need to use two of the [_AWS Step Functions_ intrinsic functions](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html) stems from how the integration between _AWS Step Functions_ and _Amazon API Gateway_ handles query parameters – the value **has to be a valid [JSON array of string values](https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html)**.

The **second step** is to amend the _method resource_ on the _Amazon API Gateway_ side.

```diff
api.root.addMethod(
  "GET",
  new aws_apigateway.HttpIntegration(
    "YOUR_API_ENDPOINT",
    {
      httpMethod: "POST",
      options: {
        requestTemplates: {
          "application/json": `{
                "static": "body_payload"
              }`
        },
        requestParameters: {
          "integration.request.header.Content-Type": "'application/json'",
+         "integration.request.querystring.token":
+           "method.request.querystring.token"
        },
        integrationResponses: [{ statusCode: "200" }]
      },
      proxy: false
    }
  ),
  {
    methodResponses: [{ statusCode: "200" }],
+   requestParameters: {
+     "method.request.querystring.token": true
+   }
  }
);
```

The above addition tells _Amazon API Gateway_ to forward the `token` query parameter from the request to the method (which will, in turn, forward it to our external API endpoint).

### Using _Amazon API Gateway_ HTTP API and _AWS CDK_

Utilizing the _Amazon API Gateway_ HTTP API for making outbound HTTP requests works on the same basis as the REST API version. The **most significant difference** is the configuration depth of the integration. The HTTP API integration is much more streamlined in terms of ways one might configure it.

Whenever I'm faced with choosing between those two options, I always fall back to [this helpful documentation page](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html) that depicts the differences between those integrations.

To avoid making this blog post overly long, I have decided not to include the configuration steps (very similar to what we have done in the previous section) in this blog post. You could find the implementation [in this example GitHub repository](https://github.com/WojciechMatuszewski/lambdaless-outbound-http-requests).

## Using _Amazon SNS_

This article would not be complete without mentioning the _Amazon Simple Notification Service_. Along with its [filtering](https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html) capabilities, the service allows for dispatching a large number of messages.

One of the medium of the data transports the service handles is HTTP. Though much less sophisticated than the approaches that we have looked at so far (mainly the inability to transform the incoming payload), it still might come in handy in some scenarios.

![SNS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zlwqeo10aq0jkii1dka7.jpeg)

The following is an IaC _AWS CDK_ code that declares an _Amazon SNS_ topic and an HTTP subscription.

```ts
import { aws_sns } from "aws-cdk-lib";

// class definition ...

const topic = new aws_sns.Topic(this, "topic");
new aws_sns.Subscription(this, "subscription", {
  // Keep in mind the filtering capabilities!
  endpoint: "YOUR_API_ENDPOINT",
  protocol: aws_sns.SubscriptionProtocol.HTTPS,
  topic
});
```

The **most crucial** thing to keep in mind is that the `endpoint` has to be [compatible with SNS subscriptions](https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.prepare.html). Do not be surprised when a request is made to the `endpoint` whenever you deploy your architecture – it is the _Amazon SNS_ requesting the subscription confirmation request!

## Using _AWS AppSync_

> I've added this section after feedback from my colleague [Ross Williams](https://twitter.com/williamsross). Thank you!

_AWS AppSync_ with its managed GraphQL offering is an excellent service for building modern APIs. Used by startups and corporations alike, since its inception allowed developers to achieve tremendous feature velocity.

> If you want to learn more about the _AWS AppSync_, I would recommend getting [_AppSync_ Masterclass course](https://appsyncmasterclass.com/).

Service allows us to retrieve data from various [_data sources_](https://docs.aws.amazon.com/appsync/latest/devguide/system-overview-and-architecture.html). In the context of this blog post, the most relevant of them being the _HTTP data source_. The _HTTP data source_ and the accompanying [_HTTP resolver_](https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-http-resolvers.html) will allow us to make HTTP outbound requests without deploying any _AWS Lambda_ functions.

![AppSync HTTP resolver](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pe24kgz58ha6p6i0cpa6.jpeg)

AWS already has a [great tutorial](https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-http-resolvers.html) on how to write and deploy the architecture we are interested in, so I will not be repeating their words. Before diving into implementation, though, **keep in mind that, to my best knowledge, _HTTP resolvers_ cannot specify any kind of resiliency configuration**.

## Closing words

Did I miss how one might perform an outbound HTTP request without using _AWS Lambda_ service? (keeping the scope to "serverless" services). Please let me know. I'm eager to learn more!

For more similar content, consider following me on Twitter - [@wm_matuszewski](https://twitter.com/wm_matuszewski).

Thank you for your time.