AWS StepFunctions HTTP Endpoint demystified
Tags: aws, serverless, stepfunctions
Published: 2024-02-14 11:54:54

---
title: AWS StepFunctions HTTP Endpoint demystified
description: Calling an API from AWS StepFunctions has for a long time required us to write a Lambda function to handle that. With the release of HTTP EndPoint there is no need for that anymore. In this post I'll go over the functionality and show how to use it.
cover_image: https://jimmydqv.com/assets/img/post-stepfunctions-http-api/cover-image-dev.png
tags: aws, serverless, stepfunctions
canonical_url: https://jimmydqv.com/stepfunctions-http-api/index.html
published: true
---

I started to play around with [StepFunctions new HTTP Endpoint](https://aws.amazon.com/about-aws/whats-new/2023/11/aws-step-functions-https-endpoints-teststate-api/) state as soon as it was released. I found it to be a very nice addition and it worked really well, I was happy with it. At a recent re:Invent reCap I did a presentation and a demo on it, so I once again dove deep into it. I still like the functionality, not having to write a Lambda function just to call an API is very refreshing. However, there are some parts on it that I feel is just a bit misplaced. In this post I will go over the good, the bad, and the ugly with StepFunctions HTTP Endpoint State.

## Solutions in this post

To demonstrate the HTTP Endpoint state we'll create two applications. The first will pull weather data from an open weather api, for a specific longitude and latitude. It will use the weather data to determine if it's shorts or not weather.

The second solution will use the Slack API to post a message to a Slack channel.

## EventBridge Connection

To call an API using a HTTP Endpoint state a EventBridge Connection is required. This so we don't have to hard code or include any authorization parameters in our StateMachine.

![Image showing the description of EventBridge connections.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/eb-connections-why.png)

The thought behind this is good, but I feel it get a bit strange, why is there not a StepFunction Authorization resource? Why reuse what was created for EventBridge API Destinations? I would prefer if this was part of StepFunctions or was a standard SecretsManager secret.

## EventBridge Connection Secrets

Speaking of secrets. When a EventBridge Connection is created there is automatically a SecretsManager secret created, with a format like this.

![Image showing the secret.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/connection-secret.png)

I understand that the secret might need to be in certain format that the connection understand, but I would have preferred that I supply the secret to the connection and that the connection just populate it with data. I have seen and got several questions from people, that has been using the HTTP Endpoint state, and then discovers this secret, wondering what it is and what created it.

## Create EventBridge Connection

Let's create an EventBridge Connection for our open weather API. To do this, we need to navigate to the EventBridge console. Yes, it feels so strange. In the EventBridge Console there is also no "Connections" in the menu. Instead we have to navigate to "API Destinations" and select the "Connections" tab. Yes it is very well hidden. It feels that connections are not a first class citizen, I understand the placement when it was only for API Destinations. But it's not anymore, make it an option in the menu please!

![Image showing where EventBridge connections are located.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/eb-connections.png)

In our first solution we'll use an Open Weather API from [yr.no](https://developer.yr.no/) a weather service in Norway. Click Create Connection and fill in the details. Supply a name and a description and then we need to set an authentication.

![Image showing the created connection.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/eb-create-yr-connection.png)

Again this is where things get highly opinionated from AWS. We must select one of the three authentication types, it's not possible to create a connection without an authentication. But the API I'm going to use is fully open and doesn't require an authentication. Sure, a fully open API might not be the best of practices, but it does exists. Anyway, let's select API Key and just fill in some bogus values. And for that matter, I think API Key should have been named "Use Header" instead, more on that later.

After the connection is created let's build the first solution.

## Shorts or not

The first solution will call an open Weather API, fetch the current conditions at a location and decide if it's short weather or not. The state machine that will be used looks like this.

![Image showing the state machine for shorts or not.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/stepfunctions-graph-shorts-or-not.png)

Call the API, use a ResultSelector to pick out the current conditions, use a Choice state to determine if it's shorts or not weather.

Create a StepFunction and add an HTTP Endpoint state to the graph, we configure it to use YR API, make sure you read the [documentation](https://api.met.no/weatherapi/locationforecast/2.0/documentation) if you are going to try this. The method will be GET and we use connection created before.

![Image showing the http endpoint state configuration.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr.png)

Now, the API require that we'll supply a longitude and latitude as query parameters, we can't set this directly in the endpoint. Instead scroll down and expand "Advanced parameters" and set the query parameters there.

![Image showing the http endpoint query parameters.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-query-params.png)

The output from this state will be a json object with the headers and the body that the endpoint responds with, Headers and Body will be in separate objects, like this.

``` json
{
  "Headers" : { },
  "ResponseBody" : { }
}
```

The data we get back from the API is an array with time series data, we are only interested in the current conditions, so here we can use a ResultSelector to get only what we need. In the output section add the following as ResultSelector.

``` json
{
  "temperature.$": "$.ResponseBody.properties.timeseries[0].data.instant.details.air_temperature"
}
```

![Image showing the result selector.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-result-selector.png)

Drag in a Choice State and two Pass state. Create a condition for the shorts weather, in the choice state. Set that if temperature is above 10 degrees C it's shorts weather. Let the default state go to Not Shorts Weather.

![Image showing the condition.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-choice.png)

This mean that your state machine should now look like this.

![Image showing the state machine.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-full.png)

Save it and let's try it out. We need to start a new execution and supply the longitude and latitude, I supply the coordinates close to where I live.

![Image showing the state machine run.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-run.png)

After the run we can see that it's apparently not shorts weather in the middle of the winter,

![Image showing the state machine finished.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/shorts-or-not-call-yr-finished.png)

### Deploy it with SAM

This SAM template will create EventBridge connection and the state machine if you prefer to use that.

``` yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Create StepFunction, is it shorts or not weather?

Parameters:
  Application:
    Type: String
    Description: Name of the application
    Default: http-endpoint-demo

Resources:
  YrWeatherApiConnection:
    Type: AWS::Events::Connection
    Properties:
      AuthorizationType: API_KEY
      AuthParameters:
        ApiKeyAuthParameters:
          ApiKeyName: notuse
          ApiKeyValue: notuse
      Description: Call YR open Weather API
      Name: yr-weather-api

  ShortsOrNotStateMachineStandard:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: statemachine/statemachine.asl.yaml
      Tracing:
        Enabled: true
      DefinitionSubstitutions:
        EBConnectionArn: !GetAtt YrWeatherApiConnection.Arn
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - Statement:
            - Effect: Allow
              Action:
                - secretsmanager:GetSecretValue
                - secretsmanager:DescribeSecret
              Resource: !GetAtt YrWeatherApiConnection.SecretArn
        - Statement:
            - Effect: Allow
              Action:
                - events:*
              Resource: !GetAtt YrWeatherApiConnection.Arn
        - Statement:
            - Effect: Allow
              Action:
                - states:InvokeHTTPEndpoint
              Resource: !Sub arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*
      Type: STANDARD

```

State machine ASL:

``` yaml
Comment: Shorts or not?
StartAt: Call Yr Weather API
States:
    Call Yr Weather API:
        Type: Task
        Resource: arn:aws:states:::http:invoke
        Parameters:
            Authentication:
                ConnectionArn: ${EBConnectionArn}
            Method: GET
            ApiEndpoint: https://api.met.no/weatherapi/locationforecast/2.0/compact
            QueryParameters:
                lat.$: $.lat
                lon.$: $.lon
        ResultSelector:
            temperature.$: $.ResponseBody.properties.timeseries[0].data.instant.details.air_temperature
        Next: Choice
    Choice:
        Type: Choice
        Choices:
            - Variable: $.temperature
              NumericGreaterThan: 10
              Next: Shorts Weather
        Default: Not Shorts Weather
    Shorts Weather:
        Type: Pass
        End: true
    Not Shorts Weather:
        Type: Pass
        End: true

```

## Post to Slack

First of all, we need to create and setup an Slack application how to do that can be found in my post [Serverless and event-driven translation bot](https://jimmydqv.com/eventdriven-translation-bot/index.html)

Now we need to create an EventBridge connection for our Slack application / bot. When posting to a channel using the [Slack API](https://api.slack.com/methods/chat.postMessage) we need to send the token in the Authorization header, setting the value to Bearer [TOKEN]. Here it become clear that this Authorization type in EventBridge Connection should be called Header and not API Key, that would have made so much more sense I think.

![Image showing the slack connection.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/slack-connection.png)

Just as with Shorts or Not we can create a StepFunction that use the connection and post to Slack. In the Body we need to supply the text and channel, we supply that as input parameters to the StepFunction invocation.

![Image showing the slack StepFunction.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/slack-post-config.png)

When calling the API from a StepFunction we need to set the Content-Type header, Slack doesn't parse our data and determine if it's json or not. This is done under Advanced Parameters.

![Image showing the slack StepFunction headers.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/slack-post-headers.png)

We can now invoke the StepFunction and supply the parameters needed.

![Image showing the slack StepFunction invocation.](https://jimmydqv.com/assets/img/post-stepfunctions-http-api/slack-post-invocation.png)

### Deploy with SAM

Just as before, this SAM template will create EventBridge connection and the state machine if you prefer to use that.

``` yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Create StepFunction calling Slack API.

Parameters:
  Application:
    Type: String
    Description: Name of the application
    Default: slack-endpoint-demo

Resources:
  SlackApiConnection:
    Type: AWS::Events::Connection
    Properties:
      AuthorizationType: API_KEY
      AuthParameters:
        ApiKeyAuthParameters:
          ApiKeyName: Authorization
          ApiKeyValue: Bearer resolve:secretsmanager:/slack/app/token
      Description: Call Slack API
      Name: slack-api

  SlackSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: Slack Oauth Token Secret
      Name: /slack/app/token

  PostToSlackStateMachineStandard:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: statemachine/statemachine.asl.yaml
      Tracing:
        Enabled: true
      DefinitionSubstitutions:
        EBConnectionArn: !GetAtt SlackApiConnection.Arn
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - Statement:
            - Effect: Allow
              Action:
                - secretsmanager:GetSecretValue
                - secretsmanager:DescribeSecret
              Resource: !GetAtt SlackApiConnection.SecretArn
        - Statement:
            - Effect: Allow
              Action:
                - events:*
              Resource: !GetAtt SlackApiConnection.Arn
        - Statement:
            - Effect: Allow
              Action:
                - states:InvokeHTTPEndpoint
              Resource: !Sub arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*
      Type: STANDARD

```

StateMachine definition

``` yaml
Comment: Post to Slack
StartAt: Post to Slack Channel
States:
    Post to Slack Channel:
        Type: Task
        Resource: arn:aws:states:::http:invoke
        Parameters:
            Authentication:
                ConnectionArn: ${EBConnectionArn}
            Method: POST
            RequestBody:
                channel.$: $.channel
                text.$: $.text
            Headers:
                Content-type: application/json
            ApiEndpoint: https://slack.com/api/chat.postMessage
        End: true
```

## Final Words

In this post I took a look at HTTP Endpoint state in StepFunctions. This functionality, to be able to call APIs directly, is a great feature. I will use this over and over again that is for sure, I love it. However, it doesn't come flawless, there are thing that I find really strange. But, I think this is such a great feature that I'm willing to live with the weirdness, for now!

Don't forget to follow me on [LinkedIn](https://www.linkedin.com/in/dahlqvistjimmy/) and [X](https://x.com/jimmydahlqvist) for more content, and read rest of my [Blogs](https://jimmydqv.com)