How to route IoT messages across multiple regions with AWS IoT Core and Amazon SNS
Tags: aws, iot, sns, sqs
Published: 2023-07-05 22:45:53

## Introduction

In this blog, I explain how to route [AWS IoT Core](https://aws.amazon.com/iot-core/) messages across multiple [AWS regions](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) using [Amazon Simple Notification Service](https://aws.amazon.com/sns/) (Amazon SNS). That is a common pattern when IoT telemetry should be ingested in different geographical regions and shipped to another region for further centralized processing of IoT messages. For instance, when an organization hosts a fleet of devices and collects telemetry in different regions providing lower latency however the collected data across regions should be proceed in one place due to operational or compliance requirements.
 
You will learn how to configure [Rules for AWS IoT](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html) and Amazon SNS for delivering of MQTT messages from one AWS region into [Amazon Simple Queue Service](https://aws.amazon.com/sqs/) (Amazon SQS) in another region. A suggested approach can be used for cross-region routing of MQTT messages to other AWS services supported as a destination by Amazon SNS as well. A combination of Amazon SNS and Amazon SQS allows you to transfer messages from one AWS account to another also. 

## Solution Overview

In this solution, you will first create an Amazon SNS topic in the data ingestion region and an Amazon SQS queue in the data processing region. Then you will grant permissions for the SNS topic to send messages to the SQS queue and subscribe the SQS queue to the SNS topic. Next, you will create an AWS IoT rule to route MQTT messages to the SNS topic. Lastly, you will test the solution by sending IoT messages to an AWS IoT Core topic and polling them from the SQS Queue in another region.

1. Create an Amazon SNS topic called `iot-transfer` in the data ingestion region.
2. In the data processing region:
   1. Create an Amazon SQS queue called `iot-data`.
   2. Gant permissions for sending messages from the Amazon SNS topic `iot-transfer` to the Amazon SQS queue `iot-data`.
   3. Subscribe the Amazon SQS queue `iot-data` to the Amazon SNS topic `iot-transfer`.
3. In the data ingestion region:
   1. Create an Identity and Access Management (IAM) Role called `iot-sns-allow` with an inline policy that allows publishing to the Amazon SNS topic `iot-transfer`.
   2. Create an IoT rule with the IAM role `iot-sns-allow` in the ingestion account to route messages from a MQTT topic called `dt/transferdata` to the Amazon SNS topic `iot-transfer`.
4. Publish messages to the MQTT topic `dt/transferdata`.
5. Verify a delivery of the messages by polling the messages from the Amazon SQS queue `iot-data` in the data processing region.

##Solution Diagram

![Solution Diagram](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4kpo8feyqbmmn1fwasks.png)

##Solution Instructions

###Prerequisites

- One ore more AWS accounts.
- Administrator privileges in the accounts.
- [AWS  Command Line Interface (AWS CLI)](https://aws.amazon.com/cli/).
- Command-line JSON processor [jq](https://github.com/jqlang/jq).

To deploy the solution, I recommend to use [AWS CloudShell](https://aws.amazon.com/cloudshell/). `AWS CloudShell` has preinstalled `AWS CLI` and `jq`. Some AWS CLI commands have `--query` parameter to filter the output of running command but I found out an usage of `jq` utility is more convenient to parse the output.

###Define variables

Define environment variables that will be used for the solution deployment. Change values of the variables if needed.

Set up the data ingestion and data processing regions. I use `us-east-2` as the data ingestion region and `us-west-2` as the data processing region.

```
AWS_INGESTION_REGION=us-east-2
AWS_PROCESSING_REGION=us-west-2
```

Define the `AWS_ACCOUNT_ID` value with an account number of your AWS account. In case if you use different AWS accounts for the ingestion and processing data, set up the variable with the account number of the data ingestion account.

```
AWS_ACCOUNT_ID=111111111111
```

Lastly, define names for a SNS topic, SQS queue, IAM roles for Rules for AWS IoT.

```
AWS_SNS_NAME=iot-transfer
AWS_SQS_NAME=iot-data
AWS_SNS_ROLE_NAME=iot-sns-allow
AWS_REPUBLISH_ROLE_NAME=iot-republish-allow
```

###Create a SQS queue

In the data processing region create a SQS queue with a name defined in the `AWS_SQS_NAME` variable and retrieve a `QueueUrl` and `QueueArn`.

```
SQS_URL=$(aws sqs create-queue \
   --region $AWS_PROCESSING_REGION \
   --queue-name $AWS_SQS_NAME \
   | jq -r .QueueUrl)

SQS_ARN=$(aws sqs get-queue-attributes \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL \
    --attribute-names QueueArn \
    | jq -r .Attributes.QueueArn)
```

Subscribe the SQS queue to the SNS topic and retrieve a `SubscriptionArn`.

```
SUBSCRIPTION_ARN=$(aws sns subscribe \
   --region $AWS_INGESTION_REGION \
   --topic-arn $SNS_TOPIC_ARN \
   --protocol sqs \
   --notification-endpoint $SQS_ARN | jq -r .SubscriptionArn)

```

###Grant permissions to the SNS topic sending messages to SQS

In order the SNS topic can send messages into the SQS Queue created in the previous steps, I need to change a resource policy for the SQS queue and allow the `sqs:SendMessage` action for the SNS `TopicArn`.

Create a file with the resource policy running the following command.

```
cat > policy.json << EOF
{
  "Id": "SNStoSQSCrossRegion",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "sqs:SendMessage",
      "Resource": "${SQS_ARN}",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "${SNS_TOPIC_ARN}"
        }
      }
    }
  ]
}
EOF
```

Create a file for SQS attributes and include the resource policy created on the previous step as a string.

```
cat > sqs-attributes.json << EOF
{
    "Policy": $(jq -Rs . policy.json)
}
EOF
```

Change the resource policy of the SQS queue running the following command.

```
aws sqs set-queue-attributes \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL \
    --attributes file://sqs-attributes.json
```

###Create the IAM role and policy for publishing to SNS

To publish to the SNS topic from AWS IoT Core, create an IAM role with a policy allowing the publishing action.

Create a file named `iot-role-trust-policy.json` with a policy allowing to assume the role by `iot.amazonaws.com` service.

```
cat > iot-role-trust-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
          "Effect": "Allow",
          "Principal":
               {
                    "Service": "iot.amazonaws.com"
               },
          "Action": "sts:AssumeRole"
        }
     ]
}
EOF
```

Create an IAM role and retrieve a `RoleArn`

```
IOT_ROLE_ARN=$(aws iam create-role \
    --role-name $AWS_SNS_ROLE_NAME \
    --assume-role-policy-document file://iot-role-trust-policy.json \
     | jq -r .Role.Arn)
```

Create a file with IAM policy allowing the `sns:Publish` action to the SNS topic.

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sns:Publish",
            "Resource": "${SNS_TOPIC_ARN}"
        }
    ]
}
EOF
```

Attach the policy to the IAM role.

```
aws iam put-role-policy \
     --role-name $AWS_SNS_ROLE_NAME \
     --policy-name iot-sns-policy \
     --policy-document file://allow_send_sns.json
```

###Create the IAM role and policy for publishing errors to IoT topic

To republish error messages to another MQTT topic with an AWS IoT rule, you will create a role and attach a needed policy to the role.

Create an AWS IAM role and retrieve a `RoleArn`.

```
IOT_REPUBLISH_ROLE_ARN=$(aws iam create-role \
    --role-name $AWS_REPUBLISH_ROLE_NAME \
    --assume-role-policy-document file://iot-role-trust-policy.json \
     | jq -r .Role.Arn)

```

Create a policy allowing publishing to the `topic/errors` topic and attach the policy to the role created in the previous step.

```
cat > allow_republish.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iot:Publish",
            "Resource": "arn:aws:iot:${AWS_INGESTION_REGION}:${AWS_ACCOUNT_ID}:topic/errors"
        }
    ]
}
EOF
aws iam put-role-policy \
     --role-name $AWS_REPUBLISH_ROLE_NAME \
     --policy-name iot-republish \
     --policy-document file://allow_republish.json

```

###Create an IoT rule in the data ingestion region to evaluate messages and republish errors

Next, create the IoT Rule that will route messages to the SNS topic and republish any messages that encounter an error to a topic named `topic/errors`.

Create a file with the rule definition.

```
cat > ing-rule.json <<EOF
{
    "sql": "SELECT * FROM 'dt/transferdata'" ,
    "description": "Publishing of IoT messages to SNS.",
    "awsIotSqlVersion": "2016-03-23",
    "ruleDisabled": false,
    "actions": [{
        "sns": {
            "roleArn": "${IOT_ROLE_ARN}",
            "targetArn": "${SNS_TOPIC_ARN}",
            "messageFormat": "JSON"
        }
    }], 
    "errorAction": {
            "republish": {
                "roleArn": "${IOT_REPUBLISH_ROLE_ARN}",
                "topic": "error/rules",
                "qos": 0
            }
    }
}
EOF
```

Create the IoT rule

```
aws iot create-topic-rule \
    --region $AWS_INGESTION_REGION \
    --rule-name "Publishing2SNS" \
    --topic-rule-payload file://ing-rule.json
```

###Publish messages in the data ingestion region and verify their delivery to the data processing region

In the data ingestion region run the following command to publish a message into the SNS topic.

```
aws iot-data publish \
    --region $AWS_INGESTION_REGION \
    --topic dt/transferdata \
    --cli-binary-format raw-in-base64-out \
    --payload '{"Default": "data"}'
```

Use Default as a field name in a json document in order to follow the message format for Amazon SNS.

Verify receiving messages in the data processing region.

```
aws sqs receive-message \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL
```

If don’t receive the messages, in the data ingestion region, check the `topic/errors` topic in AWS Console with the [AWS IoT MQTT client](https://docs.aws.amazon.com/iot/latest/developerguide/view-mqtt-messages.html) for error messages related to delivery.

##Cleaning Up

It is good practice to clean up any resources you no longer want to use. Cleaning up AWS resources prevents your account from incurring any further charges.

Delete IoT Rule.

```
aws iot delete-topic-rule \
    --region $AWS_INGESTION_REGION \
    --rule-name "Publishing2SNS"
```

Unsubscribe the SQS queue from the SNS topic.

```
aws sns unsubscribe \
    --region $AWS_INGESTION_REGION \
    --subscription-arn $SUBSCRIPTION_ARN
```

Delete the SNS topic and the SQS queue.

```
aws sns delete-topic \
    --region $AWS_INGESTION_REGION \
    --topic-arn $SNS_TOPIC_ARN
aws sqs delete-queue \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL
```

Delete the IAM roles and policies.

```
aws iam delete-role-policy \
    --role-name $AWS_SNS_ROLE_NAME \
    --policy-name iot-sns-policy
aws iam delete-role \
    --role-name $AWS_SNS_ROLE_NAME
aws iam delete-role-policy \
    --role-name $AWS_REPUBLISH_ROLE_NAME \
    --policy-name iot-republish
aws iam delete-role \
    --role-name $AWS_REPUBLISH_ROLE_NAME

```

##Conclusion

In this blog I showed you how to route AWS IoT messages from one AWS Region to Amazon SQS in another region. The pattern allows use different AWS regions for ingestion and processing data.