.NET and AWS S3 with LocalStack: How to develop with local S3 buckets
Tags: dotnet, aws, webdev, cloud
Published: 2023-02-14 09:19:51

## Introduction

LocalStack is an open-source framework that allows us to emulate the major AWS services locally, making it easier to develop and test cloud applications without incurring the cost and complexity of deploying to a real cloud environment.

In this post, I'll show how to configure it to emulate S3 buckets and how to interact with those buckets from a C# application.

## Running LocalStack

LocalStack can be run as a CLI or using a container. In this post, I'll explain how to run it in a container with `docker run` and `docker-compose`.

> ℹ️ **If you don't have docker or other container runtime installed**, click [here for Docker installation instructions](https://docs.docker.com/engine/install/) or [here for Podman installation instructions](https://podman.io/getting-started/installation).

### Container

To start a container with a LocalStack instance, run the following command:

```bash
docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 -e EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. localstack/localstack:1.3.1
```

Note that it is exposing port `4566` and ports `4510` to `4559` and allowing CORS access from `https://app.localstack.cloud.` (to allow access from the LocalStack dashboard).

> 🚨 **It's recommended to use a specific version instead of `latest`** to avoid problems with new versions updated automatically.

### Docker compose

Starting LocalStack from docker compose is just as easy. Just add the `localstack` service, as below, to a `docker-compose.yaml` file:

```yaml
version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      - DEBUG=${DEBUG-}
      - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR-}
      - DOCKER_HOST=unix:///var/run/docker.sock
      - EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. # Enable access from the dashboard
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"
```

Then run the following command:

```bash
docker-compose up
```

> 🚨 **It's recommended to use a specific version instead of `latest`** to avoid problems with new versions updated automatically.

> ⚠️ I've added the environment variable `EXTRA_CORS_ALLOWED_ORIGINS` with the value `https://app.localstack.cloud.` to allow access from the LocalStack dashboard.

> 💡 The updated docker-compose.yaml file can be found [here](https://github.com/localstack/localstack/blob/master/docker-compose.yml), in the LocalStack repo.

## LocalStack Dashboard

LocalStack has a web-based dashboard that allows us to manage and configure its services and visualize its logs.

Access [https://app.localstack.cloud.](https://app.localstack.cloud.) and it will connect to the LocalStack instance running locally.

It runs in the browser, so there is no need to expose any ports to the internet.

![LocalStack dashboard showing the services statuses](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/images/dashboard.png)

> 🚨 **If the dashboard says "Please start LocalStack to check the System Status"** and the container log shows **Blocked CORS request from forbidden origin https://app.localstack.cloud.**, the `EXTRA_CORS_ALLOWED_ORIGINS` environment variable was not correctly set to `https://app.localstack.cloud.`. See [here](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-simplifying-cloud-testing/#running-localstack).

## Interacting with LocalStack using the AWS CLI

We'll use the AWS CLI to interact with LocalStack. If you don't have the AWS CLI, look [here](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for instructions on how to install it.

### Configuring a profile for LocalStack

The AWS CLI requires credentials when running. LocalStack doesn't validate the credentials by default, so will create a profile with anything as access key and secret key just to make the CLI happy.

1. In a terminal, type `aws configure --profile localstack`;
1. For `AWS Access Key ID [None]:`, type anything;
1. For `AWS Secret Access Key [None]:`, type anything;
1. For `Default region name [None]:`, type the region you prefer (for example, `us-east-1`);
1. For `Default output format [None]:`, type `json`.

### How to create a bucket using the AWS CLI

To create a S3 bucket in LocalStack, we'll use the `aws s3 mb` command (mb is short for Make Bucket).

The command below will create a bucket with the name `local-bucket-name` using the AWS CLI profile we previously created with the name `localstack`. It's important to pass the `--endpoint` parameter or else it will try to create the bucket in AWS.

```bash
aws s3 mb s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack
```

Looking at LocalStack dashboard we can see the bucket was created:

![LocalStack dashboard showing the local buckets](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/images/dashboard-buckets.png)

### How to list a bucket contents using the AWS CLI

To look the contents of a bucket, we can use the `aws s3 ls` command:

```bash
aws s3 ls s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack
```

Or use the LocalStack dashboard:

![LocalStack dashboard showing a bucket content](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/images/dashboard-bucket-files.png)

## Accessing LocalStack from .NET

To access an S3 bucket from LocalStack in .NET we use the same libraries we use to access it in AWS.

In this example, I'll use the `AWSSDK.S3` and the `AWSSDK.Extensions.NETCore.Setup` NuGet packages, both from AWS.

## How does the AmazonS3Client get AWS access data?

When running in AWS, the `AmazonS3Client` will get its access data from the IAM Role attached to the service running it. When running locally, it will get from the AWS CLI profile named `default` or from the settings we pass to it.

In the code below, I'm checking for a configuration section with the name `AWS`, that is not present in the production environment. If it's found, I set the `Region`, `ServiceURL` and `ForcePathStyle` properties of the `AmazonS3Config` and pass it to the creation of the `AmazonS3Client`.

### Program.cs

```csharp
var builder = WebApplication.CreateBuilder(args);

builder.AddAwsS3Service();

var app = builder.Build();
```

### AwsExtensions.cs

```csharp
public static class AwsExtensions
{
    public static void AddAwsS3Service(this WebApplicationBuilder builder)
    {
        if (builder.Configuration.GetSection("AWS") is null)
        {
            builder.Services.AddAWSService<IAmazonS3>();
        }
        else
        {
            builder.Services.AddSingleton<IAmazonS3>(sc =>
            {
                var awsS3Config = new AmazonS3Config
                {
                    RegionEndpoint = RegionEndpoint.GetBySystemName(builder.Configuration["AWS:Region"]),
                    ServiceURL = builder.Configuration["AWS:ServiceURL"],
                    ForcePathStyle = bool.Parse(builder.Configuration["AWS:ForcePathStyle"]!)
                };

                return new AmazonS3Client(awsS3Config);
            });
        }
    }
}
```

The `appsettings.Development.json` has the configurations pointing to the LocalStack instance:

### appsettings.Development.json

```json
{
  "BucketName": "local-bucket-name",

  "AWS": {
    "Region": "us-east-1",
    "ServiceURL": "http://localhost:4566",
    "ForcePathStyle": "true"
  }
}
```

> ⚠️ **The `ForcePathStyle` forces the use** of `https://s3.amazonaws.com/<bucket-name>/<object-key>` styled URLs instead of `https://<bucket-name>.s3.amazonaws.com/<object-key>` URLs.
>
> **The `ForcePathStyle` needs to be set to `true`** for the `AmazonS3Client` to work with LocalStack.

## Upload an image to the S3 Bucket

Using Minimal APIs, I created an endpoint that receives a file and saves it in the S3 bucket.

The code is straight forward:

### Program.cs

```csharp
app.MapPost("/upload", async (IAmazonS3 s3Client, IFormFile file) =>
{
    var bucketName = builder.Configuration["BucketName"]!;

    var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName);

    if (!bucketExists)
    {
        return Results.BadRequest($"Bucket {bucketName} does not exists.");
    }

    using var fileStream = file.OpenReadStream();

    var putObjectRequest = new PutObjectRequest()
    {
        BucketName = bucketName,
        Key = file.FileName,
        InputStream = fileStream
    };

    putObjectRequest.Metadata.Add("Content-Type", file.ContentType);

    var putResult = await s3Client.PutObjectAsync(putObjectRequest);

    return Results.Ok($"File {file.FileName} uploaded to S3 successfully!");
});
```

Now, we can test it from Postman:

![File upload from Postman](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/images/upload-postman.png)

## Get an image from the S3 Bucket

I also created an endpoint that returns the file with the key passed by parameter from the S3 bucket:

```csharp
app.MapGet("/object/{key}", async (IAmazonS3 s3Client, string key) =>
{
    var bucketName = builder.Configuration["BucketName"]!;

    var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName);

    if (!bucketExists)
    {
        return Results.BadRequest($"Bucket {bucketName} does not exists.");
    }

    try
    {
        var getObjectResponse = await s3Client.GetObjectAsync(bucketName,
            key);

        return Results.File(getObjectResponse.ResponseStream,
            getObjectResponse.Headers.ContentType);
    }
    catch (AmazonS3Exception ex) when (ex.ErrorCode.Equals("NotFound", StringComparison.OrdinalIgnoreCase))
    {
        return Results.NotFound();
    }
});
```

Testing from Postman, we can see the image previously uploaded:

![Getting the file from Postman](https://blog.genezini.com/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/images/get-postman.png)

## Full source code

[GitHub repository](https://github.com/dgenezini/UploadImageToS3LocalStack/tree/v1)

## Liked this post?

I post extra content in my personal blog. [Click here](https://blog.genezini.com/) to see.

## Follow me

- [On Feedly](https://feedly.com/i/subscription/feed/https://blog.genezini.com/index.xml)
- [RSS](https://blog.genezini.com/index.xml)
- [Linkedin](https://www.linkedin.com/in/danielgenezini/)
- [Twitter](https://twitter.com/GeneziniDaniel)

## References and Links

- [LocalStack Website](https://localstack.cloud/)
- [LocalStack Feature Coverage](https://docs.localstack.cloud/user-guide/aws/feature-coverage/)
- [LocalStack Dashboard](https://app.localstack.cloud./)
- [awslocal CLI](https://github.com/localstack/awscli-local)