Tool for fast deletion and emptying of S3 buckets (versioning supported)
Tags: aws, s3
Published: 2023-06-28 11:41:26

I have released an OSS tool that solves the following problems.

- The button to empty an S3 bucket is present in the S3 console but not in the CLI or SDK.
  - If versioning is off, it can be done with `aws s3 rm --recursive` in CLI, but it's slow.
  - If **versioning is on**, it can't be done in the first place.
- When deleting an S3 bucket, the bucket must be empty to be deleted.
  - It can be done with `aws s3 rb --force` in CLI, but it's slow.
- You want to empty it, but can't delete + re-create the bucket with CDK's `autoDeleteObjects`, etc. because it is dependent on other resources.
- You have to wait a day to try to delete it with S3's lifecycle function.

You can now empty or delete entire buckets **very easily and fast**.

Moreover, you can do **multiple buckets at once**, and also **search buckets interactively!**

---

## What is

The CLI tool **"cls3"** is to **CL**ear Amazon **S3** Buckets(AWS).

It **empties** (so **deletes all objects and versions/delete-markers** in) S3 Buckets **or** **deletes** the buckets **themselves**.

This tool allows you to **search for bucket names** and delete or empty **multiple buckets**.

---

## Source Code

Implemented in Go language. Please see [GitHub](https://github.com/go-to-k/cls3).

---

## Features

### Bucket deletion option

Initially, the tool was intended to "empty the bucket", but since I was going to go through the trouble, I also added an option (`-f|--force`) to **"delete the bucket as well"**.

### Search for bucket names and delete multiple buckets

As described below, you can search for bucket names and delete or empty multiple buckets at once.

### Cross-region

In deleting multiple buckets, you can delete them all at once, even if they are in multiple regions.

### Versioning

**Even if versioning is turned on**, you can empty it just as if it were turned off. Therefore, you can use it **without** being aware of the versioning settings.

### Deletion of old version objects only

The `-o | --oldVersionsOnly` option allows you to **delete only old versions** and all delete-markers **without new versions and a bucket itself**.

**So you can retain the latest version objects only.**

This option cannot be specified with the `-f | --force` option.

### Number of objects that can be deleted

The delete-objects API provided by the CLI and SDK has a limit of **"the number of objects that can be deleted in one command is limited to 1000"**, but **This tool has no limit on the number**.

### Retry on 503 error

When there are tens of thousands of objects, a SlowDown error (503 error) may occur on the S3 api side in rare cases when deleting them all at once using the CLI or SDK.

When this occurs, cls3 responds by adding a mechanism that waits a few seconds and retries automatically several times.

---

## Install

### Homebrew

```sh
brew install go-to-k/tap/cls3
```

### Linux, Darwin (macOS) and Windows

```sh
curl -fsSL https://raw.githubusercontent.com/go-to-k/cls3/main/install.sh | sh
cls3 -h

# To install a specific version of cls3
# e.g. version 0.13.2
curl -fsSL https://raw.githubusercontent.com/go-to-k/cls3/main/install.sh | sh -s "v0.13.2"
cls3 -h
```

### aqua

```sh
aqua g -i go-to-k/cls3
```

### Binaries

- [Releases(GitHub)](https://github.com/go-to-k/cls3/releases)

### Source build (development version)

The Go environment is required.

```sh
git clone https://github.com/go-to-k/cls3.git
cd cls3
make install
```

---

## How to use

```sh
cls3 -b <bucketName> [-b <bucketName>] [-p <profile>] [-r <region>] [-f|--force] [-i|--interactive] [-o|--oldVersionsOnly]
```

### Options

- -b, --bucketName: optional
  - Bucket name
    - Must be specified in **not** interactive mode
    - Otherwise (so in the interactive mode), you **can not** specify this!
  - **Multiple specifications are possible.**
    - `cls3 -b test1 -b test2`
- -p, --profile: optional
  - AWS profile name
- -r, --region: optional(default: `ap-northeast-1`)
  - AWS Region
  - It is not necessary to be aware of this as it can be used **across regions**.
- -f, --force: optional
  - ForceMode (Delete the bucket together)
- -i, --interactive: optional
  - Interactive Mode for buckets selection
- -o, --oldVersionsOnly: optional
  - Delete old version objects only (including all delete-markers)
  - Do not specify the `-f` option if you specify this option.

---

## Interactive Mode

### BucketName Selection

In the interactive mode(`-i` option), you can search bucket names and select buckets.

It is designed to be searchable and deletable **across regions**, so it can be used **without being region-aware**.

It can be **empty**.

```sh
❯ cls3 -i
Filter a keyword of bucket names: test-goto
```

Then you select bucket names in the UI.

```sh
? Select buckets.
  [Use arrows to move, space to select, <right> to all, <left> to none, type to filter]
> [x]  test-goto-bucket-1
  [ ]  test-goto-bucket-2
  [x]  test-goto-bucket-3
```

---

## Finally

I created a CLI tool for fast deletion and emptying of S3 buckets.

You can use it if you like!