How to isolate your Rails blobs in subfolders
Tags: rails, activestorage, heroku
Published: 2021-06-11 04:20:04

Ruby on Rails has a very nice abstraction for storing user-provided assets and images to any cloud (AWS S3, Google, Azure, local files etc) called [Active Storage](https://guides.rubyonrails.org/active_storage_overview.html). If you are spinning up new deployments of your application with each branch/pull request, you will want the uploaded assets to go nicely into their own subfolders. Much easier to clean them up later. Unfortunately this is not possible out of the box.

Fortunately, with Ruby, we can fix anything.

Firstly, we need a solution that does not alienate all existing blobs. We only want to put new blobs into subfolders. Existing blobs can stay exactly where they are.

I've deployed two demonstration apps that show the internal `key` to demonstrate that the images are stored in different subfolders:
* https://subfolder-demo-1111.herokuapp.com/
* https://subfolder-demo-2222.herokuapp.com/

The new files in each app are now being stored in subfolders, whilst legacy files are kept in their original file in the root of the bucket:

```
$ aws s3 ls s3://mybucket/
                           PRE subfolder-demo-1111/
                           PRE subfolder-demo-2222/
2021-06-11 12:08:16    1579488 g7643rle6b6bgn38v16734062wqw
2021-06-11 13:50:33    5034683 ki79m2n0p5ib3hg8l6bf1z7yrclt
```

In the sample solution below, I'm going to prefix all uploaded blobs with the name of the deployment. On Heroku this is available at runtime from the environment variable `$HEROKU_APP_NAME` after you turn on [dyno metadata](https://devcenter.heroku.com/articles/dyno-metadata).

Each user-provided file is stored in your blobstore and registered in your database with an `ActiveStorage::Blob`. The `ActiveStorage::Blob#key` value guarantees that two files uploaded with the same name will be stored with unique file names.

If we change `#key` to have different prefix for each deployment then our blobs will be stored in isolated subfolders.

To feel good about myself as a professional, let's [write a test first](https://github.com/mocra/subfolder-rails-activestorage-blobs/blob/develop/spec/lib/activestorage_blob_spec.rb).

```ruby
# spec/lib/activestorage_blob_spec.rb
require "rails_helper"

RSpec.describe ActiveStorage::Blob do
  let(:blob) { ActiveStorage::Blob.create_and_upload! io: StringIO.new("This is a test file"), filename: "test.txt" }
  let(:key) { blob.key }

  it "keys have no special prefix by default" do
    expect(ENV).to receive(:[]).with("HEROKU_APP_NAME").and_return(nil)
    # default key looks like "nk2gxeujmuoldqr6o8ng6gov9g5c"
    expect(key).to match(%r{\w{28}})
  end

  it "keys have no special prefix by default" do
    expect(ENV).to receive(:[]).with("HEROKU_APP_NAME").and_return("rcrdcp-pr-123").twice
    # expect like "rcrdcp-pr-123/nk2gxeujmuoldqr6o8ng6gov9g5c"
    expect(key).to match(%r{rcrdcp-pr-123/\w{28}})
  end
end
```

If the application is not running on Heroku, then keep using the default `#key` &mdash; a 28-character long random string.

If the app is running on Heroku, then put the app name at the front of the string, e.g. `myapp-pr-123/nk2gxeujmuoldqr6o8ng6gov9g5c`.

Currently, the `#key` value is created indirectly by `has_secure_token :key` [before the creation](https://github.com/rails/rails/blob/b869a4e3a6e06c8d741ebf48faecfed6afb550a0/activerecord/lib/active_record/secure_token.rb#L32) of an `ActiveStorage::Blob`.

[Our solution will be to re-create](https://github.com/mocra/subfolder-rails-activestorage-blobs/blob/develop/config/initializers/active_storage.rb) this `key` value with an additional `before_create` call.

```ruby
# config/initializers/active_storage.rb

Rails.configuration.to_prepare do
  ActiveStorage::Blob.class_eval do
    before_create :generate_key_with_prefix

    def generate_key_with_prefix
      self.key = if prefix
        File.join prefix, self.class.generate_unique_secure_token
      else
        self.class.generate_unique_secure_token
      end
    end

    def prefix
      ENV["HEROKU_APP_NAME"]
    end
  end
end
```

You can use any environment variable that differentiates your deployments.

On Heroku, to access the `$HEROKU_APP_NAME` variable you need to turn on dyno metadata and deploy the app again.

```
heroku labs:enable runtime-dyno-metadata
```

Thanks to [Arian Faurtosh](https://github.com/arianf) for the starting point for this solution <https://github.com/rails/rails/issues/32790#issuecomment-844095704>. I found I needed to take a different route due to the behaviour of `has_secure_token` which might a recent change.