The future of rbs collection
Tags: ruby
Published: 2021-09-28 15:48:21

Recently I discussed the future of rbs collection in [ruby-jp](https://ruby-jp.github.io/), the Japanese online Ruby community. The discussion was exciting, so I'd like to share it.

By the way, if you don't know rbs collection, see https://dev.to/pocke/rbs-collection-was-released-4nmm . This article introduces rbs collection with a motivating example.

## TL;DR

I'm planning the following idea to scale rbs collection to grown ruby/gem_rbs_collection.

* rbs collection will migrate to use RubyGems' scoped packages in a few years, like `@types/*`of TypeScript.
* But RubyGems doesn't have the feature, so we need to develop it.

## Background

[@ybiquitous](https://github.com/ybiquitous) asked me did I consider providing ruby/gem_rbs_collection's RBS as a gem, like `@types/**` of TypeScript. I had the idea but I didn't adopt it for the first release of `rbs collection`. And I realized I didn't describe the reason publicly. I summarized the answer to the question in this article.

## Current Implementation

Currently, `rbs collection` downloads RBSs with `git clone` from [ruby/gem_rbs_collection](https://github.com/ruby/gem_rbs_collection/) GitHub repository.

It works well for now, but probably it will introduce many problems in a few years because the repository will grow as [DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped).

* It will be slow!
  * As you know, cloning a large repository is slow.
  * The repository will have many gem's RBSs.
* It will be expensive for GitHub's server.
  * CocoaPods used (or still uses?) shallow clone, and it was too expensive.
      * https://github.com/CocoaPods/CocoaPods/issues/5016 
      * https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 

[Partial clone](https://github.com/ruby/rbs/pull/805) reduces the impact, but it just procrastinates the problems.

## Why I chose `git clone`?

`git clone` is slow, but it is the simplest way.

At first I created [a PoC](https://github.com/pocke/gem_rbs_cli/) with GitHub API v4 (GraphQL) because it could keep less API calls. However it was not a good idea. GitHub API requires an API token, so the user needs to prepare a Personal Access Token to use it for the local environment and CI.
GitHub API v4 has another problem; It truncates large file contents. Therefore I needed combining GitHub API v4 and [another API](https://github.com/pocke/gem_rbs_cli/blob/8608c9411ffc623bf3f8104da11675eb9f607d4f/lib/gem_rbs_cli/github_client.rb#L77-L82).

I think `git clone` is a good compromise for now. It is the same phase of [tsd](https://github.com/DefinitelyTyped/tsd) or [typings](https://github.com/typings/typings) of TypeScript. We needed a simple RBS manager now as you can see from the previous article.

However `git clone` will introduce performance issues. I'll describe a solution idea in the next section.

## Solution Idea

The solution idea is using RubyGems' scoped packages. It is the same way of `@types/*` of TypeScript.

For example, imagine a Gemfile:

```ruby
# Gemfile

source "https://rubygems.org"

# I use the following gems
gem "rails"
gem "sidekiq"
gem "parallel"

# So, I need the following RBSs!
gem "@types/rails"
gem "@types/sidekiq"
gem "@types/parallel"
```

Put the `Gemfile`, then execute `bundle install`. That's all! Steep and so on load RBSs from `@types/*` gems.

It looks useful, but I need to solve many things to realize the idea.

First of all, we need to implement the scoped packages feature to RubyGems! Unfortunately RubyGems doesn't have the feature😭 [@hsbt](https://github.com/hsbt) told [a related issue](https://github.com/rubygems/rubygems.org/issues/2258) to me.

Just off the top of my head I can imagine we need many tasks to implement it, adding organization, adding UI of rubygems.org, the authentication and authorization, updating gemspec/Gemfile.lock/installed package's path, and so on.

We still have another issue; It needs automatically published `@types/*` packages.
It is easier than implementing scoped packages, of cause. I guess we just need to set-up a workflow of GItHub Actions.

Nothing has been decided yet, but I think it is the best way. I'll investigate it.

## Another idea

[@znz](https://github.com/znz) told another idea; Using RubyGems' `source` instead of scoped packages. For example:

```ruby
# Gemfile

source "https://rubygems.org"

# It is the same
gem "rails"
gem "sidekiq"
gem "parallel"

source "https://rbs.rubygems.org" do
  gem "rbs-rails"
  gem "rbs-sidekiq"
  gem "rbs-parallel"
end
```

Realizing the solution is easier than the scoped packages solution. It doesn't need to modify RubyGems, we can realize it with existing assets!

However this solution has problems:

* We need a server to host `rbs.rubygems.org`.
  * I don't want to maintain the server.
* The package name can conflict with a gem from `rubygems.org`.
  * If both `rubygems.org` and `rbs.rubygems.org` have `rbs-rails` gem, user cannot use both gems together.
  * We can solve the problem with more unique prefix like `rbs-types-*`, but I think it is not smart.

I think `source` is not the best way, but maybe we can use the solution as a temporary solution.

## Conclusion

`git clone` will be slow with grown ruby/gem_rbs_collection. Therefore I'm planning to migrate rbs collection to use RubyGems' scoped package, but it has difficulty.

I think scoped package is the best solution to manage third party RBSs in the long term. I work on improving RBS management. 🚀💎