Faster JetBrains IDEs with shared indexes
Tags: productivity, tooling, performance, devops
Published: 2022-01-02 21:53:11

---
canonical_url: https://coder.com/blog/faster-jetbrains-ides-with-shared-indexes
---

If you develop with IntelliJ IDEA, PyCharm, GoLand, or other JetBrains IDEs,  it’s likely you’ve waited for “indexing” to complete after opening a project. While this may be annoying, it’s necessary for IntelliJ and other heavy-weight IDEs to have features such as code search, highlighting, refactoring, and code completion.

Waiting for an IDE to finish indexing a project might not be a big problem for many workflows. After the first load, indexes are cached and subsequent runs are faster. However, indexing time can be a huge blocker for developers, especially in these cases:

- large projects (monorepos, many dependencies, monolithic applications)

- running old/slow machines (indexing is CPU-intensive)

- ephemeral developer workspaces (containers, remote IDEs)


![edit of xkcd's "compiling" comic: https://xkcd.com/303/](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ksoux60kx7ntmfu4fywr.png)

In this post, we’ll cover how [shared indexes](https://www.jetbrains.com/help/idea/shared-indexes.html) can significantly reduce IDE load times, share some examples, and a one-line command to generate these for your project. (Historically, shared indexes have been difficult to set up)

## First, how indexing works

Indexing works by traversing the project’s codebase to create a “virtual map” of classes, methods, and objects for future lookups. After the index is generated, it is cached on your device for later use.

Indexing a codebase will likely take the longest **the first time you open it on your machine**. When the codebase changes, such as pulling code or switching branches, your indexes will “update,” but significantly faster than the first time.

## Shared indexes ⚡

[Shared indexes](https://www.jetbrains.com/help/idea/shared-indexes.html) make it possible to host pre-generated indexes for others to download, significantly improving loading speeds across your team. These remote indexes work in conjunction with local indexing to ensure your IDE always has up-to-date information on the codebase.

<figure>
  <img src="https://www.datocms-assets.com/19109/1639079914-final61b24be6b9a30400a127b80d760422.gif" alt="Comparison: local vs shared indexes"><figcaption>GIF: Loading the code-server project in WebStorm</figcaption>
</figure>

## Generating shared indexes for your project

JetBrains has a [guide for creating shared indexes](https://www.jetbrains.com/help/idea/shared-indexes.html), but it involves many steps, including downloading custom tooling and uploading indexes to a CDN. It also lacks instructions for automating this process, to generate indexes in CI, for example.

Using a Docker container to generate shared indexes makes it simple to try locally or automate with cron/CI:

```sh
cd your_large_codebase/

# generate shared indexes
docker run -it --rm \
  -v "$(pwd)":/var/project \
  -v "$HOME"/indexes-output:/shared-index \
  -e INDEXES_CDN_URL=https://cdn.myserver.com/project \
	-u "$(id -u):$(id -g)" \
  bencdr/indexer:idea-2021.3
```

After generating indexes, you can upload the output folder to your CDN, or a local server. You can also use shared indexes without a CDN by using a network share or even your local filesystem for testing. Check out my [GitHub repo](https://github.com/bpmct/indexer) for details:

{% github bpmct/jetbrains-indexer no-readme %}

## Benchmarking shared indexes

I tested indexing time for some popular projects on my 2019 MacBook Pro. To benchmark your own projects, `File → Invalidate Caches` in your IDE will allow you to opt in/out of downloading shared indexes to simulate first launching your project.

| Project | Language(s) | Local indexing 🐌 | With shared indexes ⚡ | Improvement % |
| --- | --- | --- | --- | --- |
| [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) | Go | 2m 40s | 22s | 727% |
| [cdr/code-server](https://github.com/cdr/code-server) | Typescript | 2m 30s | 34s | 441% |
| Coder internal monorepo | Go & Typescript | 3m 20s | 32s | 625% |
| [jetbrains/intellij-community](https://github.com/jetbrains-intellij-community) | Java | 6m 30s | 2m 15s | 288% |


These times were averaged across two test runs. Your mileage will vary depending on network speeds, device performance, etc.

## Remote development & shared indexes

Recently, JetBrains released [remote development support](https://www.jetbrains.com/remote-development/), making it simple to develop from powerful, remote workspaces. On-demand workspaces have a lot of benefits, such as faster onboarding and better reproducibility. However, first-time indexing happens much more frequently, since, after all, workspaces are meant to be ephemeral.

Shared indexes work with [Coder](https://coder.com), our remote development platform. Coder supports all JetBrains IDEs locally, or via the web browser. If you don’t want to host a CDN for shared indexes, you can include them in the workspace image, so everything loads in a snap ⚡

If you’d like to learn more about Coder, you can [request a demo](https://coder.com/demo) or [try it for free](https://coder.com/trial).

## References

- [Talk: Indexing, or How We Made Indexes Shared and Fast](https://www.youtube.com/watch?v=xJKff0QUd3c)
- [JetBrains Remote Development](https://www.jetbrains.com/remote-development/)
- [JetBrains docs: Indexing](https://www.jetbrains.com/help/idea/shared-indexes.html)
- [GitHub: idea-shared-index-dockerfile](https://github.com/damintsew/idea-shared-index-dockerfile)