Fast full-text search in Redmine
By: Igor Olemskoi
Tags: Redmine, Ruby on Rails, Elasticsearch, Fulltext Search, Redmine Plugin

Here at Southbridge we widely use Redmine — a project management and issue tracking tool. In order to increase the quality of our services and add important features, we are continuing to make improvements, according to our needs. One of the most important tasks was to speed up the search.

Since the number of tasks in Redmine had grown to several hundred thousand, the processing time of a search query began to take tens of seconds which is far too long for us. Therefore, we decided to implement a full-text search system based on Elasticsearch, which is the topic of this post.

There is a lot of information on the Internet about Elasticsearch, so a full description of it is not necessary here. Instead, let us focus on the deployment itself.

Deployment process

First we found the redmine_elasticsearch plugin, which seemed to suit us completely, but later realised that improvements still needed to be made: this plugin was outdated and had not been supported for a couple of years (according to its changelog, it was only compatible with Redmine 2.5.x).

The ‘redmine_elasticsearch’ plugin has some dependencies. It uses Resque to run indexing background jobs, however, in our installation of Redmine a couple of plugins had already been using Sidekiq for similar purposes. It was, therefore, redundant to install Resque. So, first of all we replaced Resque with Sidekiq.

The ‘tire’ gem is necessary to work with Elasticsearch, but this ‘gem’ is outdated. Moreover, it is increasingly recommended to use elasticsearch-rails instead. Also, the ‘tire’ gem imposed severe restrictions on our version of Elasticsearch. In fact, we only managed to make it work with 1.7.X. However, at this point we decided to postpone replacing the ‘tire’ gem.

Finally, Elasticsearch requires two additional plugins:

It’s pretty straightforward to install them.

Installation

Copy the plugin from GitHub into #{REDMINE_ROOT}/plugins folder. In our installation we used the following command:

git clone https://github.com/centosadmin/redmine_elasticsearch.git /opt/redmine/plugins

Install the required gems:

bundle install

Re-index all documents using the following command:

bundle exec rake redmine_elasticsearch:reindex_all RAILS_ENV=production

Notice: re-indexing may take a long time. In our case it lasted for around 40 minutes.

Start Sidekiq for the queue named ‘index_queue’ :

sidekiq -q index_queue

Restart Redmine.

And that’s all. The plugin is ready to work, and has replaced the standard Redmine search with a full-text one.

Conclusion

To eliminate the hard dependency from the version of Elasticsearch, a lot of work should be done, which includes replacing ‘tire’ with ‘elasticsearch-rails’. However, our solution proved its utility and usefulness. Search times decreased from tens to approximately 0.11–0.16 seconds after the plugin deployment. We believe that this is a great result, which is confirmed by our happy users.

I hope you find this post useful. Please leave your questions, notes and suggestions in the comment section below. Thank you for your attention!

Here is a link to the fork with our modifications: redmine_elasticsearch