You could for example rescue a generic ConcurrencyBalancer::Error.

I would also use a newline between the constants and the (exception) class definitions to group the logical paragraphs (like proposed by [bbatsov's Ruby Style Guide](https://github.com/bbatsov/ruby-style-guide#source-code-layout) for example).