Workaround bundler issue which could cause an old extension to get used.

See https://github.com/bundler/bundler/issues/2947