Fix da build

Fixes #2016 and a race condition that showed up when running kibana as a plugin.