Stop executors even when server was never started

Ported from master branch revision 5267b79c42f9727eb23f0064de2c469b8844fbd9