Only notify about a server crash once

Fixes #715