Make it possible to configure the grace period before kill

Closes #8