Bugfix: Do not call stop from non-benchmarks

Signed-off-by: Markus Pargmann <mpargmann@allfex.org>