IS: Fix buttons not being configured correctly in some cases

Only call stop if we're started.