Remove configuration that no longer exists (lastrun)