Remove counter when inactive