Fix a small bug with removal for older clients