Fix bug that clears whitelist buffer at wrong time.