Remove deleted keys from allKeys