Ensure ID anonymizations don't erase speaker information