Check key exists before use, we expecting it to not exist sometimes