only delete the key when exist.

Not sure why different block has different key. #5049 will be the better solution