Fix memory cleanup by acctually deleting from the hash