text: 
  
  TITLE
  Tapas tokenization Different from Tensorflow Code
  
  COMMENTS
  9
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  <!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
  
  - `transformers` version: 4.9.1
  ### Who can help
  @LysandreJik @sgugger @NielsRogge
  
  ## Information
  
  Model I am using (Bert, XLNet ...): Tapas
  
  When I am trying to replicate the TAPAS table retrieval results using Huggingface Tapas implementation, I find that [Tapas tokenization in Huggingface](https://github.com/huggingface/transformers/blob/master/src/transformers/models/tapas/tokenization_tapas.py#L1314) is different from the original [Tensorflow code ](https://github.com/google-research/tapas/blob/master/tapas/utils/tf_example_utils.py#L391). The original code first checks whether the table cell is "n/a", "?" or empty. If so, it would return "[EMPTY]" token. The Huggingface code has implemented [the same tokenization](https://github.com/huggingface/transformers/blob/master/src/transformers/models/tapas/tokenization_tapas.py#L370) with the tensorflow code, but it is not used to tokenize the tables. It could be easily fixed by changing all the calls of function `self.tokenize` to `self._tokenize` in the `_tokenize_table` function. After fixing this, I could use the released table retrieval model to replicate their results on NQ dataset with Huggingface Tapas.
  
  