text: 
  
  TITLE
  Missing keys when loading weights in TF are not useful
  
  COMMENTS
  2
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## This concerns all TF models 
  
  If one loads weights of a tensorflow model these lines are run:
  https://github.com/huggingface/transformers/blob/9e68d075a4100906509170498480823e7e61874a/src/transformers/modeling_tf_utils.py#L627
  to check which layers are in the model weights file and which layer names of the model are actually loaded.
  
  The problem is that these layer names consists only of the "highest" layer names of a model. *E.g.* for *TFBertForMaskedLM*, these layer names are just:
  "bert" and "mlm",
  
  but a name for each weight as it should be.
  
  See:
  https://github.com/huggingface/transformers/blob/3c6bf8998fb6ca5aca063fed2543b7176883b004/src/transformers/modeling_tf_bert.py#L865
  
  So the missing keys argument for tensorflow will only capture the most "high" level missing weights.
  