text: 
  
  TITLE
  Continued training of the original BERT models (not to PyTorch)
  
  COMMENTS
  3
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🚀 Feature request
  
  I am looking for continued training of original tiny-bert with my own raw data using masked language modelling. 
  But I want the final model in TF.
  
  ## Motivation
  
  I tried this [LM](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py), but this only works from pytorch to pytorch.
  
  I tried with using original ckpt and from_tf, it always results in a .h5 error 
  
  Please let me know I can explain more OR help in any way.
  
  Basically, we should be able to use tf weights >> masked language modelling >> and have a domain specific pre-trained Tensorflow language model.
  
  
  