text: 
  
  TITLE
  RAG: Do we need to pretrained the doc-encoder when using a custom dataset?
  
  COMMENTS
  7
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  Now the RAG consists of a [script](https://github.com/huggingface/transformers/blob/master/examples/rag/use_own_knowledge_dataset.py) where we can use a custom dataset other than the wiki-dataset. 
  
  
  Since, in the fine-tuning phase of the RAG, we do not update the doc-encoder (we update only BART and Question Encoder), what if our custom dataset consists of different distribution compared to the wiki dataset (Ex: medical records)? 
  
  Will it still work?