RagGenerator 
Author: MaskXman
Pull request: no

### System Info

import os
from transformers import RagTokenizer, RagTokenForGeneration
from datasets import load_dataset

dataset_path = '/Volumes/WD_BLACK/Pycharm/dataset/downloads'  # 修改为外部硬盘的路径

dataset = load_dataset(dataset_path, "psgs_w100.nq.compressed")

model_name = 'facebook/rag-token-base'
tokenizer = RagTokenizer.from_pretrained(model_name)
model = RagTokenForGeneration.from_pretrained(model_name)

# 定义查询
query = "What is the capital of France?"

# 生成答案
inputs = tokenizer(query, return_tensors='pt')
generated = model.generate(input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_length=20)

# 解码生成的答案
answer = tokenizer.decode(generated[0], skip_special_tokens=True)

print("Answer:", answer)

bug:Traceback (most recent call last):
  File "/Users/maskxman/PycharmProjects/AutoKG/AutoKG/RAG.py", line 18, in <module>
    generated = model.generate(input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_length=20)
  File "/Users/maskxman/anaconda3/envs/camel/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/maskxman/anaconda3/envs/camel/lib/python3.10/site-packages/transformers/models/rag/modeling_rag.py", line 1491, in generate
    assert (context_input_ids.shape[0] % n_docs) == 0, (
AttributeError: 'NoneType' object has no attribute 'shape'

Could you pls help me? Thank you !!!!

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

import os
from transformers import RagTokenizer, RagTokenForGeneration
from datasets import load_dataset

dataset_path = '/Volumes/WD_BLACK/Pycharm/dataset/downloads'  # 修改为外部硬盘的路径

dataset = load_dataset(dataset_path, "psgs_w100.nq.compressed")

model_name = 'facebook/rag-token-base'
tokenizer = RagTokenizer.from_pretrained(model_name)
model = RagTokenForGeneration.from_pretrained(model_name)

# 定义查询
query = "What is the capital of France?"

# 生成答案
inputs = tokenizer(query, return_tensors='pt')
generated = model.generate(input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_length=20)

# 解码生成的答案
answer = tokenizer.decode(generated[0], skip_special_tokens=True)

print("Answer:", answer)


### Expected behavior

/Users/maskxman/anaconda3/envs/camel/bin/python3.10 /Users/maskxman/PycharmProjects/AutoKG/AutoKG/RAG.py
2023-07-29 15:06:31.519470: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Resolving data files: 100%|██████████| 51/51 [00:00<00:00, 229763.16it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'RagTokenizer'. 
The class this function is called from is 'DPRQuestionEncoderTokenizer'.
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'RagTokenizer'. 
The class this function is called from is 'DPRQuestionEncoderTokenizerFast'.
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'RagTokenizer'. 
The class this function is called from is 'BartTokenizer'.
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'RagTokenizer'. 
The class this function is called from is 'BartTokenizerFast'.
Some weights of the model checkpoint at facebook/rag-token-base were not used when initializing RagTokenForGeneration: ['rag.question_encoder.question_encoder.bert_model.pooler.dense.bias', 'rag.question_encoder.question_encoder.bert_model.pooler.dense.weight']
- This IS expected if you are initializing RagTokenForGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RagTokenForGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "/Users/maskxman/PycharmProjects/AutoKG/AutoKG/RAG.py", line 18, in <module>
    generated = model.generate(input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_length=20)
  File "/Users/maskxman/anaconda3/envs/camel/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/maskxman/anaconda3/envs/camel/lib/python3.10/site-packages/transformers/models/rag/modeling_rag.py", line 1491, in generate
    assert (context_input_ids.shape[0] % n_docs) == 0, (
AttributeError: 'NoneType' object has no attribute 'shape'

Process finished with exit code 1


Comments:
Hi @MaskXman, thanks for raising this issue! 

So that we can best help you, could you please: 
* Make sure there is a minimal reproducible code snippet. We don't have access to the dataset used: `/Volumes/WD_BLACK/Pycharm/dataset/downloads'`. Either a dummy dataset can be created in the script or a public dataset on the hub used. 
* Format the code and errors so they're easier to read in markdown code formatting - between a pair of three backticks: ` ``` code goes here ``` `
* Provide information about the running environment: run `transformers-cli env` in the terminal and copy-paste the output

Are you able to run the example [in the docs](https://huggingface.co/docs/transformers/v4.31.0/en/model_doc/rag#transformers.RagSequenceForGeneration.forward.example)? (This will help us pinpoint the issue)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.