RAG - ValueError: Columns ['embeddings'] not in the dataset. Current columns in the dataset: ['title', 'text'] 
Author: deema-A
Pull request: no
Labels: name: bug

### System Info

```shell
Hi, 

INFO:__main__:Step 1 - Create the dataset
WARNING:datasets.builder:Using custom data configuration default-3b4ec65e3c3d818f
Downloading and preparing dataset csv/default to /local/data/daa2182/.cache/huggingface/modules/datasets_modules/datasets/csv/default-3b4ec65e3c3d818f/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519...
Downloading data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 5322.72it/s]
Extracting data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 915.59it/s]
Dataset csv downloaded and prepared to /local/data/daa2182/.cache/huggingface/modules/datasets_modules/datasets/csv/default-3b4ec65e3c3d818f/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519. Subsequent calls will reuse this data.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1262.20ba/s]
Some weights of the model checkpoint at facebook/dpr-ctx_encoder-multiset-base were not used when initializing DPRContextEncoder: ['ctx_encoder.bert_model.pooler.dense.weight', 'ctx_encoder.bert_model.pooler.dense.bias']
- This IS expected if you are initializing DPRContextEncoder from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing DPRContextEncoder from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
/local/data/daa2182/anaconda/lib/python3.9/site-packages/torch/cuda/__init__.py:145: UserWarning: 
NVIDIA RTX A4000 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA RTX A4000 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'DPRQuestionEncoderTokenizer'. 
The class this function is called from is 'DPRContextEncoderTokenizerFast'.
INFO:__main__:Step 2 - Index the dataset
Traceback (most recent call last):
  File "/local/data/daa2182/13MAy/transformers/examples/research_projects/rag/use_own_knowledge_dataset.py", line 209, in <module>
    main(rag_example_args, processing_args, index_hnsw_args)
  File "/local/data/daa2182/13MAy/transformers/examples/research_projects/rag/use_own_knowledge_dataset.py", line 107, in main
    dataset.add_faiss_index("embeddings", custom_index=index)
  File "/local/data/daa2182/anaconda/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 4197, in add_faiss_index
    with self.formatted_as(type="numpy", columns=[column], dtype=dtype):
  File "/local/data/daa2182/anaconda/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/local/data/daa2182/anaconda/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 1809, in formatted_as
    self.set_format(type, columns, output_all_columns, **format_kwargs)
  File "/local/data/daa2182/anaconda/lib/python3.9/site-packages/datasets/fingerprint.py", line 458, in wrapper
    out = func(self, *args, **kwargs)
  File "/local/data/daa2182/anaconda/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 1868, in set_format
    raise ValueError(
ValueError: Columns ['embeddings'] not in the dataset. Current columns in the dataset: ['title', 'text']

I got this message every time I created a new CSV.
@patrickvonplaten
@lhoestq
thanx!
```


### Who can help?

@patrickvonplaten
@lhoestq

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

python examples/research_projects/rag/use_own_knowledge_dataset.py \
    --csv_path path/to/my_csv \
    --output_dir path/to/my_knowledge_dataset \

### Expected behavior

```shell
It should create a new KB
```


Comments:
Hi ! The "embeddings" column is computed at the line:

https://github.com/huggingface/transformers/blob/95b6bef624bd9dfdfcdfdedd86bb2173f7fb4bfe/examples/research_projects/rag/use_own_knowledge_dataset.py#L88-L93

Can you make sure this line is run and check that "embeddings" is in `dataset.column_names` ?

@deema-A,

Also note that we don't officially maintain code under `research_projects`

The csv files you are creating are not in the format expected by the code. 

This is the line in the code that reads the csv file:
`    
    dataset = load_dataset(
        "csv", data_files=[rag_example_args.csv_path], split="train", delimiter="\t", column_names=["title", "text"]
    )`

There is the delimiter="\t"
You can use this to create the csv:

`
import csv
row_list = [
            ["title", "text"],]

with open('my_knowledge_dataset.csv', 'w', newline='') as file:
  writer = csv.writer(file, delimiter='\t')
  writer.writerows(row_list) `

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.