# Spider

This is the unsupervised pretrained model discussed in our paper [Learning to Retrieve Passages without Supervision](https://arxiv.org/abs/2112.07708).

## Usage

We used weight sharing for the query encoder and passage encoder, so the same model should be applied for both.

**Note**! We format the passages similar to DPR, i.e. the title and the text are separated by a `[SEP]` token, but token 
type ids are all 0-s. 

An example usage:

```python
from transformers import AutoTokenizer, DPRContextEncoder

tokenizer = AutoTokenizer.from_pretrained("tau/spider")
model = DPRContextEncoder.from_pretrained("tau/spider")

input_dict = tokenizer("title", "text", return_tensors="pt")
del input_dict["token_type_ids"]

outputs = model(**input_dict)
```