Latest source has bug in pipelines for short inputs (regardless of padding)
Author: jusjosgra
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 3.0.2
- Platform: Darwin-19.2.0-x86_64-i386-64bit
- Python version: 3.6.8
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@mfuntowicz @LysandreJik 


## Information

Model I am using (Bert, XLNet ...): bert-large-uncased-whole-word-masking-finetuned-squad

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
Run the latest source (do not get the error from pip)

Steps to reproduce the behavior:

1. install master from source
2. run question answering pipeline
3. Use a document shorter than max_model_length (with or without a padding_strategy)

Traceback:
```
Traceback (most recent call last):
....
    output = pipeline(question=question, context=doc, max_seq_len=512, max_answer_len=50)
  File "/Users/justingrace/Google Drive/healx_root/code/transformers/src/transformers/pipelines.py", line 1673, in __call__
    fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
  File "/Users/justingrace/Google Drive/healx_root/code/transformers/src/transformers/pipelines.py", line 1673, in <dictcomp>
    fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
ValueError: expected sequence of length 512 at dim 1 (got 481)
```

## Expected behavior

the module should not have an error for the non-padded tensors stored in the squad feature set. These are expected to not conform to the max_model_length and need to be handled alternatively -- for instance I am not sure these objects are required at all so they may be removed and masks can be used on the padded tensors.


Comments:
Hey @jusjosgra, 

Can you post a complete code snippet to produce the error. I'm using `master` and running this code does not give me any errors:

```python
from transformers import pipeline
qa = pipeline("question-answering", model="bert-large-uncased-whole-word-masking-finetuned-squad", tokenizer="bert-large-uncased-whole-word-masking-finetuned-squad")
output = qa(question=question, context=context, max_seq_len=512, max_answer_len=50)
```

Do you use batches as an input in your example? 

Using @patrickvonplaten code I am experiencing the same problem here. Is truncation & padding of sqad examples not working correctly?
In my case, I'm using batches as an input, therefore I introduce a list of questions (the same repeated n times) and n contexts for the model to search the answer in. 

@alexvaca0 are you running on a pypi version or on the current `master` branch?

@LysandreJik on the current master branch

Okay, pinging @mfuntowicz to see if he has any insights.

I'm starting to suspect this has something to do with certain tokens... Look, this is one of the texts that the pipeline tokenizes to 485 instead of 512:

**For a city containing multiple hospitals such as NYC, we defined a 102 proximity metric in this study as population normalized number of hospital beds within Prior to analysis of potential predictors, we considered multiple base regression models. 110 Given the significant spatial correlation in the present case data as evidenced by the 111 Moran Index, I (176) = 0.481, p < 0.0005 [24] , we explored potential regression models 112 both with and without spatial effects. We compared four base models (no predictors): 113 1) a Poisson model with random intercept; 2) a Poisson Besag-York-Mollié (BYM) by Eq 1:
where υ i has an intrinsic conditional autoregressive (ICAR) structure [27] . We used the 119 reparameterization of the BYM model proposed by Riebler et al. [28] , known as the 120 BYM2 model and shown in Eq 2:
where τ γ is the overall precision hyperparameter, ϕ ∈ [0, 1] is the mixing hyperparameter 122 representing the proportional division of variance between the spatial and nonspatial 123 effects, υ * is the spatial (ICAR) effect with a scaling factor such that Var (υ * ) ≈ 1, and 124 ν * is the nonspatial random-effect with ν * ∼ N (0, 1). Penalized complexity (PC) priors 125 are applied to hyperparameters τ γ and ϕ (compared to log-gamma priors in the random 126 intercept model) [29] . All four models used ZCTA population as the exposure and a 127 log-link function. We selected the model with the lowest Deviance Information Criterion 128 (DIC) [30] , representing the best trade-off between model fit and complexity. Characteristics for the four base models examined, including hyperparameters, are 130 shown in 
i , scaled nonspatial random-effect; υ * i , scaled spatial random-effect with intrinsic conditional autoregressive structure; τ ν , precision for nonspatial random effect, log-Gamma prior; τ γ , overall precision, penalized complexity (PC) prior; ϕ, mixing parameter, PC prior; n, overdispersion parameter, PC Gamma prior. Multiple regression models were built using a method adjusted from Nikolopoulos et 136 al.**

I don't know if it's a coincidence that it has some mathematical signs that the tokenizer might not be processing correctly...

Facing the same issue when dealing with multiple examples and padding strategy="longest". I think it might have to do with the fact that the features are created per example and hence the padding is done per example. The padding code does not get to see all examples to find the longest example.

https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py#L1753-L1764

Same issue with `squad.py`
https://github.com/huggingface/transformers/blob/master/src/transformers/data/processors/squad.py#L354-L362

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.