AttributeError when running question answering models for result
Author: tanish36
Pull request: no



- `transformers` version: 4.11.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.9.0+cu102 (False)
- Tensorflow version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help

@LysandreJik

## Information

I am using bert-multi-cased-finetuned-xquadv1  for question answering I was trying it with the example from the model repo website (https://huggingface.co/mrm8488/bert-multi-cased-finetuned-xquadv1)for the example given in the website it's working fine but when I tried with custom input I fetched from the crawler it's failing with the AttributeError: 'list' object has no attribute 'tolist'

The problem arises when using:

from transformers import pipeline
qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-multi-cased-finetuned-xquadv1",
    tokenizer="mrm8488/bert-multi-cased-finetuned-xquadv1"
)
qa_pipeline({
    'context': "Yes Bank founder Rana Kapoor has approached the Bombay High Court, challenging a special court's order from August this year that had remanded him in police custody for a week in a multi-crore loan fraud case. Kapoor, who is currently lodged in Taloja Jail, is an accused in the loan fraud case and some related matters being probed by the CBI and Enforcement Directorate. A single bench presided over by Justice S K Shinde on Tuesday posted the plea for further hearing on October 14. In his plea filed through advocate Vijay Agarwal, Kapoor claimed that the special court's order permitting the CBI's request for police custody on August 14 was illegal and in breach of the due process of law. Therefore, his police custody and subsequent judicial custody in the case were all illegal. Kapoor has urged the High Court to quash and set aside the special court's order dated August 14. As per his plea, in August this year, the CBI had moved two applications before the special court, one seeking permission to arrest Kapoor, who was already in judicial custody at the time in another case, and the other, seeking his police custody. While the special court refused to grant permission to the CBI to arrest Kapoor, it granted the central agency's plea for his custody. Kapoor, however, said in his plea that before filing an application for his arrest, the CBI had not followed the process of issuing him a notice under Section 41 of the CrPC for appearance before it. He further said that the CBI had not taken prior sanction as mandated under section 17 A of the Prevention of Corruption Act for prosecuting him. The special court, however, had said in its order at the time that as Kapoor was already in judicial custody in another case and was not a free man the procedure mandated under Section 41 of the CrPC need not have been adhered to as far as issuing a prior notice of appearance was concerned. ADVERTISING It had also said that case records showed that the investigating officer had taken an approval from a managing director of Yes Bank before beginning the proceedings against Kapoor and such a permission was a valid sanction. However, Kapoor in his plea said that the above order was bad in law and sought that it be quashed and set aside. The law mandated that if initial action was not in consonance with legal procedures, then all subsequent actions must be held as illegal, he said, urging the High Court to declare the CBI remand and custody and all subsequent proceedings including the further custody as illegal and void ab-initio. In a separate plea before the High Court, Kapoor's daughter Rakhee Kapoor-Tandon has sought exemption from in-person appearance before a special PMLA court. Rakhee has stated that she is a resident of the United Kingdom and is unable to travel to India owing to restrictions imposed due to the COVID-19 pandemic. According to the CBI, in the present case, Kapoor had obtained a gratification or pecuniary advantage of ₹ 307 crore, and thereby caused Yes Bank a loss of ₹ 1,800 crore by extending credit facilities to Avantha Group, when it was not eligible for the same",
    'question': "Is this person invovled in fraud?"
    
})



## To reproduce

Steps to reproduce the behaviour:

1. Just follow the code snippet given above


![image](https://user-images.githubusercontent.com/30778739/135454415-8ea5f2f2-ca83-49e7-afc9-a72430ba04e7.png)

## Expected behavior

{'answer': 'a loan fraud case', 'end': 250, 'score': 0.552571678161621, 'start': 276}


Comments:
I also have this issue as listed above. The code is similar but instead of having specific model and tokenizer, I have the default.

<img width="633" alt="MyCode" src="https://user-images.githubusercontent.com/70382249/135517907-493203f1-983a-4e9a-a5be-2ac0113a9fc3.PNG">

I’m still a beginner to NLP, so please excuse and let me know if I make any mistakes.

**WHEN IT WORKS FINE**:
//span_idx - int value from 0 to len(p_mask) - 1 (inclusive)

I’m not sure if this is the reason but let me explain what I think it is. Whenever the text is short enough where the text doesn’t need to be converted into a batch of inputs, the issue doesn’t occur. The list inside p_mask is of type <class 'numpy.ndarray'>. This makes sense because when iterating through p_mask, we can call p_mask[span_idx].tolist() to convert it to a python list. 
As you can see with the image below, when I cut the text to a length of 500, it worked perfectly.

<img width="380" alt="CodeWithoutError" src="https://user-images.githubusercontent.com/70382249/135520827-5b86afd8-68f0-42d9-a1da-bd890d747994.PNG">


<img width="413" alt="InfoWithoutError" src="https://user-images.githubusercontent.com/70382249/135519793-c73812f4-ece2-4303-af2f-55b8acbed4ad.PNG">

**WHEN ISSUE OCCURS:**
//span_idx - int value from 0 to len(p_mask) - 1 (inclusive)

The issue occurs when the length of the text is too long and needs to be converted into batches, my example having a length of 6819. As you can see in the image below, the length of p_mask is now 6. I believe this means there are 6 batches. But now the type of each list inside of p_mask is a python list object, not of type <class 'numpy.ndarray'>. So now when we call p_mask[span_idx].tolist(), it would give this error because it doesn't make sense to convert a python list into a python list.

<img width="363" alt="InfoWhenErrors" src="https://user-images.githubusercontent.com/70382249/135520783-87363fb6-3ef6-4e3a-aa08-5ca81d293189.PNG">
<img width="329" alt="CodeWithError" src="https://user-images.githubusercontent.com/70382249/135522691-de4ae7ce-52b4-476a-8b2e-bb02a9841af7.PNG">

**WHERE p_mask IS CREATED:**
Here is the code for p_mask being created.

<img width="592" alt="p_mask" src="https://user-images.githubusercontent.com/70382249/135522345-0be39098-2f5d-4565-b55d-7119266ce9ea.PNG">

I tried changing putting the list comprehension into a numpy array, but it caused another error which I'm not sure how it's caused and is probably past my knowledge. 
I could be completely wrong if this is the cause but I would just like to share my thoughts.

Thank you for raising this error! This indeed happens when the length is too long, great diagnosis @DeepP2667!

Putting @Narsil in cc

So I was doing some more experiments and found that this works fine when I have installed transformer version 4.10.2 and 4.10.3
![image](https://user-images.githubusercontent.com/30778739/135568752-bccbbc8a-5021-46fc-9f09-5b6a0f41502f.png)

@LysandreJik @Narsil  please look into this also
and @DeepP2667 you can use downgrade the transformer version and use it for the time being

I'm not sure who to ping, @LysandreJik @Narsil 

**The file this was in**:  question_answering.py

So I was looking to this issue and I would like to share my thoughts on what’s happening. Please excuse me if this is not a correct approach. 

**CHANING p_mask**:
First off, the AttributeError was there because the lists inside of p_mask were python lists instead of numpy arrays. It can be fixed by converting them to a numpy array. By converting each list to a numpy array, the call “p_mask[span_idx].tolist()” is able to be changed to a python list. Also, the shape of p_mask becomes the same matrix as version 4.10.2.

<img width="614" alt="p_mask_withNPArray" src="https://user-images.githubusercontent.com/70382249/135661422-3b1bdd7e-53ec-4504-983a-e066865869a3.PNG">

**PADDING ERROR:**
But this caused another error. The shape of one of lists, (the last one), inside p_mask had a different shape then the rest. This is because there was no padding. 
I ran the code with the sample text and question. I saw that it went into the preprocessing function. It skipped the if statement “not self.tokenizer.is_fast” and went to the else statement. Here the padding was set to “no padding”, which is why the last list in p_mask had a different shape than the rest. In version 4.10.2, the same function had a padding of 'longest'.

Version 4.10.2 – Has padding = kwargs[“padding”] which is ‘longest’

<img width="490" alt="Version_10 4 2_PaddingLongest" src="https://user-images.githubusercontent.com/70382249/135661444-4b11d163-768a-4b88-bb18-d81f6142e097.PNG">


Latest Version – has padding = padding, where padding is “do_not_pad”

<img width="550" alt="LatestVersion_NoPadding" src="https://user-images.githubusercontent.com/70382249/135661482-ce8b7aed-76b9-40b2-8f15-4ce7cb57bacb.PNG">


**CHANGED PADDING:**
I tested in version 4.10.2 and when I ran my pipeline with the code posted in the earlier comment, in the preprocess function, it went to the else statement. So both the latest and 4.10.2 version go to the else statement with the sample text and question I provided. I saw that the padding in 4.10.2 was set to “longest” while in the latest version was set to “do_not_pad”. So I switched the padding in the latest version to “longest” to match version 4.10.2 and it seemed to fix the problem. I got a score that was the same in both the latest and 4.10.2 when using the same text and question.

Version 4.10.2, original code used.

<img width="556" alt="4 10 2VersionInfo" src="https://user-images.githubusercontent.com/70382249/135661619-6175c00c-c421-4ce0-8b32-ffe9bf1c4c2e.PNG">
 

Latest Version - I changed the padding in the preprocessing to 'longest' instead of 'do_not_pad'.

<img width="566" alt="LatestVersionInfo" src="https://user-images.githubusercontent.com/70382249/135661767-047f5697-70a4-40f9-b968-e64086de77bf.PNG">


<img width="413" alt="paddingSetToLongestLatestVersion" src="https://user-images.githubusercontent.com/70382249/135661775-875d7dab-4d69-41da-b22b-da6e3794e444.PNG">
 
Again this may not be the correct approach, which I apologize if it isn't, since it does seem like I am going back to version 4.10.2. I noticed that the start and end in the results were different but I assumed it was just an updated tokenization method in the newer version. Also I haven't tested it yet on different models instead of the default, which I can gladly do if wanted!

Hi, proposed a PR to fix this.

There seemed to have been slightly specific defaults too in earlier versions (384 max_length, and 128 stride). 
Does anyone here know where that comes from ? The default are somehow changed slightly to support the test models we're running (so it should run more configurations/models than before)