Repository: transformers
Issue: Perhaps your features (`videos` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
Labels: bug

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.46.1
- Platform: Linux-5.15.0-125-generic-x86_64-with-glibc2.35
- Python version: 3.10.16
- Huggingface_hub version: 0.27.0
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 4090

### Who can help?

@ArthurZucker 

class BatchEncoding(UserDict):

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


 
 What are the names of some famous actors that started their careers on Broadway? 
 Some famous actors that started their careers on Broad[78/1906]
de:                                                                                                                                                                
1. Hugh Jackman                                                                                                                                                    
2. Meryl Streep                                                                                                                                                    
3. Denzel Washington                                                                                                                                               
4. Julia Roberts                                                                                                                                                   
5. Christopher Walken                                                                                                                                              
6. Anthony Rapp                                                                                                                                                    
7. Audra McDonald                                                                                                                                                  
8. Nathan Lane                                                                                                                                                     
9. Sarah Jessica Parker                                                                                                                                            
10. Lin-Manuel Miranda
                                                                                                                                         
label_ids:                                                                                                                                                         
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 2909, 8376, 16760, 369, 
2774, 652, 26072, 356, 24331, 3024, 28747, 28705, 13, 28740, 28723, 22389, 4299, 1294, 28705, 13, 28750, 28723, 351, 1193, 28714, 4589, 615, 28705, 13, 28770, 2872
3, 4745, 10311, 5924, 28705, 13, 28781, 28723, 19526, 18021, 28705, 13, 28782, 28723, 17561, 9863, 269, 28705, 13, 28784, 28723, 15089, 399, 763, 28705, 13, 28787,
 28723, 14421, 520, 25999, 28705, 13, 28783, 28723, 20514, 19029, 28705, 13, 28774, 28723, 12642, 24062, 19673, 28705, 13, 28740, 28734, 28723, 6678, 28733, 2356, 
3009, 9154, 5904, 2]                                                                                                                                               
labels:                                                                                                                                                            
Some famous actors that started their careers on Broadway include:                                                                                                 
1. Hugh Jackman                                                                                                                                                    
2. Meryl Streep                                                                                                                                                    
3. Denzel Washington                                                                                                                                               
4. Julia Roberts                                                                                                                                                   
5. Christopher Walken                                                                                                                                              
6. Anthony Rapp                                                                                                                                                    
7. Audra McDonald                                                                                                                                                  
8. Nathan Lane                                                                   
9. Sarah Jessica Parker                                                                                                                                            
10. Lin-Manuel Miranda
  
11. 
key: images                       
value: [None, None, None, None]                                                                                                                                    
key: videos             
value: [None, None, None, None]                                                                                                                                    
  0%|                                                                                                                                      | 0/202 [00:00<?, ?it/s]
Traceback (most recent call last):                                                                                                                                 
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 781, in convert_to_tensors
    tensor = as_tensor(value)                                                                                                                                      
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 738, in as_tensor
    return torch.tensor(value)                                                                                                                                     
RuntimeError: Could not infer dtype of NoneType
The above exception was the direct cause of the following exception:                                                                                               
                                                                                                                                                                   
Traceback (most recent call last):                                                                                                                                 
  File "/data/kleinblue/LLaMA-Factory/scripts/cal_ppl.py", line 132, in <module>                                                                                   
    fire.Fire(calculate_ppl)                                                                                                                                       
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/fire/core.py", line 135, in Fire                                                          
    component_trace = _Fire(component, args, parsed_flag_args, context, name)                                                                                      
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire                                                         
    component, remaining_args = _CallAndUpdateTrace(                                                                                                               
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace                                           
    component = fn(*varargs, **kwargs)                                                                                                                             
  File "/data/kleinblue/LLaMA-Factory/scripts/cal_ppl.py", line 110, in calculate_ppl                                                                              
    for batch in tqdm(dataloader):                                                                                                                                 
  File "/home/yxy/.local/lib/python3.10/site-packages/tqdm/std.py", line 1181, in __iter__                                                                         
    for obj in iterable:                                                                                                                                           
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 701, in __next__                                    
    data = self._next_data()                                                                                                                                       
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 757, in _next_data                                  
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration                                                                                           
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch                                      
    return self.collate_fn(data)                                                                                                                                   
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/data/data_collator.py", line 599, in __call__                                
    batch = pad_without_fast_tokenizer_warning(                                                                                                                    
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/data/data_collator.py", line 66, in pad_without_fast_tokenizer_warning       
    padded = tokenizer.pad(*pad_args, **pad_kwargs)                                                                                                                
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3546, in pad                               
    return BatchEncoding(batch_outputs, tensor_type=return_tensors)                                                                                                
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 240, in __init__                           
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)                                                                        
  File "/home/yxy/.conda/envs/llama_factory/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 797, in convert_to_tensors                 
    raise ValueError(
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the 
same length. Perhaps your features (`videos` in this case) have excessive nesting (inputs type `list` where type `int` is expected).

### Expected behavior

i just try to calculate ppl and it's the model is a mistral-7B-0.1，i dont offer keyword like images or videos，so how can i solve it？