The output tensor's data type is not torch.long when the input text is empty.
Author: wangzhen0518
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.1
- Platform: Linux-5.15.0-130-generic-x86_64-with-glibc2.35
- Python version: 3.12.8
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: 1.3.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.10.2 (cpu)
- Jax version: 0.5.0
- JaxLib version: 0.5.0
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA GeForce RTX 3060 Ti

### Who can help?

@ArthurZucker and @itazap

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

The output tensor's data type is not torch.long when the input text is empty.

```python
t = tokenizer('', return_tensors='pt')
print(t['input_ids'].dtype)
# torch.float32
```

### Expected behavior

```python
t = tokenizer('', return_tensors='pt')
print(t['input_ids'].dtype)
# torch.int64
```

Comments:
Hi @wangzhen0518, does this happen with all tokenizer classes, or just a specific one you tested?

I have only tested it on the tokenizer of the QWen series models. Here is the complete code.
```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen2.5-1.5B-Instruct')
t = tokenizer('', return_tensors='pt')
print(t['input_ids'].dtype) #torch.float32
```

I've investigated further and I believe this is caused by the behaviour of `torch.tensor()`:

`torch.tensor([]).dtype`  # float32

`torch.tensor([1, 2, 3]).dtype`  # int64

The `torch` behaviour when converting an input list to a tensor is that the output dtype will be `int64` when all of the elements in the list are `int`, otherwise it will be `torch.float32`. This creates a strange edge case when the input is empty, but this empty input is not a valid model input regardless of dtype, and so I'm not sure if it's worth fixing this bug!

Yes, you are right. I also noticed `torch.tensor`'s behavior determined by the input's dtype.

Actually, I’m currently working with an LLM integrated into an environment. When concatenating the environment’s response (which is sometimes empty) to the LLM’s output tensor of type torch.long, the concatenation unexpectedly changes the resulting tensor’s dtype to torch.float32 when the response is empty. This dtype mismatch subsequently causes an error when feeding the concatenated tensor back into the LLM.

I know this can be easily resolved by adding type conversion when concatenating tensors, but should the tokenizer's behavior remain consistent, i.e., whether the default data type of return values should stay unchanged as long as no errors occur during tokenization?

Hmm, I think it might make sense, but I'm unsure if the expected dtypes are stored anywhere, so this could be a tricky PR.

I suspect the only way we could do this is to have a dict of common tokenizer output names like `input_ids`, and map those to the appropriate dtype like `torch.long`. Other outputs might still exhibit this edge case behaviour.

cc @arthurzucker for tokenizers - would you support a PR to add that, or does it add complexity for not enough gain?

Thanks! Can we just modify function `convert_to_tensors`?
https://github.com/huggingface/transformers/blob/c0f8d055ce7a218e041e20a06946bf0baa8a7d6a/src/transformers/tokenization_utils_base.py#L719-L777

For example, just explicitly specify the dtype when creating tensors in the `as_tensor` function as following.
```python
        elif tensor_type == TensorType.PYTORCH:
            if not is_torch_available():
                raise ImportError("Unable to convert output to PyTorch tensors format, PyTorch is not installed.")
            import torch

            is_tensor = torch.is_tensor

            def as_tensor(value, dtype=None):
                if isinstance(value, list) and isinstance(value[0], np.ndarray):
                    return torch.from_numpy(np.array(value)).to(torch.long)
                return torch.tensor(value, dtype=torch.long)
```

Hi @wangzhen0518, the problem is that some tokenizers might return `float32` values for some keys! I'm not sure which, so I'd have to scan the codebase. We could try making your change in a PR and seeing which tests fail, though - feel free to open a PR for that and ping me!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.