Issue 1:
System Info

transformers version: 4.30.2

Python version: 3.10.11

System: Ubuntu 22.04.2 LTS

Who can help?

@ArthurZucker @younesbelkada

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

For the following code,

model_name = 'TheBloke/guanaco-7B-HF'
model = AutoModel.from_pretrained(model_name, torch_dtype=torch.bfloat16, trust_remote_code=True)
model = model.to('cuda')

inputs = tokenizer(['hello'], max_length=100, truncation=True, return_tensors="pt").to(model.device)
outputs = model(**inputs)

I got following error messages:

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[34], line 2
1 inputs = tokenizer(['hello'], max_length=100, truncation=True, return_tensors="pt").to(model.device)
----> 2 outputs = model(**inputs)

File ~/miniconda/envs/vocab/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)
1496 # If we don't have any hooks, we want to skip the rest of the logic in
1497 # this function, and just call forward.
1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1499 or _global_backward_pre_hooks or _global_backward_hooks
1500 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501 return forward_call(*args, **kwargs)
1502 # Do not call functions when jit is used
1503 full_backward_hooks, non_full_backward_hooks = [], []

TypeError: LlamaModel.forward() got an unexpected keyword argument 'token_type_ids'

The inputs value is following:

{'input_ids': tensor([[ 1, 22172]], device='cuda:0'), 'token_type_ids': tensor([[0, 0]], device='cuda:0'), 'attention_mask': tensor([[1, 1]], device='cuda:0')}

There is token_type_ids in the value returned from tokenizer, however, LlamaModel.forward() don't accept the arguments.

And I compared LlamaTokenizer and LlamaTokenizerFast, I found they behave differently.

from transformers import LlamaTokenizer, LlamaTokenizerFast
print(f"LlamaTokenizer: {LlamaTokenizer.from_pretrained(model_name)('hello')}")
print(f"LlamaTokenizerFast: {LlamaTokenizerFast.from_pretrained(model_name)('hello')}")

the results are:

LlamaTokenizer: {'input_ids': [1, 22172], 'attention_mask': [1, 1]}
LlamaTokenizerFast: {'input_ids': [1, 22172], 'token_type_ids': [0, 0], 'attention_mask': [1, 1]}

Expected behavior

Should LlamaTokenizerFast remove the token_type_ids in the returned value? or should LlamaModel.forward() accept the token_type_ids in the function arguments list? Thanks.

Issue 2:
System Info

platform==Ubuntu18.04

python==3.10

transformers==4.29.2

Who can help?

@ArthurZucker

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction


 is the special token of LLaMATokenizer(Fast), it is expected that 
 can be recognized as a single token when encoding the text. However, it can be shown that the two tokenizers behave differently:

>>> t1 = transformers.AutoTokenizer.from_pretrained("huggyllama/llama-7b", use_fast=True)
>>> t2 = transformers.AutoTokenizer.from_pretrained("huggyllama/llama-7b", use_fast=False)
>>> text = "I love you.
"
>>> t1(text)
>>> {'input_ids': [1, 306, 5360, 366, 21106, 29879, 29958], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1]}
>>> t2(text)
>>> {'input_ids': [1, 306, 5360, 366, 29889, 2], 'attention_mask': [1, 1, 1, 1, 1, 1]}

also, LLaMATokenizerFast returns token_type_ids but LLaMATokenizer does not.

Expected behavior

LLaMATokenizerFast to be consistent with LLaMATokenzier.