noamaanMulla-03/transformers-issues huggingface dataset:
LlamaForCausalLM returning prompt without answer
Author: leweex95
Pull request: no

### System Info

transformers: 4.30.2
Python: 3.9.17
OS: MacOS 13.3.1 (a)

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I took the below code from the official documentation on the HuggingFace website: https://huggingface.co/openlm-research/open_llama_13b_easylm, and slightly adapted it to match my use case that is information extraction from unstructured text (named entity recognition) using LLMs.


```
import torch
from transformers import LlamaTokenizer, LlamaForCausalLM

model_path = 'openlm-research/open_llama_13b'

tokenizer = LlamaTokenizer.from_pretrained(model_path)

model = LlamaForCausalLM.from_pretrained(
    model_path, 
    torch_dtype=torch.float16, 
    device_map='auto',
)

prompt = "What are the named entities in the following text: 'The Moon revolves around the Earth for over 4 billion years.'"

input_ids = tokenizer(prompt, return_tensors="pt").input_ids

generation_output = model.generate(
    input_ids=input_ids, max_new_tokens=32
)

answer = tokenizer.decode(generation_output[0])
```


This code returns the following output:

`"
What are the named entities in the following text: 'The Moon revolves around the Earth for over 4 billion years.'?\nThe named entities in the following text are:\nThe Moon revolves around the Earth over 4 billion years.\nThe Moon revolves around the"`

which is not what I was hoping for. Interestingly, when I use the default example shown in the documentation, i.e.

`prompt = 'Q: What is the largest animal?\nA:'`

I get the following output:

`'
Q: What is the largest animal?\nA: A whale.\nQ: What is the largest animal?\nA: A whale.\nQ: What is the largest animal?\nA: A whale'`

which is slightly better, although I don't quite understand how to limit the engine not to keep repeating itself. 


### Expected behavior

The expected output would be something like: 

`{"ASTRONOMICAL_NAME": "Moon", "ASTRONOMICAL_NAME": "Earth", "PERIOD": "4 billion years"}`



Comments:
Hi @leweex95, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.
Fix `VisionTextDualEncoderIntegrationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Need a tiny update in the test files after the PR #24585

So far, CI gets errors like

```bash
RuntimeError: Only Tensors of floating point and complex dtype can require gradients
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Unpin `huggingface_hub`
Author: ydshieh
Pull request: yes

# What does this PR do?

- As the release `0.16` is out today.

- Also, use `--upgrade-strategy eager` in `pip install` which is required to respect [this comment](https://github.com/huggingface/transformers/pull/24424#pullrequestreview-1493647494).

  The default `-U` (which is associated with `only-if-needed`) won't upgrade to all available new versions. See [the doc](https://pip.pypa.io/en/stable/development/architecture/upgrade-options/#controlling-what-gets-installed):

  > packages are only upgraded if they are named in the pip command or a requirement file (i.e, they are direct requirements), or an upgraded parent needs a later version of the dependency than is currently installed.

- Since some packages are upgraded, let's change the cache version number for the new versions could be included in the cache.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you @ydshieh !
Implement Lion (EvoLved Sign Optimizer)
Author: dannyadkins
Pull request: yes

# What does this PR do?

Lion is a new optimizer from Google Brain that has seen early results improving on language modeling tasks: https://arxiv.org/abs/2302.06675

This PR implements Lion (unfused) as a drop-in replacement for Adam.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23820). All of your documentation changes will be reflected on that endpoint.

Thanks for your PR. Transformers is a library of models, not optimizers. All optimizers implemented inside the library are deprecated and we won't accept new ones. You can already use Lion via bitsandbytes for instance.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
RuntimeError
Author: lazytensor
Pull request: no

### System Info

Facing error when doing`from transformers import Trainer`

`RuntimeError: Failed to import transformers.training_args because of the following error (look up to see its traceback): cannot import name 'PartialState' from 'accelerate' (/opt/conda/lib/python3.10/site-packages/accelerate/__init__.py)`

Here is the environment

- `transformers` version: 4.29.2
- Platform: Linux-5.15.109+-x86_64-with-glibc2.35
- Python version: 3.10.10
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.0 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.10 (gpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.7
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

Solutions tried `pip install -U accelerate`. But it is still not resolved




### Who can help?

 @pacman100 @ArthurZucker @sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

`from transformers import Trainer`

### Expected behavior

Expecting no error

Comments:
Could you let us know your version of Accelerate?

same problem

This worked at my Kaggle Notebook 

https://stackoverflow.com/questions/76363436/cannot-import-name-partialstate-from-accelerate-when-using-huggingface-pipel

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix model referenced and results in documentation. Model mentioned was inaccessible
Author: rafaelpadilla
Pull request: yes

# What does this PR do?

This is a very small change on the documentation.
The mentioned model (`MariaK/detr-resnet-50_finetuned_cppe5`) was either removed or set to private. So I could not reproduce the shown example.  
I basically reference the same model but from another user, which provides a slightly better result. That's why I also updated the metrics.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @MKhalusova 

 
 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add Nucleotide Transformer notebooks and restructure notebook list
Author: Rocketknight1
Pull request: yes

As the name suggests, this adds links to the recent Nucleotide Transformer notebooks in the main `transformers` docs! It also restructures the notebooks list - right now the `Other` list is just full of bio models, so I moved them into their own section.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
fixes issue when saving fsdp via accelerate's FSDP plugin
Author: pacman100
Pull request: yes

# What does this PR do?
1. Fixes https://github.com/huggingface/transformers/issues/24057#issuecomment-1595152783
2. When using Accelerate's integration for FSDP, fsdp_plugin saves the optimizer state under various configs such as Full_Dict, Sharded_Dict ... properly. For other cases such as with FSDP-XLA, the trainer's behaviour is unchanged.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks again for this issue! I just finished training a largeish (7b param) model using this fix and have some questions. 
I noticed the directory where the model was meant to save looks a bit odd. While the training code seems to have finished without error, the directory contains some checkpoint directories and a directory called pytorch_model_0 (which has a bunch of .distcp files), but none of the files I previously would see in my trained model directories, like the config.json and .bin files. Is this expected save behavior? 

Hello @amartino1, this is because it is now using the FSDP's recommended way of saving ckpts, see this: https://github.com/pytorch/pytorch/blob/e71ab214226af1f9dbded944e939c6447e0e8f09/torch/distributed/checkpoint/examples/fsdp_checkpoint_example.py#L59

You will only notice that if you are using `SHARDED_STATE_DICT` as the `fsdp_state_dict_type`. 

with PR https://github.com/huggingface/transformers/pull/24591, it should save the whole model in transformers format as well as FSDP ckpt following what you have chosen as `fsdp_state_dict_type`.
Gradient clipping is no longer recommended?
Author: cloudygoose
Pull request: no

### System Info

Hi,

I just found that in the current examples (e.g., https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm_no_trainer.py), gradient clipping is no longer applied. Is there any particular reason? Is it okay if we add a line to do gradient clipping myself?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

N/A

Comments:
You can definitely experiment with gradient clipping.

The `transformers` GitHub pages are reserved for issues or feature requests. The question here is not in the scope, and [Hugging Face Forums](https://discuss.huggingface.co/) is a better place, if you have further question on this topic.
Language Modeling on Already Tokenized Data
Author: jorisperrenet
Pull request: no

### System Info

When I try to execute [`run_clm.py`](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py) in the language modeling example I naturally get the question to specify the `tokenizer_name`.

Yet, my data is already tokenized, i.e. my train and validation look (very crudely) like
```
0 3111 5100 2100 3100 6000
1000 4067 3031 3068 5141 3073
1000 3067 6031 3068 5141 3076
```
Thus, sequences on separate lines.

My question is: is there some kind of workaround such that I can train a model on this already tokenized data?

Any help would be great!

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Inside `transformers/examples/pytorch/language-modeling` create the folder `output` and the file `train.txt` (with some numbers in it, see above).

```
python run_clm.py --model_type gpt2 --output_dir output --do_train --train_file train.txt
```

It returns
```
ValueError: You are instantiating a new tokenizer from scratch. This is not
supported by this script.You can do it from another script, save it, and load
it from here, using --tokenizer_name.
```

### Expected behavior

I would expect/prefer that the output would be a warning specifying that a "passthrough" tokenizer will be used.

Comments:
The example scripts serve as examples 🤗 . If you need some custom modification(s), go for it.

In your case, you can probably skip the following block (and other similar places). You might need to assign your own tokenized dataset(s) to variables like `tokenized_datasets` however.

https://github.com/huggingface/transformers/blob/9a5d468ba0562e2d5edf9da787881fa227132bca/examples/pytorch/language-modeling/run_clm.py#L456-L470
DeepSpeed/FSDP ckpt saving utils fixes and FSDP training args fixes
Author: pacman100
Pull request: yes

# What does this PR do?
1. `_save` function saves `tokenizer` and `training_args.bin` in addition to model. 
2. This PR rearranges logic for saving model for DS and FSDP such that the above 2 things are also saved in addition to model ckpt. 
3. FIxes https://github.com/huggingface/transformers/issues/24641

Comments:
_The documentation is not available anymore as the PR was closed or merged._
deepspeed init during eval fix
Author: pacman100
Pull request: yes

# What does this PR do?
1. Fixes #24294 as DS Z2 and Z1 stages don't modify the model and the check was exhibiting wrong behaviour. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for working on this. I'm currently running into the same issue with V 4.30.2. Any idea when the the new V with this fix will be released?
TrainingArguments not working in transformers v 4.30
Author: Varshithavn
Pull request: no

### System Info

Hi @sgugger
I was trying to implement the same code that is present in the tutorial "https://github.com/huggingface/notebooks/blob/main/examples/language_modeling.ipynb" , but when executing the TrainingArguments function I am getting the error "ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U`" even after installing what was suggested still facing the same problem. My previous code which worked well 1 month ago also not working exactly at the **TrainingArguments** function.

Attaching the image below
<img width="1346" alt="image" src="https://github.com/huggingface/transformers/assets/96924488/01819fec-1bae-4d45-bea7-fa0ab60d63db">


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

training_args = TrainingArguments(
    f"{model_checkpoint}-wikitext2",
    evaluation_strategy = "epoch",
    learning_rate=2e-5,
    weight_decay=0.01,
    push_to_hub=True
)

From Casual language modeling task

### Expected behavior

TrainingArguments should be working with no error.

Comments:
From the discussion forum "https://discuss.huggingface.co/t/trainingargument-does-not-work-on-colab/43372" got the solution to use Transformers version 4.17 to make TrainingArguments work. Wanted to know why TrainingArguments not working in version 4.30?

After you `pip install accelerate -U`, did you restart the notebook? (It seems you are using colab notebook?)

Hi @ydshieh yes thanks that worked, yesterday I tried the same but not sure what changed but today it worked. Thanks for the comment. Yes, it is in colab notebook.
Add EMD loss
Author: wesboyt
Pull request: no

### Feature request

could we import this file for 1d EMD? its like kl div but allows us to better represent ordinal/numeric classes.
https://github.com/TakaraResearch/Pytorch-1D-Wasserstein-Statistical-Loss/blob/master/pytorch_stats_loss.py
is the best option Ive seen online.

### Motivation

I am currently using it locally for ordinal discrete density function approximation.

### Your contribution

I'm not totally sure whats necessary to incorporate it into the currently available options throughout the codebase, but it shouldn't be hard to import it.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Sliding window for finetuning 
Author: DanaTurkif
Pull request: no

### Feature request

The sliding window feature is important since many models have limited size (BERT 512 token) , as far as i know the feature is available when using the pipelines for inference however when finetuning it's not. 

 

### Motivation

I'm trying to finetune BERT models since they reached state of the art in many NLP tasks, but especially  for NER tasks and most of my documents are far larger than 512 tokens and truncating them will ruin the context. 

### Your contribution

I've been trying to implement sliding window manually by using the available features such as stride,  return_overflowing_tokens 

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask such questions. The feature is implemented via `stride` and `return_overflowing_tokens` in tokenizers as you note.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
trainer evaluation strucked when using dynamic padding in distributed evaluation
Author: CaoYiwei
Pull request: no

### System Info

tranasformer version=4.28.1
deepspeed=0.9.2

As I known, trainer evaluate func is distributed. when I use longest padding in every eval batch, the program will be strucked.  This won't happen when I use max_length padding.  I guess processes is structed because gather operation between different length tensors from different processes. Please fix this bug.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1.use longest padding
2.trainer evaluate in multi-gpus with deepspeed

### Expected behavior

the program won't struck

Comments:
Without a code reproducer, there is nothing we can do. The Trainer will pad samples to the same length before gathering them, so this is already accounted for.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
LlamaTokenizer should be picklable
Author: icyblade
Pull request: yes

# What does this PR do?
Fixes `LlamaTokenizer` not picklable and will cause `OSError` when tokenizing with Spark UDF.
Reference: #13577 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?
@ArthurZucker 



Comments:
_The documentation is not available anymore as the PR was closed or merged._