noamaanMulla-03/transformers-issues huggingface dataset:
Support ratios for `logging_steps`, `eval_steps`, and `save_steps`
Author: konstantinjdobler
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #23171 

Adds support for *ratios* to the `logging_steps`, `eval_steps`, and `save_steps` arguments, i.e. if they are a float in range `[0,1)`, the steps are calculated as a ratio of total training steps.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
CTC example: updated trainer parameters to save tokenizer
Author: MKhalusova
Pull request: yes

The current example only passes `feature_extractor` to `Trainer` and thus `tokenizer` is not saved and won't be pushed to Hub. This PR fixes this by passing the `processor` to `Trainer`. It can probably be refactored further to get the tokenizer and feature_extractor from the instantiated processor, but with regard to behavior, this small fix seems to address the problem. 


Comments:
Sorry, wrong branch, will open a new PR

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23242). All of your documentation changes will be reflected on that endpoint.
Revert "[Doctests] Refactor doctests + add CI"
Author: sgugger
Pull request: yes

Reverts huggingface/transformers#22987

This PR created a hard dependency on `pytest` which we don't want in Transformers. Looking a bit more it would be better if the whole `doctest_utils.py` module lived outside of the Transformers library, so it should be structured differently.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23245). All of your documentation changes will be reflected on that endpoint.
`from_config` errors for `bigcode/santacoder`
Author: DyeKuu
Pull request: no

### System Info

transformers commit (current main branch): c34a525d2

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoConfig, AutoModelForCausalLM
model_name = "bigcode/santacoder"
config = AutoConfig.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_config(config, trust_remote_code=True)
```

gives error:
```
Exception has occurred: ValueError
not enough values to unpack (expected 2, got 1)
  File "/fsx/kunhao/transformers/src/transformers/dynamic_module_utils.py", line 408, in get_class_from_dynamic_module
    module_file, class_name = class_reference.split(".")
  File "/fsx/kunhao/transformers/src/transformers/models/auto/auto_factory.py", line 411, in from_config
    model_class = get_class_from_dynamic_module(repo_id, module_file + ".py", class_name, **kwargs)
ValueError: not enough values to unpack (expected 2, got 1)
```
However, directly calling `model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)` works fine.

### Expected behavior

No error happens

Comments:
Not an expert but I feel like we could do something here https://github.com/huggingface/transformers/blob/main/src/transformers/models/auto/auto_factory.py#L410-L411

```diff
            class_ref = config.auto_map[cls.__name__]
            if "--" in class_ref:
                repo_id, class_ref = class_ref.split("--")
            else:
                repo_id = config.name_or_path
-           module_file, class_name = class_ref.split(".")
-           model_class = get_class_from_dynamic_module(repo_id, module_file + ".py", class_name, **kwargs)
+           model_class = get_class_from_dynamic_module(class_ref, repo_id, **kwargs)
```

Sounds like the right fix if you want to make a quick PR!
OSError: sentence-transformers/all-distilroberta-v1 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
Author: alexcoca
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: macOS-13.3-x86_64-i386-64bit
- Python version: 3.9.6
- Huggingface_hub version: 0.13.4
- Safetensors version: not installed
- PyTorch version (GPU?): 1.13.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the following code:

```python
from transformers import AutoModel
model  = AutoModel.from_pretrained("sentence-transformers/all-distilroberta-v1")
```

This fails with the OSError in the issue title. I would normally raise this issue on the hub, but then to my amazement, `sentence-transformers/all-distilroberta-v1` does not exist on the hub. Our code reliably worked for months, so I presume that this model, quite well known actually, was in the space at some point.  I wonder why model loading no longer works. @younesbelkada , any idea?

### Expected behavior

Model is loaded. 

Comments:
Experiencing the same issue with a different model. Looks like [the entire sentence-transformers page](https://huggingface.co/sentence-transformers) is down. Hopefully a fix is on the way.

UPDATE: Indeed, it looks like HuggingFace is aware of the issues and working on it: https://twitter.com/huggingface/status/1655760648926642178

Experiencing the same issue with a model from `cross-encoder`... Looks like the entire [cross-encoder page](https://huggingface.co/cross-encoder) is down. 

Hi @alexcoca, thanks for raising this issue! 

We're unfortunately experiencing a bug which means some popular organisations like sentence-transformers have had their model temporarily disappear from the Hub.

They will come back; we're working hard on getting this fixed ASAP! Apologies for the disruption.

@amyeroberts thanks for your hard work, everything is back to normal, I think. For future reference, users should know that `from_pretrained` methods have a `local_files_only` flag that can be passed to load a model that has been cached locally before. This can help in situations like this, thanks @Wauplin for pointing this out.
[docs] Audio task guides fixes
Author: MKhalusova
Pull request: yes

Related to https://github.com/huggingface/transformers/issues/23188 and https://github.com/huggingface/transformers/issues/23222

In the guide examples, only `feature_extractor` is passed to `Trainer`, so that's the only part of the processor that gets pushed to Hub. This PR fixes the docs to pass `processor` to Trainer as the `tokenizer` parameter, so both `feature_extractor` and `tokenizer` are saved. 
The behavior is confirmed with the ASR task guide example. We may also need to fix the example scripts. I'll look into it, and if a fix is needed, I'll create a separate PR. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Improve Docs of Custom Tools and Agents
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

This PR improves the docs explaining how to customize prompts and corrects some grammar, spelling, code snippets of both `transformers_agent.mdx` and `custom_tools.mdx`. Also `agent.toolbox` is made a get method / property which should help both with documentation and forbid the user to overwrite the attribute completely.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23255). All of your documentation changes will be reflected on that endpoint.
Transformers Agents
Author: sgugger
Pull request: yes

# Introducing Transformers Agents

This PR adds a new API called Transformers Agents. Agents allow you to use Transformers with zero code experience, directly talking to Transformers or Diffusers via natural language. It is based on `Agent`s and `Tool`s. The agent is an LLM prompted to generate code using the tools, which are simple functions performing a single task.

Tools can live in Transformers or on the Hub, this PR introduces both. You can read more about this in the added documentation but here is an example:

Define an agent using the starcoder model:
```py
from transformers import HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
```

Use the command `run` to execute a given problem:
```py
agent.run("Draw me a picture of rivers and lakes")
```

![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rivers_and_lakes.png)

Use the command `chat` to chat with the agent and execute instructions one after the other:

```py
agent.chat("Draw me a picture of rivers and lakes")
```

![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rivers_and_lakes.png)

```py
agent.chat("Transform the picture so that there is a rock in there")
```
![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rivers_and_lakes_and_beaver.png)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

picture = agent.run("Draw me a picture of rivers and lakes")
==Explanation from the agent==
I will use the following  tool: `image_segmenter` to generate a segmentation mask for the image.


==Code generated by the agent==
prompt = "rivers and lakes"
mask = image_segmenter(image, prompt)


==Result==
Evaluation of the code stopped at line 1 before the end because of the following error:
The variable `image` is not defined.

Ah yes we did that example with openAI. Will fine-tune the prompt so that example works before the release, thanks for the pointer!
Generate a pre-training model for GAP Computational Discrete Algebra System.
Author: hongyi-zhao
Pull request: no

### Feature request

I can't find any fine-tuned pre-training models for [GAP Computational Discrete Algebra](https://www.gap-system.org/).

### Motivation

I'm a scholar who conducts research in related fields of mathematical physics based on group theory methods. So, I would like to have a fine-tuned pre-training model for [GAP Computational Discrete Algebra](https://www.gap-system.org/).

### Your contribution

I want to know the possibilities in creating such a model based on the resources provided on huggingface. Any hints/comments will be appreciated.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
PyTorch ViTMAEModel output is not deterministic
Author: nalzok
Pull request: no

### System Info

- `transformers` version: 4.27.4
- Platform: Linux-5.13.0-1023-gcp-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.6.8 (tpu)
- Jax version: 0.4.8
- JaxLib version: 0.4.7
- Using GPU in script?: No.
- Using distributed or parallel set-up in script?: No

### Who can help?

@amyeroberts @sgugger @stevhliu @MKhalusova 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the [example snippet](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/vit_mae#transformers.ViTMAEModel) in the documentation. I'll copy & paste it below for your convenience:

```python
from transformers import AutoImageProcessor, ViTMAEModel
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

image_processor = AutoImageProcessor.from_pretrained("facebook/vit-mae-base")
model = ViTMAEModel.from_pretrained("facebook/vit-mae-base")

inputs = image_processor(images=image, return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
```

Put the code in a script, execute it twice, and you will notice that the content in `last_hidden_states` is different.

### Expected behavior

The embeddings should be deterministic across runs.

Comments:
Hi,

Yes that's expected behaviour, see https://github.com/huggingface/transformers/issues/20431

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix new line bug in chat mode for agents
Author: sgugger
Pull request: yes

# What does this PR do?

Depending on the agent used we might get too many new lines here. Just stripping everything and adding the right amount fixes this.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23267). All of your documentation changes will be reflected on that endpoint.
Every call to the generate method will repeatedly print "Generate config {config}" on the console
Author: Silypie
Pull request: no

https://github.com/huggingface/transformers/blob/3335724376319a0c453049d0cd883504f530ff52/src/transformers/generation/configuration_utils.py#L577
Is it feasible to delete this line of code? Or is there a better way?



Comments:
cc @gante 

Hey @Silypie 👋 

Consider the following points:
1. `logger.info` messages are not printed by default
2. Logging with `info` on `from_dict` configuration methods is also standard across the library
3. This line should only be reached from `.generate()` in a legacy setup

Because of these 3 points, I'm biased toward not changing this behavior. Nevertheless, it may be a bug -- can you share a short stand-alone script so I can reproduce the issue?
CTC example: updated trainer parameters to save tokenizer
Author: MKhalusova
Pull request: yes

The current example only passes `feature_extractor` to `Trainer` and thus `tokenizer` is not saved and won't be pushed to Hub. This PR fixes this by passing the `processor` to `Trainer`. It can probably be refactored further to get the tokenizer and feature_extractor from the instantiated processor, but with regard to behavior, this small fix seems to address the problem.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks @MKhalusova!
T5-Flan Resuming Int-8 / LoRA / Deepspeed Checkpoint
Author: afogarty85
Pull request: no

### System Info

transformers 4.29
accelerate 0.19
peft 0.3



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

My training ended unexpectedly and I want to resume my T5-Flan training from a checkpoint. Inside my checkpoint I have:

```
global_step25000
latest
pytorch_model.bin 
rng_state.pth 
trainer_state.json 
training_args.bin
zero_to_fp32.py
```

I am unable to load the checkpoint in the following ways:

```

# try 1
model = AutoModelForSeq2SeqLM.from_pretrained("/checkpoint-25000",
                                                           load_in_8bit=True,
                                                           device_map='auto',
                                                           )
# ValueError: weight is on the meta device, we need a `value` to put in on 0.

# try 2
model = AutoModelForSeq2SeqLM.from_pretrained("/checkpoint-25000",
                                                           )
# ValueError: weight is on the meta device, we need a `value` to put in on 0.

# try 3
config = AutoConfig.from_pretrained("google/flan-t5-large")
with init_empty_weights():
    model = AutoModelForSeq2SeqLM.from_config(config)
model.tie_weights()
device_map = infer_auto_device_map(model)
model = load_checkpoint_and_dispatch(model,  "checkpoint-25000/pytorch_model.bin", device_map=device_map)

# AttributeError: 'T5ForConditionalGeneration' object has no attribute 'model'
```




### Expected behavior

The checkpoint to attach so I could resume training.

Comments:
Totally didnt try the trainer, but it works!
`trainer.train('checkpoint-xxxx')`
[`gpt`] Gpt2 fix half precision causal mask
Author: younesbelkada
Pull request: yes

# What does this PR do?

Applies a similar fix than https://github.com/huggingface/transformers/issues/23136 but for GPT2.

To reproduce: 
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2", device_map="auto", load_in_8bit=True)
inputs = torch.LongTensor([[1, 1, 1], [1, 2, 1]]).to(0)

print(model(inputs))
```

The explanation is the same as the tagged PR: 

> When going for low_cpu_mem_usage each parameter is force-casted to the expected dtype, which is force-set to torch.float16 for 8bit models.

> Therefore, for 8bit models (and also half-precision models) the causal mask is always force casted to float16 as it is part of the model's state dict, hence expected to be loaded from the Hub if the mask is available on the state dict.

> The fix is to add persistant=False and add a field _keys_to_ignore_on_unexpected (for removing the warnings) to avoid loading that causal mask from the state dict and assign it to the buffer, and all causal masks that are saved as buffers should do the same to avoid unexpected behaviors.

Some users reported that they were also able to reproduce on PyTorch main branch but without load_in_8bit, I didn't managed to reproduce that way, I will have a deeper look

cc @amyeroberts 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Temporary tolerance fix for flaky whipser PT-TF equiv. test
Author: amyeroberts
Pull request: yes

# What does this PR do?

The PT-TF whisper tests have recently become flaky e.g. for [this CI run](https://app.circleci.com/pipelines/github/huggingface/transformers/64100/workflows/b4463c5d-b3dc-4b00-a7cd-19acd096cb07/jobs/792381). 

Although the differences are still relatively small, it represents ~2x on the largest absolute difference. 

This PR temporarily increases the tolerance until the root cause is found. An issue will be opened and linked here for reference. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Issue: #23258 

_The documentation is not available anymore as the PR was closed or merged._
Temporarily increase tol for PT-FLAX whisper tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Flax whisper equivalence tests have also started to fail in a flaky manner with small increase in the difference between the PT and FLAX model e.g. for [this run](https://app.circleci.com/pipelines/github/huggingface/transformers/64230/workflows/e2d42ca4-f367-4a85-9054-a0ea99e49849/jobs/794534).

Flax equivalent of: #23257


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Updated issue #23258 to reference this PR too

_The documentation is not available anymore as the PR was closed or merged._
`offset_alibi` is not used
Author: JaheimLee
Pull request: no

What's the meaning of `offset_alibi`? It's not used in Bloom's source codes. Does it make sense?

Comments:
Hi @JaheimLee, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/), as we try to reserve the github issues for feature requests and bug reports. Another good place to ask would be opening a discussion [on the model page on the hub](https://huggingface.co/bigscience/bloom-7b1/discussions), as it directly relates to the configuration file there. 

> Hi @JaheimLee, thanks for raising an issue!
> 
> This is a question best placed in our [forums](https://discuss.huggingface.co/), as we try to reserve the github issues for feature requests and bug reports. Another good place to ask would be opening a discussion [on the model page on the hub](https://huggingface.co/bigscience/bloom-7b1/discussions), as it directly relates to the configuration file there.

Oh, sorry.  I created an issue on the hub. I close this one now.