noamaanMulla-03/transformers-issues huggingface dataset:
Generalize CFG to allow for positive prompts
Author: oobabooga
Pull request: yes

This PR changes the `guidance_scale` restriction from `> 1` to `!= 1`, thus allowing the negative prompt in CFG (https://github.com/huggingface/transformers/pull/24654) to be used as a positive prompt.

I believe that this change opens up new ways to use CFG while having no downside.

@gante 

Comments:
You are right, I had changed the docstring for the wrong class. Now it's fixed, and I also have added a mention to the positive prompt possibility, as well as a positive prompt example.

I note that I could not reproduce the existing examples, so my new one looks different:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
inputs = tokenizer(["Today, a dragon flew over Paris, France,"], return_tensors="pt")
out = model.generate(inputs["input_ids"], guidance_scale=1.5)
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])

neg_inputs = tokenizer(["A very happy event happened,"], return_tensors="pt")
out = model.generate(inputs["input_ids"], guidance_scale=2, negative_prompt_ids=neg_inputs["input_ids"])
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])

neg_inputs = tokenizer(["A very happy event happened,"], return_tensors="pt")
out = model.generate(inputs["input_ids"], guidance_scale=0, negative_prompt_ids=neg_inputs["input_ids"])
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])

```

The existing examples, which start with an upper case character even though the input prompt ends in a comma:

```
The dragon flew over Paris, France, landing in Lyon, a city of a few million. Dragon-flying was a new form of
transport, and the dragon was the first in Europe.

The dragon flew over Paris, France, crashing into Notre Dame Cathedral in the French capital killing at least 127
people and injuring more than 350.
```

My outputs:

```
Today, a dragon flew over Paris, France, killing at least 50 people and injuring more than 100

Today, a dragon flew over Paris, France, killing at least 130 people. French media reported that

Today, a dragon flew over Paris, France, and I'm very happy to be here. I
```

@oobabooga regarding the examples: given that the default `max_length` is `20`, it seems like the original examples were crafted with additional parameterization (they have clearly more than 20 tokens). Let's leave yours as it is, and perhaps rectify it later (see below).

@Vermeille would you be able to clarify the parameterization that led to the examples added in the CFG PR? :) Otherwise, we (the transformers team) will have to rewrite the examples, as they will fail our internal daily CI

_The documentation is not available anymore as the PR was closed or merged._
Fix for #25437
Author: ydshieh
Pull request: yes

# What does this PR do?

We need to make some extra adjustment to make the code below in `.circleci/create_circleci_config.py` work
```python
if example_tests == "all":
    job.tests_to_run = [f"examples/{framework}"]
```
This is also align with at the end of `utils/tests_fetcher.py`

```python
        if commit_flags["test_all"]:
           ...
            with open(example_file, "w", encoding="utf-8") as f:
                f.write("all")
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Refactor image processor testers
Author: amyeroberts
Pull request: yes

# What does this PR do?

The tests for the image processors contained a lot of repeated logic. In particular, the logic to prepare inputs and for testing resulting outputs. Removing the class-specific logic and consolidating in the mixin should make add new image processor tests easier. 

This PR: 
* Moves `test_call_numpy`, `test_call_pil`, `test_call_pytorch` to the mixin and removes repeated logic
* Renames `ImageProcessorSavingTestMixin` -> `ImageProcessingTestMixin` as the mixin handles more than saving

To support this, two methods were added to the model tester class: 
* `prepare_image_inputs`: Each model is responsible for preparing the inputs for testing. Removes the repeated `prepare_image_inputs` logic. in some model testers
* `expected_output_image_shape` - method which returns the expected shape for 
 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Switch Transformers: remove overwritten beam sample test
Author: gante
Pull request: yes

# What does this PR do?

#25375 added a fix to beam sample + `num_return_sequences >1`, and had to tweak the corresponding tests. Switch Transformers had those tests overwritten, meaning its beam sample tests were not updated -- causing our slow CI to fail.

The cause for overwriting the tests was flakiness of the original tests with this model. However, running with `--flake-finder --flake-runs=1000` on CPU and GPU, I couldn't find any flakiness. As such, this PR simply removes the overwritten tests.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Access to Transformers example Pytorch link broken . Impact on navigation as well
Author: SoyGema
Pull request: no


Hello there! 👋

## Context
Exploring Pytorch translation example

## Issue
There seems to be a broken link in Pytorch example in the Translation example, similar to what happened in #24497 
https://github.com/huggingface/transformers/blob/347001237a8ff845fc23f678107fc505361f9f13/examples/pytorch/translation/run_translation.py#L381

When I follow the link , the following message appears. And when I click the here link

<img width="1154" alt="Captura de pantalla 2023-08-10 a las 18 19 10" src="https://github.com/huggingface/transformers/assets/24204714/9cc8450f-85bc-4be2-934b-b687dcb8c4f9">

It redirects me to https://huggingface.co/docs/transformers/main/en/examples with an 404 error.

## Potential fix 

1. Is that ok if I submit a quick fix as in #24594 ?
2. Would it be hard to try to submit a contrib of something like this? ... 
> specify doc redirects using a `redirects.yml` like in datasets and other libraries: https://github.com/huggingface/datasets/blob/main/docs/source/_redirects.yml

Do you know if this entails more than create than source folder and .yaml file? Would I have to study [this](https://github.com/huggingface/datasets/tree/main/docs#generating-the-documentation)?


@sgugger 
Keep up the good work! 




Comments:
For the redirects, you will have to see with the datasets team on their repo :-)
As for Transformers, happy to look at a PR fixing the link!
Update run_translation.py broken link example Pytoch
Author: SoyGema
Pull request: yes

# What does this PR do?

Fixes #25444 

<!-- Remove if not applicable -->




## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add input_data_format argument, image transforms
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds `input_data_format` as optional argument to all image transforms. This enables users to explicitly set the format, rather than it being inferred, which can solve issues when it's ambiguous. 

Once merged, image processors can then pass this along to allow for more robust processing - either images with different number of channels or hard-to-infer data format. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Mark flaky tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Handles two tests which semi-regularly fail on CI runs

`tests/utils/test_image_utils.py::LoadImageTester::test_load_img_url_timeout`
- Mark with an is_flaky decorator

`tests/models/wav2vec2/test_modeling_tf_wav2vec2.py::TFWav2Vec2ModelTest::test_labels_out_of_vocab`
- Already has an is_flaky decorator
- Upped the vocab size being used when generating random ids tensor to increase probability of OOV error being hit 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
openlm-research/open_llama_13b_easylm cannot be downloaded
Author: leweex95
Pull request: no

### System Info

transformers: 4.30.2.
Python: 3.9.17
OS: MacOS Ventura 13.3.1 (a) 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Code to reproduce:

```
model_id = "openlm-research/open_llama_13b_easylm"
model_name = model_id.split("/")[1]

model = pipeline(model=model_id)
model.save_pretrained(f"./models/{model_name}")
```


### Expected behavior

I expect the model to be downloadable locally for use in downstream NLP tasks. Noteworthy is that on the [website](https://huggingface.co/openlm-research/open_llama_13b_easylm) it is indicated that 0 people downloaded this model over the past month. 

With the above script, I can easily fetch other models such as `"openlm-research/open_llama_13b"`:

```
model_id = "openlm-research/open_llama_13b"
model_name = model_id.split("/")[1]

model = pipeline(model=model_id)
model.save_pretrained(f"./models/{model_name}")
```

Comments:
Looking at 

https://huggingface.co/openlm-research/open_llama_13b_easylm/tree/main

The file doesn't seem to be torch bin file. 

However, https://huggingface.co/openlm-research/open_llama_13b/tree/main has those `.bin` files.

You will have to open an issue on that Hub repo. to discuss with the repo. owner.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Trajectory Transformer - NameError: name 'self' is not defined
Author: keyuchen21
Pull request: no

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.9.16
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Using GPU in script?:  No
- Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import TrajectoryTransformerModel
import torch
import numpy as np

device = "cuda"
from transformers import TrajectoryTransformerModel
import torch

model = TrajectoryTransformerModel.from_pretrained(
    "CarlCochet/trajectory-transformer-halfcheetah-medium-v2"
)
model.to(device)
model.eval()

observations_dim, action_dim, batch_size = 17, 6, 256
seq_length = observations_dim + action_dim + 1

trajectories = torch.LongTensor([np.random.permutation(self.seq_length) for _ in range(batch_size)]).to(
    device
)
targets = torch.LongTensor([np.random.permutation(self.seq_length) for _ in range(batch_size)]).to(device)

outputs = model(
    trajectories,
    targets=targets,
    use_cache=True,
    output_attentions=True,
    output_hidden_states=True,
    return_dict=True,
)

### Expected behavior

![1687627979812](https://github.com/huggingface/transformers/assets/54015474/609d3e13-0091-40f3-815f-9cc28d7bc429)


Comments:
`self` is indeed not defined in your code. Please use the [forums](https://discuss.huggingface.co/) to debug your code :-)

Yeah, but I was using the code directly from the tutorial, so that's the reason I raised this issue...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
BLIP special additional token self.tokenizer.enc_token_id
Author: ghost
Pull request: no

### System Info

Hello,

It seems to me that the Salesforce implementation of BLIP uses a special tokem 'ENC' at the beginning of each text sentence, it is added 
https://github.com/salesforce/BLIP/blob/a176f1e9cc5a232d2cc6e21b77d2c7e18ceb3c37/models/blip.py#L190

and used when encoding text conditioned on images e.g. https://github.com/salesforce/BLIP/blob/a176f1e9cc5a232d2cc6e21b77d2c7e18ceb3c37/models/blip.py#L67
or here
https://github.com/salesforce/BLIP/blob/b7bb1eeb6e901044a9eb1016f408ee908b216bc7/models/blip_retrieval.py#L124

Shouldn't we do the same? What is special about that token? 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

When in conditional mode with images, we should add 
`encoder_input_ids[:,0] = self.tokenizer.enc_token_id`?


Comments:
cc @ArthurZucker 

Hey! Thanks for submitting this issue. I think @younesbelkada is the most familiar with Blip and should be able to answer whether or not the `enc_token_id` should be added. BLIP seems to be using a `BertTokenizer`, and looking at [this](https://huggingface.co/Salesforce/blip-image-captioning-large/blob/main/tokenizer_config.json) I don't think that we are adding it.  

Any news here? 

Hi @DianeBouchacourt 
Do you notice any notable qualitative difference when adding that token? When porting the model I got predictions matched with the current approach 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Reuse the cache created for latest `main` on PRs/branches if `setup.py` is not modified
Author: ydshieh
Pull request: yes

# What does this PR do?

For a PR or a branch, if `setup.py` is not modified (compare to the common ancestor with the `main`), let's use cache that is created for the `setup.py` from the `latest` commit on the `main` branch. 

**latest** means the latest one on `main` at the moment where a run is triggered.

Motivation:

  - avoid having cache for `main` and `pull` (for most cases), which is introduced in #24886 to avoid unexpected/undesired edge cases.
  - **avoid the cache creation and store for PRs/branches that still have (very) old `setup.py` as they don't rebase on more recent `main`**

With this PR, we expect the storage usage for `cache` could be reduced dramatically 🤞 .

[The artifact in this job run](https://app.circleci.com/pipelines/github/huggingface/transformers/70312/workflows/b93db7c8-40ac-4925-ac2a-ea9feac9b926/jobs/881950/artifacts) shows `generated_config.txt` has explicit checksum used `v0.7-pipelines_torch-main-pip-9RXs1YQ8L2beP4cdAfRDkWX0VRTtWaQodDVKzvyJwPI=` rather than `{{ checksum "setup.py" }}`

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger FYI: I need to change 2 more places to make it work correctly for all cases. See my last 2 review comments.

Hi @sgugger 

I am afraid I have to revert this PR until we do something to enabling sharing cache (see below at the end). [From this section](
https://circleci.com/docs/caching/#caching-and-open-source)

> PRs from the same fork repo share a cache (this includes, as previously stated, that PRs in the main repo share a cache with main).
> Two PRs in different fork repos have different caches.

The cache is never shared between PRs/branches from different forks. This might explains the question I posted about why the same cache key could be found sometimes but not other times. 

The PR description of #24886 is partially valid as **lhoestq** created a branch in `transformers` rather than a PR from his own forked repo.

The way to share cache is

> Enabling the sharing of[ environment variables](https://circleci.com/docs/env-vars/) allows cache sharing between the original repo and all forked builds.

But we need to be careful if we have sensitive env. variables or not.
core dumped with Wav2vec2CTC
Author: ZL92
Pull request: no

### System Info

transformers==4.31.0
protobuf==3.20.0
datasets==2.14.3

Python version: 3.9.15

### Who can help?

@sanchit-gandhi @Narsil 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi,

This one gives the core dumped error: 
```
import torchaudio
import os
import torch
from datasets import Dataset, load_dataset, Audio, load_metric
from transformers import Wav2Vec2ForCTC
```

While this one works:
```
from transformers import Wav2Vec2ForCTC
import torchaudio
import os
import torch
from datasets import Dataset, load_dataset, Audio, load_metric
```

### Expected behavior

The datasets and sentencepiece packages might be the causes.

Comments:
Hi, 

with just 

```python
import torchaudio
import os
import torch
from datasets import Dataset, load_dataset, Audio, load_metric
from transformers import Wav2Vec2ForCTC
```
I can't reproduce. There is no remaining code but just the imports causing the core dump?

Likewise - I am unable to reproduce. Could you check you're using the latest versions of all packages? (including torch, torchaudio and datasets)

> Likewise - I am unable to reproduce. Could you check you're using the latest versions of all packages? (including torch, torchaudio and datasets)

Thanks for your reply! 

It was caused by the versions of torch and torchaudio. The problem is solved after upgrading.

Perfect! Go well @ZL92 🤗
Revert "Reuse the cache created for latest `main` on PRs/branches if `setup.py` is not modified"
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#25445

The way to solve the whole huge cache issue is to enable cache sharing between PRs from different forked repositories, see [this doc](https://circleci.com/docs/caching/#caching-and-open-source).

If we decide to enable this (it's a dangerous take), then we can include the #25445 once the sharing is done.

More info: https://circleci.com/docs/oss/#pass-secrets-to-builds-from-forked-pull-requests

Comments:
_The documentation is not available anymore as the PR was closed or merged._