noamaanMulla-03/transformers-issues huggingface dataset:
Can't import Mega for causal LM model
Author: Tylersuard
Pull request: no

### System Info

ImportError: cannot import name 'MegaForCausalLM' from 'transformers' (/usr/local/lib/python3.9/dist-packages/transformers/__init__.py)

This happens while running the example code here: https://huggingface.co/docs/transformers/main/model_doc/mega

from transformers import AutoTokenizer, MegaForCausalLM, AutoConfig
import torch

tokenizer = AutoTokenizer.from_pretrained("mnaylor/mega-base-wikitext")
config = AutoConfig.from_pretrained("mnaylor/mega-base-wikitext")
config.is_decoder = True
config.bidirectional = False
model = MegaForCausalLM.from_pretrained("mnaylor/mega-base-wikitext", config=config)

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

prediction_logits = outputs.logits

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

!pip install transformers
from transformers import AutoTokenizer, MegaForCausalLM, AutoConfig



### Expected behavior

It should import MegaForCausalLM.

Comments:
The example you are referring to comes from the documentation of the main branch of Transformers, not the released version. You will thus need a [source install](https://huggingface.co/docs/transformers/installation#install-from-source) to be able to execute it.

Thank you @sgugger
Adding progress-bars to pipelines
Author: ntakouris
Pull request: no

### Feature request

In order to implement https://github.com/huggingface/evaluate/issues/442, in order to provide progress bars while using `evaluator_instance.compute(..., progress_bar=True)`, we would have to update the `base.py` pipeline in order to support this.

Quoting the referenced issue from evaluate:

After doing some digging, It's a matter of if the dataset+pipeline can support progress bars. For example, on the [call](https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/base.py#L1046) pipeline function, we can see that the actual pipeline could be many things, including but not limited to a GeneratorType (which does not advertise a __len__, a Dataset or a list (which typically have __len__), so the worse-case progress bar you can get would be a tqdm "X iterations / s" dialogue.

### Motivation

Progress bars are always nice and they are relatively simple to implement: just wrap an iterator. 

It gives us a qualitative sense of what we're doing. If the underlying unit supports `__len__`, it's extra useful for debugging or giving a rough processing estimate without having to run through everything.

### Your contribution

I'm willing to contribute given some guidance from the hf team.

Comments:
cc @Narsil 

Hey, you can already to that:

```python
import tqdm

for out in tqdm.tqdm(pipe(dataset)):
    pass
```

When using an iterating dataset instead of a real dataset you can add (`total=total` to get the "correct" progressbar).

Advantage of having the progressbar in usercode is that we don't have to choose your favorite progress bar or handle colab+jupyter weirdness here.

@Narsil I am referring to passing in a progress bar argument into the pipeline's `__call__` function, in order to accomplish 
 https://github.com/huggingface/evaluate/issues/442, not to adding progress bar to dataset iteration.

This can be done in `evaluate` directly is what I was saying.
Fix Whisper Positional Embeddings when using decoder context
Author: andyehrenberg
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21455). All of your documentation changes will be reflected on that endpoint.

cc @ArthurZucker 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

is this getting in or really not needed?

Seems to work well without 😉 Also not sure if the updates on whisper fixed the original issue, would have to check!
Enable naive Pipeline Parallelism training for Gpt neox japanese and san japanese
Author: mayankagarwals
Pull request: yes

As suggested in the https://github.com/huggingface/transformers/issues/22561, moved labels to the same device as logits for GPT Neox Japanese and Gpt San Japanese

Comments:
@sgugger Could you review this once?

_The documentation is not available anymore as the PR was closed or merged._
Clarify stride option
Author: luccailliau
Pull request: yes

# What does this PR do?

Clarify the `stride` option which refers to the number of overlapping tokens between chunks.

Fixes #22391


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Narsil, I just added a sentence in the doc to avoid confusion about the naming.

Have a good day
Remove 2 failing ONNX conversion tests
Author: ydshieh
Pull request: yes

# What does this PR do?

After ##22212, two tests start to fail.

Comments:
OK. But should we do something like removing CI regarding this? Currently failing tests pop up.

_The documentation is not available anymore as the PR was closed or merged._

I think you can remove the tests as well.

@sgugger Just to confirm, we want/could remove all things like 

- `ConvertCommand` in `src/transformers/commands/transformers_cli.py`
- `export_with_transformers` in `src/transformers/onnx/__main__.py`
- the file `src/transformers/onnx/convert.py` and any test using this fle

also cc @michaelbenayoun @fxmarty 

No we're not removing code, just the tests if they start failing.

OK, glad I ask!

Ping @sgugger again to draw a bit of his attention.
Replace -100s in predictions by the pad token
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the seq2seq examples with the Trainer on datasets with small samples. The problem is that the results on those samples get padded with a -100 by the Trainer, and this in turns gets an index error in the tokenizer decode method.

Fixes part of #22634

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Some text in the international README files are in the wrong language
Author: shermansiu
Pull request: no

### System Info

Some of the text in the international README files, such as the model descriptions, are in the wrong language.

### Who can help?

_No response_

### Reproduction

N/A

### Expected behavior

The text in each README file should be in the correct language.

A good starting point is looking at the model list for each language.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Conversion Script Tatoeba
Author: thanh767
Pull request: no

### System Info

I follow this guide https://github.com/huggingface/transformers/tree/main/scripts/tatoeba to convert models from Tatoeba Translation Challenge to Huggingface but when I ran 
`python3 src/transformers/models/marian/convert_marian_tatoeba_to_pytorch.py --models kor-eng --save_dir converted` 
it returns like the below, I found that the find_vocab_file function requires a vocab file with ext .yml, but the model from Tatoeba doesn't has.

```
  0%|                                                                                                          | 0/1 [00:01<?, ?it/s]Traceback (most recent call last):
  File "src/transformers/models/marian/convert_marian_tatoeba_to_pytorch.py", line 1324, in <module>
    resolver.convert_models(args.models[0])
  File "src/transformers/models/marian/convert_marian_tatoeba_to_pytorch.py", line 90, in convert_models
    convert(save_dir / model["_name"], dest_dir / f"opus-mt-{pair_name}")
  File "/tmp/transformers/src/transformers/models/marian/convert_marian_to_pytorch.py", line 663, in convert
    opus_state = OpusState(source_dir)
  File "/tmp/transformers/src/transformers/models/marian/convert_marian_to_pytorch.py", line 494, in __init__
    self.tokenizer = self.load_tokenizer()
  File "/tmp/transformers/src/transformers/models/marian/convert_marian_to_pytorch.py", line 592, in load_tokenizer
    add_special_tokens_to_vocab(self.source_dir, not self.share_encoder_decoder_embeddings)
  File "/tmp/transformers/src/transformers/models/marian/convert_marian_to_pytorch.py", line 409, in add_special_tokens_to_vocab
    vocab = load_yaml(find_vocab_file(model_dir))
  File "/tmp/transformers/src/transformers/models/marian/convert_marian_to_pytorch.py", line 385, in find_vocab_file
    return list(model_dir.glob("*vocab.yml"))[0]
IndexError: list index out of range
```

@sgugger @stevhliu 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1.  clone and install https://github.com/huggingface/transformers/tree/main/scripts/tatoeba
2. run python3 src/transformers/models/marian/convert_marian_tatoeba_to_pytorch.py --models kor-eng --save_dir converted

### Expected behavior

it can convert successfully

Comments:
cc @ArthurZucker 

I'll have a look thanks for reporting

Sorry, but it seems that all the formats are messed up w.r.t. the old scripts. These are not maintained and thus we don't plan on fixing this. If you want to however, feel free to contribute!
Fix decorator order
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix decorator order.

For some tests, like `test_basic_distributed`, we have the following on `main`.
```
    @require_torch_multi_gpu
    @parameterized.expand(params, name_func=parameterized_custom_name_func)
    def test_basic_distributed(self, stage, dtype):
```
but the (generated) tests are actually run even on a single GPU machine.

We need to change the order:
```python
    @parameterized.expand(params, name_func=parameterized_custom_name_func)
    @require_torch_multi_gpu
```

(PS: it doesn't mean the tests will fail on a single GPU VM. It still passes, but I am not sure if it makes sense.)

Comments:
cc @sgugger : I think this observation might be interesting to you.

_The documentation is not available anymore as the PR was closed or merged._

Yes, this requirement is documented here https://huggingface.co/docs/transformers/testing#to-gpu-or-not-to-gpu but it's not "enforced" so it's easy to miss

Perhaps it can be added to the quality checks?
Inconsistency in Model Output [ Token Classification]
Author: pratikchhapolika
Pull request: no

```
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
from transformers import pipeline
```

```
tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER")
model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER")
```

```
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "My name is Wolfgang and I live in Berlin"
```

```
ner_results = nlp(example)
print(ner_results)
```

**Output 1:**
```
[{'entity': 'B-PER', 'score': 0.9990139, 'index': 4, 'word': 'Wolfgang', 'start': 11, 'end': 19}, 
{'entity': 'B-LOC', 'score': 0.999645, 'index': 9, 'word': 'Berlin', 'start': 34, 'end': 40}]
```


```
inputs = tokenizer.encode_plus(example, return_tensors="pt", add_special_tokens=True, max_length=512, padding="max_length", truncation=True)
input_ids = inputs["input_ids"]
attention_mask = inputs["attention_mask"]
# Feed the encoded segment into the model to obtain the predicted labels for each token
outputs = model(input_ids, attention_mask=attention_mask)
logits = outputs.logits
predicted_labels = torch.argmax(logits, dim=2)[0]
```

`[0, 0, 0, 0, 3, 0, 0, 0, 0, 0]`

```
label_list = [ "O","B-MISC","I-MISC","B-PER","I-PER","B-ORG","I-ORG","B-LOC","I-LOC"]
final_label_names = [label_list[label] for label in predicted_labels]
```

**Output 2:**

`['O','O','O','O', 'B-PER', 'O','O','O','O','O']`


Comments:
ASTModel Signature doesn't work
Author: conradg
Pull request: no

### System Info

- `transformers` version: 4.27.2
- Platform: macOS-13.2.1-arm64-arm-64bit
- Python version: 3.10.7
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@NielsRogge


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When I run the following code, I expect this to do a forward pass successfully. I'm using random numbers to test the Model.

```
import torch
import numpy as np
from transformers import  ASTModel, ASTConfig
from torch.utils.data import DataLoader

configuration = ASTConfig()
model = ASTModel(configuration)

# (batch_size, channels, height, width)
dataset = torch.tensor(np.random.normal(size = (100,1,256,256)))

dataLoader = DataLoader(dataset,
                        batch_size=25,
                        pin_memory=True)

for data in dataLoader:
    model(torch.tensor(data).float())
```

the shape of the input_values is pulled from the [Official docs](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/audio-spectrogram-transformer#transformers.ASTModel.forward.input_values).

> `input_values (torch.FloatTensor of shape (batch_size, num_channels, height, width))`


What I see is the following error raised
```
Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [25, 1, 256, 1, 256]
```

It looks like the cause to me is [these lines](https://github.com/huggingface/transformers/blame/1670be4bdec19d5a8893f943bf78a8d9b3dc8911/src/transformers/models/audio_spectrogram_transformer/modeling_audio_spectrogram_transformer.py#L110-L113) in the forward pass of the ASTPatchEmbeddings

```
def forward(self, input_values: torch.Tensor) -> torch.Tensor:
        input_values = input_values.unsqueeze(1)
        input_values = input_values.transpose(2, 3)
        embeddings = self.projection(input_values).flatten(2).transpose(1, 2)
        return embeddings
```

When I step through with the debugger, I see that the unsqueeze and transpose commands are what is affecting the shape of the tensor.

### Expected behavior

I expect to see the model silently do a forward pass.

Comments:
cc @amyeroberts 

Hi @conradg, thanks for reporting this. 

I believe this is an with the documentation for `input_values` having the incorrect info. Looking at a [code example](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/audio-spectrogram-transformer#transformers.ASTModel.forward.example), the shape of the input array to the model is `(batch_size, max_length, num_mel_bins)`. Testing, on `main` the example runs successfully. I'll open a quick PR to update.
Update input values for docstring
Author: amyeroberts
Pull request: yes

# What does this PR do?

Updates docstring values for AST model for `input_values` that were incorrectly for pixel values. 

Fixes #22610 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
Asking for a sanity check from you, @sanchit-gandhi to make sure the description of the audio inputs is correct :) 

_The documentation is not available anymore as the PR was closed or merged._
[Image processors] Fix warnings
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR aims to reduce the number of warnings shown to the user for 2 use cases:

- DETR and friends' `max_size` argument which is deprecated => if this gets approved, I'll run fix-copies to fix the other DETR-based models
- the pattern matching warning when a configuration doesn't have an image processor in the config

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the review! After offline discussion, will close this PR in favor of a new one that just updates the warning levels. The removal of `max_size` can be done in a follow-up PR.
[`bnb`] Let's make serialization of int8 models possible
Author: younesbelkada
Pull request: yes

# What does this PR do?

Before this PR, it was not possible to save an 8bit model, or load an 8bit model from the Hub. This PR makes this feature possible. If this PR gets merged, users can upload 8bit models on the Hub and/or load 8bit models from the Hub, hence save 2x memory compared to half-precision models.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("bigscience/bloom-560m", device_map="auto", load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom-560m")

text = "Hello my name is"
inputs = tokenizer(text, return_tensors="pt").to(0)
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

>>> Hello my name is Nate, I am a professional photographer and I am a member of the
model.save_pretrained("./saved_int8")

model = AutoModelForCausalLM.from_pretrained("./saved_int8")

outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))
>>> Hello my name is Nate, I am a professional photographer and I am a member of the
```

Depends on https://github.com/TimDettmers/bitsandbytes/pull/159 

Let's put it as draft before I address the last TODOs and open questions & before https://github.com/TimDettmers/bitsandbytes/pull/159  gets merged.

## TODOs and open questions:

- ability to push `BitsAndBytesConfig`
- Do we want to save the serialized model under the name `pytorch_model.bin` ? I would say yes for simplicity reasons but we need to make sure that a user calls `from_pretrained` with `load_in_8bit`, hence add a warning if there is a `quantization_config.json` on the Hub repo + the user is not passing `load_in_8bit=True`.
- Force `load_in_8bit=True` if there is a `quantization_config.json` on the Hub repo?
- Update docs
- Update warnings
- Safety checkers for `bnb` versions
- Add a test to check if it works using sharded fp16 weights

cc @sgugger I left few open questions, would love to hear your thoughts on these!


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The design is not easy enough to use. If a user saves a quantized model and pushes to the Hub, it should work directly with `from_pretrained`. This is why I insisted that the quantization config should be saved inside the model config. This way you won't need to have the user pass `load_in_8_bit=True`, as you can read it from the config.

awesome ok, I'll work on that, so if there is a quantized config on the repo we should force-use `device_map=auto` & `load_in_8bit` in this case

The PR is ready for review @sgugger ! 
This PR is not mergeable before the bnb release of course

Thanks for the heads up! :D 
It should be much better now! For me the PR is ready for a review now