noamaanMulla-03/transformers-issues huggingface dataset:
Add VGCN-BERT model
Author: Louis-udm
Pull request: no
Labels: name: New model

### Model description


HI, I am the author of [VGCN-BERT paper](https://arxiv.org/abs/2004.05707), the original implementation is in my gitlab [vgcn-bert repo](https://github.com/Louis-udm/VGCN-BERT), but recently I updated the algorithm and implemented a new version for integrating in Transformer.

> Much progress has been made recently on text classification with methods based on neural networks. In particular, models using attention mechanism such as BERT have shown to have the capability of capturing the contextual information within a sentence or document. However, their ability of capturing the global information about the vocabulary of a language is more limited. This latter is the strength of Graph Convolutional Networks (GCN). In this paper, we propose VGCN-BERT model which combines the capability of BERT with a Vocabulary Graph Convolutional Network (VGCN). Local information and global information interact through different layers of BERT, allowing them to influence mutually and to build together a final representation for classification. In our experiments on several text classification datasets, our approach outperforms BERT and GCN alone, and achieve higher effectiveness than that reported in previous studies.

I actually finished the integration of my new version and opened the PR. This new VGCN-BERT algorithm has the following improvements:
- Greatly speeds up the calculation speed of embedding vocabulary graph convolutinal network (or Word Graph embedding). Taking CoLa as an example, the new model only increases the training time by 11% compared with the base model
- Updated subgraph selection algorithm.
- Currently using DistilBert as the base model, but it is easy to migrate to other models.
- Provide two graph construction methods in vgcn_bert/modeling_graph.py (the same NPMI statistical method as the paper, and the predefined entity-relationship mapping method)

I hope that after integrating into transformers, someone can discover some more practical use case. I am ashamed to say that I have not discovered too much real use cases myself, mainly because the word-grounded graph obtained through statistical methods has limited improvement on the LLM model. I think its potential application should be when there are specific/customized graphs that need to be integrated into LLM.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

https://arxiv.org/abs/2004.05707
https://github.com/Louis-udm/VGCN-BERT

Comments:
The new implement is here:
https://huggingface.co/zhibinlu/vgcn-bert-distilbert-base-uncased
Default Models of the pipeline function
Author: DiogenesBR
Pull request: no

I made issue a on the NLP Course (https://github.com/huggingface/course/issues/561):

> In the video:
> Chapter 1 Live Session with Sylvain
> https://youtu.be/aV4wfnIakSQ?t=928
> 
> There a question of what are the default models in the pipeline library.
> His answer is to look at:
> https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/__init__.py
> 
> I think that this information should be in the chapter 1, also on the documentation

Then I was asked to make a Issue here too.

After looking the Transformers Agents also have this problem
![image](https://github.com/huggingface/transformers/assets/16890195/e98316e6-75bd-48a6-8d36-bd042347099c)


For a instance Speech to text uses Whisper but it don't say what exact version of Whisper

There more than 3000 versions of Whisper on the Models Directory
https://huggingface.co/models?search=whisper

Comments:
Hi @DiogenesBR, thanks for raising this issue! 

Similar to answer in the linked video, at the moment this info is best found by looking at the source code. For example, for the [speech-to-text tool](https://github.com/huggingface/transformers/blob/918a06e25dfd6f79a20b6f07f63598c71e440161/src/transformers/tools/speech_to_text.py#L22), the checkpoint used is [openai/whisper-base](https://huggingface.co/openai/whisper-base). 

Would you be interested in opening a PR to add this information? 

cc @MKhalusova @stevhliu 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add albert resources
Author: elabongaatuo
Pull request: yes

# What does this PR do?

Fixes   #20055


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@stevhliu




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[SpeechEncoderDecoderModel] Correct Encoder Last Hidden State Output
Author: sanchit-gandhi
Pull request: yes

Corrects the `Seq2SeqLMOutput` for the PyTorch SpeechEncoderDecoderModel to return the encoder hidden-state **after** the (optional) encoder-decoder projection layer `enc_to_dec_proj`. This brings the model in alignment with the change made in #16581 to the equivalent Flax model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[JAX] Replace all jax.tree_* calls with jax.tree_util.tree_*
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Newer versions of JAX moved all tree utility methods to `jax.tree_util` and emit warnings for using the old locations under `jax`. This PR refactors all `jax.tree_*` calls to `jax.tree_util.tree_*`, thus disabling the warnings with no changes to the code functionality.

```python
import jax

inputs = jax.numpy.ones(10)

# Define identity function with tree_map -> FutureWarning 
jax.tree_map(lambda x: x, inputs)

# Define identity function with tree_util.tree_map -> no warning!
jax.tree_util.tree_map(lambda x: x, inputs)
```

```
FutureWarning: jax.tree_map is deprecated, and will be removed in a future release. Use jax.tree_util.tree_map instead.
```

All `jax.tree_*` calls in Flax were updated in this PR: https://github.com/google/flax/pull/2325. However, the `FutureWarning` message will still be raised if using newer versions of JAX and older versions of Flax. 

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Sorry for being so late here! Let's merge it
[ASR Examples] Update 'tasks' for model card
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

The task 'automatic-speech-recognition' was added to the model card creator in #19985. This PR updates all the ASR examples scripts accordingly.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[modelcard] Update for ASR
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Updates the modelcard to include ASR in the task mapping and task-tag-to-name mapping, and the WER in the metric tags.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[pipeline] fix Whisper test
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes Whisper pipeline test.

Previously, we suppressed the hyphen and apostrophe tokens from Whisper generation, meaning they were alway predicted with zero probability. This meant that these tokens could never be predicted. With the Hub PR https://huggingface.co/openai/whisper-large/discussions/12, these tokens were removed from the set of suppressed tokens, meaning they can now (correctly) be predicted with non-zero probability.

We get the correct contraction now in the Italian prediction: "allo universo" -> "all'universo"

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @ydshieh @sgugger 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Test already fixed in https://github.com/huggingface/transformers/pull/20588
[S2T, Whisper] Add copied from statements
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Adds 'copied from MBart' statements to Speech2TextEncoderLayer and Speech2TextDecoderLayer. 

Since the WhisperEncoderLayer and WhisperDecoderLayer are copied from Speech2Text, these classes are updated with 'copied from MBart' statements to minimise the chain of 'copied from' statements.

Previously:
* (mBart -> ) Speech2Text -> Whisper

Updated:
* mBart -> Speech2Text
* mBart -> Whisper

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[SpeechT5HifiGan] Handle batched inputs
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Modifies the SpeechT5 HiFiGAN model to accept batched inputs. This PR is **not** a breaking change:
* If the spectrogram inputs are un-batched, the waveform outputs are un-batched (as before)
* If the spectrogram inputs are batched, the waveform outputs are batched (new)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

(failing test is unrelated)
[Examples] Generalise run audio classification for log-mel models
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Currently, `run_audio_classification.py` hard-codes the model input name to `input_values` in the pre-processing function. This makes it compatible with Wav2Vec2-style CTC models, but not other speech models that use log-mel `input_features` (e.g. Whisper or AST).

We adopt the same strategy that we use in `run_speech_recognition_seq2seq.py` and set this to the correct model input name (based on the feature extractor's attribute `.model_input_names`):
https://github.com/huggingface/transformers/blob/1d4b79785263077f9f09ddde5a75ae4f116e85d7/examples/pytorch/speech-recognition/run_speech_recognition_seq2seq.py#L419

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[SpeechT5] Fix HiFiGAN tests
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

SpeechT5HiFiGAN tests added in the PR #21702 failed in the CI daily run: https://github.com/huggingface/transformers/actions/runs/4248850253/jobs/7388500325

This PR fixes the torch devices ✅

Comments:
_The documentation is not available anymore as the PR was closed or merged._