noamaanMulla-03/transformers-issues huggingface dataset:
add zh translation for installation
Author: yyLeaves
Pull request: yes

# What does this PR do?
- Add zh (Chinese) translation for installation section #20095
- Add installation to _toctree.yml


## Who can review?
Documentation: @sgugger @stevhliu and @MKhalusova
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26084). All of your documentation changes will be reflected on that endpoint.

> Thanks for your translation! 🤗
> 
> For some reason, the second-level headings aren't being recognized so we're going to look into that. Once it gets resolved, we'll be free to merge!

Noted with thanks! @stevhliu 

The issue should be resolved now, thanks again for your contribution and patience!
Transformers breaks when running setup.py
Author: AdamLouly
Pull request: no

### System Info

Running `python setup.py install` in transformers folder throws this error:

error: huggingface-hub 0.17.2 is installed but huggingface_hub<0.17,>=0.16.4 is required by {'tokenizers'}


This starts happening after merging this PR where line 175 was changed from:
from    "tokenizers>=0.11.1,!=0.11.3,<0.14",
to    "tokenizers>=0.14,<0.15",

https://github.com/huggingface/transformers/issues/23909

when I change it back manually it works fine.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Running `python setup.py install` in transformers folder

### Expected behavior

error: huggingface-hub 0.17.2 is installed but huggingface_hub<0.17,>=0.16.4 is required by {'tokenizers'}


Comments:
A quick fix: update `huggingface_hub` . On our side we should update the dependency requirement. Thanks for reporting! 

@ArthurZucker  have you tested re installing Transformers after this update?
I tested and its still failing.

huggingface-hub>=0.16.4,<1.0 won't solve the issue: 
huggingface_hub<0.17,>=0.16.4 is required by {'tokenizers'}

because its installing huggingface_hub 0.17.4 and that will break it.


I think tokenizers has to be updated at this point, since it's not using the latest dependency. cc @Narsil any reason why we limited to HF hub 0.17? 

@ArthurZucker Any updates on this?

Yep, this [PR](https://github.com/huggingface/tokenizers/pull/1344) was merged in tokenizers, we just need a release and this will be adressed
BLIVA
Author: rafaelpadilla
Pull request: no
Labels: name: New model

### Model description

BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions

BLIVA performs VQA tasks. It is an augmented version of InstructBLIP with Visual Assistant. It incorporates the query embeddings from InstructBLIP and also directly projects encoded patch embeddings into the LLM.

"Empirical evidence demonstrates that our model, BLIVA, significantly enhances performance in processing text-rich VQA benchmarks (up to 17.76\% in OCR-VQA benchmark) and in undertaking typical VQA benchmarks (up to 7.9\% in Visual Spatial Reasoning benchmark), comparing to our baseline InstructBLIP."


### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Official repo: https://github.com/mlpc-ucsd/BLIVA
Paper: https://arxiv.org/abs/2308.09936

Comments:
Inconsistency in BertFastTokenizer
Author: Towdo
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-4.18.0-425.19.2.el8_7.x86_64-x86_64-with-glibc2.2.5
- Python version: 3.8.6
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1+cu117 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:  No

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Program to reproduce:

from transformers import BertTokenizer, BertTokenizerFast

A = [1, 2, 3]
B = []

tokenizer = BertTokenizer.from_pretrained("prajjwal1/bert-tiny")
fast_tokenizer = BertTokenizerFast.from_pretrained("prajjwal1/bert-tiny")

X = tokenizer.build_inputs_with_special_tokens(A, B)
Y = fast_tokenizer.build_inputs_with_special_tokens(A, B)

print(X) --> [101, 1, 2, 3, 102, 102]
print(Y) --> [101, 1, 2, 3, 102]

### Expected behavior

X == Y

Comments:
Update logits_process.py docstrings to clarify penalty and reward cases
Author: larekrow
Pull request: yes

This PR fixes point 3 of #25970 by clarifying the penalty and reward cases for `RepetitionPenaltyLogitsProcessor` and `EncoderRepetitionPenaltyLogitsProcessor` within the docstrings.

`EncoderRepetitionPenaltyLogitsProcessor` may require further iterations as the class name does not accurately reflect what the class is doing.

@gante 

Comments:
Hey @gante @ArthurZucker! This is ready for review 😃
Use a custom SummaryWriter to record training parameters
Author: young-chao
Pull request: no

When I use trainer, how should I use a custom SummaryWriter to record training parameters, such as grad_norm, loss_scale, etc.  I don’t see the setting options in the trainer parameters.

Comments:
Hello @young-chao, the `Trainer`'s initialization method accepts a `callbacks` argument which is a list of `TrainerCallback`s.

You can pass a `TensorBoardCallback` to that list that you can initialize with your own `tb_writer`.

See its definition here: https://github.com/huggingface/transformers/blob/0ac3875011d32dc85e0e83970507e3afe8f0febb/src/transformers/integrations/integration_utils.py#L579



> Hello @young-chao, the `Trainer`'s initialization method accepts a `callbacks` argument which is a list of `TrainerCallback`s.
> 
> You can pass a `TensorBoardCallback` to that list that you can initialize with your own `tb_writer`.
> 
> See its definition here:
> 
> https://github.com/huggingface/transformers/blob/0ac3875011d32dc85e0e83970507e3afe8f0febb/src/transformers/integrations/integration_utils.py#L579

I will try it, thanks for your reply
tf trainer dataset cardinality issue - potentially a bug
Author: ydshieh
Pull request: no

In `trainer_tf.py`, line 138, we have

    self.num_train_examples = self.train_dataset.cardinality(self.train_dataset).numpy()

in the method 

    def get_train_tfdataset(self) -> tf.data.Dataset:

However, in the official tf documenation [cardinality](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#cardinality), it is defined as:

    cardinality()

which has no argument.

I got the following error

```
    File "/home/imo/Desktop/transformers/src/transformers/trainer_tf.py", line 138, in get_train_tfdataset
      self.num_train_examples = self.train_dataset.cardinality(self.train_dataset).numpy()
    TypeError: cardinality() takes 1 positional argument but 2 were given
```
    
I think the current version on master is a bug, which should be changed to 

    self.train_dataset.cardinality().numpy()


Could you confirm, @jplu? And if it is a bug, let's fix it. Thank you.

Comments:
I can confirm! Good catch!

I resume the work on creating `test_trainer_tf.py`, I promise I will finish it this time. After that, it might be easier to catch the errors in `tf_trainer.py`.

I take care of this!

OK, @jplu . Thank you for letting me know about it (I did some check and didn't found it on master, so I thought it was not done yet).

I encountered this issue while I was running the model.

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in __getattr__(self, item)
    265         try:
--> 266             return self.data[item]
    267         except KeyError:

KeyError: 'cardinality'

During handling of the above exception, another exception occurred:
```

Below is the code:

```
from transformers import TFAutoModelForSequenceClassification

model = TFAutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased")

model.compile(optimizer=optimizer)  # No loss argument!

from transformers import TFTrainer, TFTrainingArguments


training_args = TFTrainingArguments(
    output_dir="./sentiment_model",
    per_device_train_batch_size=32,
    per_device_eval_batch_size=32,
    num_train_epochs=3,
    evaluation_strategy="steps",
    eval_steps=500,  # Adjust as needed
    save_total_limit=2,
)

trainer = TFTrainer(
    model=model,
    args=training_args,
    train_dataset=train_tokenized,
    eval_dataset=val_tokenized,
)

trainer.train() # <- where the error occured
```

It doesn't look the same issue as the original one. Could you open a new issue, and provide a simple/small dataset along the code to show the issue. Thank you.

@ydshieh https://github.com/huggingface/transformers/issues/26632#issue-1929838712
Fix failing `MusicgenTest .test_pipeline_text_to_audio`
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix failing `MusicgenTest .test_pipeline_text_to_audio`

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @sanchit-gandhi 

I have updated the tiny models on the Hub (config/model file) manually. I have to update the commit sha info. so the pipeline tests will take this new versions.

Failing test passes now.

I will rework the tiny model creation script in a separate PR.

@LysandreJik Ready for a review and we might have a green CI during the weekend :-) 🙏
fix RoPE t range issue for fp16
Author: rui-ren
Pull request: yes

#### Issue
Sometimes training with `fp16`, the `dtype` of `self.inv_freq` will be changed from `fp32` to `fp16`. This scenario will cause the position `t` to use dtype of `fp16`, like
```
t = torch.arange(seq_len, device=device, dtype=torch.float16)
```

After converting to onnx graph, however, Range Ops  in `onnx` do not support `fp16` as [here](https://github.com/onnx/onnx/blob/e11dacfa9930eafd3b34391ef5422d09ba9896dc/onnx/defs/generator/defs.cc#L488-L557)

#### Update

Use the below to avoid this scenario

```
t = torch.arange(seq_len, device=device).to(dtype)
```



Comments:
Seems fair, WDYT @Rocketknight1 ?

This will cause outputs to change numerically a bit when running in `float16` or `bfloat16` because `freqs` will be calculated in higher precision. The memory/performance impact probably wouldn't be huge, but this might introduce a small deviation for models that were trained in those precisions! Let me run some tests before I approve it.

After testing, outputs seem equivalent for `bfloat16` models so I'm happy to approve this!

@rui-ren let me know if you want to add anything else to this PR, or if you're happy for me to merge it now!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26602). All of your documentation changes will be reflected on that endpoint.

@Rocketknight1  Please merge this PR. Thank you for your review. 

Done. Thanks for a clean and helpful PR @rui-ren!
Add classes for Multi-label classification models? 
Author: gsnidero
Pull request: no
Labels: name: Feature request

# 🚀 Feature request

It would be nice to add classes for Multi-label classification models to the library.

## Motivation

In my projects, I need to perform Multi-label classification. This problem setting is quite common in real-life modelling. 

## Your contribution

As there isn't a class implemented for this in the library, I have implemented my own. I have modified, for example, BertForSequenceClassificationton to make it usable for multi-label modelling. Then, I train the model with the Trainer class as usual.

Is there any interest or plan to add this from the library maintainers? If so, I would be happy to collaborate or start working on a PR.

Thanks


Comments:
Hi @LysandreJik is this feature request being worked on by someone? If not, I would love to take it up. Thanks!

Hello! 

There is existing support for this thanks to the `problem_type` configuration attribute for some Sequence classification models, see here: https://huggingface.co/transformers/main_classes/configuration.html#transformers.PretrainedConfig

(search for `problem_type`)

It could be better documented though by having an example for each model that supports this/more complete documentation on the model themselves. Would you like to try your hand at it?

Hi,

It seems like the documentation for this feature has already been merged so I think this issue should be closed. 


Indeed! Thanks @astern21
Cannot save/load tokenizer without special tokens
Author: jonathanasdf
Pull request: no

### System Info

main

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained('stabilityai/FreeWilly2')
tokenizer.pad_token_id = None
tokenizer.eos_token_id = None
tokenizer.bos_token_id = None
tokenizer.unk_token_id = None
tokenizer.save_pretrained('/tmp/tok_test')
transformers.AutoTokenizer.from_pretrained('/tmp/tok_test')
```

```
transformers/models/llama/tokenization_llama_fast.py", line 152, in update_post_processor
    single = f"{(bos+':0 ') * self.add_bos_token}$A:0{(' '+eos+':0') if self.add_eos_token else ''}"
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
```

### Expected behavior

This works on transformers==4.33.1

I use a separate tokenizer for user input so it doesn't convert "special tokens" in unsanitized input. Maybe there's a different way to do this than setting all the special tokens to None?

Comments:
Thanks for reporting fixing now 😉
[`LlamaTokenizerFast`] Adds edge cases for the template processor   
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #26605 by making sure the LlamaTokenizerFast handles the cases when bos or eos is None and `add_bos_token` is set to `True` by raising an error. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yep rebased on main to remove them!
Add DALL-E: Zero-Shot Text-to-Image Generation
Author: slavakurilyak
Pull request: no
Labels: name: New model

# 🚀 Feature request

Please add DALLE-E model to huggingface's Transformers library.

1. [Announcement](https://openai.com/blog/dall-e/)
2. [Abstract](https://arxiv.org/abs/2102.12092v2)
3. [Paper](https://arxiv.org/pdf/2102.12092v2.pdf)
4. Code:
    - [openai/DALL-E](https://github.com/openai/DALL-E) (official)
    - [lucidrains/DALLE-pytorch](https://github.com/lucidrains/DALLE-pytorch) ([Colab](https://colab.research.google.com/drive/1dWvA54k4fH8zAmiix3VXbg95uEIMfqQM?usp=sharing))

## Motivation

> DALL·E is a 12-billion parameter version of [GPT-3](https://huggingface.co/transformers/model_doc/gpt.html) trained to generate images from text descriptions, using a dataset of text–image pairs
>
> We (Open AI) decided to name our model using a portmanteau of the artist Salvador Dalí and Pixar’s WALL·E.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

+1

Does dall-e mini currently added to transformers?
Currently, it doesn't `eBart` is not recognized in transformers library.

cc @patil-suraj who's currently working on making it easier to use from transformers

There are dozens of DALL-E models currently listed on the Hugging Face site. Unless this is a specific variant/implementation that has yet to be added, it seems this issue can be closed.
docs(zh): review and punctuation & space fix
Author: wfjsw
Pull request: yes

# What does this PR do?

Briefly reviewed the Simplified Chinese documentation.

- Fix punctuation and spacing issues.
- Fixes #26603 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stevhliu

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26627). All of your documentation changes will be reflected on that endpoint.
[Docs] Weird rendering of a code block
Author: wfjsw
Pull request: no

See 
https://github.com/huggingface/transformers/blob/3e203f92bed937fa13c35adee1bdc45a92d18e61/docs/source/zh/quicktour.md?plain=1#L459

https://huggingface.co/docs/transformers/main/zh/quicktour#:~:text=dataset%20%3D%20dataset.map(tokenize_dataset%2C%20batched%3DTrue)

![image](https://github.com/huggingface/transformers/assets/2220320/14a17760-3fbc-4187-896f-cf4e83d0b81f)

I cannot come up with a valid reason why it is rendered in this way.

Comments:
I think it's because there's an extra line between where the function is defined and where we map it. Removing the extra line should fix it:

```diff
>>> def tokenize_dataset(dataset):
...     return tokenizer(dataset["text"])
# remove blank line here

>>> dataset = dataset.map(tokenize_dataset, batched=True)
```
[New Model] Retrieval-based Voice Conversion
Author: wfjsw
Pull request: no
Labels: name: New model

### Model description

Type: Audio-to-Audio
Framework: Pytorch / ONNX
License: MIT


### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Source Code: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
Pretrained Models by Author: https://huggingface.co/lj1995/VoiceConversionWebUI/tree/main
Models in the wild: https://www.weights.gg/

Comments:
cc @sanchit-gandhi 

Hey @wfjsw, thanks for brining this model to our attention! We have recently been trying to push for model on the hub and have as much support as we can there. This is the recommended way of adding new models, since it's easier to integrate them and they can be made available as soon as the code is ready. Here is a [tutorial](https://huggingface.co/docs/transformers/custom_models) on this integration process. I think RVC could make for a great addition to the Hub!
Added OnnxConfig for MPNet models
Author: grafail
Pull request: yes

# What does this PR do?

This PR Adds `OnnxConfig` for MPNet based models. 

In order for the conversion to be compatible with some older versions of PyTorch (In my case 1.11.0), I had to add `*args`, to the signature of `forward` functions that were using `**kwargs`. This was because `_decide_input_format` in PyTorch considered `**kwargs` as a normal parameter so it added an additional (unexpected) positional argument (with value None).

https://github.com/pytorch/pytorch/blob/33bb8ae350611760139457b85842b1d7edf9aa11/torch/onnx/utils.py#L793

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #16308 
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@ChainYo 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19009). All of your documentation changes will be reflected on that endpoint.

@ChainYo As I mentioned in the pr description, a function inside torch (with version 1.11) had an issue parsing the signature correctly and provided kwargs as an extra positional argument. Latest version worked though, but I added the fix for compatibility.

> @ChainYo As I mentioned in the pr description, a function inside torch (with version 1.11) had an issue parsing the signature correctly and provided kwargs as an extra positional argument. The latest version worked, though, but I added the fix for compatibility.

Sorry I didn't read the PR carefully. Let's see if it doesn't bother the original implementation by bringing any unexpected behaviors. It's okay, then. :hugs: 

Is there anything I can do to help with this?

> Is there anything I can do to help with this?

We are waiting for a reviewer to get feedback and see what's next!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This is still an issue