noamaanMulla-03/transformers-issues huggingface dataset:
transformers-cli serve not working
Author: jankrepl
Pull request: no

### System Info

System info
``` bash
- `transformers` version: 4.27.0
- Platform: macOS-12.3.1-arm64-arm-64bit
- Python version: 3.8.12
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


The following command fails for `transformers[serving]==4.27.0`
```bash
transformers-cli serve --task=fill-mask --model=bert-base-uncased
```
this is the traceback
```bash
Traceback (most recent call last):
  File "venv/bin/transformers-cli", line 8, in <module>
    sys.exit(main())
  File "venv/lib/python3.8/site-packages/transformers/commands/transformers_cli.py", line 54, in main
    service = args.func(args)
  File "venv/lib/python3.8/site-packages/transformers/commands/serving.py", line 49, in serve_command_factory
    nlp = pipeline(
  File "venv/lib/python3.8/site-packages/transformers/pipelines/__init__.py", line 976, in pipeline
    return pipeline_class(model=model, framework=framework, task=task, **kwargs)
  File "venv/lib/python3.8/site-packages/transformers/pipelines/base.py", line 773, in __init__
    self.model = self.model.to(device=device)
  File "venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1811, in to
    return super().to(*args, **kwargs)
  File "venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1126, in to
    device, dtype, non_blocking, convert_to_format = torch._C._nn._parse_to(*args, **kwargs)
RuntimeError: Device index must not be negative
```

### Expected behavior

However, downgrading to `transformers[serving]==4.26.1` fixes the issue

```bash
INFO:     Started server process [22054]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8888 (Press CTRL+C to quit)
```

Comments:
cc @Narsil 

This will be patched very soon, thanks for reporting!

> This will be patched very soon, thanks for reporting!

Thank you for fixing it so quickly:)
Fix LLaMATokenizer naming
Author: mbehm
Pull request: yes

# What does this PR do?

Simple fix for the naming of LLaMATokenizer class

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22217). All of your documentation changes will be reflected on that endpoint.

Ah ok, it was preventing me from loading a saved model because of the capitalization change so thought it was a mistake. In that case I'll be closing this, for anyone coming across the same issue ("Tokenizer class LLaMATokenizer does not exist or is not currently imported.") they can use my fork to load them for now.
fix code example in mgp-str doc
Author: wdp-007
Pull request: yes

# What does this PR do?
 Fix code example in mgp-str doc.


## Before submitting
- [√] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [√] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [√] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [√] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [√] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Positinal Encoding for T5 family of models
Author: SreehariSankar
Pull request: no
Labels: name: Feature request

### Feature request

Please create a hook to allow users to modify the T5 family of models and change the default positional embeddings to custom positional embeddings.

### Motivation

I am trying to build a T5 version that takes non-text input, and the traditional positional encodings are getting in the way, and there is no way to switch it off, or to make them learnable parameters, etc.
BART gives limited access to positional encodings, but T5 family gives nearly 0 access.

### Your contribution

If i knew where the positional encoding were calculated and added in to the input_ids, I could create this hook myself

Comments:
Hi @SreehariSankar, thanks for raising this issue. 

Questions on designing custom hook for modifying the models are better placed in the [forum](https://discuss.huggingface.co/). 

All of the code for the model, including producing the embeddings are in the modeling files e.g. [this one for T5](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py). Note: the T5 family of models do not use the same positional embedding logic as in the traditional transformer i.e. there isn't a fixed embedding for each position, but instead a relative position embedding.
Use `dash==2.8.1` for now for daily CI
Author: ydshieh
Pull request: yes

# What does this PR do?

Use `das==2.8.1` for now for daily CI. Currently daily CI jobs all fail, for example, [this job run](https://github.com/huggingface/transformers/actions/runs/4443525103/jobs/7800913606)

Issue reported [here](https://github.com/plotly/dash/issues/2460)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
LLaMA house-keeping
Author: sgugger
Pull request: yes

# What does this PR do?

This PR just groups a couple of nits I had on the LLaMA model PR, but didn't want to add there to merge the PR quickly. I have tested the conversion scripts on all four models and it works fine.

cc @zphang for information.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `ViTForMaskedImageModeling` doc example
Author: ydshieh
Pull request: yes

# What does this PR do?

Same #22185

Comments:
@ydshieh @alaradirik @fxmarty  The issue coming from #22152 was an oversight on my part about breaking changes. Perhaps we should revert that PR first and then agree how to introduce this change as it is intended to be added to other vision model? 

Oh, I thought it was a new model head! Indeed a breaking change there. Good for me to revert that PR, but would be nice to talk to Sylvain or Lysandre first (if you feel necessary). I will leave you judge.

Regarding a solution if we really want  to have this new attribute and the new name `MaskedImageCompletionOutput`, adding a new property (named `logits`) to `MaskedImageCompletionOutput` might be a way, but I didn't think about this deeply.

_The documentation is not available anymore as the PR was closed or merged._

Converted to draft for now

@ydshieh Yep - let's get @LysandreJik and @sgugger 's opinions. 

I think having the `logits` param is probably the best solution. As far as I know, it's very rare to check against the model output type itself. I believe `reconstruction` was chosen because of the `ImageSuperResolutionOutput` data class. As mentioned in the original PR - we probably do want a different model type to be returned as the documented shapes are incorrect. 

@alaradirik - could you open a PR to revert the changes? 

Actually, it's late for @alaradirik. I'll open the PR now.

Yes we can't rename the parameter in the outputs like that for a model that has been around for a bit. What is even more annoying is that the commit was in the release, so we will need to make a patch with the fix.

Close this PR as it's clear we will and have to definitely use the original `logits`.

Sorry for being late to comment, I added the `MaskedImageCompletionOutput` to replace the inaccurate `MaskedLMOutput `class used by the masked image modeling heads (ViT and DeiT). Neither of these models have any checkpoints on the hub as mentioned in #22152 .  Swin's MIM head has its own output class but no fine-tuned checkpoints for the MIM task either.  

With that said, ViT and Swin's MIM heads are implementations of [SimMIM](https://arxiv.org/abs/2111.09886) and SimMIM have recently released fine-tuned checkpoints for these two models (as opposed to the base model weights on the hub for Swin MIM head). I'm planning to convert these checkpoints and add a `masked-image-completion` pipeline after @sheonhan merges the ICT PR (a contemporary, better performing MIM model). It'd be great to add an output class and fix inaccurate class output (listed as a language model in the docs) before that. While logits is not an accurate output name in this case as the model returns full reconstructed images, I could replace `reconstruction` with `logits` and open a new PR.

What do you think @amyeroberts @sgugger?
CC @LysandreJik @ydshieh
fix AutoTP in deepspeed could not work for bloom
Author: sywangyi
Pull request: yes

# What does this PR do?

Fixes # (issue)
fix AutoTP in deepspeed could not work for bloom

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada

- deepspeed: HF Trainer: @stas00



Comments:
should work with https://github.com/microsoft/DeepSpeed/pull/3035

@sgugger please help review


@yao-matrix

_The documentation is not available anymore as the PR was closed or merged._

Actually, just checked the modeling file and this function is only used in this class, so it would be cleaner to just make it a method. Could you update your PR in that direction?

@sgugger I see code like "from transformers.models.bloom.modeling_bloom import build_alibi_tensor" in petals, if we make this a method, the petals code needs to be changed as well. may happen to other repo that use bloom as well.

Ok so let's keep it as a function in that module. I'd still prefer a real method (that directly returns the result of the function) to setting a function attribute like this if you don't mind.

@sgugger update the PR.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22196). All of your documentation changes will be reflected on that endpoint.
Hotfix for natten on CircleCI
Author: ydshieh
Pull request: yes

# What does this PR do?

Hotfix for natten on CircleCI.

The PR CI in #22204 run with `natten` with the version `0.14.4` which run successfully. However, when I merged that PR  into `main`, natten 0.14.5 is released and cause some issues.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Here is the update regarding this issue

https://github.com/SHI-Labs/NATTEN/issues/23#issuecomment-1473865224
fix(docs): fix task guide links in model docs
Author: Seb0
Pull request: yes

# What does this PR do?

Fixes broken links for task guides in model docs 

Fixes # (issue)

see above

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@MKhalusova 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Revert "Use `dash==2.8.1` for now for daily CI"
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#22227

New version [dash 2.9.1](https://github.com/plotly/dash/releases/tag/v2.9.1) works with our CI. Tested and it works.
We no longer need the change in #22227


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22233). All of your documentation changes will be reflected on that endpoint.
[trainer] param count for deepspeed zero3
Author: stas00
Pull request: yes

As reported in https://github.com/huggingface/transformers/issues/22179 the trainer code doesn't handle the sharded models correctly in reporting "the Number of trainable parameters" - I'm not sure if FSDP models have the same issue.

This PR fixes this situation with Deepspeed ZeRO3 which otherwise reported a count of 0.

Fixes: https://github.com/huggingface/transformers/issues/22179

Comments:
_The documentation is not available anymore as the PR was closed or merged._

An explanation is needed here. 

The Deepspeed team had to invent their own tensor substitute since 2 years ago nothing of a kind existed in pytorch. They had to replace tensors with placeholders to be able to support sharded tensors.

The meta tensors were added just recently so they are looking at possibly switching to those.

The API I used in this PR is not public per-se. And the "clean" way would be to gather tensors and then get their normal `t.numel()` - but this is extremely wasteful and expensive memory and time-wise. So I hacked it to get the internal equivalent to make it almost instant.

I'm not planning on leaving it this way and asking for deepspeed to provide an efficient method to return the sizes w/o me using a non-public API.

There are many other hidden issues wrt this tensor substitution that impacts only ZeRO stage 3 https://github.com/microsoft/DeepSpeed/issues/2650 - and yesterday I have discovered at least one bug in our examples because of that, while debugging the user report that lead to this PR. All examples resize the embedding under zero3 because their check if the vocab is larger than embedding size always returns True, since the embed size is reported to be of size 0, because it's not gathered :(

I'm working on ensuring that the Deepspeed addresses this issue because it's subtle and very problematic. 

Please let me know if you're OK with merging this now that you know more details. I can also easily recode it to gather tensors first, but it'd be very inefficient.
When I use Trainer with Deepspeed, the Number of trainable parameters is 0
Author: noob-ctrl
Pull request: no

The version information is as follows:
- Deepspeed.  0.8.1
- transformers. 4.26.1

## Problem
When I use Trainer with Deepspeed, the Number of trainable parameters is 0. Like this:

![image](https://user-images.githubusercontent.com/63763578/225277324-3650bbea-78f7-493a-97a2-3f9ef0bdcd5a.png)

And it happens when using zero3. When I use zero2, it does not have this problem.




Comments:
cc @stas00 

Thank you for the report, @noob-ctrl 

Please let me know if this fix works for you: https://github.com/huggingface/transformers/pull/22193


@stas00 Hi, it works now. Thank you!

Thank you for testing, @noob-ctrl - the PR has been merged.
[Trainer] Use of inspect for model.forward with torch.compile
Author: jaketae
Pull request: no

## Issue

In `trainer`, the `inspect` module is used to remove extraneous dataset columns.

https://github.com/huggingface/transformers/blob/60d51ef5123d949fd8c59cd4d3254e711541d278/src/transformers/trainer.py#L722-L728

However, `torch.compile` modifies the signature of the forward function of the original model, so `inspect.signature` is unable to correctly identify input arguments.

## Possible Solution

If there is a way to recover the original arguments, that would be the cleanest solution. Otherwise, we could check if the model is compiled and modify the logic of the `_set_signature_columns_if_needed` function appropriately, with perhaps added logging to the user that columns won't be dropped due to using `torch.compile`. 

## System Information
* Python 3.8
* PyTorch 2.0
* transformers 4.27.1

### Who can help?

@stas00 @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
>>> import inspect, torch; from transformers import AutoModel
>>> model = AutoModel.from_pretrained("roberta-base")
>>> inspect.signature(model.forward)
<Signature (input_ids: Union[torch.Tensor, NoneType] = None, attention_mask: Union[torch.Tensor, NoneType] = None, token_type_ids: Union[torch.Tensor, NoneType] = None, position_ids: Union[torch.Tensor, NoneType] = None, head_mask: Union[torch.Tensor, NoneType] = None, inputs_embeds: Union[torch.Tensor, NoneType] = None, encoder_hidden_states: Union[torch.Tensor, NoneType] = None, encoder_attention_mask: Union[torch.Tensor, NoneType] = None, past_key_values: Union[List[torch.FloatTensor], NoneType] = None, use_cache: Union[bool, NoneType] = None, output_attentions: Union[bool, NoneType] = None, output_hidden_states: Union[bool, NoneType] = None, return_dict: Union[bool, NoneType] = None) -> Union[Tuple[torch.Tensor], transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions]>
>>> opt_model = torch.compile(model)
>>> inspect.signature(opt_model.forward)
<Signature (*args, **kwargs)>
```

### Expected behavior

The trainer should only drop unused columns, not all of them (which is what happens when it incorrectly registers `args` and `kwargs` as input arguments).

Comments:
I realized that the proper way to use `torch.compile` with `Trainer` is through the `training_args.torch_compile` flag. Using the flag didn't cause the issue (I was manually compiling it outside the trainer). Closing, thanks!
LLaMa tokenizer is labelled incorrectly when called.
Author: GamerUntouch
Pull request: no

### System Info

Most of the transformers functions call for "LlamaTokenizer", but the actual classes (found under transformers/models/llama) are labelled as "LLaMaTokenizer"



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

There are no steps, changing the class names fixes the loading.

### Expected behavior

Either fix the class names or fix the functions that call for them.

Comments:
🐛 Saving TF model : Expected Operation, Variable, or Tensor, got None
Author: astariul
Pull request: no

# 🐛 Bug

I'm training TFElectraModel. Training goes well, but when I try to save the model, I met the following error :

`TypeError: Expected Operation, Variable, or Tensor, got None`

## Minimal reproducible example

```
!pip install transformers

from transformers import TFElectraModel

model = TFElectraModel.from_pretrained("google/electra-base-discriminator")
!mkdir swag
model.save("swag")
```

## Related

#2336
https://github.com/tensorflow/tensorflow/issues/35432


Comments:
Hi! We prefer using our own API for this, using `from_pretrained` which also takes care of the configuration:

```py
from transformers import TFElectraModel

model = TFElectraModel.from_pretrained("google/electra-base-discriminator")
!mkdir swag
model.save_pretrained("swag")
```

Is there a reason why you would rather use `.save` rather than `.save_pretrained`?

I was using `.save()` because I made a custom model, with additional layers on top of Electra, and I want to save these layers as well.

Finally, I simply sub-classed `TFElectraPreTrainedModel` and add my custom layers there, so I can use `.save_pretrained`.

Thanks for the help !

I am also facing similar issue.
The reason for using save method is i want to host the saved model using tensorflow model server

@nirajkale did you manage to solve this? I'm facing the same issue with other pre-trained.
Wrong "view source" links on main docs
Author: gau-nernst
Pull request: no

### System Info

NA

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When viewing the latest docs, "view source" link expansion leads to an invalid GitHub link.

e.g. https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertConfig

View source link: https://github.com/huggingface/transformers/blob/v4.27.0/src/transformers/models/bert/configuration_bert.py#L72

Since `v4.27.0` tag does not exist, GitHub reports an invalid link. It should be `main` instead. I believe this is a problem of configuring the auto-generate docs.

### Expected behavior

Show correct link to source code.

Comments:
cc @stevhliu 

The tag will be out in a couple of hours. The doc for the v4.27.0 release was pushed last night and the rest of the release will follow this morning. Thanks for catching this so quicly!