noamaanMulla-03/transformers-issues huggingface dataset:
Added AnyPrecisionAdamW as an optimizer
Author: Zeesky-code
Pull request: yes

Added AnyPrecisionAdamW as an optimizer
- Related Issue 
#18827

@stas00 







Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for trying, @Zeesky-code - but that won't do anything ;)

I guess my instructions weren't very instructive I've just pointed to a few places where one would start looking at mimicking the integration of other optimizers, my apologies if it wasn't obvious.

So in this case it'd follow the path of `adamw_torch` , as it's the nearest similar optimizer.

and as I said the key to the PR is tests and documentation. Again checking the existing tests and working from there is what's needed.

If it's too much and you're no longer interested please don't hesitate to comment in the feature request issue that it's open for grabs again. If you want to continue, that is great too - please don't hesitate to ask questions if any.

p.s. it might help to look at the previous PRs that added new optimizers, e.g. find the PR that added `adamw_bnb_8bit` - that could be a good model to copy from. And you can see the scope of work that needs to be done.




Ohh, I've taken a look at the PR that added adamw_bnb_8bit and I'm afraid I don't think I'll be able to work on this. 
I'll close this PR and let others know the issue is still available to work on.

Thank you😅

> Thank you for trying, @Zeesky-code - but that won't do anything ;)
> 
> I guess my instructions weren't very instructive I've just pointed to a few places where one would start looking at mimicking the integration of other optimizers, my apologies if it wasn't obvious.
> 
> So in this case it'd follow the path of `adamw_torch` , as it's the nearest similar optimizer.
> 
> and as I said the key to the PR is tests and documentation. Again checking the existing tests and working from there is what's needed.
> 
> If it's too much and you're no longer interested please don't hesitate to comment in the feature request issue that it's open for grabs again. If you want to continue, that is great too - please don't hesitate to ask questions if any.
> 
> p.s. it might help to look at the previous PRs that added new optimizers, e.g. find the PR that added `adamw_bnb_8bit` - that could be a good model to copy from. And you can see the scope of work that needs to be done.



Thank you for an honest evaluation, @Zeesky-code - and much appreciated for trying!
Add Trainer to quicktour
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR makes some edits to the `pipeline` section to focus less on all the tasks (and their definitions) it is capable of. Users probably only need a general representative idea of what it can do, and then they're more interested in diving into how to use the pipeline.

I also added a brief section on the `Trainer` here about the basic parameters it accepts and a small explanation of how to customize the training loop behavior to keep the quick tour short. I think the `Trainer` is pretty important to include since a lot of users use it for training, and we also use it in our finetune guides.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Decouple inference from preprocessing and postprocessing steps from pipeline
Author: jspablo
Pull request: no

### Feature request

Each pipeline under the pipeline abstraction has a preprocessing, forward and postprocessing method. In order to use any pipeline it should be initiliazed providing a model so preprocessing and postprocessing methods cannot be easily used by other services.
It would be great to provide the model just for the forward method, so methods that don't need it can be use independently.

For instance, if I want to serve a [zero shot classification pipeline](https://huggingface.co/docs/transformers/v4.21.0/en/main_classes/pipelines#transformers.ZeroShotClassificationPipeline) the model is a required parameter so I can't just split the preprocess logic to send it to an inference service hosting the model itself.

### Motivation

When deploying models using a pipeline I would like to use an inference server and keep any preprocessing and postprocessing logic as a different service. The goal is to improve throughput by using different services that can scale independently while using specialized inference servers for predictions and model optimizations.

### Your contribution

If my previous assumptions are correct and there is no workaround that I missed I would be happy to discuss a possible solution and create a PR

Comments:
cc @Narsil 

@jspablo ,

This is a very sound request which does make sense.

I think it's going to be tricky to implement in a streamlined fashion since a lot of postprocessing actually uses the model (to get the config). We could have the config be on its own but that require real care since then we have to be extremely consistent (to use the same location) and that would force users that use custom models (with `pipeline(model=MyAwesomeModel()`) to also pass a config along to make it fit the model.

IMO, the easiest route you can take is actually send a mocked model (like an empty class or something). 
Override the `forward` method of your model to do the remote calling. and then implement everything that might be required for later steps.

Does this make sense ?

If you have better proposal/ergonomics suggestion, I am all hears !

Thank you @Narsil
Yes, it makes sense, tested something similar for using ONNX models before they were included in the pipeline abstraction.
Just was looking for a more generic way to reuse any pipeline.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
"AttributeError: cls.seq_relationship.weight not found in PyTorch model" when converting pytorch to tensorflow
Author: BramVanroy
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.0.dev0
- Platform: Linux-5.15.0-41-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.0+cu113 (True)
- Tensorflow version (GPU?): 2.9.1 (True)

### Who can help?

 @Rocketknight1 @sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I am trying to convert [this model](https://huggingface.co/BramVanroy/bert-base-dutch-cased-hebban-reviews) to a tensorflow checkpoint on the command line. (I know that users can specify `from_pt` but I'd like the models to be present in both formats from the start). The model has been finetuned on a text classification task.

When I run

```python
from transformers.convert_pytorch_checkpoint_to_tf2 import convert_pt_checkpoint_to_tf
convert_pt_checkpoint_to_tf("bert", "/path/to/pytorch_model.bin", "/path/to/config.json", "my_output_dir")
```

I get the following error:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bram/Python/projects/transformers-finetuner/transformers/src/transformers/convert_pytorch_checkpoint_to_tf2.py", line 326, in convert_pt_checkpoint_to_tf
    tf_model = load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path)
  File "/home/bram/Python/projects/transformers-finetuner/transformers/src/transformers/modeling_tf_pytorch_utils.py", line 126, in load_pytorch_checkpoint_in_tf2_model
    return load_pytorch_weights_in_tf2_model(
  File "/home/bram/Python/projects/transformers-finetuner/transformers/src/transformers/modeling_tf_pytorch_utils.py", line 210, in load_pytorch_weights_in_tf2_model
    raise AttributeError(f"{name} not found in PyTorch model")
AttributeError: cls.seq_relationship.weight not found in PyTorch model
```

### Expected behavior

No error and a correct conversion of my model to TF.

Comments:
Hi @BramVanroy, you shouldn't ever need to directly call `convert_pt_checkpoint_to_tf`. If you want to make a native TensorFlow checkpoint for that model, then the easiest way would be just:

```
model = TFAutoModelForSequenceClassification.from_pretrained("BramVanroy/bert-base-dutch-cased-hebban-reviews", from_pt=True)
model.save_pretrained("bert-base-dutch-cased-hebban-reviews-tf")
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Only use one gpu when generating the hidden states (text embeddings) by "Model output" api
Author: zyddora
Pull request: no
Labels: name: bug

### System Info

Report an issue about model output efficiency. 
I want to get the certain hidden states from the fine-tuned roberta model, then I chose Model Output api([link](https://huggingface.co/docs/transformers/main_classes/output#transformers.utils.ModelOutput)) to get those dense embeddings. 
However, when executing below model output code, only ONE gpu is used (actually, I have one worker with 4 gpus), which severely hinders inference efficiency. Also, I cannot find the instructions or docs to help to use multiple gpus when using model output.

<img width="1397" alt="Screen Shot 2022-07-27 at 15 51 37" src="https://user-images.githubusercontent.com/12815760/181194206-1483d4ce-39db-4f20-9355-a46844f6e39c.png">

<img width="681" alt="Screen Shot 2022-07-27 at 15 52 01" src="https://user-images.githubusercontent.com/12815760/181194264-893ceab2-c63e-4fa7-87c0-a9cf5b530782.png">



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

please refer to the above info form.

### Expected behavior

Multiple gpus can be used when using transformers Model Output api.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

As an aside, we recommend looking into the [accelerate](https://github.com/huggingface/accelerate) package for multi-GPU usage.

Thanks!

Cannot loggin to hugging face, so just to pose the issue here.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Tried running Stable Diffusion GRisk GUI.exe and no go.
Author: nymb
Pull request: no
Labels: name: bug

### System Info

torchvision\io\image.py:13: UserWarning: Failed to load image Python extension:
torch\_jit_internal.py:751: UserWarning: Unable to retrieve source for @torch.jit._overload function: <function _DenseLayer.forward at 0x000001A24A9D15E0>.
  warnings.warn(f"Unable to retrieve source for @torch.jit._overload function: {func}.")
torch\_jit_internal.py:751: UserWarning: Unable to retrieve source for @torch.jit._overload function: <function _DenseLayer.forward at 0x000001A24A9E4940>.
  warnings.warn(f"Unable to retrieve source for @torch.jit._overload function: {func}.")
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
Moving 10 files to the new cache system
  0%|                                                                                           | 0/10 [00:00<?, ?it/s]
There was a problem when trying to move your cache:

  File "transformers\utils\hub.py", line 1077, in <module>

  File "transformers\utils\hub.py", line 1040, in move_cache

  File "transformers\utils\hub.py", line 997, in move_to_new_cache

  File "huggingface_hub\file_download.py", line 841, in _create_relative_symlink


Not sure how to proceed..

@LysandreJik 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Tried running Stable Diffusion GRisk GUI.exe and no go.

### Expected behavior

Expected the GUI to load..

Comments:
Sorry the GUI did open on another monitor I had turned off but the terminal did mention some issues.
The Document of LongT5 confilcts with and its example code of prefix
Author: GabrielLin
Pull request: no
Labels: name: bug

### System Info

All.

### Who can help?

@patrickvonplaten 

### Reproduction

See https://huggingface.co/docs/transformers/main/en/model_doc/longt5

### Expected behavior

In the above document, it said `Unlike the T5 model, LongT5 does not use a task prefix. Furthermore, it uses a different pre-training objective inspired by the pre-training of [PegasusForConditionalGeneration].`. But in the example code of `LongT5ForConditionalGeneration`, there is a prefix of `summarize: `.  I am confused about how to use LongT5 in different down tasks. Could you please help? Thanks.

Comments:
@stancld @patil-suraj Could you please help to solve this issue and tell me how to set and use special down tasks for LongT5? Thanks.

Hi @GabrielLin, with LongT5 model no prefix should be added to the input sentence. The doc example seems not to be accurate.

Hi, @stancld . Thank you for your reply. Could you please indicate how to use `[PegasusForConditionalGeneration]` for different down-tasks and help to fix the example code? I have no ideas.

> Hi, @stancld . Thank you for your reply. Could you please indicate how to use `[PegasusForConditionalGeneration]` for different down-tasks and help to fix the example code? I have no ideas.

Hi, example should have been already fixed by @patrickvonplaten. Fine-tuning on different down-tasks should be pretty standard. There's no prefix, you can thus use same techniques for models like BART, GPT-2, etc. :] However, the final performance is questionable as, AFAIK, only summarization and Q&A has been investigated so far.

> > Hi, @stancld . Thank you for your reply. Could you please indicate how to use `[PegasusForConditionalGeneration]` for different down-tasks and help to fix the example code? I have no ideas.
> 
> Hi, example should have been already fixed by @patrickvonplaten. Fine-tuning on different down-tasks should be pretty standard. There's no prefix, you can thus use same techniques for models like BART, GPT-2, etc. :] However, the final performance is questionable as, AFAIK, only summarization and Q&A has been investigated so far.

Thank you @stancld . Thank @patrickvonplaten . I have one more question. If having the prefix, I consider that different down-tasks can be fine-tuned in the same model. Now, without the prefix, should we use separated model for different down-tasks? Thanks.

Hey @GabrielLin 

That depends on how different the use cases are and what your limitations are exactly. In general, I'd say yes you should use different fine-tuned models for different tasks

@patrickvonplaten Got it. Thanks. This issue has been fixed and closed.
Character level models?
Author: KosayJabre
Pull request: no

Hi, are any character-level language models available? Transformer-XL mentions in their paper that they did both word level and character level stuff, yet here it seems only the word level one is available? Is that correct?

Comments:
I don't think we have any character-level language models available on [huggingface.co/models](https://huggingface.co/models), but they should be pretty straightforward to train.

BTW I'm sure you saw it already but [PyTorch's nn.Transformer tuto](https://pytorch.org/tutorials/beginner/transformer_tutorial.html) is a char-level language model.

Hi, that's actually is word-level model.
Do you know of any pretrained bidirectional transformer-based character level language models that I can use?

No I'm not aware of any pretrained one.

We will soon have a pretrained ReformerLM model on character level 

@patrickvonplaten just for my curiosity would this be the model trained on the "Crime and Punishment" from their Colab (but the vocab is not char-only), or do you have an own trained model 🤔 

Yeah that was my bad definition of char-only I guess :D. The vocab has 320 tokens, so it's more like on "very" small word units level. 

Example: 
```python 
tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
tokens = tok.encode("This is a test sentence") # [108, 265, 24, 111, 4, 3, 249, 7, 76, 25, 69]
print([tok.decode(token) for token in tokens]) # ['T', 'h', 'is', 'is', 'a', 't', 'est', 's', 'ent', 'en', 'ce']
```

"True" char-level is cool because obviously the tokenizer is then pretty trivial :)

Btw, we now have a "True" char-level reformer model here: https://huggingface.co/google/reformer-enwik8 :-) 

Any chance of having a TF version of the Reformer model?

Yes in ~2 month I would guess

@patrickvonplaten Is there any notebook/doc on how to fine tune the char level model using reformer-enwiki8 model? Having a doc showing how to pass training data to fine tune it would be helpful.

You should be able to leverage the code shown on the model card of reformer-enwik8 here: https://huggingface.co/google/reformer-enwik8#reformer-language-model-on-character-level-and-trained-on-enwik8 . It shows how data is passed to the model.

I don't understand how to use that code in place of a Tokenizer object. For example, to train a masked language model in [this example script](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm.py) the tokenizer is used in the data collator to do a lot more than just mapping to `input_ids`. I've tried loading tokenizers from character level models like CANINE but that doesn't work either. I could try instantiating an untrained tokenizer and passing all the characters in the corpus as special characters [like this person did](https://discuss.huggingface.co/t/character-level-tokenizer/12450) but it seems like I shouldn't have to do that to achieve something so basic.
`load_tf_weights` doesn't handle the weights added to the TF models at the top level
Author: ydshieh
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Windows-10-10.0.22000-SP0
- Python version: 3.9.11
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.9.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@gante

### Reproduction

(TF)MarianMTModel has weights `final_logits_bias` added at the top-level (i.e. not under any layer)

https://github.com/huggingface/transformers/blob/5f06a09b9f3f05b4860f11bbbe22861923b49d81/src/transformers/models/marian/modeling_tf_marian.py#L1287

However, the method `load_tf_weights` only handle weights under some layers

https://github.com/huggingface/transformers/blob/5f06a09b9f3f05b4860f11bbbe22861923b49d81/src/transformers/modeling_tf_utils.py#L850

This causes problem when we load TF checkpoints for `TFMarianMTModel`, i.e. `final_logits_bias` is not loaded.

```python
from transformers import MarianMTModel, TFMarianMTModel
model_name = "Helsinki-NLP/opus-mt-en-ROMANCE"

pt_model = MarianMTModel.from_pretrained(model_name)
tf_model_from_pt = TFMarianMTModel.from_pretrained(model_name, from_pt=True)
tf_model = TFMarianMTModel.from_pretrained(model_name, from_pt=False)

# Only has `TFMarianMainLayer` in `layers`
print(tf_model.layers)

print(pt_model.final_logits_bias.numpy())
print(tf_model_from_pt.final_logits_bias.numpy())
print(tf_model.final_logits_bias.numpy())
```

Outputs:

```bash
[<transformers.models.marian.modeling_tf_marian.TFMarianMainLayer object at 0x000001F00ECE9940>]
[[11.757146  -1.7759448 -7.3816853 ... -1.6559223 -1.6663467  0.       ]]
[[11.757146  -1.7759448 -7.3816853 ... -1.6559223 -1.6663467  0.       ]]
[[0. 0. 0. ... 0. 0. 0.]]
```


### Expected behavior

`load_tf_weights` should be able to load weights like `final_logits_bias`, and the TF checkpoint should be loaded correctly.

Comments:
Related to #18149.

cc @patrickvonplaten as we might need to change the core method `load_tf_weights`.
TF: TFMarianMTModel final logits bias as a layer
Author: gante
Pull request: yes

# What does this PR do?

Fixes #18802 

As stated in the issue above, `final_logits_bias` in `TFMarianMTModel` are not being loaded at `from_pretrained(...)` time. The PT model has this variable defined, and thus the outputs of the model in the two frameworks are very different (>1e-1).

Actually, these weights are also not being stored when the TF version is saved, for the same reason -- only layers are stored/loaded with the functions we are using (`.save_weights` and `.load_weights`), and this bias weight is not inside a layer.

As a solution, this PR moves the bias to a layer and creates an alias for it, resulting in no interface changes. After this change, the models from `Helsinki-NLP` can be converted with the `pt-to-tf` CLI, passing all the quality checks.

⚠️ Other models have this pattern, so I will apply the change to them in a separate PR if this one gets approved. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@gante Thanks a lot. It looks like it works well!

However, there is one thing I don't understand quite well.


```bash
(Pdb) [x.name for x in model.non_trainable_weights]
['final_logits_bias:0']
```
and this is good as it makes loading correctly. But I was thinking I will see `['final_logits_bias.final_logits_bias:0']`, as you pass the name to the layer as well as in `add_weight`.

Is it true that when we use `add_weight` inside a layer, that layer name won't appear in the variable name for that weight?

(I set a breakpoint at in `src/transformers/modeling_tf_utils.py` at line 847)

@ydshieh hah, I had the same question but I tried, it worked, and I forgot to dig deeper to understand why :D 

After some digging, I found that it is poorly documented -- variables created with `.add_weight` are set without any name scope, i.e. their name consists of the name set in `name`. This is opposed to the weights from layers, such as `tf.keras.layers.Dense`, that automatically get a scoped name according to the `name` of the layers (e.g. `foo/bar/weights:0`).

This implies that initializing `BiasLayer` with a `name` has no effect whatsoever regarding weight storing/loading. If we wanted the weights to have a scoped name (we don't here), we could either hardcode it in `name` ([example](https://github.com/huggingface/transformers/blob/811c4c9f79758235762b4f70ffae00deae494fb1/src/transformers/models/albert/modeling_tf_albert.py#L493)) or use `tf.name_scope` ([example](https://github.com/huggingface/transformers/blob/811c4c9f79758235762b4f70ffae00deae494fb1/src/transformers/models/albert/modeling_tf_albert.py#L150)).

I'm adding a link to this comment in the code, for future reference.

Thanks a lot @gante , you are the best!