noamaanMulla-03/transformers-issues huggingface dataset:
Use code on the Hub from another repo
Author: sgugger
Pull request: yes

# What does this PR do?

This makes it easier to maintain only one source of ground truth when using the code on the Hub feature by storing the repo ID on top of the module containing the class inside the config. Thus, when saving and re-pushing a model using code on the Hub, the code is not copied over anymore, but a reference to the original repo containing the code is put.

This might be breaking if some users relied on the code being copied over when `save_pretrained(xxx)` is executed. To enable that old behavior, one only needs to call the `register_for_auto_class` method:

```py
from transformers import AutoModel

model = AutoModel.from_pretrained("hf-internal-testing/test_dynamic_model", trust_remote_code=True)
model.save_pretrained(some_path)
```
then some_path only contains the config and weights of the model. The config will contain links to the repo where the code of the model is defined (`hf-internal-testing/test_dynamic_model`) so that it can be reloaded via
```py
AutoModel.from_pretrained(some_path)
```

To get the custom code file copied other (behavior before this PR) just do:
```py
from transformers import AutoModel

model = AutoModel.from_pretrained("hf-internal-testing/test_dynamic_model", trust_remote_code=True)
model.register_for_auto_class("AutoModel")
model.save_pretrained(some_path)
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Mark auto models as important
Author: sgugger
Pull request: yes

# What does this PR do?

This PR marks the auto model as important so that the corresponding tests are not skipped. This is what caused a break on main after #22698 was merged.

The change in the Korean doc file is a change of line ending, which is currently making it impossible to do anything on main (the remote branch as CRLF line endings but GitHub really wants LF, this shouldn't be possible but there was a bug when merging the PR touching that file).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22815). All of your documentation changes will be reflected on that endpoint.
The output of TFAutoModel-save_pretrained and keras-ModelCheckpoint do not equal.
Author: guotong1988
Pull request: no
Labels: name: bug

### Describe the bug

```
history = model.fit(
    tf_train_dataset, validation_split=0.01,
    epochs=int(training_args.num_train_epochs),
    callbacks=callbacks,
)

model.save_pretrained(checkpoint_local)

```
output: `h5` file


```
callbacks = [tf.keras.callbacks.ModelCheckpoint(checkpoint_local)]

history = model.fit(
    tf_train_dataset, validation_split=0.01,
    epochs=int(training_args.num_train_epochs),
    callbacks=callbacks,
)
```
output: `pb` file and `assets` and `variables`


### System info

```shell
transformers = 4.26

python = 3.8
```


Comments:
Hi @guotong1988 , I think this issue is more related to the `transformers` library so I'm transferring the issue to the corresponding repo. I'll let @sgugger @ydshieh comment about the issue itself.

@guotong1988 

It's not clear to me what question you have in mind. Do you mean one output `h5` file and another one output `pb` file (and others), and you think both of these 2 methods should output the same (set of) file(s)? Or you mean other thing(s)?


Sorry for the late response.

Yes! @ydshieh Thank you!

These 2 methods should output the same. 

`h5` file is preferred.

In fact, I need to output the model file during training, while using the `callbacks`.

I refer the code here https://github.com/huggingface/transformers/blob/main/examples/tensorflow/language-modeling/run_clm.py#L587

@guotong1988 These are two different methods of saving models to different formats. It's normal that they don't give the same format. If you need a `.h5` file as well as other files (like configuration file, tokenizers from `transformers`), you can always add a line `model.save_pretrained(checkpoint_local)` in your script/notebook.

Thank you.

How can I put `model.save_pretrained` into `callbacks`?

So that I can save the model for each epoch.

There is [PushToHubCallback](https://huggingface.co/docs/transformers/main/en/main_classes/keras_callbacks#transformers.PushToHubCallback).

The goal of this callback is to save and push to the Hub - I am not sure if we can only save but not to push though.
It might be great if you also push the checkpoints to the Hub. If you don't want to push but just save, I will cc @Rocketknight1 :-)


Yes, I don't want to push but just save.

@guotong1988 If you want to proceed quickly, you can modify the code of the class `PushToHubCallback` to remove the part that pushes the checkpoints.
How to use Distill-BERT with different datasets?
Author: sauravtii
Pull request: no

### System Info

- `transformers` version: 4.11.3
- Platform: Linux-5.4.0-58-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.12.0+cu102 (True)
- Tensorflow version (GPU?): 2.10.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I recently read [this](https://huggingface.co/docs/transformers/quicktour#train-with-tensorflow:~:text=The%20most%20important%20thing%20to%20remember%20is%20you%20need%20to%20instantiate%20a%20tokenizer%20with%20the%20same%20model%20name%20to%20ensure%20you%E2%80%99re%20using%20the%20same%20tokenization%20rules%20a%20model%20was%20pretrained%20with.) and was wondering how to use distill-BERT (which is pre-trained with imdb dataset) with a different dataset (for eg. [this](https://huggingface.co/datasets/yhavinga/imdb_dutch) dataset)?

### Expected behavior

Distill-BERT should work with different datasets.

Comments:
Closing this issue as it's a repeat of #22817
Allow initializing HuggingFaceEmbeddings from the cached weight
Author: nicolefinnie
Pull request: no

### Feature request


### Suggestion

The only change has only a few lines in `__init__()`,

```python
class HuggingFaceEmbeddings(BaseModel, Embeddings):
    """Wrapper around sentence_transformers embedding models.

    To use, you should have the ``sentence_transformers`` python package installed.

    Example:
        .. code-block:: python

            from langchain.embeddings import HuggingFaceEmbeddings
            model_name = "sentence-transformers/all-mpnet-base-v2"
            hf = HuggingFaceEmbeddings(model_name=model_name)
    """

    client: Any  #: :meta private:
    model_name: str = DEFAULT_MODEL_NAME
    """Model name to use."""

    def __init__(self, cache_folder=None, **kwargs: Any):
        """Initialize the sentence_transformer."""
        super().__init__(**kwargs)
        try:
            import sentence_transformers
            self.client = sentence_transformers.SentenceTransformer(model_name_or_path=self.model_name, cache_folder=cache_folder)
        except ImportError:
            raise ValueError(
                "Could not import sentence_transformers python package. "
                "Please install it with `pip install sentence_transformers`."
            )

    class Config:
        """Configuration for this pydantic object."""

        extra = Extra.forbid

    def embed_documents(self, texts: List[str]) -> List[List[float]]:
        """Compute doc embeddings using a HuggingFace transformer model.

        Args:
            texts: The list of texts to embed.

        Returns:
            List of embeddings, one for each text.
        """
        texts = list(map(lambda x: x.replace("\n", " "), texts))
        embeddings = self.client.encode(texts)
        return embeddings.tolist()

    def embed_query(self, text: str) -> List[float]:
        """Compute query embeddings using a HuggingFace transformer model.

        Args:
            text: The text to embed.

        Returns:
            Embeddings for the text.
        """
        text = text.replace("\n", " ")
        embedding = self.client.encode(text)
        return embedding.tolist()
```

### Usage

```python
embedding_model = HuggingFaceEmbeddings(model_name=model_name, cache_folder=cache_folder)
```

### Motivation


Right now, HuggingFaceEmbeddings doesn't support loading an embedding model's weights from the cache but downloading the weights every time. Fixing this would be a low hanging fruit.

### Your contribution

I can submit a PR if this request makes sense, and I've read `CONTRIBUTING.MD`

Comments:
This issue should be fixed in `LangChain` sorry for the misreport.
🌐 [i18n-KO] Fix anchor links for docs `auto_tutorial`, `training`
Author: gabrielwithappy
Pull request: yes

# What does this PR do?

Fixed anchor links for `auto_tutorial` and `training` docs
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

@sgugger, @ArthurZucker, @eunseojo May you please review this PR? 
<!-- Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

Comments:
Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

I fixed anchor links for documents I translated


_The documentation is not available anymore as the PR was closed or merged._

@sgugger, @ArthurZucker, @eunseojo 
May you please review this PR?
Can't load config for 'DeepPavlov/xlm-roberta-large-en-ru'
Author: doyled-it
Pull request: no

Similar to this issue: https://github.com/huggingface/transformers/issues/6226

I'm unable to load a config for a specific model when I run the following:

```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("DeepPavlov/xlm-roberta-large-en-ru")

model = AutoModel.from_pretrained("DeepPavlov/xlm-roberta-large-en-ru")
```

Version:

```python
>>> import transformers
>>> transformers.__version__
'4.17.0'
```

Then I get this error from [this part](https://github.com/huggingface/transformers/blob/main/src/transformers/configuration_utils.py#L634) of the code
```
OSError: Can't load config for 'DeepPavlov/xlm-roberta-large-en-ru'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'DeepPavlov/xlm-roberta-large-en-ru' is the correct path to a directory containing a config.json file
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

It's not completed. Just never received a response.

I'm facing the same issue . Please let me know if you have resolved it @doyled-it

> I'm facing the same issue . Please let me know if you have resolved it @doyled-it

I believe what I did was git clone the huggingface repository and then point to it when I try to load the model instead of forcing huggingface to download it for me.
Fix `test_eos_token_id_int_and_list_top_k_top_sampling`
Author: ydshieh
Pull request: yes

# What does this PR do?

In #22204, I updated the expected value in `test_eos_token_id_int_and_list_top_k_top_sampling` to pass the `CircleCI`. However, the daily CI fails with that new value. It turns out that we need a seed that could give the same (generation) output (at minimum, the same output length) on both CPU/GPU machines. The difference is very likely coming from somehow larger numerical differences after certain generation steps.

### remark
With seed `0`, the output `generated_tokens[0]` is:
- `cpu`: `[ 40, 416,  79,  12, 230,  89, 231, 432, 301, 212, 933, 225,  33,  33,
        846]`
- `gpu`: `[ 40, 416,  79,  12, 230,  89, 231, 432, 301, 212, 933, 225, 476, 682,
        319, 832, 873, 853, 873, 832]`

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use torch.TensorDicts: The output of tokenizers.batch_encode_plus/__call__ could be made to inherit from torch TensorDicts
Author: JulesGM
Pull request: no

### Feature request

Tensor dicts have recently come out as a way to manipulate dicts of tenors in a way that is analogous to pandas, eg to make it easy to work on columns on tensors that share some property and a batch dimension or set of dimensions
https://pytorch.org/rl/tensordict/

An obvious application of this is the output of `tokenizer.batch_encode_plus`. 


### Motivation

Being able to do a bunch of things on all the subtensors at once would be cool, like `.to` and `.cat`, etc.
Having a common interface with tensordict could be fun.

### Your contribution

I don't have the bandwidth to handle this myself.

Comments:
The result of the tokenizer calls can already interact with the `to` method (note that batch_encode_plus will be deprecated sometime soon) but I agree it could be interesting to look at this! The main challenge I see is that it's not packaged in PyTorch main so would require an extra dep...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
flan-t5-xl and flan-t5-xxl model deployment on Sagemaker fails on deploying from HuggingFace Hub
Author: raghu-ramesha
Pull request: no

### System Info

Code to replicate from model hub - https://huggingface.co/google/flan-t5-large/tree/main -> Deploy -> Amazon SageMaker endpoint -> AWS

```
from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'google/flan-t5-xl',
	'HF_TASK':'text2text-generation'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge' # ec2 instance type
)

predictor.predict({
	'inputs': "The answer to the universe is"
})

```

The endpoint invocation fails with below error -

`2023-03-10 19:15:14,508 [INFO ] W-google__flan-t5-xl-5-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index', 'flax_model.msgpack'] found in directory /.sagemaker/mms/models/google__flan-t5-xl or `from_tf` and `from_flax` set to False.`

This is possibly because if you look at files under "Files and Versions"( [Link](https://huggingface.co/google/flan-t5-xl/tree/main)) the model has been split up into multiple (pytorch_model-00001-of-00002.bin files because of the size) and the out of the box solution is looking for one pytorch_model.bin file and failing


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Provided code in the issue description

### Expected behavior

Should work out of the box on SageMaker deployment

Comments:
@philschmid  could you please help here? I've gone though your workaround [here](https://www.philschmid.de/deploy-t5-11b)

You need to install a more recent version of Transformers, 4.17.0 won't support sharded checkpoints.

@rags1357 you can check out this blog post: [Deploy FLAN-T5 XXL on Amazon SageMaker](https://www.philschmid.de/deploy-flan-t5-sagemaker)


Thank you @sgugger  and @philschmid , will try it with the updated Transformers version

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update accelerate version + warning check fix
Author: muellerzr
Pull request: yes
Labels: name: dependencies

name: Distributed Training / Models

# What does this PR do?

This PR bumps the accelerate version, and flips the logic for the warning to be accurate on the distributed mode check

Fixes # (issue)

Solves https://github.com/huggingface/transformers/issues/22816

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger @pacman100 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix from_pretrained when model is instantiated on the meta device
Author: sgugger
Pull request: yes

# What does this PR do?

#22437 broke the `from_pretrained` method whenever the model is instantiated on the meta device and the state dict passed is not complete (see [this issue](https://github.com/huggingface/accelerate/issues/1333) for one example).

Basically the check will remove all keys from `missing_keys` since all parameters on the meta device share the same data pointer. I had advocated to use another solution in that PR but the contributor did not listen

Since we rely on those `missing_keys` later on to re-initialize the weights that are not in the state dict, the model ends up with weights on the meta device.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Raise err if minimum Accelerate version isn't available
Author: muellerzr
Pull request: yes
Labels: name: Distributed Training / Models

# What does this PR do?

This PR will raise an explicit `ImportError` during `TrainingArguments` if `Accelerate` isn't installed (or isn't the required minimal version) and Accelerate is going to be utilized

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Make ClipSeg compatible with model parallelism
Author: youssefadr
Pull request: yes

# What does this PR do?

Add model parallelism for `ClipSeg`.

Related to #22561


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

My pleasure, I'm glad I could help!
Add an efficient vision transformer backbone in ICLR 2022: CrossFormer
Author: cheerss
Pull request: no
Labels: name: New model

### Model description

The CrossFormer has three new things that does not exist in other ViTs (such as Swin):

1. The cross-scale embedding layer(CEL) that generate cross-scale embeddings as ViT's input.
2. The long-short distance attention (LSDA) mechanism, which is an efficient replacement of the vanilla self-attention and shows better performance than Swin
3. A dynamic relative position bias, a kind of relative position bias that support dynamic group size.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The open source website: https://github.com/cheerss/CrossFormer
The paper was accepted in ICLR 2022: https://openreview.net/forum?id=_PHymLIxuI

Comments:
I'm going to close this as it's a repeat of #22852