noamaanMulla-03/transformers-issues huggingface dataset:
Fix the Conda package build
Author: bryant1410
Pull request: yes

I saw that [the Conda builds are failing since v4.12](https://github.com/huggingface/transformers/actions/workflows/release-conda.yml). The main problem is that, for some reason, [the build tries to install `setuptools` but Conda build forbids it](https://github.com/huggingface/transformers/runs/5853294684?check_suite_focus=true#step:6:2172). I found [an answer in StackOverflow](https://stackoverflow.com/a/64825075/1165181) that shows it can be fixed by adding the flags ` --single-version-externally-managed --record=record.txt` to the `python setup.py install` command in the `build.sh` file (note the `--record` flag is also needed, otherwise the command fails, stating so).

I also updated the tokenizers version specification, which seemed to have been forgotten to be updated in this file as well.

I added `conda-verify`, which `conda build` uses for some sanity checks.

Finally, I changed `conda-build` to `conda build`, which seems to be the way to use this command.

It'd be good if somebody can check this on their end, to double-check it's working fine:

```bash
conda create -n build-transformers -c huggingface python=3.8 anaconda-client conda-build conda-verify
conda activate build-transformers
TRANSFORMERS_VERSION=$(python setup.py --version) conda build .github/conda
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger I saw you have worked on the Conda packaging of this repo before. Can you look into it? IMHO this PR doesn't take a lot of time to review.

@LysandreJik friendly ping!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@LysandreJik friendly ping :)

Sorry, just got to it! I managed to get the build to run correctly by just adding the `tokenizers` line :tada: Let me know if you'd like for me to send logs.

Are you sure we need the rest? I'd be happy to merge your PR with just the tokenizers changes :)

Yeah. I explained why in the PR description

Would you like me to share the logs I got locally showing only the `tokenizers` change was necessary?

> Would you like me to share the logs I got locally showing only the `tokenizers` change was necessary?

I left only the `tokenizers` change now.

The failing tests are flaky, right?
Add Semantic Segmentation for ConvNext
Author: tanaymeh
Pull request: yes

# What does this PR do?

This PR adds Semantic Segmentation to ConvNext model, original code [here](https://github.com/facebookresearch/ConvNeXt/tree/main/semantic_segmentation).

## Who can review?
@NielsRogge 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15781). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15781). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still working on this.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still working


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix all is_torch_tpu_available issues
Author: muellerzr
Pull request: yes
Labels: name: trainer

name: bug

# What does this PR do?

This PR should fix up all `torch_xla` initialization caused on import with the new check for if a TPU is available by following the same structure as [accelerate](https://github.com/huggingface/accelerate/pull/469)

Fixes # (issue)
Fixes #17752 
Fixes #17900 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
fix regexes with escape sequence
Author: stas00
Pull request: yes

This PR fixes:

```
src/transformers/dynamic_module_utils.py:81
  /workspace/transformers/src/transformers/dynamic_module_utils.py:81: DeprecationWarning: invalid escape sequence \s
    relative_imports = re.findall("^\s*import\s+\.(\S+)\s*$", content, flags=re.MULTILINE)
src/transformers/dynamic_module_utils.py:83
  /workspace/transformers/src/transformers/dynamic_module_utils.py:83: DeprecationWarning: invalid escape sequence \s
    relative_imports += re.findall("^\s*from\s+\.(\S+)\s+import", content, flags=re.MULTILINE)
src/transformers/dynamic_module_utils.py:125
  /workspace/transformers/src/transformers/dynamic_module_utils.py:125: DeprecationWarning: invalid escape sequence \s
    imports = re.findall("^\s*import\s+(\S+)\s*$", content, flags=re.MULTILINE)
src/transformers/dynamic_module_utils.py:127
  /workspace/transformers/src/transformers/dynamic_module_utils.py:127: DeprecationWarning: invalid escape sequence \s
    imports += re.findall("^\s*from\s+(\S+)\s+import", content, flags=re.MULTILINE)
src/transformers/modeling_utils.py:222
  /workspace/transformers/src/transformers/modeling_utils.py:222: DeprecationWarning: invalid escape sequence \d
    bit_search = re.search("[^\d](\d+)$", str(dtype))
```

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use explicit torch version in deepspeed CI
Author: ydshieh
Pull request: yes

# What does this PR do?

Use explicit torch version in DeepSpeed CI docker file, as we do in

https://github.com/huggingface/transformers/blob/d49c43e93fedc1ff7d58a3617fc8a3532af054ba/docker/transformers-all-latest-gpu/Dockerfile#L12

Comments:
_The documentation is not available anymore as the PR was closed or merged._

OK for me to use 1.12. We might see more test failures (for scheduled daily CI). Current CircleCI pings 1.11 to have green CI.
PyTorch 1.12.0 for scheduled CI
Author: ydshieh
Pull request: yes

# What does this PR do?

After the slack discussion, using PyTorch 1.12.0 for scheduled CI is a better idea, so we can see what to fix.

Another reason is that this (messy) block
https://github.com/huggingface/transformers/blob/39dad9768e75460d8bf92fc27d407562eaeb6bd0/docker/transformers-pytorch-gpu/Dockerfile#L19-L22
will change torch 1.11 (if specified) back to 1.12, as torchvision and torchaudio are installed separately from torch without specifying versions. It's better to avoid such situations.

Regarding the torch/torchvision/torchaudio correspondence, I have better approach in my past-ci PR.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
ExplicitEnum subclass str (JSON dump compatible)
Author: BramVanroy
Pull request: yes

I found that when I wanted to write the parsed dataclasses that I get from `HfArgumentParser.parse_args_into_dataclasses()` to JSON, that I would get JSON errors. The reason being that `TypeError: Object of type IntervalStrategy is not JSON serializable`. While this is understandable (Enum members are not serializable), this is not ideal within `transformers`.

I checked all items in `transformers` that subclass `ExplicitEnum` and it seems that they are all `str`-only Enums. That would allow us to have them inherit from `str`, too, which solves the JSON issue. JSON can then make use of its `str` class for serialization. Below is a minimal - but full - example to show how this would work:

```
from enum import Enum
from json import dump, loads
from pathlib import Path


class ExplicitEnum(str, Enum):  # If you remove `str` you'll get a serialization error
    """
    Enum with more explicit error message for missing values.
    """

    @classmethod
    def _missing_(cls, value):
        raise ValueError(
            f"{value} is not a valid {cls.__name__}, please select one of {list(cls._value2member_map_.keys())}"
        )


class IntervalStrategy(ExplicitEnum):
    NO = "no"
    STEPS = "steps"
    EPOCH = "epoch"


if __name__ == "__main__":
    strat = IntervalStrategy("no")
    print(strat)

    p = Path("strat_dump.json")
    with p.open("w", encoding="utf-8") as out:
        dump({"strategy": strat}, out, indent=4, sort_keys=True)

    loaded = loads(p.read_text(encoding="utf-8"))
    strat = IntervalStrategy(loaded["strategy"])
    print(strat)
```

A consequence is that now these ExplicitEnums will have a Union type, which originally lead to issues when using `HfArgumentParser._parse_dataclass_field`. Therefore, I added an exception to `_parse_dataclass_field` to allow for a Union if one of the types is `str`, assuming that a given string value to the argparser will be resolved correctly, because it is one of the accepted types.


## Who can review?

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The following tests are failing but that seems unrelated:

tests/pipelines/test_pipelines_object_detection.py::ObjectDetectionPipelineTests::test_small_model_pt
tests/pipelines/test_pipelines_image_segmentation.py::ImageSegmentationPipelineTests::test_small_model_pt


Yes, I skipped those tests on main for now. Let me play a little bit with this, it seems like a good idea but I want to make sure it doesn't break anything before merging.

Tested and it all looks good, thanks a lot!
Decision Transformer Position Embedding Incorrect Implementation
Author: charlesjsun
Pull request: no
Labels: name: bug

### System Info

Not necessary (source code issue)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

From the code for decision transformer: https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L821-L822
But the actual implementation did not remove the position embedding from the GPT2 model
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L497
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L610-L611


### Expected behavior

The expected implementation should be 
https://github.com/kzl/decision-transformer/blob/e2d82e68f330c00f763507b3b01d774740bee53f/gym/decision_transformer/models/trajectory_gpt2.py#L680-L681
from the official decision transformer repo

Comments:
cc @edbeeching @simoninithomas 

Thanks for highlighting this, we set the position_ids to all zeros in the forward pass of the Decision Transformer Model: 
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L935-L942

In additional, the weights of this layer are loaded with zeros.  This is equivalent to not using the position embeddings. 
I just went through the Decision Transformer models on the hub to ensure that the model.encoder.wpe weights are indeed zeros and that is the case.

We left position embeddings in the implementation in case researchers wish to experiment with the inclusion of position embeddings. Please let us know if you find any other examples of potential bugs or require further clarification.
Ambiguous positional embedding management in LongformerEmbeddings
Author: clefourrier
Pull request: no
Labels: name: bug

### System Info

```shell
Current main version of the transformer lib (4.20.1?)
```


### Who can help?

@ydshieh

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi!

In the code of the LongformerEmbeddings ([here](https://github.com/huggingface/transformers/blob/401fcca6c561d61db6ce25d9b1cebb75325a034f/src/transformers/models/longformer/modeling_longformer.py#L459)), there is unreachable code for the `position_ids` in `forward`.

```python
   def forward(self, input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None):
        if position_ids is None:
            if input_ids is not None:
                # Create the position ids from the input token ids. Any padded tokens remain padded.
                position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx).to(input_ids.device)
            else:
                position_ids = self.create_position_ids_from_inputs_embeds(inputs_embeds)

        ...

        # Here, code is unreachable
        # as both create_position_ids_from_input_ids and create_position_ids_from_inputs_embeds always return stg 
        if position_ids is None:
            position_ids = self.position_ids[:, :seq_length]
```

So, what is the actual expected behavior of this layer for the positional embeddings ids? 
Is it supposed to use self.position_ids or not? 
(If yes, then this is indeed a bug, if not, then some code could be removed for clarity)


Comments:
Hi @clefourrier 

Looking the PR #7352, `LongformerEmbeddings` was originally using (or being) `RobertaEmbeddings` at that time, and both had the line `if position_ids is None:`.

Looking current `RobertaEmbeddings`, there is no more such line. So I think we can remove it too for `LongformerEmbeddings` without any doubt (actually, it is already very obvious, but I just to find more evidence 😄 )

Would you like to open a PR?
Fix #17893, removed dead code
Author: clefourrier
Pull request: yes

# What does this PR do?


Fixes #17893 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?
@ydshieh 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @clefourrier! The code quality error comes from a new release from `black`. Rebasing on `main` should solve the issue as you'll benefit from https://github.com/huggingface/transformers/pull/17918.

Regarding the test

from Lysandre on Slack

_There was a new release from black that has a slightly different behavior for the --preview flag that we use in the CI._

_If you see failures in the CI for the code quality test, with a large number of file changes (>500), please mention to the author that they just need to rebase on/merge main in order to benefit from the fix._


@LysandreJik @ydshieh Should be good now! :smiley: 
Ty both, I had missed it on the slack

@sgugger Done :)
Fix GPT-NeoX-20B past handling, attention computation
Author: zphang
Pull request: yes

# What does this PR do?

* Fixes GPT-NeoX-20B handing of the past object to correctly be used in .generate
* Swaps attention computation for one more similar in the original training code, to hopefully avoid NaNs
* Update docstring, removed unnecessary dropout configs in config object

<!-- Remove if not applicable -->

Fixes # (issue)

https://github.com/huggingface/transformers/issues/17632 
https://github.com/huggingface/transformers/issues/17452 (Hopefully)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

There are a few equivalence tests failing with the PR, if you can dive into it. Let us know if you need any help!

I've run the tests locally and they pass, so I can't seem to reproduce the test errors. Can someone else give them a try?

The tests pass on GPU but not on CPU on my side. So doing
```
CUDA_VISIBLE_DEVICES="" pytest tests/models/gpt_neox/test_modeling_gpt_neox.py
```
reproduces the failure.

Thanks again! Nice to be able to use GPT-Neo-X in float16 for generations :-)
[Default pipelines] Add a revision tag to all pipelines
Author: patrickvonplaten
Pull request: no

### Feature request

Continuing the discussion here: https://github.com/huggingface/transformers/pull/17286#discussion_r893716733

To make sure that the default pipelines models - some of which - are community contributed models and thus somewhat "out-of-control" for transformers, it'd be important to add a `revision` tag to those default models (maybe actually to all pipeline models).

### Motivation

To ensure that changing checkpoints don't break transformers

### Your contribution

Can open a quick PR

Comments:
cc @NielsRogge @Narsil @LysandreJik @sgugger
"num_examples" incorrect when using IterableDataset
Author: Vicky-Meng
Pull request: no

When using ```torch.utils.data.IterableDataset```, logging ```num_examlpes``` (as in https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L1518) is not the actual num_examples,  but the number of examples processed on a single process.

For example, when I have 2000 training samples and 2 gpus, with and IterableDataset, current output be like: 
```
[INFO|trainer.py:1519] 2022-06-28 12:51:44,666 >> ***** Running training *****
[INFO|trainer.py:1520] 2022-06-28 12:51:44,666 >>   Num examples = 1000
[INFO|trainer.py:1521] 2022-06-28 12:51:44,666 >>   Num Epochs = 1
[INFO|trainer.py:1522] 2022-06-28 12:51:44,666 >>   Instantaneous batch size per device = 8
[INFO|trainer.py:1523] 2022-06-28 12:51:44,666 >>   Total train batch size (w. parallel, distributed & accumulation) = 16
[INFO|trainer.py:1524] 2022-06-28 12:51:44,666 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:1525] 2022-06-28 12:51:44,666 >>   Total optimization steps = 125
```


Here's the possible cause I've found:
as defined in 
https://github.com/huggingface/transformers/blob/e02037b3524686b57c5a861ea49ac751f15568af/src/transformers/trainer.py#L1085
```num_examples``` is equal to ``` len(dataloader.dataset)```.
However, when ```isinstance(dataset, torch.utils.data.IterableDataset)```, the```dataloader.dataset``` is an instance of ```IterableDatasetShard```, which "generate samples for one of the processes only", whose ```__len__``` is the length of dataset on a single process, not the entire project.

Comments:
Thanks for flagging! Could you double-check the PR above fixes your issue?
Fix number of examples for iterable dataset in distributed training
Author: sgugger
Pull request: yes

# What does this PR do?

As pointed out in #17913, when training in distributed mode with iterable datasets, the number of examples displayed is wrong. This is because we need to go grab the length of the underlying dataset of the `IterableDatasetShard`, not the length of the  `IterableDatasetShard` itself.

Fixes #17913

Comments:
_The documentation is not available anymore as the PR was closed or merged._
CLI: convert sharded PT models
Author: gante
Pull request: yes

# What does this PR do?

This PR adds a major upgrade and a minor change to the `pt-to-tf` CLI.

Major upgrade: we can now convert sharded PT models. It updates how the `from_pt` loading works so as to be able to load from shards. It also updates how the `pt-to-tf` CLI stores the models, so it uses sharding capabilities when needed.

Minor change: adds a flag to control the maximum hidden layer admissible error. It is relatively common to find models where the outputs from the PT and TF models are nearly the same, but the hidden layers have a larger mismatch. This flag allows us to temporarily increase the admissible error if the model seems to be behaving properly (for instance, all RegNet models had a hidden layer difference between 1e-4 and 1e-2, but the outputs were behaving properly).

Example of sharded TF model PR, using the updated tools: https://huggingface.co/facebook/regnet-y-10b-seer-in1k/discussions/1 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

BTW could we add 2 tests, `test_load_sharded_tf_to_pt` and `load_sharded_pt_to_tf` 

TF shards -> PT probably won't work, but I will add the test for PT shards -> TF 👍