noamaanMulla-03/transformers-issues huggingface dataset:
[Whisper] Don't return attention mask in feat extractor
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Whisper pads all audio inputs to a fixed max length (=30s) by appending the silence token (zero) to the end of any sequences shorter than the max length. Hence, the model does **not** use an attention mask: all inputs have length 30s, padding is treated through use of the silence token rather than an attention mask.

This PR sets the default value of `return_attention_mask` to `False` in the feature extractor. In doing so, feature extractor methods such as `__call__` and `pad` will **not** return an `attention_mask` by default.

This behaviour can be overridden by passing the arg `return_attention_mask=True` to these methods.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add configuration_codegen.py
Author: AymenBer99
Pull request: yes

# What does this PR do?

Add configuration_codegen.py to utils/documentation_tests.txt for doctest.

Based on #19487 

@sgugger @ydshieh could you please check it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix a typo in the preprocessing tutorial
Author: Quasar-Kim
Pull request: yes

# What does this PR do?

Fixed a typo in transformers [tutorials > preprocess](https://huggingface.co/docs/transformers/preprocessing). Currently the code and its output does not match. Please see last two cells from this [colab notebook](https://colab.research.google.com/drive/18WjgFPtQu4n8k6qAWtpsjVwDmICDrcfD#scrollTo=4LsMdS8plf-K); `dataset[0]["image"]` is a PIL image, not a dictionary as present in the current version of the tutorial. It should be `dataset[0]`.

(Please ignore the change at line 490, Github is showing the wrong diff. There's no actual changes.)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Comments:
My fault, there's nothing wrong with the tutorial. 😮‍💨

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19645). All of your documentation changes will be reflected on that endpoint.
Fixed typos in comments of OPTDecoderLayer
Author: nickypro
Pull request: yes

# What does this PR do?
Fixed typos in comments of OPTDecoderLayer ( used in Meta OPT models )

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

@sgugger 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18909). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add DPT Flax
Author: younesbelkada
Pull request: yes

# What does this PR do?

I tried to implement DPT (Dense Prediction with Transformers) in Flax during my free time! 🚀 
By the way it is the first Segmentation and Depth Estimation model implemented in Flax on the library!

Nits/TODOs:
- [x] Figure out how to properly call `BatchNorm` and `Dropout` inside a `Sequential` 
- [x] Deal correctly with `Sequential` layers 
- [x] Test equivalency tests 
- [ ] Write documentation - For now they're just copy/pasted
Quetions:
- Why the loss is not implemented in `modeling_dpt.py` ? I can probably help on that since I have already implemented the loss for a university project: https://github.com/antocad/FocusOnDepth/blob/master/FOD/Loss.py 
 
cc @NielsRogge @sanchit-gandhi @patil-suraj 


Comments:
All the keys match now but the equivalency test does not pass with `1e-5` but `1e-4` instead

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17779). All of your documentation changes will be reflected on that endpoint.

Would be great to also incorporate the updates of #17731 

The Flax model finally predicts the correct depths for the cats (left is Flax and right is Pytorch)! 
![Screenshot 2022-06-25 at 20 12 08](https://user-images.githubusercontent.com/49240599/175785580-bd9b3b25-cc90-4ad0-a262-fbf335e1351f.png)
For that it appears that the transpose conv does not give the same result as Pytorch's implementation that uses a gradient based operation. I fixed it by creating a custom function based on this PR: https://github.com/google/jax/pull/5772 the PR does not seem to be merged soon. We can probably go for this hack for now until the PR in JAX gets merged 


As wee discussed, it seems that `align_corners` set to  `False` for both model would not require lowering the tolerance in one of the cases right? 


@ArthurZucker exact. I have put a new attribute in the `DPTConfig` and modified a bit the original modeling code but should not break backward compatibility. Now all tests pass with a tolerance of `1e-5` 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Thank you very much @sanchit-gandhi for the very detailed review! I had a second round of refactoring while catching up on Flax projects and would love to have a second round of review (left also some unresolved comments) 💪 Thanks again 🙏 


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
adding functionality to iterate pipelines over sentence pairs when using dataset
Author: rohit1998
Pull request: yes

# What does this PR do?
This PR adds a new torch data set class to pt_utils.py that helps with using pipelines and datasets for sentence pair tasks. 
Usage can be simply like
```
dataset = Dataset.from_pandas(dataset_df[['sentence1', 'sentence2']])
pipe = pipeline('text-classification', model=args.input_path_model, device=0, num_workers=4)
result = list(tqdm(pipe(KeyPairDataset(dataset, 'sentence1', 'sentence2'), batch_size=32), total=len(dataset)))
```

@LysandreJik please have a look.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19659). All of your documentation changes will be reflected on that endpoint.
Cast masks to np.unit8 before converting to PIL.Image.Image
Author: amyeroberts
Pull request: yes

# What does this PR do?

In the recent update to the image segmentation pipeline, the numpy arrays converted to `PIL.Image.Image` with mode `"L"` weren't converted to type `np.uint8`, resulting in corrupted masks. 

Running the following: 
```
import requests
from PIL import Image
from transformers import pipeline

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
pipe = pipeline("image-segmentation", model="facebook/detr-resnet-50-panoptic")
results = pipe(image)
```

**Output masks on `main`:**
![main_0_cat](https://user-images.githubusercontent.com/22614925/195832894-95ef005f-3e0a-4b09-9c10-88539a732d95.png)
![main_1_couch](https://user-images.githubusercontent.com/22614925/195832898-bc40d2fc-a33e-4a26-a97e-380b0a6f37b7.png)
![main_2_remote](https://user-images.githubusercontent.com/22614925/195832899-96cdae09-864f-420b-a902-2da2482d46e4.png)
![main_3_blanket](https://user-images.githubusercontent.com/22614925/195832901-9fa9f65b-089a-4c00-b216-7eabbbcd76a8.png)


**Output masks on this branch**: 
![fix_0_cat](https://user-images.githubusercontent.com/22614925/195832923-3835a988-e0b0-4f3e-a282-011daa95f4f0.png)
![fix_1_couch](https://user-images.githubusercontent.com/22614925/195832926-b7963092-4648-456d-b863-1820ea619f09.png)
![fix_2_remote](https://user-images.githubusercontent.com/22614925/195832929-f5cc2ca9-cf51-4dca-9954-5bf1500e156e.png)
![fix_3_blanket](https://user-images.githubusercontent.com/22614925/195832932-07ecf096-1c1e-4725-97a6-cf239a83e925.png)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @Narsil, would it be possible to update the inference widgets to include this PR?

We were notified on Twitter that the inference widgets were broken: https://twitter.com/levelsio/status/1580573108431646720?t=d9BlnF9Q2nvRaQFNei6KLw&s=19
Use multiple workers for DataLoader at prediction step for Trainer
Author: greg2451
Pull request: yes

# What does this PR do?

Fixes #17749 by adding a parameter in the DataLoader init call of the test data_loader, so that we can use multiple workers for data preparation during prediction time.

@sgugger, I tag you because this is a Trainer related PR

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add `configuration_trocr.py`
Author: thliang01
Pull request: yes

* trocr Config for doctest

* ran make style

# What does this PR do?

Add `configuration_trocr.py` to `utils/documentation_tests.txt` for doctest.

Based on issue #19487

@ydshieh could you please take a look at it?
Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add doctest info in testingmdx
Author: ArthurZucker
Pull request: yes
Labels: name: Documentation

# What does this PR do?

Finding information on how to properly test the docstring is currently pretty hard! As far as I know, the only good explanation is in `transformers/utils/prepare_for_doc_test.py` but it does not appear in the doc. I hope this will help people debug their docstring examples. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yes, gonna clean the github history
Feature-extraction pipeline to return Tensor
Author: ierezell
Pull request: no

# 🚀 Feature request

Actually, to code of the feature-extraction pipeline 
`transformers.pipelines.feature-extraction.FeatureExtractionPipeline l.82` return a `super().__call__(*args, **kwargs).tolist()`

Which gives a list[float] (or list[list[float]] if list[str] in input)

I guess it's to be framework agnostic, but we can specify `framework='pt'` in the pipeline config so I was expecting a `torch.tensor`. 

Could we add some logic to return tensors ? 

# Motivation 
Features will be used as input of other models, so keeping them as tensors (even better on GPU) would be profitable. 

Thanks in advance for the reply, 

Have a great day. 

Comments:
Hello! Indeed, this is a valid request. Would you like to open a PR and take a stab at it?

@LysandreJik Hi, thanks for the fast reply ! 

Ok will do that :) 
I will comment here when the PR will be ready

Hi @LysandreJik is there any update on this issue? If @Ierezell didn't have time, I might be able to give a shot at it in the next days

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi!
Is this issue somewhere in consideration still?
Would be awesome to be able to get tensors from the feature extraction pipeline

I think we'd still be open to that; WDYT @Narsil?

Sure ! 

Would adding an argument `return_type= "tensors"` be OK ? That way we can enable this feature without breaking backward compatibility ?

I'm baffled as to why returning the features as a list is the default behavior in the first place... Isn't one common usage of feature extraction to provide an input to another model, which means it is preferred to keep it as a tensor?

@ajsanjoaquin 

Well it depends, not necessarily. Another very common use case is to feed it to some feature database for querying later.
Those database engines are not necessarily expecting the same kind of tensors that you are sending.

But I kind of agree that it should be at least a `numpy.array` because usually conversions between numpy and PT or TF is basically free, meaning it would be much easier to use that way.

Some `pipeline` were added a long time ago where the current situation was not as clear as today, and since we are very conservative regarding breaking changes, that can explain why some defaults are the way they are.

If/When v5 is getting prepared there would be a lot of small but breaking changes in that regard.
[New Model] LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
Author: WaterKnight1998
Pull request: no
Labels: name: New model

### Model description

Structured document understanding has attracted considerable attention and made significant progress recently, owing to its crucial role in intelligent document processing. However, most existing related models can only deal with the document data of specific language(s) (typically English) included in the pre-training collection, which is extremely limited. To address this issue, we propose a simple yet effective Language-independent Layout Transformer (LiLT) for structured document understanding. LiLT can be pre-trained on the structured documents of a single language and then directly fine-tuned on other languages with the corresponding off-the-shelf monolingual/multilingual pre-trained textual models. Experimental results on eight languages have shown that LiLT can achieve competitive or even superior performance on diverse widely-used downstream benchmarks, which enables language-independent benefit from the pre-training of document layout structure. Code and model are publicly available at [this https URL](https://github.com/jpWang/LiLT).

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Code @jpWang  : https://github.com/jpWang/LiLT

Weights (Author @ManuelFay ): 
- https://huggingface.co/manu/lilt-camembert-dit-base-hf
- https://huggingface.co/manu/lilt-camembert-base
- https://huggingface.co/manu/lilt-camembert-dit-base
- https://huggingface.co/manu/lilt-infoxlm-base

Comments:
Hi,
thanks for your great effort. Contact me if any problem encountered :)

Closing as it has been added in #19450
Add return types for tensorflow GPT-J, XLM, and XLNet
Author: sirmammingtonham
Pull request: yes

# What does this PR do?

Adds return types for model classes in tensorflow GPT-J, XLM, and XLNet as tasked in #16059.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Rocketknight1 
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix pipeline predict transform methods
Author: s-udhaya
Pull request: yes

# What does this PR do?

This PR fixes pipeline's predict and transform methods which are wrapper around ` __call__` of pipeline class. `__call__ `requires one positional argument, However these wrapper methods pass the incoming argument as a keyword argument to `__call__` method which leads to failure. Hence in this commit the keyword argument is modified to positional argument and basic tests are added to make sure this does not break again in future.


Fixes #19289


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik @Narsil 

Comments:
_The documentation is not available anymore as the PR was closed or merged._