noamaanMulla-03/transformers-issues huggingface dataset:
Add MistralForQuestionAnswering
Author: nakranivaibhav
Pull request: no
Labels: name: Feature request

### Feature request

Add a MistralForQuestionAnswering class to the [modeling_mistral.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/mistral/modeling_mistral.py) so Mistral models have AutoModelForQuestionAnswering support (by also adding Mistral models to the MODEL_FOR_QUESTION_ANSWERING_MAPPING_NAMES  in the [modeling_auto.py](https://github.com/huggingface/transformers/blob/v4.36.1/src/transformers/models/auto/modeling_auto.py#L1343) file.

### Motivation

1 - Evaluation benchmarks like [Squad](https://huggingface.co/datasets/squad_v1_pt) or [FaQUAD](https://huggingface.co/datasets/eraldoluis/faquad) are commonly used to evaluate language models.
2 - Many decoder-only transformers ([BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom), [Falcon](https://huggingface.co/docs/transformers/model_doc/falcon), [OpenAI GPT-2](https://huggingface.co/docs/transformers/model_doc/gpt2), [GPT Neo](https://huggingface.co/docs/transformers/model_doc/gpt_neo), [GPT NeoX](https://huggingface.co/docs/transformers/model_doc/gpt_neox), [GPT-J](https://huggingface.co/docs/transformers/model_doc/gptj), etc.) have support for the AutoModelForQuestionAnswering.
3 - Creating a fine-tuning/evaluation procedure using things like AutoModelForQuestionAnswering and evaluate.load('squad') is very simple, making these features very helpful and desirable.
4 - On the contrary, if one cannot use AutoModelForQuestionAnswering, like in the Llama style models, everything becomes more difficult.

Hence, I would like to request the addition of a MistralForQuestionAnswering class to the modeling_mistral.py file. Hence, we could all easily perform experiments with Mistral models and squad-style Q&A benchmarks:

### Your contribution

I have recently added LlamaForQuestionAnswering class in modeling_llama.py file. I can do the same for Mistral.

Comments:
https://github.com/huggingface/transformers/pull/29168#issuecomment-1965857953

> either community pushes for this

push ;) Would be very nice to have an easy way to create a MistralForQuestionAnswering for our benchmark paper.

Thank you for the support janpf. Will be glad if PR is accepted.

Thanks for your request! We'd happily welcome it to the library, feel free to open a PR adding it if you have the bandwidth to do so!

They already did, right? 🤔 #29168 
But the PR got stale
Sink cache: crop if exceeds max cache len
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes #32233. Previous PR completely broke SinkCache and now it is not working as supposed. This PR reverts the change regarding `max_cache_length`, because for sink cache we need to get the latest `max length` masks and have position ida not exceeding the `max length`

I am aware that SinkCache is buggy and doesn't work for some edge cases and for multi-turn dialogues. But I think we can try to keep the previous working version for one-turn generation, until we figure out how to make SinkCache happy 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32315). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yes, we could but we would have to also handle cropping attention masks within `update_causal_mask`. Otherwise we could get `target_length=window_length=50` but the attention is already way beyond that. I guess this wasn't caught by sliding-window-cache users because it usually has a very high max length 

Oke, I will port cropping into `update_causal_mask` in this PR

But the sequence length is given and we create the attention mask based off sequence_length, target_length. 
```python3
            causal_mask = torch.full(
                (sequence_length, target_length), fill_value=min_dtype, dtype=dtype, device=device
            )
            exclude_mask = torch.arange(target_length, device=device) > cache_position.reshape(-1, 1)
            if self.config.sliding_window is not None:
                if not using_sliding_window_cache or sequence_length > self.config.sliding_window:
                    exclude_mask.bitwise_or_(
                        torch.arange(target_length, device=device)
                        <= (cache_position.reshape(-1, 1) - self.config.sliding_window)
                    )
```


now, if slicing the mask is the only thing we need to use potentially siding window attention (or SinkCache) on models, LGTM!

(It means we can refactor the mistral piece of code as well) 

Yes, cropping right now is the only way to make SinkCache happy. I just realized moving the crop to `update_causal_mask` will cause errors in SinkCache case because we would also have to re-inii position ids. Sink Cache position ids should not go beyond max-cache-length, though I am not sure about sliding window.

I propose to leave it in `prepare_for_generation` as is. Otherwise we have to do the same checks in `prepare_for_generation` to crop `position_ids` which will not make code look any easier. Added a test and made some changes

> Could we add a SinkCache test to make sure this does not get broken again?

We have a couple slow sink cache tests failing 🙃 I'd argue that we have so many failing slow tests that we stopped paying attention to new failures (myself included) :P 

close as sink cache will not be maintained anymore
How do LLMs identify generation start point during fine-tuning?
Author: naphatkps
Pull request: no
Labels: name: Need more information

When fine-tuning an LLM, we pass a long string containing `context`, `user input`, and `the expected answer`. How does the model know which word to start generating from.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Phi-3 maximum recursion depth exceeded when execute AutoModelForCausalLM.from_pretrained
Author: JaimeFon
Pull request: no
Labels: name: bug

name: Remote code

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.44.2
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.11.3
- Huggingface_hub version: 0.24.6
- Safetensors version: 0.4.5
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.4.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I ran this code several times successfully but I got the error and from that point I keep getting the error, this is the simple code I am running


import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

torch.random.manual_seed(0)

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3.5-mini-instruct", 
    device_map="cuda", 
    torch_dtype="auto", 
    trust_remote_code=True, 
)

### Expected behavior

I expected to initialize the tokenizer so I can use it later in the code.

Comments:
cc @Rocketknight1 

Hi @JaimeFon, I've tried running this several times and can't reproduce the error. Can you paste me the entire error message?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix a typo
Author: PengWeixuan
Pull request: yes



# What does this PR do?

Fixes typo: "If you want you tokenizer..."->"If you want your tokenizer...."


## Before submitting
- [ √] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Fix flaky tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes flakiness in some generation tests for VLMs. We have to pass the `bad_ids` in all tests that call generate with tiny models.

Plus for Mllama we don't have to check the generation stage that way anymore, since we decided not to support muti-turn conversations.

Comments:
I see, this is not a very nice workaround to force certain tokens from being generated. Actually the test for mllama would not be flaky just by changing the check in prepare-inputs, as I did in this PR

We can get rid of flakiness for now, but take a note for future that fixing it in a proper way is very important. Interestingly passing logits processot makes other tests flaky.... So yeah, that wouldn't work  

@ArthurZucker can you merge pls? The tests is failing because we have unused attribute, but we can't remove it for BC. Also I want all VLMs have such an attr by default, like `bos_token` etc.

For the default one, I'll open another PR to see if everyone agrees

@zucchini-nlp we can't force-merge, this failing check would start showing up in all PRs :P In this case we have to fix the underlying issue or add an exception to the script that checks it :) (lmk if you'd like pointers)

@gante okey, let's do the second option as part of this PR then :)

Added image token id as always allowed, so that all VLMs can have it even if not used directly in modeling. I believe this is useful already in tests + may be useful in other cases in the future

@LysandreJik FYI -- merged this PR, as it is blocking other PRs :)

Thanks for doing so!
TimeSeriesTransformerModel: add no time lags option
Author: chiara-fb
Pull request: no
Labels: name: Feature request

### Feature request

`TimeSeriesTransformerModel` has no option to insert no lags in the model. Setting `lags_sequence = []` in the `config` will return an error. An alternative to circumvent the problem, i.e. `lags_sequence = [0]` will result in having two identical copies of the temporal input (`lagged_sequence` and `time_feat`) to be concatenated in the `temporal_input`. 

### Motivation

Would make the model more user-friendly in cases where no additional lag is needed.

### Your contribution

A simple implementation would entail adding an `if len(lags_sequence) > 0` clause prior to adding the subsequences to the input tensor [here](https://github.com/huggingface/transformers/blob/7434c0ed21a154136b0145b0245ae9058005abac/src/transformers/models/time_series_transformer/modeling_time_series_transformer.py#L1292).

Comments:
cc @kashif 

@chiara-fb  so try with `lags_sequence = [1]` as the lags indices are substracted by 1
Add Translate docs into Arabic - section files CONCEPTUAL GUIDES
Author: AhmedAlmaghz
Pull request: yes

Add Translate docs into Arabic - section files CONCEPTUAL GUIDES 

---------------------------------------------------------------------------------------
# CONCEPTUAL GUIDES

- [x]  Philosophy #33064
- [x]  Glossary #33038
- [x]  What 🤗 Transformers can do #33073
- [x]  How 🤗 Transformers solve tasks #33074
- [x]  The Transformer model family #33047
- [x]  Summary of the tokenizers #33078
- [x]  Attention #33021
- [x]  Padding and truncation #33050
- [x]  BERTology #33024
- [x]  Perplexity of fixed-length models #33063
- [x]  Pipelines for webserver inference #33066
- [x]  Model training anatomy #33045
- [x] Getting the most out of LLMs #33043



Comments:
مرحبا أخي عبدالله @abodacs 
لقد حللت التعارض بطلب السحب هذا
ارجو عمل approval لطلب السحب فقد جمعت طلبات السحب السابقة هنا وقمت بإغلاقها جميعا
تحياتي 

وهذا هي المهام التالية للمراجعة

# DEVELOPER GUIDES

- [ ]  Use fast tokenizers from 🤗 Tokenizers #33034
- [ ]  Run inference with multilingual models #33048
- [ ]  Use model-specific APIs #33030
- [ ]  Share a custom model #33031
- [ ]  Templates for chat models #33026
- [ ]  Trainer #33080
- [ ]  Run training on Amazon SageMaker #33071
- [ ]  Export to ONNX #33072
- [ ]  Export to TFLite #33077
- [ ]  Export to TorchScript #33079
- [ ]  Benchmarks #33023
- [ ]  Notebooks with examples #33049
- [ ]  Community resources #33027
- [ ]  Troubleshoot #33017
- [ ]  Interoperability with GGUF files #33037


تحياتي وتقديري لك أخي @abodacs 

@stevhliu Could you please review and merge this PR?

Thank you, @stevhliu and @abodacs! 
Excellent work!

سﻻم عليكم  @AhmedAlmaghz 
بدأت العمل على مرجع للمصطلحات المسنخدمه للترجمة ﻷنى ﻻحظت اعتماد أكثر من معنى لنفس الكلمه
ملحوظة جارى العمل عليه لكنى فضلت مشاركته معك من البداية

https://docs.google.com/spreadsheets/d/1WyVvc1PMaFkpnlsO6NkYT6Sl_5XUXoXN1GC98LTMHd4/edit?usp=sharing
🚨🚨 Setting default behavior of assisted decoding
Author: jmamou
Pull request: yes

# What does this PR do?

Following https://github.com/huggingface/transformers/pull/33258#issuecomment-2361897163

Setting default behavior of assisted decoding: the number of speculative tokens per assistant model iteration is dynamically determined according to the confidence in the assistant model prediction.

By default, 
```
num_assistant_tokens=20 
num_assistant_tokens_schedule="constant" 
assistant_confidence_threshold=0.4
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? 
https://github.com/huggingface/transformers/pull/33258#issuecomment-2361897163
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@gante 

Comments:
Need to modify it.
We should set the default values only for the generation_config of the assistant model.
Currently, ConfidenceCriteria is added to any model that has generation_config.assistant_confidence_threshold > 0.  

I added `is_assistant` flag.
Fixed!

I'm consistently getting a 30% speedup over the previous default, well done @jmamou 💛 

@ArthurZucker it is not a breaking change in the sense that the API doesn't change and the model output doesn't change. Only the run time gets upgraded 🤗 

In any case, updated the PR name in case something goes wrong (cc @jmamou we add an emoji to potentially breaking PRs)

@jmamou would you kindly rebase the PR? I suspect it fixes the red CI :) 

(I hope you don't mind -- I've rebased the PR, in the hopes this PR makes the cut for our next release which will be very soon 🤗 )

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33657). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Docker container with development environment for Transformers library
Author: manuelsh
Pull request: no
Labels: name: Feature request

### Feature request

A docker container that one can use with or without GPU that has everything to start developing, if it doesn't exist and a reference to it in the `CONTRIBUTING.MD` file.

### Motivation

Reduce the barrier to start developing in any system with docker.

### Your contribution

I could build it. Would be something in the lines of:

```
FROM python:3.10

# Install system dependencies
RUN apt-get update && \
    apt-get install -y git && \
    pip install --upgrade pip --no-cache-dir && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Set a temp directory to install environment
WORKDIR /usr/src/temp

# Clone the repository
RUN git clone https://github.com/huggingface/transformers.git

# Set the working directory to the cloned repository
WORKDIR /usr/src/temp/transformers

# Install development dependencies 
RUN pip install -e .[dev]

# Go tot he working directory for development
WORKDIR /usr/src/app

```

plus a docker compose like:

```
services:
  app:
    build: .
    volumes:
      - ~:/usr/src/app
    ports:
      - "8000:8000"
    entrypoint: ["tail", "-f", "/dev/null"]
    environment:
      - RUN_SLOW=true
```

Comments:
Hi @manuelsh, thanks for opening this feature request! 

We have a suite of dockerfiles defined here: https://github.com/huggingface/transformers/tree/main/docker

Would you like to add a reference in CONTRIBUTING.md of them with a guide on how to use? 

Thanks for the answer @amyeroberts ! I saw them before, and was wondering if there is any place where they are explained (could find some of them in the documentation), maybe a README.MD in the docker folder could do it.

Also, out of all of them, which one is the one which installs the development environment? couldn't find any that runs the `pip install -e ".[dev]` command.

Oh - good point, I don't think there's a specific dev image. One thing to consider is that `pip install -e .[dev]` won't install any of the ML libraries needed to develop on models e.g. torch, tensorflow or flax, so you might need framework specific images

cc @ydshieh WDYT? 

We are mostly using

```
    container:
      image: huggingface/transformers-all-latest-gpu
```
which contains `torch` and `tensorflow` and other stuffs, but no flax/jax.
(It is built daily)

I think this is quite sufficient. An image with all 3 frameworks installed sometimes just increase the chance of chaos. (I sometimes need to uninstall tensorflow before running something).

@manuelsh would you like to add a readme on the usage of these dockers? 🤗 otherwise I can draft a small readme explaning usage, TLDR it's for our CIs which separate the frameworks! 

@ArthurZucker or @ydshieh it might be a good idea to draft a quick README indeed as it's going to be hard to know the differences/what they're used for otherwise

Fantastic to see it happening. I didn't have the context on the dockers, now I do. Thanks @ArthurZucker
Idefics: enable generation tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Enables GenerationTesterMixin for Idefics models. Tests are passing for me locally.

There are a few PRs with idefics already and they need to be reviewed/merged in the following order: https://github.com/huggingface/transformers/pull/33907 -> https://github.com/huggingface/transformers/pull/34043 -> current PR

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34062). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Idefics: fix position ids
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/33852 by cropping position ids to the length of inputs and cleans up a little bit the way inputs are prepared for generation

TF equivalence tests are failing for me locally and also failing on `main`, so those are not related to this PR

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33907). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

When this branch can be merged?

Will take a look at your comment and merge after seeing what is wrong with labels. TBH. labels should not be affected by position ids, but I'll dig into 

@ArthurZucker can you take one more look pls? I added fix for `labels`. Seems one of the very first VLMs introduced it and all other simply copied from that one. I don't see any attn mask applied in LLMs and I don't think we should be doing it under the hood, without the user knowing. So I deprecated that for 2 minor releases

> Thanks, there might be a test worth adding for this no? (we break that often for peft 😄 )

Maybe PEFT would be a better place to add this test. If so, LMK and I can work on something (probably based on [this code](https://github.com/huggingface/transformers/issues/33852#issuecomment-2389184690)). Since that would only test a single architecture, it wouldn't be super useful IMO but still better than nothing.