Add distribution params to time series output
Author: benHeid
Pull request: yes

# What does this PR do?

This PR is adding the distribution params and the name of the distribution to the `SampleTSPredictionOutput`. Additionally, it enables the distribution outputs in the autoformer, informer, and time series transformer. 
<!-- Remove if not applicable -->

Fixes #29556 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

I extended existing tests

## Who can review?

@kashif 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
thanks @benHeid having a look!

@benHeid Can we confirm that the param output is not `num_samples` copies of the parameter?

> @benHeid Can we confirm that the param output is not `num_samples` copies of the parameter?

Sorry for the late answer. I was a bit blocked with various stuff the last weeks.

* For the Informer, each time step a new param is created. This param is fed into the distribution object to create the next sample. Thus, I assume that this is definitely not the same for each future value. 
* For the TimeSeriesModel Transformer, the same argument as for the Informer could be applied. 
* For the autoformer, the params are the last prediction_length values of the parameter_projection (several neural networks). Thus, this should also not be a copy.
If desired, I can add a test regarding that.


Furthermore, I just seen that we also need to report the loc and scale values from the normalization step. Here I am wondering, if it might be possible to adapt the parameters in a way that they are reflecting the scale and loc. However, I am unsure if it is that easy for each distribution.. Otherwise, we could just return these values too. 



@benHeid  yes so for gaussian and student-t the loc and scale can be used to get the parameters of the corresponding scaled distribution. However for neg. bin. i haven't figured out how to do that with the `loc` parameter... and so only the No-ops scaling and "mean" scaling can be used at the moment with neg. bin. If you can figure out how the loc param can be applied to the neg. bin params I would appreciate it. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@kashif unfortunately, I haven't found a solution. Thus, I would propose that the scaling information are also passed to `SampleTSPredictionOutput` so that the user has the possibility to use them.

yes that would be ideal... and then one can either create the `(distrb(scaled_params).sample()  + loc) * scale` or via a `transformed` distribution

Exactly. I will check again if everything is working as expected locally and ping you again here.

@kashif, I added the scaling params. Thus, I suppose it is ready for review now :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29693). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Regarding the failed tests. It seems that at least a significant part of the failure is not related to my changes or? E.g., I haven't changed any examples and also the linter is proposing file changes in files that I haven't touched.

@benHeid agree... let me check

I applied fix-copies and pushed. Is this correct or should I revert this? 

lets revert all the un-related changes... hopefully that also fixes the merge conflicts?

I reverted. The only files that are getting change by fix-copies are: modeling_bigbird_pegasus.py, modeling_cohere.py, modeling_gemma.py, and test_tokenization_roc_bert.py. Since I haven't changed them I reverted these changes.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@kashif what is the current state ? 

i believe the last thing was to get the API also working with patchTST