[time series] Add PatchTST
Author: psinthong
Pull request: yes

# What does this PR do?

Adding PatchTST model https://arxiv.org/abs/2211.14730

@kashif 

## To-Do's:

- [x] Add generate method
- [x] Add additional integration test cases 
- [x] Make pretrained dataset publicly available 
- [ ] Make pretrained weights publicly available 



Comments:
@psinthong  do you have a training script I can use?

@kashif, yes, I added an example training script on a dummy dataset for our forecasting model [here](https://github.com/namctin/transformers/blob/patchtst/src/transformers/models/patchtst/patchtst_forecast_test.py). 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25927). All of your documentation changes will be reflected on that endpoint.

@ArthurZucker green!

@ArthurZucker the failing CI is due to the hub and i believe not related

Hi, it seems the attention in the PatchTST is not the vanilla attention. It uses `residual_attention` with `res_attention:bool=True`. https://github.com/yuqinie98/PatchTST/blob/204c21efe0b39603ad6e2ca640ef5896646ab1a9/PatchTST_supervised/layers/PatchTST_backbone.py#L16-L23

@Hannibal046 thats true that the official implementation had those two flags for learning the scale and RealFormer, but as far as I know those flags were always kept to `False` 

@kashif Hi, Thanks for replying! According to this [official script](https://github.com/yuqinie98/PatchTST/blob/main/PatchTST_supervised/scripts/PatchTST/traffic.sh) to launch a supervised PatchTST model for traffic dataset, the flag `res_attention` is set to the default value which is `True`.

Actually, there are two codebase in the official PatchTST repo, `PatchTST-self-supervised` and `PatchTST-supervised`. For the former, they didn't use `res_attention` but in the latter, the `res_attention` is the default and recommended setting.

@Hannibal046  can you check if the flag makes a big difference, they also didn't mention RealFormer in the paper either...

They indeed didn't mention RealFormer in the paper. I left an [issue](https://github.com/yuqinie98/PatchTST/issues/81) here and I think we should let the authors decide it is a bug or feature.

Hi, It seems that the current HF version of PatchTST doesn't include `Reversible Instance Normalization` as shown in the original repo:
https://github.com/yuqinie98/PatchTST/blob/204c21efe0b39603ad6e2ca640ef5896646ab1a9/PatchTST_supervised/layers/PatchTST_backbone.py#L63-L65

@Hannibal046 I switched out the RevIN (instance norm) with the scaling heuristics originally from the DeepAR paper which essentially serves the same purpose... If I remember, PatchTST was not learning some affine transformation to the mu and std of the context window data so it should be the same... also as in the DeepAR paper, we should append the mean and sigma to the input encodings (which would be like the Non-stationary transformer paper) Note both the RevIN and NS-transformer paper didn't mention this prior, I believe related, work/technique... 

Thanks so much for the response! Now I understand it!

I'll review this one as well 😄 


> Most of the comments I did for the other PR also apply here, so no single letter variables, comments on the line above to not always split the line as it is not necessary, avoid sequential and dummy classes, as @NielsRogge suggested, use the config when args can be taken from the config, resolving all previous comments (answering or code changes). I can review again afterwards

Hi @ArthurZucker @NielsRogge , we have addressed all of your comments. Can you please review and let us know if there is anything that we have to resolve? 

@amyeroberts  the failing tests are from the `test_run_semantic_segmentation` timing out

@amyeroberts  all the `foward` have a dedicated docstring now, so do we need to include the `add_start_docstrings_to_model_forward` decorator?


@amyeroberts I have made the doc fixes here #27476

Hi @amyeroberts, thank you for your details comments and requests. We have addressed all your concerns including adding examples on the use of these model in the modeling page. Can you please kindly review? CC @kashif @vijaye12 .

@namctin As this PR was merged it's closed and can't be reopened. Could you open a new PR for the review?