[Time series] Add PatchTSMixer
Author: ajati
Pull request: yes

[PatchTSMixer](https://arxiv.org/pdf/2306.09364.pdf) ([KDD 2023](https://dl.acm.org/doi/abs/10.1145/3580305.3599533)) is a lightweight time-series modeling approach based on the MLP-Mixer architecture. In this HuggingFace implementation, we provide PatchTSMixer's capabilities to effortlessly facilitate lightweight mixing across patches, channels, and hidden features for effective multivariate time-series modeling. It also supports various attention mechanisms starting from simple gated attention to more complex self-attention blocks that can be customized accordingly. The model can be pretrained and subsequently used for various downstream tasks such as forecasting, classification and regression.

@kashif 

Done: ~~TODOs~~

- [x] Add generate method
- [x] Make pretrained dataset publicly available
- [x] Make pretrained weights publicly available 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26247). All of your documentation changes will be reflected on that endpoint.

depends on #25927 

I am not sure I understand should I review this ? It's all red, while the other PR does not seem ready either (a lot of recent commits + red as well) ? 


ok @ArthurZucker let me have a look! will ping you when ready

> Main comments:
> 
> * updating docstrings to HF format
> * use clear variable names everywhere
> * use copied from wherever possible, especially after [[time series] Add PatchTST #25927](https://github.com/huggingface/transformers/pull/25927) is merged
> * do not use `context_values` if we already use the term `past_values` for all our time series models (keep in mind we can't change this afterwards)

Hi @NielsRogge - Thanks for your inputs. We have completed all the suggested changes. Requesting your review.  Feel free to let us know if any issue is missed out. Only pending step is "copied from PatchTST" which we will bring it back as soon as PatchTST is merged.. CC: @kashif @ajati @namctin 

Hi @patrickvonplaten @NielsRogge - We have completed most of the requested code changes. Requesting your review and approval if things are in place. 

All test-cases are passing now. 

Copied from PatchTST is still pending and will be done as soon as the PatchTST branch is merged.

CC: @kashif @namctin @ajati 

I'll review 😉 

> I don't see a conversion script so I suppose the idea was to match 1-1 the original structure but let's avoid this if possible. The code would be a lot cleaner if we have 1 class for each mode mixing. Keep it simple and isolate what can be done in a single class, then use a mapping to get the properclass. Otherwise there is too much granularity, and too many if else controlled by config arguments. It's hard to follow and not really transformers like. It's fine when it's one layer but here it's for all forward all layers. `PatchTSMixerLinearHead` Is fine with me but there are ways to make it a bit readable
> 
> Moreover depending on the mode it's most often just the input / output shape that changes. So for this we should control the linear with this arg see for example this function
> 
> ```python
>         if self.mode in ["common_channel", "mix_channel"]:
>             self.base_pt_block = nn.Sequential(
>                 nn.Dropout(head_dropout),
>                 nn.Linear(num_features, patch_len),
>             )
>         else:
>             self.base_pt_block = nn.Sequential(
>                 nn.Dropout(head_dropout),
>                 nn.Linear(num_features, patch_len * num_input_channels),
>             )
> ```
> 
> Sorry for the time sensitivity and the late review! Open to discussions if these models are super good, would be a pity not to make them into a reference implementation!

Yes. We removed "flatten" mode which is just a baseline and not used mostly. Now, we do not have else-if block in every forward call. 

Hi @ArthurZucker - We have addressed most of your comments and also the comments from other reviewers (@NielsRogge @patrickvonplaten)


TODO: CopiedFrom from PatchTST pending after its merge.

CC: @kashif @ajati @namctin 

> Looks a lot better thanks all for bearing with me 🤗 In principle looks good to me, left a few nits and will have to wait for other PR to be merged. I'll let @amyeroberts handle the last review and merge as I'm off for next week! Good work!

Sure. Thank you @ArthurZucker for your comments. We will resolve the newer comments soon and will wait for the other PR to get merged.

> Main comments:
> 
> * updating docstrings to HF format
> * use clear variable names everywhere
> * use copied from wherever possible, especially after [[time series] Add PatchTST #25927](https://github.com/huggingface/transformers/pull/25927) is merged
> * do not use `context_values` if we already use the term `past_values` for all our time series models (keep in mind we can't change this afterwards)

All corrections completed

@amyeroberts ready for review

@kashif @ajati Holding off on a review until PatchTST has been merged. There's quite a few comments I would make e.g. removing the `PatchTSTTranspose` which have been [applied to PatchTST](https://github.com/huggingface/transformers/pull/25927#discussion_r1387015192) and will be reflected there after rebasing, running `make fix-copies` and updating the code. 

All the suggested changes from PatchTST, copied from are resolved in this branch too.. Ready for review and testcases passing now.

@amyeroberts - Greetings! We have enabled all the changes suggested in PatchTST and also the corrections suggested in this PR from the past reviewers. Requesting your review and approval.

Hi @ArthurZucker -Thanks for approving this PR. We have resolved all the final changes you mentioned. Please review and help with merge, if all good.

PS: change with respect to adding segment names to docstring is the only one comment pending. When we add segment names in docstring following the syntax ssuggested - it getting auto removed during make fix-ups. Other than this - all other comments are resolved.

@vijaye12 thanks for bearing with us in this long review! and congrats for the merge! 🚀