add model scaling section
Author: lvwerra
Pull request: yes

# What does this PR do?

This PR adds a practical scaling guide to the documentation. When starting to work with large transformer models one of the most common issues is running out of GPU memory. There are several easy to implement strategies to counteract this. This guide shows how to use gradient accumulation, gradient checkpointing, mixed precision training, and choice of optimizer to decrease the memory footprint. It shows how to do this with both the `Trainer` and `accelerate` in a hands-on fashion.

## What does this PR not do?
This PR does **not** replace the in-depth explanations at ["Model Parallelism"](https://huggingface.co/docs/transformers/parallelism) and ["Performance and Scalability: How To Fit a Bigger Model and Train It Faster"](https://huggingface.co/docs/transformers/performance). Rather the proposed guide should be an easy entry point for new users whereas these other sections are for more advanced users.

## TensorFlow?
If this guide resonates well it would make sense to extend it with a TensorFlow/Keras section (similar to the `accelerate` section). Maybe TensorFlow boy (aka @Rocketknight1) has some ideas?

Comments:
We could definitely make a TF/Keras section! I believe Keras doesn't support gradient accumulation or gradient checkpointing out of the box, though, but mixed precision, Adafactor, etc. are all very doable.

Hi @sgugger I integrated your feedback, let me know if this is what you had in mind. 

In the case of documentation, I don't think repetition is a bad thing since users rarely look at every page in a row. Making sure the docs are all linked to each other for further reading is important. The two doc pages reach different target audience IMO and present the techniques in different ways, so I think having both of them is good.


Repetition is a bad thing in this situation, IMHO:

1. as it'd make it more difficult to point users to the right doc. For example @LysandreJik has been pointing users to the performance doc when they have difficulties with getting their model to run memory-wise. Now you will have 2 competing docs, which one should he point users to? It shouldn't take more than a few secs for a maintainer to find where to point the user to and not needing to reread each doc and contemplate is it A or B?
2. how do you maintain 2 overlapping documents?
3. some users do read docs and now they will need to read 2 docs with confusing overlaps

But the big question is this: why make things more complicated and messy when the whole performance/scalability documentation is due for a revamp anyway. It's far from great at the moment. I propose to discuss how we best serve both users and ourselves by improving the performance/scalability doc structure/layout. I'm not proposing to drop anything from the current PR but to integrate rather than creating a diverging doc.

and I'd be happy to work together with @lvwerra to make it happen.

update: @lvwerra and I will discuss a revamp/integration on Fri.

Hi there, I updated the doc and this is ready for another review:
- use @stas00's suggestion to initialize the measurement by loading a small tensor to the GPU
- added a note about the embedding issue with BnB optimizer

I disusssed with @stas00 that we could start the revamp by merging this PR and then move things around as we start adding the other documents in #15213. 

Good point, I merged `scaling.mdx` and `performance.mdx`. The document starts with the new practical guide and I added the existing material to a section called `Further discussions`. There is still a bit of redundancy between the two parts but I think we can refactor it when we flesh out the other parts.

@sgugger are you happy with the current state to be merged?

Thanks @sgugger, I integrated your feedback! @stas00 do you want to have a last look before merging?

Thanks, @lvwerra 

1. Please revert all the header level changes in the existing doc - the menu on the right only handles ## and ### and your proposed change will practically eliminate most of the menu entries as you added an additional level

2. We discussed adding this PR's content at the end of the existing document as at the moment it's the extra and not the main document as it was laid out. 95% of this PR's material will be spread out over the new documents, and the current performance doc will remain here as a reference that the more specific docs will point to for indepth information. (e.g. mixed precision section we aren't going to repeat the full discussion about why and how in each sub-document and will point to the performance reference doc instead). We may rename the main reference doc and make performance.mdx the entry point instead.

Does it make sense? If not, and we need to sync on the big picture - let's discuss it then first. 

Hi @stas00 

Sure, I'll revert the headers, I was not aware of that limitation.

Regarding the order of the content: I know that we will change that document radically in later PRs but I thought since this is gonna be public and maybe here for a few weeks it would be nicer to have the practical guide at the beginning and the theoretical part at the end. If you feel strongly about this I can change it back. 

> Hi @stas00
> 
> Sure, I'll revert the headers, I was not aware of that limitation.

In fact I asked when the new design was added to support more than 3-levels but nothing came out of that. i.e. the menu is missing already a bunch of entries where there is a deeper nesting like the deepspeed doc. :(

> Regarding the order of the content: I know that we will change that document radically in later PRs but I thought since this is gonna be public and maybe here for a few weeks it would be nicer to have the practical guide at the beginning and the theoretical part at the end. If you feel strongly about this I can change it back.

Sure, that works. I think what I will do instead in the new PR is to move this original document out to a new document. I'm just concerned with broken links for any pre-existing links to this url.


Thanks a lot for working on this important part of the docs!