Add a new script to check model testers' config
Author: ydshieh
Pull request: yes

# What does this PR do?

Add a new check to check model testers will give tiny config.

The objective is to ensure no test will run with large configuration values (such as `BridgeTowerModelTest(er)`, except the integration or slow tests.

The check is not added to PR/daily CI workflow: we don't want to add more burden to contributors.

### The effect
<img width="720" alt="Screenshot 2023-03-09 212533" src="https://user-images.githubusercontent.com/2521628/224147644-62fb5f9c-60e1-4801-b257-08d9679cb06b.png">



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Request a review from @amyeroberts too - to celebrate her new core maintainer role 🔥 🚀 


> I think that's way too specific

I agree that `100` is way to specific.

> and will require lots of exceptions

Not really a lot of exceptions. So far, there are `245` errors given by this new check.
- `max_position_embeddings`: 78 places --> this is better to be changed to smaller values
- `hidden_size` or `xxx_dim`: 41 places --> also better to be changed
- `xxx_size` (other than `hidden_size`): 39 places  --> I guess better to change
- `xxx_token_id`: 19 places --> this could be skipped (they look weird, but it doesn't really matter)
- **we can skip other places for now**

> It would be more helpful to have something in a PR added a new model that would extract the time the tests took (as reported in the artifacts) and put it somewhere clearly visible, for instance the comment regarding the documentation.

The `test_torch` job of PR #20775 (Add BridgeTower model) took `37` minutes, while it took about `33` minutes on nightly run one day before. So yes, this might be a good way to look. However:
- even if we show the timing, **without the timing of the previous (full) run**, no one really knows if we should look into the timing/speed issue, and we/contributor just would not pay attention
  - try to grab  (automatically) **without the timing of the previous (full) run** seems to me not a super easy task
- we will have to identity PRs that are new model addition PRs (use diff tool?) + we will have to show in PR comments
  - This, plus the necessity to grab the previous running time, seems to me requiring much more work than just simply fix things I mentioned in the first part above

So let me continue a bit and see what would happen!




I also don't think this check is super valuable as it will add another burden on the contributors for something that really only affects us (we're the ones to handle CI, memory, timeouts, etc.).

I think aiming for tiny tiny models is great, but it's not the end of the world if a few slip through the cracks which we have to correct after.

> it will add another burden on the contributors

Yeah, that's a super valid point. I don't feel strong, so more than happy to close the PR.

But still give you a rough numbers: 

> if a few slip through the cracks which we have to correct after.

currently there are `245` places this check identified, a few examples

```bash
config["max_speech_positions"] = 4000 which is too large for testing!
config["num_block_records"] = 13353718 which is too large for testing!
config["max_2d_position_embeddings"] = 1024 which is too large for testing!
```

Without the check, more such cases will accumulate (but most of them are not as extreme as `BridgeModelTest` . Also maybe we will only have to deal them after a long long long period of time.


_The documentation is not available anymore as the PR was closed or merged._

Well, although we are not going to add a check on PR CI, I  think it might be handy if we add the script - just in case when we need to perform housekeeping. It's not complex this new script, but always better if it is already there when we need it.

I also change the check to check just a few major attributes for now.

@sgugger @LysandreJik Let me know if you are happy with this addition (without it being added to CI workflows).