added the max_matching_ngram_size to GenerationConfig
Author: mosheber
Pull request: yes

# What does this PR do?

* Added the max_matching_ngram_size parameter into the GenerationConfig, for the PromptLookupCandidateGenerator.
* Included the max_matching_ngram_size when calling the __init__ of PromptLookupCandidateGenerator in   _get_candidate_generator, in case it is specified. 

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante , would appreciate it if you could give this PR a glance, and thank you in advance.


Comments:
Hi @mosheber 👋 

I'd be happy to merge the PR, conditional on the answer to the following question being yes (preferably backed with data): have you found significant benefits of changing the flag you added?

On the [original issue](https://github.com/huggingface/transformers/issues/27722), the author's experiments showed that there were little benefits in changing this option. As such, we don't want to add new flags unless they result in clear benefits :)

Hi @gante @mosheber 👋 

The results below show a `3ms` speedup in latency with a 7B target model, when comparing the default `max_matching_ngram_size=2` with `max_matching_ngram_size=4`.
The bottom graph shows `n_matches` vs `max_matching_ngram_size`.
**As the target size increases further, the speedup will be greater.**
Note that these results use an optimized routine for PLD subsequence matching (~`70x` faster on avg, uses `numba`): 
```bash
get_candidates:       0.3467 ms
get_candidates_opt:   0.0047 ms
```

![Screenshot 2024-02-26 at 18 17 31](https://github.com/huggingface/transformers/assets/32893314/89cb9242-baab-4282-9165-1fb2aa72522e)
![Screenshot 2024-02-26 at 18 17 11](https://github.com/huggingface/transformers/assets/32893314/ac83f2a6-c198-45f3-b1d7-08edc7dde9e0)


@danielkorat convinced by your numbers 👍 

Let's add this PR!

@mosheber after applying the fix, please run `make fixup` from the `transformers` folder and commit the result. It will fix the CI issues you're seeing :)

> @mosheber after applying the fix, please run `make fixup` from the `transformers` folder and commit the result. It will fix the CI issues you're seeing :)

I ran the `make fixup` as well.

> @mosheber
> 
> Actually, there is a tiny missing thing: `max_matching_ngram_size` should have an entry in the docstring of `GenerationConfig`
> 
> our CI is still complaining about code formating, make sure you have the latest version installed when running `make fixup` again :)

Great idea! I added the doc string to the GenerationConfig class. 
Regarding the tests, I fixed the formatting issue, it seems that the CI fails due to timing reasons, which are not related to this PR. 

> Regarding the tests, I fixed the formatting issue, it seems that the CI fails due to timing reasons, which are not related to this PR.

Yeah don't worry about it :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29131). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.