Adding MLPSpeculator support for assisted generation
Author: sahilsuneja1
Pull request: yes

# What does this PR do?
This PR adds support to use [MLPSpeculator](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) models for assisted generation, similar to it's support in [TGI](https://github.com/huggingface/text-generation-inference/pull/1865) and [vLLM](https://github.com/vllm-project/vllm/pull/4947)

Model code [originally](https://github.com/foundation-model-stack/fms-extras/blob/main/fms_extras/models/speculator.py) authored by Davis Wertheimer @daviswer

List of already existing speculators [here](https://huggingface.co/collections/ibm-ai-platform/speculators-66a1b6838f0d2327e0a3a8c3) and [here](https://huggingface.co/collections/ibm-granite/granite-speculators-664b97a44ddc5640e8cd73ac)

Training recipes new speculators [here](https://github.com/foundation-model-stack/fms-fsdp/tree/main/speculator) and [here](https://github.com/foundation-model-stack/fms-fsdp/tree/main/scripts)

Usage example:
```
import torch
import time
from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed, MLPSpeculatorPreTrainedModel

def compare_assisted_generation(prompts, checkpoint, assistant_checkpoint):
    tokenizer = AutoTokenizer.from_pretrained(checkpoint)
    inputs = tokenizer(prompts, return_tensors="pt").to(device=device)

    model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device=device, dtype=torch.bfloat16)
    assistant_model = MLPSpeculatorPreTrainedModel.from_pretrained(assistant_checkpoint).to(device=device, dtype=torch.bfloat16)
    model.eval()
    assistant_model.eval()

    if model.generation_config.pad_token_id is None:
        model.generation_config.pad_token_id = model.generation_config.eos_token_id

    generate_kwargs = {
        "do_sample":False,
        "temperature":None,
        "max_new_tokens":50,
        "output_hidden_states":True,
    }

    # warmup
    for _ in range(0,2):
        model.generate(**inputs, **generate_kwargs)
        model.generate(**inputs,  assistant_model=assistant_model, **generate_kwargs)

    start_time = time.time()
    outputs = model.generate(**inputs, **generate_kwargs)
    end_time = time.time()
    print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
    print(f"Generation without assistant; Time taken: {end_time-start_time} seconds")

    start_time = time.time()
    outputs = model.generate(**inputs,  assistant_model=assistant_model, **generate_kwargs)
    end_time = time.time()
    print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
    print(f"Generation with assistant; Time taken: {end_time-start_time} seconds")


torch.set_grad_enabled(False)
prompt = "Alice and Bob"
checkpoint = "meta-llama/Meta-Llama-3-8B-Instruct"
speculator_checkpoint = "ibm-ai-platform/llama3-8b-accelerator"
compare_assisted_generation(prompt, checkpoint, speculator_checkpoint)

```

Output from the above example on A100:
```
['Alice and Bob are two friends who are trying to solve a puzzle. They are given a set of numbers, and they need to find the sum of the numbers that are multiples of 3 or 5.\n\nHere is the set of numbers: 1, ']
Generation without assistant; Time taken: 1.150806188583374 seconds
['Alice and Bob are two friends who are trying to solve a puzzle. They are given a set of numbers, and they need to find the sum of the numbers that are multiples of 3 or 5.\n\nHere is the set of numbers: 1, ']
Generation with assistant; Time taken: 0.6626832485198975 seconds
```


## Who can review?
@gante 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hey @sahilsuneja1 👋 

We're currently pausing the addition of all non-critical decoding methods, including assisted generation variations. This is because we're designing a new way of adding transformers-compatible decoding methods (see [this draft PR](https://github.com/huggingface/transformers/pull/36405))

TL;DR, if the plan goes forward, new decoding methods will live on the hub, and `transformers` will only hold the core decoding strategies 🤗 

Thanks @gante, will track progress on it and revisit when the change is made!