An ability to pass a function to tokenizer to transform prompt
Author: nikitastaf1996
Pull request: no

### Feature request

Models often work best with specific prompt template.And sometimes [prefix](https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/configuration#transformers.PretrainedConfig.prefix) is not enough.
An ability to pass function to transform prompt would be excellent.

Example
```
def transform_prompt(prompt:str) -> str:
return f"""
### Instruction: {prompt}
### Assistant:
"""

pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer,max_new_tokens=256,temperature=0.1,transform=transform_prompt)
```

Providing text template is also an option.But in my belief it would be more holistic approach.

### Motivation

I am a little frustrated with the need to define CustomLLM in langchain in order to accomodate the model prompt template. And i believe thats not the only use case for it. Many models require special prompt template.

### Your contribution

I am sorry but no. I am not python programmer.

Comments:
Hi @nikitastaf1996, thanks for raising this issue. 

So that we can best understand the proposed feature - could you explain a bit more about the utility this would unlock? At the moment, it seems that a user could transform their prompts as needed before passing to the pipeline e.g.:

```python
def transform(prompt: str) -> str:
    ...

prompt = transform(original_prompt_string)
pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer, max_new_tokens=256, temperature=0.1)
outputs = pipeline(prompt)
```

cc @Narsil @gante 

I will give you code. You will decide if it nessesary to implement. After some time has passed i am not so sure it is nesserally at all. It might be niche nuisance exclusive to me,
Current code i use:
```
#@title Custom llm
from langchain.llms.base import LLM
from typing import Optional, List, Mapping, Any
from transformers import TextGenerationPipeline
class CustomLLM(LLM):
        
    @property
    def _llm_type(self) -> str:
        return "custom"
    
    def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
      pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer,max_new_tokens=256,temperature=0.1)
      prompt = f"""User:{prompt}
      Assistant:"""
      prompt_len = len(prompt)
      result = pipeline(prompt)[0]["generated_text"][prompt_len:]
      if stop is not None:
        for stop_string in stop:
          index = result.find(stop_string)  # find the index of the substring
          if index != -1:
              result = result[:index + len(stop_string)]
              break
      return result
```


```
#@title Langchain Agent
from langchain.agents import load_tools,initialize_agent,AgentType

llm = CustomLLM()
tools = load_tools(["terminal","python_repl"])
agent = initialize_agent(tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,verbose=True,handle_parsing_errors=True)
agent.run("Please install ffmpeg")
```

What i would like to see:
```
from langchain.agents import load_tools,initialize_agent,AgentType
from langchain.llms import HuggingFacePipeline
from transformers import pipeline

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer,max_new_tokens=256,temperature=0.1,transform=TRANSFORMFUNCTION)
llm = HuggingFacePipeline(pipeline=pipe)
tools = load_tools(["terminal","python_repl"])
agent = initialize_agent(tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,verbose=True,handle_parsing_errors=True)
agent.run("Please install ffmpeg")
```

I think your custom LLM is perfectly fine imo.

You have other ways to define it actually. Introducting `transform` function would bloat everything up IMHO and what you really need to do is just send a preprompted text to the pipelines like what you are doing. The `stop` part can also be implemented with `generate` arguments I think. (`stop_sequence`)

https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/text_generation.py#L138

The it's finished.