Streaming final agent response callbacks
Labels: name: enhancement

This is a comprehensive version of callbacks when we want to streaming the only the last response of the agent.

This fix issue #2483 but in a more comprehensive way. This is NOT duplicate of #4630 .

I know there is a great work has been done by @UmerHA for **FinalStreamingStdOutCallbackHandler** class, and this PR work is inspired by him, but I want that class to be a simple use case for final streaming. What this PR does is a comprehensive version of final streaming with the following additional feature:

- **Multiple answer prefix phrases**: Just like OpenAI they allows multiple stop sequences, this class can detect multiple stop sequences before it begin streaming.
- **Post-process on-the-fly**: This utilize the above algorithm (matching multiple prefix phrases) so I think this is suitable to place it here.
- **Easy-to-use Callback** on each new token **during streaming of final agent response**.
- **Abnormal streaming detection**: if the streaming contains unexpected tokens generated by model, then raise an Exception.

#### Quick setup

```python
from langchain.agents import load_tools, initialize_agent, AgentType
from langchain.llms import OpenAI
from langchain.callbacks import StreamingLastResponseCallbackHandler

llm = OpenAI(temperature=0, streaming=True)
tools = load_tools(["serpapi", "llm-math"], llm=llm)
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False)
```

#### Usage 1 - use as callback function for **last response new token**
```python
stream = StreamingLastResponseCallbackHandler.from_agent_type(agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
@stream.on_last_response_new_token()
def on_new_token(token: str):
    if token is StopIteration:
        print("\n[Done]")
        return
    else:
        print(token, end="", flush=True)

agent.run("Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?", callbacks=[stream])
```
#### Usage 2 - use as iterator
```python
stream = StreamingLastResponseCallbackHandler.from_agent_type(agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
def _run():
    agent.run("Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?", callbacks=[stream])
threading.Thread(target=_run).start()

for token in stream:
    print(token, end="", flush=True)
```

#### Usage 3 - Post process on-the-fly
```python
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
stream = StreamingLastResponseCallbackHandler.from_agent_type(agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)

@stream.postprocess(sliding_window_step=1, window_size=3)
def postprocess_func(tokens: List[str]) -> List[str]:
    sentence = "".join(tokens).replace("LLM", "LangChain")
    words = [enc.decode([t]) for t in enc.encode(sentence)]  # postprocess output can have different size!
    return words

def _run():
    agent.run("Say 'Large Language Model (LLM) is great!'", callbacks=[stream])
threading.Thread(target=_run).start()

for token in stream:
    print(token, end="", flush=True)
```

#### Who can review?

Hi @hwchase17 and @agola11, please review my PR, I do have an usage example in the docstring of this **StreamingLastResponseCallbackHandler** class.

I appreciate any feedback, please review my PR soon.

P/s: Updated using decorator approach.