Repository: xorbitsai/inference

Issue:
BUG: Unable to stream response on GPT-2
### Describe the bug
An error occurs when I stream response using the GPT-2 model.

### To Reproduce
Versions:
Python 3.11.4
Latest Github commit of Xinference

Minimal Code:
```python
from xinference.client import Client

# Get model
client = Client("http://localhost:9997")
model_uid = client.launch_model(model_name="gpt-2", model_format="ggmlv3")
model = client.get_model(model_uid)

# Define the prompt and generate
prompt = "What is the largest animal?"

result = model.generate(
    prompt,
    generate_config={
        "max_tokens": 5, 
        "stream": True
    }
)

# Print the results
for chunk in result:
    print(chunk["choices"][0]["text"], end="")
```

Error:
```
Exception: [address=127.0.0.1:54027, pid=19432] Error when sending message 5f371380c6de21150c71e241ba67f117ef2599dd8fc86f662801d8971cbf5ae7. Caused by AttributeError("Can't pickle local object 'DecoratorBaseModel.check_args'"). 
Original error: ValidationError(model='Generate', errors=[{'loc': ('stream',), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}])
```

<details> 
<summary>Full Error Log</summary> 

```
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[24], line 21
     12 result = model.generate(
     13     prompt,
     14     generate_config={
   (...)
     17     }
     18 )
     20 # Print the results
---> 21 for chunk in result:
     22     time.sleep(0.1)
     23     print(chunk["choices"][0]["text"], end="")

File ~/anaconda3/lib/python3.11/site-packages/xinference/client/common.py:49, in streaming_response_iterator(response_lines)
     47 error = data.get("error")
     48 if error is not None:
---> 49     raise Exception(str(error))
     50 yield data

Exception: [address=127.0.0.1:54027, pid=19432] Error when sending message 5f371380c6de21150c71e241ba67f117ef2599dd8fc86f662801d8971cbf5ae7. Caused by AttributeError("Can't pickle local object 'DecoratorBaseModel.check_args'"). 
Original error: ValidationError(model='Generate', errors=[{'loc': ('stream',), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}])Traceback: 
  File "/Users/username/anaconda3/lib/python3.11/site-packages/xoscar/backends/pool.py", line 657, in send
    result = await self._run_coro(message.message_id, coro)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xoscar/backends/pool.py", line 368, in _run_coro
    return await coro
           ^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xoscar/api.py", line 306, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "xoscar/core.pyx", line 558, in __on_receive__

  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__

  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__

  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/core/utils.py", line 33, in wrapped
    ret = await func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/core/model.py", line 77, in wrapped_func
    ret = await fn(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/core/model.py", line 250, in generate
    return await self._call_wrapper(_wrapper)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/core/model.py", line 214, in _call_wrapper
    return await asyncio.to_thread(_wrapper)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/core/model.py", line 240, in _wrapper
    getattr(self._model, "generate")(prompt, *args, **kwargs)

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/model/llm/ggml/ctransformers.py", line 235, in generate
    generate_config = self._sanitize_generate_config(generate_config_raw)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/username/anaconda3/lib/python3.11/site-packages/xinference/model/llm/ggml/ctransformers.py", line 161, in _sanitize_generate_config
    **CreateCompletionCTransformers(**generate_config).dict()
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
```
</details>

### Expected behavior
There should be five tokens generated and printed.

### Additional context
Seems like an issue specific to models using the `ctransformers` library.



Discussion:

Resolving patch:
diff --git a/xinference/types.py b/xinference/types.py
--- a/xinference/types.py
+++ b/xinference/types.py
@@ -355,7 +355,10 @@ class CreateCompletionTorch(BaseModel):
     CreateCompletionCTransformers = get_pydantic_model_from_method(
         LLM.generate,
         exclude_fields=["tokens"],
-        include_fields={"max_tokens": (Optional[int], max_tokens_field)},
+        include_fields={
+            "max_tokens": (Optional[int], max_tokens_field),
+            "stream": (Optional[bool], stream_field),
+        },
     )
 except ImportError:
     CreateCompletionCTransformers = create_model("CreateCompletionCTransformers")


Test patch: