Repository: xorbitsai/inference

Issue:
BUG: pytorch stream gen produces last token twice
### Describe the bug
```python
In [1]: from xinference.client import RESTfulClient

In [2]: uid = 'db10d45c-431e-11ee-97cf-fa163e74fa2d'

In [3]: client = RESTfulClient('http://127.0.0.1:13334')

In [4]: m = client.get_model(uid)

In [5]: for c in m.generate('Once upon a while,', generate_config={'stream': True, 'max_tokens': 10}):
   ...:     print(c)
   ...:
{'id': '14072df6-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' I', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '1409d43e-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' saw a', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '140c45c0-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' group of', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '140eb1a2-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' guys,', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '14111992-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' who were', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '1412657c-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' drunk', 'index': 0, 'logprobs': None, 'finish_reason': None}]}
{'id': '14128b9c-431f-11ee-beb1-fa163e74fa2d', 'object': 'text_completion', 'created': 1692951125, 'model': 'db10d45c-431e-11ee-97cf-fa163e74fa2d', 'choices': [{'text': ' drunk', 'index': 0, 'logprobs': None, 'finish_reason': 'length'}]}
```

### To Reproduce
To help us to reproduce this bug, please provide information below:

1. Your Python version.
2. The version of xinference you use.
3. Versions of crucial packages.
4. Full stack of the error.
5. Minimized code to reproduce the error.

### Expected behavior
A clear and concise description of what you expected to happen.

### Additional context
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/xinference/model/llm/pytorch/utils.py b/xinference/model/llm/pytorch/utils.py
--- a/xinference/model/llm/pytorch/utils.py
+++ b/xinference/model/llm/pytorch/utils.py
@@ -282,15 +282,15 @@ def generate_stream(
             break
 
     # finish stream event, which contains finish reason
-    if i == max_new_tokens - 1:
-        finish_reason = "length"
-    elif stopped:
+    if stopped:
         finish_reason = "stop"
+    elif i == max_new_tokens - 1:
+        finish_reason = "length"
     else:
         finish_reason = None
 
     completion_choice = CompletionChoice(
-        text=output, index=0, logprobs=None, finish_reason=finish_reason
+        text="", index=0, logprobs=None, finish_reason=finish_reason
     )
     completion_chunk = CompletionChunk(
         id=str(uuid.uuid1()),


Test patch: