| 2024-07-03 09:12:28 | INFO | model_worker | args: Namespace(awq_ckpt=None, awq_groupsize=-1, awq_wbits=16, controller_address='http://localhost:21001', conv_template=None, cpu_offloading=False, debug=False, device='cuda', dtype=None, embed_in_truncate=False, enable_exllama=False, enable_xft=False, exllama_cache_8bit=False, exllama_gpu_split=None, exllama_max_seq_len=4096, gptq_act_order=False, gptq_ckpt=None, gptq_groupsize=-1, gptq_wbits=16, gpus=None, host='127.0.0.1', limit_worker_concurrency=5, load_8bit=False, max_gpu_memory=None, model_names=None, model_path='lmsys/vicuna-7b-v1.5', no_register=False, num_gpus=1, port=31000, revision='main', seed=None, ssl=False, stream_interval=2, worker_address='http://localhost:31000', xft_dtype=None, xft_max_seq_len=4096) | |
| 2024-07-03 09:12:28 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker 8276027a ... | |
| 2024-07-03 09:12:29 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/torch/storage.py:315: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. | |
| 2024-07-03 09:12:29 | ERROR | stderr | warnings.warn(message, UserWarning) | |
| 2024-07-03 09:12:29 | ERROR | stderr | Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] | |
| 2024-07-03 09:12:33 | ERROR | stderr | Loading checkpoint shards: 50%|██████████████████████████████████████▌ | 1/2 [00:04<00:04, 4.65s/it] | |
| 2024-07-03 09:12:35 | ERROR | stderr | Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00, 3.05s/it] | |
| 2024-07-03 09:12:35 | ERROR | stderr | Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00, 3.29s/it] | |
| 2024-07-03 09:12:35 | ERROR | stderr | | |
| 2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-03 09:12:35 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-03 09:12:35 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. | |
| 2024-07-03 09:12:35 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. | |
| 2024-07-03 09:12:35 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 09:13:01 | INFO | model_worker | Register to controller | |
| 2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last): | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 196, in _new_conn | |
| 2024-07-03 09:13:01 | ERROR | stderr | sock = connection.create_connection( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 85, in create_connection | |
| 2024-07-03 09:13:01 | ERROR | stderr | raise err | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 73, in create_connection | |
| 2024-07-03 09:13:01 | ERROR | stderr | sock.connect(sa) | |
| 2024-07-03 09:13:01 | ERROR | stderr | ConnectionRefusedError: [Errno 111] Connection refused | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | The above exception was the direct cause of the following exception: | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last): | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 789, in urlopen | |
| 2024-07-03 09:13:01 | ERROR | stderr | response = self._make_request( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 495, in _make_request | |
| 2024-07-03 09:13:01 | ERROR | stderr | conn.request( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 398, in request | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.endheaders() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders | |
| 2024-07-03 09:13:01 | ERROR | stderr | self._send_output(message_body, encode_chunked=encode_chunked) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.send(msg) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/lib/python3.8/http/client.py", line 951, in send | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.connect() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 236, in connect | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.sock = self._new_conn() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 211, in _new_conn | |
| 2024-07-03 09:13:01 | ERROR | stderr | raise NewConnectionError( | |
| 2024-07-03 09:13:01 | ERROR | stderr | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | The above exception was the direct cause of the following exception: | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last): | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 667, in send | |
| 2024-07-03 09:13:01 | ERROR | stderr | resp = conn.urlopen( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 843, in urlopen | |
| 2024-07-03 09:13:01 | ERROR | stderr | retries = retries.increment( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 519, in increment | |
| 2024-07-03 09:13:01 | ERROR | stderr | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] | |
| 2024-07-03 09:13:01 | ERROR | stderr | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=21001): Max retries exceeded with url: /register_worker (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused')) | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | During handling of the above exception, another exception occurred: | |
| 2024-07-03 09:13:01 | ERROR | stderr | | |
| 2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last): | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main | |
| 2024-07-03 09:13:01 | ERROR | stderr | return _run_code(code, main_globals, None, | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/lib/python3.8/runpy.py", line 87, in _run_code | |
| 2024-07-03 09:13:01 | ERROR | stderr | exec(code, run_globals) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 414, in <module> | |
| 2024-07-03 09:13:01 | ERROR | stderr | args, worker = create_model_worker() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 385, in create_model_worker | |
| 2024-07-03 09:13:01 | ERROR | stderr | worker = ModelWorker( | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 102, in __init__ | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.init_heart_beat() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/base_model_worker.py", line 81, in init_heart_beat | |
| 2024-07-03 09:13:01 | ERROR | stderr | self.register_to_controller() | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/base_model_worker.py", line 99, in register_to_controller | |
| 2024-07-03 09:13:01 | ERROR | stderr | r = requests.post(url, json=data) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 115, in post | |
| 2024-07-03 09:13:01 | ERROR | stderr | return request("post", url, data=data, json=json, **kwargs) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 59, in request | |
| 2024-07-03 09:13:01 | ERROR | stderr | return session.request(method=method, url=url, **kwargs) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 589, in request | |
| 2024-07-03 09:13:01 | ERROR | stderr | resp = self.send(prep, **send_kwargs) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 703, in send | |
| 2024-07-03 09:13:01 | ERROR | stderr | r = adapter.send(request, **kwargs) | |
| 2024-07-03 09:13:01 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 700, in send | |
| 2024-07-03 09:13:01 | ERROR | stderr | raise ConnectionError(e, request=request) | |
| 2024-07-03 09:13:01 | ERROR | stderr | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=21001): Max retries exceeded with url: /register_worker (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused')) | |