2024-07-05 07:25:35 | INFO | model_worker | args: Namespace(awq_ckpt=None, awq_groupsize=-1, awq_wbits=16, controller_address='http://127.0.0.1:21002', conv_template=None, cpu_offloading=False, debug=False, device='cuda', dtype=None, embed_in_truncate=False, enable_exllama=False, enable_xft=False, exllama_cache_8bit=False, exllama_gpu_split=None, exllama_max_seq_len=4096, gptq_act_order=False, gptq_ckpt=None, gptq_groupsize=-1, gptq_wbits=16, gpus=None, host='127.0.0.1', limit_worker_concurrency=5, load_8bit=False, max_gpu_memory=None, model_names=None, model_path='lmsys/vicuna-7b-v1.5', no_register=False, num_gpus=1, port=21003, revision='main', seed=None, ssl=False, stream_interval=2, worker_address='http://127.0.0.1:21003', xft_dtype=None, xft_max_seq_len=4096)
2024-07-05 07:25:35 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker da6b602e ...
2024-07-05 07:25:36 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/torch/storage.py:315: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.
2024-07-05 07:25:36 | ERROR | stderr |   warnings.warn(message, UserWarning)
2024-07-05 07:25:36 | ERROR | stderr | 
Loading checkpoint shards:   0%|            | 0/2 [00:00<?, ?it/s]
2024-07-05 07:25:41 | ERROR | stderr | 
Loading checkpoint shards:  50%|██  | 1/2 [00:04<00:04,  4.79s/it]
2024-07-05 07:25:43 | ERROR | stderr | 
Loading checkpoint shards: 100%|████| 2/2 [00:06<00:00,  3.15s/it]
2024-07-05 07:25:43 | ERROR | stderr | 
Loading checkpoint shards: 100%|████| 2/2 [00:06<00:00,  3.39s/it]
2024-07-05 07:25:43 | ERROR | stderr | 
2024-07-05 07:25:43 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-05 07:25:43 | ERROR | stderr |   warnings.warn(
2024-07-05 07:25:43 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-05 07:25:43 | ERROR | stderr |   warnings.warn(
2024-07-05 07:25:43 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
2024-07-05 07:25:43 | ERROR | stderr |   warnings.warn(
2024-07-05 07:25:43 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.
2024-07-05 07:25:43 | ERROR | stderr |   warnings.warn(
2024-07-05 07:25:59 | INFO | model_worker | Register to controller
2024-07-05 07:25:59 | ERROR | stderr | INFO:     Started server process [89278]
2024-07-05 07:25:59 | ERROR | stderr | INFO:     Waiting for application startup.
2024-07-05 07:25:59 | ERROR | stderr | INFO:     Application startup complete.
2024-07-05 07:25:59 | ERROR | stderr | INFO:     Uvicorn running on http://127.0.0.1:21003 (Press CTRL+C to quit)
2024-07-05 07:26:25 | INFO | stdout | INFO:     127.0.0.1:53916 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-07-05 07:26:39 | INFO | stdout | INFO:     127.0.0.1:48166 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-07-05 07:26:44 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:27:29 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:28:14 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:28:59 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:29:44 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:30:29 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:31:14 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:31:59 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:32:44 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:33:29 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:34:14 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:35:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:35:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:36:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:37:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:38:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:38:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:39:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:40:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:41:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:41:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:42:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:43:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:44:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:44:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:45:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:46:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:47:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:47:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:48:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:49:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:50:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:50:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:51:30 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:52:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:53:00 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:53:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:54:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:55:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:56:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:56:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:57:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:58:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:59:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 07:59:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:00:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:01:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:02:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:02:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:03:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:04:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:05:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:05:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:06:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:07:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:08:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:08:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:09:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:10:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:11:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:11:46 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:12:31 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:13:16 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:14:01 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:14:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:15:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:16:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:17:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:17:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:18:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:19:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:20:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:20:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:21:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:22:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:23:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:23:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:24:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:25:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:26:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:26:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:27:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:28:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:29:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:29:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:30:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:31:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:32:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:32:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:33:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:34:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:35:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:35:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:36:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:37:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:38:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:38:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:39:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:40:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:41:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:41:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:42:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:43:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:44:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:44:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:45:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:46:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:47:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:47:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:48:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:49:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:50:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:50:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:51:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:52:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:53:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:53:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:54:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:55:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:56:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:56:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:57:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:58:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:59:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 08:59:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:00:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:01:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:02:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:02:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:03:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:04:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:05:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:05:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:06:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:07:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:08:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:08:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:09:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:10:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:11:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:11:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:12:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:13:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:14:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:14:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:15:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:16:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:17:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:17:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:18:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:19:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:20:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:20:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:21:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:22:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:23:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:23:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:24:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:25:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:26:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:26:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:27:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:28:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:29:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:29:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:30:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:31:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:32:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:32:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:33:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:34:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:35:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:35:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:36:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:37:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:38:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:38:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:39:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:40:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:41:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:41:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:42:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:43:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:44:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:44:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:45:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:46:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:47:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:47:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:48:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:49:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:50:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:50:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:51:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:52:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:53:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:53:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:54:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:55:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:56:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e. 
2024-07-05 09:56:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: da6b602e.