2024-02-26 21:43:32 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40000, worker_address='http://localhost:40000', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-05B-Chat', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False)
2024-02-26 21:43:32 | INFO | model_worker | Loading the model ['MobiLlama-05B-Chat'] on worker c61286d6 ...
2024-02-26 21:43:34 | INFO | model_worker | Register to controller
2024-02-26 21:43:34 | ERROR | stderr | INFO:     Started server process [455216]
2024-02-26 21:43:34 | ERROR | stderr | INFO:     Waiting for application startup.
2024-02-26 21:43:34 | ERROR | stderr | INFO:     Application startup complete.
2024-02-26 21:43:34 | ERROR | stderr | INFO:     Uvicorn running on http://0.0.0.0:40000 (Press CTRL+C to quit)
2024-02-26 21:44:19 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: None. call_ct: 0. worker_id: c61286d6. 
2024-02-26 21:45:04 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: None. call_ct: 0. worker_id: c61286d6. 
2024-02-26 21:45:49 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: None. call_ct: 0. worker_id: c61286d6. 
2024-02-26 21:46:01 | INFO | stdout | INFO:     127.0.0.1:52628 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:46:20 | INFO | stdout | INFO:     127.0.0.1:34082 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:46:34 | INFO | stdout | INFO:     127.0.0.1:49126 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-02-26 21:46:34 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=4, locked=False). call_ct: 1. worker_id: c61286d6. 
2024-02-26 21:47:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: c61286d6. 
2024-02-26 21:47:22 | INFO | stdout | INFO:     127.0.0.1:34094 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-02-26 21:48:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:48:50 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:49:35 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:50:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:51:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:51:50 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:52:00 | INFO | stdout | INFO:     127.0.0.1:34224 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:52:20 | INFO | stdout | INFO:     127.0.0.1:44660 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:52:35 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:53:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:53:23 | INFO | stdout | INFO:     127.0.0.1:44436 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:53:54 | INFO | stdout | INFO:     127.0.0.1:46838 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:54:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:54:50 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:54:56 | INFO | stdout | INFO:     127.0.0.1:34368 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:55:14 | INFO | stdout | INFO:     127.0.0.1:57892 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:55:35 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:56:16 | INFO | stdout | INFO:     127.0.0.1:45362 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:56:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:56:49 | INFO | stdout | INFO:     127.0.0.1:56122 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:57:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:57:19 | INFO | stdout | INFO:     127.0.0.1:36316 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:57:36 | INFO | stdout | INFO:     127.0.0.1:52210 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:57:50 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:58:35 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 21:59:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:00:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:00:37 | INFO | stdout | INFO:     127.0.0.1:37368 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:00:50 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:01:01 | INFO | stdout | INFO:     127.0.0.1:59848 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:24 | INFO | stdout | INFO:     127.0.0.1:35582 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:35 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:01:43 | INFO | stdout | INFO:     127.0.0.1:52444 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:02:20 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:02:37 | INFO | stdout | INFO:     127.0.0.1:54302 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:02:55 | INFO | stdout | INFO:     127.0.0.1:60566 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:03:05 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:03:35 | INFO | stdout | INFO:     127.0.0.1:59700 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:03:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:03:56 | INFO | stdout | INFO:     127.0.0.1:44720 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:04:24 | INFO | stdout | INFO:     127.0.0.1:58308 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:04:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:04:49 | INFO | stdout | INFO:     127.0.0.1:56838 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:05:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:06:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:06:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:07:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:07:46 | INFO | stdout | INFO:     127.0.0.1:50666 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:08:12 | INFO | stdout | INFO:     127.0.0.1:48668 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:08:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:09:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:09:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:10:01 | INFO | stdout | INFO:     127.0.0.1:43170 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:10:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:11:09 | INFO | stdout | INFO:     127.0.0.1:44282 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:11:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:12:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:12:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:13:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:14:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:14:41 | INFO | stdout | INFO:     127.0.0.1:52596 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:14:59 | INFO | stdout | INFO:     127.0.0.1:50848 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:15:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:15:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:16:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:17:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:17:57 | INFO | stdout | INFO:     127.0.0.1:50102 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:18:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:18:14 | INFO | stdout | INFO:     127.0.0.1:54672 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:18:51 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:19:36 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:20:21 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:21:06 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:21:52 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:22:37 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:23:22 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:24:07 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:24:52 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:25:37 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:26:22 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:27:07 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 2. worker_id: c61286d6. 
2024-02-26 22:27:31 | ERROR | stderr | INFO:     Shutting down
2024-02-26 22:27:31 | ERROR | stderr | INFO:     Waiting for application shutdown.
2024-02-26 22:27:31 | ERROR | stderr | INFO:     Application shutdown complete.
2024-02-26 22:27:31 | ERROR | stderr | INFO:     Finished server process [455216]