2024-02-26 21:45:17 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40001, worker_address='http://localhost:40001', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-1B-Chat', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False)
2024-02-26 21:45:17 | INFO | model_worker | Loading the model ['MobiLlama-1B-Chat'] on worker 1639f093 ...
2024-02-26 21:45:23 | INFO | model_worker | Register to controller
2024-02-26 21:45:23 | ERROR | stderr | INFO:     Started server process [455699]
2024-02-26 21:45:23 | ERROR | stderr | INFO:     Waiting for application startup.
2024-02-26 21:45:23 | ERROR | stderr | INFO:     Application startup complete.
2024-02-26 21:45:23 | ERROR | stderr | INFO:     Uvicorn running on http://0.0.0.0:40001 (Press CTRL+C to quit)
2024-02-26 21:46:01 | INFO | stdout | INFO:     127.0.0.1:60500 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:46:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: None. call_ct: 0. worker_id: 1639f093. 
2024-02-26 21:46:20 | INFO | stdout | INFO:     127.0.0.1:48788 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:46:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: None. call_ct: 0. worker_id: 1639f093. 
2024-02-26 21:47:06 | INFO | stdout | INFO:     127.0.0.1:55144 - "POST /worker_generate_stream HTTP/1.1" 200 OK
2024-02-26 21:47:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:48:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:49:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:49:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:50:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:51:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:52:01 | INFO | stdout | INFO:     127.0.0.1:59092 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:52:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:52:20 | INFO | stdout | INFO:     127.0.0.1:37148 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:52:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:53:23 | INFO | stdout | INFO:     127.0.0.1:58944 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:53:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:53:54 | INFO | stdout | INFO:     127.0.0.1:41798 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:54:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:54:56 | INFO | stdout | INFO:     127.0.0.1:53180 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:55:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:55:14 | INFO | stdout | INFO:     127.0.0.1:36798 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:55:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:56:16 | INFO | stdout | INFO:     127.0.0.1:34376 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:56:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:56:49 | INFO | stdout | INFO:     127.0.0.1:38786 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:57:19 | INFO | stdout | INFO:     127.0.0.1:50820 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:57:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:57:36 | INFO | stdout | INFO:     127.0.0.1:45772 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 21:58:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:58:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 21:59:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:00:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:00:37 | INFO | stdout | INFO:     127.0.0.1:43378 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:01 | INFO | stdout | INFO:     127.0.0.1:43798 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:01:24 | INFO | stdout | INFO:     127.0.0.1:54958 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:43 | INFO | stdout | INFO:     127.0.0.1:57684 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:01:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:02:37 | INFO | stdout | INFO:     127.0.0.1:34256 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:02:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:02:55 | INFO | stdout | INFO:     127.0.0.1:55556 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:03:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:03:35 | INFO | stdout | INFO:     127.0.0.1:50278 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:03:56 | INFO | stdout | INFO:     127.0.0.1:40924 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:04:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:04:24 | INFO | stdout | INFO:     127.0.0.1:52980 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:04:49 | INFO | stdout | INFO:     127.0.0.1:37202 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:04:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:05:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:06:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:07:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:07:46 | INFO | stdout | INFO:     127.0.0.1:59246 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:07:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:08:12 | INFO | stdout | INFO:     127.0.0.1:53410 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:08:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:09:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:10:01 | INFO | stdout | INFO:     127.0.0.1:44898 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:10:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:10:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:11:09 | INFO | stdout | INFO:     127.0.0.1:58684 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:11:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:12:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:13:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:13:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:14:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:14:41 | INFO | stdout | INFO:     127.0.0.1:41514 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:14:59 | INFO | stdout | INFO:     127.0.0.1:48850 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:15:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:16:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:16:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:17:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:17:57 | INFO | stdout | INFO:     127.0.0.1:55990 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:18:14 | INFO | stdout | INFO:     127.0.0.1:36960 - "POST /worker_get_status HTTP/1.1" 200 OK
2024-02-26 22:18:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:19:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:19:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:20:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:21:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:22:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:22:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:23:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:24:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:25:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:25:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:26:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:27:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. 
2024-02-26 22:27:37 | ERROR | stderr | INFO:     Shutting down
2024-02-26 22:27:38 | ERROR | stderr | INFO:     Waiting for application shutdown.
2024-02-26 22:27:38 | ERROR | stderr | INFO:     Application shutdown complete.
2024-02-26 22:27:38 | ERROR | stderr | INFO:     Finished server process [455699]