2024-02-26 22:09:14 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40002, worker_address='http://localhost:40002', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-1B', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False) 2024-02-26 22:09:14 | INFO | model_worker | Loading the model ['MobiLlama-1B'] on worker 23461ac5 ... 2024-02-26 22:09:15 | ERROR | stderr | Loading checkpoint shards: 0%| | 0/2 [00:00: Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:14 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:19 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:24 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:29 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:34 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:39 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:44 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:49 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:52 | ERROR | stderr | INFO: Shutting down 2024-02-26 22:28:52 | ERROR | stderr | INFO: Waiting for application shutdown. 2024-02-26 22:28:52 | ERROR | stderr | INFO: Application shutdown complete. 2024-02-26 22:28:52 | ERROR | stderr | INFO: Finished server process [458643]