2024-02-26 22:10:44 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40004, worker_address='http://localhost:40004', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-08B', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False) 2024-02-26 22:10:44 | INFO | model_worker | Loading the model ['MobiLlama-08B'] on worker 7c759f2b ... 2024-02-26 22:10:45 | ERROR | stderr | Loading checkpoint shards: 0%| | 0/2 [00:00: Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:20 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:25 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:30 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2024-02-26 22:28:32 | ERROR | stderr | INFO: Shutting down 2024-02-26 22:28:32 | ERROR | stderr | INFO: Waiting for application shutdown. 2024-02-26 22:28:32 | ERROR | stderr | INFO: Application shutdown complete. 2024-02-26 22:28:32 | ERROR | stderr | INFO: Finished server process [459676]