2023-11-18 09:03:28 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40000, worker_address='http://localhost:40000', controller_address='http://localhost:10000', model_path='4bit/llava-v1.5-7b-5GB', model_base=None, model_name=None, device='cuda', multi_modal=False, limit_model_concurrency=5, stream_interval=1, no_register=False, load_8bit=True, load_4bit=False) 2023-11-18 09:03:28 | INFO | model_worker | Loading the model llava-v1.5-7b-5GB on worker 72f41b ... 2023-11-18 09:03:28 | ERROR | stderr | tokenizer_config.json: 0%| | 0.00/749 [00:00: Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:01 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:06 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:12 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:17 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:22 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:27 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:32 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:37 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:42 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:47 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:52 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:13:57 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:02 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:07 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:12 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:17 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:22 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:27 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:32 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:37 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:42 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:47 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:52 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:14:57 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:15:02 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:15:07 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) 2023-11-18 09:15:12 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused'))