File size: 11,554 Bytes
5472531 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
2024-02-26 22:09:53 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40003, worker_address='http://localhost:40003', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-05B', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False)
2024-02-26 22:09:53 | INFO | model_worker | Loading the model ['MobiLlama-05B'] on worker ddd61ea9 ...
2024-02-26 22:09:55 | ERROR | stderr |
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
2024-02-26 22:10:01 | ERROR | stderr |
Loading checkpoint shards: 50%|█████████████████████████████████████▌ | 1/2 [00:06<00:06, 6.51s/it]
2024-02-26 22:10:02 | ERROR | stderr |
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00, 2.89s/it]
2024-02-26 22:10:02 | ERROR | stderr |
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00, 3.43s/it]
2024-02-26 22:10:02 | ERROR | stderr |
2024-02-26 22:10:03 | INFO | model_worker | Register to controller
2024-02-26 22:10:03 | ERROR | stderr | [32mINFO[0m: Started server process [[36m459212[0m]
2024-02-26 22:10:03 | ERROR | stderr | [32mINFO[0m: Waiting for application startup.
2024-02-26 22:10:03 | ERROR | stderr | [32mINFO[0m: Application startup complete.
2024-02-26 22:10:03 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://0.0.0.0:40003[0m (Press CTRL+C to quit)
2024-02-26 22:10:48 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: None. call_ct: 0. worker_id: ddd61ea9.
2024-02-26 22:11:09 | INFO | stdout | [32mINFO[0m: 127.0.0.1:54416 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:11:31 | INFO | stdout | [32mINFO[0m: 127.0.0.1:51800 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:11:33 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: ddd61ea9.
2024-02-26 22:11:51 | INFO | stdout | [32mINFO[0m: 127.0.0.1:50382 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:12:15 | INFO | stdout | [32mINFO[0m: 127.0.0.1:56256 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:12:18 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 3. worker_id: ddd61ea9.
2024-02-26 22:13:03 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 3. worker_id: ddd61ea9.
2024-02-26 22:13:48 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 3. worker_id: ddd61ea9.
2024-02-26 22:14:33 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 3. worker_id: ddd61ea9.
2024-02-26 22:14:41 | INFO | stdout | [32mINFO[0m: 127.0.0.1:50640 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:14:59 | INFO | stdout | [32mINFO[0m: 127.0.0.1:42930 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:15:10 | INFO | stdout | [32mINFO[0m: 127.0.0.1:43100 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:15:18 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 4. worker_id: ddd61ea9.
2024-02-26 22:16:03 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 4. worker_id: ddd61ea9.
2024-02-26 22:16:48 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 4. worker_id: ddd61ea9.
2024-02-26 22:17:33 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 4. worker_id: ddd61ea9.
2024-02-26 22:17:57 | INFO | stdout | [32mINFO[0m: 127.0.0.1:42952 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:18:14 | INFO | stdout | [32mINFO[0m: 127.0.0.1:45540 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:18:18 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 4. worker_id: ddd61ea9.
2024-02-26 22:18:43 | INFO | stdout | [32mINFO[0m: 127.0.0.1:58682 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:18:56 | INFO | stdout | [32mINFO[0m: 127.0.0.1:35474 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:19:03 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 6. worker_id: ddd61ea9.
2024-02-26 22:19:48 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 6. worker_id: ddd61ea9.
2024-02-26 22:20:33 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 6. worker_id: ddd61ea9.
2024-02-26 22:21:18 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 6. worker_id: ddd61ea9.
2024-02-26 22:21:36 | INFO | stdout | [32mINFO[0m: 127.0.0.1:44274 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:21:45 | INFO | stdout | [32mINFO[0m: 127.0.0.1:60316 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m
2024-02-26 22:22:03 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:22:49 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:23:34 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:24:19 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:25:04 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:25:49 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:26:34 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:27:19 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:28:04 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-05B']. Semaphore: Semaphore(value=5, locked=False). call_ct: 8. worker_id: ddd61ea9.
2024-02-26 22:28:04 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1845a75030>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:09 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1845806f20>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:14 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1844051db0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:19 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f184581c1c0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:24 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1845a75840>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:29 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1845a74a30>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:34 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f18440512d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:39 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1844052920>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:44 | ERROR | model_worker | heart beat error: HTTPConnectionPool(host='localhost', port=10000): Max retries exceeded with url: /receive_heart_beat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f18440531f0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-02-26 22:28:46 | ERROR | stderr | [32mINFO[0m: Shutting down
2024-02-26 22:28:46 | ERROR | stderr | [32mINFO[0m: Waiting for application shutdown.
2024-02-26 22:28:46 | ERROR | stderr | [32mINFO[0m: Application shutdown complete.
2024-02-26 22:28:46 | ERROR | stderr | [32mINFO[0m: Finished server process [[36m459212[0m]
|