File size: 1,713 Bytes
5472531 |
1 2 3 4 5 6 7 8 9 10 11 12 |
2024-02-26 21:44:44 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40001, worker_address='http://localhost:40001', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-05B-Chat', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False)
2024-02-26 21:44:44 | INFO | model_worker | Loading the model ['MobiLlama-05B-Chat'] on worker 4d3d60b1 ...
2024-02-26 21:44:46 | INFO | model_worker | Register to controller
2024-02-26 21:44:46 | ERROR | stderr | [32mINFO[0m: Started server process [[36m455481[0m]
2024-02-26 21:44:46 | ERROR | stderr | [32mINFO[0m: Waiting for application startup.
2024-02-26 21:44:46 | ERROR | stderr | [32mINFO[0m: Application startup complete.
2024-02-26 21:44:46 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://0.0.0.0:40001[0m (Press CTRL+C to quit)
2024-02-26 21:45:04 | ERROR | stderr | [32mINFO[0m: Shutting down
2024-02-26 21:45:05 | ERROR | stderr | [32mINFO[0m: Waiting for application shutdown.
2024-02-26 21:45:05 | ERROR | stderr | [32mINFO[0m: Application shutdown complete.
2024-02-26 21:45:05 | ERROR | stderr | [32mINFO[0m: Finished server process [[36m455481[0m]
|