| 2024-07-03 02:40:41 | INFO | model_worker | args: Namespace(awq_ckpt=None, awq_groupsize=-1, awq_wbits=16, controller_address='http://127.0.0.1:21002', conv_template=None, cpu_offloading=False, debug=False, device='cuda', dtype=None, embed_in_truncate=False, enable_exllama=False, enable_xft=False, exllama_cache_8bit=False, exllama_gpu_split=None, exllama_max_seq_len=4096, gptq_act_order=False, gptq_ckpt=None, gptq_groupsize=-1, gptq_wbits=16, gpus=None, host='127.0.0.1', limit_worker_concurrency=5, load_8bit=False, max_gpu_memory=None, model_names=None, model_path='lmsys/vicuna-7b-v1.5', no_register=False, num_gpus=1, port=21003, revision='main', seed=None, ssl=False, stream_interval=2, worker_address='http://127.0.0.1:21003', xft_dtype=None, xft_max_seq_len=4096) | |
| 2024-07-03 02:40:41 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker b8347ea9 ... | |
| 2024-07-03 02:40:42 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/torch/storage.py:315: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. | |
| 2024-07-03 02:40:42 | ERROR | stderr | warnings.warn(message, UserWarning) | |
| 2024-07-03 02:40:42 | ERROR | stderr | Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] | |
| 2024-07-03 02:40:47 | ERROR | stderr | Loading checkpoint shards: 50%|█████████████████████████████████ | 1/2 [00:05<00:05, 5.21s/it] | |
| 2024-07-03 02:40:49 | ERROR | stderr | Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.45s/it] | |
| 2024-07-03 02:40:49 | ERROR | stderr | Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.72s/it] | |
| 2024-07-03 02:40:49 | ERROR | stderr | | |
| 2024-07-03 02:40:49 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-03 02:40:49 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 02:40:49 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-03 02:40:49 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 02:40:49 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. | |
| 2024-07-03 02:40:49 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 02:40:49 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. | |
| 2024-07-03 02:40:49 | ERROR | stderr | warnings.warn( | |
| 2024-07-03 02:40:52 | INFO | model_worker | Register to controller | |
| 2024-07-03 02:40:53 | ERROR | stderr | [32mINFO[0m: Started server process [[36m92401[0m] | |
| 2024-07-03 02:40:53 | ERROR | stderr | [32mINFO[0m: Waiting for application startup. | |
| 2024-07-03 02:40:53 | ERROR | stderr | [32mINFO[0m: Application startup complete. | |
| 2024-07-03 02:40:53 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://127.0.0.1:21003[0m (Press CTRL+C to quit) | |
| 2024-07-03 02:41:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:42:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:43:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:43:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:44:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:45:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:46:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:46:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:47:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:48:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:49:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:49:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:50:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:51:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:52:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:52:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:53:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |
| 2024-07-03 02:54:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: b8347ea9. | |