Initializing zero-element tensors is a no-op
#3
by
ndurkee
- opened
Not sure if this is something on my end or something with the model. Does this mean that the model didn't download properly for me?
RuntimeError: Cannot find any model weights with `/home/ndurkee/dolphin-2.5-mixtral-8x7b`
(RayWorkerVllm pid=2530) /usr/local/lib/python3.8/dist-packages/torch/nn/init.py:412: UserWarning: Initializing zero-element tensors is a no-op [repeated 3x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.)
(RayWorkerVllm pid=2530) warnings.warn("Initializing zero-element tensors is a no-op") [repeated 3x across cluster]
Not sure if this is something on my end or something with the model. Does this mean that the model didn't download properly for me?
RuntimeError: Cannot find any model weights with `/home/ndurkee/dolphin-2.5-mixtral-8x7b` (RayWorkerVllm pid=2530) /usr/local/lib/python3.8/dist-packages/torch/nn/init.py:412: UserWarning: Initializing zero-element tensors is a no-op [repeated 3x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.) (RayWorkerVllm pid=2530) warnings.warn("Initializing zero-element tensors is a no-op") [repeated 3x across cluster]
Same error here
So I tested the mixtral instruct and that does work on my VLLM. I also redownloaded all the files from here. The main difference I'm seeing is that these files are .bin and the mixtral files are .safetensors. I wonder if that impacted anything. I might open an issue on the vllm github.
ndurkee
changed discussion status to
closed