runtime error

Exit code: 1. Reason: 58, in run [rank0]: result = self.fn(*self.args, **self.kwargs) [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/_snapshot_download.py", line 306, in _inner_hf_hub_download [rank0]: return hf_hub_download( [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn [rank0]: return fn(*args, **kwargs) [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1010, in hf_hub_download [rank0]: return _hf_hub_download_to_cache_dir( [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1171, in _hf_hub_download_to_cache_dir [rank0]: _download_to_tmp_and_move( [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1723, in _download_to_tmp_and_move [rank0]: xet_get( [rank0]: File "/home/user/.local/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 629, in xet_get [rank0]: download_files( [rank0]: RuntimeError: Data processing error: CAS service error : Error : single flight error: Real call failed: ReqwestMiddlewareError(Reqwest(reqwest::Error { kind: Request, url: "https://s3.us-east-1.amazonaws.com/xethub-xorb-bucket/xorbs/default/4b37e2e25198cb7cf93df4cb1f3d1f265610dc72c01b085eb8546921f00b886a?x-id=GetObject&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2JU7TKAQDPQNWMNT%2F20250814%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250814T233436Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host%3Brange&X-Amz-Signature=b0870128cc34cf37d29d020e7545ddc4c6147da939aaecadabae86a5e9e50376", source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(IncompleteMessage)) })) [rank0]:[W814 23:35:20.334118778 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

Container logs: