Error message: XPU out of memory. Tried to allocate 4.00 GiB (GPU 0; 15.11 GiB total capacity; 0 bytes already allocated; 0 bytes reserved in total by PyTorch)
Author: fablevi
Pull request: no
Labels: name: bug

### System Info

Soo the problem is simple:
If i only install xpu torch and intel_extension_for_pytorch then all my 16gbvram is ready but if i install transformers  its only 4gb vram is aviable.
With this the 16gb vram works:

The code:

      import torch
      from torch import xpu
      import os
      import intel_extension_for_pytorch
      
      os.environ['UR_L0_USE_RELAXED_ALLOCATION_LIMITS'] = '1'
      os.environ['IGC_ExtraOCLOptions'] = "-cl-intel-greater-than-4GB-buffer-required"
      
      
      def xpu_memory_test():
          if not xpu.is_available():
              print("XPU not available!")
              return
      
      device = torch.device("xpu")
      print(f"\n=== XPU Memory Test ===")
      
      try:
          print(f"Device Name: {torch.xpu.get_device_name(device)}")
      
          max_alloc = torch.xpu.max_memory_allocated(device) / (1024**3)
          total_mem = torch.xpu.get_device_properties(device).total_memory / (1024**3)
          
          print(f"\nDevice Memory: {total_mem:.2f}GB total")
          print(f"Max allocated during session: {max_alloc:.2f}GB")
      
          size_step = 0.1
          current_size = 1.0
          last_success = 0
          
          while current_size <= total_mem:
              tensor_size = int(current_size * (1024**3 / 4))
              print(f"\nAttempting to allocate {current_size}GB tensor...")
              
              try:
                  test_tensor = torch.empty(tensor_size, dtype=torch.float32, device=device)
                  torch.xpu.synchronize(device)
      
                  allocated = torch.xpu.memory_allocated(device) / (1024**3)
                  print(f"Success! Current allocated: {allocated:.2f}GB")
                  os.system("free -h")
                  
                  del test_tensor
                  torch.xpu.empty_cache()
                  last_success = current_size
                  current_size += size_step
                  
              except RuntimeError as e:
                  print(f"\nAllocation failed at {current_size}GB (last success: {last_success}GB)")
                  print(f"Error message: {str(e)}")
                  os.system("free -h")
                  break
                  
      except Exception as e:
          print(f"\nError during memory test: {str(e)}")
          
      finally:
          allocated = torch.xpu.memory_allocated(device) / (1024**3)
          print(f"\n! Final allocated memory: {allocated:.2f}GB")
          print("Test completed.")
      
      if __name__ == "__main__":
          xpu_memory_test()
          torch.xpu.empty_cache()

pip list (no transformers)

      Package                     Version
      --------------------------- ----------
      accelerate                  1.10.1
      certifi                     2025.8.3
      charset-normalizer          3.4.3
      dpcpp-cpp-rt                2025.0.4
      filelock                    3.13.1
      fsspec                      2024.6.1
      hf-xet                      1.1.10
      huggingface-hub             0.35.1
      idna                        3.10
      impi-devel                  2021.14.1
      impi-rt                     2021.14.1
      intel-cmplr-lib-rt          2025.0.4
      intel-cmplr-lib-ur          2025.0.4
      intel-cmplr-lic-rt          2025.0.4
      intel_extension_for_pytorch 2.7.10+xpu
      intel-opencl-rt             2025.0.4
      intel-openmp                2025.0.4
      intel-pti                   0.10.1
      intel-sycl-rt               2025.0.4
      Jinja2                      3.1.4
      MarkupSafe                  2.1.5
      mkl                         2025.0.1
      mkl-dpcpp                   2025.0.1
      mpmath                      1.3.0
      networkx                    3.3
      numpy                       2.1.2
      oneccl                      2021.14.1
      oneccl-bind-pt              2.7.0+xpu
      oneccl-devel                2021.14.1
      onemkl-sycl-blas            2025.0.1
      onemkl-sycl-datafitting     2025.0.1
      onemkl-sycl-dft             2025.0.1
      onemkl-sycl-lapack          2025.0.1
      onemkl-sycl-rng             2025.0.1
      onemkl-sycl-sparse          2025.0.1
      onemkl-sycl-stats           2025.0.1
      onemkl-sycl-vm              2025.0.1
      packaging                   25.0
      pillow                      11.0.0
      pip                         24.0
      psutil                      7.1.0
      pytorch-triton-xpu          3.3.0
      PyYAML                      6.0.2
      regex                       2025.9.18
      requests                    2.32.5
      ruamel.yaml                 0.18.15
      ruamel.yaml.clib            0.2.14
      safetensors                 0.6.2
      setuptools                  65.5.0
      sympy                       1.13.3
      tbb                         2022.2.0
      tcmlib                      1.2.0
      tokenizers                  0.22.1
      torch                       2.7.0+xpu
      torchaudio                  2.7.0+xpu
      torchvision                 0.22.0+xpu
      tqdm                        4.67.1
      typing_extensions           4.12.2
      umf                         0.9.1
      urllib3                     2.5.0



But if i install transformers its go back to 4gb vram:

pip list (with transformers)

      Package                     Version
      --------------------------- ----------
      accelerate                  1.10.1
      certifi                     2025.8.3
      charset-normalizer          3.4.3
      dpcpp-cpp-rt                2025.0.4
      filelock                    3.13.1
      fsspec                      2024.6.1
      hf-xet                      1.1.10
      huggingface-hub             0.35.1
      idna                        3.10
      impi-devel                  2021.14.1
      impi-rt                     2021.14.1
      intel-cmplr-lib-rt          2025.0.4
      intel-cmplr-lib-ur          2025.0.4
      intel-cmplr-lic-rt          2025.0.4
      intel_extension_for_pytorch 2.7.10+xpu
      intel-opencl-rt             2025.0.4
      intel-openmp                2025.0.4
      intel-pti                   0.10.1
      intel-sycl-rt               2025.0.4
      Jinja2                      3.1.4
      MarkupSafe                  2.1.5
      mkl                         2025.0.1
      mkl-dpcpp                   2025.0.1
      mpmath                      1.3.0
      networkx                    3.3
      numpy                       2.1.2
      oneccl                      2021.14.1
      oneccl-bind-pt              2.7.0+xpu
      oneccl-devel                2021.14.1
      onemkl-sycl-blas            2025.0.1
      onemkl-sycl-datafitting     2025.0.1
      onemkl-sycl-dft             2025.0.1
      onemkl-sycl-lapack          2025.0.1
      onemkl-sycl-rng             2025.0.1
      onemkl-sycl-sparse          2025.0.1
      onemkl-sycl-stats           2025.0.1
      onemkl-sycl-vm              2025.0.1
      packaging                   25.0
      pillow                      11.0.0
      pip                         24.0
      psutil                      7.1.0
      pytorch-triton-xpu          3.3.0
      PyYAML                      6.0.2
      regex                       2025.9.18
      requests                    2.32.5
      ruamel.yaml                 0.18.15
      ruamel.yaml.clib            0.2.14
      safetensors                 0.6.2
      setuptools                  65.5.0
      sympy                       1.13.3
      tbb                         2022.2.0
      tcmlib                      1.2.0
      tokenizers                  0.22.1
      torch                       2.7.0+xpu
      torchaudio                  2.7.0+xpu
      torchvision                 0.22.0+xpu
      tqdm                        4.67.1
      transformers                4.56.2
      typing_extensions           4.12.2
      umf                         0.9.1

Comments:
cc @IlyasMoutawwakil 

@yao-matrix @jiqing-feng 

Any update?

@yao-matrix @jiqing-feng 

I don't have access to an XPU so I can't reproduce. Hopefully intel devs can 🤗

@IlyasMoutawwakil 

Soo there is no update, where can i go with my problem? Where should i wrote? 

I tested out on an IGPU Intel Iris XE with windows. The error is the same.
Soo, on every platform there is this bug.

Would you please upgrade your torch xpu version to 2.8 or install the latest pre-released torch by `pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu `. Please uninstall intel_extension_for_pytorch as it will no longer be maintained in the future.

@jiqing-feng 

I done it
If intel extension is not installed(without transformers) and torch is 2.8.0+xpu, then there are still the same issue. 

Cannot allocate more than 4gb vram, even with UR_L0_USE_RELAXED_ALLOCATION_LIMITES = 1

I use torch2.9 by `pip install torch==2.9.0 pytorch-triton-xpu==3.5.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/test/xpu` and test your script on a Windows system. 

It shows the max memory allocated can reach to 16GB without ipex. 

<img width="1605" height="780" alt="Image" src="https://github.com/user-attachments/assets/5a800faf-04eb-41b9-b427-66f4d0852ea9" />

I just uninstalled ipex and remove `import intel_extension_for_pytorch` in your script. Could you please try with torch 2.9 as I mentioned in the beginning and uninstall ipex to try it again?

BTW, could you share your xpu type?

(Now i can test on my linux system, i will test out in windows to, there is only an Iris Xe)

I have an Intel Arc A770 16GB version

I done what you sad. 
I made a new venv and installed ```pip install torch==2.9.0 pytorch-triton-xpu==3.5.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/test/xpu```

The problem is the same: 

      pip list
      Package            Version
      ------------------ ----------
      dpcpp-cpp-rt       2025.2.1
      filelock           3.19.1
      fsspec             2025.9.0
      impi-rt            2021.16.1
      intel-cmplr-lib-rt 2025.2.1
      intel-cmplr-lib-ur 2025.2.1
      intel-cmplr-lic-rt 2025.2.1
      intel-opencl-rt    2025.2.1
      intel-openmp       2025.2.1
      intel-pti          0.13.1
      intel-sycl-rt      2025.2.1
      Jinja2             3.1.6
      MarkupSafe         2.1.5
      mkl                2025.2.0
      mpmath             1.3.0
      networkx           3.5
      numpy              2.3.3
      oneccl             2021.16.1
      oneccl-devel       2021.16.1
      onemkl-sycl-blas   2025.2.0
      onemkl-sycl-dft    2025.2.0
      onemkl-sycl-lapack 2025.2.0
      onemkl-sycl-rng    2025.2.0
      onemkl-sycl-sparse 2025.2.0
      pillow             11.3.0
      pip                24.0
      pytorch-triton-xpu 3.5.0
      setuptools         79.0.1
      sympy              1.14.0
      tbb                2022.2.0
      tcmlib             1.4.0
      torch              2.9.0+xpu
      torchvision        0.24.0+xpu
      typing_extensions  4.15.0
      umf                0.11.0

-

      Attempting to allocate 3.9000000000000026GB tensor...
      Success! Current allocated: 3.90GB
                     total        used        free      shared  buff/cache   available
      Mem:            31Gi       5.0Gi       8.9Gi       386Mi        18Gi        26Gi
      Swap:          8.0Gi          0B       8.0Gi
      
      Attempting to allocate 4.000000000000003GB tensor...
      
      Allocation failed at 4.000000000000003GB (last success: 3.9000000000000026GB)
      Error message: XPU out of memory. Tried to allocate 4.00 GiB. GPU 0 has a total capacity of 15.11 GiB. Of the allocated memory 0 bytes is allocated by PyTorch, and 0 bytes is reserved by PyTorch but unallocated. Please use `empty_cache` to release all unoccupied cached memory.
                     total        used        free      shared  buff/cache   available
      Mem:            31Gi       5.0Gi       8.9Gi       386Mi        18Gi        26Gi
      Swap:          8.0Gi          0B       8.0Gi
      
      ! Final allocated memory: 0.00GB
      Test completed.

And the code: 

      import torch
      from torch import xpu
      import os
      
      os.environ['PYTORCH_XPU_MAX_ALLOC_MB'] = '15000' 
      os.environ['PYTORCH_XPU_ALLOC_STRATEGY'] = 'dpc'
      os.environ['UR_L0_USE_RELAXED_ALLOCATION_LIMITS'] = '1'
      
      
      def xpu_memory_test():
          if not xpu.is_available():
              print("XPU not available!")
              return
      
      device = torch.device("xpu")
      print(f"\n=== XPU Memory Test ===")
      
      try:
          print(f"Device Name: {torch.xpu.get_device_name(device)}")
      
          max_alloc = torch.xpu.max_memory_allocated(device) / (1024**3)
          total_mem = torch.xpu.get_device_properties(device).total_memory / (1024**3)
          
          print(f"\nDevice Memory: {total_mem:.2f}GB total")
          print(f"Max allocated during session: {max_alloc:.2f}GB")
      
          size_step = 0.1
          current_size = 1.0
          last_success = 0
          
          while current_size <= total_mem:
              tensor_size = int(current_size * (1024**3 / 4))
              print(f"\nAttempting to allocate {current_size}GB tensor...")
              
              try:
                  test_tensor = torch.empty(tensor_size, dtype=torch.float32, device=device)
                  torch.xpu.synchronize(device)
      
                  allocated = torch.xpu.memory_allocated(device) / (1024**3)
                  print(f"Success! Current allocated: {allocated:.2f}GB")
                  os.system("free -h")
                  
                  del test_tensor
                  torch.xpu.empty_cache()
                  last_success = current_size
                  current_size += size_step
                  
              except RuntimeError as e:
                  print(f"\nAllocation failed at {current_size}GB (last success: {last_success}GB)")
                  print(f"Error message: {str(e)}")
                  os.system("free -h")
                  break
                  
      except Exception as e:
          print(f"\nError during memory test: {str(e)}")
          
      finally:
          allocated = torch.xpu.memory_allocated(device) / (1024**3)
          print(f"\n! Final allocated memory: {allocated:.2f}GB")
          print("Test completed.")
      
      if __name__ == "__main__":
          xpu_memory_test()
          torch.xpu.empty_cache()

@jiqing-feng 
Any update or idea?

Sorry, I don't have arc A770 device. Is there any way I can access your node? 

Probably not i am sorry

Soo guys its 2 weeks and i don't have any idea what should i do.

I don't close this issue, if somebody have any solution for this, but I will follow this thread in the future.

Thanks for the help, guys!

@fablevi , sorry that I missed this issue, I'll try to find one A770 internally and check this issue.

@fablevi ,  after I set `export UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1` in my shell env, then execute the python scripts you supplied, both cases(w/ or w/o transformers) are OK for me, as below.

<img width="686" height="302" alt="Image" src="https://github.com/user-attachments/assets/69aa43ff-1de5-486f-b515-054d043ab06c" />

But if I don't set it in shell env, both failed in `4GB` allocation, even if I can see you set it in python script.

Could you pls try my method? Thx very much.

@yao-matrix 

I don't want to close this thread rn, but somehow it works.

I added it to .bashrc and its working fine.
Thanks for help.

in your script change 
`os.environ['UR_L0_USE_RELAXED_ALLOCATION_LIMITS'] = '1'` 
to 
`os.environ['UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS'] = '1'`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.