Mock torch device for unit testing

Daniel_Crawford: <p>I’m interested in mocking a torch device like torch.device(‘cuda’) but with a cpu instead. This way I can use configure my unit tests to check if the devices are being mapped properly in my program (i.e. catch errors such as “Expected all tensors to be on the same device, but found at least two devices”)</p>
<p>Ideally I would want to simply make a dummy device like this:</p>
<pre><code class="lang-auto"># create two devices
dv1 = torch.device('dummy1') 
dv2 = torch.device('dummy2')

# map to devices
a = torch.rand(1).to(dv1)
b = torch.rand(1).to(dv2)

# raises RuntimeError
a + b
</code></pre>
<p>Right now I’ve found the best way to do this is to use my own GPU to test for these situations but its a bit overkill and also causes issues if another program is using it or a GPU is not available. Any chance there is some functionality like this in PyTorch or some trick to get this to work?</p>
<p>Also would like to mention I have found a similar thread here, but the solution I found recommended using a library which literally simulates the GPU, but again thats a bit much for what I want to do. Somewhat related problem here: <a href="https://stackoverflow.com/questions/3087361/gpu-emulator-for-cuda-programming-without-the-hardware" class="inline-onebox" rel="noopener nofollow ugc">GPU Emulator for CUDA programming without the hardware - Stack Overflow</a></p>

BrianPugh: <p>I’m also interested in a solution for this. <a class="mention" href="/u/daniel_crawford">@Daniel_Crawford</a> have you found a solution?</p>

Daniel_Crawford: <p>Unfortunately did not find anything to test in a way similar to this. Haven’t looked in months though so maybe something has changed.</p>

Ilya_Kamenshchikov: <p>I’ve learned about “meta” device today, maybe it solves your goals. See <a href="https://github.com/pytorch/pytorch/issues/61654#issuecomment-879989145" class="inline-onebox" rel="noopener nofollow ugc">Allow creation of pseudo devices for testing purposes · Issue #61654 · pytorch/pytorch · GitHub</a></p>

Ilya_Kamenshchikov1: <p>Even better, MacOS users can have “mps” device (metal performance shaders). It seems to be intended for M1+ macs, but on my intel macbook pro with AMD gpu, it’s also available. Note <code>torch.has_mps</code> boolean and test with .to(device=“mps”).</p>

Ilya_Kamenshchikov1: <p>there’s a problem with “meta” when you try more diverse pytorch code. e.g.:</p>
<blockquote>
<pre><code>      return x.detach().to(device="cpu").numpy()
</code></pre>
</blockquote>
<p>E           NotImplementedError: Cannot copy out of meta tensor; no data!</p>

Aknw_Fen: <p>I’m unsure if this is what  you are searching for, but pytest provides mocking, one blunt way to achieve it is (the fn to test is my local <code>get_device</code>):</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
from torch_practice.utils.get_device import get_device

def mock_available() -&gt; bool:
  return True
def mock_unavailable() -&gt; bool:
  return False

def test_get_device(monkeypatch) -&gt; None:
  m = monkeypatch
  m.setattr(torch.cuda, "is_available", mock_available)
  m.setattr(torch.mps, "is_available", mock_unavailable)
  m.setattr(torch.cpu, "is_available", mock_unavailable)
  result = get_device()
  assert result == "cuda"

  m.setattr(torch.cuda, "is_available", mock_unavailable)
  m.setattr(torch.mps, "is_available", mock_available)
  result = get_device()
  assert result == "mps"

  m.setattr(torch.mps, "is_available", mock_unavailable)
  m.setattr(torch.cpu, "is_available", mock_available)
  result = get_device()
  assert result == "cpu"
</code></pre>
<p>Or split into several tests:</p>
<pre data-code-wrap="python"><code class="lang-python"># same imports
def test_cuda_device(monkeypatch) -&gt; None:
  m = monkeypatch
  m.setattr(torch.cuda, "is_available", mock_available)
  m.setattr(torch.mps, "is_available", mock_unavailable)
  m.setattr(torch.cpu, "is_available", mock_unavailable)
  result = get_device()
  assert result == "cuda"

def test_mps_device(monkeypatch) -&gt; None:
  m = monkeypatch
  m.setattr(torch.cuda, "is_available", mock_unavailable)
  m.setattr(torch.mps, "is_available", mock_available)
  m.setattr(torch.cpu, "is_available", mock_unavailable)
  result = get_device()
  assert result == "mps"

def test_cpu_device(monkeypatch) -&gt; None:
  m = monkeypatch
  m.setattr(torch.cuda, "is_available", mock_unavailable)
  m.setattr(torch.mps, "is_available", mock_unavailable)
  m.setattr(torch.cpu, "is_available", mock_available)
  result = get_device()
  assert result == "cpu"

</code></pre>