A series questions caused by torch.load and tensor.to(device) during grouped_gemm on multiple gpus

NiuMa-1234: <p>I build several tensor groups to test the grouped_gemm. the program went well on single gpu, but when I change the environment to multiple gpus and load the tensors to different device at different time, the program would error and have different errors each time:<br>
the 1st case:</p>
<blockquote>
<p>def randn(bs, x, y, device_id):<br>
out = (torch.rand(bs, x, y) - 0.5 * 2) / (y * x)<br>
device = torch.device(“cuda:%d”%device_id if torch.cuda.is_available() else “cpu”)<br>
return out.to(device).to(torch.bfloat16)<br>
a = randn(bs, x, y, device_id)<br>
b = randn(bs, x, y, device_id)</p>
</blockquote>
<p>yes, I load the tensor to the specific device when I build it, and the error would be:<br>
group_1: RuntimeError: Failed to run CUTLASS Grouped GEMM<br>
group_2…n:<br>
RuntimeError: CUDA error: an illegal memory access was encountered<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>
<p>the 2nd case:</p>
<blockquote>
<p>def randn(bs, x, y):<br>
out = (torch.rand(bs, x, y) - 0.5 * 2) / (y * x)<br>
return out.cuda().to(torch.bfloat16)<br>
a = a.to(torch.device(‘cuda:%d’%device_id))<br>
b = b.to(torch.device(‘cuda:%d’%device_id))</p>
</blockquote>
<p>I build the tensors  on default gpu and then load them to specific gpus, the program went well!</p>
<p>the 3rd case:</p>
<blockquote>
<pre><code>a = torch.load(a_path,map_location=torch.device('cuda:%d'%device_id))
b = torch.load(b_path,map_location=torch.device('cuda:%d'%device_id))
</code></pre>
</blockquote>
<p>when the device_id  is 0, it went well, but if not, it would error at following calculation at  the backend.gmm: RuntimeError: Failed to run CUTLASS Grouped GEMM</p>
<p>*and even I settle the device_id as 0 forever, the next tensor group would have the same error</p>
<p>why would this happen, so weird</p>

ptrblck: <p>Could you run your first code snippet via <code>cuda-gdb</code> and post the backtrace of the illegal memory access here?</p>

NiuMa-1234: <p>Thank you for your reply! The following is the output of the 1st test using cuda-gdb</p>
<blockquote>
<p>[Thread debugging using libthread_db enabled]<br>
Using host libthread_db library “/lib/x86_64-linux-gnu/libthread_db.so.1”.</p>
<p>[New Thread 0x7fff60ac4700 (LWP 1001)]<br>
[New Thread 0x7fff602c3700 (LWP 1002)]<br>
[New Thread 0x7fff5bac2700 (LWP 1003)]<br>
[New Thread 0x7fff55375700 (LWP 1004)]<br>
[Thread 0x7fff5bac2700 (LWP 1003) exited]<br>
[Thread 0x7fff602c3700 (LWP 1002) exited]<br>
[Thread 0x7fff60ac4700 (LWP 1001) exited]<br>
[Detaching after fork from child process 1005]<br>
[New Thread 0x7fff5bac2700 (LWP 1012)]<br>
[New Thread 0x7fff602c3700 (LWP 1013)]<br>
Tensor created successfully on GPU 0<br>
[New Thread 0x7fff60ac4700 (LWP 1015)]<br>
Tensor created successfully on GPU 1<br>
[New Thread 0x7fff48af1700 (LWP 1016)]<br>
Tensor created successfully on GPU 2<br>
[New Thread 0x7fff41dff700 (LWP 1017)]<br>
Tensor created successfully on GPU 3<br>
[New Thread 0x7fff4115d700 (LWP 1018)]<br>
Tensor created successfully on GPU 4<br>
[New Thread 0x7fff2dfff700 (LWP 1019)]<br>
Tensor created successfully on GPU 5<br>
[New Thread 0x7fff2d7fe700 (LWP 1020)]<br>
Tensor created successfully on GPU 6<br>
[New Thread 0x7fff2cb5c700 (LWP 1021)]<br>
Tensor created successfully on GPU 7<br>
GPU 3 memory summary:<br>
|===========================================================================|</p>
<div class="md-table">
<table>
<thead>
<tr>
<th>PyTorch CUDA memory summary, device ID 3</th>
</tr>
</thead>
<tbody>
<tr>
<td>CUDA OOMs: 0</td>
</tr>
<tr>
<td>===========================================================================</td>
</tr>
<tr>
<td>Metric</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Allocated memory</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Active memory</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Requested memory</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>GPU reserved memory</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Non-releasable memory</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Allocations</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Active allocs</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>GPU reserved segments</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Non-releasable allocs</td>
</tr>
<tr>
<td>from large pool</td>
</tr>
<tr>
<td>from small pool</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Oversize allocations</td>
</tr>
<tr>
<td>---------------------------------------------------------------------------</td>
</tr>
<tr>
<td>Oversize GPU segments</td>
</tr>
<tr>
<td>===========================================================================</td>
</tr>
</tbody>
</table>
</div><p>a.device:  cuda:3<br>
–Type  for more, q to quit, c to continue without paging–</p>
<p>CUDA Exception: Warp Illegal Address<br>
The exception was triggered at PC 0x7fff27455280</p>
<p>Thread 1 “python” received signal CUDA_EXCEPTION_14, Warp Illegal Address.<br>
[Switching focus to CUDA kernel 0, grid 2, block (0,0,0), thread (0,0,0), device 0, sm 0, warp 1, lane 0]<br>
0x00007fff27455310 in void cutlass::Kernel&lt;cutlass::gemm::kernel::GemmGrouped&lt;cutlass::gemm::threadblock::MmaMultistage&lt;cutlass::gemm::GemmShape&lt;128, 128, 32&gt;, cutlass::transform::threadblock::PredicatedTileAccessIterator&lt;cutlass::MatrixShape&lt;128, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajor, 1, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;32, 128&gt;, 128, cutlass::PitchLinearShape&lt;4, 8&gt;, 8&gt;, cutlass::Array&lt;cutlass::bfloat16_t, 8, false&gt;, false, cutlass::layout::NoPermute&gt;, cutlass::transform::threadblock::RegularTileAccessIterator&lt;cutlass::MatrixShape&lt;128, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswise&lt;16, 32&gt;, 0, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;32, 128&gt;, 128, cutlass::PitchLinearShape&lt;4, 8&gt;, 8&gt;, 16&gt;, (cutlass::arch::CacheOperation::Kind)1, cutlass::transform::threadblock::PredicatedTileAccessIterator&lt;cutlass::MatrixShape&lt;32, 128&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajor, 0, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;128, 32&gt;, 128, cutlass::PitchLinearShape&lt;8, 4&gt;, 8&gt;, cutlass::Array&lt;cutlass::bfloat16_t, 8, false&gt;, false, cutlass::layout::NoPermute&gt;, cutlass::transform::threadblock::RegularTileAccessIterator&lt;cutlass::MatrixShape&lt;32, 128&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, 0, cutlass::transform::PitchLinearWarpRakedT–Type  for more, q to quit, c to continue without paging–<br>
hreadMap&lt;cutlass::PitchLinearShape&lt;128, 32&gt;, 128, cutlass::PitchLinearShape&lt;8, 4&gt;, 8&gt;, 16&gt;, (cutlass::arch::CacheOperation::Kind)1, float, cutlass::layout::RowMajor, cutlass::gemm::threadblock::MmaPolicy&lt;cutlass::gemm::warp::MmaTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswise&lt;16, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, float, cutlass::layout::RowMajor, cutlass::gemm::warp::MmaTensorOpPolicy&lt;cutlass::arch::Mma&lt;cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, 32, cutlass::bfloat16_t, cutlass::layout::RowMajor, cutlass::bfloat16_t, cutlass::layout::ColumnMajor, float, cutlass::layout::RowMajor, cutlass::arch::OpMultiplyAdd&gt;, cutlass::MatrixShape&lt;1, 1&gt; &gt;, 1, false, bool&gt;, cutlass::MatrixShape&lt;0, 0&gt;, cutlass::MatrixShape&lt;0, 0&gt;, 1&gt;, 4, (cutlass::gemm::SharedMemoryClearOption)0, bool&gt;, cutlass::epilogue::threadblock::Epilogue&lt;cutlass::gemm::GemmShape&lt;128, 128, 32&gt;, cutlass::gemm::warp::MmaTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswise&lt;16, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, float, cutlass::layout::RowMajor, cutlass::gemm::warp::MmaTensorOpPolicy&lt;cutlass::arch::Mma&lt;cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, 32, cutlass::bfloat16_t, cutlass::layout::RowMajor, cutlass::bfloat16_t, cutlass::layout::ColumnMajor, float, cutlass::layout::RowMajor, cutlass::arch::OpMultiplyAdd&gt;, cutlass::MatrixShape&lt;1, 1&gt; &gt;, 1, false, bool&gt;, 1, cutlass::epilogue::threadblock::PredicatedTileIterator&lt;cutlass::epilogue::threadblock::OutputTileOptimalThreadMap&lt;cutlass::epilogue::threadblock::OutputTileShape&lt;128, 8, 2, 1, 1&gt;, cutlass::epilogue::threadblock::OutputTileShape&lt;1, 8, 1, 1, 8&gt;, 128, 8, 16&gt;, cutlass::bfloat16_t, false, cutlass::layout::NoPermute, false&gt;, cutlass::epilogue::warp::FragmentIteratorTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, float, cutlass::Array&lt;float, 4, true&gt;, cutlass::layout::RowMajor&gt;, cutlass::epilogue::warp::TileIteratorTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, float, cutlass::layout::RowMajor&gt;, cutlass::epilogue::threadblock::SharedLoadIterator&lt;cutlass::epil–Type  for more, q to quit, c to continue without paging–<br>
ogue::threadblock::OutputTileOptimalThreadMap&lt;cutlass::epilogue::threadblock::OutputTileShape&lt;128, 8, 2, 1, 1&gt;, cutlass::epilogue::threadblock::OutputTileShape&lt;1, 8, 1, 1, 8&gt;, 128, 8, 16&gt;::CompactedThreadMap, float, 32&gt;, cutlass::epilogue:<img src="https://discuss.pytorch.org/images/emoji/apple/thread.png?v=12" title=":thread:" class="emoji" alt=":thread:" loading="lazy" width="20" height="20">:LinearCombination&lt;cutlass::bfloat16_t, 8, float, float, (cutlass::epilogue:<img src="https://discuss.pytorch.org/images/emoji/apple/thread.png?v=12" title=":thread:" class="emoji" alt=":thread:" loading="lazy" width="20" height="20">:ScaleType::Kind)0, (cutlass::FloatRoundStyle)2, cutlass::bfloat16_t&gt;, cutlass::MatrixShape&lt;0, 8&gt;, 1, 1&gt;, cutlass::gemm::threadblock::GemmBatchedIdentityThreadblockSwizzle, (cutlass::gemm::kernel::GroupScheduleMode)0, false&gt; &gt;(cutlass::gemm::kernel::GemmGrouped&lt;cutlass::gemm::threadblock::MmaMultistage&lt;cutlass::gemm::GemmShape&lt;128, 128, 32&gt;, cutlass::transform::threadblock::PredicatedTileAccessIterator&lt;cutlass::MatrixShape&lt;128, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajor, 1, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;32, 128&gt;, 128, cutlass::PitchLinearShape&lt;4, 8&gt;, 8&gt;, cutlass::Array&lt;cutlass::bfloat16_t, 8, false&gt;, false, cutlass::layout::NoPermute&gt;, cutlass::transform::threadblock::RegularTileAccessIterator&lt;cutlass::MatrixShape&lt;128, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswise&lt;16, 32&gt;, 0, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;32, 128&gt;, 128, cutlass::PitchLinearShape&lt;4, 8&gt;, 8&gt;, 16&gt;, (cutlass::arch::CacheOperation::Kind)1, cutlass::transform::threadblock::PredicatedTileAccessIterator&lt;cutlass::MatrixShape&lt;32, 128&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajor, 0, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;128, 32&gt;, 128, cutlass::PitchLinearShape&lt;8, 4&gt;, 8&gt;, cutlass::Array&lt;cutlass::bfloat16_t, 8, false&gt;, false, cutlass::layout::NoPermute&gt;, cutlass::transform::threadblock::RegularTileAccessIterator&lt;cutlass::MatrixShape&lt;32, 128&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, 0, cutlass::transform::PitchLinearWarpRakedThreadMap&lt;cutlass::PitchLinearShape&lt;128, 32&gt;, 128, cutlass::PitchLinearShape&lt;8, 4&gt;, 8&gt;, 16&gt;, (cutlass::arch::CacheOperation::Kind)1, float, cutlass::layout::RowMajor, cutlass::gemm::threadblock::MmaPolicy&lt;cutlass::gemm::warp::MmaTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswis–Type  for more, q to quit, c to continue without paging–<br>
e&lt;16, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, float, cutlass::layout::RowMajor, cutlass::gemm::warp::MmaTensorOpPolicy&lt;cutlass::arch::Mma&lt;cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, 32, cutlass::bfloat16_t, cutlass::layout::RowMajor, cutlass::bfloat16_t, cutlass::layout::ColumnMajor, float, cutlass::layout::RowMajor, cutlass::arch::OpMultiplyAdd&gt;, cutlass::MatrixShape&lt;1, 1&gt; &gt;, 1, false, bool&gt;, cutlass::MatrixShape&lt;0, 0&gt;, cutlass::MatrixShape&lt;0, 0&gt;, 1&gt;, 4, (cutlass::gemm::SharedMemoryClearOption)0, bool&gt;, cutlass::epilogue::threadblock::Epilogue&lt;cutlass::gemm::GemmShape&lt;128, 128, 32&gt;, cutlass::gemm::warp::MmaTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCrosswise&lt;16, 32&gt;, cutlass::bfloat16_t, cutlass::layout::RowMajorTensorOpMultiplicandCongruous&lt;16, 64&gt;, float, cutlass::layout::RowMajor, cutlass::gemm::warp::MmaTensorOpPolicy&lt;cutlass::arch::Mma&lt;cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, 32, cutlass::bfloat16_t, cutlass::layout::RowMajor, cutlass::bfloat16_t, cutlass::layout::ColumnMajor, float, cutlass::layout::RowMajor, cutlass::arch::OpMultiplyAdd&gt;, cutlass::MatrixShape&lt;1, 1&gt; &gt;, 1, false, bool&gt;, 1, cutlass::epilogue::threadblock::PredicatedTileIterator&lt;cutlass::epilogue::threadblock::OutputTileOptimalThreadMap&lt;cutlass::epilogue::threadblock::OutputTileShape&lt;128, 8, 2, 1, 1&gt;, cutlass::epilogue::threadblock::OutputTileShape&lt;1, 8, 1, 1, 8&gt;, 128, 8, 16&gt;, cutlass::bfloat16_t, false, cutlass::layout::NoPermute, false&gt;, cutlass::epilogue::warp::FragmentIteratorTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, float, cutlass::Array&lt;float, 4, true&gt;, cutlass::layout::RowMajor&gt;, cutlass::epilogue::warp::TileIteratorTensorOp&lt;cutlass::gemm::GemmShape&lt;64, 64, 32&gt;, cutlass::gemm::GemmShape&lt;16, 8, 16&gt;, float, cutlass::layout::RowMajor&gt;, cutlass::epilogue::threadblock::SharedLoadIterator&lt;cutlass::epilogue::threadblock::OutputTileOptimalThreadMap&lt;cutlass::epilogue::threadblock::OutputTileShape&lt;128, 8, 2, 1, 1&gt;, cutlass::epilogue::threadblock::OutputTileShape&lt;1, 8, 1, 1, 8&gt;, 128, 8, 16&gt;::CompactedThreadMap, float, 32&gt;, cutlass::epilogue:<img src="https://discuss.pytorch.org/images/emoji/apple/thread.png?v=12" title=":thread:" class="emoji" alt=":thread:" loading="lazy" width="20" height="20">:LinearCombination&lt;cutlass::bfloat16_t, 8, float, float, (cutlass::epilogue:<img src="https://discuss.pytorch.org/images/emoji/apple/thread.png?v=12" title=":thread:" class="emoji" alt=":thread:" loading="lazy" width="20" height="20">:ScaleType::Kind)0–Type  for more, q to quit, c to continue without paging–<br>
, (cutlass::FloatRoundStyle)2, cutlass::bfloat16_t&gt;, cutlass::MatrixShape&lt;0, 8&gt;, 1, 1&gt;, cutlass::gemm::threadblock::GemmBatchedIdentityThreadblockSwizzle, (cutlass::gemm::kernel::GroupScheduleMode)0, false&gt;::Params)&lt;&lt;&lt;(1,1,1),(128,1,1)&gt;&gt;&gt; ()</p>
</blockquote>

ptrblck: <p>Thank you! Would you mind creating an issue on <a href="https://github.com/pytorch/pytorch/issues">GitHub</a> ideally with a minimal code snippet to reproduce the issue?<br>
I don’t know if we are missing to set the current device inside these CUTLASS kernels properly or what might be causing the memory violation.</p>

NiuMa-1234: <p>Thank you! I put the  code snippet in the issue:<a href="https://github.com/pytorch/pytorch/issues/134441" class="inline-onebox" rel="noopener nofollow ugc">Cuda memory error when testing GroupedGemm on a multi-Gpu environment · Issue #134441 · pytorch/pytorch · GitHub</a>. And all the three  snippets and their original test (in a single gpu environment) are placed at <a href="https://github.com/NiuMa-1234/Test_of_GroupedGemm_on_multiGpus.git" class="inline-onebox" rel="noopener nofollow ugc">GitHub - NiuMa-1234/Test_of_GroupedGemm_on_multiGpus: This is a test to GroupedGemm on multi-gpu environment which has encountered Cuda error</a></p>

sharanr: <p>Hello, Im using a Nvidia jetson orin nano and im facing compatibility issues with the cuda and pytorch. i have an ubuntu 22.04 , jetpack= 5.1 but when i check the drivers i get.</p>
<p>| NVIDIA-SMI 540.3.0                Driver Version: N/A          CUDA Version: 12.2     |<br>
|-----------------------------------------±---------------------±---------------------+<br>
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                      |               MIG M. |<br>
|=========================================+======================+======================|<br>
|   0  Orin (nvgpu)                  N/A  | N/A              N/A |                  N/A |<br>
| N/A   N/A  N/A               N/A /  N/A | Not Supported        |     N/A          N/A |<br>
|                                         |                      |                  N/A |<br>
±----------------------------------------±---------------------±---------------------+</p>
<p>±--------------------------------------------------------------------------------------+<br>
| Processes:                                                                            |<br>
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |<br>
|        ID   ID                                                             Usage      |<br>
|=======================================================================================|<br>
|  No running processes found                                                           |<br>
could this be the problem? How to solve this?</p>

ptrblck: <aside class="quote no-group" data-username="sharanr" data-post="6" data-topic="208392">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/97f17d/48.png" class="avatar"> sharanr:</div>
<blockquote>
<p>| 0 Orin (nvgpu) N/A | N/A N/A | N/A |<br>
| N/A N/A N/A N/A / N/A | Not Supported | N/A N/A |<br>
| | | N/A |</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="sharanr" data-post="6" data-topic="208392">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/97f17d/48.png" class="avatar"> sharanr:</div>
<blockquote>
<p>could this be the problem?</p>
</blockquote>
</aside>
<p>Yes, it seems your system is unable to detect the GPU so you might want to fix it first before trying to install and use PyTorch.</p>

sharanr: <p>Hello, thank you for your message. But how do I fix that?. I flashed it already like two times, do I have to flash it once again? If yes ,what details should I pay attention to. If there is some other way could you please let me know.? I’ve been struggling for weeks now.  Thanks in advance.</p>

sharanr: <p>Hello, is there a way to fix that?</p>