Segment Reduce memory problems while building?

NateTalley: <p>I’m trying to build PyTorch from source (cloned <code>7a79de1</code>) on a fresh Windows 11 system (Ryzen 7 5800XT, 32GB DDR4-2666 RAM) with an <strong>NVIDIA RTX 5060 Ti 16GB</strong> and <strong>CUDA 12.9</strong>. Building from source is a critical requirement as existing PyTorch builds don’t yet support this GPU.</p>
<p>My build consistently fails with an <strong>“LLVM ERROR: out of memory”</strong> and <code>nvcc</code> error (<code>'""%CICC_PATH%\cicc"' died with status 0xC0000409</code>) when compiling <code>aten\src\ATen\native\cuda\SegmentReduce.cu</code>. The log shows this is specifically during the object file generation for this <code>.cu</code> file.</p>
<p>I’ve already taken several troubleshooting steps:</p>
<ul>
<li><strong>Increased Windows Virtual Memory (Page File):</strong> Set to Initial: 49152 MB (48GB) &amp; Maximum: 65536 MB (64GB). PC was restarted.</li>
<li><strong>Limited Build Jobs:</strong> Ran <code>set MAX_JOBS=1</code> before <code>python setup.py install</code>.</li>
<li><strong>Clean Build:</strong> Deleted the <code>build</code> directory before each attempt.</li>
<li><strong>Environment:</strong> Using Visual Studio 2022 Community (MSVC 19.44.35208.0) Developer Command Prompt, Python 3.10.6 in a virtual environment (<code>sd_env</code>), and all Python build dependencies installed.</li>
<li><strong>System Checks:</strong> Windows Memory Diagnostic reported no errors. Event Viewer showed no relevant errors during previous failures.</li>
</ul>
<p>The build log (partial paste below) clearly shows the failure at <code>SegmentReduce.cu</code>. It seems the compiler (or related tools) is hitting a memory limit even with substantial virtual memory and single-threaded compilation.</p>
<p><code>[3400/7521] Building CUDA object caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj [cite: 1983] FAILED: caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/cuda/SegmentReduce.cu.obj [cite: 1983] C:\PROGRA~1\NVIDIA~2\CUDA\v12.9\bin\nvcc.exe -forward-unknown-to-host-compiler ... -x cu -c C:\StableDiffusion\pytorch\aten\src\ATen\native\cuda\SegmentReduce.cu -o caffe2\CMakeFiles\torch_cuda.dir\__\aten\src\ATen\native\cuda\SegmentReduce.cu.obj -Xcompiler=-Fdcaffe2\CMakeFiles\torch_cuda.dir\,-FS [cite: 1983, 1987] LLVM ERROR: out of memory [cite: 1988] SegmentReduce.cu [cite: 1988] nvcc error   : '""%CICC_PATH%\cicc"' died with status 0xC0000409 [cite: 1988] ninja: build stopped: subcommand failed. [cite: 1988]</code></p>
<p>Could this be a compatibility issue between CUDA 12.9 and the RTX 5060 Ti’s specific architecture during this particular compilation step? Or is there another compiler flag/environmental variable that might help with memory management during <code>nvcc</code>’s LLVM backend process?</p>

ptrblck: <aside class="quote no-group" data-username="NateTalley" data-post="1" data-topic="220488">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/natetalley/48/77228_2.png" class="avatar"> NateTalley:</div>
<blockquote>
<p>Building from source is a critical requirement as existing PyTorch builds don’t yet support this GPU.</p>
</blockquote>
</aside>
<p>Our nightly binaries support Blackwell architectures for a few months already as well as the latest stable PyTorch 2.7.0 release, so just install one of these binaries if you are running into issues building from source.</p>

atalman: <p>Hi <a class="mention" href="/u/natetalley">@NateTalley</a> any luck finding workaround for this issue ? Looks like we are hitting it in PyTorch CD now as well: <a href="https://github.com/pytorch/pytorch/issues/156181" class="inline-onebox" rel="noopener nofollow ugc">GitHub · Where software is built</a> . We are using CPU machines to build CUDA 12.9 PyTorch Wheels.</p>