Cuda programming using python

Sumesh_Sankar: <p>When i search, cuda programming, all the programming techniques are in C language, i want in python and pytorch framework, where do i get, please help me.</p>

ptrblck: <p>Could you describe your use case in more detail please?<br>
In case you want to use a Python CUDA frontend, you might be able to hook it into PyTorch by writing a custom <code>autograd.Function</code> as described <a href="https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html">here</a>.</p>

Sumesh_Sankar: <p>Thank you for the reply!, I have a for loop that handle 18 lakh float parameters, i need to know how to handle these much parameters at a time so that i can avoid “out of memory” issue. the function given in the screen shot is been called in a loop and it handles a lot of parameters at a time. this is my issue.how to manage the memory?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a.png" data-download-href="https://discuss.pytorch.org/uploads/default/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a" title="Screenshot from 2022-11-22 16-58-59"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a_2_690x387.png" alt="Screenshot from 2022-11-22 16-58-59" data-base62-sha1="uMDln92AmXTwc9OaBCCqpwhZUVQ" width="690" height="387" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a_2_690x387.png, https://discuss.pytorch.org/uploads/default/original/3X/d/7/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/7/d7c09ef9b5b2f4c8a7921cba1e86d3fda55ce84a.png 2x" data-dominant-color="F1F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-11-22 16-58-59</span><span class="informations">903×507 48 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/b/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62.png" data-download-href="https://discuss.pytorch.org/uploads/default/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62" title="Screenshot from 2022-11-22 17-10-26"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62_2_690x388.png" alt="Screenshot from 2022-11-22 17-10-26" data-base62-sha1="3XoCBwR7bvUHzDxHZYYMXsr69mG" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/b/1bbd5fa11dc57d5bdc38e7a2614b7ed46217eb62.png 2x" data-dominant-color="F6F5F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-11-22 17-10-26</span><span class="informations">1347×759 117 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I’m unsure how the screenshot showing PyTorch code is related to the original question.<br>
In any case, you might need to decrease the batch size or the number of iterations to save memory.</p>

Sumesh_Sankar: <p>Ok… i can explain, look this training loop shown, i have moved my inputs and labels to GPU by .to(device) command, as the inputs and labels needed only to the point of loss calculation, i need to simply erase the memmory allocated in the GPU for inputs and labels. so that i can utilise the same memory or without loss of memory for the next batch<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/d/ad847b41c94394f6d59ffee6c21a077d8422b940.png" data-download-href="https://discuss.pytorch.org/uploads/default/ad847b41c94394f6d59ffee6c21a077d8422b940" title="Screenshot from 2022-11-23 10-13-16"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad847b41c94394f6d59ffee6c21a077d8422b940_2_690x388.png" alt="Screenshot from 2022-11-23 10-13-16" data-base62-sha1="oL0rrtYS3MQJuCiEZWxCdP3SIlq" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad847b41c94394f6d59ffee6c21a077d8422b940_2_690x388.png, https://discuss.pytorch.org/uploads/default/original/3X/a/d/ad847b41c94394f6d59ffee6c21a077d8422b940.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/d/ad847b41c94394f6d59ffee6c21a077d8422b940.png 2x" data-dominant-color="F0F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-11-23 10-13-16</span><span class="informations">937×528 71.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>