Identifying the Conv Algo Selected by cuDNN v8 within PyTorch

Haoke_Xu: <p>Hello,</p>
<p>I’ve been working on developing a new convolution algorithm. During my benchmarks with cuDNN in PyTorch, I encountered an issue: I’m unable to determine which convolution algorithms were selected by cuDNN.</p>
<p>I reviewed the <a href="https://github.com/pytorch/pytorch/blob/8d08b103be936d78d5d4ed90c0547aeccb8ce166/aten/src/ATen/native/cudnn/Conv_v8.cpp" rel="noopener nofollow ugc">Conv_v8.cpp</a> file but couldn’t find any relevant clues. Even after extracting the logs from cuDNN, I couldn’t locate any related information within the descriptors. The conv descriptor is as below:</p>
<p>i!     convDesc: type=cudnnConvolutionDescriptor_t:<br>
i!         mode: type=cudnnConvolutionMode_t; val=CUDNN_CROSS_CORRELATION (1);<br>
i!         dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);<br>
i!         mathType: type=cudnnMathType_t; val=CUDNN_DEFAULT_MATH (0);<br>
i!         reorderType: type=int; val=0;<br>
i!         arrayLength: type=int; val=2;<br>
i!         padA: type=int; val=[1,1];<br>
i!         strideA: type=int; val=[1,1];<br>
i!         dilationA: type=int; val=[1,1];<br>
i!         groupCount: type=int; val=1;</p>
<p>Would anyone be able to assist me in determining how to identify the convolution algorithm selected by cuDNN within PyTorch?</p>
<p>Thank you,</p>

J_Johnson: <p>cuDNN is developed and maintained by NVidia. See here for documentation:</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://developer.nvidia.com/cudnn">
  <header class="source">
      <img src="https://dirms4qsy6412.cloudfront.net/nv/assets/favicon-81bff16cada05fcff11e5711f7e6212bdc2e0a32ee57cd640a8cf66c87a6cbe6.ico" class="site-icon" width="48" height="48">

      <a href="https://developer.nvidia.com/cudnn" target="_blank" rel="noopener nofollow ugc">NVIDIA Developer</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/362;"><img src="https://developer.download.nvidia.com/images/og-default.jpg" class="thumbnail" width="690" height="362"></div>

<h3><a href="https://developer.nvidia.com/cudnn" target="_blank" rel="noopener nofollow ugc">CUDA Deep Neural Network</a></h3>

  <p>cuDNN provides researchers and developers with high-performance GPU acceleration.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Haoke_Xu: <p>cuDNN backend API(which was used by PyTorch) was not well documented. I also posted a ticket in cuDNN forum and <a href="https://forums.developer.nvidia.com/t/specify-conv-algorithm-in-backend-api/270593" rel="noopener nofollow ugc">they asked me to get help in PyTorch forum</a>. Just hope someone who knows cuDNN backend API could give a hint about it.</p>

ptrblck: <p>The <a href="https://github.com/pytorch/pytorch/blob/8d08b103be936d78d5d4ed90c0547aeccb8ce166/aten/src/ATen/native/cudnn/Conv_v8.cpp#L108-L111"><code>ExecutionPlanBuilder</code></a> contains the engine config responsible for the algorithm selection.</p>

Haoke_Xu: <p>Thank you for your response!</p>
<p>I discovered a relatively straightforward solution. By making modifications to <a href="https://github.com/pytorch/pytorch/blob/8d08b103be936d78d5d4ed90c0547aeccb8ce166/aten/src/ATen/native/cudnn/Conv_v7.cpp" rel="noopener nofollow ugc">Conv_v7.cpp</a>, I can enable PyTorch to use cuDNN in the v7 manner. This approach simplifies the process of checking and controlling the convolution algorithm, especially since NVIDIA provides more comprehensive documentation for their frontend API.</p>
<p>Thanks again for your assistance!</p>