How to inspect each grad_fn?

XINGYU_ZHU: <h1><a name="background-1" class="anchor" href="#background-1"></a>Background</h1>
<p>Note that this is not a CUDA-related question!</p>
<p>Hi, I want to implement the following function on CUDA (which is already achieved).</p>
<pre><code class="lang-auto">"""
This is a function of calculating vertex normals where
      xyz: (batch_size, number of vertices, 3)
      faces: (batch_size, number of faces, 3)
"""
def computeVN(xyz, faces):
    vertex_normals = torch.zeros_like(xyz)
    vertex_per_face = grouping_operation(
        xyz.permute(0, 2, 1).contiguous(),
        faces
    ).permute(0, 2, 3, 1)
    v2 = vertex_per_face[:, :, 2, :]
    v1 = vertex_per_face[:, :, 1, :]
    v0 = vertex_per_face[:, :, 0, :]
    vec1 = v2 - v1 # (B, M, 3)
    vec2 = v0 - v1
    face_normals = torch.cross(vec1, vec2, -1)
    for bidx, batch in enumerate(face_normals):
        for fidx, face_normal in enumerate(batch):
            v0, v1, v2 = faces[bidx, fidx]
            vertex_normals[bidx, v0] += face_normal
            vertex_normals[bidx, v1] += face_normal
            vertex_normals[bidx, v2] += face_normal
    from torch.nn import functional as F
    vertex_normals = F.normalize(vertex_normals, dim=-1)
    return vertex_normals
</code></pre>
<p>I use the following wrapper so that it can be applied in gradient descent:</p>
<pre><code class="lang-auto">class ComputeVertexNormals(Function):
    @staticmethod
    def forward(ctx, xyz, faces):
        """
            xyz: (B, N, 3)
            faces: (B, M, 3)
        """
        ctx.for_backwards = (xyz, faces)
        return _ext.compute_vertex_normals(xyz, faces)
    @staticmethod
    def backward(ctx, grad_out):
        """
            grad_out: (B, N, 3)
        """
        xyz, faces = ctx.for_backwards
        return _ext.compute_vertex_normals_grad(grad_out, xyz, faces), None

compute_vertex_normals = ComputeVertexNormals.apply
</code></pre>
<p>However, my <code>compute_vertex_normals_grad</code> is not working as expected. Now I want to debug it by inspecting each gradient.</p>
<h1><a name="question-2" class="anchor" href="#question-2"></a>Question</h1>
<ul>
<li>Is it possible for me check the detailed implementation of each <code>grad_fn</code>? For example, the grad_fn for <code>face_normals</code> is a <code>LinalgCrossBackward0 object</code>. But I dont know what is inside this function.</li>
</ul>
<p><strong>Why Do I Want to Inspect its Implementation</strong>:</p>
<p>I have calculate the derivation by hands, and I got the following equaltion (Here we take v2 as an example):</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/a/ea77ef3ce902d2a9cb31a7b64d66fe543ce70dbd.png" alt="image" data-base62-sha1="xscGzlMXMBvArLsierYomurKnql" width="309" height="267"></p>
<p>But the result of this equation is totally different from the one by <code>loss.backward()</code></p>

ptrblck: <p>One approach would be to check <a href="https://github.com/pytorch/pytorch/blob/a76bc410df7d20eac8b2a85ea7902c3257173b9a/tools/autograd/derivatives.yaml#L474-L477"><code>derivatives.yaml</code></a> for the corresponding function name (I just used <code>linalg_cross</code>) and then to search for the defined backward implementations.</p>

XINGYU_ZHU: <p>Thx for your help, but I still cannot find its <code>grad_fn</code> implementation. Is there any documents which can help me understand the source code of Pytorch?</p>