Feedback on PEP 646 (relevant to tensor typing)

mrahtz: <p>Hi all,</p>
<p>A few of us have been working on the draft of PEP 646, enabling support for variadic generics in Python. This is relevant to PyTorch because the primary use case is to support array types that are generic in their shapes, e.g. <code>Tensor[Batch, Height, Width]</code>. (For a more detailed example of how this might work, see our experimental <a href="https://github.com/deepmind/tensor_annotations" rel="noopener nofollow ugc">tensor_annotations</a> library. We currently only support TensorFlow and JAX, but we’d love to support PyTorch in the future too.)</p>
<p>Before setting the PEP in stone, we’d like to get some feedback from folks working on libraries like PyTorch to reduce the chances of making design decisions that are going to bite us later on. If you’re interested, please leave some comments on the current draft in <a href="https://docs.google.com/document/d/1oXWyAtnv0-pbyJud8H5wkpIk8aajbkX-leJ8JXsE318" rel="noopener nofollow ugc">this doc</a>.</p>
<p>Thanks!<br>
Matthew</p>

ptrblck: <p>Thank you for posting the draft here and asking for feedback!<br>
Just by skimming through the code snippets in <code>tensor_annotations</code>, it looks similar to PyTorch’s named tensors, but this utility would be built into the Python standard now?</p>

mrahtz: <p>Right, thanks for asking. The problem we’re trying to solve is indeed very similar to what named tensors is trying to solve.</p>
<p>I say “similar”, because (as I understand it - I haven’t used named tensors in anger) the feature sets are slightly different:</p>
<ul>
<li>Pro named tensors: Named tensors goes a step further in allowing use of semantic axis labels within function calls too. With named tensors, we can do <code>sum('C')</code>. In the <code>tensor_annotations</code> approach, we still have to specify axes by index, e.g. <code>sum(dim=3)</code> (even if the resulting type, e.g. <code>Tensor[Height, Width]</code>, allows us to check that we really did sum over the <code>Channels</code> axis).</li>
<li>Pro the <code>tensor_annotations</code> approach: <code>tensor_annotations</code> allows expected shapes be checked statically, and also allows us to specify the expected shapes of e.g. arguments and returns (<code>def foo(x: Tensor[Height, Width]): ...</code>).</li>
</ul>
<p>Our original motivation was that, seeing how named tensors were never taken up properly by TensorFlow, we were worried that the named tensors approach wouldn’t take off in general because of the amount of work that library authors would have to do to support them. We wanted to see whether it was possible to make a solution solving roughly the same problem in a way that was library-agnostic, without changing existing APIs.</p>
<p>I’m still not personally sure whether the <code>tensor_annotations</code> approach is the right way to go (though of course I wouldn’t be championing the <code>tensor_annotations</code> approach if I didn’t feel <em>somewhat</em> more optimistic about it than named tensors). It’s also a bit of an experiment.</p>
<p>Our intention with PEP 646 is to add the single feature necessary to let us run the experiment and gather some data on how well it works. (Without PEP 646, we have to use different <code>Tensor</code> classes for each rank, e.g. <code>Tensor1[Logits]</code>, <code>Tensor2[Height, Width]</code>, etc, and that feels cumbersome enough that I suspect it’d put people off.) Even if <code>tensor_annotations</code> doesn’t work out, the <code>TypeVarTuple</code> of PEP 646 is also a useful language feature to have in general, allowing the signatures of e.g. <code>map</code> and <code>zip</code> to be specified properly (in combination with one extra typing operator, which we’ll likely be introducing in a future PEP).</p>
<p>I hope that gives some useful context. Do let me know if you have more questions.</p>

ptrblck: <aside class="quote no-group" data-username="mrahtz" data-post="3" data-topic="111791">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/8baadc/48.png" class="avatar"> mrahtz:</div>
<blockquote>
<p>Pro the <code>tensor_annotations</code> approach: <code>tensor_annotations</code> allows expected shapes be checked statically, and also allows us to specify the expected shapes of e.g. arguments and returns (<code>def foo(x: Tensor[Height, Width]): ...</code>).</p>
</blockquote>
</aside>
<p>This sounds really great and I think might be a really useful feature.</p>
<p>Also, thank you very much for the detailed explanation. I’ll dig a bit into the document and ping others who might be interested. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>