Why is Tensor not subclass of collections.abc.Sequence

username: adanielnoel
cooked: <p>When using type annotations in Python, what base type should we use for an argument that can either be a <code>list</code>, <code>tuple</code>, <code>torch.Tensor</code>, <code>numpy.ndarray</code>, etc.?</p>
<p>I would have expected the following to return true, since <code>torch.Tensor</code> implements both <code>__len__</code> and <code>__getitem__</code> methods:</p>
<pre><code class="lang-auto">from collections.abc import Sequence
import torch
a = torch.tensor((3, 4, 5.0))
isinstance(a, Sequence)
&gt;&gt;&gt; False
</code></pre>

username: Haydnspass
cooked: <p>Great question. I wonder if I can rely on it never going to be a sequence, cause I want to differentiate between a list of tensors (or rather  a sequence of tensors) or a tensor.</p>
<p>If I do</p>
<pre><code class="lang-python">if isinstance(t, Sequence):
   ...
elif isinstance (t, torch.Tensor):
   ...

</code></pre>
<p>this works now, but if tensors are one day going to be sequences, it’ll break.</p>

username: rehno
cooked: <p>For posterity, you can see why <code>Tensor</code> can’t be made into a <code>Sequence</code> by adding a type parameter.</p>
<p>For example, this function takes a <code>Sequence</code> and returns an <code>int</code>:</p>
<pre><code class="lang-python">def head(x: Sequence[int]) -&gt; int:
    return x[0]
</code></pre>
<p>While this function takes a <code>Tensor</code> and returns another <code>Tensor</code>:</p>
<pre><code class="lang-python">def head(x: torch.IntTensor) -&gt; torch.IntTensor:
    return x[0]
</code></pre>

username: Haydnspass
cooked: <p>I am not quite sure I understand the argument, could you elaborate on it?<br>
Take, for example, str where any str with len &gt; 1 is also a Sequence[str]</p>

username: rehno
cooked: <p>Yeah, dynamic language typing rules is obscuring the issue.</p>
<p>You are right to point out that a hypothetical</p>
<pre><code class="lang-auto">def head(x: Sequence[torch.IntTensor]) -&gt; torch.IntTensor
</code></pre>
<p>would be similar to</p>
<pre><code class="lang-auto">def head(x: Sequence[str]) -&gt; str
</code></pre>
<p>Perhaps a simpler illustration of the problem is this:</p>
<pre><code class="lang-python">&gt;&gt;&gt; len(torch.tensor([0]))
1
&gt;&gt;&gt; len(torch.tensor(0))
TypeError: len() of a 0-d tensor
&gt;&gt;&gt; [*torch.tensor([0])]
[tensor(0)]
&gt;&gt;&gt; [*torch.tensor(0)]
TypeError: iteration over a 0-d tensor
</code></pre>
<p>That is, I suppose an n-dim tensor could be a <code>Sequence</code> but a 0-dim tensor can’t.</p>
<p>This would be obvious with a type like <code>List[int]</code>, but unfortunately it’s not so obvious with pytorch’s <code>Tensor</code> type.</p>
<p>In an alternative world we’d have <code>Tensor[int]</code> for multi-dimensional tensors and <code>Prim[int]</code> for 0-dimensional primitives, and then <code>Tensor</code> would be able to support the <code>Sequence</code> interface.</p>
<p>At least, I’m assuming that if they inherit <code>collections.abc.Sequence</code> then they would want to make sure that it can be statically type checked as a <code>typing.Sequence</code> and not throw any unexpected run-time exceptions.</p>

username: rehno
cooked: <p>Actually, even assuming the problem above got a work-around, <a href="https://docs.python.org/3/library/stdtypes.html#common-sequence-operations" rel="noopener nofollow ugc">Sequence</a> also demands <code>__add__</code> and <code>__mul__</code> operators:</p>
<pre><code class="lang-python">&gt;&gt;&gt; def concat(x: Sequence, y: Sequence):
...     return x + y

&gt;&gt;&gt; concat([1,2,3], [4,5,6])
[1, 2, 3, 4, 5, 6]

&gt;&gt;&gt; concat(torch.tensor([1,2,3]), torch.tensor([4,5,6]))
tensor([5, 7, 9])
</code></pre>
<p>Generic code will be expecting the semantics of <code>__add__</code> to be monoidal addition rather than vector addition, so it would break things quite badly.</p>