Kronecker Product

username: vitchyr
cooked: <p>Is there a way to perform a Kronecker product between two matrices? I know there’s <a href="http://pytorch.org/docs/torch.html#torch.ger" rel="nofollow noopener">outer products for vectors</a>, but is there something like that for 2D Tensors?</p>

username: fmassa
cooked: <p>I’m afraid not. But you can probably replicate the result by replicating one of the tensors and reshaping the result.</p>

username: vitchyr
cooked: <p>Okay. I was going to use the “upsample” method to replicate this, but this will only work with square matrices. Is there a way to upsample with a different scale factor along the different dimensions?</p>

username: moskomule
cooked: <p>How about this? I dont think this is efficient way, though.</p>
<pre><code class="lang-auto">h, w = t1.size()
c = t2.size()[0]
t2.repeat(h, w, 1) * t1.unsqueeze(2).repeat(1, 1, c)
</code></pre>

username: vitchyr
cooked: <p>That doesn’t seem to work (I just tried <code>t1 = t2 = torch.randn(2, 2)</code>)</p>
<p>This should work for arbitrarily sized matrices, though I’m not sure how fast/slow it will be:</p>
<pre><code class="lang-auto">def kronecker_product(t1, t2):
    """
    Computes the Kronecker product between two tensors.
    See https://en.wikipedia.org/wiki/Kronecker_product
    """
    t1_height, t1_width = t1.size()
    t2_height, t2_width = t2.size()
    out_height = t1_height * t2_height
    out_width = t1_width * t2_width

    tiled_t2 = t2.repeat(t1_height, t1_width)
    expanded_t1 = (
        t1.unsqueeze(2)
          .unsqueeze(3)
          .repeat(1, t2_height, t2_width, 1)
          .view(out_height, out_width)
    )

    return expanded_t1 * tiled_t2
</code></pre>

username: moskomule
cooked: <p>No. I supposed <code>t2</code> is a vector, indeed.</p>

username: OCY
cooked: <p>Below seems to work.</p>
<pre><code class="lang-auto">def kronecker(matrix1, matrix2):
    return torch.ger(matrix1.view(-1), matrix2.view(-1)).reshape(*(matrix1.size() + matrix2.size())).permute([0, 2, 1, 3]).reshape(matrix1.size(0) * matrix2.size(0), matrix1.size(1) * matrix2.size(1))

</code></pre>

username: purvanshi_mehta
cooked: <p>This works for 2d tensorsof size [B,D] with the first dimension as the batch size[B] -<br>
size_t1 is the D</p>
<pre><code class="lang-auto">def kronecker_product(t1, t2,size_t1,size_t2):
    fusion_tensor = torch.bmm(t1.unsqueeze(2), t2.unsqueeze(1))
    fusion_tensor = fusion_tensor.view(-1, size_t1 * size_t2)
    return fusion_tensor

</code></pre>

username: SaiSurya
cooked: <p>The following github repo has implemented the Kronecker product of multiple tensors efficiently.<br>
<a href="https://github.com/cornellius-gp/gpytorch/blob/master/gpytorch/lazy/kronecker_product_lazy_tensor.py" rel="nofollow noopener">gpytorch/kronecker_product_lazy_tensor.py</a></p>

username: chunyuan_yuan
cooked: <aside class="quote no-group" data-username="OCY" data-post="7" data-topic="3919">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ocy/48/154_2.png" class="avatar"> OCY:</div>
<blockquote>
<p>def kronecker(matrix1, matrix2):</p>
</blockquote>
</aside>
<p>Another simple way:</p>
<pre><code class="lang-auto">def kronecker(A, B):
    return torch.einsum("ab,cd-&gt;acbd", A, B).view(A.size(0)*B.size(0),  A.size(1)*B.size(1))
</code></pre>

username: yulkang
cooked: <p>I made a broadcasting-based implementation that works with batch dimensions: <a href="https://gist.github.com/yulkang/4a597bcc5e9ccf8c7291f8ecb776382d" rel="nofollow noopener">https://gist.github.com/yulkang/4a597bcc5e9ccf8c7291f8ecb776382d</a></p>

username: panaali
cooked: <p>I guess this should be the top choice since it’s lazy, meaning it would calculate the elements just when you access them so less memory usage.</p>

username: I-Love-U
cooked: <p>This way is so interesting.</p>

username: Leockl
cooked: <p>Hi <a class="mention" href="/u/saisurya">@SaiSurya</a>, can you give an example on how to use KroneckerProductLazyTensor? Many thanks.</p>