Elegant way to get a symmetric Torch Tensor over diagonal

username: eduardo4jesus
cooked: <p><strong>Is there an elegant way to build a <code>Torch.Tensor</code> like this from a given set of values?</strong></p>
<p>Here is an 3x3 example, but in my application I would have a matrix of any odd-size.</p>
<p>A function call <code>gen_matrix([a, b, c, d, e, f])</code> should generate</p>
<p><a href="https://i.stack.imgur.com/ySOmZ.gif" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/ySOmZ.gif" alt="enter image description here" width="74" height="66"></a></p>
<hr>
<p>EDIT: As of now, I implemented the following solution. A more elegant way is desirable without a <code>for</code> loop. Using plain <code>torch</code> operations is desirable.</p>
<pre><code class="lang-auto">def weights_to_symmetric(weights, N):
  assert(weights.ndim == 3)
  tensor = torch.zeros((*weights.shape[:2], N, N))

  idx = 0
  for diag in range(N):
    size = N - diag
    w = weights[:, :, idx:idx+size]

    tensor += torch.diag_embed(w, offset=diag)
    if diag &gt; 0:
      tensor += torch.diag_embed(w, offset=-diag)

    idx += size

  return tensor
</code></pre>

username: Aditya_Ranganath
cooked: <p>I’m not sure about pytorch, but <a href="https://github.com/cornellius-gp/gpytorch" rel="noopener nofollow ugc">gpytorch</a> has something called toeplitz. Maybe that’s what you are looking for ? (look into sym_toeplitz)</p>

username: eduardo4jesus
cooked: <p>Thank you for the suggestion. <code>sym_toeplitz</code> is a resembles want, but it is not quite the same thing.</p>
<pre><code class="lang-auto">from gpytorch import utils
c = torch.tensor([1, 6, 4, 5], dtype=torch.float)
res = utils.toeplitz.sym_toeplitz(c)
res
# tensor([[1., 6., 4., 5.],
#        [6., 1., 6., 4.],
#        [4., 6., 1., 6.],
#        [5., 4., 6., 1.]])
</code></pre>

username: eduardo4jesus
cooked: <p>Here is a solution by <a href="https://stackoverflow.com/users/14527267/swag2198" rel="noopener nofollow ugc">swag2198</a> from <a href="https://stackoverflow.com/a/68029042/2313889" rel="noopener nofollow ugc">stackoverflow</a>.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; N = 5
&gt;&gt;&gt; vals = torch.arange(N*(N+1)/2) + 1
 
&gt;&gt;&gt; A = torch.zeros(N, N)
&gt;&gt;&gt; i, j = torch.triu_indices(N, N)
&gt;&gt;&gt; A[i, j] = vals
&gt;&gt;&gt; A.T[i, j] = vals
  
&gt;&gt;&gt; vals
 tensor([ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14.,
         15.])
&gt;&gt;&gt; A
 tensor([[ 1.,  2.,  3.,  4.,  5.],
         [ 2.,  6.,  7.,  8.,  9.],
         [ 3.,  7., 10., 11., 12.],
         [ 4.,  8., 11., 13., 14.],
         [ 5.,  9., 12., 14., 15.]])
</code></pre>

username: Foivos_Diakogiannis
cooked: <p>It is possible to also use <code>torch.nn.utils.parametrize.register_parametrization</code> for symmetrizing the matrix on the fly, however it comes to the expense of more parameters, but if this does not induce memory problem is more elegant and faster. Example:</p>
<pre><code class="lang-auto">import torch
class SymmetricTensor(torch.nn.Module):
    def __init__(self):
        super().__init__()        
        
    def forward(self, x):
        # standard symmetrization equation of tensors. 
        return 0.5*(x+x.transpose(1,0))
    
    
class ToyModel(torch.nn.Module):
    def __init__(self,N):
        super().__init__()
        self.symmetric_param = torch.nn.Parameter(torch.rand(N,N),requires_grad=True) 
        # This makes the Parameter symmetric_param symmetric  
        torch.nn.utils.parametrize.register_parametrization(self, "symmetric_param", SymmetricTensor())
        
        
    def forward(self,input):
        # assuming input.shape B x C x  H x W, summing over channels index   
        return torch.einsum('bchw,cj-&gt;bjhw',input,self.symmetric_param)  

net = ToyModel(4)
xx = torch.rand(2,4,24,24)

out = net(xx)
out.shape
# torch.Size([2, 4, 24, 24])

net.symmetric_param

# output: 
tensor([[0.0248, 0.2363, 0.6852, 0.5177],
        [0.2363, 0.3860, 0.4304, 0.7496],
        [0.6852, 0.4304, 0.6364, 0.3511],
        [0.5177, 0.7496, 0.3511, 0.6647]], grad_fn=&lt;MulBackward0&gt;)
</code></pre>