RuntimeError: mat1 and mat2 must have the same dtype

Benjamin_Tan: <p>I was trying to use TORCH.NN.FUNCTIONAL.LINEAR on my model. However,  I got an error message saying that “mat1 and mat2 must have the same dtype”. It is just a linear function, I don’t get why the matrices have to be in the same dtype. Thank you for any reply, it will help me to gain a better understanding.</p>

ptrblck: <p>The internal calls expect to get data in the same <code>dtype</code>. I guess you might be manually casing the input to this linear layer to another <code>dtype</code> than its parameters, so cast it back or could you explain what’s your use case and why you expect the <code>dtype</code> mismatch to work?</p>

Benjamin_Tan: <p>Thanks for your reply. So what is the expected dtype requires? I only have one input X (a tensor of floats) to the linear function.</p>

Benjamin_Tan: <p>Could you tell me what is mat1 and mat2 is? I suppose it is x and transpose of A in the linear equation?</p>

ptrblck: <p><code>mat1</code> and <code>mat2</code> refer most likely to the input tensor and the weight matrix of the linear layer.<br>
Here is a small example showing one way to run into this error which is caused by the <code>dtype</code> mismatch between the input tensor and the layer’s parameters:</p>
<pre><code class="lang-python"># initialzie linear layer
linear = nn.Linear(10, 1, bias=False)

# by default float32 is used as the dtype
print(linear.weight.dtype)
# torch.float32

# create input tensor
x = torch.randn(10, 10)

# by default float32 is also used
print(x.dtype)
# torch.float32

# linear layer works and output dtype is also float32
out = linear(x)
print(out.dtype)
# torch.float32

# transform to float64
x = x.to(torch.float64)
print(x.dtype)
# torch.float64

# create dtype mismatch
out = linear(x)
# RuntimeError: expected scalar type Double but found Float

# same for an explicit matmul
out = torch.matmul(x, linear.weight.T)
# RuntimeError: expected scalar type Double but found Float
</code></pre>

Benjamin_Tan: <p>I think I got it now. Apprecite your help!</p>

ptrblck: <p>Also, in case you are trying to use mixed-precision training, use the util. functions from <code>torch.amp</code> as e.g. the <code>autocast</code> context will cast the tensors to the appropriate types for you.</p>

Amer_Yasser: <p>i also faced same problem<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb" title="prob"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/c/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb_2_690x281.jpeg" alt="prob" data-base62-sha1="fpKSMpCSXyBWeyfz3uAWynMU5Z1" width="690" height="281" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/c/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb_2_690x281.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c09f313e908241c09d0e6b91b4b0881eb6ec3bb.jpeg 2x" data-dominant-color="292828"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">prob</span><span class="informations">916×374 34 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
i dont know what could be the issue here<br>
it give me this error:<br>
RuntimeError: mat1 and mat2 must have the same dtype</p>

ptrblck: <p>I guess your input might be using <code>float64</code> while the model’s parameters use <code>float32</code>?<br>
Could you check and fix it if that’s the case?</p>

Amer_Yasser: <p>it was indeed the problem,Thanks</p>

Muhamood_KHALID: <p>am trying this but am geting an error<br>
RuntimeError: mat1 and mat2 must have the same dtype</p>
<p>Any posible solution<br>
class ClassNa(nn.Module):<br>
# create the methods<br>
def <strong>init</strong>(self, in_size, hidden1_size, num_class):</p>
<pre><code>    super(ClassNa, self).__init__()#
    self.fc1 = nn.Linear(in_size, hidden1_size)
</code></pre>
<h1>
<a name="selfrelu1-nnrelu-1" class="anchor" href="#selfrelu1-nnrelu-1"></a>self.relu1 = nn.ReLU()</h1>
<pre><code>    self.fc2 = nn.Linear(hidden1_size, num_class)

def forward(self, x):
    """this is for propagation"""
    out = F.relu(self.fc1(x))
</code></pre>
<h1>
<a name="out-selfrelu1out-2" class="anchor" href="#out-selfrelu1out-2"></a>out = self.relu1(out)</h1>
<pre><code>    out = self.fc2(out)
    return out     
</code></pre>
<p>model= ClassNa(3,6,2)<br>
model.to(device)<br>
x,y= next(iter(train_dataload))<br>
x= x[:4].to(device)<br>
score =model(x)<br>
print(score)<br>
RuntimeError: mat1 and mat2 must have the same dtype</p>

ptrblck: <p>Check the <code>dtype</code>s via <code>model.fc1.weight.dtype</code> and <code>x.dtype</code>, and make sure both are equal. If not, transform either the input to the model <code>dtype</code> or vice versa.</p>

Muhamood_KHALID: <p>Thanks<br>
I used<br>
Out= x.view(size (0),-1)<br>
In the in the forward function</p>

ptrblck: <p>The <code>view</code> operation does not change the <code>dtype</code> of the tensor and will thus also not solve the issue.<br>
Check the <code>.dtype</code> attribute and make sure the parameters as well as the input are using the same.</p>

opeyemiolasunkanmi: <p>Great man. you are very helpful. thanks</p>

Swetha_Kerahalli: <p>I have input to the model of same dtype, but still getting error “RuntimeError: mat1 and mat2 must have the same dtype”</p>
<p>self.fc = nn.Linear(n_inputs, n_hidden)<br>
torch.rand(x.shape).to(self.device)<br>
input = self.fc(x)</p>
<p>output:<br>
fc: Linear(in_features=784, out_features=100, bias=True)<br>
fc: Linear(in_features=100, out_features=10, bias=True)<br>
torch.Size([480, 8568])<br>
torch.float64<br>
torch.Size([480, 8568])<br>
torch.float64</p>
<p>Can anyone please suggest some solution.</p>

ptrblck: <p>Could you post a minimal and executable code snippet reproducing the error, please?</p>

Swetha_Kerahalli: <p>I am working on spiking neural network and when I tried checking the dtype of fc.weight, it seems there is mismatch in dtype.</p>
<p>self.fc = nn.Linear(n_inputs, n_hidden)<br>
print(self.fc)<br>
print(self.fc.weight.dtype)</p>
<p>output:<br>
fc: Linear(in_features=784, out_features=100, bias=True)<br>
torch.float32<br>
fc: Linear(in_features=100, out_features=10, bias=True)<br>
torch.float32</p>
<p>def forward(self,x):<br>
# 1. Weight matrix multiplies the input x<br>
x = x.to(torch.float32)<br>
print(x.shape)<br>
print(x.dtype)<br>
torch.rand(x.shape).to(self.device)<br>
print(x.shape)<br>
print(x.dtype)<br>
input_excitation = self.fc(x)</p>
<p>output:<br>
torch.Size([480, 8568])<br>
torch.float32<br>
torch.Size([480, 8568])<br>
torch.float32</p>
<p>fc.weight is float32 and my input is float64. So I have changed input to float32. Tried changing weights also to float64. In both cases I am getting same error.</p>
<p>I have changed it to x.to(torch.float32), but after doing this getting error as<br>
RuntimeError: mat1 and mat2 shapes cannot be multiplied (480x8568 and 784x100)</p>

ptrblck: <p>Your code is still not executable and not properly formatted.<br>
However, it seems you have solved the issue by transforming everything to <code>float32</code>.</p>
<aside class="quote no-group" data-username="Swetha_Kerahalli" data-post="18" data-topic="166759">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/swetha_kerahalli/48/62133_2.png" class="avatar"> Swetha_Kerahalli:</div>
<blockquote>
<p>I have changed it to x.to(torch.float32), but after doing this getting error as<br>
RuntimeError: mat1 and mat2 shapes cannot be multiplied (480x8568 and 784x100)</p>
</blockquote>
</aside>
<p>This shape mismatch is raised if the <code>in_features</code> of the linear layer do not match the feature dimension of the input activation, so use <code>in_features=8568</code> and it should work.</p>

Swetha_Kerahalli: <p>trainData = data[‘trainData’].transpose((3,0,1,2))<br>
print(trainData.shape[0]) <span class="hashtag-raw">#output:</span> 480</p>
<p>trainLabels = data[‘trainLabels’].transpose((3,0,1,2))<br>
print(trainLabels.shape[0]) <span class="hashtag-raw">#output:32</span></p>
<p>valData = data[‘valData’].transpose((3,0,1,2))<br>
print(valData.shape[0]) <span class="hashtag-raw">#output:</span> 480</p>
<p>valLabels = data[‘valLabels’].transpose((3,0,1,2))<br>
print(valLabels.shape[0]) <span class="hashtag-raw">#output:32</span></p>
<p>train_set_loader = DataLoader(trainData, batch_size=480, shuffle=True)<br>
test_set_loader = DataLoader(valData, batch_size=480, shuffle=True)</p>
<p>use_cuda = torch.cuda.is_available()<br>
device = torch.device(“cuda” if use_cuda else “cpu”)</p>
<p>def neuron_state(self):<br>
batch_size = x.shape[0]</p>
<p>prev_inner = torch.zeros([self.batch_size, self.n_hidden]).to(device)<br>
prev_outer = torch.zeros([self.batch_size, self.n_hidden]).to(device)</p>
<p>input_excitation = self.fully_connected(x)<br>
inner_excitation = input_excitation + self.prev_inner * self.decay_multiplier<br>
outer_excitation = F.relu(inner_excitation - self.threshold)</p>
<p>do_penalize_gate = (outer_excitation &gt; 0).float()<br>
inner_excitation = inner_excitation - (self.penalty_threshold + outer_excitation) * do_penalize_gate</p>
<p>delayed_return_state = self.prev_inner<br>
delayed_return_output = self.prev_outer<br>
self.prev_inner = inner_excitation<br>
self.prev_outer = outer_excitation<br>
return delayed_return_state, delayed_return_output</p>
<p>def train(model, device, train_set_loader, optimizer, epoch, logging_interval=100):<br>
model.train()<br>
for batch_idx, data in enumerate(train_set_loader):<br>
for batch_idx, target in enumerate(train_set_loader):<br>
data = data.to(device)<br>
target = target.to(device)<br>
optimizer.zero_grad()<br>
output = model(data)<br>
loss = F.nll_loss(output, target)<br>
loss.backward()<br>
optimizer.step()</p>
<pre><code>    if batch_idx % logging_interval == 0:
        pred = output.max(1, keepdim=True)[1]
        correct = pred.eq(target.view_as(pred)).float().mean().item()
        print('Train Epoch: {} [{}/{} ({:.0f}%)] Loss: {:.6f} Accuracy: {:.2f}%'.format(
            epoch, batch_idx * len(data), len(train_set_loader.dataset),
            100. * batch_idx / len(train_set_loader), loss.item(),
            100. * correct))
</code></pre>
<p>def train_many_epochs(model):<br>
epoch = 1000<br>
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)<br>
train(model, device, train_set_loader, optimizer, epoch, logging_interval=10)<br>
test(model, device, test_set_loader)</p>
<p>def test(model, device, test_set_loader):<br>
model.eval()<br>
test_loss = 0<br>
correct = 0</p>
<pre><code>with torch.no_grad():
    for data, target in test_set_loader:
        data, target = data.to(device), target.to(device)
        output = model(data)
        test_loss += F.nll_loss(output, target, reduce=True).item()
        pred = output.max(1, keepdim=True)[1]
        correct += pred.eq(target.view_as(pred)).sum().item()

test_loss /= len(test_set_loader.dataset)
print("")
print('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)'.format(
    test_loss,
    correct, len(test_set_loader.dataset),
    100. * correct / len(test_set_loader.dataset)))
print("")
</code></pre>
<p>class SpikingNeuronLayerRNN(nn.Module):</p>
<pre><code>def __init__(
    self, device, n_inputs=480, n_hidden=8568,
    decay_multiplier=0.9, threshold=2.0, penalty_threshold=2.5
):
    super(SpikingNeuronLayerRNN, self).__init__()
    self.device = device
    self.n_inputs = n_inputs
    self.n_hidden = n_hidden
    self.decay_multiplier = decay_multiplier
    self.threshold = threshold
    self.penalty_threshold = penalty_threshold
    self.fc = nn.Linear(n_inputs, n_hidden)
    print("fc:", self.fc)
    print(self.fc.weight.dtype)
    # self.fc = self.fc.to(torch.float64)
    # print((self.fc).dtype)

    self.init_parameters()
    self.reset_state()
    self.to(self.device)

def init_parameters(self):
    for param in self.parameters():
        if param.dim() &gt;= 2:
            nn.init.xavier_uniform_(param)

def reset_state(self):
    self.prev_inner = torch.zeros([self.n_hidden]).to(self.device)
    self.prev_outer = torch.zeros([self.n_hidden]).to(self.device)

def forward(self,x):
    if self.prev_inner.dim() == 1:
        batch_size = x.shape[0]
        self.prev_inner = torch.stack(batch_size * [self.prev_inner])
        self.prev_outer = torch.stack(batch_size * [self.prev_outer])

    x = x.to(torch.float32)
    print(x.shape)
    print(x.dtype)
    torch.rand(x.shape).to(self.device)
    # x = x.view(x.size(0), -1)
    # print(x)
    print(x.shape)
    print(x.dtype)
    input_excitation = self.fc(x)
    inner_excitation = input_excitation + self.prev_inner * self.decay_multiplier
    outer_excitation = F.relu(inner_excitation - self.threshold)
    delayed_return_state = self.prev_inner
    delayed_return_output = self.prev_outer
    self.prev_inner = inner_excitation
    self.prev_outer = outer_excitation
    return delayed_return_state, delayed_return_output
</code></pre>
<p>class InputDataToSpikingPerceptronLayer(nn.Module):</p>
<pre><code>def __init__(self, device):
    super(InputDataToSpikingPerceptronLayer, self).__init__()
    self.device = device

    self.reset_state()
    self.to(self.device)
	
def forward(self, x, is_2D=True):
    x = x.view(x.size(0), -1)
    random_activation_perceptron = torch.rand(x.shape).to(self.device)
    return random_activation_perceptron * x
</code></pre>
<p>class OutputDataToSpikingPerceptronLayer(nn.Module):</p>
<pre><code>def __init__(self, average_output=True):
    
    super(OutputDataToSpikingPerceptronLayer, self).__init__()
    if average_output:
        self.reducer = lambda x, dim: x.sum(dim=dim)
    else:
        self.reducer = lambda x, dim: x.mean(dim=dim)

def forward(self, x):
    x = x.view(x.size(0), -1)
    if type(x) == list:
        x = torch.stack(x)
    return self.reducer(x, 0)
</code></pre>
<p>class SpikingNet(nn.Module):</p>
<pre><code>def __init__(self, device, n_time_steps, begin_eval):
    super(SpikingNet, self).__init__()
    assert (0 &lt;= begin_eval and begin_eval &lt; n_time_steps)
    self.device = device
    self.n_time_steps = n_time_steps
    self.begin_eval = begin_eval

    self.input_conversion = InputDataToSpikingPerceptronLayer(device)

    self.layer1 = SpikingNeuronLayerRNN(
        device, n_inputs=480, n_hidden=8568,
        decay_multiplier=0.9, threshold=1.0, penalty_threshold=1.5
    )
    """
    self.layer2 = SpikingNeuronLayerRNN(
        device, n_inputs=100, n_hidden=10,
        decay_multiplier=0.9, threshold=1.0, penalty_threshold=1.5
    )
    """
    self.output_conversion = OutputDataToSpikingPerceptronLayer(
        average_output=False)

    self.to(self.device)

def forward_through_time(self,x):
    
    self.input_conversion.reset_state()
    self.layer1.reset_state()
    # self.layer2.reset_state()

    out = []

    all_layer1_states = []
    all_layer1_outputs = []
    all_layer2_states = []
    all_layer2_outputs = []
    for _ in range(self.n_time_steps):
        xi = self.input_conversion(x)
        layer1_state, layer1_output = self.layer1(xi)
        layer2_state, layer2_output = self.layer2(layer1_output)

        all_layer1_states.append(layer1_state)
        all_layer1_outputs.append(layer1_output)
        all_layer2_states.append(layer2_state)
        all_layer2_outputs.append(layer2_output)
        out.append(layer2_state)

    out = self.output_conversion(out[self.begin_eval:])
    return out, [[all_layer1_states, all_layer1_outputs], [
        all_layer2_states, all_layer2_outputs]]

def forward(self,x):
    out, _ = self.forward_through_time(x)
    return F.log_softmax(out, dim=-1)

def visualize_all_neurons(self, x):
    assert x.shape[0] == 1 and len(x.shape) == 4, (
        "Pass only 1 example to SpikingNet.visualize(x) with outer dimension shape of 1.")
    _, layers_state = self.forward_through_time(x)

    for i, (all_layer_states, all_layer_outputs) in enumerate(layers_state):
        layer_state  =  torch.stack(all_layer_states).data.cpu(
            ).numpy().squeeze().transpose()
        layer_output = torch.stack(all_layer_outputs).data.cpu(
            ).numpy().squeeze().transpose()

        self.plot_layer(layer_state, title="Inner state values of neurons for layer {}".format(i))
        self.plot_layer(layer_output, title="Output spikes (activation) values of neurons for layer {}".format(i))

def visualize_neuron(self, x, layer_idx, neuron_idx):
    assert x.shape[0] == 1 and len(x.shape) == 4, (
        "Pass only 1 example to SpikingNet.visualize(x) with outer dimension shape of 1.")
    _, layers_state = self.forward_through_time(x)

    all_layer_states, all_layer_outputs = layers_state[layer_idx]
    layer_state  =  torch.stack(all_layer_states).data.cpu(
        ).numpy().squeeze().transpose()
    layer_output = torch.stack(all_layer_outputs).data.cpu(
        ).numpy().squeeze().transpose()

    self.plot_neuron(
        layer_state[neuron_idx],
        title="Inner state values neuron {} of layer {}".format(neuron_idx, layer_idx))
    self.plot_neuron(
        layer_output[neuron_idx],
        title="Output spikes (activation) values of neuron {} of layer {}".format(neuron_idx, layer_idx))
</code></pre>
<p>spiking_model = SpikingNet(device, n_time_steps=100, begin_eval=0)<br>
train_many_epochs(spiking_model)</p>
<h2><a name="this-is-the-code-i-have-even-tried-changing-the-input-but-still-get-this-error-output-fc-linearin_features480-out_features8568-biastrue-torchfloat32-torchsize480-8568-torchfloat32-torchsize480-8568-torchfloat32-1" class="anchor" href="#this-is-the-code-i-have-even-tried-changing-the-input-but-still-get-this-error-output-fc-linearin_features480-out_features8568-biastrue-torchfloat32-torchsize480-8568-torchfloat32-torchsize480-8568-torchfloat32-1"></a>This is the code, I have even tried changing the input, but still get this error<br>
output:<br>
fc: Linear(in_features=480, out_features=8568, bias=True)<br>
torch.float32<br>
torch.Size([480, 8568])<br>
torch.float32<br>
torch.Size([480, 8568])<br>
torch.float32</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in &lt;cell line: 406&gt;()<br>
404<br>
405 spiking_model = SpikingNet(device, n_time_steps=100, begin_eval=0)<br>
 → 406 train_many_epochs(spiking_model)<br>
407<br>
408 <span class="hashtag-raw">#Training</span> a Feedforward Neural Network (for comparison)</p>
<p>8 frames<br>
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input)<br>
112<br>
113     def forward(self, input: Tensor) → Tensor:<br>
 → 114         return F.linear(input, self.weight, self.bias)<br>
115<br>
116     def extra_repr(self) → str:</p>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (480x8568 and 480x8568)</p>