RuntimeError: matrices expected, got 1D, 2D tensors

username: Hell_Razor
cooked: <p>I’m working on my first project with PyTorch and Syft. Here I am facing an issue of matrices expected. This same dataset works for training and it gets completed with no issue. As soon as I try to run test loop, it shows the above error.</p>
<p>The batch size is 1024, and input features 30.</p>
<pre><code class="lang-auto">class MLPUpdated(nn.Module):
    def __init__(self):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(30, 32),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(32,16),
            nn.ReLU(),
            nn.Dropout(0.1),
            nn.Linear(16, 8),
            nn.ReLU(),
            nn.Linear(8, 1),
            nn.Sigmoid()
        )

    def forward(self, x):
        return self.layers(x)
</code></pre>
<blockquote>
<p>Error:<br>
Traceback (most recent call last):<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\tensors\interpreters\native.py”, line 336, in handle_func_command<br>
new_args, new_kwargs, new_type, args_type = hook_args.unwrap_args_from_function(<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook_args.py”, line 167, in unwrap_args_from_function<br>
new_args = args_hook_function(args_)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook_args.py”, line 356, in <br>
return lambda x: f(lambdas, x)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook_args.py”, line 534, in three_fold<br>
lambdas[0](args_[0], **kwargs),<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook_args.py”, line 331, in <br>
else lambda i: forward_func<a>type(i)</a><br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\hook\hook_args.py”, line 27, in <br>
else (_ for _ in ()).throw(PureFrameworkTensorFoundError),<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\hook\hook_args.py”, line 27, in <br>
else (_ for _ in ()).throw(PureFrameworkTensorFoundError),<br>
syft.exceptions.PureFrameworkTensorFoundError<br>
During handling of the above exception, another exception occurred:<br>
Traceback (most recent call last):<br>
File “federated_learning.py”, line 274, in <br>
test_loss, test_acc =  test(args, model, device, test_loader, epoch)<br>
File “federated_learning.py”, line 145, in test<br>
output = model(data)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py”, line 532, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “D:\internship\Indo_Aus\Federated Learning\Split-Learning-and-Federated-Learning\src\models.py”, line 213, in forward<br>
return self.layers(x)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py”, line 532, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\container.py”, line 100, in forward<br>
input = module(input)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py”, line 532, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\linear.py”, line 87, in forward<br>
return F.linear(input, self.weight, self.bias)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook.py”, line 335, in overloaded_func<br>
response = handle_func_command(command)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\tensors\interpreters\native.py”, line 380, in handle_func_command<br>
response = cls.<em>get_response(cmd, args</em>, kwargs_)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\tensors\interpreters\native.py”, line 414, in <em>get_response<br>
response = command_method(*args</em>, **kwargs_)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\functional.py”, line 1370, in linear<br>
ret = torch.addmm(bias, input, weight.t())<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook.py”, line 335, in overloaded_func<br>
response = handle_func_command(command)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\tensors\interpreters\native.py”, line 380, in handle_func_command<br>
response = cls.<em>get_response(cmd, args</em>, kwargs_)<br>
File “C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\frameworks\torch\tensors\interpreters\native.py”, line 414, in <em>get_response<br>
response = command_method(*args</em>, **kwargs_)<br>
RuntimeError: matrices expected, got 1D, 2D tensors at C:\w\1\s\windows\pytorch\aten\src\TH/generic/THTensorMath.cpp:131</p>
</blockquote>

username: KFrank
cooked: <p>Hi Hell!</p>
<aside class="quote no-group" data-username="Hell_Razor" data-post="1" data-topic="125038" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hell_razor/48/39179_2.png" class="avatar"> Hell_Razor:</div>
<blockquote>
<p>As soon as I try to run test loop, it shows the above error.</p>
<p>The batch size is 1024, and input features 30.</p>
</blockquote>
</aside>
<p>You must not be calling your model correctly in your test loop; your<br>
code works for me:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; import torch.nn as nn
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; class MLPUpdated(nn.Module):
...     def __init__(self):
...         super().__init__()
...         self.layers = nn.Sequential(
...             nn.Linear(30, 32),
...             nn.ReLU(),
...             nn.Dropout(0.2),
...             nn.Linear(32,16),
...             nn.ReLU(),
...             nn.Dropout(0.1),
...             nn.Linear(16, 8),
...             nn.ReLU(),
...             nn.Linear(8, 1),
...             nn.Sigmoid()
...         )
...
...     def forward(self, x):
...         return self.layers(x)
...
&gt;&gt;&gt; _ = torch.manual_seed (2021)
&gt;&gt;&gt; MLPUpdated() (torch.randn (1024, 30))
tensor([[0.4777],
        [0.4868],
        [0.4884],
        ...,
        [0.4624],
        [0.4619],
        [0.4893]], grad_fn=&lt;SigmoidBackward&gt;)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: Hell_Razor
cooked: <aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="125038">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<pre><code class="lang-auto">import torch
&gt;&gt;&gt; import torch.nn as nn
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; class MLPUpdated(nn.Module):
...     def __init__(self):
...         super().__init__()
...         self.layers = nn.Sequential(
...             nn.Linear(30, 32),
...             nn.ReLU(),
...             nn.Dropout(0.2),
...             nn.Linear(32,16),
...             nn.ReLU(),
...             nn.Dropout(0.1),
...             nn.Linear(16, 8),
...             nn.ReLU(),
...             nn.Linear(8, 1),
...             nn.Sigmoid()
...         )
...
...     def forward(self, x):
...         return self.layers(x)
...
&gt;&gt;&gt; _ = torch.manual_seed (2021)
&gt;&gt;&gt; MLPUpdated() (torch.randn (1024, 30))
</code></pre>
</blockquote>
</aside>
<p>Hey KFrank,<br>
Thanks for the reply.<br>
The model is working perfectly, and the data passed inside the forward function is also tensor. But still I’m getting this error.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "federated_learning.py", line 283, in &lt;module&gt;
    test_loss, test_acc =  test(args, model, device, test_loader, epoch)
  File "federated_learning.py", line 145, in test
    output = model(data)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\internship\Indo_Aus\Federated Learning\Split-Learning-and-Federated-Learning\src\models.py", line 206, in forward
    return self.layers(x)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\container.py", line 100, in forward
    input = module(input)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\syft\generic\frameworks\hook\hook.py", line 335, in overloaded_func
    response = handle_func_command(command)
</code></pre>
<p>This error is indicating that something is wrong in the self.layers<br>
Given below are the tensors being passed inside the self.layers function.</p>
<pre><code class="lang-auto">[ 3.9251e+00,  6.7676e-01,  1.3711e+00,  ...,  4.6753e-01,
         -8.6226e-02, -3.1110e-02],
        [ 8.0766e+00, -1.7748e-01,  1.2341e-01,  ..., -2.9790e-01,
         -9.5155e-02,  1.1550e-01],
        [-2.8701e-01, -9.5842e-02, -3.6565e+00,  ...,  3.8841e-01,
         -2.5426e+00, -2.0138e+00]])
</code></pre>
<p>The test function is given below.</p>
<pre><code class="lang-auto">def test(args, model, device, test_loader, epoch):
    """Evaluate the model

    Args:
        args: Hyper-parameters for testing
        model: The model to be evaluated
        device: Training device (e.g. cpu/gpu)
        test_loader: Federated data loader for testing
        epoch: Current epoch

    Returns:
        test_loss: Test loss for current epoch
        test_acc: Test accuracy for current epoch
    """
    model.eval()
    test_loss, correct = 0.0, 0

    with torch.no_grad():
        for data, target in test_loader:
            data, target = data.to(device), target.to(device)

            # Feed-forward data through the model
            # print("Test: ", data.location.size)
            # print("Test Type: ", type(target))
            output = model(data)

            # Sum up the batch loss
            binary_loss = torch.nn.BCEWithLogitsLoss()
            target = target.unsqueeze(1)
            target = target.float()
            test_loss += binary_loss(output, target, reduction="sum").item()
            # test_loss += F.nll_loss(output, target, reduction="sum").item()

            # Retrieve the index of the maximum log-probability
            pred = output.argmax(1, keepdim=True)

            # Get the number of correct classifications
            correct += pred.eq(target.view_as(pred)).sum().item()

    # Average the test_loss
    test_loss /= len(test_loader.dataset)

    # Test accuracy
    test_acc = correct / len(test_loader.dataset)

    print("\nTest set - Epoch: {} - Loss: {:.4f}, Acc: {:.2f}%\n".format(
        epoch, test_loss, 100 * test_acc))

    return test_loss, test_acc
</code></pre>
<p>Test Loader is given below.</p>
<pre><code class="lang-auto">test_ds = PeopleDataset(test_file)
    # train_loader = torch.utils.data.DataLoader(train_ds,batch_size=args.batch_size, shuffle=True)

test_loader = sy.FederatedDataLoader(test_ds.federate(workers),
                  batch_size=args.batch_size, shuffle=True, **kwargs)
</code></pre>

username: KFrank
cooked: <p>Hi Hell!</p>
<aside class="quote no-group" data-username="Hell_Razor" data-post="3" data-topic="125038" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hell_razor/48/39179_2.png" class="avatar"> Hell_Razor:</div>
<blockquote>
<p>The model is working perfectly</p>
</blockquote>
</aside>
<p>From what I understand calling your model in your training loop<br>
works, but it fails in your test loop.  The error message you quote<br>
suggests that you are applying your model to an input tensor of<br>
the wrong shape.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">            # print("Test: ", data.location.size)
</code></pre>
</blockquote>
</aside>
<p>As an aside, this looks fishy.  It’s only a comment, of course, but<br>
if the <code>data</code> you pass to your <code>model</code> really has a <code>location</code> property,<br>
it probably isn’t what your model is expecting.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">            output = model(data)
</code></pre>
</blockquote>
</aside>
<p>I assume that your training loop also has an equivalent<br>
<code>model (data)</code> call in it somewhere.</p>
<p>Print out the <code>shape</code> of <code>data</code> just before your training-loop<br>
<code>model (data)</code> call and your test-loop <code>model (data)</code> call.<br>
I expect that they will not be the same.</p>
<p>If they are the same, capture the training-loop and test-loop<br>
<code>data</code> tensors, and post a complete, self-contained, runnable<br>
script that reproduces your error that uses those example<br>
<code>data</code> tensors.</p>
<p>That is, get rid of the data loaders and loops – just have<br>
something like:</p>
<pre><code class="lang-auto">train_data = torch.tensor (...)
model (train_data)
test_data = torch.tensor (...)
model.eval()
with torch.no_grad():
    model (test_data)
</code></pre>
<p>And also post the complete results of your self-contained,<br>
runnable script.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Hell_Razor
cooked: <p>Hey KFrank,<br>
Thanks again for the reply.</p>
<p>Here’s the code snippet which you asked for.</p>
<pre><code class="lang-auto">class MLPUpdated(nn.Module):
    def __init__(self):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(30, 32),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(32,16),
            nn.ReLU(),
            nn.Dropout(0.1),
            nn.Linear(16, 8),
            nn.ReLU(),
            nn.Linear(8, 1),
            nn.Sigmoid()
        )

    def forward(self, x):
        return self.layers(x)

device = "cuda" if torch.cuda.is_available() else "cpu"
device = torch.device(device)
model = MLPUpdated().to(device)

train_data = torch.tensor([ 0.3018, -0.3064,  1.1054, -0.0751,  0.4352,  0.9034, -0.4978, -0.4954,
         0.0315, -0.0492,  0.4202, -0.2261, -0.2327,  0.4352, -0.5217,  0.1191,
        -0.0143, -0.7541,  0.4915, -1.1997, -0.2744, -0.1156, -0.0913, -0.1326,
        -0.0158,  0.4527,  0.4410,  0.3742, -0.0228,  0.0137])
print(train_data.shape)

test_data = torch.tensor([ 0.5229,  0.8923, -1.6663, -1.8197,  1.1027, -1.2481,  0.3701,  1.5603,
        -1.6727,  1.3396,  0.1646, -0.6536, -0.4564,  1.0487, -0.0249, -0.4200,
        -1.7048, -1.9665,  0.4084,  0.8154, -1.8285, -0.3640, -0.0078,  0.1991,
         0.2119, -1.6663, -1.0810, -0.4766,  0.2329, -0.0569])
print(test_data.shape)

model(train_data)
model.eval()
with torch.no_grad():
    model(test_data)
print("SUCCESS")
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">2021-06-26 12:06:47.330716: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2021-06-26 12:06:47.340872: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.        
torch.Size([30])
torch.Size([30])
SUCCESS
</code></pre>
<p>This is running successfully.</p>
<p>But when I run the below code, it gets the error of matrices expected.</p>
<pre><code class="lang-auto">#!/usr/bin/python3

"""federated.py Contains an implementation of federated learning with ten
                workers applied to the Fashion MNIST data set for image
                classification using a slightly modified version of the LeNet5
                CNN architecture.

For the ID2223 Scalable Machine Learning course at KTH Royal Institute of
Technology"""

__author__ = "Xenia Ioannidou and Bas Straathof"


import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms

from sys import argv
from time import time
from argparse import ArgumentParser, Namespace
from syft.federated.floptimizer import Optims

# The Pysyft Federated Learning library
import syft as sy

from models import LeNetComplete, MLP, MLPUpdated
from plotting import generate_simple_plot

from load_data import PeopleDataset

torch.autograd.set_detect_anomaly(True)

def parse_args() -&gt; Namespace:
    """Parses CL arguments

    Returns:
        Namespace object containing all arguments"""
    parser = ArgumentParser()

    parser.add_argument("-bs", "--batch_size", type=int, default=64)
    parser.add_argument("-tbs", "--test_batch_size", type=int, default=1000)
    parser.add_argument("-ls", "--log_steps", type=int, default=50)
    parser.add_argument("-lr", "--learning_rate", type=float, default=0.01)
    parser.add_argument("-e", "--epochs", type=int, default=25)

    return parser.parse_args(argv[1:])


def train(args, model, device, train_loader, optimizer, epoch):
    model.train()
    """Train the model

    Args:
        args: Hyper-paramters for training
        model: The model to be trained
        device: Training device (e.g. cpu/gpu)
        train_loader: Federated data loader for training
        optimizer: Training optimizer (e.g. SGD)
        epoch: Current epoch

    Returns:
        training_time: Time it took for one epoch of training
    """
    start = time()
    # data, target = next(iter(train_loader))
    # batch_idx = 0
    for batch_idx, (data, target) in enumerate(train_loader):
        # Ensure that thet data is send to the right worker
        # print("Train: " ,data.shape)
        model.send(data.location)

        # Write the data and targets to the compute device
        data, target = data.to(device), target.to(device)

        #Call the optimizer for the worker using get_optim
        # print(data.location)
        # print(optimizer.optimizers)
        opt = optimizer.get_optim(data.location.id)

        # Reset the optimizers gradients to zero
        # optimizer.zero_grad()
        opt.zero_grad()

        # Pass the data to the model
        print("Train: ", data.shape)
        # print("Train Type: ", type(data.location))
        output = model(data)

        # Compute the loss
        # binary_loss = nn.BCELoss()
        binary_loss = torch.nn.BCEWithLogitsLoss()
        target = target.unsqueeze(1)
        target = target.float()
        loss = binary_loss(output, target)
        # loss = F.nll_loss(output, target)

        # Execute the backpropagation step
        loss.backward()

        # Apply the optimizer
        opt.step()

        # Retrieve the model (PySyft feature)
        model.get()

        if batch_idx % args.log_steps == 0:
            # Retrieve the loss (PySyft feature)
            loss = loss.get()
            print(' {} - Epoch: {} [{}/{} ({:.0f}%)] Loss: {:.6f}'.format(
                data.location.id, epoch, batch_idx * args.batch_size,
                len(train_loader) * args.batch_size,
                100. * batch_idx / len(train_loader), loss.item()))

    end = time()
    training_time = end-start

    return training_time

def test(args, model, device, test_loader, epoch):
    """Evaluate the model

    Args:
        args: Hyper-parameters for testing
        model: The model to be evaluated
        device: Training device (e.g. cpu/gpu)
        test_loader: Federated data loader for testing
        epoch: Current epoch

    Returns:
        test_loss: Test loss for current epoch
        test_acc: Test accuracy for current epoch
    """
    model.eval()
    test_loss, correct = 0.0, 0

    with torch.no_grad():
        for data, target in test_loader:
            data, target = data.to(device), target.to(device)

            # Feed-forward data through the model
            print("Test: ", data.shape)
            # print("Test Type: ", type(target))
            output = model(data)

            # Sum up the batch loss
            binary_loss = torch.nn.BCEWithLogitsLoss()
            target = target.unsqueeze(1)
            target = target.float()
            test_loss += binary_loss(output, target, reduction="sum").item()
            # test_loss += F.nll_loss(output, target, reduction="sum").item()

            # Retrieve the index of the maximum log-probability
            pred = output.argmax(1, keepdim=True)

            # Get the number of correct classifications
            correct += pred.eq(target.view_as(pred)).sum().item()

    # Average the test_loss
    test_loss /= len(test_loader.dataset)

    # Test accuracy
    test_acc = correct / len(test_loader.dataset)

    print("\nTest set - Epoch: {} - Loss: {:.4f}, Acc: {:.2f}%\n".format(
        epoch, test_loss, 100 * test_acc))

    return test_loss, test_acc

if __name__ == "__main__":
    args = parse_args()

    # Pysyft needs to be hooked to PyTorch to enable its features
    hook = sy.TorchHook(torch)

    # Define the workers
    alfa    = sy.VirtualWorker(hook, id="alfa")
    bravo   = sy.VirtualWorker(hook, id="bravo")

    workers = (alfa, bravo)
    workers_id = (alfa.id, bravo.id)

    # Check if a GPU is available
    device = "cuda" if torch.cuda.is_available() else "cpu"
    device = torch.device(device)
    kwargs = {'num_workers': 1, 'pin_memory': True} if device=="cuda" else {}


    # Federated learning needs a special train loader to distribute the data
    # over the various workers
    print("Loading data...")
    train_file = '../dataset/creditcard_train.csv'
    train_ds = PeopleDataset(train_file)

    train_loader = sy.FederatedDataLoader(train_ds.federate(workers),
        batch_size=args.batch_size, shuffle=True, **kwargs)


    test_file = '../dataset/creditcard_test.csv'
    test_ds = PeopleDataset(test_file)
    
    test_loader = sy.FederatedDataLoader(test_ds.federate(workers),
        batch_size=args.batch_size, shuffle=True, **kwargs)


    print("Data loaded...")

    # Instantiate the CNN model
    model = MLPUpdated().to(device)

    # Load the optimizer
    optimizer = Optims(workers_id, optim=optim.Adam(params=model.parameters(),lr=0.1))

    # Train and evaluate for a number of epochs
    total_train_time, test_losses, test_accs = 0.0, [], []
    for epoch in range(1, args.epochs + 1):
        train_time = train(args, model, device, train_loader, optimizer, epoch)
        total_train_time += train_time

        test_loss, test_acc =  test(args, model, device, test_loader, epoch)
        test_losses.append(test_loss)
        test_accs.append(test_acc)



    print("Total training time: {:.2f}s".format(total_train_time))
    print("Final test accuracy: {:.4f}".format(test_acc))
    print("Final test loss: {:.4f}".format(test_loss))


</code></pre>
<p>This gives the following output:</p>
<pre><code class="lang-auto">Train:  torch.Size([1024, 30])
Train:  torch.Size([1024, 30])
Train:  torch.Size([1024, 30])
 alfa - Epoch: 1 [102400/257024 (40%)] Loss: 0.693148
Train:  torch.Size([1024, 30])
Train:  torch.Size([1024, 30])
Train:  torch.Size([1024, 30])
.
.
.
Train:  torch.Size([1024, 30])
Train:  torch.Size([1024, 30])
 bravo - Epoch: 1 [256000/257024 (100%)] Loss: 0.693148
Train:  torch.Size([163, 30])
Test:  torch.Size([1024, 30])
Traceback (most recent call last):
.
.
.
Traceback (most recent call last):
  File "federated_learning.py", line 284, in &lt;module&gt;
    test_loss, test_acc =  test(args, model, device, test_loader, epoch)
  File "federated_learning.py", line 146, in test
    output = model(data)
  File "C:\Users\het\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "\Federated Learning\Split-Learning-and-Federated-Learning\src\models.py", line 206, in forward
    return self.layers(x)
.
.
.
RuntimeError: matrices expected, got 1D, 2D tensors at C:\w\1\s\windows\pytorch\aten\src\TH/generic/THTensorMath.cpp:131
</code></pre>

username: KFrank
cooked: <p>Hi Hell!</p>
<p>You are posting misleading code.</p>
<p>This version of <code>MLPUpdated</code>:</p>
<aside class="quote no-group quote-modified" data-username="Hell_Razor" data-post="5" data-topic="125038" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hell_razor/48/39179_2.png" class="avatar"> Hell_Razor:</div>
<blockquote>
<pre><code class="lang-auto">class MLPUpdated(nn.Module):
    def __init__(self):
...
</code></pre>
</blockquote>
</aside>
<p>comes from self-contained code you posted (which would be good<br>
if it were relevant, but it isn’t), while this version:</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">from models import LeNetComplete, MLP, MLPUpdated
</code></pre>
</blockquote>
</aside>
<p>comes from some package called “models.”</p>
<p>And we know that these two versions of <code>MLPUpdated</code> differ<br>
substantively because, among other reasons, your self-contained<br>
version does not contain a <code>.send()</code> method.</p>
<p>You are probably using your “federated learning” Syft framework<br>
incorrectly.  (But I don’t know …)</p>
<p>Note, you call your <code>model</code> differently in your train and test loops:</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">def train(args, model, device, train_loader, optimizer, epoch):
    model.train()
...
    for batch_idx, (data, target) in enumerate(train_loader):
        model.send(data.location)
...
        print("Train: ", data.shape)
        # print("Train Type: ", type(data.location))
        output = model(data)
...
        model.get()
</code></pre>
</blockquote>
</aside>
<p>vs.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">def test(args, model, device, test_loader, epoch):
    model.eval()
...
    with torch.no_grad():
        for data, target in test_loader:
            data, target = data.to(device), target.to(device)

            # Feed-forward data through the model
            print("Test: ", data.shape)
            # print("Test Type: ", type(target))
            output = model(data)
...
</code></pre>
</blockquote>
</aside>
<p>I have two suggestions for you:</p>
<ol>
<li>
<p>Debugging 101:  Trim down your code to the simplest version that<br>
still shows the error.  Can you get rid of the <code>ArgumentParser()</code><br>
stuff?  The <code>plotting</code> stuff?  Can you get rid of <code>.to (device)</code>?<br>
Can you get rid of the data loaders?  Can you get rid of the loops<br>
and just make a single train and test call to <code>model (data)</code>?  Can<br>
you get rid of the syft stuff and still reproduce the error?  (I doubt it.)</p>
</li>
<li>
<p>Try printing out <code>type (model)</code>, <code>type (model).mro()</code>,<br>
<code>type (data)</code>, and <code>type (data).mro()</code> just before your train<br>
and test calls to <code>model (data)</code> to see what Syft is doing under<br>
the hood.  Maybe you’re trying to call a “syft-hooked” <code>model</code> with<br>
an ordinary pytorch tensor, or vice versa.</p>
</li>
</ol>
<p>(As an aside, let me note that the code you posted that reproduces<br>
your error is not a “complete, self-contained, runnable script.”)</p>
<p>Best.</p>
<p>K. Frank</p>

username: Hell_Razor
cooked: <p>Hey K.Frank,<br>
Thanks for the reply.<br>
MLPUpdated from models is same as MLPUpdated. I just wrote it together for your convinence, but sorry for that.</p>
<p>Thanks for the help.<br>
Still I don’t understand the difference between train and test call for model. Can you please eloborate that part?</p>

username: KFrank
cooked: <p>Hi Hell!</p>
<aside class="quote no-group" data-username="Hell_Razor" data-post="7" data-topic="125038" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hell_razor/48/39179_2.png" class="avatar"> Hell_Razor:</div>
<blockquote>
<p>MLPUpdated from models is same as MLPUpdated.</p>
</blockquote>
</aside>
<p>No it’s not:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; class MLPUpdated(nn.Module):
...     def __init__(self):
...         super().__init__()
...         self.layers = nn.Sequential(
...             nn.Linear(30, 32),
...             nn.ReLU(),
...             nn.Dropout(0.2),
...             nn.Linear(32,16),
...             nn.ReLU(),
...             nn.Dropout(0.1),
...             nn.Linear(16, 8),
...             nn.ReLU(),
...             nn.Linear(8, 1),
...             nn.Sigmoid()
...         )
...
...     def forward(self, x):
...         return self.layers(x)
...
&gt;&gt;&gt; model = MLPUpdated()
&gt;&gt;&gt; type (model)
&lt;class '__main__.MLPUpdated'&gt;
&gt;&gt;&gt; model.send
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;...&gt;\torch\nn\modules\module.py", line 1131, in __getattr__
    type(self).__name__, name))
AttributeError: 'MLPUpdated' object has no attribute 'send'
</code></pre>
<p>I guarantee that your “models” version of <code>MLPUpdated</code> does not have<br>
type <code>&lt;class '__main__.MLPUpdated'&gt;</code>.  Also, in your code that is<br>
not self-contained you call <code>model.send()</code>.  Your free-standing version<br>
of <code>MLPUpdated</code> does not have a <code>.send()</code> method.</p>
<p>Perhaps you copy-pasted some lines of code out of some more<br>
elaborate context and thought things would be the same, but lacking<br>
that larger context, there is no reason they should be.</p>
<aside class="quote no-group">
<blockquote>
<p>Still I don’t understand the difference between train and test call for model.</p>
</blockquote>
</aside>
<p>In my previous reply I excerpted the relevant portions of the code you<br>
posted.</p>
<p>Among other things you call <code>model.send()</code> / <code>model.get()</code> in your<br>
training loop, but not in your test loop.  Also, you get your <code>data</code><br>
object from different data loaders, so there is no reason to expect<br>
them to be of the same or similar type.</p>
<p>Did you try any of the things I suggested in my previous post?</p>
<p>Best.</p>
<p>K. Frank</p>