How could i change my output size as i expected?

username: astri
cooked: <p>hello all i am a beginner in deep learning and pytorch. I have made sequential model in pytorch like code below.<br>
input shape : (1934,1024)<br>
expected output shape : (1934,8)<br>
batch size = 32</p>
<p>when i train my model and check the output the size turn out to be (14,8). How i can have my output size as my expected size</p>
<pre><code class="lang-auto">class MLP(nn.Module):
    def __init__(self):
        super(MLP, self).__init__()
        self.layers = nn.Sequential(
            nn.Linear(1024, 100),
            nn.Sigmoid(),
            nn.Linear(100, 8),
            #tempsigmoid()
            TempSigmoid()
            )
        
    def forward(self, x):
        x = self.layers(x)
        return x
</code></pre>
<p>this is how i train my model</p>
<pre><code class="lang-auto"># Model TRAINING
epochs = 5
train_losses=[]
elapsed=[]

for epochs in range(epochs):
    t0 = time.time()
    model.train()
    for batch_idx, (x,y) in enumerate(loader):
        x,y = x.to(device), y.to(device)
        optimizer.zero_grad()
        outputs = model(x)
        loss = loss_fn(outputs,y)
        loss.backward()
        optimizer.step()
        StopatLossValue()
        
        # program timing
        torch.cuda.synchronize() #the right way to measure time in pytorch
        t1 = time.time()
        eta = t1-t0
        
        if batch_idx % 100 == 0:
            print('Train Epoch: {} \t Loss: {:.6f} \t Elapsed time :{:.2f} ms'.format(epochs, loss.item(), (eta*1000)))
            elapsed.append(float(eta))
            np.savetxt("eta.csv", elapsed, delimiter=",")
</code></pre>

username: JuanFMontesinos
cooked: <p>Well, As far as I can see you should get the proper size. What is TempSigmoid? is it a pytorch’s function. That’s probably what is reducing your output size.</p>

username: astri
cooked: <p>i was custom sigmoid function with tempsigmoid</p>
<pre><code class="lang-auto">class TempSigmoid(nn.Module):
    def __init__(self):
        super(TempSigmoid, self).__init__() # writing super in python 2.7 (difference with python 3)
        #self.nd = nd
        global nd # define global variable in python
        nd = 3.0
  
    def forward(self,x):
        temp=nd/torch.log(torch.tensor(0.9))
        return torch.sigmoid(x/(temp)) 
</code></pre>
<p>But i think it doesn’t effect the outputs size, because when i checked by not using tempsigmoid its still the same size (14,8)</p>

username: JuanFMontesinos
cooked: <aside class="quote no-group" data-username="astri" data-post="1" data-topic="60347">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/astri/48/12946_2.png" class="avatar"> astri:</div>
<blockquote>
<p>(1934,1024)</p>
</blockquote>
</aside>
<p>I’m afraid your input is not the size you think <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"></p>
<pre><code class="lang-auto">import torch
from torch import nn



class MLP(nn.Module):
    def __init__(self):
        super(MLP, self).__init__()
        self.layers = nn.Sequential(
            nn.Linear(1024, 100),
            nn.Sigmoid(),
            nn.Linear(100, 8),
            # tempsigmoid()
        )

    def forward(self, x):
        x = self.layers(x)
        return x

a= torch.rand(32,1934,1024)

model = MLP()
out = model(a)
print(out.shape)
</code></pre>
<pre><code class="lang-auto">torch.Size([32,1934, 8])
</code></pre>
<p>Btw I didn’t use TempSigmoid as temp variable is not defined</p>

username: astri
cooked: <p>so the problem is my input data is not as i am thinking?<br>
Below is my dataset .</p>
<pre><code class="lang-auto">x = np.loadtxt("./input/set3/new_set3.csv", delimiter=",")
y = np.loadtxt("./input/set3/ideal_full.csv", delimiter=",")

x_train = torch.from_numpy(x).float()
y_train = torch.from_numpy(y).float()


dataset = torch.utils.data.TensorDataset(x_train, y_train)
loader = DataLoader(
    dataset=dataset,
    batch_size = 32,
    shuffle=True,
    num_workers=0,
    pin_memory=torch.cuda.is_available()
)
</code></pre>
<p>I check when prepare dataset my input still (1934,1024). I’ve checked too in training loop after model.train() still the same size, after that it change to (14,1024) what causing it changed?</p>

username: JuanFMontesinos
cooked: <p>I have no idea as I have no acess to your dataset<br>
I would iterate over the dataset asserting the size</p>
<pre><code class="lang-auto">dataset = torch.utils.data.TensorDataset(x_train, y_train)
loader = DataLoader(
    dataset=dataset,
    batch_size = 32,
    shuffle=True,
    num_workers=0,
    pin_memory=torch.cuda.is_available()
)
for sample in loader:
    assert sample.shape == expected
</code></pre>
<p>But for sure you have some sneaky line somewhere you forgot.</p>

username: astri
cooked: <p>i had solved it by changing my batch size to 1934 the same number with my samples</p>

username: JuanFMontesinos
cooked: <p>I don’t know It shouldn’t have any effect.<br>
Linear layer takes a (*,N) input. At least in the later versions.</p>

username: astri
cooked: <p>i dont know why it works. i am using torch 1.2.0</p>

username: ecdrid
cooked: <p>that would mean that you only have a single batch now, isn’t effective as this BS might hinder your performance as well…</p>

username: astri
cooked: <p>excuse for my not understanding. So this batch size make my model performance not good?</p>

username: ecdrid
cooked: <p>it depends on your task at hand as well; I am . pretty sure you have some data from csv’s probably; Had it been some image data etc, the model might take a longer time to generalize [speaking in general], plus everytime we don’t have the possibility to increase the batch size to 1k+ as well; (system constraints)<br>
I would suggest, try to narrow down things with the example you have first <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: astri
cooked: <p>i am a beginner in DL and pytorch and doing research with custom activation function and weight clip and stop training with loss value.</p>
<p>My input is a handwriting number with 32x32 and has 1934 samples number. and my target data is 1934 samples with 8 bit so i need output with 1934 and 8 bit so i can compare and count loss with my target data.</p>
<p>this is the first time to i am using batch size to 1k+ and i dont know whether is it right or no.</p>
<p>do you have suggestion the good example which i can learn. I am googling all and make my model just like now</p>

username: JuanFMontesinos
cooked: <p>Here you have a fully runnable script which fits exactly what you specified.</p>
<pre><code class="lang-auto">import torch
from torch import nn


class TempSigmoid(nn.Module):
    def __init__(self):
        super(TempSigmoid, self).__init__()  # writing super in python 2.7 (difference with python 3)
        self.nd =3.0


    def forward(self, x):
        temp = self.nd / torch.log(torch.tensor(0.9))
        return torch.sigmoid(x / (temp))

class MLP(nn.Module):
    def __init__(self):
        super(MLP, self).__init__()
        self.layers = nn.Sequential(
            nn.Linear(1024, 100),
            nn.Sigmoid(),
            nn.Linear(100, 8),
            # tempsigmoid()
            TempSigmoid()
        )

    def forward(self, x):
        x = self.layers(x)
        return x

N = 10
epochs = 1

inp= torch.rand(N*32,1934,1024)
gt= torch.rand(N*32,1934,8)
dataset = torch.utils.data.TensorDataset(inp,gt)
loader = torch.utils.data.DataLoader(
    dataset=dataset,
    batch_size = 32,
    shuffle=True,
    num_workers=0,
)
model = MLP()
loss_fn = torch.nn.L1Loss()
optimizer = torch.optim.SGD(model.parameters(),lr=0.1)
for epoch in range(epochs):
    model.train()
    for batch_idx, (x, y) in enumerate(loader):
        optimizer.zero_grad()
        outputs = model(x)
        loss = loss_fn(outputs, y)
        loss.backward()
        optimizer.step()
        print('Input shape: %r. Output shape %r' %(x.shape,outputs.shape))
</code></pre>
<pre><code class="lang-auto">Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
Input shape: torch.Size([32, 1934, 1024]). Output shape torch.Size([32, 1934, 8])
</code></pre>
<p>I have to insist that if it’s not working you probably has any issue in your data. Mainly because linear layer takes a …,Q tensor and map it to a …,Q’.<br>
Thus, the mistake is in the data or any line of code<br>
Ofc, the suitability of the chosen architecture depends on what kinda data are you using.<br>
In general the higher batch size, the better.</p>
<p>The thing is that, according to what you say here</p>
<blockquote>
<pre><code class="lang-auto">My input is a handwriting number with 32x32 and has 1934 samples number. and my target data is 1934 samples with 8 bit so i need output with 1934 and 8 bit so i can compare and count loss with my target data.
</code></pre>
</blockquote>
<p>I think Your input is not 1934,1024 but you have 1934 samples. That’s what’s not clear to me. Maybe it’s related to that.<br>
Realize x and y (in your code) must be shape shape everywhere but the last dimension (depending on the loss you are using).</p>
<p>Can you print x.shape, y.shape, x_train.shape and y_train.shape</p>

username: astri
cooked: <p>Thank you for your reply</p>
<p>I tried the code that you write and the result like below</p>
<pre><code class="lang-auto">Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([32, 1024]). Output shape torch.Size([32, 8])
Input shape: torch.Size([14, 1024]). Output shape torch.Size([14, 8])
</code></pre>
<p>in preparing my dataset i had print the shape</p>
<pre><code class="lang-auto">((1934, 1024), (1934, 8), (1934, 1024), (1934, 1024))
</code></pre>
<p>data shape are changing when it come to loop in training</p>

username: JuanFMontesinos
cooked: <p>I don’t know what those shapes corresponds to but it seems you did not properly set something right? dataset maybe</p>

username: astri
cooked: <p>thank you for your reply, yes i think something its not right and still try to analyze it</p>

username: Wei_Liu
cooked: <p>I noticed that 1934%32=14, I think it is because you just output the last batch.</p>

username: astri
cooked: <p>thank you for your respond. excuse me for my lack of knowledge , if i want to get all the output how i can do it?</p>