Cannot Set Batch Size to Anything Aside from 1 on my CNN Model

tired_grad_student: <p>Hi there. I built a CNN that I am training on data I generated. The CNN uses 1d convolutions since each data sample is a row of numbers rather than a picture. I am running into an issue where the model only works if the batch size is 1. I’ve seen other posts (like this one: <a href="https://discuss.pytorch.org/t/batchnorm1d-input-shape/91364" class="inline-onebox">BatchNorm1d - input shape</a>) in this forum talk about resizing or permuting data so it will be the proper shape, but I’ve spent a month trying to figure out this issue on my own and nothing seems to work. When I run the model with a batch size that is not 1, I get the following error:</p>
<p>RuntimeError: expand(torch.cuda.FloatTensor{[100, 3]}, size=[3]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)</p>
<p>Please see the model below.<br>
Any help in this matter would be greatly appreciated. Thank you!</p>
<p>import time<br>
from numpy import savetxt<br>
import torch<br>
from torch.utils.data import DataLoader<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import torchvision<br>
import torchvision.transforms as transforms<br>
import matplotlib.pyplot as plt<br>
import numpy as np<br>
import csv<br>
from numpy import genfromtxt</p>
<p>#%%</p>
<p>start = time.time()  # Begin Timer</p>
<h1><a name="device-configuration-1" class="anchor" href="#device-configuration-1"></a>Device configuration</h1>
<p><span class="hashtag-raw">#device</span> = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)<br>
device = torch.device(‘cuda:0’ if torch.cuda.is_available() else ‘cpu’)</p>
<h1><a name="hyperparameters-2" class="anchor" href="#hyperparameters-2"></a>Hyperparameters</h1>
<p>ngrid = 16200;<br>
num_epochs = 500;<br>
batch_size = 1<br>
learning_rate = 0.0003 # originally 0.001<br>
<span class="hashtag-raw">#momentum</span> = 0.5;<br>
neg_slope = 0.4   ; # Neg slope for a Leaky Relu<br>
alpha = 1; # ELU constant</p>
<h1><a name="store-the-data-in-a-numpy-variable-3" class="anchor" href="#store-the-data-in-a-numpy-variable-3"></a>Store the data in a Numpy variable</h1>
<p>with open(‘current working data/train_data.xlsx’, ‘r’) as f1:<br>
<span class="hashtag-raw">#with</span> open(‘train_data.xlsx’, ‘r’) as f1:</p>
<pre><code>reader = csv.reader(f1) # store excel file in variable 'reader'
training_data_np_list = list(reader) # convert 'reader' to a list
training_data_np = np.array(training_data_np_list, dtype = np.float32) # convert list into a np array
training_data_np = training_data_np[:, 0:16200]; # only keep N_grid data (get rid of labels)
</code></pre>
<p>with open(‘current working data/train_data_labels.xlsx’, ‘r’) as f2:<br>
<span class="hashtag-raw">#with</span> open(‘train_data_labels.xlsx’, ‘r’) as f2:</p>
<pre><code>reader = csv.reader(f2)
training_data_labels_list = list(reader)
training_data_labels_np = np.array(training_data_labels_list, dtype = np.float32)
</code></pre>
<p>with open(‘current working data/test_data.xlsx’, ‘r’) as f3:<br>
<span class="hashtag-raw">#with</span> open(‘test_data.xlsx’, ‘r’) as f3:</p>
<pre><code>reader = csv.reader(f3)
test_data_np_list = list(reader)
test_data_np = np.array(test_data_np_list, dtype = np.float32)
test_data_np = test_data_np[:, 0:16200]; # only keep n_grid data
</code></pre>
<p>with open(‘current working data/test_data_labels.xlsx’, ‘r’) as f4:<br>
<span class="hashtag-raw">#with</span> open(‘test_data_labels.xlsx’, ‘r’) as f4:</p>
<pre><code>reader = csv.reader(f4)
test_data_labels_list = list(reader)
test_data_labels_np = np.array(test_data_labels_list, dtype = np.float32)
</code></pre>
<h1><a name="convert-numpy-data-into-pytorch-tensors-the-data-includes-the-labels-4" class="anchor" href="#convert-numpy-data-into-pytorch-tensors-the-data-includes-the-labels-4"></a>Convert Numpy data into Pytorch Tensors - The data includes the labels</h1>
<p><span class="hashtag-raw">#training_data_pth</span> = torch.from_numpy(training_data_np, requires_grad=True);<br>
<span class="hashtag-raw">#training_data_labels_pth</span> = torch.from_numpy(training_data_labels_np, requires_grad=True);<br>
<span class="hashtag-raw">#test_data_pth</span> = torch.from_numpy(test_data_np, requires_grad=True);<br>
<span class="hashtag-raw">#test_data_labels_pth</span> = torch.from_numpy(test_data_labels_np, requires_grad=True);</p>
<h1><a name="convert-numpy-data-into-pytorch-tensor-original-42124-5" class="anchor" href="#convert-numpy-data-into-pytorch-tensor-original-42124-5"></a>Convert Numpy data into Pytorch Tensor - Original 4.21.24</h1>
<p>training_data_pth = torch.from_numpy(training_data_np);<br>
training_data_labels_pth = torch.from_numpy(training_data_labels_np);<br>
test_data_pth = torch.from_numpy(test_data_np);<br>
test_data_labels_pth = torch.from_numpy(test_data_labels_np);</p>
<p>print(‘Training Data Type:’, training_data_pth.dtype)<br>
print(‘Training Data Labels Type:’, training_data_labels_pth.dtype)<br>
print(‘Test Data Type:’, test_data_pth.dtype)<br>
print(‘Test Data Labels Type:’, test_data_labels_pth.dtype)</p>
<h1><a name="transform-the-tensor-into-a-dataset-6" class="anchor" href="#transform-the-tensor-into-a-dataset-6"></a>Transform the tensor into a dataset</h1>
<p>training_data = torch.utils.data.TensorDataset(training_data_pth, training_data_labels_pth);<br>
test_data = torch.utils.data.TensorDataset(test_data_pth, test_data_labels_pth);</p>
<h1><a name="wrap-the-dataset-with-a-data-loader-this-makes-it-an-iterable-object-7" class="anchor" href="#wrap-the-dataset-with-a-data-loader-this-makes-it-an-iterable-object-7"></a>Wrap the dataset with a data loader - this makes it an iterable object</h1>
<p>train_loader = DataLoader(training_data, batch_size = batch_size, shuffle = False)<br>
test_loader = DataLoader(test_data, batch_size = batch_size, shuffle = False)</p>
<h1><a name="one-batch-of-random-training-data-samples-8" class="anchor" href="#one-batch-of-random-training-data-samples-8"></a>one batch of random training data samples</h1>
<p><span class="hashtag-raw">#data_iter</span> = iter(train_loader)<br>
<span class="hashtag-raw">#label_iter</span> = iter(train_loader_labels)<br>
<span class="hashtag-raw">#MUSIC_data</span> = data_iter.next()<br>
<span class="hashtag-raw">#labels</span> = label_iter.next()</p>
<h1><a name="one-batch-of-random-training-data-samples-9" class="anchor" href="#one-batch-of-random-training-data-samples-9"></a>one batch of random training data samples</h1>
<p>dataiter = iter(train_loader)<br>
MUSIC_data, labels = next(dataiter)<br>
print(‘Size of MUSIC_data’, MUSIC_data.size())<br>
print(‘Size of labels’, labels.size())</p>
<h1><a name="cnn-model-10" class="anchor" href="#cnn-model-10"></a>CNN Model</h1>
<p>class ConvNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()</p>
<pre><code>    # First attempt - input sizes not correct bc they should be 3D unbatched
    # Should there be 3 input channels? One for azimuth, one for colat, one for MUSIC output? Also consider channels as number of freq bins
    
    #self.bn1 = nn.BatchNorm1d(1); # input to batch size is number of features (channels)
    self.conv1 = nn.Conv1d( batch_size , 64 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) 
    self.bn1 = nn.BatchNorm1d(batch_size);
    self.conv2 = nn.Conv1d( 64 , 64 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn2 = nn.BatchNorm1d(batch_size);
    self.pool1 = nn.MaxPool1d(2, 2) 
    
    self.conv3 = nn.Conv1d( 64 , 128 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output is 1 x 900
    self.bn3 = nn.BatchNorm1d(batch_size);
    self.conv4 = nn.Conv1d( 128 , 128 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn4 = nn.BatchNorm1d(batch_size);
    self.pool2 = nn.MaxPool1d(2, 2)
    
    self.conv5 = nn.Conv1d( 128 , 256 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output is 1 x 900
    self.bn5 = nn.BatchNorm1d(batch_size);
    self.conv6 = nn.Conv1d( 256 , 256 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn6 = nn.BatchNorm1d(batch_size);
    self.conv7 = nn.Conv1d( 256 , 256 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn7 = nn.BatchNorm1d(batch_size);
    self.pool3 = nn.MaxPool1d(3, 3)
    
    self.conv8 = nn.Conv1d( 256 , 512 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output is 1 x 900
    self.bn8 = nn.BatchNorm1d(batch_size);
    self.conv9 = nn.Conv1d( 512 , 512 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn9 = nn.BatchNorm1d(batch_size);
    self.conv10 = nn.Conv1d( 512 , 512 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn10 = nn.BatchNorm1d(batch_size);
    self.pool4 = nn.MaxPool1d( 3 , 3 ) 
    
    self.conv11 = nn.Conv1d( 512 , 512 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output is 1 x 900
    self.bn11 = nn.BatchNorm1d(batch_size);
    self.conv12 = nn.Conv1d( 512 , 512 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn12 = nn.BatchNorm1d(batch_size);
    self.conv13 = nn.Conv1d( 512 , 1 , kernel_size = 3, stride = 1, dilation = 1, groups = 1, bias = True) # output size = [(W−K+2P)/S]+1, W = flattened image length, K = kernel size, P = padding, S = Stride
    self.bn13 = nn.BatchNorm1d(batch_size);
    self.pool5 = nn.MaxPool1d(3, 3) 
    
    self.fc1 = nn.Linear( 147 , 147 , bias = False)
    self.bn14 = nn.BatchNorm1d(147);
    self.fc2 = nn.Linear( 147 , 147 , bias = False) # originally 1000, 225
    self.bn15 = nn.BatchNorm1d(147);
    self.fc3 = nn.Linear( 147, 3 )   # originally 225, 3
    
def forward(self, x):
   
   # This was an attempt to reshape the input for use with batch sizes other than 1 
   # Transpose data to correct shape -
   # Current shape = [embedded dim, batch size, length] = [1, 50, 16207]
   # New Shape = [batch size, embedded dim, length] = [50, 1, 16207]
   # swap 2nd and 3rd dims
   #print('Size of x before transpose', x.size())
   #x = torch.transpose(x, 1, 2);
   #print('Size of x before resizing into CNN', x.size())
   #x_temp = torch.empty(( 1 , batch_size , x.size( dim=1 ) ) ) # empty tensor
   #x_temp[ 0 , : , : ] = x;                                    # assign input 'x' values to empty tensor
   ##x = torch.transpose( x_temp, 0, 1 );                       # transpose 'x_temp' and assign it to input 'x' 
   #x = x_temp;                                                 # assign x_temp to input 'x'
   #x = x.to(device);                                           # push data to device
   
   # BLOCK 1
   #x = self.bn1(x)
   #print('x dtype after batchnorm:', x.dtype)
   #print('Size of x after batchnorm', x.size())
   
   # define leaky relu Activation Fcn, then apply as shown below
   m = nn.LeakyReLU(neg_slope) 
  
   x = self.conv1(x)
   x = m(x) # apply LeakyReLU       
   #x = torch.permute(x, (1, 0))
   #x = torch.permute(x, (0, 1))
   x = self.conv2(x)
   x = m(x) # apply LeakyReLU
   x = self.pool1(x)            
   
   # BLOCK 2
   x = self.conv3(x)            
   x = m(x) # apply LeakyReLU
   x = self.conv4(x)
   x = m(x) # apply LeakyReLU
   x = self.pool2(x)            
   
   # BLOCK 3
   x = self.conv5(x)           
   x = m(x) # apply LeakyReLU
   x = self.conv6(x)  
   x = m(x) # apply LeakyReLU
   x = self.conv7(x)            
   x = m(x) # apply LeakyReLU
   x = self.pool3(x)
   
   # BLOCK 4
   x = self.conv8(x)            
   x = m(x) # apply LeakyReLU
   x = self.conv9(x)
   x = m(x) # apply LeakyReLU
   x = self.conv10(x)      
   x = m(x) # apply LeakyReLU
   x = self.pool4(x)  
   
   # BLOCK 5
   x = self.conv11(x)            
   x = m(x) # apply LeakyReLU       
   x = self.conv12(x) 
   x = m(x) # apply LeakyReLU    
   x = self.conv13(x)            
   x = m(x) # apply LeakyReLU   
   x = self.pool5(x)  
   
   # BLOCK 6
   x = self.fc1(x)  
   x = m(x) # apply LeakyReLU
   x = self.fc2(x)  
   x = m(x) # apply LeakyReLU
   x = self.fc3(x)              
   
   return x
</code></pre>
<p>model = ConvNet().to(device) # Send model to GPU</p>
<p><span class="hashtag-raw">#if</span> torch.cuda.is_available(): # send model to GPU</p>
<h1><a name="modelcuda-11" class="anchor" href="#modelcuda-11"></a>model.cuda()</h1>
<p>criterion = nn.MSELoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)<br>
<span class="hashtag-raw">#optimizer</span> = torch.optim.SGD(model.parameters(), lr = learning_rate, momentum = momentum)</p>
<p>training_loss_np_array = np.zeros([ 1 , num_epochs ]);<br>
loss_np_array_index = 0;</p>
<p>n_total_steps = len(train_loader)<br>
for epoch in range(num_epochs):</p>
<pre><code>n_train_samples = len(train_loader)

training_comparison_data = torch.zeros([n_train_samples, 6]) # empty tensor for comparing  samples
training_index = 0; # iterable index variable    

running_loss = 0.0

for i, (MUSIC_data, labels) in enumerate(train_loader):
    MUSIC_data = MUSIC_data.to(device) # send to GPU
    labels = labels.to(device) # send to GPU

    # Forward pass
    outputs = model(MUSIC_data)
    #loss = criterion(outputs, labels) # use this for non-augmented losses
    loss = torch.sqrt(criterion(outputs, labels)) # changes MSEloss to euclidean distance since MSEloss is (X**2 + Y**2 + Z**2)
    
    # Output Comparison and Predicted vs Actual Sound Source distance calculation
    training_predicted = outputs ;
    training_comparison_data[training_index, :3 ] = training_predicted ; 
    training_comparison_data[training_index, 3:6 ] = labels ; 
    
    training_index += 1; 
    
    # Backward and optimize
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

    running_loss += loss.item()
#training_loss_np_array[ 0 : loss_np_array_index]  = loss.detach().numpy();
#loss_np_array_index += 1; # index to next column in training loss array
#print(f'[{epoch + 1}] loss: {running_loss:.3f}')
print(f'[{epoch + 1}] loss: {running_loss :.3f}')
</code></pre>
<p>training_comparison_data_np = training_comparison_data.detach().numpy(); <span class="hashtag-raw">#convert</span> to numpy array<br>
print(‘Finished Training’)<br>
<span class="hashtag-raw">#PATH</span> = ‘./VGG16_rev_DOE3_Trial3.pth’<br>
<span class="hashtag-raw">#torch</span>.save(model.state_dict(), PATH)</p>
<p>end = time.time()  #  End Timer<br>
print(‘Total Time to Execute Training[Seconds]’)<br>
print(end - start)  #  Print Time to Execute Program<br>
print(‘Total Time to Execute Training[minutes]’)<br>
print((end - start)/60)  #  Print Time to Execute Program<br>
print(‘Total Time to Execute Training[hours]’)<br>
print((end - start)/3600)  #  Print Time to Execute Program</p>
<p>#%%</p>
<h1><a name="evaluation-section-12" class="anchor" href="#evaluation-section-12"></a>Evaluation Section</h1>
<p>dataiter = iter(test_loader)<br>
MUSIC_data, labels = next(dataiter)</p>
<p>testing_running_loss = 0.0</p>
<p>start = time.time()  # Begin Timer<br>
with torch.no_grad():</p>
<pre><code>n_samples = len(test_loader)

comparison_data = torch.zeros([n_samples, 6]) # empty tensor for comparing  samples
index = 0; # iterable index variable

for MUSIC_data, labels in test_loader:
    MUSIC_data = MUSIC_data.to(device)
    labels = labels.to(device)
    outputs = model(MUSIC_data)
    testing_loss = torch.sqrt(criterion(outputs, labels)) # changes MSEloss to euclidean distance since MSEloss is (X**2 + Y**2 + Z**2)

    
    # Output Comparison and Predicted vs Actual Sound Source distance calculation
    testing_predicted = outputs ;
    #distance = torch.sqrt( (outputs[0] - labels[0] )**2 + (outputs[1] - labels[1] )**2 + ( outputs[2] - labels[2] )**2 );
    comparison_data[index, :3 ] = testing_predicted ; 
    comparison_data[index, 3:6 ] = labels ; 
    #comparison_data[index, 6] = distance 
    
    index += 1; 
   
    testing_running_loss += testing_loss.item()
print(' Testing loss: {testing_running_loss :.3f}')
</code></pre>
<h1><a name="convert-data-to-numpy-13" class="anchor" href="#convert-data-to-numpy-13"></a>Convert data to numpy</h1>
<p>comparison_data_np_array = comparison_data.numpy();</p>
<p>comparison_data_np_array = comparison_data.numpy() # convert tensor to np array<br>
<span class="hashtag-raw">#savetxt</span>(‘predicted_SSL_vs_real_SSL_CNN3_ngrid_16200_LR0.0005_LeakyRelu0.4_epochs150_5.15.24.csv’, comparison_data_np_array, delimiter=‘,’, header = ‘Predicted XYZ, Real XYZ, Euclidean Distance’)<br>
<span class="hashtag-raw">#savetxt</span>(‘DOE_Trial_16_Test_data_Loss_CNN3_ngrid_16200_LR0.0005_LeakyRelu0.4_epochs150_5.15.24.csv’, comparison_data_np_array, delimiter=‘,’, header = ‘Predicted XYZ, Real XYZ, Euclidean Distance’)</p>
<p>end = time.time()  #  End Timer<br>
print(‘Total Time to Execute Testing[Seconds]’)<br>
print(end - start)  #  Print Time to Execute Program<br>
print(‘Total Time to Execute Testing[minutes]’)<br>
print((end - start)/60)  #  Print Time to Execute Program<br>
print(‘Total Time to Execute Testing[hours]’)<br>
print((end - start)/3600)  #  Print Time to Execute Program</p>
<pre><code>#acc = 100.0 * n_correct2 / n_samples
#print(f'Accuracy of the loaded model: {acc} %')
</code></pre>

Brock_Brown: <p>Take a look at your model. <code>BatchNorm1d</code> expects the number of channels as input, <em>not the size of the batch</em>. Adjust your <code>BatchNorm1d</code> layers so that it takes in the appropriate number of channels. Or maybe try <a href="https://pytorch.org/docs/stable/generated/torch.nn.LazyBatchNorm1d.html" rel="noopener nofollow ugc">LazyBatchNorm1d</a> if you’re not sure about the number of channels, it will infer it from <code>input.size(1)</code>.<br>
<sub>Fun fact: There’s a Lazy version of lots of different kinds of layers, such as <a href="https://pytorch.org/docs/stable/generated/torch.nn.LazyLinear.html" rel="noopener nofollow ugc">LazyLinear</a>, that will infer the correct input size.</sub></p>

tired_grad_student: <p>Hi Brock. Thanks for your input. I adjusted the batchnorm layers to take the correct number of channels and it did not work. I got the following error:</p>
<p>RuntimeError: expand(torch.cuda.FloatTensor{[10, 3]}, size=[3]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)</p>
<p>I then tried your other suggestion of using LazyBatchNorm1d and got the same error.</p>

ptrblck: <p>Could you post the input activation shape used in this batchnorm layer as well as the code to initialize this layer, please?</p>

tired_grad_student: <p>Hi ptrblck,</p>
<p>My answer is long. Please bear with me. Also, hank you all for your help. It is greatly appreciated!</p>
<p>By activation shape, do you mean the shape coming out of the activation function that goes into the batchnorm layer? For the first convolution (conv1) with 64 output channels, kernel size = 3, and stride =1, the output of the activation function has size [64 , 16198]. As stated in my original post, each “sample” is a row of data from an excel spreadsheet of size [1 , 16200].</p>
<p>As for your second question, I am not sure I understand. I did not initialize the batchnorm layer. I followed some examples in order to figure out how to format the batchnorm layers (see below).</p>
<aside class="quote quote-modified" data-post="1" data-topic="216">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kalamaya/48/381_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/example-on-how-to-use-batch-norm/216">Example on how to use batch-norm?</a> 
  </div>
  <blockquote>
    TLDR: What exact size should I give the batch_norm layer here if I want to apply it to a CNN? output? In what format? 
I have a two-fold question: 

So far I have only <a href="http://pytorch.org/docs/nn.html?highlight=batch%20norm#torch.nn.BatchNorm2d" rel="noopener nofollow ugc">this</a> link here, that shows how to use batch-norm. My first question is, is this the proper way of usage? For example


bn1 = nn.BatchNorm2d(what_size_here_exactly?, eps=1e-05, momentum=0.1, affine=True) 
x1= bn1(nn.Conv2d(blah blah blah)) 

Is this the correct intended usage? Maybe an example of the syntax for it’s usage with a CN…
  </blockquote>
</aside>

<p>From a previous post ( <a href="https://discuss.pytorch.org/t/batchnorm1d-input-shape/91364" class="inline-onebox">BatchNorm1d - input shape</a> ), you answered another user and stated that the input size for a batchnorm1d layer should be [batch size, feature size, temp. dim]. I have two questions about this:</p>
<ol>
<li>What is temp dim and feature size?</li>
<li>Brock (the first commentor on my post) stated that the input to the batchnorm1d layer should be [# channels]. So is either [# channels] or [batch size, feature size, temp. dim] acceptable?</li>
</ol>
<p>Finally, I found what may be causing the error I previously shared. I store the output of the CNN in a tensor at line 283, as well as the labels so that I can export it later for analysis. When I increase the batch size to greater than 1, the output of the CNN stays at size [1, 3] (the output is the x, y, z location in space of a signal emitter). However, the labels that I am comparing them to has size [batch size, 3]. For batch sizes other than 1, these two do not match.</p>
<p>This may be a silly question, but if I have a batch size of 10 samples, how do I ensure that I am comparing 10 output to their appropriate labels? I assume this is taken care of by the data loaders. My last conv1d layer has output channels = 1 to ensure I have an output of size [1, 3]. Should this be changed to output channels = batch size for cases when I have batch sizes greater than 1?</p>