Batch normalisation in 1D CNN architecture

Fathima: <p>I am performing a binary classification task with ECG signals. I didn’t normalise in the beginning because I read some papers that say pre and post-processing are not required for the deep learning model and batch normalization should be done in the CNN architecture and that should be sufficient.<br>
I created the architecture and trained the model but I got a zig-zag curve for train and valid loss.</p>
<p>If I want to normalize in the beginning, how can I normalize? I tried to add normalization but it doesn’t work. Here is the code for beat extraction from signal</p>
<pre><code class="lang-auto">def extract_beat(signal, win_pos, qrs_positions, win_msec=40, fs=360, start_beat=36, end_beat=108):

    #extract signal 
    signal = np.array(signal)
   
    #print(signal.shape)
    
    #segment the beats
    start = int(max(win_pos-start_beat,0))
    stop  = start+start_beat+end_beat+1 
    
    beat =  signal[start:stop]
  
    
    #print(" =========== BEAT = ",len(beat))
     # do the normalisation
     #beat = transforms.Normalize(beat)
    
    #compute the nearest neighbor of win_pos among qrs_positions
    tolerance = (fs*win_msec)//1000  #samples at a distance &lt;tolerance are matched
    nbr = NearestNeighbors(n_neighbors=1).fit(qrs_positions)
    distances, indices = nbr.kneighbors(np.array([[win_pos]]).reshape(-1,1))
    
    #label
    if distances[0][0] &lt;= tolerance:
        label =  1
        
    else: 
        label =  0
    #print(distances[0],tolerance,label)
        
    return beat, label

</code></pre>
<p>This is my CNN architecture, how can I do the batch normalisation here ?</p>
<pre><code class="lang-auto">class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        
        
        self.conv1 = nn.Sequential(
            nn.Conv1d(1, 32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU())
        
        self.conv2 = nn.Sequential(
            nn.Conv1d(32,32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU(),
            nn.MaxPool1d(2,stride=3))
          
        self.conv3 = nn.Sequential(
            nn.Conv1d(32, 32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU())
        
        #fully connected layers
        self.fc1 = nn.Linear(32*47,32)
        #self.fc2 = nn.Linear(32,1)
        self.fc2 = nn.Linear(32,1)
        self.activation = nn.Softmax()
    
    
    def forward(self, x):
        # input x : 
        #expected conv1d input = minibatch_size * num_channel * width
        batch_size=x.size(0)
        y = self.conv1(x.view(batch_size,1,-1))
        y = self.conv2(y)
        y = self.conv3(y)
        
        #print(y.size())
        batch_size= y.size(0)
        y = y.flatten(start_dim=1)
        #print(y.size())
        y = self.fc1(y.view(y.size(0), -1))
        #y = self.fc1(y.view(batch_size,1,-1))
        y = self.fc2(y.view(batch_size,1,-1))

        return y

</code></pre>

ptrblck: <pre data-code-wrap="python"><code class="lang-python">beat = transforms.Normalize(beat)
</code></pre>
<p>This code won’t work, as <code>transforms.Normalize</code> expects the <code>mean</code> and <code>std</code> input arguments as described in the <a href="https://pytorch.org/vision/stable/generated/torchvision.transforms.Normalize.html#torchvision.transforms.Normalize">docs</a>. After creating the transformation object, you could apply it:</p>
<pre data-code-wrap="python"><code class="lang-python">transform = transforms.Normalize(mean=..., std=...)
out = transform(input)
</code></pre>
<aside class="quote no-group" data-username="Fathima" data-post="1" data-topic="153885">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/f6c823/48.png" class="avatar"> Fathima:</div>
<blockquote>
<p>This is my CNN architecture, how can I do the batch normalisation here ?</p>
</blockquote>
</aside>
<p>Just add it to the layers via <code>nn.BatchNorm1d(...)</code>.</p>

Fathima: <p>Do I have to normalize in the beginning and do a batch normalisation too? Or in deep learning can you normalisé the data in architecture?</p>

Fathima: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="153885">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<pre><code class="lang-auto">transform = transforms.Normalize(mean=..., std=...)
out = transform(input)
</code></pre>
</blockquote>
</aside>
<p>The beat is a NumPy array. what’s the best way to normalise?</p>

harpone: <p>just use torch.tensors? I.e. <code>input=torch.tensor(input)</code></p>

iliasslasri: <p>Check <a href="https://discuss.pytorch.org/t/understanding-convolution-1d-output-and-input/30764">understanding-convolution-1d-output-and-input</a> to understand more.</p>