Advice for machine learning task with chemical data

CopyOfA: <p>I am relatively new to CNNs, and I’m working on a machine learning classification problem with chemical data. I’m looking for advice on 1) how to structure the input data, and 2) architecture for the neural network classifier.</p>
<p>For this task I have raman spectra to use as input. I’ve already removed the baseline and normalized the spectra to lie between 0 and 1. Next I want to input them into the model. I have been operating under the assumption that the data should be input into the model in the shape [batch, 1, length_of_spectra], that is if each spectrum is length 1000, and I have a batch size of 32, then the input data shape would be [32, 1, 1000]. Is this the correct shape? I’m a little confused about the difference between the number of (input) channels and the signal length. Should I consider each wavenumber to be a channel or is the entire spectrum a single channel? Effectively I want the neural net operators to operate on each spectrum independently of the other spectra in a particular batch. That is, if I do a 1-D convolution layer, I want the kernel to operate only on a single spectrum at a time (not convolve multiple spectra together).</p>
<p>I’ve so far tried two different structures, all relatively simple and taken from literature on using CNNs with raman spectroscopy data. Structure 1 (taken from <a href="https://pubs.rsc.org/en/content/articlepdf/2019/an/c8an02212g" class="inline-onebox" rel="noopener nofollow ugc">Deep learning-based component identification for the Raman spectra of mixtures - Analyst (RSC Publishing)</a>):</p>
<pre><code class="lang-auto">layer1 = Dropout(p=0.5)(
   Maxpool1d(kernel_size=2, stride=2, padding=1)(
      leaky_relu(
         Conv1d(in_channels=1, out_channels=32, kernel_size=5, stride=2, padding=2)
      )
   )
)

layer2 = Dropout(p=0.5)(
   Maxpool1d(kernel_size=2, stride=2, padding=1)(
      leaky_relu(
         Conv1d(in_channels=32, out_channels=64, kernel_size=5, stride=2, padding=2)
      )
   )
)

layer3 = Linear(8128, 1024)(
   torch.flatten(start_dim=1)
)

layer4 = Linear(1024, 1)
</code></pre>
<p>When I’m training this, I do get <code>nan</code>  loss after some time, and I’m not entirely sure why. If you have any suggestions on that, I’d very much appreciate it. Structure 2 (taken from <a href="https://www.sciencedirect.com/science/article/pii/S1386142521003085" class="inline-onebox" rel="noopener nofollow ugc">Classifying breast cancer tissue by Raman spectroscopy with one-dimensional convolutional neural network - ScienceDirect</a>):</p>
<pre><code class="lang-auto">layer1 =  maxpool1d(kernel_size=2, stride=2)(
   leaky_relu(
      batchnorm1d(num_features=5)(
         conv1d(in_channels=1, out_channels=5, kernel_size=10, stride=2)
      )
   )
)

layer2 = dropout(p=0.5)(
   leaky_relu(
      batchnorm1d(num_features=5)(
         Linear(in_features=2125, out_features=5)(torch.flatten(start_dim=1))
      )
   )
)

layer3 = Linear(in_features=5, out_features=1)
</code></pre>
<p>This model also reached <code>nan</code> loss after some time, and I’m not sure why. Anyhow, this is what I’ve done so far, and I appreciate any advice on either of these models, but especially on the data input format/shape. Thanks!</p>

Mwni: <p><code>[batch, 1, length_of_spectra]</code> is the correct shape for 1D convolution. Normalization and regularization is the proper way too.<br>
Regarding your loss becoming <code>NaN</code> after a while, try applying this function to your model before you update weights</p>
<pre><code class="lang-auto">def sanitize_grads(module):
	for param in module.parameters():
		if param.grad is not None:
			torch.nan_to_num(param.grad, nan=0, posinf=1e5, neginf=-1e5, out=param.grad)
</code></pre>