Which loss function is suitable?

username: learner47
cooked: <p>I am using PyTorch and am still quite new to the library.</p>
<p>I have a relation between my input and output given by <code>y = ax + b</code>, where <code>a</code> and <code>b</code> are sampled from some distribution (say Uniform), that is, they are random. I would like to train a network to predict <code>x</code> upon seeing <code>y</code> and <code>a</code>. I am employing a network, named <code>probability_network</code>, with <code>nn.Linear</code> layers. There are <code>N</code> (say 10) classes to choose from for <code>x</code>.</p>
<pre><code class="lang-auto">class ProabilityNetwork(nn.Module):
    def __init__(self):
        super(ProabilityNetwork, self).__init__()
        self.fc1 = nn.Linear(8, 76)
        self.fc2 = nn.Linear(76, 150)
        self.fc3 = nn.Linear(150, 75)
        self.fc4 = nn.Linear(75, 14)
        self.fc5 = nn.Linear(14, 10)
        self.tanh = nn.Tanh()
        self.sigmoid = nn.Sigmoid()
        self.relu = nn.ReLU()
        self.softmax = nn.Softmax(dim=1)

    def forward(self, inputs):
        return self.softmax(self.fc5(self.fc4(self.relu(self.fc3(self.relu(self.fc2(self.relu(self.fc1(inputs)))))))))

probabilty_network = ProbabilityNetwork()
</code></pre>
<p>Upon seeing <code>y</code>, the loss function should help the network predict an <code>x</code> that minimizes <code>||y-ax||^2</code>. All the quantities in <code>y = ax + b</code> are vectors (each of length 4, in this example). I have already tried the following loss function.</p>
<pre><code class="lang-auto">prob_values = probabilty_network(torch.cat([y, a], dim=0))  # shape: (batch_size, 10)
x_hat = mapping_tensor[torch.argmax(prob_values, dim=1)]  # Mapping from probability to one of 10 classes, mapping_tensor is an array of shape (10, 4)

mse_loss = nn.MSELoss()
loss = mse_loss(y, a*x_hat)
</code></pre>
<p>As an example, the <code>mapping_tensor</code> could contain binary representation of values from <code>0</code> (<code>0000</code>) to <code>9</code> (<code>1001</code>). The reason I need the binary representation of the class is that I need a vector <code>x</code> for the loss <code>||y-ax||^2</code>. In this case, <code>x</code> is a <code>4</code> length vector whereas the output of the neural network is a <code>10 </code> length vector.</p>
<p>The above setup doesn’t work. Half the values in the predicted class (written out in binary) are always wrong, implying the network is confused while training.</p>
<p>Further, this is not an unachievable problem. Solution to the above mentioned loss function exists (with errors, of course, but the errors are far lesser than 50%) but is computationally intensive. I am trying to check if the network can somehow learn to predict at lower complexity. Any help is appreciated. Thanks.</p>
<p>Furthermore, from an optimization standpoint, the loss function is the best possible solution (that I know of). So, changing the loss function would only lead to poorer results.</p>
<p>Another way to look at the problem is as follows. Suppose the network sees <code>y</code> and <code>a</code>. The network then computes <code>|y-ax|</code> for each class <code>x</code> (out of 10 possible classes), and then chooses the class with least value of the computed value. And my question is, what loss function can I use to make the network train in this fashion?</p>

username: KFrank
cooked: <p>Hi Learner!</p>
<p>I have a couple of comments and questions, in line, below:</p>
<aside class="quote no-group" data-username="learner47" data-post="1" data-topic="111861" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> learner47:</div>
<blockquote>
<p>There are <code>N</code> (say 10) classes to choose from for <code>x</code>.</p>
</blockquote>
</aside>
<p>Does this mean that you have 10 fixed values of <code>x</code> (say, 0.33, 0.52,<br>
0.57, 0.68, etc.) that you know ahead of time?  Or is <code>x</code> continuous,<br>
but you choose to break it up into 10 categories somehow?</p>
<p>If the latter, I would just predict <code>x</code> directly, and not (artificially) impose<br>
10 categories on the problem.</p>
<p>If the former, I would <em>still</em> predict <code>x</code> directly – unless you are<br>
purposely using 10 values / categories in order to set up a toy<br>
classification problem with 10 categories.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">self.softmax(self.fc5(self.fc4(self.relu(self.fc3(self.relu(self.fc2(self.relu(self.fc1(inputs)))))))))
...
mse_loss = nn.MSELoss()
</code></pre>
</blockquote>
</aside>
<p><code>Softmax</code> and <code>MSELoss</code> don’t really go together.  <code>Softmax</code> would be<br>
used for a classification (categorical) problem with a cross-entropy<br>
loss (although you would actually leave out the <code>Softmax</code> and use<br>
pytorch’s <code>CrossEntropyLoss</code> that has, in effect, <code>Softmax</code> built in).</p>
<p><code>MSELoss</code> would be used for a regression-like problem (which is what<br>
yours seems to be), but you would then predicts <code>x</code> directly, not pass<br>
it through a <code>Softmax</code>, and not categorize it.</p>
<aside class="quote no-group">
<blockquote>
<p>All the quantities in <code>y = ax + b</code> are vectors (each of length 4, in this example).</p>
</blockquote>
</aside>
<p>It looks to me as if your “vectors of length 4” make up batches of<br>
batch size <code>nBatch = 4</code>, but that you are building your batch size<br>
into your network (<code>self.fc1 = nn.Linear(8, 76)</code>).  Pytorch models<br>
work implicitly with batches, so, with two input features (<code>y</code> and <code>a</code>), you<br>
would want your network to start off with <code>Linear (2, 76)</code>.  You would<br>
then pass into your network a batch of shape<br>
<code>[nBatch, nFeatures = 2]</code>.  <code>nBatch</code> could be <code>4</code>, but it could be other<br>
values, say <code>nBatch = 100</code>, <em>without</em> changing the structure of the<br>
network.</p>
<p>In short, I would have your network predict a single value of <code>x</code> (or more<br>
precisely, a batch of single values of <code>x</code>, but pytorch does that for you<br>
automatically), predict <code>x</code> directly (no <code>Softmax</code> or other non-linearity<br>
following the final <code>Linear</code> layer).</p>
<p>So your network would return something like<br>
<code>return self.fc5 (self.relu (self.fc4 ( ... ) ) )</code>, but with<br>
<code>self.fc5 = nn.Linear (14, 1)</code>.</p>
<p>(Note that <code>self.fc5 (self.fc4 (...) )</code>, without an intervening non-linearity<br>
such as <code>self.relu()</code> acts as just a single <code>Linear</code> layer, not two.)</p>
<p>I would then use <code>MSELoss</code> as the loss criterion.</p>
<p>Best.</p>
<p>K. Frank</p>

username: learner47
cooked: <p><a class="mention" href="/u/kfrank">@KFrank</a>, thank you so much for taking time to write this answer. Really appreciate it. I would like to answer pointwise to your questions.</p>
<p>My <code>x</code>s are known before hand. It can be, for example, binary representation of 0 to 10 (each representation having length 4).</p>
<p>The reason for 10 classes is that the loss function that I use, <code>||y-ax||^2</code>, is only accurate when <code>x</code> is the representation of one of the 10 classes.</p>
<aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="111861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p><code>Softmax</code> and <code>MSELoss</code> don’t really go together. <code>Softmax</code> would be<br>
used for a classification (categorical) problem with a cross-entropy<br>
loss (although you would actually leave out the <code>Softmax</code> and use<br>
pytorch’s <code>CrossEntropyLoss</code> that has, in effect, <code>Softmax</code> built in).</p>
</blockquote>
</aside>
<p>This is exactly why I am on the hunt for another loss function that can help me train the network <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"> .</p>
<aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="111861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>It looks to me as if your “vectors of length 4” make up batches of<br>
batch size <code>nBatch = 4</code></p>
</blockquote>
</aside>
<p>The sizes of <code>y</code>, <code>a</code>, <code>x</code> and <code>b</code> are all <code>(batch_size, 4)</code>.</p>
<p>To summarize, from an optimization perspective, it can be shown that the loss function that I have mentioned in my question is optimal under this setting. Since the network has access to both <code>a</code> and <code>y</code>, I assume, the knowledge of both must be utilized for training the network. In essence, since the network has access to <code>y</code> and <code>a</code> and the all the possible classes (<code>x</code>), I believe the decision must based on the values of <code>||y-ax||^2</code>. Thank you again!</p>

username: KFrank
cooked: <p>Hi Learner!</p>
<aside class="quote no-group" data-username="learner47" data-post="3" data-topic="111861" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> learner47:</div>
<blockquote>
<p>My <code>x</code>s are known before hand. It can be, for example, binary representation of 0 to 10 (each representation having length 4).</p>
</blockquote>
</aside>
<p>I would say that I still don’t really understand your use case.</p>
<p>Your <code>x</code>s:  Would you say conceptually that they represent classes<br>
(categories) that, although they can be labelled or encoded with<br>
numbers, are not really numerical, but are something like “cat”,“dog”,<br>
“goat”, where there is not really a sense of “dog” being in between<br>
“cat” and “goat”.  Or are your <code>x</code>s fundamentally numerical, in the sense<br>
that <code>2</code> is closer to <code>1</code> than it is to <code>4</code>?</p>
<p>In the first case, you would want to understand your problem as a<br>
multi-class (presumably ten-class) classification problem, have your<br>
model output ten logits from its final <code>Linear</code> layer, and use (something<br>
like) <code>CrossEntropyLoss</code> as your loss criterion.</p>
<p>In the second case, you would want to understand your problem as<br>
a regression problem, have your final <code>Linear</code> layer output a single<br>
predicted numerical value, and use (something like) <code>MSELoss</code> as<br>
your loss criterion.</p>
<p>The underlying question is what, conceptually, do your <code>x</code>s represent?</p>
<p>You talk about <code>x</code> being a “representation of one of the 10 classes,” so<br>
I tend to think that <code>x</code>, although perhaps a number or set of numbers,<br>
encodes a non-numerical category.</p>
<aside class="quote no-group">
<blockquote>
<p>The sizes of <code>y</code>, <code>a</code>, <code>x</code> and <code>b</code> are all <code>(batch_size, 4)</code>.</p>
</blockquote>
</aside>
<p>Understood.  4 is not the batch size.</p>
<p>So it sounds to me like you have one <code>x</code> that is encoded somehow with<br>
four numbers, and four randomly chosen <code>a</code>s and <code>b</code>s that you use<br>
with the four pieces of <code>x</code> to produce four <code>y</code>s.  The four <code>y</code>s and four<br>
<code>a</code>s are together input to your network, so, indeed, one input sample<br>
consists of eight numbers.  (The four <code>b</code>s are not input to your model,<br>
so they are in some sense “hidden” or noise.)</p>
<p>That’s fine.  The question – to repeat – is what is the conceptual<br>
meaning of <code>x</code> (regardless of how it is encoded), so should you be<br>
performing a (ten-class) classification, or a single-predicted-value<br>
regression (or something else)?</p>
<p>Best.</p>
<p>K. Frank</p>

username: learner47
cooked: <p><a class="mention" href="/u/kfrank">@KFrank</a>, okay, let me tell you my exact use case.</p>
<p>I have an encoder (fixed) that maps <code>m</code> bit vector, say <code>z</code>, to <code>n</code> bit (4 in this case, n &gt; m) vector, <code>x</code>. This is then passed through a box, which causes random distortions, namely, the multiplicative distortion <code>a</code> and additive distortion <code>b</code>. <code>a</code> and <code>b</code> are random but the statistics of the distribution are known. The network that I am designing needs to able to get back the original <code>m</code> bit length vector (<code>z</code>) upon seeing <code>y</code> (<code>y=ax+b</code>) and <code>a</code>. In literature, for such a setting, it is optimal to choose an <code>x</code> (and hence <code>z</code> because there is one-one fixed mapping through the encoder) which minimizes squared norm of <code>y-ax</code>. I would like to implement this loss function to train the network to give out <code>x</code> (or the class of <code>x</code>), so that I meet the optimum performance. My implementation through the neural network may be a round about approach to achieve what I want, because I am still a novice.</p>
<p>There can be at most <code>2^m</code> classes for <code>z</code> and therefore <code>x</code>. These classes are assumed to be known to the network. Its job is then to predict the class which gives the minimum loss function value.</p>
<p>Thank you again for replying. Please let me know if you need more clarifications.</p>

username: KFrank
cooked: <p>Hi Learner!</p>
<aside class="quote no-group" data-username="learner47" data-post="5" data-topic="111861" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> learner47:</div>
<blockquote>
<p>The network that I am designing needs to able to get back the original <code>m</code> bit length vector (<code>z</code>) upon seeing <code>y</code> (<code>y=ax+b</code>) and <code>a</code>.</p>
</blockquote>
</aside>
<p>Yes, this clarifies things.  Your “ground truth” (your known “target” for<br>
training) is a bit vector, <code>z</code>, of length <code>m</code> (say, <code>m = 3</code>).</p>
<p><em>Unless</em> <code>z</code> has some additional structure / meaning (that you have not<br>
shared with us, e.g., being the binary representation of an integer), then<br>
you should understand your problem as being a so-called <em>multi-label,<br>
multi-class</em> classification problem.  This means that you have <code>m</code><br>
classes, any number of which (including none or all of them) can<br>
be “present” in any sample.  That is, any of the <code>m</code> bits can be<br>
independently on.  (“Multi-class” means you have the <code>m</code> classes;<br>
“multi-label” means that the classes are not exclusive, so that any<br>
given sample can carry a label for multiple classes at a time.)</p>
<p>The most common approach to such a problem is to have your<br>
final <code>Linear</code> layer output <code>m</code> logits (one for how strongly each of<br>
the <code>m</code> bits is predicted to be “on”).  Then use <code>BCEWithLogitsLoss</code><br>
to compare your prediction with your ground-truth <code>z</code>.  The output<br>
of your model will be of type <code>float32</code> and have shape <code>[nBatch, m]</code>.<br>
Your ground-truth <code>target</code> will have the same shape and also be of<br>
type <code>float32</code> and will have value <code>1.0</code> whenever the corresponding<br>
bit of <code>z</code> is set.</p>
<p>(As an aside, regardless of exactly how you choose to approach<br>
your problem, I would start with a shallower architecture – maybe<br>
just one or two hidden layers – and then add depth / layers if I<br>
could show that doing so yielded better performance.)</p>
<aside class="quote no-group">
<blockquote>
<p>My implementation through the neural network may be a round about approach to achieve what I want, because I am still a novice.</p>
</blockquote>
</aside>
<p>I’m assuming that this is a toy problem (given your <code>m</code> to <code>n</code> bit<br>
“encoding” followed by the introduction of additive and multiplicative<br>
noise).  If it is a toy problem, then carry on, and use it to practice<br>
building neural networks.</p>
<p>If it represents a real problem, then I suspect that it has enough<br>
structure that there would be a more optimal specialized algorithm<br>
for recovering (<em>filtering</em>) <code>z</code> from <code>y</code> and <code>a</code> (not that I would know<br>
what such an algorithm would be).</p>
<aside class="quote no-group">
<blockquote>
<p>There can be at most <code>2^m</code> classes for <code>z</code> and therefore <code>x</code>.</p>
</blockquote>
</aside>
<p>It is true that a multi-label, <code>m</code>-class problem can be understood as<br>
a single-label <code>2^m</code>-class problem.  As a general rule, however, you<br>
will be better off treating is as a multi-label problem (unless it has<br>
some special structure your network can take advantage of).</p>
<p>The problem, of course, is the <code>2^m</code> classes, as you lose information<br>
about how the classes are “correlated,” and the potentially large<br>
number of classes means you have fewer samples per class.</p>
<p>Best.</p>
<p>K. Frank</p>

username: learner47
cooked: <p>Hello <a class="mention" href="/u/kfrank">@KFrank</a>!</p>
<aside class="quote no-group" data-username="KFrank" data-post="6" data-topic="111861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p><em>Unless</em> <code>z</code> has some additional structure / meaning (that you have not<br>
shared with us, e.g., being the binary representation of an integer), then<br>
you should understand your problem as being a so-called <em>multi-label,<br>
multi-class</em> classification problem.</p>
</blockquote>
</aside>
<p><code>z</code> is the binary representation of the class with <code>m</code> bits. As an example, for <code>m=3</code> <code>z</code> runs from <code>000</code> to <code>111</code>. So <code>z</code> has a particular structure in that it is the exhaustive list of combinations of <code>1</code>s and <code>0</code>s.</p>
<aside class="quote no-group" data-username="KFrank" data-post="6" data-topic="111861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>The most common approach to such a problem is to have your<br>
final <code>Linear</code> layer output <code>m</code> logits (one for how strongly each of<br>
the <code>m</code> bits is predicted to be “on”). Then use <code>BCEWithLogitsLoss</code><br>
to compare your prediction with your ground-truth <code>z</code>. The output<br>
of your model will be of type <code>float32</code> and have shape <code>[nBatch, m]</code>.<br>
Your ground-truth <code>target</code> will have the same shape and also be of<br>
type <code>float32</code> and will have value <code>1.0</code> whenever the corresponding<br>
bit of <code>z</code> is set.</p>
</blockquote>
</aside>
<p>I have two questions here. One, for this to work should the activation function at the end of my network be <code>nn.Sigmoid</code> (I checked the documentation and found that <code>nn.Sigmoid</code> is added internally in the loss function)? And two, is this loss function optimal for the setting? What I mean is does it lead to a trained network, which can perform as well as <code>||y-ax||^2</code> being the loss function? I apologize if my questions do not make sense.</p>
<aside class="quote no-group" data-username="KFrank" data-post="6" data-topic="111861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>The problem, of course, is the <code>2^m</code> classes, as you lose information<br>
about how the classes are “correlated,” and the potentially large<br>
number of classes means you have fewer samples per class.</p>
</blockquote>
</aside>
<p>The relation <code>y=ax+b</code> can produce infinite number of training samples, theoretically. This is because the there are infinite values of <code>a</code> and <code>b</code> for a fixed distribution with fixed statistics.</p>
<p>Thank you so much Frank, for taking time to make me understand the working of the loss functions. I feel I am really close to the correct architecture for this problem. Appreciate it.</p>

username: KFrank
cooked: <p>Hi Learner!</p>
<aside class="quote no-group" data-username="learner47" data-post="7" data-topic="111861" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> learner47:</div>
<blockquote>
<p><code>z</code> is the binary representation of the class with <code>m</code> bits. As an example, for <code>m=3</code> <code>z</code> runs from <code>000</code> to <code>111</code>. So <code>z</code> has a particular structure in that it is the exhaustive list of combinations of <code>1</code>s and <code>0</code>s.</p>
</blockquote>
</aside>
<p>Then (barring other noteworthy details) I would understand this as a<br>
multi-label, three-class classification problem.</p>
<p>To illustrate what I meant by “additional structure”, consider the mapping<br>
between a set of three bits and the integers 0, …, 7, where, e.g., “011” is<br>
the binary representation of the integer 3.</p>
<p>Let’s say your ground-truth <code>z</code> is “011”, while your predicted <code>z</code> is “100”.<br>
Would you consider this a maximally bad prediction because all three<br>
of the bits were predicted incorrectly?  Or would you consider it a pretty<br>
good (but not perfect) predictions because the ground-truth <code>z</code> of “011”<br>
corresponds to the integer 3, while the predicted <code>z</code>, “100”, corresponds<br>
to 4, which is only off by 1?</p>
<p>I’m presuming the former, so viewing this as a multi-label classification<br>
problem (rather than, say, a regression) is appropriate.</p>
<aside class="quote no-group">
<blockquote>
<p>One, for this to work should the activation function at the end of my network be <code>nn.Sigmoid</code></p>
</blockquote>
</aside>
<p><code>BCEWithLogitsLoss</code> is the “go-to” loss function for the multi-label problem.<br>
(There are other reasonable loss functions, but I would use this unless I<br>
had good reason to think that another loss function would be better, and,<br>
even then, I would want to demonstrate that the alternative loss function<br>
did, if fact, perform better.)</p>
<p>If you use <code>BCEWithLogitsLoss</code>, then, no, you do not want a <code>Sigmoid</code><br>
for the output of your network.</p>
<aside class="quote no-group">
<blockquote>
<p>(I checked the documentation and found that <code>nn.Sigmoid</code> is added internally in the loss function)?</p>
</blockquote>
</aside>
<p>You do not want the <code>Sigmoid</code> because, as you note, the <code>Sigmoid</code> is, in<br>
effect, included in <code>BCEWithLogitsLoss</code>.  (The numerically-less-stable<br>
<code>BCELoss</code> does not have the <code>Sigmoid</code> built in.)</p>
<aside class="quote no-group">
<blockquote>
<p>And two, is this loss function optimal for the setting?</p>
</blockquote>
</aside>
<p>If my understanding of your problem as being a multi-label, multi-class<br>
classification problem is correct, then, yes, <code>BCEWithLogitsLoss</code> is<br>
reasonably likely to be your “optimal” loss function.</p>
<aside class="quote no-group">
<blockquote>
<p>What I mean is does it lead to a trained network, which can perform as well as <code>||y-ax||^2</code> being the loss function?</p>
</blockquote>
</aside>
<p>I’m certainly not able to write a mathematical proof showing that one loss<br>
function or another will be better – the whole business of training realistic<br>
neural networks is sufficiently complicated that such choices  are generally<br>
made based on rules of thumb and empirical results.</p>
<p>You can certainly try using both loss functions to train your network and<br>
see which works better.  But don’t <em>not</em> try <code>BCEWithLogitsLoss</code>, at least<br>
as a baseline.</p>
<aside class="quote no-group">
<blockquote>
<p>The relation <code>y=ax+b</code> can produce infinite number of training samples, theoretically.</p>
</blockquote>
</aside>
<p>The beauty of this is that you don’t have to worry about overfitting – just<br>
keep training until your model converges to an acceptable accuracy (or<br>
reaches a plateau that could represent the best it can do).</p>
<p>And again, you might want to experiment with the architecture of your<br>
network, probably including some shallower (fewer layers) networks.</p>
<p>Best.</p>
<p>K. Frank</p>

username: learner47
cooked: <p><a class="mention" href="/u/kfrank">@KFrank</a>, thank you for all your comments. I certainly learnt a lot from the discussion. I have been able to make it work, but the performance could be improved upon. I think this would come in the form of hyperparameter tuning.</p>
<p>I really appreciate your effort in all this. It was not just a mere reply, rather a detailed and wonderful explanation of what my problem should be looked as. I am sure others who run into such problems would find your replies helpful. That said, I am able to do what I set out to do, so I am going to mark this thread as solved.</p>
<p>Thank you so much!</p>