Making Predictions for a highly Overlapping Data

Harsh_Choudhary: <p>I am solving a problem where I have 4 distinct outputs. I need to train a Neural Net as a <strong>regressor</strong> so as to predict the correct value corresponding to my input features, The Problem however is that the features I am using are highly overlapping. Till now, I have tried training a Neural Net over the whole dataset but that didn’t give any promising results. I even tried using K-means first to cluster the data into 4 distinct clusters and then training a different Neural Net on each of the cluster for regression. That didn’t seem to work as well the way I wanted it to. As a brief insight to my problem: suppose the 4 targets are 0, 1, 2, 4. and I need to train a Neural Net as a regressor as I want to generalize the model as there can be a lot of values in between as well in the future. I am attaching some images which are scatter plots of my input features. The main problem I face is that my neural Net never performs good on all 4 of the classes. If the predictions for the extreme classes i.e. 0 and 4 are consistently good, the predictions for the middle classes are bad and sometimes the other way around.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc.png" data-download-href="https://discuss.pytorch.org/uploads/default/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc" title="torch0_4"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/a/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc_2_690x439.png" alt="torch0_4" data-base62-sha1="hsRfiGsJjHEkqKTmwq0Bv9HErEM" width="690" height="439" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/a/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc_2_690x439.png, https://discuss.pytorch.org/uploads/default/original/3X/7/a/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/a/7a681514bd12f63dcbdd14723d9ec8c8b4fb94dc.png 2x" data-dominant-color="D8BEE4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch0_4</span><span class="informations">888×566 211 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/701d407b375e8a07cb2e352ccd836de033b3c11e.png" data-download-href="https://discuss.pytorch.org/uploads/default/701d407b375e8a07cb2e352ccd836de033b3c11e" title="torch1_2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/701d407b375e8a07cb2e352ccd836de033b3c11e_2_690x435.png" alt="torch1_2" data-base62-sha1="fZO9d2GfVaXVXuz2MDQJIbUgyHc" width="690" height="435" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/701d407b375e8a07cb2e352ccd836de033b3c11e_2_690x435.png, https://discuss.pytorch.org/uploads/default/original/3X/7/0/701d407b375e8a07cb2e352ccd836de033b3c11e.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/0/701d407b375e8a07cb2e352ccd836de033b3c11e.png 2x" data-dominant-color="CAE2BE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch1_2</span><span class="informations">879×555 249 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, When I tried using the clustering first to separate the data, The 2 clusters were good but the other 2 clusters have a lot of misclassifications so my neural nets couldn’t make a good predictions, also the data to each neural net has been reduced to a fourth the data that the single neural net was getting over the whole dataset. I am also attaching the image of the clustering that I made. where each color represents item types (0,1,2,4) respectively<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c.png" data-download-href="https://discuss.pytorch.org/uploads/default/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c" title="torch_cluster"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c_2_690x467.png" alt="torch_cluster" data-base62-sha1="aV9oQ9aFjWibNMXB8X1sFlPwF3K" width="690" height="467" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c_2_690x467.png, https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c8b9c24c68721c4b0c5f52642d1a6722ccd935c.png 2x" data-dominant-color="E3DEDC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch_cluster</span><span class="informations">882×597 20.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>