Constrain MLP to predict values between 0 and 1

ays: <p>how do i force my MLP model to predict values between 0 and 1(I want to do this because my target value is a continuous value between 0 and 1)</p>
<p>NB: This is a regression problem and my loss is MSE</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/ays">@ays</a>,</p>
<p>You could pass your output via a <code>nn.Sigmoid</code> function that’ll force the range to be [0,1)</p>

KFrank: <p>Hi Ays and Alpha!</p>
<aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="2" data-topic="185199" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>You could pass your output via a <code>nn.Sigmoid</code> function that’ll force the range to be [0,1)</p>
</blockquote>
</aside>
<p>An alternative approach that may or may not train more effectively with a<br>
mean-squared-error loss would be to use the output of your model (that I<br>
assume ranges from <code>-inf</code> to <code>inf</code>) unchanged – that is, not pass is through<br>
a <code>Sigmoid</code> – but pass your target value through <code>torch.logit()</code>, which will<br>
transform your target’s range from <code>[0.0, 1.0]</code> to <code>[-inf, inf]</code>.</p>
<p><code>logit()</code> is the inverse function of <code>sigmoid()</code>.  So, in a sense, these two<br>
schemes are comparing the same quantities, but doing so with a transformed<br>
loss function.</p>
<p>(Note that if your target value represents a probability, then treating this<br>
as a regression problem with a mean-squared-error loss, while logically<br>
consistent, will probably not train as well as treating it as a classification<br>
problem with binary cross entropy as the loss.)</p>
<p>Best.</p>
<p>K. Frank</p>

ays: <p>Thanks <a class="mention" href="/u/kfrank">@KFrank</a> and <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a>  for the reply. Isn’t it just for classification task that I pass it through a sigmoid (or it doesnt matter and I can use it in this case too)? Also my target ranges between 0 and 1 (because I’m trying to predict a dice score which ranges between 0 and 1), and there is no class so it’s a continues value.</p>

vdw: <p>Probably a naive opinion: I wouldn’t try to enforce anything.</p>
<p>You clearly say it’s a regression task. This also means, I assume, that the ground truth values in your training dataset are between 0 and 1. So if the model is trained well, and unseen data samples stems from a similar distribution as the training data, the prediction will be within 0 and 1.</p>
<p>In contrast, if the prediction is indeed outside this range, this might mean that the unseen sample “looks” very different from the training data. And in this case forcing it to be within 0 and 1 doesn’t feel right. Of course, in practice, you can always map negative values to 0 and larger values to 1. But enforcing the boundaries during training seems not a good idea.</p>
<p>After all, this is the common situation that a trained regressor is only valid within the range of the training data. For example, if you train a regressor predicting the resale price of houses based on <span class="hashtag-raw">#rooms</span>, <span class="hashtag-raw">#floors</span>, age, area_size, etc., you can predict the prices for hypothetical houses with 0 rooms or a million rooms. The regressor will give you some prediction, but it’s arguably not very meaningful as this unseen sample is so different compared to the training data. In short, while you can extrapolate with the regressor, the more you extrapolate the less meaningful the prediction might become.</p>
<p>Despite this, I’ve never seen approaches to enforce any lower and upper bound during training. That being said, you could try tree-based models (Decision Trees, Random Forests, Gradient-Boosted Trees, etc.) as those models cannot extrapolate (compared to, say, Logistic Regression). So with tree-based models, your prediction will always be between the lowest and highest value in your training data.</p>

ays: <p>That makes sense. Thanks!</p>