Why does the .clamp function exist?

username: pinocchio
cooked: <p>I was looking at the example:</p>
<p><a href="http://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_autograd.html" class="onebox" target="_blank" rel="nofollow noopener">http://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_autograd.html</a></p>
<p>and has the line:</p>
<pre><code># Forward pass: compute predicted y using operations on Variables; these
# are exactly the same operations we used to compute the forward pass using
# Tensors, but we do not need to keep references to intermediate values since
# we are not implementing the backward pass by hand.
y_pred = x.mm(w1).clamp(min=0).mm(w2)
</code></pre>
<p>I read the documentation for clamp:</p>
<pre><code>torch.clamp(input, min, max, out=None) → Tensor
Clamp all elements in input into the range [min, max] and return a resulting Tensor.
</code></pre>
<p><a href="http://pytorch.org/docs/master/torch.html#torch.clamp" class="onebox" target="_blank" rel="nofollow noopener">http://pytorch.org/docs/master/torch.html#torch.clamp</a></p>
<p>but it didn’t make sense to me. Why do we need to do such a weird thing? Tensorflow doesn’t “clamp” anything during matrix multiplication why does pytorch?</p>

username: ruotianluo
cooked: <p>clamp(min=0) is exactly ReLU.</p>

username: pinocchio
cooked: <p>Oh I see. Thanks. So there is no relu function? we just use clamp?</p>
<p>I was very confused cuz I was just trying to implement linear regression and there was a random clamp command in most examples I’ve seen which was confusing to me.</p>

username: moskomule
cooked: <p>Of course there is. Relu in PyTorch is <code>torch.nn.functional.relu</code>. So</p>
<pre><code class="lang-auto">import torch.nn.functional as F
y_pred = F.relu(x.mm(w1)).mm(w2)
</code></pre>
<p>should work in same way.</p>

username: dgriff
cooked: <p>nice…   <img src="//discuss.pytorch.org/images/emoji/apple/+1.png?v=5" title=":+1:" class="emoji" alt=":+1:"></p>

username: qbx2
cooked: <p>Someone would use clamp to clip the rewards in reinforcement learning.</p>

username: santi-pdp
cooked: <p>Or to clip the weight values in a Wasserstein GAN</p>

username: ruotianluo
cooked: <p>There are relu, and relu6. However what if you want some thing like relu7?</p>

username: uapatira
cooked: <p>I think I have a pretty good grasp of the situations which might lead one to use ReLU6; but I’m kinda interested in it’s history. Got any good links?</p>

username: Jiawei_Zhuang
cooked: <p>That’s exactly the same as <a href="https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.clip.html" rel="nofollow noopener">numpy.clip</a>.</p>

username: Hyungjoo_Cho
cooked: <p>import torch.nn.functional as F<br>
y_pred = F.threshold(<a href="http://x.mm" rel="nofollow noopener">x.mm</a>(w1), 0, 7).mm(w2)</p>
<p>is it right?</p>

username: ruotianluo
cooked: <p>I don’t think F.threshold can accept min and max.</p>
<p><a href="http://x.mm" rel="nofollow noopener">x.mm</a>(w1).clamp(0,7).mm(w2)</p>

username: Hyungjoo_Cho
cooked: <p>import torch.nn.functional as F<br>
y_pred = F.hardtanh(<a href="http://x.mm" rel="nofollow noopener">x.mm</a>(w1), 0, 7).mm(w2)</p>
<p>I found my mistake. Not threshold.</p>
<p>Just use hardtanh!!</p>

username: Tf_Hou
cooked: <p>In detection, i just found the code use clamp to ensure right poit minus lift point more than 0.</p>

username: Arcadianlee
cooked: <p>Could also use clamp() to constrain the range of your update tensors in SGD optimization</p>