When I use the optimizer, there is no gradient due to the use of unit8, but I have to use unit8

username: wang1528186571
cooked: <p>def apply_relighting_tensor(tensor, alpha, beta):<br>
tensor = tensor * 255.0<br>
new_tensor = tensor.to(torch.uint8)<br>
new_tensor = new_tensor * alpha + beta / 255.0<br>
new_tensor = torch.abs(new_tensor)<br>
new_tensor = new_tensor.to(torch.float32)<br>
new_tensor = new_tensor / 255.0<br>
return new_tensor</p>
<p>I know that this value is normal, but it has no gradient<br>
But when I use floating point numbers, the effect of the generated image is not what I want at all. I don’t know what to do.</p>
<p>def apply_relighting_tensor(tensor, alpha, beta):<br>
tensor_float = tensor * 255.0<br>
new_tensor = tensor_float * alpha + beta</p>
<pre><code>new_tensor = torch.clamp(new_tensor, 0, 255)
new_tensor = new_tensor / 255.0
</code></pre>

username: soulitzer
cooked: <aside class="quote no-group" data-username="wang1528186571" data-post="1" data-topic="214893">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wang1528186571/48/74860_2.png" class="avatar"> wang1528186571:</div>
<blockquote>
<p>But when I use floating point numbers, the effect of the generated image is not what I want at all.</p>
</blockquote>
</aside>
<p>Since you weren’t able to train the model using the uint8 approach, could it also be possible that the cause of the issue with generated image quality is the result of some other part of your model/training process rather than this particular function.</p>

username: wang1528186571
cooked: <p>I tried what you said, but in fact the final positioning is at the place I mentioned before. If this problem is solved, the code will also be solved.</p>

username: wang1528186571
cooked: <p>Theoretically, I can use CV2 to implement it, but there will be no gradient in training. My process is in the training graph, so I have to use tensor. But using the actual tensor is not enough and must be converted to unit8, which gives me a headache.</p>

username: Sayed_Nadim
cooked: <p>The main issue is that you are using <code>torch.clamp(new_tensor, 0, 255)</code> and this is probably why you are getting very different outputs for int vs float operations. One workaround is using rounding operations.</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import numpy as np

 # your int method
def apply_relighting_tensor_int(tensor, alpha, beta):
    tensor = tensor * 255.0
    new_tensor = tensor.to(torch.uint8)
    new_tensor = new_tensor * alpha + beta / 255.
    new_tensor = torch.abs(new_tensor)
    new_tensor = new_tensor.to(torch.float32)
    new_tensor = new_tensor / 255.0
    return new_tensor

# new float method
def apply_relighting_tensor_float(tensor, alpha, beta):
    tensor_float = tensor * 255.0
    new_tensor = torch.round(tensor_float * alpha + beta, decimals=1)
    new_tensor = new_tensor / 255.0
    return torch.round(new_tensor, decimals=1)


if __name__ == "__main__":
    image_tensor = torch.rand(1, 3, 64, 64)
    # random alpha and beta
    alpha = 2.5
    beta = 0.1

    # apply functions
    int_tensor = apply_relighting_tensor_int(image_tensor, alpha=alpha, beta=beta)
    float_tensor = apply_relighting_tensor_float(image_tensor, alpha=alpha, beta=beta)

    # check stats
    print(f"INT VS FLOAT MAX: {int_tensor.max()} vs {float_tensor.max()}")
    print(f"INT VS FLOAT MIN: {int_tensor.min()} vs {float_tensor.min()}")
    print(f"INT VS FLOAT MEAN: {int_tensor.mean()} vs {float_tensor.mean()}")
    print(f"INT VS FLOAT STD: {int_tensor.std()} vs {float_tensor.std()}")

</code></pre>
<p>Though it’s not exactly giving the <code>int8</code> outputs, the stats show that the float method is almost same as the int method.</p>
<pre data-code-wrap="python"><code class="lang-python">INT VS FLOAT MAX: 2.4901974201202393 vs 2.5
INT VS FLOAT MIN: 1.537870048196055e-06 vs 0.0
INT VS FLOAT MEAN: 1.2464064359664917 vs 1.2522379159927368
INT VS FLOAT STD: 0.7196686863899231 vs 0.7208324670791626
</code></pre>
<p>This <code>apply_relighting_tensor_float</code> method shouldn’t have any issues with the gradient flow.</p>

username: wang1528186571
cooked: <p>Thank you very much for your reply. The round operation may affect the gradient.<br>
Since the round operation is not differentiable or has zero gradient at most points, if rounding is used in the neural network or optimization process:</p>
<p>Gradient blocking: This operation causes the gradient on the relevant path to be zero, thereby blocking the gradient flow. This may cause the model to be unable to learn effectively.<br>
Optimization difficulties: Using non-differentiable operations such as round makes the optimization process more difficult because the gradient information is lost or inaccurate.</p>
<p>The image after apply will be sent to the detector for detection, so I can’t let it break the gradient. I’ll try it first. If it still doesn’t work, I’ll come back and tell you.</p>

username: wang1528186571
cooked: <aside class="quote no-group" data-username="Sayed_Nadim" data-post="5" data-topic="214893">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sayed_nadim/48/34051_2.png" class="avatar"> Sayed_Nadim:</div>
<blockquote>
<pre><code class="lang-auto">def apply_relighting_tensor_float(tensor, alpha, beta):
    tensor_float = tensor * 255.0
    new_tensor = torch.round(tensor_float * alpha + beta, decimals=1)
    new_tensor = new_tensor / 255.0
    return torch.round(new_tensor, decimals=1)
</code></pre>
</blockquote>
</aside>
<p>This is done using the apply_relighting_tensor_float method, which currently does not display properly.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/7/77bd49c8d5cd757a9cc5044a0d18064eef4ed322.png" data-download-href="https://discuss.pytorch.org/uploads/default/77bd49c8d5cd757a9cc5044a0d18064eef4ed322" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77bd49c8d5cd757a9cc5044a0d18064eef4ed322_2_690x340.png" alt="image" data-base62-sha1="h5gmLepyUwtxYjaIIOyslitJIgG" width="690" height="340" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77bd49c8d5cd757a9cc5044a0d18064eef4ed322_2_690x340.png, https://discuss.pytorch.org/uploads/default/original/3X/7/7/77bd49c8d5cd757a9cc5044a0d18064eef4ed322.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/7/77bd49c8d5cd757a9cc5044a0d18064eef4ed322.png 2x" data-dominant-color="433D3B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">695×343 72.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The following is done with unit8, it shows correctly, I tried many ways but still can’t do it</p>

username: wang1528186571
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/0/8042801a4ab6a7b28fa87f3912f0655c38d15dfd.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/8042801a4ab6a7b28fa87f3912f0655c38d15dfd" title="DeVEG3HoRc"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/0/8042801a4ab6a7b28fa87f3912f0655c38d15dfd.jpeg" alt="DeVEG3HoRc" data-base62-sha1="iiDAKdPOWX7Pp0NOHp4iii8yiJv" width="592" height="289"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">DeVEG3HoRc</span><span class="informations">592×289 54.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: wang1528186571
cooked: <p>Please help me! because this is significant!</p>