Matching Augmentation with Tensorflow

ayushm-agrawal: <p>I noticed that Tensorflow’s random_brightness and random_contrast function calculation is different from PyTorch. While tensorflow adds the delta value to the pixels, PyTorch multiplies the value.</p>
<p>I am trying to replicate a paper that has provided the code in Tensorflow. How do I replicate the transformation for random brightness and contrast based on the code provided in tensorflow?</p>
<pre><code class="lang-auto">reshaped_image = tf.image.random_brightness(reshaped_image,
                                               max_delta=63)
  # Randomly changing contrast of the image
  reshaped_image = tf.image.random_contrast(reshaped_image,
                                             lower=0.2, upper=1.8)
</code></pre>

ptrblck: <p><code>torchvision.transforms</code> uses either <code>PIL</code> as the backend for <code>PIL.Image</code>s or the equivalent tensor manipulations (which should yield the same results as <code>PIL</code>).<br>
Could you explain which delta is wrongly multiplied in which function?</p>

ayushm-agrawal: <p>Hello Peter,<br>
Thank you for your reply. The function is not wrongly calculated but just different. Please check this adjust_brightness function in Tensorflow where the value ‘delta’ is added to all the pixels of the image.<br>
<a href="https://github.com/tensorflow/tensorflow/blob/b36436b087bd8e8701ef51718179037cccdfc26e/tensorflow/python/ops/image_ops_impl.py#L1833" rel="nofollow noopener">Tensorflow Adjust_brightness</a></p>
<p>In PyTorch, brightness in colorJitter is calculated as a factor value. So a delta value of 52 in tensorflow for random_brightness would make the image completely white in PyTorch.</p>
<p>The main reason I ask this question is how do I replicate a paper’s augmentation that has source code in Tensorflow?</p>
<p>Thank you,<br>
Ayush</p>

ayushm-agrawal: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, Also, the tensorflow method for per_image_normalization. How do we achieve that in PyTorch?</p>

ptrblck: <p>If TF uses an absolute value for the brightness and PyTorch/PIL a relative one, you could try to map the TF value by dividing it with <code>255</code>.</p>
<p>However, I’m a bit confused by the TF description, which claims:</p>
<blockquote>
<p>The value <code>delta</code> is added to all components of the tensor <code>image</code>. <code>image</code> is<br>
converted to <code>float</code> and scaled appropriately if it is in fixed-point<br>
representation, and <code>delta</code> is converted to the same data type. For regular<br>
images, <code>delta</code> should be in the range <code>[0,1)</code>, as it is added to the image in<br>
floating point representation, where pixel values are in the <code>[0,1)</code> range.</p>
</blockquote>
<p>Also the example uses a <code>delta</code> value of <code>0.1</code>, so I’m unsure where the <code>52</code> comes from.</p>

ayushm-agrawal: <p>This was a bit confusing to us as well. We were using the random_brightness function in TF and the closest thing to that was the ColorJitter in PyTorch. But it seems like they just derive a random uniform distribution from the max_delta value that you provide and generate a sample delta value. This value is then added to each pixel of the image. Here is the implementation that my teammate created that we use as a lambda function in our transforms.</p>
<pre><code class="lang-auto">def random_brightness(image, max_delta):
    """
    This function creates a custom random brightness
    transform which is used for data augmentation.
    params:
        - image (torch Tensor): Image Tensor that needs to be standardized.
        - max_delta (int) : Number used to generate random brightness
    
    returns:
        - adjusted_image (torch Tensor): Image Tensor post standardization.
    """

    # check if max delta is greater than 0
    if max_delta &lt; 0:
        raise ValueError('Max Delta should be non-negative!')

    # Generates uniformly distributed random samples from the half-open interval [low, high)
    delta = Uniform(-max_delta, max_delta).sample()

    # Add the delta to each pixel of the image
    adjusted_image = torch.add(image, delta)

    return adjusted_image
</code></pre>