Image Effects for Android using OpenCV: Image Blending
By: Ahmed Gad
Tags: Opencv, Java, Heartbeat, Android App Development, Image Processing

What is the Alpha Channel?

When an image is visualized, the values assigned to its color space channels are what control the appearance of its pixels. If each pixel is represented as an 8-bit channel, then the range of the pixel values is from 0 to 255, where 0 means no color and 255 means full color.

For a color space such as RGB, there are 3 channels—red, green, and blue. If a color is to be assigned to a pixel using RGB, then all you need to do is just specify the values of the 3 channels. For example, if the pure red color is to be represented using the RGB color space, then the red channel will be assigned the highest possible value, and the other 2 channels the lowest possible values. Thus, the values for the red, green, and blue channels will be 255, 0, and 0 respectively.

Besides specifying the values for each channel of the color space, there’s another factor that controls how the color appears on the screen—pixel transparency, which allows you to control whether the pixel is to be transparent or not. We do this using a new channel added to the color space named alpha.

For making the RGB colors transparent, a fourth channel is added, representing the alpha channel. This channel has the same size (rows and columns) as all other channels. Its values are assigned the same way that values for the other channels are assigned. That is, if each pixel is represented as 8-bit, then the values for the alpha channel range from 0 to 255.

We know that the 0 value for any of the RGB channels means no color (black) and 255 means full color (white). For the alpha channel, what do values 0 and 255 represent? These values help calculate the fraction of pixel color that will appear on the screen, according to the equation below. This fraction is then multiplied by the pixel value to get the result.

Fraction = Alpha/255

When the alpha is 0, the fraction is also 0.0, and thus the old pixel value is multiplied by 0 to return 0. This means nothing from the pixel is taken, and thus the result is 100% transparent.

When the alpha is 255, the fraction will be 1.0, and thus the full value of the pixel is multiplied by 1.0 to return the same pixel value unchanged. This means no transparency at all.

As a summary, the 0 value for the alpha channel means that the pixel is 100% transparent. The alpha value 255 means the full pixel value is used unchanged and thus the result is not transparent at all. The pixel is more transparent when the alpha value approaches 0.

According to the above discussion, we know how to make an image transparent. This leaves us with a question. How is this useful? Let’s discuss one benefit—alpha blending or compositing.

Assume that there is no alpha channel at all and we need to add the color of the 2 pixels listed below:

Pixel1 = (R1, G1, B1) = (200, 150, 100) Pixel2 = (R2, G2, B2) = (100, 250, 220)

The result of adding Pixel1 to Pixel2 is as follows:

Pixel3 = Pixel1 + Pixel2 = (200, 150, 100) + (100, 250, 200) = (300, 400, 320)

If each channel is represented using 8 bits, then the channel values range from 0 to 255. The combined colors above exceed this range. When the channel value is outside the range (below 0 or above 255), it will be clipped to be inside the range.

Thus, the result of adding the above 2 pixels is: 255, 255, 255. Assuming that the result of adding most of the pixels in 2 images exceeds the range, then the output image will be nearly white, and thus the details in the 2 images are lost. To overcome this issue, we use the alpha channel.

When adding 2 pixels together while using the alpha channel, the value in the alpha channel specifies how much of the pixel contributes to the result of the addition. Let’s discuss this.

Assume that the alpha channel is added to the previous 2 pixels, as follows:

Pixel1 = (R1, G1, B1, A1) = (200, 150, 100, 120) Pixel2 = (R2, G2, B2, A2) = (100, 250, 220, 80)

The alpha values for these 2 pixels are 20 and 80. These values are used to calculate how much color from each pixel is used in the addition. For 8-bit images, the percentage is calculated as follows:

Fraction = Alpha/255

For the first pixel with a value of 120 to the alpha channel, the result is (120)/255=0.471. This means a fraction of 0.471 will be used from the first pixel. In other words, 47.1% of the pixel color will appear in the result.

For the second pixel, the result is 0.314, which means 31.4% of the second pixel color is used in the addition.

After calculating the fractions, the result of the addition is calculated as follows:

Pixel3 = Fraction1*Pixel1 + Fraction2*Pixel2

By substituting the values, the result is as follows after being rounded:

Pixel3 = 0. 471*(200, 150, 100) + 0. 314*(100, 250, 220) = (125.6, 149.15, 116.18) = (126, 149, 116)

The new result of combining the 2 pixels after using the alpha channel is within the 0 to 255 range. By changing the values of the alpha channels, you can change the amount of color taken from each pixel.

In the previous example, using the alpha channel helps the preserve the result within the 0 to 255 range. Note that there are 2 alpha values used for the addition, one for each image. This leaves us with a question. What if the values for the 2 alpha channels are 0?

Note that an alpha of 255 means that the color isn’t transparent at all. When adding a pixel with alpha 255, the full pixel color is used. So, if the alpha values for the previous 2 pixels are 255, the result of addition will be (300, 400, 320) where all values are outside the 0–255 range. How do we solve this issue? This is by using just a single alpha value rather than 2, according to the equation below:

Pixel3 = Fraction*Pixel1 + (1- Fraction)*Pixel2

This equation confirms the values in Pixel3 will be in the 0–255 range. Assuming that the alpha value to be used in the addition is the one saved in the first pixel, which is 120, then the fraction is 0.471.

Let’s edit the pixel values to work with the worst case, where the pixels values are high, as given below. The values in the red channels of the 2 pixels are 255.

Pixel1 = (R1, G1, B1, A1) = (255, 150, 100, 120) Pixel2 = (R2, G2, B2, A2) = (255, 250, 220, 80)

The new pixel value is:

Pixel3 = 0.471*(255, 150, 100) + 0.529*(255, 250, 220) = (255, 202.9, 163.48) = (255, 203, 164)

The result of addition is that all values are inside the range (especially the red channel).

At this point, the alpha channel is introduced to highlight its benefit in the blending or compositing of 2 or more images. The next section works with the alpha channel in OpenCV.

Working with the Alpha Channel in OpenCV

When an image is read in OpenCV using the Imgcodecs.imread() method, the default color space of the image will be BGR (Blue-Green-Red). This color space doesn’t include an alpha channel, but OpenCV supports other color spaces that include the alpha channel. The one we’re going to use in this tutorial is BGRA, which is the regular BGR color space after adding a fourth channel for alpha.

So the image is read as BGR, and we want to convert it to BGRA—how do we do that conversion? Note that the previous tutorial used the Imgproc.cvtColor() method for converting an image from BGR to gray. It can also be used to convert BGR to BGRA, according to the line below.

The Imgproc.cvtColor() method accepts 3 arguments. The first one is the source Mat. The second one is the destination Mat in which the result of the color conversion will be saved. And the third argument specifies the current color space of the source Mat and the desired color space of the destination Mat.

Mat img = Imgcodecs.imread("img.jpg");

Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2BGRA);

The default value for the alpha channel is 255. Thus, the original image will appear as it is. Using the for loops listed below, we can access each pixel and edit its alpha channel. All values for the alpha channel are set to 50.

Inside the for loops, each pixel is returned using the get() method, by specifying the current row and column indices. This returns a single pixel in a double array named pixel . Because the color space is RGBA, which includes 4 channels, then each pixel has 4 values. Thus, the length of the returned array is 4. In order to access the alpha channel, which is the fourth channel, index 3 is used. The alpha channel is changed for the current pixel to be 50.

To apply these changes, the current pixel is set to the edited array, according to the put() method. Finally, the image is written using Imgcodecs.imwrite() . Note that the image extension is PNG, because PNG supports transparency.

The image returned from the above code is shown below. Note that the image has high transparency because a small alpha value (50) is used.

If the above code is changed to use a value of 160 for the alpha channel, the result will be less transparent, as shown below.

This section used the alpha channel for just a single image. We can also blend 2 images together using the alpha channel. We’ll look at how to do that in the following section.

Image Blending

This section adds 2 images together while using the alpha channel. The code for adding such images is listed below.

The 2 images are read in the 2 Mat arrays img and img2 . These images have to be of equal size at the current time.

A third Mat is created named result to hold the result of the addition. To make this Mat of the same size as the previous 2 Mat arrays, it’s initially set equal to the first Mat img .

Using the 2 for loops, the current pixel is returned from the 2 Mat arrays. The used alpha value is set equal to 200, and then the fraction is calculated. After that, the new pixel is calculated by multiplying the fraction by the value returned from the first image, and the inverse of the fraction by the second image. The results of these multiplications are added together. The new pixel is then put into the result Mat. Finally, the image is saved.

The saved image returned from the above code is shown below. Note that the first image appears stronger than the second image. The reason for this is that the fraction taken from the first image is 200/250=0.8, but just the fraction 0.2 is used with the second image. In other words, 80% of the pixels values in the result Mat are taken from the second image, and the remaining 20% are taken from the second Mat.

If the alpha value is made 50, then 80% of pixels values in the result Mat are taken from the second image and only 20% are from the second image. The result is shown below.

If the alpha is set to 0.0, then the result will be identical to the second image. If 255.0, then it will be identical to the first image.

This section blends 2 images together by blending each pixel. In the next section, we’ll look at blending a specific region from one image into another image.