This AI Removes the Water from Underwater Images!
By: Louis, What S Ai
Tags: Underwater, Machine Learning, Water, Photography, Computer Vision

Technique explanation

Underwater photography is a challenging task. It requires expensive cameras, lights, all sorts of equipment, and a lot of skills. Even then, there is still a blue-greenish tint from the water covering the objects, or animals in any pictures or video you will take. Fortunately for all the underwater enthusiasts, researchers from the University of Haifa were able to remove the water from these pictures and videos by using computer vision and machine learning algorithms.

Image from the paper

They called their new algorithm: Sea-thru. It takes an image from underwater with the same blue-green tint and makes it look like it was photographed through the air instead of water! It doesn’t only remove this color filter from the water. It also removes the visual distortion caused by the light travels underwater to accurately correct both the colors and physical aspects of the images.

Let’s dive a little deeper for all the machine learning enthusiasts that are wondering how this algorithm work. As I said, it takes an underwater image as input. But you don’t even need an expensive set of lights anymore since it takes a raw image, or even video, with natural lighting. Normally, this was a big issue that is completely solved by this algorithm. Objects that were far away from the light source wouldn’t get caught clearly by the camera. They now appear as clear as the objects closer to the lens, if the distance is specified.

Image from the paper

Indeed, it takes raw RGB-D images, where the D stands for the distance, or depth map, between the camera and the object we want to have a clear view. This depth map is later used, with the raw image, to reveal the original colors of the image by calculating the attenuation of each wavelength of light.

Image from the paper

They’ve done that by placing objects of known sizes in the scene at various ranges, analyzing the backscatter effect, and reconstructing them using the known scene’s information. Then, it uses an estimate of the spatially varying illuminant calculated in the training phase of the model to obtain the underwater range-dependent attenuation coefficient.

But this is just half the work that needs to be done to have colors that match an image that was taken at the sea surface. It also needs to estimate the backscatter using the dark pixels and their known range-information.

Image by Matt Testoni

The backscatter is the reflection of light waves bouncing on particles in the water and going into the camera, considering the underwater environment deflecting them from their original direction. It illuminates these water particles and can frequently obstruct the object. To find this backscatter value and reverse it, they start by partitioning the range map into 10 clusters covering all the possible values of z, estimated previously. Then, using the captured image and known distance of the object, they find the perfect z correlation. Following that, they can do the same process across the whole image estimating the backscatter from the information previously computed.

There is still one step left to explain, which is how they can find the wideband attenuation and backscatter coefficients of the RGB image. Governing both the backscatter value and the information about the scene.

Spectral response | Image from Stemmer-Imaging

This is a complex calculation explained in detail in the paper, but it is mainly done using the spectral response of the camera and the optical water type which they measured previously using their trained model.

Finally, using their known range map of z estimated previously, they can refine these coefficients to obey the given z and improve the final results.

This will be very beneficial for coral biologists and other experts in the field since this work really aims to obtain the true colors compared to the current enhanced version you can find using photoshop.