<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
<figure><img src="image_2.png" /></figure>

# Multi-feature Based Sea-land Segmentation for Multi-spectral and Panchromatic Remote-Sensing Imagery

D. K. Savitha(✉), Rajkumar Kethavath, and Arshad Jamal

Centre for Artificial Intelligence and Robotics (CAIR), Defense Research and Development Organization (DRDO), Bangalore 093, India

savitha.cair@gov.in

**Abstract.** Robust sea-land segmentation in optical remote-sensing images is difficult task because of complex sea-land environment. Low contrast difference between sea and land in the case of panchromatic images and water color variation in case of multispectral images makes the segmentation process highly challenging. In this paper, we propose a multi-feature sea-land segmentation approach for optical remote sensing images where features such as color information in CMYK, intensity in HSV, statistical information and texture features are used to segment the image into sea and land. Morphological operations are applied to fine tune the segmentation results. In the proposed approach, these features are extracted from resized image and take very less computation time. The proposed approach was tested on Google-Earth images and yields robust, accurate sea-land segmentation results and it works very well on both panchromatic as well as multi-spectral images.

**Keywords:** CMYK · HSV · Gabor Texture · Connected-Component and Morphological Operation · Unsupervised-Segmentation

## 1 Introduction

Ship target detection in remote sensing images is challenging task as they indicate false targets on the land area because of many ship like elongated objects present in the land. Accurate sea-land segmentation in such images becomes important to reduce the false targets. Reliable segmentation of sea and land would improve the detection accuracy by removing these false positive targets on the land area. Also by masking the land area speed of the detection process increases as we skip the masked area and process only the unmasked area of the image for target detection.

There exist many supervised and unsupervised techniques for sea land segmentation. With all the supervised techniques including deep learning based ones the challenging part is in training the model. In the unsupervised technique the challenge is in selecting minimum appropriate features and execution time.
To overcome these challenges we propose multi-feature based sea-land segmentation method which works on Multi-Spectral as well as panchromatic remote-sensing images. In case of Multi-spectral images rich color information present in the images are used to segment the images in CMYK color space. Along with this the Gabor texture feature and Otsu segmentation on intensity is used as features to get final sea-land segmentation. Morphological operations are applied to fill the holes on the land area. Our method is simple to compute i.e. can work on CPU without any requirement of training data or geographical information.

## 2 Related Work

There are many sea-land segmentation techniques based on supervised learning methods including deep learning based [1-4]. But they need large number of data for learning and model creation. Many existing geographic information [12] based methods depend on database of geographic coordinate for image matching for sea-land segmentation. But their results are dependent on accuracy of geographic coordinates. There are several methods based on features such as grey scale and texture. Grey scale represents intensity information of the image. Based on these grey scale values which varies from 0 to 255, pixels in images are grouped into sea and land. In grey scale based approaches [5-7]. You and Li [5] proposed a sea-land segmentation method which computes the threshold value according to adaptively established statistical model of the sea, and removes the incorrectly segmented land region by considering the variance difference in the statistical model of land and sea. Zhang and Wang [6] used minimum filter method to reduce sea surface texture effect for coastline extraction. However, thresholds alone usually cannot guaranty proper segmentation when applied on images with low contrast between sea and land. Texture based methods [10, 13] also fail to give good segmentation of sea-land as land portion may have very smooth texture in some regions. When it comes to color information based land-sea classification methods, spectral information [9] are used to capture the sea variation. But many a times this alone is not sufficient for proper classification. In case of graph-cut [8] based sea land segmentation initialization of the proper seed points for land and sea plays very important role in proper segmentation. To overcome many of these limitations multiple features [9-12] are extracted from images for proper segmentation of sea-land and SVM based classifiers are used however it works well when the features are non-overlapping. Towards this effort our paper proposes new combination of features for fast unsupervised panchromatic as well as multispectral sea-land segmentation.

## 3 Theory

Some of the concepts used in our sea-land segmentation approach are as explained below is shown in Fig. 1. Image tiles up to 4096 x 4096 pixels are resized into 512 x 512 pixels before extracting the features used in segmenting the image to sea and land area. As the histogram of the resized image and original images are of same shape this step helps in fast processing of Otsu segmentation. Also the small window size (5 x 5) is sufficient to extract Gabor texture features quickly.
## 3.1 CMYK Color Space

This is known as subtractive color model, it is mainly used in printing. As we can know that the sea is dominated by cyan color the image is first converted to CMYK color space to get the cyan color information. The information content in C component is used as a color feature for the segmentation process. Otsu segmentation [15, 16] is applied on this component to get probable sea-land segmentation areas.

## 3.2 HSV Color Space

HSV color space characterizes the color by segregating the Hue, Saturation and Intensity. This color model represents the way humans interpret color. On the V component of the image, Otsu segmentation is applied to segment the image into sea and land regions using the intensity values. Gabor texture features are also extracted from the V component.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Proposed Method</figcaption></figure>

## 3.3 Gabor Feature

To capture the texture part of the images, intensity variation of the pixels and its neighboring pixel values are considered. In our method, we selected Gabor filtering technique to extract texture feature. Gabor filter is a multi-channel filtering approach which allows us to capture different texture features in different sizes and orientations [13]. As it uses
edge information Gabor filtering technique is generally used in texture extraction. Gabor texture feature are computed with 30 degree separation (0, 30, 60, 90, 120 and 150) in a 5x5 window. These features are added together to get Gabor energy. In a small 5x5 window mean value of this feature are calculated. If the mean value is greater than 10 it is considered as land pixel else sea pixel. The definition of Gabor filter is given below:

$$g(x, y : \lambda, \theta, \sigma) = \exp\left(-\frac{x^{2}+y^{2}}{2\sigma^{2}}\right) \cos\left(i\left(2\pi\frac{x^{t}}{\lambda}\right)\right) \quad (1)$$

$$x^t = x \cos\theta + y \sin\theta$$

$$y^t = -x \sin\theta + y \cos\theta$$

where $\lambda$ denotes the wavelength $\theta$ denotes the orientation and $\sigma$ is the standard deviation of the Gaussian envelope

## 3.4 Otsu Segmentation

Finding proper threshold value is very critical operation in image segmentation. Through this step we can assign the pixels of image into different classes. But manually specifying the threshold value is quite tedious process as the environment is very dynamic. In case of Otsu's method it automatically identifies the threshold value quite accurately when histogram is bimodal in nature. Otsu's algorithm groups the image into 2 classes land and sea. The threshold t which is selected maximizes the inter-class variance and also minimizes the weighted within-class variance.

Class probabilities $q_1$ and $q_2$ are given by:

$$q_1(t) = \sum_{i=1}^{t} p(i) \quad (2)$$

$$q_2(t) = \sum_{i=t+1}^{l} p(i) \quad (3)$$

Class means $\mu_1$ and $\mu_2$ are given by:

$$\mu_1 = \sum_{i=1}^{t} \frac{ip(i)}{q_1(t)} \quad (4)$$

$$\mu_2 = \sum_{i=t+1}^{l} \frac{ip(i)}{q_2(t)} \quad (5)$$

Class variances $\sigma_1$ and $\sigma_2$ are given by

$$\sigma_1(t) = \sum_{i=1}^{t} [i - \mu_1(t)]^2 \frac{p(i)}{q_1(t)} \quad (6)$$

$$\sigma_2(t) = \sum_{i=t+1}^{l} [i - \mu_2(t)]^2 \frac{p(i)}{q_2(t)} \quad (7)$$

The weighted within-class variance is given by

$$\sigma_w(t) = q_1(t)\sigma_1(t) + q_2(t)\sigma_2(t) \quad (8)$$
## 3.5 Gray Connectedness

Gray connectedness when applied on a binary image gives us different regions present in the image with their area in pixels. Pixels values with 0 are considered as background and 255 as foreground region for further processing. It works on the assumption that objects with similar class have similar gray values connected to each other [18]. These connected components checks 4 or 8 neighborhoods of any foreground pixel to find regions in a binary image. Depending on the area size of these regions very small size regions are filtered out.

## 3.6 Morphological Operations

Morphological operations are applied on the segmented binary image. The image obtained after segmentation are often distorted by noise. These morphological operations fill small holes and remove the protrusion to give smooth output final image [14, 17]. The two main morphological operations are erode and dilate. The size and shape of the structuring element determines the pixels added or removed from the regions. Morphological dilate operation fills the holes in the image and erode removes pixels from the image boundary.

# 4 Proposed Method

In the proposed method the first step is to resize the image to 512x512 pixels so the texture information can be captured in a small window and the processing is very fast. Also histogram computation for Otsu's segmentation will be very fast as shape of the histogram is same for 512 x 512 pixels image and the original image is same. The image is converted to both CYMK color space and HSV color space. Figure 2. shows the image after converting to CMYK color space. The Cyan (C) component captures the water body color information present in the image. Otsu segmentation is applied on this to get cyan component based initial land-sea segmentation. It is considered as feature (a) Fig. 3. shows the HSV representation of the original image. Where Hue represents the dominant color, S represents gray information present in the color and V represents intensity value of the image. Here we use V component of the HSV color space to extract Gabor texture feature, which is represented as feature (b). Gabor filter is applied in six directions with 30 degree separation i.e. (0, 30, 60, 90, 120 and 150) in a 5x5 windows and the Gabor energy of the image is obtained by adding all the filter outputs. These values are contrast stretched between 0 to 255. In a small window image statistics mean is calculated and if it is more than threshold value 10, image pixel value is grouped to land segment else it is grouped to sea segment. When we see original image in Fig. 2 the sea area is highly textured. When the sea is rough Gabor texture segmentation of the sea area is also segmented as likely land area. The last feature is obtained by applying Otsu segmentation on V component to segment sea-land. Here we try to capture the uniform low intensity feature of the sea, which is represented as feature (c).

Figure 4 (a) shows Otsu segmentation on Cyan component, where we can see that the sea being segmented to one class based on color information. The color of the sea
<figure><img src="image_4.png" /><figcaption>Fig. 2. Original Multispectral Image and its CYMK representation</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 3. Original Multispectral Image and its HSV representation</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. a. Otsu on C component, b. Otsu on V component, c. Gabor texture d. output</figcaption></figure>

is highly varying in nature the two main colors are blue and green. So the cyan color space captures this blue and color variation quite accurately. As shown in Fig. 4(b), Otsu segmentation on V component captures the uniform low intensity feature of the sea and groups sea pixels into one class. Land area generally has high intensity pixels compare to sea and dominated by edges due to manmade construction. Gabor texture captures dominant edge feature in a local window as we can see in the output land area is grouped into one class as in Fig. 4(c). When the sea is rough edge feature becomes prominent in the sea area because of this some parts of the sea is segmented as land. As sea becomes rougher more pixels from sea are grouped into land region resulting in wrong segmentation.

In all these segmentation which captures different features of the sea and land there is overlapping. To correct this majority logic is applied on feature a, b and c to finally segment image into land and sea. Figure 4(d) shows the output of segmentation based on majority logic. For every image pixel out of these three features at least two features has to agree for assigning a pixel to sea or land. Finally gray connectedness operation is applied to remove small disjoint regions. Morphological dilate operation applied finally to close small holes in the land class.
For Panchromatic images as there is no color information present in it only Gabor texture and Otsu segmentation on V component are captured as their feature. Gabor feature on the input image are computed as explained earlier as feature (b). Otsu segmentation on input image is computed as feature (c). These two features are merged using OR logic to segment the image pixels as sea-land regions.

To remove very small unconnected land regions gray connectedness is applied. Any land region less than 0.01% of image size is removed by setting these pixel values to 0. At this stage morphological dilate operation is applied to fill the small holes in the land area. The size of the structuring element is 5x5 with values 1 in all the cells. The final result is as shown in Fig. 5 for panchromatic images.

<figure><img src="image_7.png" /><figcaption>Fig. 5. (a) Panchromatic Image (b) Otsu Segmentation (c) Gabor Texture (d) Output</figcaption></figure>

## 5 Experimental Results

To verify the correctness of the proposed method 5 multispectral and 2 panchromatic images from Google earth maps have been used. As we can see in the Fig. 6 applying only the Otsu segmentation on the intensity or color information will not result in good land-sea segmentation but as we can see in the Table 1. The color component has good accuracy. Also using Gabor texture feature alone will not help much as sea may be rough and it will be grouped as land pixels. Combining these features based on the majority logic will result in good segmentation of land-sea area as shown in the Table 1.

To compute the accuracy of the method, ground truth of an image is required which gives us the correct pixel belongingness to the land and sea region. This ground truth was created using MATLAB 2017b Image-Segmenter application by marking all the land pixels as intensity value 256 and the sea pixels as intensity value 0 i.e. the land region pixels have white color manually changed and the sea region pixels have black color manually changed. Figure 6 shows the input image result of the proposed method and ground truth. Every pixel of the output image was compared with the corresponding pixel of the ground truth image. Pixel accuracy is calculated [19] [20] using the formula given below:

$$ \eta = \frac{\text{total number of pixels matched with ground truth}}{\text{total number of pixels}} \qquad (9) $$

Dice coefficient is calculated according to the formula given below:

$$ DSC = \frac{2|x \cap y|}{|x| + |y|} \qquad (10) $$
<figure><img src="image_8.png" /><figcaption>Fig. 6. a Input Image b. Otsu segmentation on Cyan component c. Otsu segmentation on Intensity d. Gabor Texture Segmentation e. output f. ground truth</figcaption></figure>
where X and Y are output and ground truth mask pixel values.

Jaccard index was also computed according to the formula given below:

$$jc = \frac{|x \cap y|}{|x \cup y|} \qquad (11)$$

The following Table1 shows the accuracy of proposed method using different metric approaches like Pixel Accuracy, Dice Similarity Coefficient and Jaccard Index.

<table><caption>**Table 1.** Accuracy of proposed method using different metric approaches</caption><thead><tr><th>Metric Name</th><th>Proposed method</th><th>Otsu</th><th>Gabor Texture</th><th>Color (CMYK)</th></tr></thead><tbody><tr><td>Pixel accuracy</td><td>0.9467</td><td>0.7901</td><td>0.9112</td><td>0.9350</td></tr><tr><td>Dice Similarity Coefficient</td><td>0.9156</td><td>0.7111</td><td>0.8932</td><td>0.9107</td></tr><tr><td>Jaccard Index</td><td>0.8533</td><td>0.6018</td><td>0.8120</td><td>0.8467</td></tr></tbody></table>

# 6 Conclusion

Proposed land-sea segmentation converts given image to a binary image with land and sea segments. The multi-feature un-supervised segmentation method does not require any training. An experimental result on Google images demonstrates the proposed multi-feature unsupervised segmentation method gives fast and reliable segmentation results on both multi-spectral and panchromatic images. The Segmentation algorithm clocked 532 milli-seconds per image when run on an Intel Xeon E5-2650 @ 2.30 GHz CPU with 64 GB RAM.

# References

1. Li, R., et al.: DeepUNet: a deep fully convolution network for pixel-level sea-land segmentation. IEEE J. Selected Topics in Applied Earth Observations and Remote Sensing 11(11), 3954-3962 (2018). https://doi.org/10.1109/JSTARS.2018.2833382

2. Cheng, D., Meng, G., Cheng, G., Pan, C.: SeNet: structured edge network for sea-land segmentation. IEEE Geosci. Remote Sens. Lett. 14(2), 247-251 (2017). https://doi.org/10.1109/LGRS.2016.2637439

3. Yang, T., et al.: Sea-land segmentation using deep learning techniques for landsat-8 OLI Imagery. Mar. Geodesy 43(2), 105-133 (2020). https://doi.org/10.1080/01490419.2020.1713266

4. Alam, M., Wang, J.-F., Guangpei, C., Yunrong, L.V., Chen, Y.: Convolutional neural network for the semantic segmentation of remote sensing images. Mobile Networks Appl. 26(1), 200-215 (2021). https://doi.org/10.1007/s11036-020-01703-3

5. You, X., Li, W.: A sea-land segmentation scheme based on statistical model of sea. In: 2011 4th International Congress on Image and Signal Processing, pp. 1155-1159 (2011). https://doi.org/10.1109/CISP.2011.6100503.
6. Zhang, X., Wang, Z.: Coastline extraction from remote sensing image based on improved minimum filter. In: 2nd IITA International Conference on Geoscience and Remote Sensing **2**, 44–47 (2010)

7. Xiao, H., Xu, Q., Lei, H.: A sea-land segmentation algorithm based on gray smoothness ratio. In: 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA), pp. 117–121 (2016). https://doi.org/10.1109/EORSA.2016.7552778

8. Huang, Z.: A sea-land segmentation algorithm based on graph theory. In: Proceedings SPIE **9901**, 2nd ISPRS International Conference on Computer Vision in Remote Sensing (CVRS 2015), 990111 (2 March 2016). https://doi.org/10.1117/12.2235136

9. Beşbınar, B., Gürbüz, Y.Z., Alatan, A.A.: Inshore ship detection in multispectral satellite images. In: 2015 23rd Signal Processing and Communications Applications Conference (SIU), pp. 2029–2032 (2015). https://doi.org/10.1109/SIU.2015.7130264

10. Beşbınar, B., Alatan, A.A.: Sea detection on high-resolution panchromatic satellite images using texture and intensity. In: 2014 22nd Signal Processing and Communications Applications Conference (SIU), pp. 2213–2216 (2014). https://doi.org/10.1109/SIU.2014.683 0704

11. Wang, D., Cui, X., Xie, F., Jiang, Z., Shi, Z.: Multi-feature sea-land segmentation based on pixel-wise learning for optical remote-sensing imagery. Int. J. Remote Sens. **38**(15), 4327–4347 (2017)

12. Li, J., Xie, W., Pei, J.: A sea-land segmentation algorithm based on multi-feature fusion for a large-field remote sensing image. Remote Sensing Image Processing, Geographic Information Systems, and Other Applications, 10611 (2018). https://doi.org/10.1117/12.2285779

13. Jain, A.K., Farrokhnia, F.: Unsupervised texture segmentation using Gaborfilters. Pattern Recognition **24**(12), 1167–1186 (1991). ISSN 0031–3203

14. Soille, P.: Morphological Image Analysis: Principles and Applications. Springer-Verlag, Berlin Heidelberg (2004). https://doi.org/10.1007/978-3-662-05088-0

15. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. **9**(1), 62–66 (1979). https://doi.org/10.1109/TSMC.1979.4310076

16. Youse, J.: Image binarization using otsu thresholding algorithm. 05 (2015)

17. Anuar, K., Jambek, A., Sulaiman, N.: A study of image processing using morphological opening and closing processes. Int. J. Control Theory Appl. **9**, 15 (2016)

18. Wang, Y., Bhattacharya, P.: Image analysis and segmentation using gray connected components. In: 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929), **1**, p. 444 {449 vol.1.} (1996)

19. Zhang, Y.J.: A review of recent evaluation methods for image segmentation. In: Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467), **1**, p. 148{151 vol.1 (2001)

20. Taha, A., Hanbury, A.: Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool. In: BMC Medical Imaging (2015)