# Research on Automatic Inspection Methods of Image Quality of Digital Aerial Photography Results

Yong Liang, Yanwei Zeng, Wencong Jiang, Xiaojun Wang

► To cite this version:

Yong Liang, Yanwei Zeng, Wencong Jiang, Xiaojun Wang. Research on Automatic Inspection Methods of Image Quality of Digital Aerial Photography Results. 5th Computer and Computing Technologies in Agriculture (CCTA), Oct 2011, Beijing, China. pp.320-331, 10.1007/978-3-642-27275-2_37. hal-01361156

HAL Id: hal-01361156

https://inria.hal.science/hal-01361156

Submitted on 6 Sep 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Research on Automatic Inspection Methods of Image Quality of
Digital Aerial Photography Results

Liang Yong¹, Zeng Yanwei *², Jiang Wencong², Wang Xiaojun²

1. School of Information Science & Engineering, Shandong Agricultural University, Tai'an, Shandong 271018,
P. R. China;

*2. Quality Supervision and Testing Center of National Surveying and Mapping Products, Chengdu, Si Chuan
Province 610081,China; zengyw@sbsm.gov.cn

2. School of Information Science & Engineering, Shandong Agricultural University, Tai'an, Shandong 271018,
P. R. China

**Abstract.** Aerial photogrammetry is one of the main methods obtaining geospatial information. After entering the 21st century, aerial photogrammetry technology has fully entered the digital age, and the quality of digital aerial photography results will directly affect the quality and accuracy of results of surveying and mapping. Therefore, it is necessary to inspect the quality of digital aerial photography results. In the process of digital aerial photography, image quality directly affects the quality of the results. It is of great importance to make its image quality inspection. Based on the analysis of image quality index of the results of digital aerial photography and domestic and international current situation of quality inspection technology of aerial photography results, the author has put forward the index system which consists of subjective evaluation index and objective evaluation index. Based on objective evaluation index which includes the maximal displacement of image points, image definition, uniform degree of image contrast and brightness and image integral color performance, the author has researched automatic inspection methods of image quality of the results of digital aerial photography, and has realized automatic quality inspection of image quality of the results of digital aerial photography.

**Key words:** Digital aerial photography; Image quality index; Image quality; Quality inspection; Automatic inspection

# 1 Introduction

Digital aerial photography results are the important data sources of basic surveying and mapping, which quality will directly affect the quality of results of follow-up surveying and mapping. Therefore, it is necessary steps to make the comprehensive quality inspection to guarantee the quality of data, and it's of important application value and meaning [1]. In the process of digital aerial photography, image quality directly affects the quality of the results, and it's important to check image quality. Digital aerial photography provides digital aviation image and makes it possible to take comprehensive and rigorous automatic inspection using computers.
## 2 Index of Image Quality

In this paper, based on the national standards and regulations(in table 1), the author studied and put forward quality inspection index system of image quality of the box type digital aerial photography results(in table 2).

The index of image quality includes the maximal displacement of image points, image definition, image color, image contrast, cloud and cloud shadow, splash and bad point and image noise and so on [2] [3].

<table><caption>**Table1.** National standards and regulations</caption><tr><td rowspan="10">National standards and regulations</td><td>GB/T 6962-2005 《Aerial Photographic specification for 1: 500、1:1000、1:2000 Scale Topographic maps》</td></tr><tr><td>GB/T 15661-1955 《Aerial Photographic specification for 1: 5000、1:10000、1:25000、1:50000、1:100000 Scale Topographic maps》</td></tr><tr><td>GB/T 19294-2003 《Design specification of Aerial Photography Technology》</td></tr><tr><td>MH/T 1006-1996 《Inspection specification of Aerial Photography Instruments》</td></tr><tr><td>《Provisions of GPS Supplemental Aerial Photography Technology》 (Trying)</td></tr><tr><td>GB/T 24356-2009 《Quality Inspection and Acceptance of Surveying and Mapping Results》</td></tr><tr><td>《Topographic Map Based on The Scale of 1:10000、1: 50000 IMU/DGPS Supplemental Aerial Photography Technology Provisions 》 (Trying)</td></tr><tr><td>《Implementation Detailed Rules of Inspection and Acceptance and Quality Evaluation of The National Basic Aerial Photography Results》 (Trial Draft)</td></tr><tr><td>《Supplement Technology Regulations of The National Basic Aerial Photography 》</td></tr><tr><td>《Implementation Detailed Rules of Inspection and Acceptance and Quality Evaluation of The National Basic Aerial Photography Results》 (Submissions)</td></tr><tr><td>《The Format and Note of Material of The National Basic Aerial Photography Results》</td></tr><tr><td>《Data Arrangement and Explains of Digital Aerial Photography Results》 (Trying)</td></tr></table>

<table><caption>**Table2.** Image quality index system of digital aerial photography results [4] [5]</caption><thead><tr><th>Index of the second class</th><th>Description of function</th></tr></thead><tbody><tr><td>the maximal displacement of image points</td><td>Check the deformation of features in images to judge whether to be beyond the range or not</td></tr><tr><td>image definition</td><td>Check image definition to judge whether to be read or not</td></tr><tr><td>image color</td><td>Check the partial color of images</td></tr><tr><td>image contrast</td><td>Check image contrast to judge whether to be moderate or not</td></tr><tr><td>cloud and cloud shadow</td><td>Check the coverage of cloud and cloud shadow and the percentage of the area of cloud and cloud shadow on images</td></tr><tr><td>image noise</td><td>Check the range of image noise</td></tr><tr><td>splash and bad point</td><td>Check whether there are splashes in images</td></tr></tbody></table>
# 3 Inspection Methods of Image Quality

It's a complicated process to make image quality assessment of digital images. The reasons of complication are that the images are two dimensional representations of three dimensional real worlds and not comprehensively reflect the properties of features. It's a perception process to make image quality evaluation and it's a comprehensive judgment using the brain and other organs. In addition, the evaluation results also depend on people's knowledge ability and image quality requirements of the evaluation. The complexity shows that the evaluation doesn't only depend on the quantitative index and it will appear a better result to use the method of the combination between subjective evaluation and objective evaluation.

## 3.1 Subjective Evaluation Index

It's difficult to distinguish characteristics texture, imaging injury and image quality degradation in the process and artificial qualitative evaluation results are given. The human eyes judge images whether there are cloud and cloud shadow, bad points and noise, according to the subjective feelings to give the evaluation.

### 3.1.1 Cloud and Cloud Shadow

View the data of images, select problem areas through the user circle, and calculate the area size of cloud and cloud shadow.

### 3.1.2 Splash and Bad Point

View the data of images and users judge splashes and bad points of images.

### 3.1.3 Image Noise

View the data of images and users judge noise of images.

## 3.2 Objective Evaluation Index

### 3.2.1 The Maximal Displacement of Image Points

Calculate by reading the data of images.

Space frequency can be used to make quantitative evaluation of the maximal displacement of image points. If the calculation value is smaller, the displacement is bigger, and if the calculation value is bigger, the displacement is smaller [6].

Space frequency: It reflects the overall activity degree of image space. And it includes space line frequency RF and space column frequency CF which can be expressed as:
$$RF = \sqrt{\frac{1}{m \times n} \sum_{i=1}^{m} \sum_{j=2}^{n} (f_{i,j} - f_{i,j-1})^2} \quad (1)$$

$$CF = \sqrt{\frac{1}{m \times n} \sum_{i=2}^{m} \sum_{j=1}^{n} (f_{i,j} - f_{i-1,j})^2} \quad (2)$$

Get the whole space frequency value:

$$SF = \sqrt{RF^2 + CF^2} \quad (3)$$

Among them, *m* and *n* stand for the width and height of images, and *f*(*i*, *j*) is the gray value of image pixels (*i*, *j*).

Take a few pictures below as an example according to the above algorithms and calculate their maximal displacement of image points. From the results we can see that the SF value of three pictures above from left to right becomes smaller and smaller, and the results of three pictures below are the same. This conclusion is consistent with subjective evaluation.

<figure><img src="image_2.png" /><figcaption>**Fig.1.** Inspection of the maximal displacement of image points according to Space frequency</figcaption></figure>
### 3.2.2 Image Definition

It depicts the clarity degree of image features, and it's the contrast degree of features and its background region.

It's more index of quantitative evaluation about definition. Definition is used to reflect diffusion degree of some characteristics of the edge of images, and if the diffusion degree is bigger, the definition is smaller, and if the diffusion degree is smaller, the definition is bigger. At present, the most mature evaluation method of definition is MTF which full name is modulation transfer function, but the determination of MTF is relatively complex [7]. In order to make it simple, we can take global point sharpness algorithm as the approximate evaluation. It can accurately and fast reflect tendency of definition of digital images. It's calculated by:

$$P = \frac{\sum_{i=1}^{m \times n} \sum_{a=1}^{8} |df / dx|}{m \times n} \qquad (4)$$

Among them, *m* and *n* is the size of images, *df* is the grayscale amplitude and *dx* is the distance increment between the pixels. This formula can be described: Get the difference of each point and its eight neighborhood points and calculate the eight differences according to the average of the weighted distance and finally obtain the average results of the whole image and regard it as the relative evaluation index of image definition.

The value domain of this index is from 0 to 255 and the value is bigger, then the definition is bigger.

Take a few pictures below as an example. From the results we can see that the definition degree of three pictures above from left to right becomes smaller and smaller, and the results of three pictures below are the same. The results are also the same to the subjective evaluation.
<figure><img src="image_3.png" /><figcaption>Fig.2. Detection of image definition</figcaption></figure>

### 3.2.3 Uniform Degree of Image Contrast and Brightness

The common data detection about remote sensing images and aviation images is image brightness uniformity and according to the absolute evaluation, we put forward a kind of quantitative index. Firstly according to image brightness distribution fitting quadric surface we judge whether it is level, that is, whether its variances is close to zero, and the variance is more closer to zero, then the brightness distribution is uniform. On the contrary it's not uniform [8]. And its calculation process is as follows:

Input sampling of image brightness component and obtain the size ($M \times N$) of images.

(1) Generate average template according to window size ($w_w \times h_w$, such as 11×11) and get its three-dimensional coordinate $(x, y, l)$. Among them, $x, y$ is coordinates of each point $x \in [0, \lceil N/w_w \rceil], y \in [0, \lceil M/h \rceil]$ and $\lceil \cdot \rceil$ stands for getting integer up and $\lfloor \cdot \rfloor$ stands for getting integer down. $l$ is the brightness of each point.

(2) According to the explicit two-dimensional surface equation:
$$z = A + Bx + Cy + Dxy + Ex^2 + Fy^2$$

Generate vertex equations based on least square principle:

$$ \begin{bmatrix} m & \sum_{i=1}^{m} x & \sum_{i=1}^{m} y & \sum_{i=1}^{m} xy & \sum_{i=1}^{m} x^2 & \sum_{i=1}^{m} y^2 \\ \sum_{i=1}^{m} x & \sum_{i=1}^{m} x^2 & \sum_{i=1}^{m} xy & \sum_{i=1}^{m} x^2 y & \sum_{i=1}^{m} x^3 & \sum_{i=1}^{m} xy^2 \\ \sum_{i=1}^{m} y & \sum_{i=1}^{m} xy & \sum_{i=1}^{m} y^2 & \sum_{i=1}^{m} xy^2 & \sum_{i=1}^{m} x^2 y & \sum_{i=1}^{m} y^3 \\ \sum_{i=1}^{m} xy & \sum_{i=1}^{m} x^2 y & \sum_{i=1}^{m} xy^2 & \sum_{i=1}^{m} x^2 y^2 & \sum_{i=1}^{m} x^3 y & \sum_{i=1}^{m} xy^3 \\ \sum_{i=1}^{m} x^2 & \sum_{i=1}^{m} x^3 & \sum_{i=1}^{m} x^2 y & \sum_{i=1}^{m} x^3 y & \sum_{i=1}^{m} x^4 & \sum_{i=1}^{m} x^2 y^2 \\ \sum_{i=1}^{m} y^2 & \sum_{i=1}^{m} xy^2 & \sum_{i=1}^{m} y^3 & \sum_{i=1}^{m} xy^3 & \sum_{i=1}^{m} x^2 y^2 & \sum_{i=1}^{m} y^4 \end{bmatrix} \begin{bmatrix} \sum_{i=1}^{m} l \\ \sum_{i=1}^{m} xl \\ \sum_{i=1}^{m} yl \\ \sum_{i=1}^{m} xyl \\ \sum_{i=1}^{m} x^2 l \\ \sum_{i=1}^{m} y^2 l \end{bmatrix} = \begin{bmatrix} A \\ B \\ C \\ D \\ E \\ F \end{bmatrix} \quad (6) $$

Among them $m$ is the number of points in template,

(3) Use Gaussian elimination method of main elements selected to solve the above equation and get the coefficient of the explicit two-dimensional surface equation.

(4) Calculate square error according to the two-dimensional surface equation and it's shown as follows

$$ \sigma_t^2 = \frac{1}{M' \cdot N'} \sum_{i=0}^{M'-1} \sum_{j=0}^{N'-1} [\hat{l} - \mu]^2 \quad (7) $$

Among them, $M', N'$ is the size of template, $M' = \lceil M/w_w \rceil, N' = \lceil N/h_w \rceil$, $\hat{l}$ is fitting brightness of each point in template according to the two-dimensional surface and $\mu$ is the average width of template.

If square error is smaller, the brightness distribution is uniform; On the contrary it's not uniform. From the calculation process we can found that the domain still is from 0 to 255. When square error is close to 0, the template is close to ideal level plane and it indicates that brightness distribution is uniform.

The pictures below give several square error values after processing. From the results we can see that the square error value of three pictures above from left to right becomes bigger and bigger, and the results of three pictures below are the same. This conclusion is consistent with subjective evaluation.
<figure><img src="image_4.png" /><figcaption>Fig.3. Detection of uniform degree of image brightness</figcaption></figure>

3.2.4 Image Integral Color Performance

It's used to depict integral tonal of images and evaluate the difference of adjacent images in image mosaic.

The method based on gray balance is to make statistics of the average brightness of three channels
and judge whether the result is neutral grey according to assumption scene meeting the gray world. If it
is neutral grey, there will be no partial color. Otherwise there will be partial color. The algorithm is
shown below:

(1) Make statistics of the average brightness of three channels and they are R<sub>mean</sub>, G<sub>mean</sub>, B<sub>mean</sub>.

(2) Suppose for the standard RGB color space coordinates and get the relatively uniform chromaticity coordinates through the color space transformation. The method of color space transformation is shown as follows:

The conversion of RGB and Lab chromaticity coordinates:

There are two kinds of methods to establish color space: One is the color reference system based on
the psychological physical experiment (such as munsell table color system); the second is the color
analytic expression system based on the psychological physics experiment (such as CIELAB, CIELUV
color system). Color space used commonly can be divided into the base color space, CIE color space orthogonal color space and cognitive color space.

Base color space is the most basic and common RGB color space in image processing. Because the existing image acquisition and display equipment is mostly based on RGB value and intermediate storage is also mostly based on RGB value. Other color space used in image processing is the transformation from RGB color space and the processing results that need to display or storage out also need to convert back to RGB color space [9].

CIE chromaticity space based on colorimetric theories is launched by CIE (international lighting committee) and has been generally accepted as international measurement standards [10]. The common characteristics of all CIE chromaticity space have nothing to do with equipment.

The main principle of orthogonal color space is using color antagonist phenomenon of human visual perception to find out the orthogonal color component. According to different TV formats, color space which color component is approximating orthogonal has YUV color space (PAL TV), YIQ (NTSC TV) and so on. At present color space of the digital coding of standard television images which are recommended is YCbCr (ITU/ITU-R, 1994). In addition, there is the I1I2I3 color space put forward by Ohta, and three color components are completely irrelevant. Pratt made K-L transform for RGB and the results show that, Ohta component (I1 I2, I3) is a good approximation of K-L transforms for RGB.

In order to comply with people's subjective cognitive characteristics for the color, many researchers have studied various algorithms for get color attribute from RGB value, and they are collectively called cognitive color space which includes HIS, VHS, LHS HSB and GLHS and so on, and these color space in computer vision and digital image processing has a wide range of applications, so they also known as color models. Their common characteristics are based on the subjective psychological evaluation of human visual system for the color and mostly based on hue, saturation and brightness.

In comparison to other color space, RGB color space has the following characteristics:

It's not intuitive and it is difficult to directly recognize color from RGB value through the human visual system.

RGB color space is one of the least uniform color spaces. The perception difference of the two colors is nonlinear, and it can't be expressed as Euclidean distance between two color points.

There is a high correlation between RGB values (B-R: 0.78, R-G: 0.98, G-B: 0.94) (Palus, 1998). These high correlations make the challenge on the various treatments of the color information.

It is related to device. RGB value is actually the relative value of the device dynamic range quantity, and the same RGB values have different color performance on different display and output devices.

Just like cognition color space of HSV, HSL, and HSB, it is a subset of color space that the human can perceive.

In order to improve color space and imaging devices of the RGB and display devices relevance, the International Color Consortium introduced the sRGB color space to make image exchange in 1996, especially the standard format of network image exchange. sRGB color space provides handling the color methods from the operating system and device drivers, and uses a simple and robust device independent color to definite add the current ICC color management strategy. The definition of sRGB color space is based on the average effect of a typical CRT monitor under the conditions of reference and observation, but it is suitable for the flat panel display devices, television, scanners, digital cameras and printing systems. Due to these advantages, it has been adopted as an international standard by ICC.
Because of the defined similarity of the display device of reference and real CRT monitors, when you use the sRGB color space, you usually do not need extra color space conversion to display images. However, you need to convert the data to sRGB, and then output to different dynamic range, color gamut and output device of the observe conditions. With regards to sRGB standard color space and ICC profile format, more details can be referred to the relevant references (Gasparin & Schettini, 2004).

The RGB color space is related to the device, but independent CIE color space has no relation with the device, therefore in the conversion process from RGB to CIE LAB, you need to describe the device properties.In the absence of profile and experimental conditions, you can usually assume image data based on the sRGB standard color space, and the light source referred is CIE D65, according to the following formula to achieve color coordinate transformation from the RGB to Lab.

$$R'_{sRGB} = R_{8bit} / 255.0; G'_{sRGB} = G_{8bit} / 255.0; B'_{sRGB} = B_{8bit} / 255.0 \quad (8)$$

$$I_{sRGB} = \begin{cases} I'_{sRGB} / 12.92, & \text{if } (I'_{sRGB} \le 0.04045) \\ [(I'_{sRGB} + 0.055) / 1.055]^{2.4}, & \text{else} \end{cases} \quad (9)$$

$$\text{Among them, } I'_{sRGB} = \{R'_{sRGB}, G'_{sRGB}, B'_{sRGB}\}, I_{sRGB} = \{R_{sRGB}, G_{sRGB}, B_{sRGB}\} \quad (10)$$

$$\begin{bmatrix} X \\ Y \\ Z \end{bmatrix} = \begin{bmatrix} 0.4124 & 0.3576 & 0.1805 \\ 0.2126 & 0.7152 & 0.0722 \\ 0.0193 & 0.1192 & 0.9505 \end{bmatrix} \begin{bmatrix} R_{sRGB} \\ G_{sRGB} \\ B_{sRGB} \end{bmatrix} \quad (11)$$

$$S' = S / S_0, S' = \{X', Y', Z'\}, S = \{X, Y, Z\}, S_0 = \{X_0, Y_0, Z_0\} \quad (12)$$

Among them, $X_0, Y_0, Z_0$ is the stimulation value of D65 light source:

$$X_0 = 0.9505, Y_0 = 1.00, Z_0 = 1.0891 \quad (13)$$

$$S'' = \begin{cases} (S')^{1/3}, & \text{if } (S' > 0.008856) \\ 7.787 \times S' + 16.0 / 116.0, & \text{else} \end{cases}, S'' = \{X'', Y'', Z''\} \quad (14)$$

$$\begin{cases} L^* = 903.3Y', a^* = 3893.5(X' - Y'), b^* = 1557.4(Y' - Z'), \\ \text{if } (X' \le 0.008856 \& Y' \le 0.008856 \& Z' \le 0.008856) \\ L^* = 116Y'' - 16, a^* = 500(X'' - Y''), b^* = 200(Y'' - Z''), \\ \text{else} \end{cases} \quad (15)$$

(3) Calculate the distance of chromaticity of neutral grey

$$dis_{gray} = \sqrt{a^2 + b^2} \quad (16)$$

(4) The value is bigger then the integral partial color is more serious.

Take three pictures below as an example. From the results we can see that the overall partial color of the left image is weak, that of the middle image turns to be blue and red and that of the right image becomes yellow and red. And the situation of partial color of the right image is the most serious and the left is the most light. This conclusion is consistent with subjective evaluation.
<figure><img src="image_5.png" /><figcaption>$dis_{gray} = 2.89363$</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>$dis_{gray} = 22.7274$</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>$dis_{gray} = 36.3411$</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>$R_{mean} = 97$; $G_{mean} = 94$;</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>$R_{mean} = 77$; $G_{mean} = 79$;</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>$R_{mean} = 120$; $G_{mean} = 119$;</figcaption></figure>

<caption>$B_{mean} = 55$;</caption>

<caption>**Fig.4.** Calculation of partial color based on the method of gray balance</caption>

## 4 Conclusions

In this paper, based on research and analysis of domestic and international situations of quality inspection technology of aerial photography results, the author studies and proposes inspection index and methods of image quality of box type digital aerial photography. Subjective evaluation index system which includes cloud and cloud shadow, splashes and bad points of images and image noise is difficult to recognize by computers and it needs to give the evaluation according to the subjective feelings. Based on objective evaluation index system which consists of the maximal displacement of image points, image definition, uniform degree of image contrast and brightness and image integral color performance, the author has put forward the quality inspection method of image appearance based on characteristics of the human visual system and image structure distortion, and has realized automatic quality inspection of image appearance quality of the results of digital aerial photography.
References

[1]Yu, Ch., Zeng, Y.: Design on Automatic Quality Inspection System of Digital Aerial Photography Results. Information and Engineering of Surveying and Mapping36 (1), 8-10(2011)

[2] GB/T 6962-2005: Aerial Photographic Specification for 1:500、1:1000、1:2000 Scale Topographic maps

[3] GB/T 15661-1955: Aerial Photographic Specification for 1:5000、1:10000、1:25000、1:50000、1:100000 Scale Topographic Maps

[4] Implementation Detailed Rules of Inspection and Acceptance and Quality Evaluation of the National Basic Aerial Photography Results. Bureau of National Surveying and Mapping (2001)

[5] Data Arrangement and Explains of Digital Aerial Photography Results. Bureau of National Surveying and Mapping (2007)

[6]Wu, Sh.: Design and Implementation on Digital Acceptance System of Aerial Photography Flight Quality. Information Engineering University of Zhengzhou (2005)

[7]Wang, H.: MTF Measurement Research and Its Application of Satellite in Orbit. Nanjing University of Science and Technology, 2004

[8]Li, Q., Guo, B., Dong, L.: The Semi-automatic Extraction Algorithm Research of the road centerline on Aerial images. Information and Engineering of Surveying and Mapping, 2009, 34(1):23-25

[9]Yuan, G., Huang, J.: Quality Control and Inspection of Digital Aerial images. Surveying and Mapping society of Jiangsu Province (2003)

[10]Yuan, J.: Information Management System of Aerial Photography. Information Engineering University of Zhengzhou (2002)