<figure><img src="image_1.png" /></figure>

HAL
open science

# Detection of Surface Defects of Fruits Based on Fractal Dimension

Yongxiang Sun, Yong Liang, Qiulan Wu

► To cite this version:

Yongxiang Sun, Yong Liang, Qiulan Wu. Detection of Surface Defects of Fruits Based on Fractal Dimension. 4th Conference on Computer and Computing Technologies in Agriculture (CCTA), Oct 2010, Nanchang, China. pp.547-554, 10.1007/978-3-642-18333-1_65. hal-01559587

HAL Id: hal-01559587

https://inria.hal.science/hal-01559587

Submitted on 10 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Detection of Surface Defects of Fruits Based on Fractal Dimension

Yongxiang Sun, Yong Liang, Qiulan Wu

School of Information Science and Engineering,
Shandong Agricultural University,
Taian, Shandong Province, P. R. China, 271018;
E-mail: syx@sdau.edu.cn

**Abstract.** As the identification of surface defects is very important in fruit automatic detection, a new method for the detection of fruit surface defects based on fractal dimension is suggested. In this method, fruit image was collected using computer vision system. The fractal dimension of fruit image was calculated by an improved ‘box dimension’. The fruit fractal dimension reflects the three dimensional characteristics of the fruit as well as information of the fruit surface. The detection of surface defects of fruits was performed according to a given threshold of fruit image fractal dimension. The results on Fuji apple fruits showed that the improved ‘box dimension’ method was effective and reliable in the detection of fruit defects for its improvement in the accuracy in the calculation of the fractal dimension.

**Keywords:** Fruit; Surface defects; Detection; Fractal dimension

## 1 Introduction

Although much progress has been made in fruit automatic grading world widely, the detection of fruit surface defects was the most difficult and became one of the limiting factors. [1] In recent years, the development of the theory of fractal provides a new way for the detection of fruit surface defects. [2, 3]

Fractal theory is an important branch of nonlinear scientific that gained much attention, which focuses on objects with irregular shape in nonlinear systems in nature. As fractal geometry has many advantages in describing and analyzing the chaotic, irregular and random phenomenon in nature in comparison with traditional geometry, fractal theory was widely used in mathematics, physics, chemistry, material science, biology, medicine, geography, earthquake, astronomy, computer science, and so on. In particular, in computer science, the ideas and methods of fractal have been made much success in pattern recognition, natural images simulation, and signal processing. [4] In this paper, the detection of fruit defects was successfully performed by analyzing the fractal feature of the fruit image, which is obtained from a computerized image processing system.
## 2. The Calculation of Fractal Dimension of Fruit Image

The fractal dimension represents the irregularities of an object, reflecting the shape as well as the surface characteristics of geometric solids. At present, there are several methods concerning the calculation of fractal dimension, in which the ‘box dimension’ became one of the most widely used as it can be easily performed in computer.[5]

### 2.1. The Traditional Method for the Calculation of ‘Box Dimension’

Let $F$ be a nonempty finite subset of $R$, $N_{\delta}(F)$ be the amounts of the boxes covering $F$ which has the maximum diameter $\delta$.[6] The lower and upper box dimension of $F$ could be expressed as in formula (1) and (2), respectively.

$$ \underline{\dim}_B F = \lim_{\delta \to 0^+} \frac{\log N_{\delta}(F)}{-\log \delta} \qquad (1) $$

$$ \overline{\dim}_B F = \lim_{\delta \to 0^-} \frac{\log N_{\delta}(F)}{-\log \delta} \qquad (2) $$

If (1) and (2) are equal, the box dimension of $F$ could be expressed as in formula (3).

$$ \dim_B F = \lim_{\delta \to 0} \frac{\log N_{\delta}(F)}{-\log \delta} \qquad (3) $$

In fact, the box dimension of $F$ could be considered to be the increasing logarithmic ratios when $\delta \to 0$, which can be estimated by the slope of $\log N_{\delta}(F)$ and $-\log \delta$.

The calculation of box dimensions could be described as follows.[7] A gray image could be considered to be a three-dimensional space $(x, y, z)$, $z$ being the gray values of the pixel $(x, y)$. Thus, a three-dimensional curved surface can be formed with the total three-dimensional pixels of the gray image, which can be considered to be a nonempty finite subset $F$ in the three-dimensional space. In the three-dimensional space, cube boxes of size $r \times r \times r$ are piled in the $x, y, z$ directions. If these boxes are enough to cover the whole curved surface of the gray image, the amounts of boxes that intersect with the curved surface of the image should be $N_{\delta}(F)$ (Figure 1). $N_{\delta}(F)$ can be calculated as follows. The $M \times M$ gray image is divided into grids of $r \times r$ size in the $x-y$ plane ($I<r \le M/2$, $r$ being an integer), with a cube box of $r \times r \times r$ size existing in each grid. Let the minimum and maximum gray value of the image in the $(i, j)$ grid lie in the $k$ box and the $l$ box from bottom-up, respectively, there comes formula (4).

$$ n_r(i, j) = l - k + 1 \qquad (4) $$

In formula (4), $n_r(i, j)$ is the amounts of boxes that is needed to cover the image of the $(i, j)$ grid, and $N_r$ is the total boxes covering the whole image, which can be expressed in formula (5).

$$ N_r = \sum_{i,j} n_r(i, j) \qquad (5) $$
According to formula (5), different values of $N_r$ can be obtained with different values of $r$. Thus, the slope of $(\log(Nr), -\log(r))$, namely the box dimension $D$, can be fitted using linear least squares regression.

<figure><img src="image_3.png" /><figcaption>Fig. 1. The traditional box dimension</figcaption></figure>

In computer, the algorithm for the calculation of the fractal dimension of a gray image is as follows.

i. Binarization of the image. Namely, the value of matrix elements of the image is let to be either 1 or 0.

ii. Partition of the binary image. Within each parts of the image, its row = column=$K(K=1, 2, 4, ..., 2i)$. Thus the image is partitioned into parts of $2^i \times 2^i$, $2^{i-1} \times 2^{i-1}$, $2^{i-2} \times 2^{i-2}$, ..., $2^l \times 2^l$, $2^0 \times 2^0$, $2^i \le$ the length of the image.

iii. Calculation of the pixels. The amounts of image parts that contain the pixel “1” are calculated which is denoted as $N_K$, Thus a serious of values $N_1, N_2, ..., N_{(i+1)}$ as well as data pairs $(K, N_K)$ with the amounts of $(i+1)$, are obtained.

iv. Curve fitting. A straight line can be obtained using least squares method (-$\log K, \log N_K$).

v. Calculation of the fractal dimension of the image. The slope of the fitted line, namely the fractal dimension of the image, was determined.

In this method, as all of the cubes are located in a fixed position in the calculation of amounts of boxes $N_\delta(F)$, there are some disadvantages in the calculation of box dimensions. For example, there are situations that although the curves of some image surfaces are slight, they cover between two cubes. Therefore, the amounts of boxes covering this kind of surface are even more than the amounts of boxes covering image surfaces with larger curves. In this situation, there are some boxes that are not in set $F$. In the condition of $\delta \to 0$, this effect on the calculation of box dimension is negligible. But for digital images, $\delta$ is not always very small. In this condition, the calculation of box dimensions is affected as the surface is not totally covered by cubes. This situation might get even worse with even larger $\delta$ values.
## 2.2. An Improved Method for the Calculation of ‘box dimension’

To solve this problem, an improved method for the calculation of box dimensions was suggested in this paper, which eliminated the effects of ‘empty boxes’ in comparison with the traditional method, by covering the image surface with the least amount of δ-cubes. In the improved method, the cubes are not confined to a fixed position; instead, they can move along the z axis[8]. In this method, great improvement was achieved in the calculation of box dimensions due to reduced amounts of boxes as well as improvement in the tightness that the boxes cover the image surface. The essence of this improved method is that the cuboids of size $r×r×r'$ with variable heights are adopted to cover the image surface instead of the fixed-size cubes of size $r×r×r$. This method does not disobey the definition of box dimension; instead, it approaches even more close to the essence of the definition of box dimension in comparison with the traditional method for the improvement in the tightness that the boxes cover the image surface (Figure 2).

<figure><img src="image_4.png" /><figcaption>Fig. 2. An improved box dimension</figcaption></figure>

The realization process is as follows. The $M×M$ gray image is partitioned into grids of $r×r$ size in the x-y plane ($M^{1/3} ≤ r ≤ M/2$, r being an integer). Each grid encloses a series of boxes of $r×r×r'$ size, whose height $r'$ is a variable, as shown in Figure 2.

i. Calculation on the amounts of boxes within the (i, j) grid. Firstly, the serial number of the boxes in each pixel is tracked and scanned within the (i, j) grid; secondly, the gray value of each pixel is calculated within the (i, j) grid; finally, the statistical result, namely the set of $index_{i,j}$, can be obtained.

ii. Scan of the set $index_{i,j}$. In the scanning, elements that appear only once are totally adopted while elements that appear more than once are considered to be the same. A new set $index_{new}$ is used to record the serial number of different boxes. The set $index_{new}$ can be expressed in formula (6).

$$Index_{new} = \{index_{i,j}(1), index_{i,j}(2),..., index_{i,j}(Q)\} \quad (6)$$

iii. Let the amounts of elements in $Index_{new}$ be $Q$. It means that in the (i, j) grid, the amounts of boxes that cover curved surface of the image is $n_r(i, j) = Q$. Thus the amounts of boxes in all the grids are the sum of the amounts of boxes in each grid. It could be expressed in formula (5).

iv. Finally, the value of $N_r$ with different values of r could be calculated according to formula (5), and thus the fractal dimension $D$ of the image can be obtained according to formula (7).
$$D = \log N_r / \log(1/r) \qquad (7)$$

However, it should be pointed out that much costs would be needed in the calculation of the amounts of boxes in the (i, j) grid in comparison with the traditional method, which should be resolved to work more efficiently in this algorithm.

### 3. Detection of Surface Defects of Fruits Based On Fractal Dimension

This improved algorithm of differential box-counting can be used in fruit surface defects detection. Firstly, fruit image was collected and preprocessed (including graying, noise filtering). Secondly, an appropriate threshold was set, and then removal of the background and binarization of the isolated fruit image was performed.[9] Thirdly, fractal dimension *D* of the fruit image was calculated using the improved box dimension algorithm. Finally, the threshold of fractal dimension *D*<sub>thd</sub> was determined according to different kinds of fruits, which is used to determine whether the fruit has defects or not. The flow chart of this process is shown in Figure 3.

<figure><img src="image_5.png" /><figcaption>Fig. 3. Detection of fruit surface defects based on fractal dimension</figcaption></figure>
4. Detection of Surface Defects in Fuji Apple Fruits

To test whether this method is reliable in fruit defects detection, 50 Fuji apple
fruits that have surface defects (defective fruits) and 50 normal fruits were selected.
The experimental device was shown in Figure 4. The image acquisition card is DH-
CG410 (Daheng Company, China). The image size is 512 × 512 pixels. CCD camera
is WV-CP230 (Panasonic).

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Detection system configuration
1. Light box 2. Sample 3.CCD camera 4. Light sources 5. Computer (inside image acquisition card)</figcaption></figure>

In order to reflect the entire surface of the fruit and to eliminate the effect of
position and directions of the fruit on the detection results, eight images were
collected for each fruit from random directions. Totally 800 fruit images were
collected, which were numbered by letters plus numbers. For example, the 8 images
of sample 1 were numbered as A1, B1, C1, D1, E1, F1, G1, and H1, respectively. The
representative gray images of 6 samples were shown in Figure 5 and the fractal
dimensions derived from this improved method were shown in Table 1.

<figure><img src="image_7.png" /><figcaption>Fig. 5. Gray image of 6 Fuji apple fruits</figcaption></figure>
<table><caption>Table 1. Fractal dimensions of 6 Fuji apples</caption><thead><tr><th rowspan="2">Samples</th><th colspan="3">Normal fruits</th><th colspan="3">Defective fruits</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th></tr></thead><tbody><tr><th>Images</th><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><th>A</th><td>1.186</td><td>1.299</td><td>1.178</td><td>1.355</td><td>1.412</td><td>1.499</td></tr><tr><th>B</th><td>1.073</td><td>1.291</td><td>1.117</td><td>1.387</td><td>1.358</td><td>1.438</td></tr><tr><th>C</th><td>1.129</td><td>1.261</td><td>1.132</td><td>1.304</td><td>1.435</td><td>1.456</td></tr><tr><th>D</th><td>1.129</td><td>1.284</td><td>1.142</td><td>1.372</td><td>1.450</td><td>1.491</td></tr><tr><th>E</th><td>1.129</td><td>1.284</td><td>1.142</td><td>1.304</td><td>1.404</td><td>1.496</td></tr><tr><th>F</th><td>1.073</td><td>1.261</td><td>1.144</td><td>1.355</td><td>1.412</td><td>1.538</td></tr><tr><th>G</th><td>1.126</td><td>1.299</td><td>1.148</td><td>1.372</td><td>1.400</td><td>1.456</td></tr><tr><th>H</th><td>1.188</td><td>1.269</td><td>1.175</td><td>1.371</td><td>1.341</td><td>1.444</td></tr><tr><th>Maximum</th><td>1.188</td><td>1.299</td><td>1.178</td><td>1.387</td><td>1.450</td><td>1.538</td></tr><tr><th>Minimum</th><td>1.073</td><td>1.261</td><td>1.117</td><td>1.304</td><td>1.341</td><td>1.438</td></tr><tr><th>Difference between maximum and minimum</th><td>0.115</td><td>0.038</td><td>0.060</td><td>0.083</td><td>0.109</td><td>0.100</td></tr><tr><th>Mean of each fruit</th><td>1.129</td><td>1.281</td><td>1.147</td><td>1.352</td><td>1.401</td><td>1.477</td></tr><tr><th>Mean between normal and defective fruits</th><td></td><td>1.186</td><td></td><td></td><td>1.410</td><td></td></tr></tbody></table>

Table 1 shows that for the same sample, there was no much difference between the maximum and the minimum box dimensions, indicating that this improved box-counting method was reliable.

The fractal dimension threshold $D_{thd}$ is set to be 1.30 based on this experiment. According to this value, the detection results of the selected 100 apples are shown in Table 2.

<table><caption>Table 2. Detection results of selected 100 Fuji apples</caption><thead><tr><th>Samples</th><th>Results</th><th>Accuracy(%)</th></tr></thead><tbody><tr><td>Normal fruits (50)</td><td>48</td><td>96.0</td></tr><tr><td>Defective fruits (50)</td><td>49</td><td>98.0</td></tr><tr><td>Average accuracy(%)</td><td></td><td>97.0</td></tr></tbody></table>
**5. Conclusion and Discussion**

This method of surface defects detection on fruits based on fractal dimension provides a new way in fruit automatic detection. The fractal dimension of the fruit image reflects the defective area as well as the characteristics of its spatial distribution. The experimental results with Fuji apples showed that this algorithm was effective in the identification of fruit defects. In practice, a database that contains the fractal characteristics of different kinds of fruits should be established. It should be pointed out that the effects of fruit stem on the detection results in this method should be considered in future research.[10]

**References**

1. Blasco J., Aleixos N., Moltó E.: Machine vision system for automatic quality grading of fruit. Biosystems Engineering, 85(4), pp. 415-423(2003)

2. Nirupam S., B.B.chaudhuri: An efficient differential box-counting approach to compute fractal dimension of image. IEEE Trans SMC, 24(1), pp. 115-120(1994)

3. Njoroge, J. B., Ninomiya K., Kondo N., et al: Automated fruit grading system using image processing. SICE2002. Proceedings of the 41st SICE Annual Conference, 5-7 Aug, pp. 1346-1351(2002)

4. Falconer K. J.: Techniques in Fractal Geometry. Chichester: John Wiley and Sons Ltd(1996)

5. Xie, H., Wang, J. A.: Direct Fractal Measurement of Fracture Surfaces. Int J Solids & Structures 36, pp. 3073-3084(1999)

6. B.B.chaudhuri, Nirupam Sarkar: Texture segmentation using fractal dimension. Trans on Pattern Analysis and Machine Intelligence,17(1), pp. 72-77(1995)

7. T. Ojala, M. Pietikainen, D Harwood: A comparative study of texture measures with classification based feature distributions. Pattern Recognition, 29(1), pp. 51-59(1996)

8. Zhang, T., Yang, Z.B., Huang, A. M.: Improved Extracting Algorithm of Fractal dimension of Remote Sensing Image. Journal of Ordnance Engineering College, Vol·18 No·5, pp.61-65. (in Chinese)(2006)

9. Lin, K.Y., Wu J.H, Xu, L.H.: Separation approach for shape grading of fruits using computer vision. Transactions of the CSAM, 36(6), pp. 71-74. (in Chinese) (2005)

10.Cai J.R., Xu Y.M.: Identification and classification of apple shape based on active shape models. Transactions of the CSAE, 22(6), pp. 123-126. (in Chinese)(2006)