# Nondestructive Quantification of the Ripening Process in Banana (Musa AAB Simmonds) Using Multispectral Imaging

Authors: Mauro Santoyo-Mora; Agustin Sancen-Plaza; Alejandro Espinosa-Calderon; Alejandro Israel Barranco-Gutierrez; Juan Prado-Olivarez
Journal: Journal of Sensors
 (2019)

---

## Abstract

The ripening process in bananas causes the waste of a significant part of the production of this fruit. The aim of this research is to find a new technique useful for identifying, registering, and quantifying the ripening process of a banana (Musa AAB Simmonds) at the seventh stage of the growing process. This quantification is proposed with a nondestructive technique based on processing multispectral images. This experiment used a set of multispectral imagery registered in a range of 270-1000 nm (from UV to IR) with the aid of a monochromatic camera and a set of 10 optical filters. Multispectral images were analyzed with three different techniques: Fourier fractal analysis, Hotelling transform, and homogeneity texture analysis based on cooccurrence matrix. First, a characteristic index was computed for each technique for a daily set of multispectral imagery. These indexes are slope index, for Fourier fractals; the average of the computed eigenvalues, with Hotelling transform; and the texture homogeneity value. These indexes were evaluated using the behavior of the resulting graphs for a seven-day period, being preferred those graphs with a tendency of decreasing values. Finally, the repeatability of each technique was evaluated by reproducing similar values for each day during the evaluation period. These three methods will be compared in this article in order to select the one with the best performance for measuring the ripening process in bananas. The obtained results show that it is possible to effectively isolate the brown spots from the banana peel with the Hotelling transform by using only 2 optical filters: visible (410-690 nm) and Near-IR (820-910 nm). With the resulting spectral image fusion it was possible to effectively describe the evolution of the brown spots present in the ripening process through the texture homogeneity criteria.


---

## Body

## 1. Introduction

In Mexico, the banana is one of the most important crops in agriculture occupying the first place when we refer to the production of tropical fruits. This fruit is considered part of the basic feeding for Mexican people due to its low price, all-year availability, and good nutrition facts, contributing with potassium, iron, and K vitamin. The annual worldwide banana production is likely to be around 12 million tons from which the Latin-American and Caribbean countries produce 10 million tons, making it an important source of employment and earnings for growing countries. Actually, Mexico plants 769.3 km2 (4.8% of the available area) of bananas counting with a suitable area of 16,299 km2 for this type of plantation [1]. This fruit can be harvested all over the year when it is completely ripe and shows a yellowish skin. Most of the times, mainly during the winter, the harvesting is anticipated letting the bananas grow hanging in a dry, warm, dark, and closed storage. The short life of the banana is caused by its fast ripening process, degrading the visual appearance that starts with a greenish yellow peel and finishing with a yellow peel with brown spots [2]. Banana ripeness is visually assessed by comparing the color of the peel with standardized color charts.Fruit quality is defined as a measure of characteristics or attributes that determine the suitability of the fruit to be eaten as fresh or to be stored for reasonable period without deterioration. The quality of a fresh product includes appearance (size, shape, color, gloss, and freedom from defects and decay); texture (firmness, crispness, and toughness); flavor (sweetness, sourness, aroma, and off-flavors); and nutritive value (vitamins, minerals, nutrients, and carbohydrates) [3]. Fruit internal quality attributes such as dry matter content, total soluble solids content, sugar content, and juice acidity are very important in the modern quality evaluation industries. Most of the current instrumental techniques that measure these properties are destructive and involve a considerable amount of manual work. The development of nondestructive measurements of these quality attributes is convenient for producers, processors, and distributors to ascertain fast evaluation.Such standardized color evolution in banana is defined in seven stages (Figure1), starting in a greenish yellow (stage 1) and ending in a yellow peel with brown spots (stage 7) [4]. The evolution of all the stages are shown in Figure 1 and described in Table 1. Sometimes, banana maturation can be assessed by instrumental techniques; however, instrumental colorimetric techniques are difficult to apply since they give average color values and they do not quantify spotting [5].Table 1
Description of the banana ripening process.
Stage
Peel color
1
Green
2
Green with some yellow
3
Green more dominant than yellow
4
Yellow more dominant than green
5
Yellow with green tip
6
Completely yellow
7
Yellow, flecked with brownFigure 1
Seven stages of the banana ripening process.At the 7th stage, where spotting starts, the fruit has a good taste and aroma [6, 7]. For export markets, the spotting, also called senescent spotting, is typical of overripened bananas which already have a yellow peel as a hallmark for good eatable quality; therefore, this stage must be prevented [5]. Most of the fruits and vegetables show their ripening process by transforming its color and texture; with this manifestation is how producers decide the different stages for harvesting, storage, transport, and sale the product.There are some nondestructive applications based on image analyses which are useful to quantifying regions with texture analysis techniques. Such applications have proved to be effective for the quantification and classification of food quality [8] and plant diseases [9]. Therefore texture analysis techniques become a good alternative to be used for measuring food quality. This is possible because the image texture reflects the changes in intensity pixel values, which might contain information about the color and the geometric structure of objects. In fact, the food industry is one of the top 10 industries that regularly use computer vision and image texture [10].Image analysis in different color spaces has been used before for quantifying the ripening process in bananas. Color space Red, Green, and Blue (RGB) representation of banana image was used with an artificial neural network (ANN) to build automatic classification systems [11, 12].CIEL
∗
a
∗
b
∗ (CIE) color space is widely used, alone or in combination with other color spaces. Mendoza & Aguilera [13] used the average color of each channel in CIE color space, in combination with brown area percentage (% BSA), homogeneity, contrast, correlation, and entropy of image texture. An exhaustive search was performed in order to identify which of these features are suitable for ripening classification. However, the authors conclude that the algorithms need to be optimized in order to improve performance for real-time applications.Santos Gomes et al. [14] proposed the use of a general equation to determine a ripening color index (RCI). This index has a direct correlation with ripening seven stages. In RCI, a constant was empirically obtained in combination with mean values ofa
∗ andb
∗ channels. RCI was used to analyze the ripening process ofMusa sapientum AAB with 92% of correct classification,but they concluded that constant must be obtained experimentally for other banana species.Color spaces HSV and CIE were used by Marimuthu and Roomi [15] to formulate fuzzy model for banana classification in unripe, ripe, and overripe stages. Hue channel (H) in HSV color was used because it represents the true color of banana peels. Additionally, a scale invariance unsupervised k-means clustering technique was performed in order to separate the senescent spots (brown color) from the banana peel usinga
∗ andb
∗. The classification performance was 93.11%. Nevertheless, there are no correspondence between standard seven stage and three stage classification systems proposed by authors.Quevedo et al. [5] proposed the application of fractal texture analysis as an indicator of senescent spotting in banana. They used a computerized image system implemented with a RGB digital camera, to afterwards transform the registered image into its corresponding grayscale image. A surface intensity (SI) was generated from each banana image by plotting pixel coordinates (x,y) against their gray level in the z-axis. This SI was quantified by using fractal theory [5].Prabha and Kuhmar [16] identified maturity by color and size features obtained from segmented banana images acquired under controlled lighting conditions. The color feature was obtained from the mean color intensity in image histogram. Size was computed by four image features: area, perimeter, major axis length, and minor axis length. Two classifiers were designed, one for the color and the other for size features. Color and size classification algorithms showed a 99.1% accuracy in mature bananas and 85% accuracy when bananas are undermature.This trend offers a great opportunity, but also an enormous challenge, to technologies related to spectroscopic and hyperspectral imaging [3]. Traditional optical sensing techniques, such as imaging and spectroscopy, have limitations to acquire adequate spatial and spectral information for nondestructive evaluation of food and agricultural products. Generally, conventional imaging cannot acquire spectral information, and spectroscopy measurement cannot cover large sample areas. However, in recent years spectral imaging has emerged as a better tool for safety and quality inspection of various agricultural commodities. Spectral imaging technique combines conventional imaging and spectroscopy techniques. Besides, it enables obtaining both spatial and spectral information from the target, which is extremely useful for evaluating individual food items. This technique has drawn tremendous interest from both academic and industrial areas and has been developed rapidly during the past decade.Qin et al. [10] report the importance of the fusion of conventional imaging and spectroscopy techniques with the objective of developing new applications that register information from different biological processes in which are involved food and agricultural products. For this purpose, bandpass filters, with the nearest central wavelengths to the identified important bands, are used in these area-scan systems to obtain narrowband images. The filters can be physically changed to different central wavelengths and bandwidths for different applications. Some applications using this type of technique, summarized in [10], show the analysis for feces detection (450, 550, 680, 730 nm); surface defects (450, 500, 740, 750, 800, 950 nm) and firmness evaluation (680, 880, 905, 940 nm) in apples; canker detection for citrus (730, 830 nm); maturity evaluation for peach (450, 675, 800 nm); category sorting for tea (580, 680, 800 nm); and maturity evaluation for tomato (530, 595, 630, 850 nm).Derived from the spectral imaging emerges the multispectral imaging, which aims to acquire spatial and spectral information that are directly useful for real-time applications in the field (e.g., fruit packing houses and food processing plants). The process generally involves fast image acquisition and relatively simple algorithms for image processing and decision making. In practice, this means acquiring images with relatively low spatial resolutions at a few important wavelengths [10].So, as it has been explained, image analysis has been used before for quantifying the ripening process in bananas. In spite of the high percentages of certainty reached by these techniques, the referenced articles present high complexity in their algorithms or experimental set ups, which block their effectivity in real-time applications. Besides, we noticed that none of these previous techniques used multispectral imaging. Therefore, this article aims to find a new technique, based on multispectral imaging, useful for identifying, registering, and quantifying the ripening process of a banana (Musa AAB Simmonds) from the seventh stage of the growing process.


## 2. Materials and Methods

### 2.1. Experimental Setup

The experimental setup was constituted by the elements showed in Figure2. The vision system was a monochromatic camera model UI-2210 from the manufacturer IDS. This camera uses a (1/2)” CCD monosensor with a resolution of 640 × 480 pixels with a 400 nm to 900 nm spectral range, showing a spectral sensibility of 1.0 at 500 nm. The communication between the camera and the computer was established via USB connection, which also worked for voltage supply [18]. The setup used ten optical filters from Midwest Optical Systems [17] which are listed according to their corresponding spectral band range in Table 2. The multispectral setup counted with an isolation box and an illumination system built with a set of white light LEDs that emitted an average intensity of 700 mcd and a domain wavelength of 700 nm [19]. The vision system used a black mate wooden chart as background of the scene for avoiding a segmentation process in the analysis of the captured multispectral imagery. It is important to remark that the direction of the sample in the plane is not important, because the proposed methods are rotation invariant.Table 2
Spectral band range of the optical filters for the multispectral images setup [17].
No. of filter
Description
Passes
1
Near-UV bandpass
UV portions 320-380 nm
and 280-320 nm
2
Blue bandpass
425-495 nm
3
Light green bandpass
550-555 nm
4
Near-IR/UV-block, visible bandpass
410-690 nm
5
Orange bandpass
560-600 nm
6
Light red bandpass
615-645 nm
7
Dark red bandpass
640-680 nm
8
Near-IR bandpass
820-910 nm
9
Linear polarizer
400-700 nm
10
Light balancing filter (minus blue)
400-700 nmFigure 2
Experimental setup for multispectral images.


### 2.2. The Experimental Organism

The specimen used in this work was the bananaMusa AAB Simmonds. The experiment used 7 hands of bananas, from where 1 individual specimen was taken from each hand. This means 7 bananas in total. All the bananas were selected at the end of the 5th stage, or nearly beginning of the 6th stage, that is when the banana peel starts to become completely yellow (see Figure 1).


### 2.3. Theory

#### 2.3.1. Flat-Field Correction

Flat-field correction is a common preprocessing method for reflectance measurements. The correction can be conducted using the following equation:(1)
R
s
λ
=
I
s
λ
-
I
d
λ
I
r
λ
-
I
d
λ
×
R
r
λwhere R
s is the relative reflectance image of the sample, I
s is the intensity image of the sample, I
r is the reference image of the white panel, I
d is the dark current image acquired with the light source off and the lens covered, R
r is the reflectance factor of the white panel, and λis the wavelength [10].


#### 2.3.2. Fourier Fractal Analysis

Fractal refers to entities, especially sets of pixels, which display a degree of self-similarity at a different scale. The fractal dimension for a rough surface is higher than the dimension for a smooth one. A fractal dimension (F
D
h) is determined from the Fourier power spectrum of the image data [20]. After making the fast Fourier transform of the image of interest in the horizontal directions, the average horizontal power spectrum F
h of the surface is a function of the frequency (f) and satisfies the following relationships:(2)
F
h
∝
f
2
-
β
hand (3)
β
h
=
2
·
H
n
+
2where H
n is the Hurst coefficient. If a linear variation is established from the log(magnitude Fourier coefficients)
2 vs. log(frequencies), then a F
D
h value can be calculated as (4)
F
D
h
=
6
+
β
h
2where β
h is the slope of the linear portion of the graph [6].


#### 2.3.3. Hotelling Transform

Suppose that we are givenn registered images of size M
×
N. There are n pixels for each coordinate i
,
j with a corresponding value for each registered image. These pixels are represented individually as a n-dimensional vector:(5)
x
=
x
1
x
2
⋯
x
n
TThe mean vector for this population is defined as(6)
m
x
=
1
K
∑
k
=
1
K
x
kwith K
=
M
N. For this same population, the covariance matrix can be approximated from the samples as follows:(7)
C
x
=
1
K
∑
k
=
1
K
x
k
x
k
T
-
m
x
m
x
TBecause C
x is real and symmetric, finding a set of n orthonormal eigenvectors always are possible. Let e
i and λ
i, i
=
1
,
2
,
…
,
n, be the eigenvectors and corresponding eigenvalues of C
x, arranged in descending order so that λ
j
≥
λ
j
+
1 for j
=
1
,
2
,
…
,
n
-
1. Let A be a matrix whose rows are formed from the eigenvectors of C
x, ordered so that the first row of A is the eigenvector corresponding to the smallest eigenvalue.Now, suppose that we useA as a transformation matrix to map the xs into vectors denoted by ys, as follows:(8)
y
=
A
x
-
m
xThis expression is called the Hotelling transform. An important property of the Hotelling transform deals with the reconstruction of x from y. Because the rows of A are orthonormal vectors, which followsA
-
1
=
A
T, and any vector x can be recovered from its corresponding y by using the expression x
=
A
T
y
+
m
x [21].In multispectral imagery, a pixel is a set ofn different intensities in each spectral image: (9)
x
i
j
=
λ
1
⋯
λ
kwhere i
,
j represents the position of the pixel in the k-th spectral image.As a first approximation for quantifying the spots growing, it was proposed to calculate the eigenvalues average obtained with the respective covariance for each combination, but also was proposed a qualitative analysis of the Hotelling transform by computing a new image obtained as(10)
y
=
A
x
-
m
xwhere y is the resulting image, x represents one of the spectral images combined, m
x is the mean vector, and A is the matrix of the eigenvectors.


#### 2.3.4. Texture Analysis Computing the Homogeneity Criteria Based on Cooccurrence Matrix

An important approach to region description is to quantify its texture content. Intuitively this descriptor provides measures of properties such as smoothness, coarseness, and regularity. One of the principal approaches used in image processing to describe texture of a region is statistics. A measure of texture with statistics is possible by considering both the distribution of intensities and the relative positions of pixels in an image.LetQ be an operator that defines the position of two pixels relative to each other, and consider an image, f, with L possible intensity levels. Let G be a matrix whose element g
i
j is the number of times that pixel pairs with intensities z
i and z
j occur in f in the position specified by Q, where 1
≤
i
,
j
≤
L. A matrix formed in this manner is referred to as a gray-level cooccurrence matrix. The total number, n, of pixel pairs that satisfy Q is equal to the sum of the elements of G. Then, the quantity(11)
p
i
j
=
g
i
j
nis an estimate of the probability that a pair of points satisfying Q will have values z
i
,
z
j. There is a set of descriptors useful for characterizing the contents of G as maximum probability, correlation, contrast, uniformity (also called energy), entropy, and homogeneity. From these descriptors we take the homogeneity criterion. Such criterion is defined as the measure of the spatial closeness of the distribution of elements in G to the diagonal. The homogeneity is computed as(12)
∑
i
=
1
K
∑
i
=
1
K
p
i
j
1
+
i
-
jwhere K is the size of the cooccurrence matrix [21].


### 2.4. Methodology

Two experiments were performed. Both experiments followed the same methodology, but with different levels of room temperature and relative humidity (see Table3).Table 3
Conditions for the experiments.
Number of specimens
Room temperature
Room humidity
Experiment I
5
19.9°C
68.5%
Experiment II
2
13.3°C
51.4%The evaluation of the brown spots during the ripening process was performed through the following methodology.(a) The selected specimens were placed in a wood chart for a period of 7 days. Each specimen was moved to the front of the camera for a daily capture of the multispectral imagery.(b) Since the setup includes ten different optical filters, the daily database consisted of ten images per specimen, one image with each filter. This resulted in 70 multispectral images per specimen at the end of the 7 days. A total of 490 images were processed.(c) Every multispectral imagery in this work was preprocessed with a flat-field correction, using (1), in order to remove undesired signals and collect the useful data.(d) In order to measure the ripening process, quantification of the growing of the brown spots in the peel was carried out. Three techniques were selected for quantification, Fourier fractal analysis, Hotelling transform, and texture analysis computing the homogeneity criteria based on cooccurrence matrix.These methods were selected because they all are useful for determining the degree of similarity of a set of pixels [5, 7]. These methods will be compared in this article in order to select the one with the best performance for measuring the ripening process in bananas.


## 2.1. Experimental Setup

The experimental setup was constituted by the elements showed in Figure2. The vision system was a monochromatic camera model UI-2210 from the manufacturer IDS. This camera uses a (1/2)” CCD monosensor with a resolution of 640 × 480 pixels with a 400 nm to 900 nm spectral range, showing a spectral sensibility of 1.0 at 500 nm. The communication between the camera and the computer was established via USB connection, which also worked for voltage supply [18]. The setup used ten optical filters from Midwest Optical Systems [17] which are listed according to their corresponding spectral band range in Table 2. The multispectral setup counted with an isolation box and an illumination system built with a set of white light LEDs that emitted an average intensity of 700 mcd and a domain wavelength of 700 nm [19]. The vision system used a black mate wooden chart as background of the scene for avoiding a segmentation process in the analysis of the captured multispectral imagery. It is important to remark that the direction of the sample in the plane is not important, because the proposed methods are rotation invariant.Table 2
Spectral band range of the optical filters for the multispectral images setup [17].
No. of filter
Description
Passes
1
Near-UV bandpass
UV portions 320-380 nm
and 280-320 nm
2
Blue bandpass
425-495 nm
3
Light green bandpass
550-555 nm
4
Near-IR/UV-block, visible bandpass
410-690 nm
5
Orange bandpass
560-600 nm
6
Light red bandpass
615-645 nm
7
Dark red bandpass
640-680 nm
8
Near-IR bandpass
820-910 nm
9
Linear polarizer
400-700 nm
10
Light balancing filter (minus blue)
400-700 nmFigure 2
Experimental setup for multispectral images.


## 2.2. The Experimental Organism

The specimen used in this work was the bananaMusa AAB Simmonds. The experiment used 7 hands of bananas, from where 1 individual specimen was taken from each hand. This means 7 bananas in total. All the bananas were selected at the end of the 5th stage, or nearly beginning of the 6th stage, that is when the banana peel starts to become completely yellow (see Figure 1).


## 2.3. Theory

### 2.3.1. Flat-Field Correction

Flat-field correction is a common preprocessing method for reflectance measurements. The correction can be conducted using the following equation:(1)
R
s
λ
=
I
s
λ
-
I
d
λ
I
r
λ
-
I
d
λ
×
R
r
λwhere R
s is the relative reflectance image of the sample, I
s is the intensity image of the sample, I
r is the reference image of the white panel, I
d is the dark current image acquired with the light source off and the lens covered, R
r is the reflectance factor of the white panel, and λis the wavelength [10].


### 2.3.2. Fourier Fractal Analysis

Fractal refers to entities, especially sets of pixels, which display a degree of self-similarity at a different scale. The fractal dimension for a rough surface is higher than the dimension for a smooth one. A fractal dimension (F
D
h) is determined from the Fourier power spectrum of the image data [20]. After making the fast Fourier transform of the image of interest in the horizontal directions, the average horizontal power spectrum F
h of the surface is a function of the frequency (f) and satisfies the following relationships:(2)
F
h
∝
f
2
-
β
hand (3)
β
h
=
2
·
H
n
+
2where H
n is the Hurst coefficient. If a linear variation is established from the log(magnitude Fourier coefficients)
2 vs. log(frequencies), then a F
D
h value can be calculated as (4)
F
D
h
=
6
+
β
h
2where β
h is the slope of the linear portion of the graph [6].


### 2.3.3. Hotelling Transform

Suppose that we are givenn registered images of size M
×
N. There are n pixels for each coordinate i
,
j with a corresponding value for each registered image. These pixels are represented individually as a n-dimensional vector:(5)
x
=
x
1
x
2
⋯
x
n
TThe mean vector for this population is defined as(6)
m
x
=
1
K
∑
k
=
1
K
x
kwith K
=
M
N. For this same population, the covariance matrix can be approximated from the samples as follows:(7)
C
x
=
1
K
∑
k
=
1
K
x
k
x
k
T
-
m
x
m
x
TBecause C
x is real and symmetric, finding a set of n orthonormal eigenvectors always are possible. Let e
i and λ
i, i
=
1
,
2
,
…
,
n, be the eigenvectors and corresponding eigenvalues of C
x, arranged in descending order so that λ
j
≥
λ
j
+
1 for j
=
1
,
2
,
…
,
n
-
1. Let A be a matrix whose rows are formed from the eigenvectors of C
x, ordered so that the first row of A is the eigenvector corresponding to the smallest eigenvalue.Now, suppose that we useA as a transformation matrix to map the xs into vectors denoted by ys, as follows:(8)
y
=
A
x
-
m
xThis expression is called the Hotelling transform. An important property of the Hotelling transform deals with the reconstruction of x from y. Because the rows of A are orthonormal vectors, which followsA
-
1
=
A
T, and any vector x can be recovered from its corresponding y by using the expression x
=
A
T
y
+
m
x [21].In multispectral imagery, a pixel is a set ofn different intensities in each spectral image: (9)
x
i
j
=
λ
1
⋯
λ
kwhere i
,
j represents the position of the pixel in the k-th spectral image.As a first approximation for quantifying the spots growing, it was proposed to calculate the eigenvalues average obtained with the respective covariance for each combination, but also was proposed a qualitative analysis of the Hotelling transform by computing a new image obtained as(10)
y
=
A
x
-
m
xwhere y is the resulting image, x represents one of the spectral images combined, m
x is the mean vector, and A is the matrix of the eigenvectors.


### 2.3.4. Texture Analysis Computing the Homogeneity Criteria Based on Cooccurrence Matrix

An important approach to region description is to quantify its texture content. Intuitively this descriptor provides measures of properties such as smoothness, coarseness, and regularity. One of the principal approaches used in image processing to describe texture of a region is statistics. A measure of texture with statistics is possible by considering both the distribution of intensities and the relative positions of pixels in an image.LetQ be an operator that defines the position of two pixels relative to each other, and consider an image, f, with L possible intensity levels. Let G be a matrix whose element g
i
j is the number of times that pixel pairs with intensities z
i and z
j occur in f in the position specified by Q, where 1
≤
i
,
j
≤
L. A matrix formed in this manner is referred to as a gray-level cooccurrence matrix. The total number, n, of pixel pairs that satisfy Q is equal to the sum of the elements of G. Then, the quantity(11)
p
i
j
=
g
i
j
nis an estimate of the probability that a pair of points satisfying Q will have values z
i
,
z
j. There is a set of descriptors useful for characterizing the contents of G as maximum probability, correlation, contrast, uniformity (also called energy), entropy, and homogeneity. From these descriptors we take the homogeneity criterion. Such criterion is defined as the measure of the spatial closeness of the distribution of elements in G to the diagonal. The homogeneity is computed as(12)
∑
i
=
1
K
∑
i
=
1
K
p
i
j
1
+
i
-
jwhere K is the size of the cooccurrence matrix [21].


## 2.3.1. Flat-Field Correction

Flat-field correction is a common preprocessing method for reflectance measurements. The correction can be conducted using the following equation:(1)
R
s
λ
=
I
s
λ
-
I
d
λ
I
r
λ
-
I
d
λ
×
R
r
λwhere R
s is the relative reflectance image of the sample, I
s is the intensity image of the sample, I
r is the reference image of the white panel, I
d is the dark current image acquired with the light source off and the lens covered, R
r is the reflectance factor of the white panel, and λis the wavelength [10].


## 2.3.2. Fourier Fractal Analysis

Fractal refers to entities, especially sets of pixels, which display a degree of self-similarity at a different scale. The fractal dimension for a rough surface is higher than the dimension for a smooth one. A fractal dimension (F
D
h) is determined from the Fourier power spectrum of the image data [20]. After making the fast Fourier transform of the image of interest in the horizontal directions, the average horizontal power spectrum F
h of the surface is a function of the frequency (f) and satisfies the following relationships:(2)
F
h
∝
f
2
-
β
hand (3)
β
h
=
2
·
H
n
+
2where H
n is the Hurst coefficient. If a linear variation is established from the log(magnitude Fourier coefficients)
2 vs. log(frequencies), then a F
D
h value can be calculated as (4)
F
D
h
=
6
+
β
h
2where β
h is the slope of the linear portion of the graph [6].


## 2.3.3. Hotelling Transform

Suppose that we are givenn registered images of size M
×
N. There are n pixels for each coordinate i
,
j with a corresponding value for each registered image. These pixels are represented individually as a n-dimensional vector:(5)
x
=
x
1
x
2
⋯
x
n
TThe mean vector for this population is defined as(6)
m
x
=
1
K
∑
k
=
1
K
x
kwith K
=
M
N. For this same population, the covariance matrix can be approximated from the samples as follows:(7)
C
x
=
1
K
∑
k
=
1
K
x
k
x
k
T
-
m
x
m
x
TBecause C
x is real and symmetric, finding a set of n orthonormal eigenvectors always are possible. Let e
i and λ
i, i
=
1
,
2
,
…
,
n, be the eigenvectors and corresponding eigenvalues of C
x, arranged in descending order so that λ
j
≥
λ
j
+
1 for j
=
1
,
2
,
…
,
n
-
1. Let A be a matrix whose rows are formed from the eigenvectors of C
x, ordered so that the first row of A is the eigenvector corresponding to the smallest eigenvalue.Now, suppose that we useA as a transformation matrix to map the xs into vectors denoted by ys, as follows:(8)
y
=
A
x
-
m
xThis expression is called the Hotelling transform. An important property of the Hotelling transform deals with the reconstruction of x from y. Because the rows of A are orthonormal vectors, which followsA
-
1
=
A
T, and any vector x can be recovered from its corresponding y by using the expression x
=
A
T
y
+
m
x [21].In multispectral imagery, a pixel is a set ofn different intensities in each spectral image: (9)
x
i
j
=
λ
1
⋯
λ
kwhere i
,
j represents the position of the pixel in the k-th spectral image.As a first approximation for quantifying the spots growing, it was proposed to calculate the eigenvalues average obtained with the respective covariance for each combination, but also was proposed a qualitative analysis of the Hotelling transform by computing a new image obtained as(10)
y
=
A
x
-
m
xwhere y is the resulting image, x represents one of the spectral images combined, m
x is the mean vector, and A is the matrix of the eigenvectors.


## 2.3.4. Texture Analysis Computing the Homogeneity Criteria Based on Cooccurrence Matrix

An important approach to region description is to quantify its texture content. Intuitively this descriptor provides measures of properties such as smoothness, coarseness, and regularity. One of the principal approaches used in image processing to describe texture of a region is statistics. A measure of texture with statistics is possible by considering both the distribution of intensities and the relative positions of pixels in an image.LetQ be an operator that defines the position of two pixels relative to each other, and consider an image, f, with L possible intensity levels. Let G be a matrix whose element g
i
j is the number of times that pixel pairs with intensities z
i and z
j occur in f in the position specified by Q, where 1
≤
i
,
j
≤
L. A matrix formed in this manner is referred to as a gray-level cooccurrence matrix. The total number, n, of pixel pairs that satisfy Q is equal to the sum of the elements of G. Then, the quantity(11)
p
i
j
=
g
i
j
nis an estimate of the probability that a pair of points satisfying Q will have values z
i
,
z
j. There is a set of descriptors useful for characterizing the contents of G as maximum probability, correlation, contrast, uniformity (also called energy), entropy, and homogeneity. From these descriptors we take the homogeneity criterion. Such criterion is defined as the measure of the spatial closeness of the distribution of elements in G to the diagonal. The homogeneity is computed as(12)
∑
i
=
1
K
∑
i
=
1
K
p
i
j
1
+
i
-
jwhere K is the size of the cooccurrence matrix [21].


## 2.4. Methodology

Two experiments were performed. Both experiments followed the same methodology, but with different levels of room temperature and relative humidity (see Table3).Table 3
Conditions for the experiments.
Number of specimens
Room temperature
Room humidity
Experiment I
5
19.9°C
68.5%
Experiment II
2
13.3°C
51.4%The evaluation of the brown spots during the ripening process was performed through the following methodology.(a) The selected specimens were placed in a wood chart for a period of 7 days. Each specimen was moved to the front of the camera for a daily capture of the multispectral imagery.(b) Since the setup includes ten different optical filters, the daily database consisted of ten images per specimen, one image with each filter. This resulted in 70 multispectral images per specimen at the end of the 7 days. A total of 490 images were processed.(c) Every multispectral imagery in this work was preprocessed with a flat-field correction, using (1), in order to remove undesired signals and collect the useful data.(d) In order to measure the ripening process, quantification of the growing of the brown spots in the peel was carried out. Three techniques were selected for quantification, Fourier fractal analysis, Hotelling transform, and texture analysis computing the homogeneity criteria based on cooccurrence matrix.These methods were selected because they all are useful for determining the degree of similarity of a set of pixels [5, 7]. These methods will be compared in this article in order to select the one with the best performance for measuring the ripening process in bananas.


## 3. Results and Discussion

Seven specimens of banana were used in the seven days during which the experiment lasted. Ten images were taken, daily, from each specimen, one per each optical filter. Therefore, a total of 490 images were processed. Figure3 shows an example of how the banana ripening process advanced during the seven days of the experiment.Figure 3
Example of the imagery used during the 7 days that lasted the experiment. (a) Color image of the specimen. (b) Grayscale imagery from a selected spectral region for the same specimen.
(a)
(b)


### 3.1. Fourier Fractal Analysis

The spectral image of interest for this analysis was captured with the orange bandpass filter of 560-600 nm (see filter number 5 in Table2), which allows the transference of the orange color. This was the nearest wavelength to the brown spotting shown on the banana peel at the seventh stage. This technique searches for the slope in the Fourier spectrum where it is perceptible a change in the different intensity levels as the brown spots grow all over the banana peel. First of all, it is needed to stand out the brown spots from the registered spectral image (Figure 4(a)) by computing its negative image N (Figure 4(b)).Figure 4
Spectral image with its negative. (a) Original. (b) Negative.
(a)
(b)From the resulting negative imageN is obtained its fast Fourier transform (FFT), from which it is necessary to compute the magnitude for each coefficient. This new information, as well as the spatial frequency, is logarithmically evaluated for plotting l
o
g
f vs. l
o
g
F
u
,
v
2 as shown in Figure 5.Figure 5
Graph of logarithms for frequency vs. magnitude.After analyzing different ranges with increasing values of 0.1 for similar graphs to Figure5, it was found that in the range of 1.7-1.8, for the axis of l
o
g
f, the computed data could behave as a linear portion, from which the slope would be useful for quantifying the growing of the brown spots in the peel. Such hypothesis is true if we consider that, as the covered area by the spots grows, there will be an increment in the values of the intensity as well as the computed values from the FFT. If this is possible, an increment or decrement in the value of the slope is expected to allow a process for quantifying the growing of the brown spots.The present Fourier fractal analysis is proposed in order to identify the ripening process of the banana by quantifying the growth of the spots based on a slope in a closed region near the maximum value of the magnitude, where the most significant intensity changes are present. As an example of this analysis, Figure6(b) shows the evaluated images of a specimen, which correspond to the negative image of the wavelengths captured with the bandpass filter of the orange range (see filter number 5 in Table 2).Figure 6
Spectral images registered with bandpass filter of the orange range (see filter number 5 in Table2), for 7 days. (a) Original. (b) Negative.
(a)
(b)Figure7 shows the resulting plots of l
o
g
f vs. l
o
g
F
u
,
v
2 for experiment I in three different days (third, fifth, and seventh days). The corresponding slope β
h for each day was calculated with least squares. This least square approximation obtained a correlation factor of around 0.9, expressing enough reliability in this method.Figure 7
Graph of logarithms for frequency vs. magnitude for (a) third day, (b) fifth day, and (c) seventh day.
(a)
(b)
(c)It was proposed the quantification for the growing brown spots by calculating the value ofβ
h for each one of the seven days of evaluation. In Figure 8 it is possible to see that on the first four days the values of β
h show a decreasing tendency, but at the fifth day there is an abrupt change in this tendency by increasing its value instead of continuing the decrement, disserving in a correct analysis.Figure 8
β
h values for Experiment I.Experiment II had a similar behavior to Experiment I, but there was not a clear tendency that would be helpful to quantify the grow of the brown spots in the peel caused by a continuous change of increment and decrement of the slopeβ
h. This behavior demonstrates that it is not possible to measure the quantity of characteristic brown spots of the ripening process in a repeatedly manner using this method. Furthermore, the repeatability for this method, shown in Figure 9, presents a high variability for the calculated value of β
h, which indicates an unstable ripeness characterizer.Figure 9
Repeatability of ripening process with Fourier analysis.


### 3.2. Hotelling Transform Analysis

The Hotelling transform is proposed with the objective of analyzing all the possible combinations of ten different spectral images. Because of this reason, there exist 210-11 possible combinations, if we consider the use of more than one image to fuse and excluding two cases: the use of none images and the use of only one spectral image.From all the possible combinations, it was proposed a characterization of the ripening process using an approach based on the average of the calculated eigenvalues for each day during the evaluation period. In this case, it was expected that with a least squares approximation it could be possible to quantify the evolution of the ripeness in the banana. The proposed equation with least squares had a correlation factor (r) of around 0.74, for the worst case, and 0.84 for the best case. Figures 10 and 11 show the best and the worst cases, respectively, belonging to Experiment I.Figure 10
Average of eigenvalues for Experiment I with r=0.84.Figure 11
Average of eigenvalues for Experiment I with r=0.74.Based on these results, we decided to compute the Hotelling transform (see equation (8)) for both of these cases that resulted in a correlation factor of 0.84 and 0.74. Therefore, it was found that merging the two spectral ranges registered with both optical bandpass filters, visible (410-690 nm) and Near-IR (820-910 nm), (indicated in Table 2 as filters number 4 and 8, respectively), resulted in the isolation of the brown spots in the banana peel. As can be seen in Figure 12, the spectral fusion results in the isolation of the brown spots that are starting to appear in the banana peel during the evaluation of the experiment.Figure 12
Brown spots growing in banana peel. Images obtained after the fusion of images taken with the optical filters visible (410-690 nm) and Near-IR (820-910 nm), (indicated in Table2 as filters number 4 and 8, respectively).The repeatability of the experiments is shown in Figure13. It is observable that during the first days of the ripening process, the variability of the calculated values has not a remarkable difference, allowing the quantification of the process. However, as the ripening process continues, the variability considerably increases as the days pass by, provoking a low certainty in the evaluation of the ripening process. Despite this process did not result propitious to evaluate the ripening process, it was possible to identify the spectral bands necessary to isolate the brown spots that start to appear in the banana peel at the seventh stage of the maturation process.Figure 13
Repeatability for Hotelling analysis to evaluate ripening process.


### 3.3. Texture Analysis Computing the Homogeneity Criteria Based on Cooccurrence Matrix

As the brown spots start to spread all over the peel in the banana during the ripening process (7th stage in Figure1), it is noticeable that these spots grow and start to group until they cover the whole peel. This phenomenon in the images is represented as an increasing in the intensity values where are located the brown spots. Considering this, it is likely to believe that there will be an increment in homogeneity of the contrast in the image, which is a reflex of the texture.In these experiments, the texture was described by evaluating the cooccurrence of equal intensities between the nearest pixel neighbors and using the homogeneity criteria. For example, taking the set of images of a specimen used in the Experiment I, the resulting cooccurrence matrices for the initial and final day had the behavior shown in Figure14. Then, Figure 14 shows that the cooccurrence matrix starts to form a main diagonal as the values of intensities in the evaluated image start to be similar in proximate regions, or in other words as long as the brown spots start to group all over the peel. For this reason, it was decided to approximate the ripening process with the calculation of the homogeneity for a seven-day evaluation. The resulting value of the homogeneity for each day is shown with the blue plot of Figure 15.Figure 14
Cooccurrence matrices for Experiment I.Figure 15
Ripening process evaluated with homogeneity criteria for Experiment I (blue line) and Experiment II (red line).Experiment II had a similar behavior as Experiment I, but with the difference that the tendency of decrement in the homogeneity criteria starts on the fourth day and continues until the final day of the evaluation (red line in Figure15). Notice that at the very first moment that spots appear, the homogeneity takes a decrement tendency as the spots spread all over the area comprising the peel as has been shown by Figure 12.For this analysis, as well with the other techniques descripted in this article, the repeatability of the experiments was evaluated taking as a reference the first day of the appearance of the brown spots in the peel. Figure16 shows the analysis of repeatability, where can be seen that as the ripening process advances, the variability has low levels. Based on the decreasing behavior of the graph shown in Figure 16, it is possible to propose the texture analysis based on cooccurrence matrix technique as an effective method for evaluating the ripening process of a banana at the seventh stage of maturation.Figure 16
Repeatability for texture analysis.The Fourier fractal analysis took a portion of the magnitude of the spectrum. Such portion was approached to a linear section (with a correlation around 0.9); nevertheless, the quantification of the ripeness was not possible because the value of the correlated slope had an unstable tendency as the brown spots spread in the peel. Despite this inconvenience, it was shown that with the bandpass filter of the orange specter (indicated in Table2 as filters number 5) it is possible to take register of the characteristic brown spots during the ripening process. This allows its use in a system to handle much less information related to the covered area by the brown spots in the peel, instead of the use of an RGB image containing three different wavelengths as Quevedo did [5].The Hotelling transform requires a bigger quantity of computational resources at the beginning due to all the possible combinations with the total spectral images acquired with the system. Nevertheless, this process aids to recognize which are the most relevant spectral images for its fusion. With this analysis it was possible to identify that the bandpass filters of the visible and Near-IR ranges (indicated in Table2 as filters number 4 and 8, respectively) are useful for obtaining the spectral ranges necessary to isolate the brown spots present in the banana peel, which are characteristic of the ripeness at the seventh stage of the maturation process in a banana.The texture analysis based on homogeneity, proposed in this work, does not depend on the change of color space of the banana images. The transformation from an RGB image to HSV, CIE or grayscale, process in which could be a possible loss of information if there is not an adequate handle of the color spaces used, is like the identification of the most representative color of the phenomenon to analyze.The standard deviation (SD) and average (A) of the three techniques applied to quantify the ripening process in the maturation of the banana during the 7-day evaluation is shown in Table4. In the case of the Fourier fractals technique, the A and SD are defined for the β
h value; the values corresponding to the Hotelling transform are from the average of the eigenvalues; finally the A and SD for the texture analysis are related to the computed value of the homogeneity.Table 4
Normalized values of average and standard deviation for the experiments.
Applied technique
Value
Day 1
Day 2
Day 3
Day 4
Day 5
Day 6
Day 7
Fourier fractal
A
0.43
0.39
0.24
0.20
0.28
0.29
0.24
SD
0.11
0.24
0.23
0.13
0.19
0.17
0.19
Hotelling transform
A
0.88
0.88
0.87
0.84
0.81
0.72
0.65
SD
0.06
0.07
0.07
0.11
0.20
0.28
0.31
Texture analysis through homogeneity criteria based on cooccurrence matrix
A
0.86
0.83
0.74
0.61
0.45
0.35
0.29
SD
0.11
0.10
0.15
0.20
0.21
0.17
0.19Based on the SD comparison of the three techniques, it is possible to see that the SD of the texture analysis with the homogeneity criteria has the lowest variation among the three compared techniques.The computational complexity of the three methods was calculated in order to compare the efficiency in time of each method. Such computational complexity resulted to be as follows:(i)
The Fourier fractal analysis computational time is dominated byO(
n
2
). This computational time was obtained by using the maximum likelihood estimator for calculating the Hurst coefficient.
(ii)
Hotelling transform is dominated byO(min(
p
3
,
n
3
)).This was obtained by the computation of matrix singular value decomposition, for n samples with p features.
(iii)
The homogeneity value from the cooccurrence matrix can be computed inO(
k
2
), where k is the number of image gray levels. For this specific experiment 8-bits images were used.By comparing the computational complexity of the methods we can observe that Hotelling transform is the most complex because it has the highest order (in this case 3). The other two methods present a complexity of the same order (in this case 2).Other techniques are not considered for comparison because they depend on the extraction of some features from a specific channel in color spaces (i.e.,RGB images for generating an automated classification system [11, 12]; brown area percentage in CIE L∗a∗b∗ [13]; ripening color index from the channels a∗b∗ from CIE L∗a∗b∗[14]; fuzzy system of images in two color spaces, HSV and CIE L∗a∗b∗ [15]). In all these cases, a three-channel image representation is needed. Conversely, in this study, we use only monochromatic image representations in order to quantifying the ripening. Such monochromatic image representations have the benefit of reducing the use of memory and the processing time. Besides, change of color space needs an additional computation inO(n). By using directly monochromatic images we avoid such additional computation.Hence, the texture analysis with the homogeneity criteria results to be the best candidate, among the three compared techniques, for evaluating the ripening process of the banana using the growing of the brown spots because this method has the lowest SD variation and presents a low computational complexity. Additionally, image homogeneity analysis has the advantage of being rotational invariant because of the cooccurrence matrix properties [22].


## 3.1. Fourier Fractal Analysis

The spectral image of interest for this analysis was captured with the orange bandpass filter of 560-600 nm (see filter number 5 in Table2), which allows the transference of the orange color. This was the nearest wavelength to the brown spotting shown on the banana peel at the seventh stage. This technique searches for the slope in the Fourier spectrum where it is perceptible a change in the different intensity levels as the brown spots grow all over the banana peel. First of all, it is needed to stand out the brown spots from the registered spectral image (Figure 4(a)) by computing its negative image N (Figure 4(b)).Figure 4
Spectral image with its negative. (a) Original. (b) Negative.
(a)
(b)From the resulting negative imageN is obtained its fast Fourier transform (FFT), from which it is necessary to compute the magnitude for each coefficient. This new information, as well as the spatial frequency, is logarithmically evaluated for plotting l
o
g
f vs. l
o
g
F
u
,
v
2 as shown in Figure 5.Figure 5
Graph of logarithms for frequency vs. magnitude.After analyzing different ranges with increasing values of 0.1 for similar graphs to Figure5, it was found that in the range of 1.7-1.8, for the axis of l
o
g
f, the computed data could behave as a linear portion, from which the slope would be useful for quantifying the growing of the brown spots in the peel. Such hypothesis is true if we consider that, as the covered area by the spots grows, there will be an increment in the values of the intensity as well as the computed values from the FFT. If this is possible, an increment or decrement in the value of the slope is expected to allow a process for quantifying the growing of the brown spots.The present Fourier fractal analysis is proposed in order to identify the ripening process of the banana by quantifying the growth of the spots based on a slope in a closed region near the maximum value of the magnitude, where the most significant intensity changes are present. As an example of this analysis, Figure6(b) shows the evaluated images of a specimen, which correspond to the negative image of the wavelengths captured with the bandpass filter of the orange range (see filter number 5 in Table 2).Figure 6
Spectral images registered with bandpass filter of the orange range (see filter number 5 in Table2), for 7 days. (a) Original. (b) Negative.
(a)
(b)Figure7 shows the resulting plots of l
o
g
f vs. l
o
g
F
u
,
v
2 for experiment I in three different days (third, fifth, and seventh days). The corresponding slope β
h for each day was calculated with least squares. This least square approximation obtained a correlation factor of around 0.9, expressing enough reliability in this method.Figure 7
Graph of logarithms for frequency vs. magnitude for (a) third day, (b) fifth day, and (c) seventh day.
(a)
(b)
(c)It was proposed the quantification for the growing brown spots by calculating the value ofβ
h for each one of the seven days of evaluation. In Figure 8 it is possible to see that on the first four days the values of β
h show a decreasing tendency, but at the fifth day there is an abrupt change in this tendency by increasing its value instead of continuing the decrement, disserving in a correct analysis.Figure 8
β
h values for Experiment I.Experiment II had a similar behavior to Experiment I, but there was not a clear tendency that would be helpful to quantify the grow of the brown spots in the peel caused by a continuous change of increment and decrement of the slopeβ
h. This behavior demonstrates that it is not possible to measure the quantity of characteristic brown spots of the ripening process in a repeatedly manner using this method. Furthermore, the repeatability for this method, shown in Figure 9, presents a high variability for the calculated value of β
h, which indicates an unstable ripeness characterizer.Figure 9
Repeatability of ripening process with Fourier analysis.


## 3.2. Hotelling Transform Analysis

The Hotelling transform is proposed with the objective of analyzing all the possible combinations of ten different spectral images. Because of this reason, there exist 210-11 possible combinations, if we consider the use of more than one image to fuse and excluding two cases: the use of none images and the use of only one spectral image.From all the possible combinations, it was proposed a characterization of the ripening process using an approach based on the average of the calculated eigenvalues for each day during the evaluation period. In this case, it was expected that with a least squares approximation it could be possible to quantify the evolution of the ripeness in the banana. The proposed equation with least squares had a correlation factor (r) of around 0.74, for the worst case, and 0.84 for the best case. Figures 10 and 11 show the best and the worst cases, respectively, belonging to Experiment I.Figure 10
Average of eigenvalues for Experiment I with r=0.84.Figure 11
Average of eigenvalues for Experiment I with r=0.74.Based on these results, we decided to compute the Hotelling transform (see equation (8)) for both of these cases that resulted in a correlation factor of 0.84 and 0.74. Therefore, it was found that merging the two spectral ranges registered with both optical bandpass filters, visible (410-690 nm) and Near-IR (820-910 nm), (indicated in Table 2 as filters number 4 and 8, respectively), resulted in the isolation of the brown spots in the banana peel. As can be seen in Figure 12, the spectral fusion results in the isolation of the brown spots that are starting to appear in the banana peel during the evaluation of the experiment.Figure 12
Brown spots growing in banana peel. Images obtained after the fusion of images taken with the optical filters visible (410-690 nm) and Near-IR (820-910 nm), (indicated in Table2 as filters number 4 and 8, respectively).The repeatability of the experiments is shown in Figure13. It is observable that during the first days of the ripening process, the variability of the calculated values has not a remarkable difference, allowing the quantification of the process. However, as the ripening process continues, the variability considerably increases as the days pass by, provoking a low certainty in the evaluation of the ripening process. Despite this process did not result propitious to evaluate the ripening process, it was possible to identify the spectral bands necessary to isolate the brown spots that start to appear in the banana peel at the seventh stage of the maturation process.Figure 13
Repeatability for Hotelling analysis to evaluate ripening process.


## 3.3. Texture Analysis Computing the Homogeneity Criteria Based on Cooccurrence Matrix

As the brown spots start to spread all over the peel in the banana during the ripening process (7th stage in Figure1), it is noticeable that these spots grow and start to group until they cover the whole peel. This phenomenon in the images is represented as an increasing in the intensity values where are located the brown spots. Considering this, it is likely to believe that there will be an increment in homogeneity of the contrast in the image, which is a reflex of the texture.In these experiments, the texture was described by evaluating the cooccurrence of equal intensities between the nearest pixel neighbors and using the homogeneity criteria. For example, taking the set of images of a specimen used in the Experiment I, the resulting cooccurrence matrices for the initial and final day had the behavior shown in Figure14. Then, Figure 14 shows that the cooccurrence matrix starts to form a main diagonal as the values of intensities in the evaluated image start to be similar in proximate regions, or in other words as long as the brown spots start to group all over the peel. For this reason, it was decided to approximate the ripening process with the calculation of the homogeneity for a seven-day evaluation. The resulting value of the homogeneity for each day is shown with the blue plot of Figure 15.Figure 14
Cooccurrence matrices for Experiment I.Figure 15
Ripening process evaluated with homogeneity criteria for Experiment I (blue line) and Experiment II (red line).Experiment II had a similar behavior as Experiment I, but with the difference that the tendency of decrement in the homogeneity criteria starts on the fourth day and continues until the final day of the evaluation (red line in Figure15). Notice that at the very first moment that spots appear, the homogeneity takes a decrement tendency as the spots spread all over the area comprising the peel as has been shown by Figure 12.For this analysis, as well with the other techniques descripted in this article, the repeatability of the experiments was evaluated taking as a reference the first day of the appearance of the brown spots in the peel. Figure16 shows the analysis of repeatability, where can be seen that as the ripening process advances, the variability has low levels. Based on the decreasing behavior of the graph shown in Figure 16, it is possible to propose the texture analysis based on cooccurrence matrix technique as an effective method for evaluating the ripening process of a banana at the seventh stage of maturation.Figure 16
Repeatability for texture analysis.The Fourier fractal analysis took a portion of the magnitude of the spectrum. Such portion was approached to a linear section (with a correlation around 0.9); nevertheless, the quantification of the ripeness was not possible because the value of the correlated slope had an unstable tendency as the brown spots spread in the peel. Despite this inconvenience, it was shown that with the bandpass filter of the orange specter (indicated in Table2 as filters number 5) it is possible to take register of the characteristic brown spots during the ripening process. This allows its use in a system to handle much less information related to the covered area by the brown spots in the peel, instead of the use of an RGB image containing three different wavelengths as Quevedo did [5].The Hotelling transform requires a bigger quantity of computational resources at the beginning due to all the possible combinations with the total spectral images acquired with the system. Nevertheless, this process aids to recognize which are the most relevant spectral images for its fusion. With this analysis it was possible to identify that the bandpass filters of the visible and Near-IR ranges (indicated in Table2 as filters number 4 and 8, respectively) are useful for obtaining the spectral ranges necessary to isolate the brown spots present in the banana peel, which are characteristic of the ripeness at the seventh stage of the maturation process in a banana.The texture analysis based on homogeneity, proposed in this work, does not depend on the change of color space of the banana images. The transformation from an RGB image to HSV, CIE or grayscale, process in which could be a possible loss of information if there is not an adequate handle of the color spaces used, is like the identification of the most representative color of the phenomenon to analyze.The standard deviation (SD) and average (A) of the three techniques applied to quantify the ripening process in the maturation of the banana during the 7-day evaluation is shown in Table4. In the case of the Fourier fractals technique, the A and SD are defined for the β
h value; the values corresponding to the Hotelling transform are from the average of the eigenvalues; finally the A and SD for the texture analysis are related to the computed value of the homogeneity.Table 4
Normalized values of average and standard deviation for the experiments.
Applied technique
Value
Day 1
Day 2
Day 3
Day 4
Day 5
Day 6
Day 7
Fourier fractal
A
0.43
0.39
0.24
0.20
0.28
0.29
0.24
SD
0.11
0.24
0.23
0.13
0.19
0.17
0.19
Hotelling transform
A
0.88
0.88
0.87
0.84
0.81
0.72
0.65
SD
0.06
0.07
0.07
0.11
0.20
0.28
0.31
Texture analysis through homogeneity criteria based on cooccurrence matrix
A
0.86
0.83
0.74
0.61
0.45
0.35
0.29
SD
0.11
0.10
0.15
0.20
0.21
0.17
0.19Based on the SD comparison of the three techniques, it is possible to see that the SD of the texture analysis with the homogeneity criteria has the lowest variation among the three compared techniques.The computational complexity of the three methods was calculated in order to compare the efficiency in time of each method. Such computational complexity resulted to be as follows:(i)
The Fourier fractal analysis computational time is dominated byO(
n
2
). This computational time was obtained by using the maximum likelihood estimator for calculating the Hurst coefficient.
(ii)
Hotelling transform is dominated byO(min(
p
3
,
n
3
)).This was obtained by the computation of matrix singular value decomposition, for n samples with p features.
(iii)
The homogeneity value from the cooccurrence matrix can be computed inO(
k
2
), where k is the number of image gray levels. For this specific experiment 8-bits images were used.By comparing the computational complexity of the methods we can observe that Hotelling transform is the most complex because it has the highest order (in this case 3). The other two methods present a complexity of the same order (in this case 2).Other techniques are not considered for comparison because they depend on the extraction of some features from a specific channel in color spaces (i.e.,RGB images for generating an automated classification system [11, 12]; brown area percentage in CIE L∗a∗b∗ [13]; ripening color index from the channels a∗b∗ from CIE L∗a∗b∗[14]; fuzzy system of images in two color spaces, HSV and CIE L∗a∗b∗ [15]). In all these cases, a three-channel image representation is needed. Conversely, in this study, we use only monochromatic image representations in order to quantifying the ripening. Such monochromatic image representations have the benefit of reducing the use of memory and the processing time. Besides, change of color space needs an additional computation inO(n). By using directly monochromatic images we avoid such additional computation.Hence, the texture analysis with the homogeneity criteria results to be the best candidate, among the three compared techniques, for evaluating the ripening process of the banana using the growing of the brown spots because this method has the lowest SD variation and presents a low computational complexity. Additionally, image homogeneity analysis has the advantage of being rotational invariant because of the cooccurrence matrix properties [22].


## 4. Conclusions

This article has proposed, evaluated, and compared 3 methods for identifying, registering, and quantifying the ripening process of a banana (Musa AAB Simmonds) at the seventh stage of the growing process. Such methods are nondestructive techniques based on processing multispectral images. Once the brown spots were isolated, it was found that the technique of texture analysis based on the cooccurrence matrix and the homogeneity criteria gave the best results. This is concluded due to the low variability that such technique showed in the developed experiments, in addition to the clear tendency of decrement as the brown spots spread all over the yellow peel. It can be concluded that the homogeneity decreases as the brown spots in the banana acquire a more uniform contrast and extends all over the peel, making it possible to quantify and register the evolution of the ripening process when the brown spots are present in the surface of the banana peel. Therefore, the best nondestructive technique of image processing, from the three methods compared in this article, is concluded to be the preprocessing of the flat-field correction and the texture analysis computing the homogeneity criteria based on a cooccurrence matrix. Such method has the following advantages: does not require complex calculations; is rotational invariant; and has relatively low variations (SD).This process must be applied to images taken with the optical bandpass filters of the visible (410-690 nm) and Near-IR (820-910 nm) specters (indicated in Table2 as filters number 4 and 8, respectively). Due the use of monochromatic images, it is not required to have the image representation stored in different channels, as would occur in RGB, CIEL
∗
a
∗
b
∗ or HSV. This results in saving memory space and processing time.Besides the detection of the most suitable computational technique, another important contribution of the present article is the reduction in the use of computational resources due to the use of optical filters. Optical filters reduce the time in preprocessing images, generated from changes, or conversions, in color spaces. All these advantages make this method efficient and suitable for use in nondestructive real-time identification systems.Besides, temperature and relative humidity directly affect the development of the ripening process; therefore, previous articles run their experiments controlling these variables. But in our case, we did not control such variables; furthermore, we ran the experiments under two different conditions of temperature and relative humidity, obtaining in both cases successful results.The fusion of multispectral images can potentially be used in the analysis of maturation in other types of bananas and fruits. It will be necessary to define which filters and spectral band images should be used depending on the fruit to be analyzed. The presented methodology of experimentation will facilitate the election of filters and spectral band images.It is the author’s opinion that the proposed method has can be extended to noninvasive applications in real time to quantify the level of maturity in different fruits. Research is underway in those areas and will be reported in subsequent papers.



---

*Source: 6742896-2019-04-07.xml*