# Fuzzy Classification of the Maturity of the Tomato Using a Vision System

Authors: Marcos J. Villaseñor-Aguilar; J. Enrique Botello-Álvarez; F. Javier Pérez-Pinal; Miroslava Cano-Lara; M. Fabiola León-Galván; Micael-G. Bravo-Sánchez; Alejandro I. Barranco-Gutierrez
Journal: Journal of Sensors
 (2019)

---

## Abstract

Artificial vision systems (AVS) have become very important in precision agriculture applied to produce high-quality and low-cost foods with high functional characteristics generated through environmental care practices. This article reported the design and implementation of a new fuzzy classification architecture based on the RGB color model with descriptors. Three inputs were used that are associated with the average value of the color components of four views of the tomato; the number of triangular membership functions associated with the componentsR and B were three and four for the case of component G. The amount of tomato samples used in training were forty and twenty for testing; the training was done using the Matlab© ANFISEDIT. The tomato samples were divided into six categories according to the US Department of Agriculture (USDA). This study focused on optimizing the descriptors of the color space to achieve high precision in the prediction results of the final classification task with an error of 536,995×10-6. The Computer Vision System (CVS) is integrated by an image isolation system with lighting; the image capture system uses a Raspberry Pi 3 and Camera Module Raspberry Pi 2 at a fixed distance and a black background. In the implementation of the CVS, three different color description methods for tomato classification were analyzed and their respective diffuse systems were also designed, two of them using the descriptors described in the literature.


---

## Body

## 1. Introduction

Tomato is one of the main vegetables consumed by humans for its antioxidant content, vitamins (A, B1, B2, B6, C, and E), and minerals such as potassium, magnesium, manganese, zinc, copper, sodium, iron, and calcium [1]. This fruit provides health benefits in the prevention of chronic diseases such as cancer, osteoporosis, and cataracts. One of the main indicators that allows to know the internal composition of the tomato is its degree of maturity. This characteristic is very important to determine the logistic processes of harvest, transport, commercialization, and food consumption. In this respect, the Department of Agriculture of the United States (USDA) establishes six states of maturity that are Green, Breaker, Turning, Pink, Light red and Red ([2]); these are shown in Figure 1.Figure 1
Maturity classification according to the US Department of Agriculture (USDA 2018).In literature, there are research on artificial vision that reports methodologies to estimate the maturity states of the tomato that uses the color as a main characteristic. Tomato maturity estimation models have been proposed based on the use of different models of color space. For example, theL∗a∗b∗ model allows to identify the six stages of maturity of the tomato using the Minolta a∗/b∗ [3–5]. Reference [6] conducted a study of the firmness and color of the tomato; they reported that the recommended firmness for commercialization was 1.46 N mm-1; they also determined that in the stage of pink maturity of the tomato, the values from Minolta to ∗/b∗ change from negative to positive magnitude. When the ratio of Minolta a∗/b∗ of the tomatoes reached 0.6-0.95, those can be easily marketed. On the other hand, [4] estimated the lycopene content in the different stages of maturity of the tomato by means of the foliar area and the color parameters (L∗, a∗, b∗, and hue). This model was done using an artificial neural network (ANN).On the other hand, the use of the RGB color model has allowed the identification of the maturity of the tomato. As reported by [7], which proposed a methodology to identify red tomatoes for automatic cutting through the use of a robot, this used RGB images analyzed using the relationship between the red-blue component (RB) and red-green (RG) that allowed to formulate the inequalities: B≤0.8972R and G≤0.8972R, when these conditions are met, the fruit can be harvested. A similar investigation was carried out by [8], where they compared RGB images with hyperspectral images (in the range of 396-736 nm with a spectral resolution of 1.3 nm using a spectrograph). A linear discriminant analysis was applied to both groups for the classification of the tomatoes in five stages of maturity, which was weighted by a majority vote strategy of the analysis of the individual pixels. The authors document that hyperspectral images were more discriminant than RGB in tomato maturity analysis.In 2018, [9] developed a system of maturity classification of tomatoes, the system used two types of tomatoes: with defects and without defects. For the fruit’s classification, an artificial backpropagation neural network (BPNN) was used, which was implemented in Matlab©. This system identified the degrees of maturity: red, orange, turning, and green. The architect of the neural network had thirteen inputs that were associated with six functions of color and seven functions of forms, twenty neurons in the hidden layers and one in the output. Reference [10] proposed a method using a BPNN to detect maturity levels (green, orange, and red) of tomatoes of the Roma and Pera varieties.The color characteristics were extracted from five concentric circles of the fruit, and the average shade values of each subregion were used to predict the level of maturity of the samples; these values were the entries of the BPNN. The average precision to detect the three maturity levels of the tomato samples in this method was 99.31%, and the standard deviation was 1.2%. Reference [11] implemented a classification system based on convolutional neural networks (CNN). The proposed classification architecture was composed of three stages; the first stage managed the color images of three channels that are 200 pixels in height and width. In the second part, it used five layers of CNN that extracted the main characteristics. The convolution kernels are of sizes 9×9, 5×5, and 3×3 in order to conserve characteristics, reduce unnecessary parameters, and improve the speed of calculations. Together, it has two layers of max-pooling in the CNN layers. The last part is for the classification of results by a fully connected layer. The experimental results showed an average accuracy of 91.9% with a prediction time less than 0.01 s. Another research was that of [3], who proposed an algorithm in Matlab© Simulink, which employed a 4 megapixel camera, with a resolution of 640×480 and a frame rate of 30 for the capture of the images; they received a processing that consisted of an erosion and expansion. The classification and identification of the maturity of the tomato were with the use of obtaining the red chroma of the YCbCr color model, which was between 135 and 180. Reference [6] developed a system of classification of maturity of the cherry tomatoes based on artificial vision; in this proposal, they used color, texture, and shape of the nearest K-neighbor and classifiers of vectorial support machines to classify the ripened tomatoes.Currently, with Computer Vision Systems (CVS) and Fuzzy Logic (FL), applications of maturity classification of tomatoes, guavas, apples, mangoes, and watermelons employee have been developed [12]. FL is an artificial intelligence technique that models human reasoning from the linguistics of an expert to solve a problem. Therefore, the logical processing of the variables is qualitative based on quantitative belonging functions [13]. References [14, 4] argue that the classification of the maturity of the elements of study is composed of two systems that are the identification of color and its labeling. For color representation, they used image histograms based on the RGB, HSI, and CIELab color space models; for the automatic labeling of the fruits, they designed a fuzzy system that handled the knowledge base that was transferred by an expert. On the other hand, the proposal made by [15] estimated the level of maturity in apples using the RGB color space; their methodology used four images of different views of the matrix. They proposed four maturity classes, based on a fuzzy system, which were defined as mature, low mature, near to mature, or too mature. The inputs of the diffuse system were the average values of each color map of the segmented images. Reference [13] developed an image classification system of apple, sweet lime, banana, guava, and orange; the system was implemented in Matlab©. The characteristics extracted from each fruit’s image were area, major and minor axis of each sample; these were used as inputs in the diffuse system for their classification. Another similar study was reported by [16], which implemented a diffuse system to classify the guavas in the stages of maturity raw, ripe, and overripe. The proposed classification was based on the apparent color, and their considered three inputs: hue value, saturation, and luminosity.Following this trend, this paper reports the behavior of tomato maturity based on color in the RGB model, which is the model with commonly commercial digital cameras work because they are mostly built with an optical Bayer filter on the photosensors. A fuzzy system was used in the classification stage. The main contribution of this work focuses on the comparison of color models for the description of tomato maturity stages. In addition, a Raspberry PI was used for the capture and estimation of the output variables.


## 2. Materials and Methods

### 2.1. Sample Preparation

In the proposed method, sixty tomato samples were used (acquired in a local trade) and were classified in six stages of maturity (Green, Breaker, Turning, Rosa, Light red and Red). The classification was based on the criteria of the United States Department of Agriculture USDA (1997). The samples were divided into two groups: the training and validation sets as shown in Table1.Table 1
Tomato sample division used to train and detection sets.
Samples per each maturity level
Green (G)
Breaker (B)
Turning (T)
Pink (P)
Light red (LR)
Red (R)
Training
Test
Training
Test
Training
Test
Training
Test
Training
Test
Training
Test
3
2
3
2
2
4
11
5
8
5
13
2


### 2.2. Artificial Vision System

Artificial vision systems (AVS) are intended to emulate the functionality of human vision to describe elements in captured images. Some AV’s advantages compared with other proposal are a reduction of cost, improvement of accuracy, increase of precision, and good reliability estimation [14]. Figure 2 shows the AVS system, which is integrated by three sections: (a) the image capture, (b) the lighting subsystem, and (c) the processing subsystem. The first one obtains spatial information and fruit characteristics, the second one maintains the experimental conditions, and the third one obtains several characteristics such as equalizing histograms, highlighting edges, segmenting, labeling components, and tomato maturity [17–19].Figure 2
Elements of the artificial vision system (AVS): (a) subsystem of capture of images, (b) subsystem of illumination, and (c) image processing subsystem.The images were acquired with the AVS, which was installed in a black box of dimensions38cm×38cm×43cm; this last to prevent the influences of the lighting as shown in Figure 2. The AVS was integrated by the Raspberry Pi 3 camera (8 megapixels) placed vertically at 30 cm from the sample at an angle of 28.8°. The lighting used had a ring geometry [20] with a power of 5.4 W and a diameter of 23 cm, and it was placed 30 cm above the samples where the intensity was 200 lux. The processing subsystem was implemented on a Raspberry Pi 3 card that features a Quadcom 1.2 GHz Broadcom BCM2837 64-bit processor with a 1 GB RAM memory. This device has the flexibility to be used in the solution of versatile problems [21].The proposed system is shown in Figure3; in the first stage, the RGB images of the samples were acquired. After that, images were segmented to create a vector with averages of the red, green, and blue components, which worked as an input to the fuzzy system.Figure 3
Workflow of the proposed system.


### 2.3. Image Acquisition

Four images of each fruit were acquired in each view of the tomato, obtaining a total of 240 images corresponding to 60 fruits. Figure4 shows the four views of a sample in the green maturity state. The captured images have a resolution of (1050×1680×3); they were scaled to a size of (600×900×3) with an intensity of 200 lux.Figure 4
Sample views in four different directions.
(a)
(b)
(c)
(d)


### 2.4. Image Segmentation

Figures5(a)–5(c) show the process performed on samples using Python 3.7 and OpenCV. The first step captured the images and assigned a maturity level. The second step binarized them in HSV space by using 100<=H<=156, 90<=S<=255, and 0<=V<=255 with a range between 0 and 255. The forth step was to segment each tomato image and labeled them by using an algorithm of the component’s connection. The fifth step was to separate the image segments under 500 pixels, and finally, their respective masks were used to obtain the areas of interest per each sample.Figure 5
Segmentation of one sample: (a) capture and scaling of the sample image, (b) binarization of the image and noise, and (c) image segmentation by means of the minor area discrimination.
(a)
(b)
(c)


### 2.5. Attribute Selection

The attributes were selected based on the methodology proposed by [15]. The mean channel’s values of the segmented images were used, and it was also considered that in the initial stages of maturity, the studied tomatoes had a high green content, and its content of red color was very low. As the fruit reached full maturity, the behavior was inverse [14]. The segment mean behavior was mapped by using the image channels of 40 training samples. It also used the RGB color models, the CIELab 1976, and the Minolta a∗/b∗ ratio as shown in Figures 6, 7, and 8. By using the process previously described, the identification of the six tomato maturity stages was possible, basically due to the direct relationship between the axis’s orthogonality with the data classes.Figure 6
Mapping of the means of the segments of the RGB channels of the training set.Figure 7
Mapping of the means of the segments of the channel CIELab 1976 of the training set.Figure 8
Mapping of the channel segment mean CIELab 1976 of the training set using the Minolta (a∗/b∗) relation.


### 2.6. Fuzzification

In this stage, fuzzification had the main purpose to translate the input values into linguistic variables [22]. In this proposed system, a vector created by the average values of the RGB components is used as input variable. The input fuzzification was done using triangular membership functions as shown in Figure 9. These functions were selected for their easy hardware implementation.Figure 9
Membership functions of the fuzzy system for maturity tomato classification.It is well known that in the first three maturity stages, a greater sensitivity is required to identify the changes compared with the rest of them. Therefore, in this paper, the membership function related to the green variable entries consisted of four sections. On the other hand, three membership functions were proposed for the blue and red cases, which resulted on six maturity states. Finally, the range value, for the most significant input and output stage, was determined by selecting the linguistic states for each variable, i.e., very, medium, and less.


### 2.7. Fuzzy System Implementation

The fuzzy system was implemented with the Matlab ANFISEDIT Tool and image capture using Raspberry Pi camera, where a set of data was found and integrated by the mean of the RGB channels of the image and the output was labeled for the samples.Four variants of the fuzzy system were designed to classify the state of maturity of the tomato. In these, several parameters were maintained, which were the inputs of the system, the number of training epcohs, and the type of membership functions. Table2 shows the architectures used for each fuzzy system and the error obtained after training, where it can be seen that the designs that presented the least errors were Models 3 and 4. The selected membership function is triangular because of its easy implementation.Table 2
Fuzzy system training results.
Fuzzy system
Inputs
Number of membership functions
Type of membership functions
Epochs
Error
Model 1
Mean RGB component
3,3,3
Triangular
100
0.70536
Model 2
Mean RGB component
3,4,3
Triangular
100
0.53892
Model 3
Mean RGB component
7,7,7
Triangular
100
0.01044
Model 4
Mean RGB component
10,10,10
Triangular
100
8.49
×
10
-5The programing was carried out using the methodology proposed by [23]. The description of each function is shown to follow, where the variable is LR (Low Red), MR (Middle Red), HB (High Red), Low Green (LG), Medium Low Green (MLG), Medium High Green (MHG), High Green (HG), LB Low (Blue), MB (Middle Blue), and HB (High Blue).
(1)LR=16.48−R16.48,0<R≤16.48,0,16.48<R≤34.73,(2)MR=16.48−R16.48,0<R≤16.48,R−34.7318.25,16.48<R≤34.37,(3)HR=0,0<R≤16.48,R−16.4818.25,16.48<R≤34.53,(4)LG=16.76−G16.76,0<G≤16.76,0,16.76<G≤23.91,(5)LMG=G−16.7616.76,0<G≤16.76,20.32−G3.56,16.76<G≤20.32,0,20.32<G≤23.9,(6)HMG=0,0<G≤16.76,G−20.343.56,16.76<G≤20.32,23.9−G3.57,20.32<G≤23.9,(7)HG=0,0<G≤20.37,G−27.97.57,20.37<G≤27.9,(8)LB=16.48−B16.48,0<B≤16.48,0,16.48<B≤41.61,(9)MB=B−16.816.8,0<B≤16.8,16.8−B8.81,16.52<B≤45.61,(10)HB=0,0<B≤16.8,B−16.825,16.8<B≤45.61.


### 2.8. Inferential Logic

The inferential logic was determined by identifying the maximum and minimum averages’ ranges of the RGB components of the training set images. Table3 shows the maximum and minimum averages of each maturity state according to the USDA. By using the last procedure, it was possible to determine a set of 36 rules that were used in the fuzzy system; the linguistic variables used were low, medium, low average, high, and high average, Table 4.Table 3
Maximum and minimum range of the averages of the RGB channels for each state of maturity.
Maturity level
Minimum red mean
Maximum red mean
Minimum green mean
Maximum green mean
Minimum blue mean
Maximum blue mean
Green (G)
21.5402641
23.4607073
21.4570773
22.9846567
17.4503043
20.5893361
Breaker (B)
19.1914739
25.6090892
19.7009942
23.9158162
19.788143
24.2440957
Turning (T)
8.29093793
25.4734785
13.1834743
22.9724406
19.0287402
29.0377504
Pink (P)
17.9856155
24.126667
17.6915075
21.1138724
17.4557533
19.9197693
Light red (LR)
7.38083985
24.0121635
15.451138
21.1648058
3.99488988
20.7513841
Red (R)
7.35927338
24.064192
15.9823308
21.1106179
5.10223285
20.3244826Table 4
Inference rules.
Class
Red mean
Green mean
Blue mean
(1) Green (G)
Middle
Middle
Medium high
High
Low
Middle
(2) Breaker (B)
Middle
High
Medium low
High
Middle
High
(3) Turning (T)
Low
High
Low
High
Middle
High
(4) Pink (P)
Low
High
Medium low
Medium high
Low
Middle
(5) Light red (LR)
Low
Middle
Low
Medium high
Low
High
(6) Red (R)
Low
High
Low
Medium low
Low
High


### 2.9. Defuzzification

Defuzzification was done by equation (11), with the 36 rules of inferences obtained for the modeling of maturity. The Takagi-Sugeno fuzzy model is illustrated in Figure 10; Zi represents the weight of the fuzzy rule in the output, and wi is the weight of the membership function; N is the number of rule inferences.
(11)Finaloutput=∑i=1NwiZi∑i=118wi.Figure 10
Operation of Takagi-Sugeno rules to classify the maturity of the tomato.


### 2.10. Fuzzy System Proposal

Three proposed architectures of the fuzzy systems were evaluated for the fruit identification maturity as shown in Figure11. These used the means of the RGB channels of the segments associated with the image. In the first architecture, it uses the R, G, and B channels as inputs, the second one uses the difference of the R and G channels that allow identifying the maturity according to the methodology proposed by [7], and the last construction was a change of color model from RGB to CIELab 1976, and the inputs used were L∗, a∗, and b∗ and Minolta a∗/b∗ relation proposed by [4].Figure 11
Architecture of the fuzzy models: (a) model that uses mean RGB channels of the tomato image segment, (b) model that usesR-B mean of the tomato image segment, and (c) model that uses L, ∗a, ∗b, and ∗a/∗b means of the tomato image segment.
(a)
(b)
(c)To perform the ANFIS’s training, forty samples in the six stages of maturity were used. Table5 shows the results of the training using 100 epochs of the three proposed models. It can be observed that Model 1 has the lowest training error that is 0.046; this model uses the entries R, G, and B with 3.4 and 3 belonging functions, respectively.Table 5
Proposed fuzzy system.
Model
Reference
Input
Number of membership functions (triangular)
Training error (100 epochs)
1
Proposed in this work
R, G, B
3,4,3
0.046
2
[7]
(R-G)
10
1.16
3
[4]
L, a, b y a/b
3,3,3,3
0.81


## 2.1. Sample Preparation

In the proposed method, sixty tomato samples were used (acquired in a local trade) and were classified in six stages of maturity (Green, Breaker, Turning, Rosa, Light red and Red). The classification was based on the criteria of the United States Department of Agriculture USDA (1997). The samples were divided into two groups: the training and validation sets as shown in Table1.Table 1
Tomato sample division used to train and detection sets.
Samples per each maturity level
Green (G)
Breaker (B)
Turning (T)
Pink (P)
Light red (LR)
Red (R)
Training
Test
Training
Test
Training
Test
Training
Test
Training
Test
Training
Test
3
2
3
2
2
4
11
5
8
5
13
2


## 2.2. Artificial Vision System

Artificial vision systems (AVS) are intended to emulate the functionality of human vision to describe elements in captured images. Some AV’s advantages compared with other proposal are a reduction of cost, improvement of accuracy, increase of precision, and good reliability estimation [14]. Figure 2 shows the AVS system, which is integrated by three sections: (a) the image capture, (b) the lighting subsystem, and (c) the processing subsystem. The first one obtains spatial information and fruit characteristics, the second one maintains the experimental conditions, and the third one obtains several characteristics such as equalizing histograms, highlighting edges, segmenting, labeling components, and tomato maturity [17–19].Figure 2
Elements of the artificial vision system (AVS): (a) subsystem of capture of images, (b) subsystem of illumination, and (c) image processing subsystem.The images were acquired with the AVS, which was installed in a black box of dimensions38cm×38cm×43cm; this last to prevent the influences of the lighting as shown in Figure 2. The AVS was integrated by the Raspberry Pi 3 camera (8 megapixels) placed vertically at 30 cm from the sample at an angle of 28.8°. The lighting used had a ring geometry [20] with a power of 5.4 W and a diameter of 23 cm, and it was placed 30 cm above the samples where the intensity was 200 lux. The processing subsystem was implemented on a Raspberry Pi 3 card that features a Quadcom 1.2 GHz Broadcom BCM2837 64-bit processor with a 1 GB RAM memory. This device has the flexibility to be used in the solution of versatile problems [21].The proposed system is shown in Figure3; in the first stage, the RGB images of the samples were acquired. After that, images were segmented to create a vector with averages of the red, green, and blue components, which worked as an input to the fuzzy system.Figure 3
Workflow of the proposed system.


## 2.3. Image Acquisition

Four images of each fruit were acquired in each view of the tomato, obtaining a total of 240 images corresponding to 60 fruits. Figure4 shows the four views of a sample in the green maturity state. The captured images have a resolution of (1050×1680×3); they were scaled to a size of (600×900×3) with an intensity of 200 lux.Figure 4
Sample views in four different directions.
(a)
(b)
(c)
(d)


## 2.4. Image Segmentation

Figures5(a)–5(c) show the process performed on samples using Python 3.7 and OpenCV. The first step captured the images and assigned a maturity level. The second step binarized them in HSV space by using 100<=H<=156, 90<=S<=255, and 0<=V<=255 with a range between 0 and 255. The forth step was to segment each tomato image and labeled them by using an algorithm of the component’s connection. The fifth step was to separate the image segments under 500 pixels, and finally, their respective masks were used to obtain the areas of interest per each sample.Figure 5
Segmentation of one sample: (a) capture and scaling of the sample image, (b) binarization of the image and noise, and (c) image segmentation by means of the minor area discrimination.
(a)
(b)
(c)


## 2.5. Attribute Selection

The attributes were selected based on the methodology proposed by [15]. The mean channel’s values of the segmented images were used, and it was also considered that in the initial stages of maturity, the studied tomatoes had a high green content, and its content of red color was very low. As the fruit reached full maturity, the behavior was inverse [14]. The segment mean behavior was mapped by using the image channels of 40 training samples. It also used the RGB color models, the CIELab 1976, and the Minolta a∗/b∗ ratio as shown in Figures 6, 7, and 8. By using the process previously described, the identification of the six tomato maturity stages was possible, basically due to the direct relationship between the axis’s orthogonality with the data classes.Figure 6
Mapping of the means of the segments of the RGB channels of the training set.Figure 7
Mapping of the means of the segments of the channel CIELab 1976 of the training set.Figure 8
Mapping of the channel segment mean CIELab 1976 of the training set using the Minolta (a∗/b∗) relation.


## 2.6. Fuzzification

In this stage, fuzzification had the main purpose to translate the input values into linguistic variables [22]. In this proposed system, a vector created by the average values of the RGB components is used as input variable. The input fuzzification was done using triangular membership functions as shown in Figure 9. These functions were selected for their easy hardware implementation.Figure 9
Membership functions of the fuzzy system for maturity tomato classification.It is well known that in the first three maturity stages, a greater sensitivity is required to identify the changes compared with the rest of them. Therefore, in this paper, the membership function related to the green variable entries consisted of four sections. On the other hand, three membership functions were proposed for the blue and red cases, which resulted on six maturity states. Finally, the range value, for the most significant input and output stage, was determined by selecting the linguistic states for each variable, i.e., very, medium, and less.


## 2.7. Fuzzy System Implementation

The fuzzy system was implemented with the Matlab ANFISEDIT Tool and image capture using Raspberry Pi camera, where a set of data was found and integrated by the mean of the RGB channels of the image and the output was labeled for the samples.Four variants of the fuzzy system were designed to classify the state of maturity of the tomato. In these, several parameters were maintained, which were the inputs of the system, the number of training epcohs, and the type of membership functions. Table2 shows the architectures used for each fuzzy system and the error obtained after training, where it can be seen that the designs that presented the least errors were Models 3 and 4. The selected membership function is triangular because of its easy implementation.Table 2
Fuzzy system training results.
Fuzzy system
Inputs
Number of membership functions
Type of membership functions
Epochs
Error
Model 1
Mean RGB component
3,3,3
Triangular
100
0.70536
Model 2
Mean RGB component
3,4,3
Triangular
100
0.53892
Model 3
Mean RGB component
7,7,7
Triangular
100
0.01044
Model 4
Mean RGB component
10,10,10
Triangular
100
8.49
×
10
-5The programing was carried out using the methodology proposed by [23]. The description of each function is shown to follow, where the variable is LR (Low Red), MR (Middle Red), HB (High Red), Low Green (LG), Medium Low Green (MLG), Medium High Green (MHG), High Green (HG), LB Low (Blue), MB (Middle Blue), and HB (High Blue).
(1)LR=16.48−R16.48,0<R≤16.48,0,16.48<R≤34.73,(2)MR=16.48−R16.48,0<R≤16.48,R−34.7318.25,16.48<R≤34.37,(3)HR=0,0<R≤16.48,R−16.4818.25,16.48<R≤34.53,(4)LG=16.76−G16.76,0<G≤16.76,0,16.76<G≤23.91,(5)LMG=G−16.7616.76,0<G≤16.76,20.32−G3.56,16.76<G≤20.32,0,20.32<G≤23.9,(6)HMG=0,0<G≤16.76,G−20.343.56,16.76<G≤20.32,23.9−G3.57,20.32<G≤23.9,(7)HG=0,0<G≤20.37,G−27.97.57,20.37<G≤27.9,(8)LB=16.48−B16.48,0<B≤16.48,0,16.48<B≤41.61,(9)MB=B−16.816.8,0<B≤16.8,16.8−B8.81,16.52<B≤45.61,(10)HB=0,0<B≤16.8,B−16.825,16.8<B≤45.61.


## 2.8. Inferential Logic

The inferential logic was determined by identifying the maximum and minimum averages’ ranges of the RGB components of the training set images. Table3 shows the maximum and minimum averages of each maturity state according to the USDA. By using the last procedure, it was possible to determine a set of 36 rules that were used in the fuzzy system; the linguistic variables used were low, medium, low average, high, and high average, Table 4.Table 3
Maximum and minimum range of the averages of the RGB channels for each state of maturity.
Maturity level
Minimum red mean
Maximum red mean
Minimum green mean
Maximum green mean
Minimum blue mean
Maximum blue mean
Green (G)
21.5402641
23.4607073
21.4570773
22.9846567
17.4503043
20.5893361
Breaker (B)
19.1914739
25.6090892
19.7009942
23.9158162
19.788143
24.2440957
Turning (T)
8.29093793
25.4734785
13.1834743
22.9724406
19.0287402
29.0377504
Pink (P)
17.9856155
24.126667
17.6915075
21.1138724
17.4557533
19.9197693
Light red (LR)
7.38083985
24.0121635
15.451138
21.1648058
3.99488988
20.7513841
Red (R)
7.35927338
24.064192
15.9823308
21.1106179
5.10223285
20.3244826Table 4
Inference rules.
Class
Red mean
Green mean
Blue mean
(1) Green (G)
Middle
Middle
Medium high
High
Low
Middle
(2) Breaker (B)
Middle
High
Medium low
High
Middle
High
(3) Turning (T)
Low
High
Low
High
Middle
High
(4) Pink (P)
Low
High
Medium low
Medium high
Low
Middle
(5) Light red (LR)
Low
Middle
Low
Medium high
Low
High
(6) Red (R)
Low
High
Low
Medium low
Low
High


## 2.9. Defuzzification

Defuzzification was done by equation (11), with the 36 rules of inferences obtained for the modeling of maturity. The Takagi-Sugeno fuzzy model is illustrated in Figure 10; Zi represents the weight of the fuzzy rule in the output, and wi is the weight of the membership function; N is the number of rule inferences.
(11)Finaloutput=∑i=1NwiZi∑i=118wi.Figure 10
Operation of Takagi-Sugeno rules to classify the maturity of the tomato.


## 2.10. Fuzzy System Proposal

Three proposed architectures of the fuzzy systems were evaluated for the fruit identification maturity as shown in Figure11. These used the means of the RGB channels of the segments associated with the image. In the first architecture, it uses the R, G, and B channels as inputs, the second one uses the difference of the R and G channels that allow identifying the maturity according to the methodology proposed by [7], and the last construction was a change of color model from RGB to CIELab 1976, and the inputs used were L∗, a∗, and b∗ and Minolta a∗/b∗ relation proposed by [4].Figure 11
Architecture of the fuzzy models: (a) model that uses mean RGB channels of the tomato image segment, (b) model that usesR-B mean of the tomato image segment, and (c) model that uses L, ∗a, ∗b, and ∗a/∗b means of the tomato image segment.
(a)
(b)
(c)To perform the ANFIS’s training, forty samples in the six stages of maturity were used. Table5 shows the results of the training using 100 epochs of the three proposed models. It can be observed that Model 1 has the lowest training error that is 0.046; this model uses the entries R, G, and B with 3.4 and 3 belonging functions, respectively.Table 5
Proposed fuzzy system.
Model
Reference
Input
Number of membership functions (triangular)
Training error (100 epochs)
1
Proposed in this work
R, G, B
3,4,3
0.046
2
[7]
(R-G)
10
1.16
3
[4]
L, a, b y a/b
3,3,3,3
0.81


## 3. Results

The results were obtained from the models using a set of 20 samples that were not part of the training set, and they are shown in Table6. By looking Model 1, it can be noticed that it presented an error of 536.995×10-6, which is the smallest value compared with the other two. On the other hand, Models 2 and 3 managed to correctly classify the entire sample of tests. However, Model 2 did not classify twelve samples of the test set; those are market in italic. The classification error is lower in Model 1 because the descriptor mean of the components of the channels R, G, and B can identify the increase in the mean of the red channel, the decrease in the mean value of the green channel, and the nonlinear behavior of the average values of the blue cannel [14].Table 6
Output and error of different classification systems.
Test set class
Model 1
Model 2
Model 3
Output
MSE (1×10-6)
Output
MSE
Output
MSE (1×10-6)
1
0.9848
230.0
4.0274
9.1653
0.9883
135.1337
5
5.0015
2.380
4.9622
0.0014
5.0002
0.0831
6
5.9993
0.4.20
6.0460
0.0014
6.0003
0.15688
3
3.0000
.00042
3.5879
0.3457
2.9999
0.0003
3
2.9995
.24200
3.5936
0.3524
2.9995
0.1605
5
5.0142
203.00
4.0408
0.9200
4.9626
1394.9366
2
1.9989
1.1500
2.6855
0.4699
2.0091
83.24365
4
4.0041
17.400
3.9967
1.08
×
10
-5
3.9806
372.6216
4
3.9857
17.400
4.1620
0.0262
4.0371
1383.8290
1
0.9848
230.00
4.0274
9.1653
0.9883
135.1337
6
5.9998
0.0109
5.2491
0.5637
5.9863
187.4693
2
1.9882
139.00
3.9156
3.6697
1.9989
1.0575
3
2.9999
.00185
4.1862
1.4070
2.9999
0.0021
5
4.9954
20.600
3.0531
3.7902
5.0375
1407.2835
5
5.0158
251.00
4.0636
0.8767
4.9669
1093.2020
3
2.9045
9110.0
3.8458
0.7154
2.9958
17.0728
5
4.9946
28.600
4.2339
0.5867
5.0047
22.7437
4
3.9909
81.200
4.1841
0.0339
4.0414
1721.7774
4
3.9857
204.00
3.1609
0.7040
3.9314
4702.4978
4
4.0167
204.00
4.2202
0.0485
4.0129
167.4634
Sum
—
10739.9
—
32.8434
—
12825.86
Average error
—
536.995
—
1.64217
—
641.293


## 4. Discussion

According to the results, Models 1 and 3 correctly classified the set of test samples. Additionally, these presented the sum of the lowest squared errors, the fuzzy system designed for RGB components with an averaged value of536.995×10-6. The architecture used ten membership functions for red, four for green, and three for blue color, which had a reliable performance.Additionally, Model 3 was a diffuse system that used the averages (L, ∗a, ∗b, and ∗a/∗b) of the tomato as inputs; its architecture integrated twelve membership functions, i.e., each input used three. The sum of error of this system was 12825.86×10-6. On the other hand, the fuzzy system with an RGB data entry (R-G) had 10 membership functions and a sum of its quadratic classification error was 32.8434.It can be inferred that using the subtraction (R-G) as a descriptor, the fuzzy classifier hided the information of the R and G components, while discarded the blue component. This system presented difficulties in classifying classes 3, 4, and 5; consequently, their efficiency was very low compared with others. The color representation with the components (L, ∗a, ∗b, and ∗a/∗b) seems to help, because few membership functions were used in each entry, but when four entries were considered, the work area was divided into 81 sectors. The color representation with the RGB model collected the direct values of the sensor image, generally of the Bayer type, so that it had the complete information to classify without noise due to the data transformation; this is a clear theoretical advantage of this work. Therefore, the fuzzy system designed with RGB averages used only 36 sectors generated by three membership functions in R, 4 in G, and 3 in B; this is practical advantage.In other words, the six tomato’s maturity classification can be reliably done in a RGB color space, mainly due to the nonlinear surfaces created by the fuzzy system or other mathematical functions, which separates each stage. However, the main limitation of the proposed system is that the overall experimentation was carried out in a controlled environment (fixed lighting, fixed distance from the camera to the sample, and a matt black background). This weakness is already considered in the research team, and a proposal will be reported in an upcoming paper.


## 5. Conclusion

In this work, a CVS was designed using a Raspberry Pi 3, which used tomato maturity degrees according to the USDA criteria with an average error of536.995×10-6. The acquisition of CVS images was done with the camera module Raspberry Pi 2 in a controlled environment with an illumination intensity of 200 lux, with the aim of reducing the noise on the fruit’s segmentation. Subsequently, several fuzzy systems were evaluated while maintaining the use of the four views to optimize the number of triangular membership functions to reduce the classification error. Based on the result, the system obtained a good classification, surpassing the system that uses the CIELab color space model and the R-G color space model. Together with this study, the report on the relationship with ∗a/∗b for the identification of tomato maturity by ∗ is confirmed [3–6].One aspect that can be highlighted is the use of the Raspberry Pi 3 and the camera module Raspberry Pi 2, which allowed to create applications of easy technology transfer and rapid implementation focused on the classification of fruit and vegetable maturity. This system can be extended to the CVS’s estimation of soluble solids, vitamins, and antioxidants in tomato.



---

*Source: 3175848-2019-07-04.xml*