# Using Machine Learning Approaches for Food Quality Detection

Authors: Junming Han; Tong Li; Yun He; Quan Gao
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Food quality detection is an important method for ensuring food safety. Efficient quality detection methods can improve the efficiency of food circulation and reduce storage and labor costs. Traditional methods use instrumentation, testing reagents, or manual labor. These methods take a long time to detect, are time-consuming and labor-intensive, and require professionals to operate. Fruit, as a high-value food that provides essential nutrition for human beings, is susceptible to spoilage during packaging, transportation, and sales, so the freshness and safety assurance of fruit are a hot and difficult area of current research. Therefore, for the detection of fruit freshness, this paper proposes an efficient and nondestructive way to detect fruit freshness by using the machine learning algorithm convolutional neural network (CNN). This paper shows that convolutional neural networks have good performance in identifying the freshness of fruits through extensive experimental results and discusses the overfitting of machine learning based on the experimental results.


---

## Body

## 1. Introduction

Food safety is the main factor affecting public health and social security [1] and the primary goal of food analysis [2]. Especially in some underdeveloped countries, frequent food safety incidents can reduce consumer confidence, affect the normal order of the market, and even cause social problems, so issues related to food safety have attracted increasing social attention [3], and related research has become particularly important.According to web research data (from ourworldindata.org), these kinds of fruits’ growth in production from 1961 to 2018 are shown in Table1. The production of these fruits increased dramatically over 57 years, the average of the absolute change increased by about 240 million tons, and the average of the relative change increased by about 403%. With a dramatic increase in the production of fruits, fruit selection, transportation, and quality control have become an important issue.Table 1
Production of fruits from 1961 to 2018 (unit: ton).
Name19612018Absolute changeRelative change (%)Apple77075584369813253292737669379.81Avocado35411852977702726235842740.88Banana104571704565151188460579484440.44Grapes19860210336030258216170047981.42Orange70741342334086151263344809372.26Fresh fruits are an important part of the human diet and contain essential minerals, vitamins, and dietary fibers [4]; therefore, finding an efficient and nondestructive method to identify the quality of fruits has become a hot research direction in recent years. According to the current research, there are about three directions.


### 1.1. Based on the Supply Chain

Wang et al. [3] proposed a food traceability system, which not only enables quality tracking but also allows quality monitoring and information sharing in real time.Pal and Kant [5] used big data collected by the Internet of things to quickly remove poor-quality food from the supply chain while reducing food waste and improving transportation efficiency.Due to the increasing demand for fresh fruit, Elavarasi et al. [6] suggested that by using the Internet of things, fresh fruits can be monitored to avoid wastage during transportation.Wu and Defraeye [7] researched thermal heterogeneity and associated differences in quality evolution for large collections of packed fruits and investigated the thermal behavior of an entire pallet of fruit throughout the entire cold chain. Therefore, three cold chain options were evaluated with the expectation of achieving the elimination of heat to improve the quality of cold chain transportation.Dora et al. [8] adopted artificial intelligence in the food supply chain to improve transparency and traceability and address challenges in food safety, quality, and waste.


### 1.2. Based on Hyperspectral

Zhang et al. [9] developed an algorithm for detecting early rottenness in apples using a hyperspectral reflectance imaging system combined with spectral analysis and image processing, and chemometrics and pattern recognition methods were performed for spectral analysis in the spectral domain. Finally, the results with 98% overall detection accuracy were obtained.Wang et al. [10] applied hyperspectral transmittance data combined with convolutional neural networks to detection of internal mechanical damage in blueberries and achieved an average accuracy and F1 score of 0.8844/0.8784 and 0.8952/0.8905, respectively.Tao et al. [11] designed a low-cost, cloud-based, portable near-infrared (NIR) system and achieved an F1 score of 0.89.


### 1.3. Based on New Equipment

Weng and Neethirajan [2] reported portable microfluidic devices for food safety and quality control, which are used for the determination of food-borne pathogens (microorganisms), food allergens, biotoxins, heavy metal ions, and other chemical components that may be present in food.Yousefi et al. [12] suggested implanting sensors into the packaging material of food products and monitoring microbial contamination in food products in real time using this sensor.Alfian et al. [13] proposed a proposal for controlling temperature and humidity during food storage and transportation using radio frequency identification and Internet of things technology, which can track perishable foods.Gonzalez-Viejo et al. [14] developed a low-cost e-nose and combined it with machine learning modeling to predict aroma in beer for food quality monitoring purposes.The supply chain focuses more on improving traceability and fast delivery. Hyperspectral and new equipment are the methods to detect the quality of fruit. However, these three methods have drawbacks such as not touching end sales, having high investment costs, and needing specialized domain knowledge, and moreover, the hyperspectral imaging system is mainly composed of five parts: an array camera, a light splitter, a light source, a transmission mechanism, and computer software and hardware, so large datasets of hyperspectral images are difficult to establish. Therefore, this paper uses a convolutional neural network to classify fruit freshness RGB-images and confirm the freshness of fruits based on classification results.Some damages or lesions on the surface of fruit are visible to the naked eye, and features can also be clearly reflected in RGB images. The convolutional neural network has been widely used as efficient and nondestructive fruit quality detection.Azizah et al. [15] and Suistika et al. [16] studied CNN models for automatic classification of mangosteen and strawberry, respectively. Ciocca et al. [17] researched convolutional neural networks to classify food productions and studied CNN in food information retrieval. Hameed et al. [18] proposed that the classification of fresh produce, such as fruits and vegetables, has become a complex problem, and convolutional neural networks are considered a promising approach for its application. Research by JahanBakhshi et al. [19] shows that image processing as a nondestructive method plays a key role in the quality assessment of agricultural products and experimentally demonstrates that convolutional neural networks have significant advantages in processing sour lemon images.These surveyed studies indicated that the types of fruits and some physical and chemical indicators of them will be reflected in the RGB image or spectral image. Deep learning methods showed better performance than traditional data analysis methods and deserve further study in the future for the quality detection of fruits.


## 1.1. Based on the Supply Chain

Wang et al. [3] proposed a food traceability system, which not only enables quality tracking but also allows quality monitoring and information sharing in real time.Pal and Kant [5] used big data collected by the Internet of things to quickly remove poor-quality food from the supply chain while reducing food waste and improving transportation efficiency.Due to the increasing demand for fresh fruit, Elavarasi et al. [6] suggested that by using the Internet of things, fresh fruits can be monitored to avoid wastage during transportation.Wu and Defraeye [7] researched thermal heterogeneity and associated differences in quality evolution for large collections of packed fruits and investigated the thermal behavior of an entire pallet of fruit throughout the entire cold chain. Therefore, three cold chain options were evaluated with the expectation of achieving the elimination of heat to improve the quality of cold chain transportation.Dora et al. [8] adopted artificial intelligence in the food supply chain to improve transparency and traceability and address challenges in food safety, quality, and waste.


## 1.2. Based on Hyperspectral

Zhang et al. [9] developed an algorithm for detecting early rottenness in apples using a hyperspectral reflectance imaging system combined with spectral analysis and image processing, and chemometrics and pattern recognition methods were performed for spectral analysis in the spectral domain. Finally, the results with 98% overall detection accuracy were obtained.Wang et al. [10] applied hyperspectral transmittance data combined with convolutional neural networks to detection of internal mechanical damage in blueberries and achieved an average accuracy and F1 score of 0.8844/0.8784 and 0.8952/0.8905, respectively.Tao et al. [11] designed a low-cost, cloud-based, portable near-infrared (NIR) system and achieved an F1 score of 0.89.


## 1.3. Based on New Equipment

Weng and Neethirajan [2] reported portable microfluidic devices for food safety and quality control, which are used for the determination of food-borne pathogens (microorganisms), food allergens, biotoxins, heavy metal ions, and other chemical components that may be present in food.Yousefi et al. [12] suggested implanting sensors into the packaging material of food products and monitoring microbial contamination in food products in real time using this sensor.Alfian et al. [13] proposed a proposal for controlling temperature and humidity during food storage and transportation using radio frequency identification and Internet of things technology, which can track perishable foods.Gonzalez-Viejo et al. [14] developed a low-cost e-nose and combined it with machine learning modeling to predict aroma in beer for food quality monitoring purposes.The supply chain focuses more on improving traceability and fast delivery. Hyperspectral and new equipment are the methods to detect the quality of fruit. However, these three methods have drawbacks such as not touching end sales, having high investment costs, and needing specialized domain knowledge, and moreover, the hyperspectral imaging system is mainly composed of five parts: an array camera, a light splitter, a light source, a transmission mechanism, and computer software and hardware, so large datasets of hyperspectral images are difficult to establish. Therefore, this paper uses a convolutional neural network to classify fruit freshness RGB-images and confirm the freshness of fruits based on classification results.Some damages or lesions on the surface of fruit are visible to the naked eye, and features can also be clearly reflected in RGB images. The convolutional neural network has been widely used as efficient and nondestructive fruit quality detection.Azizah et al. [15] and Suistika et al. [16] studied CNN models for automatic classification of mangosteen and strawberry, respectively. Ciocca et al. [17] researched convolutional neural networks to classify food productions and studied CNN in food information retrieval. Hameed et al. [18] proposed that the classification of fresh produce, such as fruits and vegetables, has become a complex problem, and convolutional neural networks are considered a promising approach for its application. Research by JahanBakhshi et al. [19] shows that image processing as a nondestructive method plays a key role in the quality assessment of agricultural products and experimentally demonstrates that convolutional neural networks have significant advantages in processing sour lemon images.These surveyed studies indicated that the types of fruits and some physical and chemical indicators of them will be reflected in the RGB image or spectral image. Deep learning methods showed better performance than traditional data analysis methods and deserve further study in the future for the quality detection of fruits.


## 2. Convolutional Neural Networks

The convolutional neural network is a kind of feedforward neural network that includes convolutional computation and has a deep structure. It is one of the representative algorithms of deep learning [20].A typical architecture of the CNN model is displayed in Figure1. CNN is one of the most popular neural network models for images, including convolutional layers, pooling layers, and fully connected layers [21]. Convolutional layers can be understood as filters for image feature extraction, which is implemented by traversing input images with convolution kernel channels, kernel size, strides, padding, and other parameters, and these parameters are set and optimized according to the problem. Pooling layers are used for sampling (upsampling, downsampling, and average sampling), and we get more obvious features by sampling. Fully connected layers composed of fully connected neural units are used to generate numerical output for regression or classification problems.Figure 1
Typical architecture of the CNN model.In a typical CNN model, the feature is only extracted from an individual layer and not transported to the next layer, and features are concatenated from different layers. To achieve better results, residual deep residual networks (ResNets) and densely connected convolutional networks (DenseNets) are used in this paper.


### 2.1. Deep Residual Networks (ResNets)

In convolutional neural network, when deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated and then degrades rapidly [22]. This problem is called degradation.He et al. [22] developed an improved version of CNN called ResNets. In ResNets, a deep residual learning framework to solve the degradation problem is proposed and can be realized by “shortcut connections” [23–25]. Figure 2 shows connections by simply performing identity mapping, and their outputs are added to the outputs of stacked layers; pure identity mappings are used as bypassing paths. It means that ResNets can improve their performance, provided depth is sufficient, and combine image features through summation before they are passed into a layer.Figure 2
Shortcut connections in deep residual networks.


### 2.2. Dense Convolutional Networks (DenseNets)

Huang et al. [26] designed a model named dense convolutional network. In a feedforward fashion, Figure 3 shows that DenseNets concatenate feature maps learned by different layers, so for each layer, the feature maps of the input are the output of all preceding layers’ feature maps.Figure 3
Concatenate in Dense Convolutional Networks, they are listed as (a) Concatenate is Dense Convolutional Networks, (b) Architecture of Dense Convolutional Networks.
(a)(b)


### 2.3. Evaluation Metrics and Parameters

In the machine learning field, almost all researchers use precision, recall, and theF1 score to evaluate models.(1)precision=TPTP+FP,recall=TPTP+FN,F1=2×precision×recallprecision+recall.TP represents true positive; i.e., the number of positive samples which are classified correctly.FP represents false positive; i.e., the number of negative samples which are classified incorrectly as positive ones.FN represents false negative; i.e., the number of positive samples which are classified incorrectly as negative ones [27].Table2 shows parameters used in this paper's methods.Table 2
The parameters of this paper’s method are listed as ResNets and DenseNets.
NameValues of ResNetsValues of DenseNetsKernel size7 × 77 × 73 × 33 × 31 × 11 × 1Stride2211Epoch number200200Padding3311Learning rate0.0010.001Number of convolutional layers1799Number of pooling layers24


## 2.1. Deep Residual Networks (ResNets)

In convolutional neural network, when deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated and then degrades rapidly [22]. This problem is called degradation.He et al. [22] developed an improved version of CNN called ResNets. In ResNets, a deep residual learning framework to solve the degradation problem is proposed and can be realized by “shortcut connections” [23–25]. Figure 2 shows connections by simply performing identity mapping, and their outputs are added to the outputs of stacked layers; pure identity mappings are used as bypassing paths. It means that ResNets can improve their performance, provided depth is sufficient, and combine image features through summation before they are passed into a layer.Figure 2
Shortcut connections in deep residual networks.


## 2.2. Dense Convolutional Networks (DenseNets)

Huang et al. [26] designed a model named dense convolutional network. In a feedforward fashion, Figure 3 shows that DenseNets concatenate feature maps learned by different layers, so for each layer, the feature maps of the input are the output of all preceding layers’ feature maps.Figure 3
Concatenate in Dense Convolutional Networks, they are listed as (a) Concatenate is Dense Convolutional Networks, (b) Architecture of Dense Convolutional Networks.
(a)(b)


## 2.3. Evaluation Metrics and Parameters

In the machine learning field, almost all researchers use precision, recall, and theF1 score to evaluate models.(1)precision=TPTP+FP,recall=TPTP+FN,F1=2×precision×recallprecision+recall.TP represents true positive; i.e., the number of positive samples which are classified correctly.FP represents false positive; i.e., the number of negative samples which are classified incorrectly as positive ones.FN represents false negative; i.e., the number of positive samples which are classified incorrectly as negative ones [27].Table2 shows parameters used in this paper's methods.Table 2
The parameters of this paper’s method are listed as ResNets and DenseNets.
NameValues of ResNetsValues of DenseNetsKernel size7 × 77 × 73 × 33 × 31 × 11 × 1Stride2211Epoch number200200Padding3311Learning rate0.0010.001Number of convolutional layers1799Number of pooling layers24


## 3. Experiments

### 3.1. Dataset

To train a good network, the dataset used in this paper was collected from Kaggle (kaggle.com) and other websites. All datasets are RGB images, containing three types of freshness (fresh, medium, and rotten), and six kinds of fruit (apples, bananas, cucumber, lemon, orange, and tomatoes). The number of these RGB images is more than 40,000, as shown in Table3.Table 3
Example of datasets.
NameFreshMediumRottenAppleBananaCucumberLemonOrangeTomato


### 3.2. Three-Classification Experiment

This three-classification experiment aims to detect freshness without considering types of fruit. Therefore, the dataset was divided into three parts: fresh, medium, and rotten, 70% of all images are used for training, and the other 30% for testing. The results are shown in Figure4.Figure 4
Results of the three-classification experiment are listed as (a) precision, (b) recall, and (c)F1 score.
(a)(b)(c)Compared with other CNNs, DenseNets have several compelling advantages, such as adding dense blocks to enhance feature reuse and propagation, being easy to train due to improved flow of information and gradients, alleviating the vanishing gradient problem of the deep network, and reducing model parameters.


### 3.3. Eighteen-Classification Experiment

The eighteen-classification experiment aims to detect each kind of fruit whether it is fresh or not. Therefore, the dataset was divided into eighteen parts: fresh apple, medium apple, rotten apple, fresh banana, medium banana, rotten banana, fresh cucumber, medium cucumber, rotten cucumber, fresh lemon, medium lemon, rotten lemon, fresh orange, medium orange, rotten orange, fresh tomato, medium tomato, and rotten tomato, 70% of all images are used for training, and the other 30% for testing. The results are shown in Figure5.Figure 5
Results of the eighteen-classification experiment are listed as (a) precision, (b) recall, and (c)F1 score.
(a)(b)(c)


## 3.1. Dataset

To train a good network, the dataset used in this paper was collected from Kaggle (kaggle.com) and other websites. All datasets are RGB images, containing three types of freshness (fresh, medium, and rotten), and six kinds of fruit (apples, bananas, cucumber, lemon, orange, and tomatoes). The number of these RGB images is more than 40,000, as shown in Table3.Table 3
Example of datasets.
NameFreshMediumRottenAppleBananaCucumberLemonOrangeTomato


## 3.2. Three-Classification Experiment

This three-classification experiment aims to detect freshness without considering types of fruit. Therefore, the dataset was divided into three parts: fresh, medium, and rotten, 70% of all images are used for training, and the other 30% for testing. The results are shown in Figure4.Figure 4
Results of the three-classification experiment are listed as (a) precision, (b) recall, and (c)F1 score.
(a)(b)(c)Compared with other CNNs, DenseNets have several compelling advantages, such as adding dense blocks to enhance feature reuse and propagation, being easy to train due to improved flow of information and gradients, alleviating the vanishing gradient problem of the deep network, and reducing model parameters.


## 3.3. Eighteen-Classification Experiment

The eighteen-classification experiment aims to detect each kind of fruit whether it is fresh or not. Therefore, the dataset was divided into eighteen parts: fresh apple, medium apple, rotten apple, fresh banana, medium banana, rotten banana, fresh cucumber, medium cucumber, rotten cucumber, fresh lemon, medium lemon, rotten lemon, fresh orange, medium orange, rotten orange, fresh tomato, medium tomato, and rotten tomato, 70% of all images are used for training, and the other 30% for testing. The results are shown in Figure5.Figure 5
Results of the eighteen-classification experiment are listed as (a) precision, (b) recall, and (c)F1 score.
(a)(b)(c)


## 4. Discussion

Table4 shows an overview of recent articles where machine learning is adopted for fruit quality detection. As can be seen in the table, approaches used in this paper (ResNets and DenseNets) perform better than others. First, ResNets and DenseNets only use RGB images as the input data, and several other methods used hyperspectral, laser backscattering, or infrared videos to improve the evaluation precision of models. Second, other research studies have a binary classification issue, while the experiments in this paper increase classification numbers to three and eighteen, so the machine learning model needs to have a better classification effect. Last, this research lists key parameters and their values: precision, recall, and F1 score, and these values show this paper’s methods are better than others.Table 4
Evaluation index of surveyed articles.
MethodClassification numberData typesPrecision (%)Recall (%)F1 score (%)[28]2Hyperspectral imaging83.3[29]2Hyperspectral imaging91.6[30]2Hyperspectral imaging93.3[31]2RGB image91–97[10]2Hyperspectral imaging88.1489.28[11]2Spectroscopy89[15]2RGB image97[32]2RGB image90[33]2Laser backscattering92.5[19]2Spectroscopic images97.3[34]2RGB image92[35]2RGB image87.27[19]2RGB image65–96.5[36]2RGB image97.5ResNets3Infrared video97.5397.5097.51ResNets18RGB image95.6994.7295.07DenseNets3RGB image97.7397.6997.7DenseNets18RGB image95.5594.7395In the machine learning area, another key issue to evaluating a model whether it is good or not is overfitting. Overfitting is a modeling error in statistics that occurs when a function is too close to a finite set of data points. Therefore, the model is only applicable to its initial dataset and not to any other dataset.Figure6 shows the loss function values of ResNets and DenseNets for training and testing. These figures illustrate that ResNets and DenseNets do not show overfitting and have good performance on both the training and test sets.Figure 6
Results of the three-classification experiment are listed as follows: (a) ResNets for three-classification; (b) ResNets for eighteen-classification; (c) DenseNets for three-classification; (d) DenseNets for eighteen-classification.
(a)(b)(c)(d)


## 5. Conclusions and Future Work

In this paper, we introduce two machine learning models (ResNets and DenseNets) to solve rapid and nondestructive detection of fruit. We selected a big dataset and conducted experiments within the dataset. From the results of the experiments, the key parameters and their values show that the models used are better than others and show consistent improvement in precision, recall, and theF1 score, without any signs of overfitting.However, in any machine learning model, data imbalance is an important point, which means that the number of images for each class is not the same. It may result in classification bias towards the majority class. Therefore, our future research is to use the data augmentation technique and generative adversarial networks to solve data imbalance, and we hope that machine learning models could perform better than now. On the other hand, solving the limitation of RGB images, such as illumination and occlusion, is also our future research.



---

*Source: 6852022-2022-12-31.xml*