# Development of Algorithms for an IoT-Based Smart Agriculture Monitoring System

Authors: Kazy Noor-e-Alam Siddiquee; Md. Shabiul Islam; Ninni Singh; Vinit Kumar Gunjan; Wong Hin Yong; Mohammad Nurul Huda; D. S. Bhupal Naik
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Sensor-based agriculture monitoring systems have limited outcomes on the detection or counting of vegetables from agriculture fields due to the utilization of either conventional color transformations or machine learning-based methods. To overcome these limitations, this research is aimed at proposing an IoT-based smart agriculture monitoring system with multiple algorithms such as detection, quantification, ripeness checking, and detection of infected vegetables. This paper presents smart agriculture monitoring systems for Internet of Things (IoT) applications. The CHT has been applied to detect and quantify vegetables from the agriculture field. Using color thresholding and color segmentation techniques, defected vegetables have also been detected. A machine learning method-convolutional neural network (CNN) has been used for the development and implementation of all algorithms. A comparison between traditional methods and CNN has been simulated in MATLAB to find out the optimal method for its implementation in this agricultural monitoring system. Compared to the traditional methods, the CNN is the optimal method in this research work which performed better over the previously developed algorithms with an accuracy of more than 90%. As an example (case study), a tomato field in Chittagong, Bangladesh, was chosen where a camera-mounted mobile robot captured images from the agriculture field for which the proposed IoT-based smart monitoring system was developed. This system will benefit farmers through the digitally monitored output at an agriculture field in Bangladesh as well as in Malaysia. Since this proposed smart IoT-based system is still driven by bulky, costly, and limited powered sensors, in a future work, for the required power of sensors, this research work is aimed at the design and development of an energy harvester (hybrid) (HEH) based on ultralow power electronics circuits to generate the required power of sensors. Implementation of multiple algorithms using CNN, circular Hough transformation (CHT), color thresholding, and color segmentation methods for the detection, quantification, ripeness checking, and detection of infected crops.


---

## Body

## 1. Introduction

A smart agriculture monitoring system outcomes a digitized scenario of existing crops in the fields to the farmers who usually have an economic concern on their products of agriculture field [1] using sensors or transducers. Sensors or transducers receive physical quantities or signals from the agriculture environment and return data in digitized forms. The economic concern of farmers has a functional dependency on these data (digitized form) as digitized outcome scenarios are received from the smart monitoring system since both their concerns and outcome scenarios consist of common attributes such as quantity, ripening, damaged crops, infected crops, and many more. Hence, farmers will obtain information on the economic values of crops in advance. This research will benefit farmers and upgrade the traditional methodologies used in Bangladesh and Malaysia from a nondigitized agricultural field to an IoT-based smart agricultural system. Since the supply power of sensors and transducers during working hours in the field is another challenge [2], the research intends to harvest energies from the surrounding environment and drain it to the input pin (VCC) of sensors and transducers. Hence, we have decided to design and simulate a hybrid energy harvester [3] to save up energies from solar power and electromagnetic (EM) sources.In this research, we have developed and simulated a smart agriculture monitoring system and a proposal that will receive power from a designed and simulated hybrid energy harvester. The primary outcome of the smart agriculture monitoring system is the detection of ripen/unripe vegetables with defects in the crop field. A tomato field has been chosen as the case study.


### 1.1. The Problem Statement and Its Justification

FA Farmers’ traditional way of cultivating and perceptions on crops in the field [1] do not reflect the accurate economic value much, since ripening of tomatoes likely depends on colors. That is why, the detection and the quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields.Traditional methods are appropriate for agricultural monitoring on a small scale [1]. In such systems, the circular Hough transformation (CHT) is appropriate for objects with regular forms, such as circles [4] [5] [6] [7] [8]. The thresholding and color segmentation techniques are advantageous [9]. Simultaneously, the detection and quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields.Traditional methods are appropriate for agricultural monitoring on a small scale [1]. In such systems, the circular Hough transformation (CHT) is appropriate for objects with regular forms, such as circles [4] [5] [6] [7] [8]. The thresholding and color segmentation techniques are advantageous [9]. Simultaneously, classifiers [10] [11] [12] [13], image filters [14], support vector machines [15], and neural networks [15] may get higher outcomes when detecting fruits or crops in the agriculture industry.CHT can be used to detect and quantify tomatoes, while thresholding and segmentation can be used to detect defects or black patches on the outer peel [16].Detecting and quantifying ripened, green, and tomatoes with black spots on the outer peel in the field would undoubtedly lower farmers’ workloads while continuing to play a critical role in sustaining market food values throughout tomato package distribution.Due to energy constraints of sensors, to provide a continuous required supply of power, an energy harvester will be used, and a hybrid energy harvester will be designed and implemented, which will harvest energies from multiple sources of ambient energies from the environment [17] [18] [19].Previous studies at a glimpse are highlighted in Tables1–5, where those researched were limited to detecting or counting crops, fruits, or vegetables from fields. Detection quantification of tomatoes and classifying them as ripened, semiripened, or green followed by identifying defective vegetables have not yet been addressed. Moreover, a digitized monitoring system with a hybrid energy harvester that will scavenge energies from multiple sources added another novelty in this research.Table 1
Performance comparison of image processing methods for agriculture monitoring systems.
Sl. no.AuthorsMethods and schemePerformance1Ni et al. [7]The method, CHT was applied on overlapped objects2Sa et al. [20]Applied the deep CNN along with RGB color followed by near-infrared and faster R-CNN80% to 83%.3Jimenez et al. [21]The CCD sensor85%4Zhang et al. [22]The FSCABC algorithm and feed-forward neural-network (FNN)87%5Zheng et al. [23]Deep learning method was applied, and deep learning classifier was trained having a larger dataset99%6Mureșan and M. Oltean [24]Neural network—7Kuang et al. [25]HOG features were applied along with LBP and Gabor-LBP followed by global color, global shape features, and histogramA lower miss rate of 0.0377 with false positives at 0.0682 per image8Ali et al. 2017 [26]Applied HOG features for detection and counting mangoes in trees—Table 2
Comparison on related works.
Sl. no.AuthorsComparison1Crow [27]Applied the object-detection procedures along with mapping of the texture procedures2Zhuang et al. [13]Applied the cascaded-classifiers using Haar-features with an objective of improvement of the paleness3BaoHua et al. [28]Color transformation and machine learning were applied4Barbedo [1]Mukhopadhyay and Chaudhuri [6]Yuen et al. [8]Applied the CHT method for detection of objects5Liebig [9]Color transformation and color segmentation were applied6Rizon et al. [29]D’Orazio et al. [5]Applied the object counting method7Ni et al. [7]Applied the CHT for detection among intersection objects8Sa et al. [20]Applied the deep convolutional neural networks for fruit detection9Zheng et al. [23]Deep learning procedures for the large datasets have been implemented here10Kuang et al. [25]HOGm LBP-, Gabor-, and histogram-based methods have been utilized here11Wang et al. [30]With the cascaded classifiers, Otsu method, with color thresholding, and an RGB camera, a laser rangefinder was used to estimate the mangoes length12Ali et al. [26]Applied the HOG features for detection and counting of mangoes in treesTable 3
Output efficiencies of energy harvesters from sources.
Sl. no.Source of powerOutput1Photovoltaic solar panels(i) 1781 MWh/lane-mile, Wang et al. [34](ii) 15 m W/cm2, Prauzek et al. [35]2Solar collectors588.634 kW/lane-mile, Wang et al. [34]3Thermoelectric0.748 kW/lane-mile, Wang et al. [34]4Geothermal4447.270 kW/lane-mile, Wang et al. [34]5Thermal(i) 15μW/cm3, Prauzek et al. [35](ii) 20–60 μW/cm2, Prauzek et al. [35](iii) 40 μW/cm3, Prauzek et al. [35]6Electromagnetic(i) 35.67 mW per system, Wang et al. [34](ii) 12nW/cm2, Prauzek et al. [35](iii) 0.2 mW/cm2–1 μW/cm2, Prauzek et al. [35]7Piezoelectric188 kW h/lane-mile, Wang et al. [34]8Wind(i) 28.5 mW/cm2, Prauzek et al. [35](ii) 3.8 x 10-4 W/cm3, Prauzek et al. [35]Table 4
Literature review on energy harvesters using single sources.
Sl. no.Ambient sourcesLiterature scheme1Solar: photovoltaic(i) Kang-Won and Correia [36](ii) Paradiso and Starner [37](iii) Bialasiewicz [38](iv) Wakulat [39](v) Andriopoulou [40](vi) Efthymiou et al. [41](vii) Golden et al. [42](viii) Nordmann and Clavadetscher [43](ix) Nordmann et al. [44]2Solar: solar collectors(i) Paradiso and Starner [37](ii) Eugster [45](iii) Chen et al. [46](iv) Pan et al. [47](v) Zwarycz [48](vi) Wu and Yu [49](vii) Matrawy and Farkas [50](viii) Chen et al. [51](ix) García and Partl [52](x) Pascual-Muñoz et al. [53](xi) Nasir et al. [54](xii) Nasir et al. [55](xiii) Guldentops et al. [56]3Thermoelectric(i) Paradiso and Starner [37](ii) Yildiz [57](iii) Abdelkefi et al. [58](iv) Hasebe et al. [59](v) Wu and Yu [49](vi) Uchida et al. [60](vii) Goldsmid [61](viii) Datta et al. [62]4Electromagnetic (EMF)(i) Paradiso and Starner [37](ii) Fthenakis and Kim [63](iii) Rabaey et al. [64](iv) Lu et al. [65](v) Elliott and Zilletti [66](vi) Peralta et al. [67]5Vibration based piezoelectric(i) Moure et al. [19](ii) Chongfeng Wei [68](iii) Liu et al. [69](iv) Roshani et al. [70](v) Yesner et al. [71](vi) Fan et al. [72]6Vibration based electromagneticChongfeng Wei [68]7Vibration based electrostatic/human walking(i) Paradiso and Starner [37](ii) Chongfeng Wei [68](iii) Yang et al. [73]8Wind/airflowParadiso and Starner [37]9LightParadiso and Starner [37]10Push buttonsParadiso and Starner [37]11Human motionYlli et al. [74]12Hand generatorsParadiso and Starner [37]13Heel strikeParadiso and Starner [37]14Geo-thermal(i) Tota-Maharaj and Paul [75](ii) Shen et al. [76]15Kinetic energy from running vehicleZhang et al. [77]16Chemical(i) McCarty and Whitesides [78](ii) Wang [79](iii) Fan et al. [80]17Wave energyLin et al. [81]Table 5
Literature review on hybrid energy harvesters using multiple sources.
Sl. no.Ambient sourcesLiterature scheme1Triboelectric and electromagnetic(i) Seol et al. [3](ii) Hu et al. [82](iii) Chen et al. [83](iv) Salauddin et al. [84]2Ferrofluid based vibration energySeola et al. [85]3Electromagnetic and piezoelectric(i) Uluşan et al. [86](ii) Muhammad Iqbal [87](iii) Fokou et al. [88](iv) Tao Yang [89](v) Toyabur et al. [90]4Vibration(i) Quintero et al. [91](ii) Sang et al. [92]5Thermo-electrics and photovoltaicsMuhtaroglu et al. [93]6Light and vibration energyYu et al. [94]7Thermal and infraredGhosh et al. [95]8Piezoelectric and triboelectricLi et al. [96]9Solar cells and triboelectricCho et al.; Rashid et al.; Ahmed et al.; Iwendi et al.; Alazab et al. [97–103]


### 1.2. Objectives

The research outlines the following objectives:(i)
To design and simulate a hybrid energy harvester to achieve the required power using solar and EM energy(ii)
To develop and implement algorithms for the smart agriculture monitoring system using convolutional neural network to detect, quantify, and classify objects in machine learning platform(iii)
To integrate and validate the hybrid energy harvester IoT-based smart agriculture monitoring system


### 1.3. Description of the Proposed System

A tomato field in Chittagong of Bangladesh has been chosen as the small-scaled agriculture field. A small lower-cost camera container robot was built for the mobility of the small camera over the land, as depicted in Figures1 and 2 through guard lanes (20 inches in length and 20 inches in gap).Figure 1
The block diagram (proposed) of the smart agriculture monitoring system with hybrid energy harvester.Figure 2
Work flow diagram of the proposed smart agriculture monitoring system with hybrid energy harvester.This robot and the camera will consume a maximum power of approximately 8500 MWh. Such an amount of power supply from a single source will be difficult, and multiple external sources need to be considered to design and integrate within a system. That is why, a hybrid energy harvester has been planned as a significant component of this research to be designed and prototyped.


## 1.1. The Problem Statement and Its Justification

FA Farmers’ traditional way of cultivating and perceptions on crops in the field [1] do not reflect the accurate economic value much, since ripening of tomatoes likely depends on colors. That is why, the detection and the quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields.Traditional methods are appropriate for agricultural monitoring on a small scale [1]. In such systems, the circular Hough transformation (CHT) is appropriate for objects with regular forms, such as circles [4] [5] [6] [7] [8]. The thresholding and color segmentation techniques are advantageous [9]. Simultaneously, the detection and quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields.Traditional methods are appropriate for agricultural monitoring on a small scale [1]. In such systems, the circular Hough transformation (CHT) is appropriate for objects with regular forms, such as circles [4] [5] [6] [7] [8]. The thresholding and color segmentation techniques are advantageous [9]. Simultaneously, classifiers [10] [11] [12] [13], image filters [14], support vector machines [15], and neural networks [15] may get higher outcomes when detecting fruits or crops in the agriculture industry.CHT can be used to detect and quantify tomatoes, while thresholding and segmentation can be used to detect defects or black patches on the outer peel [16].Detecting and quantifying ripened, green, and tomatoes with black spots on the outer peel in the field would undoubtedly lower farmers’ workloads while continuing to play a critical role in sustaining market food values throughout tomato package distribution.Due to energy constraints of sensors, to provide a continuous required supply of power, an energy harvester will be used, and a hybrid energy harvester will be designed and implemented, which will harvest energies from multiple sources of ambient energies from the environment [17] [18] [19].Previous studies at a glimpse are highlighted in Tables1–5, where those researched were limited to detecting or counting crops, fruits, or vegetables from fields. Detection quantification of tomatoes and classifying them as ripened, semiripened, or green followed by identifying defective vegetables have not yet been addressed. Moreover, a digitized monitoring system with a hybrid energy harvester that will scavenge energies from multiple sources added another novelty in this research.Table 1
Performance comparison of image processing methods for agriculture monitoring systems.
Sl. no.AuthorsMethods and schemePerformance1Ni et al. [7]The method, CHT was applied on overlapped objects2Sa et al. [20]Applied the deep CNN along with RGB color followed by near-infrared and faster R-CNN80% to 83%.3Jimenez et al. [21]The CCD sensor85%4Zhang et al. [22]The FSCABC algorithm and feed-forward neural-network (FNN)87%5Zheng et al. [23]Deep learning method was applied, and deep learning classifier was trained having a larger dataset99%6Mureșan and M. Oltean [24]Neural network—7Kuang et al. [25]HOG features were applied along with LBP and Gabor-LBP followed by global color, global shape features, and histogramA lower miss rate of 0.0377 with false positives at 0.0682 per image8Ali et al. 2017 [26]Applied HOG features for detection and counting mangoes in trees—Table 2
Comparison on related works.
Sl. no.AuthorsComparison1Crow [27]Applied the object-detection procedures along with mapping of the texture procedures2Zhuang et al. [13]Applied the cascaded-classifiers using Haar-features with an objective of improvement of the paleness3BaoHua et al. [28]Color transformation and machine learning were applied4Barbedo [1]Mukhopadhyay and Chaudhuri [6]Yuen et al. [8]Applied the CHT method for detection of objects5Liebig [9]Color transformation and color segmentation were applied6Rizon et al. [29]D’Orazio et al. [5]Applied the object counting method7Ni et al. [7]Applied the CHT for detection among intersection objects8Sa et al. [20]Applied the deep convolutional neural networks for fruit detection9Zheng et al. [23]Deep learning procedures for the large datasets have been implemented here10Kuang et al. [25]HOGm LBP-, Gabor-, and histogram-based methods have been utilized here11Wang et al. [30]With the cascaded classifiers, Otsu method, with color thresholding, and an RGB camera, a laser rangefinder was used to estimate the mangoes length12Ali et al. [26]Applied the HOG features for detection and counting of mangoes in treesTable 3
Output efficiencies of energy harvesters from sources.
Sl. no.Source of powerOutput1Photovoltaic solar panels(i) 1781 MWh/lane-mile, Wang et al. [34](ii) 15 m W/cm2, Prauzek et al. [35]2Solar collectors588.634 kW/lane-mile, Wang et al. [34]3Thermoelectric0.748 kW/lane-mile, Wang et al. [34]4Geothermal4447.270 kW/lane-mile, Wang et al. [34]5Thermal(i) 15μW/cm3, Prauzek et al. [35](ii) 20–60 μW/cm2, Prauzek et al. [35](iii) 40 μW/cm3, Prauzek et al. [35]6Electromagnetic(i) 35.67 mW per system, Wang et al. [34](ii) 12nW/cm2, Prauzek et al. [35](iii) 0.2 mW/cm2–1 μW/cm2, Prauzek et al. [35]7Piezoelectric188 kW h/lane-mile, Wang et al. [34]8Wind(i) 28.5 mW/cm2, Prauzek et al. [35](ii) 3.8 x 10-4 W/cm3, Prauzek et al. [35]Table 4
Literature review on energy harvesters using single sources.
Sl. no.Ambient sourcesLiterature scheme1Solar: photovoltaic(i) Kang-Won and Correia [36](ii) Paradiso and Starner [37](iii) Bialasiewicz [38](iv) Wakulat [39](v) Andriopoulou [40](vi) Efthymiou et al. [41](vii) Golden et al. [42](viii) Nordmann and Clavadetscher [43](ix) Nordmann et al. [44]2Solar: solar collectors(i) Paradiso and Starner [37](ii) Eugster [45](iii) Chen et al. [46](iv) Pan et al. [47](v) Zwarycz [48](vi) Wu and Yu [49](vii) Matrawy and Farkas [50](viii) Chen et al. [51](ix) García and Partl [52](x) Pascual-Muñoz et al. [53](xi) Nasir et al. [54](xii) Nasir et al. [55](xiii) Guldentops et al. [56]3Thermoelectric(i) Paradiso and Starner [37](ii) Yildiz [57](iii) Abdelkefi et al. [58](iv) Hasebe et al. [59](v) Wu and Yu [49](vi) Uchida et al. [60](vii) Goldsmid [61](viii) Datta et al. [62]4Electromagnetic (EMF)(i) Paradiso and Starner [37](ii) Fthenakis and Kim [63](iii) Rabaey et al. [64](iv) Lu et al. [65](v) Elliott and Zilletti [66](vi) Peralta et al. [67]5Vibration based piezoelectric(i) Moure et al. [19](ii) Chongfeng Wei [68](iii) Liu et al. [69](iv) Roshani et al. [70](v) Yesner et al. [71](vi) Fan et al. [72]6Vibration based electromagneticChongfeng Wei [68]7Vibration based electrostatic/human walking(i) Paradiso and Starner [37](ii) Chongfeng Wei [68](iii) Yang et al. [73]8Wind/airflowParadiso and Starner [37]9LightParadiso and Starner [37]10Push buttonsParadiso and Starner [37]11Human motionYlli et al. [74]12Hand generatorsParadiso and Starner [37]13Heel strikeParadiso and Starner [37]14Geo-thermal(i) Tota-Maharaj and Paul [75](ii) Shen et al. [76]15Kinetic energy from running vehicleZhang et al. [77]16Chemical(i) McCarty and Whitesides [78](ii) Wang [79](iii) Fan et al. [80]17Wave energyLin et al. [81]Table 5
Literature review on hybrid energy harvesters using multiple sources.
Sl. no.Ambient sourcesLiterature scheme1Triboelectric and electromagnetic(i) Seol et al. [3](ii) Hu et al. [82](iii) Chen et al. [83](iv) Salauddin et al. [84]2Ferrofluid based vibration energySeola et al. [85]3Electromagnetic and piezoelectric(i) Uluşan et al. [86](ii) Muhammad Iqbal [87](iii) Fokou et al. [88](iv) Tao Yang [89](v) Toyabur et al. [90]4Vibration(i) Quintero et al. [91](ii) Sang et al. [92]5Thermo-electrics and photovoltaicsMuhtaroglu et al. [93]6Light and vibration energyYu et al. [94]7Thermal and infraredGhosh et al. [95]8Piezoelectric and triboelectricLi et al. [96]9Solar cells and triboelectricCho et al.; Rashid et al.; Ahmed et al.; Iwendi et al.; Alazab et al. [97–103]


## 1.2. Objectives

The research outlines the following objectives:(i)
To design and simulate a hybrid energy harvester to achieve the required power using solar and EM energy(ii)
To develop and implement algorithms for the smart agriculture monitoring system using convolutional neural network to detect, quantify, and classify objects in machine learning platform(iii)
To integrate and validate the hybrid energy harvester IoT-based smart agriculture monitoring system


## 1.3. Description of the Proposed System

A tomato field in Chittagong of Bangladesh has been chosen as the small-scaled agriculture field. A small lower-cost camera container robot was built for the mobility of the small camera over the land, as depicted in Figures1 and 2 through guard lanes (20 inches in length and 20 inches in gap).Figure 1
The block diagram (proposed) of the smart agriculture monitoring system with hybrid energy harvester.Figure 2
Work flow diagram of the proposed smart agriculture monitoring system with hybrid energy harvester.This robot and the camera will consume a maximum power of approximately 8500 MWh. Such an amount of power supply from a single source will be difficult, and multiple external sources need to be considered to design and integrate within a system. That is why, a hybrid energy harvester has been planned as a significant component of this research to be designed and prototyped.


## 2. Literature Review

A comparative tabulated list is shown in Table1 on the accuracy of different image processing methods and technologies.


### 2.1. Technologies Used in Smart Agriculture Monitoring System

Table2 focuses on a comparison of characteristics of related research.In addition to the related works mentioned (Table2), the authors in [31] have worked on object recognition with a cascaded structured type of classifiers, and in [12, 32], the work was implemented using HAAR based transformations along with Boosting filters. The researchers in [11] worked on the Kalman filters to attain the optimal results, where the researchers in [33] implemented hardware-based interface for better outcomes. In [21], a computer vision-based survey system was developed, whereas in [22, 28] the research work was mainly based on design and development of machine learning-based system.


### 2.2. Technologies for Hybrid Energy Harvester

Wang et al. [34] investigated performances of energy harvesters in terms of material design, system optimization, laboratory, and field works led by a numerical model in theory. For the proposal, we have tabulated the comparison of different sources in Table 3.Reviews on energy harvesters using single sources are highlighted in Table2, and those of using multiple sources are highlighted in Table 4.


## 2.1. Technologies Used in Smart Agriculture Monitoring System

Table2 focuses on a comparison of characteristics of related research.In addition to the related works mentioned (Table2), the authors in [31] have worked on object recognition with a cascaded structured type of classifiers, and in [12, 32], the work was implemented using HAAR based transformations along with Boosting filters. The researchers in [11] worked on the Kalman filters to attain the optimal results, where the researchers in [33] implemented hardware-based interface for better outcomes. In [21], a computer vision-based survey system was developed, whereas in [22, 28] the research work was mainly based on design and development of machine learning-based system.


## 2.2. Technologies for Hybrid Energy Harvester

Wang et al. [34] investigated performances of energy harvesters in terms of material design, system optimization, laboratory, and field works led by a numerical model in theory. For the proposal, we have tabulated the comparison of different sources in Table 3.Reviews on energy harvesters using single sources are highlighted in Table2, and those of using multiple sources are highlighted in Table 4.


## 3. Methodology

### 3.1. Development of Smart Agriculture Monitoring System

According to the flow chart (Figure3), the circular Hough transformation (CHT) method has been applied to detect and quantify tomatoes in the fields. Outer skin black spots as defects of tomatoes have been detected using thresholding and segmentation.Figure 3
Flow chart of the system.The two approaches listed below have been implemented for the detection, quantification, ripeness checking, and identification of damaged (or defected) tomatoes.(i)
The color thresholding (CT)(ii)
The cascaded object detection (COD) methodA machine learning algorithm using CNN with the COD was also applied for the same purpose. In the end, a comparison between those traditional methods and machine learning algorithms (CNN) was projected to conclude the optimal feasible method to obtain research objectives.


### 3.2. Development of Hybrid Energy Harvester

From Table4, it is shown that solar panels and electromagnetic sources are optimal sources of energies to be saved up by energy harvesters. Hence, this research also proposes to harvest energies from these two sources. As limitation and availability of power is a challenge, we are planning to use both the sources (solar and electromagnetic) within a single hybrid generator to harvest energy to supply power to sensors of the smart agriculture monitoring system.Algorithm 1: Pseudocode for this algorithm.
a. steps to detect tomatoes (with presence of solid nature of background)1 Input image acquisition Process for Tomatoes2 Detection process and also Counting by means of carrying out circular Hough transformationb. steps to detect tomatoes (with presence of natural type of background)1 Input image acquisition Process for Tomatoes2 Carrying out adjustment process of colors in the images3 Modification of Internal type of Threshold values4 Modification of sensitivity values5 Modifying the polarity of Objects recognised6 Detection process and also Counting by means of carrying out circular Hough transformationStep 1: Image acquired and ReadStep 2: Calculate range of radius with minimum and maximum acceptable valuesIterative process from Step 3 and upto 7th Step until the desired value is attainedStep 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94Step 4: Call function to find circleStep 5: Lower value of edge thresholdStep 6: Return values of centre and radiusStep 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1Step 8: Add label reading COUNT variable and displayENDIFINDCIRCLESStep 1: Call function corresponding to circles and their methods


#### 3.2.1. For Checking Ripened Tomatoes

Algorithm 2: Pseudocode for ripeness checking using color thresholding.
Step 1: Read the input imageStep 2: Compute radii range by Minimum radius and Maximum radiusRepeat from Step 3 to Step 7 until the desired output is NOT found from a single imageStep 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94Step 4: Call function to find circleStep 5: Lower value of edge thresholdStep 6: Return values of centre and radiusStep 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1Step 8: Add label reading COUNT variable and displayENDIFINDCIRCLESStep 1: Call function corresponding to circles and their methods


#### 3.2.2. For Classification of Ripened Tomatoes

(1)
Get all the regions of detected tomatoes(2)
Average values of R, G, B channels are attained(3)
Using the intensity value rules, categorise tomatoes as follows:(a)
The ripened stage(b)
The semiripened stage(c)
The green stage


#### 3.2.3. Algorithm using Machine Learning

Datasets for validation have been set as null (positive samples: 117; negative samples: 288).(i)
The positive valued dataset is to be loaded(ii)
The negative valued dataset is to be loaded(iii)
A cascaded object detector is trained for extracting HOG-related image features(iv)
There is image reading from the considered datasets(v)
Tomato detection is from the image dataset(vi)
Look up, and verify for ripened type of tomatoesAlgorithm 3:
(i) Read image IMGRepeat following steps for J =1--3Color Segmentation are performed on Red , Green and BlueBreak up the image in to 3 channels as per Red , Green and Blue(ii) plot the three-separate channel of RGB(iii) Threshold value is defined for identification of Colors (110 – Red, 115 – Green and 240- Blue)(iv) Functions for the purpose of filling, structuring element and morphological dilation are called for operation(v) The image is processed and displayed in a window(vi) Threshold levels are set for RGB Colors(vii) Image conversion to Binary form using im2bw functions for each level(viii) Carryout the summation of the subdivision values and display the attained binary type of imageFinally END the procedure.


## 3.1. Development of Smart Agriculture Monitoring System

According to the flow chart (Figure3), the circular Hough transformation (CHT) method has been applied to detect and quantify tomatoes in the fields. Outer skin black spots as defects of tomatoes have been detected using thresholding and segmentation.Figure 3
Flow chart of the system.The two approaches listed below have been implemented for the detection, quantification, ripeness checking, and identification of damaged (or defected) tomatoes.(i)
The color thresholding (CT)(ii)
The cascaded object detection (COD) methodA machine learning algorithm using CNN with the COD was also applied for the same purpose. In the end, a comparison between those traditional methods and machine learning algorithms (CNN) was projected to conclude the optimal feasible method to obtain research objectives.


## 3.2. Development of Hybrid Energy Harvester

From Table4, it is shown that solar panels and electromagnetic sources are optimal sources of energies to be saved up by energy harvesters. Hence, this research also proposes to harvest energies from these two sources. As limitation and availability of power is a challenge, we are planning to use both the sources (solar and electromagnetic) within a single hybrid generator to harvest energy to supply power to sensors of the smart agriculture monitoring system.Algorithm 1: Pseudocode for this algorithm.
a. steps to detect tomatoes (with presence of solid nature of background)1 Input image acquisition Process for Tomatoes2 Detection process and also Counting by means of carrying out circular Hough transformationb. steps to detect tomatoes (with presence of natural type of background)1 Input image acquisition Process for Tomatoes2 Carrying out adjustment process of colors in the images3 Modification of Internal type of Threshold values4 Modification of sensitivity values5 Modifying the polarity of Objects recognised6 Detection process and also Counting by means of carrying out circular Hough transformationStep 1: Image acquired and ReadStep 2: Calculate range of radius with minimum and maximum acceptable valuesIterative process from Step 3 and upto 7th Step until the desired value is attainedStep 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94Step 4: Call function to find circleStep 5: Lower value of edge thresholdStep 6: Return values of centre and radiusStep 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1Step 8: Add label reading COUNT variable and displayENDIFINDCIRCLESStep 1: Call function corresponding to circles and their methods


### 3.2.1. For Checking Ripened Tomatoes

Algorithm 2: Pseudocode for ripeness checking using color thresholding.
Step 1: Read the input imageStep 2: Compute radii range by Minimum radius and Maximum radiusRepeat from Step 3 to Step 7 until the desired output is NOT found from a single imageStep 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94Step 4: Call function to find circleStep 5: Lower value of edge thresholdStep 6: Return values of centre and radiusStep 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1Step 8: Add label reading COUNT variable and displayENDIFINDCIRCLESStep 1: Call function corresponding to circles and their methods


### 3.2.2. For Classification of Ripened Tomatoes

(1)
Get all the regions of detected tomatoes(2)
Average values of R, G, B channels are attained(3)
Using the intensity value rules, categorise tomatoes as follows:(a)
The ripened stage(b)
The semiripened stage(c)
The green stage


### 3.2.3. Algorithm using Machine Learning

Datasets for validation have been set as null (positive samples: 117; negative samples: 288).(i)
The positive valued dataset is to be loaded(ii)
The negative valued dataset is to be loaded(iii)
A cascaded object detector is trained for extracting HOG-related image features(iv)
There is image reading from the considered datasets(v)
Tomato detection is from the image dataset(vi)
Look up, and verify for ripened type of tomatoesAlgorithm 3:
(i) Read image IMGRepeat following steps for J =1--3Color Segmentation are performed on Red , Green and BlueBreak up the image in to 3 channels as per Red , Green and Blue(ii) plot the three-separate channel of RGB(iii) Threshold value is defined for identification of Colors (110 – Red, 115 – Green and 240- Blue)(iv) Functions for the purpose of filling, structuring element and morphological dilation are called for operation(v) The image is processed and displayed in a window(vi) Threshold levels are set for RGB Colors(vii) Image conversion to Binary form using im2bw functions for each level(viii) Carryout the summation of the subdivision values and display the attained binary type of imageFinally END the procedure.


## 3.2.1. For Checking Ripened Tomatoes

Algorithm 2: Pseudocode for ripeness checking using color thresholding.
Step 1: Read the input imageStep 2: Compute radii range by Minimum radius and Maximum radiusRepeat from Step 3 to Step 7 until the desired output is NOT found from a single imageStep 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94Step 4: Call function to find circleStep 5: Lower value of edge thresholdStep 6: Return values of centre and radiusStep 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1Step 8: Add label reading COUNT variable and displayENDIFINDCIRCLESStep 1: Call function corresponding to circles and their methods


## 3.2.2. For Classification of Ripened Tomatoes

(1)
Get all the regions of detected tomatoes(2)
Average values of R, G, B channels are attained(3)
Using the intensity value rules, categorise tomatoes as follows:(a)
The ripened stage(b)
The semiripened stage(c)
The green stage


## 3.2.3. Algorithm using Machine Learning

Datasets for validation have been set as null (positive samples: 117; negative samples: 288).(i)
The positive valued dataset is to be loaded(ii)
The negative valued dataset is to be loaded(iii)
A cascaded object detector is trained for extracting HOG-related image features(iv)
There is image reading from the considered datasets(v)
Tomato detection is from the image dataset(vi)
Look up, and verify for ripened type of tomatoesAlgorithm 3:
(i) Read image IMGRepeat following steps for J =1--3Color Segmentation are performed on Red , Green and BlueBreak up the image in to 3 channels as per Red , Green and Blue(ii) plot the three-separate channel of RGB(iii) Threshold value is defined for identification of Colors (110 – Red, 115 – Green and 240- Blue)(iv) Functions for the purpose of filling, structuring element and morphological dilation are called for operation(v) The image is processed and displayed in a window(vi) Threshold levels are set for RGB Colors(vii) Image conversion to Binary form using im2bw functions for each level(viii) Carryout the summation of the subdivision values and display the attained binary type of imageFinally END the procedure.


## 4. Result and Discussion

Consider the following scenario: assume that thepnumber of the positive type of datasets, on the same notennumber of negative datasets, are available, and also, there is availability of database of optimum amount of bootstrapping negative version of the datasets represented by d.Let there be “g” learning objectives mentioned; subsequently, a cascade of useful binary nodes of the classifiers are available as stated below.There is a seriesH1,H2,H3,⋯⋯⋯⋯⋯,Hn… up to “n” number of stages.Here,n>>p.The learning will be based on the number of positive samples at each of the learning stageH given by
(1)No.of+iveSample=floorp1+n–1∗1–TPR,where TPR stands for true positive rate value, whereas FPR for false positive rate values, and on the same note FNR represents false negative rate values.Tomatoes are sorted into three groups using a cascaded classifier: the ripened stage, the semiripened stage, and the green colored stage as shown in (Figure4).Figure 4
Input images (1) and (3) for algorithms of detection of quantification of tomatoes and output images (2) and (4) using color thresholding method.


### 4.1. Method-Wise Comparative Analysis Process

The decision table (Table6) accurately reflects the classification of the stage of the ripened tomatoes is based on the rubrics using diverse color weighted values. Based on the category, wise applicability rules are divided into three parts. The first rule applies to ripened selections, whereas the second and third rules apply to the semiripened and also the green selections, correspondingly.Table 6
Decision table.
Rule no.RavgGavgBavgDecision1.>160.0<90.0<60.0Ripened2.<160.0>50.0<60.0Semiripened3.<50.0.>50.0<50.0GreenBased on Table7, sensitivity was set to 0.92 (comparing with the previous value 0.90) for obtaining optimal detection. Since using circles, the tomato detection was lower when the color-transformation-method was utilized.Table 7
Optical detection parameters [16].
SubjectRegularRevisedCircle finding methodTwo-stagePhase codeSensitivity0.90≥0.92Object polarityDarkDarkEdge threshold0.110.11The object polarity here is established as dark and gloomy (the detector performs better detection of objects which are brighter than the visibility of the background portion of the image)The method of circle finding was also updated yet again, this time using Phase-code, which is both faster type and also more vigorous to noise when compared to the previous method involving two-stage procedure.In addition, the type of outcome has been revised from Table8. False positive type of outcomes has been eliminated; nevertheless, the number of undiscovered tomatoes has been increased to two.Table 8
Comparative analysis between revised and regular parameters.
TopicRegular parametersRevised parametersTotal tomatoes in image77Detected85False positive20Undetected12The machine learning method outperformed the traditional method in terms of accuracy and run time complexity when it came to tomato detection. However, from the analysis, it has been found that by utilizing the COD, the ripeness complexity isOn2. In contrast, with the CSM, the ripeness complexity is reduced to On (refer to Table 9).Table 9
Comparison of COD and CSM methods.
TopicCSMCODDatasetPositiveX138Negative288False detection accuracy85%95%Time complexityOnOn2Ion the basis of Table10, it has been found that the detection ratio is higher in proportion to the actual number of tomatoes harvested. In addition, the rate of false-positive type of the results has been lowered in relation to the rate of detected tomatoes (Table 11).Table 10
No. of detected tomatoes.
Sl. no.No. of tomatoes existedNo. of tomatoes detected175286397411851196131071413Table 11
No. of false positives.
Sl. no.No. of tomatoes detectedFalse positive15126337248359261037133


### 4.2. Performances of Algorithm Detection and Quantification

With both the color thresholding and machine learning methods, from input images (Figures4(1), 5, and 6(a)), Tomatoes from fields were detected and quantified (Figures 6(b) and 7–9 in sunny light) using detection and quantification algorithms (whose flow chart is in Figure 10). In contrast, from input images of Figure 4(1), ripened tomatoes have been detected and quantified (Figures 4(2) and 4(4)) using color thresholding, color transformation methods, and cascaded object detector (COD) of machine learning method (whose flow chart is in Figure 7).Figure 5
Cascaded object detector (COD) method.Figure 6
(a) Input image for detection and quantification of tomatoes and (b) detected and quantified output 1.
(a)(b)Figure 7
Image used as a COD input (captured in the light of sun).Figure 8
Output (detected tomatoes) (in sunny lighting).Figure 9
Detected and quantified output 2.Figure 10
Algorithm1: detection of crops.


### 4.3. Performances of Algorithm in Identification of Tomatoes with Defects

With the color segmentation method, using an input image (Figure4), defected tomatoes were identified and output images are given in Figures 11 and 12 where RGB values were applied before the binary conversion and in Figure 13 where RGB values were applied after the binary conversion.Figure 11
After the color segmentation method of red, green, and blue (before BW).Figure 12
Original image of a damaged tomato as input.Figure 13
After the color segmentation method of red, green, and blue (after BW).


## 4.1. Method-Wise Comparative Analysis Process

The decision table (Table6) accurately reflects the classification of the stage of the ripened tomatoes is based on the rubrics using diverse color weighted values. Based on the category, wise applicability rules are divided into three parts. The first rule applies to ripened selections, whereas the second and third rules apply to the semiripened and also the green selections, correspondingly.Table 6
Decision table.
Rule no.RavgGavgBavgDecision1.>160.0<90.0<60.0Ripened2.<160.0>50.0<60.0Semiripened3.<50.0.>50.0<50.0GreenBased on Table7, sensitivity was set to 0.92 (comparing with the previous value 0.90) for obtaining optimal detection. Since using circles, the tomato detection was lower when the color-transformation-method was utilized.Table 7
Optical detection parameters [16].
SubjectRegularRevisedCircle finding methodTwo-stagePhase codeSensitivity0.90≥0.92Object polarityDarkDarkEdge threshold0.110.11The object polarity here is established as dark and gloomy (the detector performs better detection of objects which are brighter than the visibility of the background portion of the image)The method of circle finding was also updated yet again, this time using Phase-code, which is both faster type and also more vigorous to noise when compared to the previous method involving two-stage procedure.In addition, the type of outcome has been revised from Table8. False positive type of outcomes has been eliminated; nevertheless, the number of undiscovered tomatoes has been increased to two.Table 8
Comparative analysis between revised and regular parameters.
TopicRegular parametersRevised parametersTotal tomatoes in image77Detected85False positive20Undetected12The machine learning method outperformed the traditional method in terms of accuracy and run time complexity when it came to tomato detection. However, from the analysis, it has been found that by utilizing the COD, the ripeness complexity isOn2. In contrast, with the CSM, the ripeness complexity is reduced to On (refer to Table 9).Table 9
Comparison of COD and CSM methods.
TopicCSMCODDatasetPositiveX138Negative288False detection accuracy85%95%Time complexityOnOn2Ion the basis of Table10, it has been found that the detection ratio is higher in proportion to the actual number of tomatoes harvested. In addition, the rate of false-positive type of the results has been lowered in relation to the rate of detected tomatoes (Table 11).Table 10
No. of detected tomatoes.
Sl. no.No. of tomatoes existedNo. of tomatoes detected175286397411851196131071413Table 11
No. of false positives.
Sl. no.No. of tomatoes detectedFalse positive15126337248359261037133


## 4.2. Performances of Algorithm Detection and Quantification

With both the color thresholding and machine learning methods, from input images (Figures4(1), 5, and 6(a)), Tomatoes from fields were detected and quantified (Figures 6(b) and 7–9 in sunny light) using detection and quantification algorithms (whose flow chart is in Figure 10). In contrast, from input images of Figure 4(1), ripened tomatoes have been detected and quantified (Figures 4(2) and 4(4)) using color thresholding, color transformation methods, and cascaded object detector (COD) of machine learning method (whose flow chart is in Figure 7).Figure 5
Cascaded object detector (COD) method.Figure 6
(a) Input image for detection and quantification of tomatoes and (b) detected and quantified output 1.
(a)(b)Figure 7
Image used as a COD input (captured in the light of sun).Figure 8
Output (detected tomatoes) (in sunny lighting).Figure 9
Detected and quantified output 2.Figure 10
Algorithm1: detection of crops.


## 4.3. Performances of Algorithm in Identification of Tomatoes with Defects

With the color segmentation method, using an input image (Figure4), defected tomatoes were identified and output images are given in Figures 11 and 12 where RGB values were applied before the binary conversion and in Figure 13 where RGB values were applied after the binary conversion.Figure 11
After the color segmentation method of red, green, and blue (before BW).Figure 12
Original image of a damaged tomato as input.Figure 13
After the color segmentation method of red, green, and blue (after BW).


## 5. Discussion

The color transformation method was added with color thresholding (for input image in Figure4(1)) due to uneven brightness all over the image. The color adjustment was required for input Figure 4(1) to get both ripened and green tomatoes individually detected and quantified (Figures 4(2) and 4(3)). The object polarity was set to dark for common thresholding and object polarity. With this, the obtained accuracy for the color thresholding method was 85% approximately, and that for machine learning was 95% applying the Convolutional Neural Method (CNN) (see Figures 14 and 15).Figure 14
False positives vs. true positives using traditional approach (without machine learning).Figure 15
False positives vs. true positives using machine learning.


## 6. Conclusion

The traditional method comprising CHT and CSM performed an accuracy of 84%, whereas the COD with the CNN outperformed the traditional method with an accuracy of 92%. For detection and ripeness checking, this research was only conducted in sunlight, whereas in the future, the research will apply these methods under shadow or different weather parameters. Besides, damaged tomatoes with black spots at their outer surface have been considered only. In the future, damaged tomatoes with other characteristics such as changing shapes water elements in outer skin will be identified to help farmers achieve more accurate economic value. The research has only compared the output of different energy sources. In contrast, a hybrid energy harvester will be designed and simulated to achieve the required power using solar and EM energy.Hence, the future research of this study is aimed at developing a hybrid energy harvester that will harvest energies for sensors of this digitized agriculture monitoring system from multiple sources of energy. Additional vegetables will be added under monitoring with the different methodology for a better comparison of the system’s performance.



---

*Source: 7372053-2022-04-04.xml*