Counting pear flower buds
using worm’s-eye view images
Takumi Hanakawa∗ , Masashi Nishiyama∗‡ , Yoshio Iwai∗‡ , Eiji Morimoto† and Shintaro Nakatani∗ ,∗ Graduate School of Sustainability Science, Tottori University, Japan† Faculty of Agriculture, Tottori University, Japan‡ Cross-informatics Research Center, Tottori University

Abstract—We propose a method of counting pear flower buds
from sequential images acquired from a worm’s-eye view by adriven camera system. When only flower buds are detected ateach time point, the problem of duplicate counting of the same
flower buds may occur. Thus, in addition to the detection at eachtime point, our method identifies flower buds through keypoint
matching for sequential image pairs and then counts the flowerbuds. Experimental results show that our method has less error
in counting flower buds compared with using only a detector.
Index Terms—Pear flower buds, counting, worm’s-eye view,detection, keypoint matching𝐼𝑡
S1. Acquiring a worm'seye view image pair ofpear flower buds

S2. Detecting candidate
regions of flower buds
from image pairs

I. I NTRODUCTION
The social background of agriculture faces the problems ofan increase in physical labor, a decrease in the number ofagricultural workers, and an increase in the amount of work.
Smart agriculture, in which advanced information technology
is introduced into daily operations, is expected to overcome
these problems. In smart agriculture, crop growth information is acquired from driving devices equipped with cameras
and analyzed using artificial intelligence technologies. Highyielding and high-quality conditions are expected to be foundby comparing the growth information currently acquired withthat accumulated previously. Additionally, smart agriculture isexpected to optimize crop production control, reduce physicallabor, secure new farmers, and reduce labor.
In Tottori Prefecture, the introduction of smart agriculture
is being promoted to farms targeting pears, a specialty of theprefecture. We here consider the acquisition of crop growth
information on pear trees as an initial step in introducing
smart agriculture. Specifically, information on the growth ofpear trees includes information on flower buds, fruit, branches,and leaves. For example, the number of pears in summer [1],[2] has been counted for smart agriculture. In contrast, weconsider how to count the number of pear flower buds in early
spring for smart agriculture in the present study. Obtaining thenumber of pear flower buds in early spring and comparing itwith the number of fruit shipped at the beginning of fall canhelp farmers manage production. If a large volume of thisgrowth information is accumulated, there is the possibility to
predict the yield of pears accurately in the future.
We here propose a method of counting flower buds fromsequential images of pear trees acquired using a camera system
driven on the ground. Our method finds the regions of pear
flower buds from worm’s-eye view images using detectors

𝐼𝑡+1

Red: Candidate regions of flower buds

S3. Identifying thecandidate regions ofthe same flower budsin 𝐼𝑡 and 𝐼𝑡+1
Yellow: Regions of the same buds
Blue: Keypoints

S4. Counting flower buds
by removing the regionsof the same buds in 𝐼𝑡+1
Green: Regions for counting

Fig. 1. Overview of our method.

and then counts the number of pear flower buds throughkeypoint matching of the image pairs. In our experiments, weused a single-shot multibox detector (SSD) [3] and a youonly-look-once (YOLO) detector [4]. Furthermore, we used
SuperPoint [5] and SuperGlue [6] for keypoint matching. Theexperiments confirmed that the use of our method reduces
the error in counting flower buds compared with the simple
method of using only detectors.
II. O UR METHOD
A. Overview
We describe a method of counting flower buds adopting adetector and keypoint matching. Figure 1 shows an overviewof our method. First, sequential image pairs are acquired instep S1. An image pair comprises an image It and image It+1acquired consecutively in a time series. The camera system

B. Detection of flower buds
Shared areas

𝐼𝑡(a)

𝐼𝑡+1

Shared areas

𝐼𝑡(b)

𝐼𝑡+1

Shared areas

𝐼𝑡(c)𝐼𝑡+1Fig. 2. Examples of the same flower buds between images. These buds appear
in the shared areas of the image pairs.

is driven on the ground and acquires sequential images fromthe ground by looking up at the branches of a pear tree.
In step S2, our method detects candidate regions of flowerbuds from It and It+1 using a detector. However, it is notedthat conducting only step S2 generates duplicate-counting
problems. The ground of a pear farm is bumpy, and even if
the time interval between image acquisitions is constant, thedistance interval is not always constant. Thus, camera fieldsof view often overlap between time points. Examples of thisoverlapping are shown in Fig. 2. In the following, we referto the overlapping parts of an image pair of It and It+1 as
the shared areas. If only the detection of flower buds for It
and It+1 is adopted, there can be no determination of whether
the detected flower buds are the same, which results in theduplicate counting of flower buds at the two time points.
Flower buds being the same means that they appear in thetwo shared areas of the image pair. We here consider solving
the problem of duplicate counting in steps S3 and S4 of ourmethod. In step S3, our method identifies the candidate regionsof the same flower buds detected from the shared areas in Itand It+1 . Our method extracts keypoints within the candidate
regions to identify the candidate regions. If the keypoints
extracted from the candidate regions of It correspond to thekeypoints extracted from the candidate region of It+1 , ourmethod determines that they are identical flower buds. In stepS4, our method excludes the same flower buds from It+1
through keypoint matching of the candidate regions. After
this exclusion process, the remaining candidate regions in theimage pair It and It+1 are counted as the number of flower
buds.

Object detection is a method of finding specific object
regions in images. Recent technological advances in object
detection have led to the use of deep learning techniques, suchas in [3], [4]. These techniques have the advantage of detecting
objects more accurately than methods without deep learningtechniques.
Even with deep-learning-based detection methods, there areissues that we must consider. If the detection target is small
compared with the image size, the detection accuracy is low.
As an example, we consider the case that an SSD [3] is used
as a detector. The SSD300, one of the SSD implementations,internally reduces the size of input images It and It+1 to 300×300 pixels. In this case, the bounding box of the candidate
flower bud region in the resized input images in SSD300 issmall. Specifically, the median size of the resized boundingbox of a flower bud is 9×13 pixels. Thus, the resized boundingbox is small, making it difficult to detect candidate regions offlower buds.
In this paper, to improve the accuracy of detecting flower
bud candidate regions, we add a method of dividing the input
images It and It+1 into equal parcels and applying them toSSD. These images are divided equally into B parcels ofbItb (b = 1, . . . , B), and It+1
(b = 1, . . . , B). Inside SSD300,bbinput images It and It+1 are resized to 300 × 300 pixels. Thebbounding box of flower buds after the resizing of Itb and It+1
is approximately B times as large as that of the original It andIt+1 . By dividing It and It+1 equally, the bounding box ofthe candidate region after resizing is enlarged, which mitigates
the issue of candidate regions not being detected.
C. Keypoint matching of candidate regions of flower budsKeypoint matching is a method of extracting local descriptors representing features around keypoints such as corner
points from an image pair and identifying the same local
descriptors for the images. There are several methods for
keypoint matching. In this paper, we use SuperPoint [5] forextracting local descriptors and SuperGlue [6] for identifying
the local descriptors.
SuperPoint is a method of extracting the local descriptorsof keypoints from images with a deep neural network. WithinSuperPoint, the input image is resized to 1600 × 1067 pixels.
The median size of the bounding box of the candidate flower
bud region is then 48×48 pixels. The image size after resizing
is sufficient to extract the local descriptors. SuperPoint uses
It and It+1 as the original input images. After extracting
local descriptors from It and It+1 adopting SuperPoint, thesame local descriptors for these images are identified adopting
SuperGlue. SuperGlue is a deep neural-based method that
performs identification by finding the corresponding local descriptors in an image pair and rejecting the non-corresponding
local descriptors.
We suppose that the identification results obtained adopting
SuperGlue are used directly. In this case, candidate and noncandidate regions are occasionally misidentified. We make an(a) Camera system(c) Path froma bird’s-eye view600

Frequency

Frequency

600

4002000200400
Width [pixel]

600

400
2000200
400
Height [pixel]

600

Fig. 4. Histograms of the width and height of the bounding boxes of the pear
flower buds (manually labeled).

TABLE I
ACCURACY IN COUNTING PEAR FLOWER BUDS USING EACH OF THE
METHODS C1 , C2 , O1 AND O2 . W E PRESENT THE PREDICTED NUMBER OF
PEAR FLOWER BUDS AND THE ERROR RELATIVE TO THE CORRECT VALUE .

(b) Pear trees planted on a pear farm
(Red: Path of the camera system)
Fig. 3. We used a camera system driven along a path in the pear farm forimage acquisition.

improvement that identifies only within the detected candidate
regions to avoid this issue.
III. E XPERIMENTS
A. Dataset
We used sequential images acquired on a pear farm of
Fukube-Cho, Tottori, Tottori Prefecture, Japan. Figure 3(a)shows our camera system. This camera system comprises acamera (Canon EOS8000D), camera stabilizer (DJI RoninSC), and dolly. We considered using a drone to acquire images,
but strong spring winds in Fukube-Cho made it impossible
to acquire the images stably, and we thus used a camera
system that looked upward at the branches from the ground.
Figure 3(b) shows the pear trees planted on the pear farmand the paths of the camera system. Pear trees were planted
at equal spatial intervals. We acquired the sequential imagesof the pear flower buds along the path indicated by the redarrow. Figure 3(c) shows the path of the camera system froma bird’s-eye view. We used a path that was passable for thecamera system. The sequential images were acquired from10:28 AM to 4:39 PM on April 3, 2020. Pear leaves growquickly and the images were thus acquired on a single day
so that the additional growth of leaves would not obscure
flower buds. The number of worm’s-eye view images was 550,
and the image size was 6000 × 4000 pixels. A total of 7684
bounding boxes of flower buds were labeled manually. We
present histograms in Fig. 4 to visualize the frequency of theheight and width of the bounding boxes. The median widthand height of the bounding box were 181 and 180 pixels,respectively.
B. Basic performance
We evaluated whether our method is effective in preventing the duplicate counting of flower buds. The methods ofMethod
C1 (SSD)
C2 (YOLO)
O1 (SSD+SuperPoint+SuperGlue)
O2 (YOLO+SuperPoint+SuperGlue)

Prediction
1006
1082
755
832

Error
231
307
2057

evaluation are described below.
• C1 (SSD): Comparative method 1. We used only the
SSD [3].
• C2 (YOLO): Comparative method 2. We used only theYOLO detector [4].
• O1 (SSD+SuperPoint+SuperGlue): Our method 1. We
used the SSD [3] to find flower bud regions in step S2. We
then used SuperPoint [5] and SuperGlue [6] to identify
the same flower buds in step S3.
• O2 (YOLO+SuperPoint+SuperGlue): Our method 2. We
used the YOLO detector [4] to find flower bud regions instep S2. We then used SuperPoint [5] and SuperGlue [6]to identify the same flower buds in step S3.
Four-hundred images were used in detector training and 100
images in detector validation. The number of image divisions
described in Sec. II-B was set at B=8. The number of epochsin training of the SSD was set at 50 and the reliability at 0.4.
The number of epochs in training of the YOLO detector was
set at 90 and the reliability at 0.3. We selected YOLOv5x1
from several YOLO models. In SuperPoint, the maximum
number of keypoints was set at 3072, and the threshold for
keypoint extraction was set at 0.0001.
We describe an index for evaluating the accuracy of counting flower buds. In obtaining the correct number of flower
buds, we counted the buds manually to ensure no overlapsof flower buds between sequential images. We predicted thenumber of flower buds using each method. The absolutedifference between the correct number of flower buds and thepredicted number of flower buds was taken as the error.
Table I gives the accuracy in counting pear flower buds
using each of the methods C1 , C2 , O1 , and O2 . We evaluatedthe accuracy using 25 image pairs not used to train or validate
1 https://github.com/ultralytics/yolov5

TABLE II
ACCURACY WITH AND WITHOUT DIVIDING THE IMAGE EQUALLY INTO
PARCELS IN OUR METHOD O1 .
Image division
Without (B = 1)
With (B = 8)
𝐼𝑡(a)

𝐼𝑡+1

Prediction
133
755

Error
642
20

buds in image pair (b) is larger than that in image pair (a), thecandidate regions are still successfully identified. In Fig. 5(c),the sequential images were acquired in the evening, and thesky is thus darker than the sky in the images in (a) and (b).
We see that candidate regions are successfully identified evenin the evening when the sun was setting.
D. Effect of dividing the image equally into parcels

𝐼𝑡(b)

𝐼𝑡+1

𝐼𝑡(c)

𝐼𝑡+1

Fig. 5. Visualization of identifying the same flower buds in step S3 of ourmethod O1 . (Red: candidate regions of pear flower buds, Yellow: regions of
the same flower buds, Blue: keypoints).

the detector of candidate flower bud regions. The number offlower buds was counted manually for the 25 image pairs,resulting in a correct number of 775 flower buds. We compared
the accuracy between methods with the same base detector in
the following. The error for C1 was 231, and the error for O1was 20. The error for C2 was 307, and the error for O2 was
57. The errors of O1 and O2 were less than those of C1 andC2 . Furthermore, the error for O1 was smaller than that forO2 . These results confirm that our method has less error than
the simple method of using only detectors.
C. Visualization of identifying the same flower buds
We visualize the results of identifying the same flower buds
in sequential images in step S3 of our method O1 in Fig. 5. The
accuracy of O1 used in this figure is that of O1 in Table I. InFig. 5(a), It is the image acquired immediately after the branchentered the field of view. In It+1 , in addition to branches
and flower buds in It , other branches and flower buds comeinto view. We see that the candidate regions are successfully
identified because the yellow boxes correspond between theshared areas in the images. In Fig. 5(b), It is an image ofbranches and flower buds acquired from directly below thepear tree. In the sequential images, many flower buds are in
the field of view. We see that even when the number of flower

We conducted an experiment to confirm whether dividing
the image equally into parcels as described in Sec. II-B iseffective for our method. Table II shows the accuracy of ourmethod O1 with image division (B = 8) and without image
division (B = 1). The error of O1 with image division isless than that of O1 without image division. We confirm thatdetection using our method with image division is effective forsequential images of worm’s-eye view acquired by the camerasystem.
IV. C ONCLUSIONSWe proposed a method of detecting candidate regions ofpear flower buds from sequential images of worm’s-eye viewusing a detector. Candidate regions of the same flower buds are
identified through keypoint matching, and the flower buds arecounted. Experiments confirmed that our method prevents duplicate counting and reduces the error in counting pear flower
buds. In the experiments, false negatives in detecting candidate
flower bud regions were more frequent than false positives andkeypoint matching errors. In future work, we intend to develop
a method of reducing false negatives for flower buds with asmall bounding box. This work was partially supported by
Research Center for Sustainable Science, Tottori University.
R EFERENCES[1] A. I. B. Parico and T. Ahamed, “Real time pear fruit detection andcounting using yolov4 models and deep sort,” Sensors, vol. 21, no. 14,4803, 2021.
[2] K. Itakura, Y. Narita, S. Noaki, and F. Hosoi, “Automatic pear andapple detection by videos using deep learning and a kalman filter,” OSA
Continuum, vol. 4, no. 5, pp. 1688–1695, 2021.
[3] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, and A. Berg,“Ssd: Single shot multibox detector,” in Proceedings of the European
Conference on Computer Vision (ECCV), 2016, pp. 21–37.
[4] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only lookonce: Unified, real-time object detection,” in Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2016,
pp. 779–788.
[5] D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Selfsupervised interest point detection and description,” in Proceedings ofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition
Workshops (CVPRW), 2018, pp. 337–33712.
[6] P. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue:learning feature matching with graph neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition (CVPR), 2020, pp. 4937–4946.