Estimate Optimal Harvest Time by Cross-scale
Assimilated Digital Broccoli Farmland
Haozhou WANG, Tang LI, Erika NISHIDA, Yuya FUKANO, Yoichiro KATO, Wei GUO
haozhou-wang@outlook.com

U. TokyoU. Chiba

guowei@g.ecc.u-tokyo.ac.jp

Calculate cross-scale assimilated size

Background

Automatic ID generation for individual broccoli

In field food loss of fresh broccoli caused by non-standard shipping size is acommon problem nowadays, which is mainly affected by non-proper harvestingtime. A proper harvesting time means the maximum the proportion of standardshipping size, hard to estimate by conventional method.

Cropping

Training & detection

Labeling

ID generating byridge

model

Contribution
Propose a solution to predict the proper harvesting time.
Build digital twin of 7000+ broccoli from individual to whole field.

buffer area

Calculate the cross-scale assimilated size of broccoli.

Detected location(part)

Labelme

DOMseedling

Head identification & segmentation

Predict the broccoli size by growth model.

ID transferring to raw image

Predict profits by market price and growth model.

Select the closest one

1500 px

DOM~ 100 px

Coded groundcontrol points

DSM

Powered by EasyIDP(scales)

RTK Phantom 4• broccoli ID

Croppedsector(3D reconstruction)

Batch
processingscripts

Segment result

1500 px

Digital twin of in-field grown broccoli
Aerial scale by drone

Save to *.shp file

BiSeNetV2 modelInteractiveannotation
On raw UAV image

ROI defining

Cross-scale head assimilation

Time-series geo-referenced data

Ground scale by smart phone

DL

Same sampled regiontransfer(3D reconstruction)
Poor UAV (15m) 3D model & DOM

Simple 2D polygon

Link by sampledbroccoli ID

Simple 2D
geometry traits

Obtain the regression
model from UAV traits
to indoor traits

Selfie stick

Batch processing
scripts
Ground scalebar

Segment

Time-series ground data
Same sampled broccoli

Indoor scale by photo studio

Normal 3D point
cloud

Normal quality ground 3D point cloud(3D reconstruction)

Simple 3D
geometry traits(validation)

Put indoor model backto outdoor by templatematching andtransformation

Pickvertex

Rotating platform

scalebar
Deep learning
broccoli masking

Masked 3D
reconstruction

High quality 3D model(also rotate object itself)(UNet +CascadePSP)(fix camera, rotate object)(200+ databases)

Market survey and broccoli grading
HD: head diameter
T: Sum of daily average temperature
Model 1other size can not be sold and wasted(grade-out)

Case 2: grades with smallest
difference ΔC2

Cases
Cases

3000

Predict size ofeach broccoli,and show
distribution

2000

L size = $1
Case 1:
2L = $1 + ΔC1
M = $1 - ΔC1

Case 2:
2L = $1 + ΔC2
M = $1 - ΔC2

Income predict

Last year’s UAV
size regression
model

Income ($)

Case1

Case1

Case2

Case2

Case3

Case3

Grade-out
Proportion(%)

𝑀,𝐿,2𝐿

𝐼𝑛𝑐𝑜𝑚𝑒 = ෍ 𝐶𝑜𝑢𝑛𝑡𝑔 × 𝑃𝑟𝑖𝑐𝑒𝑔𝑔Grade-out price = 0

M
28

ayMay

In this case, May 17 is the optimal
harvest date

28

M
2a7yay
M
2a6y

27

26

date

M

ay
M
2a5y

M

ay
M
M
2a4yay25

MM
2a3yayMM
2a2yay
M
M
2a1y

22

M
M
2a0y

ay

1000

211000

ay

2000

M

For this year:
Flight when broccoli size is visible (May 12), using model 1
get T0, and weather forecast ΔT and model 2 to predict size

3000

20

Model prediction

Case 1: grades with largest
difference ΔC15000From HD to4000
normalized
T0 for4000data in model 2
UAV

countShipping standard in Japan(11cm – 14cm)

Market managerconsultation

24

5000

Model 223

Income(The price of one L size broccoli was set to 1 yen.)Normalized TIncome(The price of one L size broccoli was set to 1 yen.)−𝑐⋅𝑇Model initialization
By field measured data

Accurate 3D
geometry traits

Income prediction model

Model definition𝑙𝑛 𝐻𝐷 = 𝑎 − 𝑏 ⋅ 𝑒

High quality 3D
point cloudHigh quality indoor 3D point cloudSize prediction by growth model(still in processing)

It can be more precise with time going(weather data & UAV measure iteration)

dateHD(cm)May 202x