UTokyo Field Phenomics Lab

Virtual Broccoli Farmland
Implementation by Drone-based
Phenotyping and Cross-scale
Data Fusion
Haozhou Wang Tang Li, Nishida Erika, Kato Yoichiro,Fukano Yuya, Wei Guo

2023/11/29✉: haozhou-wang@outlook.com

Vegetables around us

Problems of vegetable farmers
L :12-13 cm(100 円)Too big(0 円)

Too small(0 円)

2L :13-15 cm(90 円)

Multi-time harvest：labor cost
Uneven growth rate

One-time harvest：food loss
Farmer’s income decrease3

Determine the optimal harvest dateUneven growth rate

Field check(longest + shortest length)High labor cost↓
Growth condition in the field↓
Subjective estimate harvest
date4

Smart farming / digital twin virtual farmland
High-throughput
data collection

3D reconstruction

OrganSegmentation

Traitscalculation

Simulation

Data visualization

Individual growth prediction[1]

Price / food loss prediction[1] Augmented Reality Company | VR & MR Solution | AR Development Agency (yeppar.com)

5Challenges：big image data
Need to analysis huge amount of image data
(difficult to process in time)
Cost of data storage for 1ha200+

One image
One flight(2.7GB+)

5742 x 3648 pixels

One map
Large amount of time-series data
(38 flights in 2022)(1GB+)

200 x 20 billion=
4 trillion pixels
per flight

0.3 trillion
pixelsper flight19572 x 17664 pixels
6Challenges：quality for organ-level analysis
Aerial reconstruction products hard to achieveorgan-level analysis quality

3D canopy model(PCD)

2D field map(DOM)
7Challenges：occlusion？
Canopy occlusion affects
traits accuracy

Non-complete structure affects
the virtual farmland visualization
8

Solutions：temporal data fusion
Narrow the processing regions by using prior knowledge of agriculture

Broccoli head position is almost thesame as its seedling position

Narrow the processing areaaround the seedling area

(100 x 100) pixels x 3000 count = 30 billion pixels per flight ~ 1.5 raw image
per crop5742 x 3648 ~ 20 billon pixels
9Solutions：spatial data fusion
Combine raw images (pixel coordinates) with field maps (geo coordinates)(2341,1492)

Pixel coordinatesraw

pixel-coordinates

Better quality
Lacks spatial context

Spatialfusing(35.7393N,139.5414E, 96.34m)

Geo coordinates[1]

geo-coordinates

Lower quality
Has spatial context[1] Wang, H., Duan, Y., Shi, Y., Kato, Y., Ninomiya, S., Guo, W., 2021. EasyIDP: A python package for intermediate data processing in UAV-based plant phenotyping. RemoteSensing 13, 2622. https://doi.org/10.3390/rs13132622

10

Solutions：paired data fusion

Destructivesampling

Close-range 3D reconstruction

Paired aerial images

High-quality
3D models

Traits without
occlusion

Auto-ML
Calibrationmodel[1] https://automl.github.io/auto-sklearn/master

Traits withocclusion

11

Results：temporal data fusion

Seedling position on
flowering stage on fieldmap (geo-coordinate)12

Results：head segmentation & traits calculation
Segmented head results

broccoli
center points

13

Results：occlusion calibration
The Auto-ML calibration improved the traits closer to actual size

14

Results：growth and profit prediction
Individual growth prediction modelby temperature sum (first year)

Predicted
size indistribution
ProfitOne day shift
harvest from the optimal
dateprediction
could lead to(second year)(secondyear)
considerable
income loss (3.7% to 20.4%reduction)

Wang, H., Li, T., Nishida, E., Kato, Y., Fukano, Y., Guo, W., 2023. Drone-Based Harvest Data Prediction Can Reduce On-Farm Food Loss and Improve Farmer Income. Plant
Phenomics 5, 0086. https://doi.org/10.34133/plantphenomics.0086

15

Results：3D virtual farmland visualization

Aerial survey obtained
position and traits

Aerial field low quality
3D models16

Conclusionn Smart farming / virtual farmland has shown its potential to：
Evaluate and predict individual plant growth
Reducing the effects of occlusion & provide 3D visualization
Reducing on-farm food loss & Increase farmer’s income

n Future work:
Collect more valuable data & robust model
Test and apply to commercial farmland

17Thank you✉: haozhou-wang@outlook.com