# A 3D Canopy Reconstruction and Phenotype Analysis Method for Wheat

Boxiang Xiao, Sheng Wu, Xinyu Guo, Weiliang Wen

► To cite this version:

Boxiang Xiao, Sheng Wu, Xinyu Guo, Weiliang Wen. A 3D Canopy Reconstruction and Phenotype Analysis Method for Wheat. 11th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Aug 2017, Jilin, China. pp.244-252, 10.1007/978-3-030-06137-1_23. hal-02124256

HAL Id: hal-02124256

https://inria.hal.science/hal-02124256

Submitted on 9 May 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A 3D Canopy Reconstruction and Phenotype Analysis Method for Wheat

Boxiang Xiao<sup>1,2,3</sup>, Sheng Wu<sup>1,2,3✉</sup>, Xinyu Guo<sup>1,2,3</sup>, and Weiliang Wen<sup>1,2,3</sup>

<sup>1</sup>Beijing Research Center for Information Technology in Agriculture, Beijing Academy of Agriculture and Forestry Science, Beijing 100097, China;

{xiaobx,wus,guoxy,wenwl}@nercita.org.cn

<sup>2</sup>National Engineering Research Center for Information Technology in Agriculture, Beijing 100097, China;

<sup>3</sup>Beijing Key Lab of Digital Plant, Beijing 100097, China;

**Abstract.** A high precise and high realistic three-dimensional wheat canopy model is important in modern agriculture. In this paper, we proposed a 3D reconstruction and quantitative calculation for phenotype analysis method for wheat. First, we made use of a 3D digitizer to acquire spatial structure and distribution data of wheat canopy. After data processing, we constructed three-dimensional organ models including stalks, leaves and others, based on a surface modeling algorithm. Under this process, we constructed a 3D canopy model by frames of wheat colony. Furthermore, we made phenotype analyses on structure and organs distribution features including leaf length, azimuth and obliquity values. By use of constructed 3D canopy model, we used a light distribution computing algorithm to analyze the potential light interception, and we also calculated interception in different layers and different organs. The synchronous light intensity and leaf area index (LAI) measured by a PAR device were used to compare and examine the constructed canopy models. We also compute macroscopic canopy attributes including leaf area, leaf area index, projection area, shading area, and so on. Finally, parts of experimental results are shown, and the results show that our method is feasible and effective for wheat as well as other similar crops. At the end, the main contributions and limitations are also discussed, and some future works are addressed.

**Keywords:** Wheat, Canopy Reconstruction, 3D, Phenotype Analysis, Light Distribution.

## 1 Introduction

In past decades, high precise and high realistic three-dimensional crop canopy models were becoming more important in many applications, such as scientific researches in agriculture, cultivars estimation and phenotype analysis, virtual technological training, etc. [1-8]. One of the most important applications of 3D quantitative virtual models is calculation of light distribution within a crop canopy [6,7]. Wheat is a main crop both in China and whole world, and many researchers have made fruitful works about cultivars evaluation, phenotype analysis and planting technologies[8-13].

The main problem in 3D virtual canopy model applications is reconstruction, in both precision and efficiency. There are several effective reconstruction ways: measure-based, 3D digitizer, 3D scanning and vision-based technologies. Each method has its advantages and limitations. For example, traditional measure-based
way is generally low efficient and precise, but it could be used in more widely scene
without complex influences. Nowadays, with the fast development of 3D laser
scanning technology and popularizing of scanning devices, people have made use of
scanning to reconstruct crops' canopies models. Hosoi and Omasa [13] estimated
vertical plant area density and analyzed growth parameters of a wheat canopy by
using 3D portable lidar imaging. LiDAR is becoming a new important technology and
tool in next-generation plant phenotyping applications[14]. Multi-sensor systems have
been also developed for high throughput field phenotyping, including video camera,
NDVI, distance sensor, infrared radiometers, and so on[15]. Benefit from that, image-
based 3D canopy reconstruction technology has been used in many applications, such
as potential productivity[16].

To sum up, the main problems of 3D canopy modeling and analysis in applications
is twofold. One is how to reconstruct precise 3D canopy models. Another is how to
make use of virtual models to analyze the features and differences between cultivars.
The first problem is more important than the second one, and it is the basic condition
for any applications. Especially for wheat, because of its complex structures of
canopy, precise reconstruction and modeling with all 3D models of organ details are
still challenges. Existing works have only focus on large scale simulation and
approximative simplified morphologic modeling for organ details. To tackle the
problems, we present a 3D canopy reconstruction and phenotype analysis method for
wheat in this paper. Fig.1 shows the flowchart of this paper. First, we acquire multi-
type 3D structure, shape and distribution data and 2D images, as well as synchronous
light intensity and leaf area index (LAI). And then, we reconstruct canopy models of
wheat based on processed data. By use of canopy models, we conduct phenotype
analysis to calculate light distribution and parameters.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The flowchart of this method</figcaption></figure>

## 2 Materials and Method

We have planted two varieties of wheat with two densities: Jingdong18 Jing8043,
in our experimental farm in Beijing Academy Agriculture and Forest Science located
at (39°57'52"N, 116°17'84"E). The line distance is 300mm, and the sowing date is Oct
30, 2016. In our experimenting field, we have installed a group of cameras for wheat
growth status monitoring. The images are automatically sent to service machine and
image collecting frequency is 1 per hour.
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Wheat planting in experiment field</figcaption></figure>

**2.1 Data Acquisition**

Here, our 3D reconstruction method depends on real 3D data of shape, structure and distribution of wheat organs in the canopy. So we make use a 3D digitizer named FastRak produced by Polhemus Company to obtain feature points of wheat plants in colony. For leaf shape, we use an image-based way to measure leaf width. On the other hand, we also acquire real light intensity and LAI values by a PAR device to examine reconstruction results.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Wheat planting in experiment field</figcaption></figure>

For 3D digitization, we have picked up all feature points on wheat plant, including all node joints of stalks, and all leaf nervation curve feature points. Generally, we took 5 to 8 feature points on a leaf mid curve, in turn from base to tip. The FastRak 3D digitizer is shown in Fig. 3. Fig. 4 (a) shows a scene of data collecting and an example of captured frame data are shown in Fig.4 (b). For each blocks, we selected 3-5plants including 10-30 tillers. By 3D digitizer, we collected the real shape, structure and organ distribution information.
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** 3D data collecting and example data</figcaption></figure>

To obtain precise leaf width parameters, we captured images of all leaves fixed on a
plotting paper, and then we measured values of leaf width and fitted a leaf width value
sequence for each leaf. We have investigated the leaf width values of different plants
in a same canopy, and the results have shown that the differences of values with same
leaf order between different plants were very small. So we assumed the distribution of
leaf width value was uniform and chose a fixed leaf width function to generate
parameters according to leaf length values.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Images of wheat plant and group of leaves.</figcaption></figure>

**2.2 Three-dimensional Canopy Reconstruction**

Based on the multi-type acquired data, we processed and got key parameters of 3D
models of wheat organs including stalks, leaves and ears. We presented a multi-scale
and multi-layer 3D modeling method to reconstruct virtual wheat models. The organ
modeling is shown in Fig. 6. For each organ's feature points, we constructed a cubic
spline interpolation curve (Eq.1) to generate the smooth nervation of leaf. And then,
we construct a local coordinate on each point of nervation, as shown in Fig. 6, and the
local coordinate determines the deformation of corresponding point set both on two
edges.
$$ 
\begin{aligned}
S_k(x) = H_3^{(k)}(x) &= y_k \alpha_0^{(k)}(x) + y_{k+1} \alpha_1^{(k)}(x) + m_k \beta_0^{(k)}(x) + m_{k+1} \beta_1^{(k)}(x) \\
&= y_k \left(1 + 2 \frac{x - x_k}{x_{k+1} - x_k} \left(\frac{x - x_{k+1}}{x_k - x_{k+1}}\right)^2 + y_{k+1} \left(1 + 2 \frac{x - x_{k+1}}{x_k - x_{k+1}} \left(\frac{x - x_k}{x_{k+1} - x_k}\right)^2\right) \right. \\
&\qquad \left. + m_k (x - x_k) \left(\frac{x - x_{k+1}}{x_k - x_{k+1}}\right)^2 + m_{k+1} (x - x_{k+1}) \left(\frac{x - x_k}{x_{k+1} - x_k}\right)^2 \right)
\end{aligned} 
\quad (1) $$

We translate each row of control points to according to the middle point of them, and the translation can be expressed by Eq. (2) and (3) where N is the new model.

$$ 
\left\{ 
\begin{array}{l}
\text{Trans}_i = Vc[i] - Ve[i] \\
N_i = \text{Trans}_i \cdot E_i'
\end{array}
\right. 
\quad i = 1, 2, \dots, n 
\quad (2) $$

$$ N = \begin{bmatrix} N_x \\ N_y \\ N_z \\ 1 \end{bmatrix} = \text{TransgE}' = \begin{bmatrix} 1 & 0 & 0 & \text{Trans}_x \\ 0 & 1 & 0 & \text{Trans}_y \\ 0 & 0 & 1 & \text{Trans}_z \\ 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} E_x' \\ E_y' \\ E_z' \\ 1 \end{bmatrix} \quad (3) $$

Finally, we recalculate the surface model according to Eq. (1) and a new leaf model is generated, as shown in Fig.6. The same process is conducted on stalk and ear feature points, we use a simplified whole cylinder ear model to get a higher efficiency. Some modeling results are illustrated in Fig.7, here (a) and (b) are frame and 3D models of a tiller of wheat respectively, (c) and (d) are small canopy's frame and 3D models including 5 plants of wheat.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Wheat leaf generation algorithm, (a) is the local coordinate on mid-point, (b) deformation.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>**Fig. 7.** 3D reconstruction of a tiller of wheat and a small colony.</figcaption></figure>

# 3 Experimental Results

We have conducted various experiments by different scene of wheat canopies. To implement our proposed method, we have developed a prototype system for wheat's three-dimensional reconstruction and phenotype analysis and the system is named as "PlantCAD Wheat". The software system consists of functional modules: 3D data view and processing, wheat organs shape reconstruction by free-form curves and surface, 3D canopy and scene generation and management, light distribution computing, canopy parameters phenotype analysis, and other additional modules such as 3D scene rendering, view operating, data I/O, and so on. To examine the proposed method, we have performed 4 canopies models on a DELL mobile workstation with i7 2.8GHz CPU, 16GB memory and NVIDIA Quadro K2100M graphics card.

## 3.1 Results of Three-dimensional Reconstruction

By use of modeling and reconstruction algorithm introduced in Section 2.2, we analyze local features of acquired data and generate an assigned wheat colony by group copy both in row and line directions. Fig.8 shows an example of 3D reconstruction of wheat canopy with 5 rows including about 180 plants. We also add land and light sensor models into canopy to support additionally light distribution calculation. The visualization is implemented by OpenGL Library.

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** 3D reconstruction of wheat canopy with 5 rows including about 180 plants.</figcaption></figure>
## 3.2 Results of Light Distribution Computing

Quantitative light distribution within a wheat canopy is very important for simulation of canopy photosynthesis, and precise 3D virtual models could support effective and accurate calculation of light distribution. We take a light distribution algorithm introduced in literatures [17] to calculate both direct sun light and scattering of sky. We also combine a visualization module based on OpenGL Library to render the light distribution results, as shown in Fig. 8(b).

## 4 Conclusions

In this paper, we have proposed a 3D canopy reconstruction and phenotype analysis method for wheat. By means of a set of multiple type real data measured from wheat canopy in field, we present a 3D reconstruction algorithm for all wheat organs including stalks, leaves and ear. And then, we construct canopy models by acquired frame feature points. Based on reconstructed 3D wheat canopy, we conduct light distribution calculation and phenotype characters calculation. The main contribution is that our data-based 3D reconstruction method provides a feasible and effective way for wheat canopy modeling and phenotype analysis. On the other hand, the main limitation of our method is data acquisition capacity and efficiency, because 3D feature points are manually collected for aim of high precision and completeness. So we have to make use a local distribution to express the expanded canopy. Involving 3D laser scanning and LiDAR technologies as well as high throughput phenotype analyzing approaches, automatically 3D canopy model reconstruction should be a potential trend for wheat and other crops in future.

## Acknowledgment

This work is supported by National Natural Science Foundation of China (Grant No. 61300079); by Beijing Municipal Science and Technology Project (Grant No. D151100004215004); by Creative Team Project of Beijing Academy of Agriculture and Forestry Science (Grant No. JNKYT201604).

## References

1. P. Prusinkiewicz. Modeling of spatial structure and development of plants: a review. Scientia Horticulturae, 74:113-149, 1998.

2. O. Deussen. Dynamic a framework for geometry generation and rendering of plants with applications in landscape architecture. Landscape and Urban Planning, 64:105-113, 2003.

3. Y. Guo, B. Li. Progress in Virtual plant Research. Chinese Science Bulletin, 2001, 46(4):273-280. (In Chinese).

4. Zhao C J, Lu S L, Guo X Y, et al. Exploration of Digital Plant and Its Technology System. Scientia Agricultura Sinica, 2010, 43(10):2023-2030. (In Chinese)

5. B. Xiao, X. Guo, X.Du, et al. An interactive digital design system for corn modeling. Mathematical and Computer Modelling, 51(11-12), pp1383-1389, 2010.

6. N.M. Munier-Jolain, S.H.M. Guyot, N. Colbach. A 3D model for light interception in heterogeneous crop:weed canopies: Model structure and evaluation. Ecological Modelling. 250 (2013) 101-110.

7. W. Zhang, L. Tang, X. Yang, et al. A simulation model for predicting canopy structure and light distribution in wheat. European Journal of Agronomy, 2015, 67:1-11.

8. Fang Gou, Martin K. van Ittersum, et al. Intercropping wheat and maize increases total radiation interception and wheat RUE but lowers maize RUE. European Journal of Agronomy. 84 (2017) 125-139.

9. Rim Baccar, Christian Fournier, Tino Dornbusch, et al. Modelling the effect of wheat canopy architecture as affected by sowing density on Septoria tritici epidemics using a coupled epidemic-virtual plant model. Annals of Botany, 2011, 108: 1179-1194.

10. Evers JB, Vos J, Chelle M. Simulating the effects of localized red:far-red ratio on tillering in spring wheat (Triticum aestivum) using a three-dimensional virtual plant model.
New Phytologist, (2007)176: 325-336.

11. C. Fournier, B. Andrieu, S. Ljutovac, et al. ADEL-wheat: a 3D architectural model of wheat development. Proc. of International Symposium on Plant Growth Modeling, Simulation, Visualization, and their Applications. Beijing, China: Tsinghua University Press/Springer, 2003.54-63.

12. J.B. Evers, J. Vos, C. Fournier, et al. 2007. An architectural model of spring wheat: evaluation of the effects of population density and shading on model parameterization and performance. Ecological Modelling 200: 308-320.

13. F. Hosoi, K. Omasa. Estimating vertical plant area density profile and growth parameters of a wheat canopy at different growth stages using three-dimensional portable lidar imaging. ISPRS Journal of Photogrammetry and Remote Sensing, 64 (2009) 151-158

14. M. Bietresato, G. Carabin, R. Vidoni, et al. Evaluation of a LiDAR-based 3D-stereoscopic vision system for crop-monitoring applications. Computers and Electronics in Agriculture, 124 (2016) 1-13.

15. G. Bai, Y. Ge, W. Hussain, et al. A multi-sensor system for high throughput field phenotyping in soybean and wheat breeding. Computers and Electronics in Agriculture, 128 (2016) 181-192.

16. A. J. Burgess, R. Retkute, M. P. Pound, et al. Image-based 3D canopy reconstruction to determine potential productivity in complex multi-species crop systems. Annals of Botany 119: 517-532, 2017.

17. W. Wen, J. Meng, X. Guo, et al. Calculation system of light distribution within crop canopy based on radiosity methods. Transactions of The Chinese Society of Agricultural Machinery, 2009, 40(S1):190-193.