# Multi-scale 3D Data Acquisition of Maize

Weiliang Wen, Xinyu Guo, Xianju Lu, Yongjian Wang, Zetao Yu

► To cite this version:

Weiliang Wen, Xinyu Guo, Xianju Lu, Yongjian Wang, Zetao Yu. Multi-scale 3D Data Acquisition of Maize. 11th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Aug 2017, Jilin, China. pp.108-115, 10.1007/978-3-030-06137-1_11. hal-02124252

HAL Id: hal-02124252

https://inria.hal.science/hal-02124252v1

Submitted on 9 May 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Multi-scale 3D data acquisition of maize

Weiliang Wen<sup>1,2</sup>, Xinyu Guo<sup>1,2(✉)</sup>, Xianju Lu<sup>1,2</sup>, Yongjian Wang<sup>1,2</sup>, and Zetao Yu<sup>1,2</sup>

<sup>1</sup>Beijing Research Center for Information Technology in Agriculture, Beijing, China

<sup>2</sup>Beijing Key Lab of Digital Plant, Beijing, China.

{wenwl, guoxy, luxj, wangyj, yuzt}@nercita.org.cn

**Abstract:** In recent years, three-dimensional (3D) data acquisition and model reconstruction of plants have been developed as a hot topic of plant scientific researches. However, the morphological structure of plants is very complex and it is hard to describe the details. The data acquisition approaches are diverse for different parts of plants. This study introduces the data acquisition methods of different scales of maize. The grain, leaf and ear, individual plant and maize colony represents the target models for different scales. 3D data acquisition instruments are used to acquire the morphometric of each target. It is found that the organ scale is the simplest to obtain and process. The smallest grain needs high-resolution scanner to acquire the morphological details, while the plant canopy is the hardest one for point cloud process and modeling. The data and reconstructed models are oriented to digital plant, phenotyping analysis, FSPMs research, and popular science education application.

**Keywords:** maize, three-dimensional scanning, three-dimensional point cloud, phenotypic.

# 1. Introduction

The intersection of information science and traditional agricultural science has become a powerful approach to scientific and technological innovation. To quantitatively express, analyze, evaluate, simulate and predict the production and life system of plants, it is necessary to analyze plant morphological structure characteristics and construct three-dimensional (3D) model through measured *in situ* morphological structure data by using 3D scanning technology. 3D point cloud based analysis and reconstruction [1] is one important content of digital plant [2], functional-structural plant modeling [3], and plant phenotyping [4] research. However, plants are living objects and have very complicated appearance characteristics, and their morphology changes with varieties,
regions and production conditions. It is a challenge to obtain high-quality 3D point
clouds of plant shape at different scales and to produce realistic 3D models of plant.
Maize is one of the most important grain crops all over the world, which has great potential for increasing production. Meanwhile, the morphology and structure of maize is comparatively simple, which make it easy to describe and study. Rapid and accurate 3D morphological data acquisition, analysis and reconstruction [5] of maize is of great significance for maize genotype-phenotype analysis [6], canopy photosynthetic productivity calculation [7] and plant type or density optimization [8].

A wide region of researches and applications related to plant 3D analysis and reconstruction can be found in recent years literature. With respect to maize, a 3D template resource database of maize organs, by 3D scanning and digitization, was constructed to provide 3D models with complete agronomic information and high fidelity for the construction of digital maize system [9]. Supawadee et al. [10] constructed an automatic system for maize plant phenotyping parameter extraction and 3D holographic reconstruction by using time-of-flight 3D camera acquired point cloud data, while the system was only capable of processing the early stage maize plants. A non-destructive camera and image processing software based methodology [11] to digitize maize plants was proposed to track the development of individual plants. Researchers also use electromagnetic digitizer to achieve the same purpose [12]. Geometric models of maize canopy were constructed using the acquired 3D digitized data [13]. Miguel et al. [14] constructed a georeferenced 3D reconstruction system for plants to acquire 3D morphological data of maize plants, including an IMU, a total station, and an autonomous vehicle loading several 2D LiDARs towards different orientations. It provided an approach for developing a georeferenced 3D plant reconstruction of an entire crop. These studies focused on different scales 3D data acquisition and analysis of maize respectively.

The aim of this study was to describe 3D data acquisition from different scales systematically, including maize seeds, leaf and ear organs, individual plants, and canopies. After that, 3D phenotyping analysis based on point clouds was conducted for different scales. Finally, existing point cloud processing problems were discussed. It provides technical support for the construction of digital maize research and applications.

# 2. Three dimensional point cloud data acquisition

## 2.1 Seed data acquisition

Maize seed, or grain, is the most important index to describe yield, and has important biological significance and function in agricultural production. Maize seed actually belongs to the organ scale. However, because of its importance and significant size difference with other organs, it is introduced as an independent scale.
It is difficult for conventional scanners to obtain detailed surface features of maize seeds [15]. SmartSCAN³D-5.0M color 3D scanner together with S-030 camera were used to capture the point cloud of seed surface. The precision of the scanner system is ±7µm and the acquisition principle is structured light measurement. The precision promises detailed features of small seeds. The system obtains the surface points by a semi-automatic registration of each target seed part. It costs about 0.5-1 hour of scanning a maize seed. The original point clouds of maize seed are relatively complete with some surface noise. They are capable of describing the morphological 3D details including the protuberance and depression. Fig. 1 shows the scanning system and point cloud data visualization of two seeds. Seed or grain point cloud data could be used for 3D phenotyping analysis, such as volume and surface area calculation and comparison. They are more useful for illustrating the detail differences of different cultivars in a visual way for popular science education.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Scanning system and point cloud data visualization</figcaption></figure>

## 2.2 Organs data acquisition

Organ is the basic structure unit of maize plant. With respect of maize, it mainly includes leaf, stem, ear, tassel, and root. Due to the branching structure and invisibility, it is difficult to scan tassels and roots. Here we introduce the scanning process of maize leaf and ear. Particularly, the point cloud data of maize stem could be obtained using the above SmartSCAN³D-5.0M system in a destructive way, and the morphological trait of maize tassel and root could be captured using electromagnetic 3D digitizer [16].

Leaf is the dominant organ of maize plant morphology and appearance and it also plays the photorespiration and photosynthesis role for maize plants. Therefore, acquiring accurate morphology of maize leaf [17] effectively is very important for 3D maize plant or canopy modeling and further functional-structural simulation research [18]. We use handheld Artec EVA 3D scanner to acquire the point cloud. The precision of the scanner is 0.1mm and it is also structured light based. During the scanning process, each frame needs to continuously track the object to be measured. Because maize leaves are long and narrow, especially the tip and the initial growth stage leaves, a reference object with larger smooth faces has to be placed on the back of the target leaf to promise the continuity of the acquisition process and the integrity of the point cloud. It costs
about 1-2 minutes of scanning a maize leaf. Fig. 2 shows three maize leaves of different cultivars. There are many point noise near the edge, especially the tips. Due to the instability of the leaves, the scanned leaf data exists some regional layered points. And sometimes there are point cloud loss because of the curling shape of the blade. Well reconstructed leaf mesh could be used to accurately estimate the leaf area, and also be the geometric templates for maize plant modeling [9].

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Three cultivar maize leaves point cloud visualization.</figcaption></figure>

One of the most important problem of maize production is to test, calculate and analyze the phenotypic information of maize ear. Hence acquiring and analyzing the ear morphological data using 3D scanner is an available way to realize automatic ear analysis. Artec Spider 3D scanner, with the precision of 30μm, is used for ear point cloud acquisition. We fix the scanner in three positions and rotate the target ear slowly before the scanner. Fig. 3(a) shows a high-throughput maize ear point cloud data acquisition system and three ear model of different shapes reconstructed using the acquired point cloud [19] (Fig. 3(b)). Because of the stability of the ear, the quality of the acquired point clouds are relatively complete and could describe the 3D morphometrics of the ears with only a few noise points. The acquired ear point cloud could be used for phenotyping parameter analysis, such as row and grain number extraction, volume calculation. Meanwhile, it could preserve the original ear shape for all the ears, users could look over the corresponding ears at any time after the experiment.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Ear 3D scanning system (a) and three ear point clouds visualization (b).</figcaption></figure>

## 2.3 Individual plant data acquisition

The individual plant of maize is composed of organs and is the unit of canopy. Field grown plant data acquisition is sensitive to external environment, such as breeze and light, and there is a large number of occluded plants in the surrounding canopy. Researchers usually focus on greenhouse cultivated maize plants [10,20], but the acquisition and analyzing system could only process the early stage plants. To solve these problems, we transplanted the target plant into pot with soil and some of its roots. The plant was then moved to indoor laboratory and watered to promise the smallest deformation occurred of the plant. FARO Focus³D 130 3D laser scanner was used to acquire the point cloud. The precision of the scanner is 2mm within 25m distance. During each scanning process, the scanner was put in three or four stations around the target plant to capture complete point clouds at each angle of view. It takes 5 or 15 minutes of each station depending on the accuracy that the user specifies. To improve the acquisition efficiency, four or six maize plants, placed dependent with each other, could be scanned together. Even indoor, due to the invisible weak airflow, there are small displacement of the plant leaves and produce a lot of noise at the edge and organ connections, while the whole plant morphological data is complete. Points near the tassel after silking are rather concentrated, which made it difficult to segment the individual leaves and tassel points. Fig. 4 shows three groups of point clouds of different growth period maize plant. Accurate point clouds of individual plant are quite useful for 3D reconstruction, realistic rendering and phenotyping analysis.
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Point cloud visualization of three growth period maize plant. (a) Six leaf extension period, (b) huge bellbottom period, (c) silking stage.</figcaption></figure>

**2.4 Maize canopy data acquisition**

Maize canopy, similar to individual plants, also faces with the problem of occlusion and environmental disturbances. In addition, the canopies are not capable of transplanted into laboratory for scanning, because transplanting will lead to the loss of the deformation characteristics of the organs in the canopy. Therefore, it must be measured *in situ*. To acquire the canopy point clouds, we selected a maize canopy including 25 plants in 4 rows at silking stage. The planting density is 4000 plants/mu. To ensure the working distance of the scanner, outlier plants of the canopy were cut off if their distance to the canopy were smaller than 2m (fig. 5(a)). The data acquisition was conducted on a calm day. Besides the noises produced by the airflow, there are still some problems of the canopy data, such as the lack of large numbers of points resulting from the occlusion of organs, and overlapping of organs within the canopy, which are difficult to separate. Because of this, the acquisition and processing methods of point clouds for plant canopy are rare. Only a few of them are used in the analysis of population statistical characteristics [21]. Fig. 5(b) shows a top view of the scanned maize canopy. The point cloud could describe the space distribution characteristics. However, it is hard to segment the individual plants and organs within the canopy due to the occlusion, connection, and deficiency of the organs. It is also difficult to extract the skeleton of each individual plant in the canopy and even harder to reconstruct the canopy using the point cloud.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Maize canopy data acquisition scene (a) and a top view visualization (b).</figcaption></figure>
### 3. Discussion

An alternative way for 3D morphological data acquisition of plants is image-based plant modeling. Researchers obtain multi-view images of plants and use algorithms [22] and software [23] to recover 3D point clouds. Because image acquisition only depends on cameras, thus multi-view stereo (MVS) approach is much cheaper than 3D scanning with general applicability. However, because of the 3D recovery algorithms, MVS has a lower accuracy than 3D scanning and is relatively suitable for organs [24] and individual plants [25]. Comparing with MVS, 3D point cloud acquisition using 3D scanners could get better result when facing the micro-scale and large scale canopy data acquisition problems.

Phenotypic analysis [4] with high-accuracy, high-throughput technologies is a hot issue in plant science. Multi-scale 3D data acquisition offers the opportunity of high-accuracy phenotyping, especially for detailed phenotyping parameters and large scale canopy structures [26]. Accurate 3D reconstruction of plants using high quality 3D point cloud provides more detailed phenotypic parameters such as leaf area, biomass, and canopy volume, etc.

There are many problems about maize plant point cloud processing to be solved in the future, such as point cloud denoising of maize leaf edge and the connection of plant organs, point cloud repairing of each plant canopy, skeleton extraction of leaves and individual plants, 3D phenotyping parameters extraction using point clouds, and multi-scale 3D reconstruction via point cloud. The above problems relies on well-directed computer graphics algorithms and need closely cooperation between computer graphics and agricultural researchers.

### 4. Conclusion

In the present study, we introduced multi-scale 3D data acquisition approaches of maize, including what scanners were suitable for the acquisition, how to build the scanning scene, the accuracy and efficiency of each scale, and the visualization results of the point clouds. It can be known that the seeds need high resolution scanner to acquire the morphological details. It is relatively simple to scan the organs, such as leaves and ears. The individual plant could be transplanted indoor laboratory to be scanned. And maize canopy is the hardest one to acquire its point cloud. High-accuracy 3D data acquisition and reconstruction could help us to study and understand of plants from a high dimension in digital and visual way.

**Acknowledgements.** This work was supported by the National Natural Science
Foundation of China (31601215), the Natural Science Foundation of Beijing Municipality (4162028), the Beijing Academy of Agricultural and Forestry Sciences Youth Research Fund (QNJJ201625), and the Scientific and Technological Innovation Team of Beijing Academy of Agricultural and Forestry Sciences (JNKYT201604).

## References

1. Yin, K.X., Huang, H., Long, P.X., Gaissinski, A., Gong, M.L.Sharf, A.: Full 3D Plant Reconstruction via Intrusive Acquisition. Comput. Graph. Forum 35 (1), 272-284 (2016).

2. Zhao, C., Lu, S., Guo, X., Xiao, B.Wen, W.: Exploration of digital plant and its technology system. Sci. Agr. Sinica 43 (10), 2023-2030 (2010).

3. Henke, M., Kurth, W.Buck-Sorlin, G.H.: FSPM-P:towards a general functional-structural plant model for robust and comprehensive model development. Front. Comput. Sci. (6), 1-15 (2016).

4. Tardieu, F., Cabrera-Bosquet, L., Pridmore, T.Bennett, M.: Plant Phenomics, From Sensors to Knowledge. Curr. Biol. 27 (15), R770-R783 (2017).

5. Gibbs, J.A., Pound, M., French, A.P., Wells, D.M., Murchie, E.Pridmore, T.: Approaches to three-dimensional reconstruction of plant shoot topology and geometry. Funct. Plant Biol. 44 (1), 62-75 (2017).

6. Struik, P.C.: Bridging the genotype-phenotype gap in 3D. J. Exp. Bot. 67 (15), 4427-4430 (2016).

7. Cabrera-Bosquet, L., Fournier, C., Brichet, N., Welcker, C., Suard, B.Tardieu, F.: High-throughput estimation of incident light, light interception and radiation-use efficiency of thousands of plants in a phenotyping platform. New Phytol. 212 (1), 269-281 (2016).

8. Ma, Y.T., Wen, M.P., Guo, Y., Li, B.G., Cournede, P.H.De Reffye, P.: Parameter optimization and field validation of the functional-structural model GREENLAB for maize at different population densities. Ann. Bot. 101 (8), 1185-1194 (2008).

9. Wen, W., Guo, X., Lu, X., Wang, Y., Zhao, D.Liao, W.: Three-dimensional template resource library construction of maize organs. Transactions of the CSAM 47 (8), 266-272 (2016).

10. Chaivivatrakul, S., Tang, L., Dailey, M.N.Nakarmi, A.D.: Automatic morphological trait characterization for corn plants via 3D holographic reconstruction. Comput. Electron. Agric. 109, 109-123 (2014).

11. Renatopratademoraes, F.Witoldf, K.: Three-dimensional digital model of a maize plant. Agr. Forest Meteorol. 150 (3), 478-488 (2010).

12. Guo, Y., Ma, Y.T., Zhan, Z.G., Li, B.G., Dingkuhn, M., Luquet, D.De Reffye, P.: Parameter optimization and field validation of the functional-structural model GREENLAB for maize. Ann. Bot. 97 (2), 217-230 (2006).

13. Xiao, B., Wen, W.Guo, X.: Digital plant calony modeling based on 3d digitization. ICIC express letters. Part B, Applications 2 (6), 1363-1367 (2011).

14. Garrido, M., Paraforos, D.S., Reiser, D., Vazquez Arellano, M., Griepentrog, H.W.Valero,
C.: 3D Maize Plant Reconstruction Based on Georeferenced Overlapping LiDAR Point Clouds. Remote Sensing 7 (12), 17077-17096 (2015).

15. Roussel, J., Geiger, F., Fischbach, A., Jahnke, S.Scharr, H.: 3D Surface Reconstruction of Plant Seeds by Volume Carving: Performance and Accuracies. Front. Plant Sci. 7, 13 (2016).

16. Wu, J.Guo, Y.: An integrated method for quantifying root architecture of field-grown maize. Ann. Bot. 114 (s), 841-851 (2014).

17. Wang, Y., Wen, W.L., Guo, X.Zhao, C.J.: Three-dimensional reconstruction of plant leaf blade based on point cloud data. J. Agr. Sci. Tech. 16 (5), 83-89 (2014).

18. Wang, X., Guo, Y., Wang, X., Ma, Y.Li, B.: Estimating photosynthetically active radiation distribution in maize canopies by a three-dimensional incident radiation model. Funct. Plant Biol. 35 (10), 867-875 (2008).

19. Wen, W., Wang, Y., Xu, T., Yang, T., Guo, X., Zhu, H.Dong, C.: Geometric modeling of maize ear based on three dimensional point cloud. J. Agr. Sci. Tech. 18 (5), 88-93 (2016).

20. Guo, Q., Wu, F., Pang, S., Zhao, X., Chen, L., Liu, J., Xue, B., Xu, G., Li, L., Jing, H., and Chu, C.: Crop 3D: a platform based on LiDAR for 3D high-throughput crop phenotyping. Scientia Sinica 46 (10), 1210-1221 (2016).

21. Fumiki, H., Kazushige, N.Kenji, O.: 3-D Modeling of Tomato Canopies Using a High-Resolution Portable Scanning Lidar for Extracting Structural Information. Sensors 11 (2), 2166-2174 (2011).

22. Pound, M.P., French, A.P., Murchie, E.H.Pridmore, T.P.: Automated Recovery of Three-Dimensional Models of Plant Shoots from Multiple Color Images. Plant Physiol. 166 (4), 1688-U1801 (2014).

23. Wu, C., "VisualSFM: a visual structure from motion system", (2011).

24. He, J.Q., Harrison, R.J.Li, B.: A novel 3D imaging system for strawberry phenotyping. Plant Methods 13, 8 (2017).

25. Pound, M.P., French, A.P., Fozard, J.A., Murchie, E.H.Pridmore, T.P.: A patch-based approach to 3D plant shoot phenotyping. Mach. Vision Appl. 27 (5), 767-779 (2016).

26. Liu, S.Y., Baret, F., Abichou, M., Boudon, F., Thomas, S., Zhao, K.G., Fournier, C., Andrieu, B., Irfan, K., Hemmerle, M., and de Solan, B.: Estimating wheat green area index from ground-based LiDAR measurement using a 3D canopy structure model. Agr. Forest Meteorol. 247, 12-20 (2017).