# Research on Plant Growth Simulation Method Based on ARToolkit

Peng-Fei Zhao, Tian-En Chen, Wei Wang, Fang-Yi Chen

► To cite this version:

Peng-Fei Zhao, Tian-En Chen, Wei Wang, Fang-Yi Chen. Research on Plant Growth Simulation Method Based on ARToolkit. 10th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Oct 2016, Dongying, China. pp.189-196, 10.1007/978-3-030-06155-5_18 . hal-02179996

HAL Id: hal-02179996

https://inria.hal.science/hal-02179996

Submitted on 12 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Research on Plant Growth Simulation

method Based on ARToolkit

Peng-fei Zhao<sup>1,2,3,4,a</sup>, Tian-en Chen<sup>1,2,3,4,b,c</sup>, Wei Wang<sup>1,2,3,4,c</sup>, Fang-yi Chen<sup>1,2,3,4,d</sup>

<sup>1</sup>Beijing Research Center for Information Technology in Agriculture, Beijing 100097, China;

<sup>2</sup>National Engineering Research Center for Information Technology in Agriculture, Beijing 100097, China;<sup>3</sup>Key Laboratory of Agri-Infomatics, Ministry of Agriculture, Beijing 100097, China;

<sup>4</sup>Beijing Engineering Research Center of Argicultural Internet of Things, Beijing 100097, China;

<sup>a</sup>zhaopf@nercita.org.cn, <sup>b</sup>chente@nercita.org.cn,

<sup>c</sup>wangw@nercita.org.cn, <sup>d</sup>chenfy@nercita.org.cn

**Abstract:** Augmented Reality is a new technology which can combine the real world information with the virtual world information seamless. In order to increase the sense of immersion and interactivity, thereby improving agricultural information service capabilities and expanding the exchange of seed industry exhibition and promotion capabilities, this paper provides a kind of interactive method to simulate the plant growth. Based on the target tracking localization algorithm of ARToolkit, through the relative distance positioning two rectangular markers, corresponding to different plant growth development period, the method can interact to realize the control of virtual objects, such as translation, rotation and zoom.

The experimental results show that the method can simulate the true process of the wheat plant growth, that can simulate different kinds of virtual wheats. The method has a good compatibility, low requirement on the hardware, stable capability, better expansibility and transferability.

**Keywords:** ARToolkit; augmented reality; simulate; plants

# 1 Introduction

Augmented reality is a new technology that combines the virtual objects with the real world, so the technology can construct the combination of the interactive 3D scene. With the help of computer and visualization technology, augmented reality can add the application of the virtual information to the real world, and superimpose the real environment and virtual objects on the same picture or the same space in real time [1]. In order to present the user with a true sensory effects of the new environment, so as to realize the enhancement of the real world. This paper uses the ARToolkit tool to develop augmented reality application, and simulates the real situation of wheat plant growth in the unity3d engine platform.

# 2 Materials and method
## 2.1 ARToolkit technology architecture

Through the computer vision technology, ARToolkit compute the relative position of the rectangular marker that captured by the camera, in order to track and position the rectangular marker. The ARToolkit analysis the region of the connected and computes the image matching values. When the match is successful, the ARToolkit can compute the position of the rectangular marker, and overlay of the virtual three-dimensional objects[2]. The basic process of ARToolkit system as shown in Fig.1.

<figure><img src="image_2.png" /><figcaption>Fig.1. Basic process of ARToolkit system</figcaption></figure>

## 2.2 ARToolkit coordinate system

The main problem of the augmented reality system is the matrix transformation between the world marker coordinates system and the camera coordinates system. Based on visual registration technology, the ARToolkit can track and positioning for the marker, so as to complete corresponding matrix of coordinate system transformation[3].

According to the basic theory of computer vision, set the real world marker coordinate system is $(X_M, Y_M, Z_M)^T$, the camera coordinate system is $(X_C, Y_C, Z_C)^T$, The camera change matrix is $T_{CM}$, The camera rotation and translation transformation is R and T. And R is 3 * 3 orthogonal matrix[4], T is The three dimensional bit vector, the ARToolkit coordinate system structure as shown in Fig.2, The relationship between the camera coordinate system and the real world marker as shown in formula (1).
<figure><img src="image_3.png" /><figcaption>Fig.2. ARToolkit coordinate system</figcaption></figure>

Set screen coordinate system is $(X_1, Y_1)^T$, S is the camera internal parameters, the internal parameters has been given in the ARToolkit system, the internal parameters is initialized after application startup, and the internal parameters can be modified by the developer in the library function. H says at some point[5], f is the focal length, s slope coefficient, usually take 0, a is the longitudinal width ratio of the pixels, the camera coordinate system and the screen to see formula(2) shows,and the results as shown in formula (3):

$$ \begin{bmatrix} X_c \\ Y_c \\ Z_c \\ 1 \end{bmatrix} = \begin{bmatrix} R & T \\ 0 & 1 \end{bmatrix} \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} = \begin{bmatrix} R_{11} & R_{12} & R_{13} & T_1 \\ R_{21} & R_{22} & R_{23} & T_2 \\ R_{31} & R_{32} & R_{33} & T_3 \\ 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} = T_{CM} \begin{bmatrix} X_M \\ Y_M \\ 1_M \\ 1 \end{bmatrix} \quad (1) $$

$$ \begin{bmatrix} hX_1 \\ hY_1 \\ h \end{bmatrix} = \begin{bmatrix} f & s & x_c & 0 \\ 0 & af & y_c & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} X_c \\ Y_c \\ Z_c \\ 1 \end{bmatrix} = S \begin{bmatrix} X_c \\ Y_c \\ Z_c \\ 1 \end{bmatrix} \quad (2) $$

$$ \begin{bmatrix} hX_1 \\ hY_1 \\ h \end{bmatrix} = ST_{CM} \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} = \begin{bmatrix} sf_x & 0 & x_c & 0 \\ 0 & sf_y & y_c & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} R_{11} & R_{12} & R_{13} & T_1 \\ R_{21} & R_{22} & R_{23} & T_2 \\ R_{31} & R_{32} & R_{33} & T_3 \\ 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} \\ = \begin{bmatrix} c_{11} & c_{12} & c_{13} & c_{14} \\ c_{21} & c_{22} & c_{23} & c_{24} \\ c_{31} & c_{32} & c_{33} & c_{34} \end{bmatrix} \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} = C \begin{bmatrix} X_M \\ Y_M \\ Z_M \\ 1 \end{bmatrix} \quad (3) $$

## 2.3 Maya 3-D modeling

The maya function is flexible operation, high production efficiency, highly realistic rendering. Bone is a joint structure, it can be animated and positioning has been skinned deformable objects, the skeleton of wheat as shown in Fig.3(a)and the local wheat skeleton ad shown in Fig.3(b). Once established the skeletal, smooth skin and body skin can be used for character building skin. The user can make objects become joint and skeletal sub objects, and use the skeleton to control the movement of objects,the process as shown in Fig.4[6].
<figure><img src="image_4.png" /><figcaption>Fig.3. Skeleton of wheat Figure</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig.4. Bone control growth process</figcaption></figure>

## 2.4 Target marker production

Before using ARToolkit developing augmented reality system, it is necessary to make and plant model corresponding marker image, while the system is based on these marker images to match the corresponding plants, so as to complete the virtual scene of the stack. ARToolkit provide standard template is one with a black border around the square, according to the need to the white area to add custom graphics, to make a different marker image[7]. Put the marker image on camera capture range, by training, generate patt file types that can be identified and analyzed by the system. As shown in Fig.5 for the production of the trigger graph. As shown in Fig.6 for the production of wheat plant target marking graph.
<figure><img src="image_6.png" /><figcaption>Fig.5. The trigger graph</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig.6. The wheat grahp</figcaption></figure>

## 2.5 Interactive simulation

In order to improve the system of interactive and immersive, in this paper, by controlling the relative distance of target recognition, the system simulate the growth of wheat plants model . Camera by capturing the relative distance between the two identify images, the distance of the equipment corresponds to the wheat growth and development stage. Through the matrix transformation ,the relative distance in the real world is converted into three-dimensional relative distance in three-dimensional coordinate system ,which can reflect the topology of the wheat, and control the process of virtual wheat growth. In the system , we set the relative distance as the initial value of 0, wheats will remain zero growth. With relative distance increases, the wheats will be the growth of the real time, gradually from the initial state of plants develop into mature model of wheat plants.

## 2.6 The interactive user interface

In the interactive interface of the system,when the camera to capture images of two markers, wheat plants will automatically play audio commentary, and endowed with text description. In the interface, the user can control the volume of voice introduction, and real-time switching system interface fullscreen / windowed mode. Through the interface, the system can improve the user immersive, and let users interact with the system better.

# 3. Results and discussion
In the Unity3d platform environment, this paper simulates the growing status of wheat. As shown in Fig.7, dist =154, the growing status of wheat seeds. As shown in Fig.8, dist =191, the growing status of wheat seedling . As shown in Fig.9, dist = 236, the growing status of wheat maturity.

According to the Fig.9, The frame rate of the system is 51 frames / sec. This method can simulate the true process of the wheat plant growth. Moreover, the system runs smoothly and the FPS can reache 51 frames /sec, so that the system consumes less IT resources, and the operation is simple.

<figure><img src="image_8.png" /><figcaption>Fig.7. The status of seeds</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig.8. The status of seedling</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Fig.9. The status of maturity</figcaption></figure>

## 4. Conclusions

Based ARToolkit augmented reality, this paper implements the wheat plant growth process and important period in character. The experimental results show that the method can be a very good simulation of the wheat plant growth process, through simple and friendly interactive multi-touch, users can easily control the plant growth process. System has good operability, users use the intuitive and lively, the result is accurate, stable and suitable for new wheat varieties in the exhibition hall, the popular science exhibition display and exhibition, etc.

## Acknowledgements

Funds for this research was provided by the Beijing agri-science-city”Internet + modern agriculture” Industries Integration Promotion Strategy Research Report: Beijing Municipal Science and Technology Commission of Science and Technology Innovation Center construction strategy research and expert advisory subject (Z161100003116001). The research work also has been done under the help of the team of the Information engineering.

## References

1. Zhou Jian-guang, Shi Gang, Ma Xiao-hu.: Virtual Interactive Method in Augmented Reality System. Computer Engineering. 38(1):251-256(2012)

2. Hu Yingqun, Yin Changqing.: Design and Implementation of 3D Virtual Commodity demo system based on ARToolkit. Computer Applications and Software. 29(8):259-263(2012)

3. Sheng Jun, Wang Yi-gang.: Research on finger interactive system based on ARToolkit. Journal of Mechanical and Electrical Engineering. 27(6):116-119(2010)
4. Cheng Shi, Gu Qi, Pan Jian-sheng, Chen Shan-li: The Research on Augmented Reality of Virtual Dolls Imaging Based on ARToolkit-MMD. Microelectronics and Computer. 28(10):150-154(2011)

5. Jiang Zheng-zheng, Gao Zhan, Chen Xiang, Wang Dan-dan, Sun Wnag-jie, Sun Qi-ke: Implementation of Virtual Watch Try-on System Based on ARToolkit.Informatization Research. 39(5):35-37(2013)

6. Li Zhan-Li, Yan Jia-Xuan-Zi: Maya-based Simulation of Tree Movement with Electronic Image Processing. Advances in Mechanical and Electronic Engineering. 3(10):541-547(2013)

7. Dong Zi-long, Zhang Guo-feng, Shao Yuan-long, Hua Wei: Chinese-character-marker Based Augmented Reality System. Journal of Image and Graphics. 14(7):372-375(2009)