<figure><img src="image_1.png" /></figure>

HAL
open science

# A Data Acquisition System Based on Outlier Detection Method for Weighing Lysimeters

Wenqian Huang, Chi Zhang, Xuzhang Xue, Liping Chen

<figure><img src="image_2.png" /></figure>

## To cite this version:

Wenqian Huang, Chi Zhang, Xuzhang Xue, Liping Chen. A Data Acquisition System Based on Outlier Detection Method for Weighing Lysimeters. 5th Computer and Computing Technologies in Agriculture (CCTA), Oct 2011, Beijing, China. pp.471-478, 10.1007/978-3-642-27281-3_53. hal-01351849

HAL Id: hal-01351849

https://inria.hal.science/hal-01351849

Submitted on 4 Aug 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A data acquisition system based on outlier detection method for weighing lysimeters¹

Wenqian Huang¹,², Chi Zhang¹,²*, Xuzhang Xue¹,², Liping Chen¹,²

1 Beijing Research Center of Intelligent Equipment for Agriculture, Beijing Academy of Agriculture and Forestry Sciences, Beijing, 100097, China
2 National Research Center of Intelligent Equipment for Agriculture, Beijing, 100097, China

*Corresponding author: zhangchi@nercita.org.cn

**Abstract.** The weighing lysimeters provide scientist the basic information for research related to the evapotranspiration, high quality of the collected data from lysimeters is of great significance. However there are many factors that can affect the measurement accuracy of the weighing lysimeter. In this paper, a data acquisition system was developed to collect the data from 24 weighing lysimeters. The calibration process of the load cell was described. An outlier detection method based on the 3-sigma rule and the median filter was proposed to improve the measurement accuracy of the weighing lysimeters. The performance of the proposed method was compared with the method based on Savitzky-Golay filter. Results show that the standard deviations of the 15-point median filter and the 15-point Savitzky-Golay filter applied to the 283 data points were 0.413Kg and 0.422Kg respectively, which means that the performance of the median filter was better than the Savitzky-Golay filter. Moreover the outliers were successfully eliminated using the median filter and were not removed by the Savitzky-Golay filter.

**Keywords:** Weighing lysimeter, outlier detection, median filter, Savitzky-Golay filter

## 1 Introduction

Evapotranspiration (ET) is used to describe the sum of evaporation and plant transpiration from the Earth's land surface to atmosphere [1]. Evapotranspiration is an important part of the water cycle. Precision measurement of the evapotranspiration is of great importance for accurate irrigation scheduling due to economic factors and the scarcity of water resources in many parts of the world [2]. It is also very important for the protection of ground water resources by studying soil water movement and nutrient transportation through the lysimeters.

There are various methods to measure the evapotranspiration, but weighing lysimeters are most accurate and widely applied. The weighing lysimeter was used to

¹ National Natural Science Foundation of China (Project No. 31071324) and National Key Technologies R&D Program of China (Project No. 2008BAB38B06)
measure the evapotranspiration directly by measuring the change in mass of an isolated soil tank. When the drainage and the water input such as irrigation and precipitation are taken into account, the evapotranspiration of the crop and soil can be accurately determined. Many works have been done in the construction and installation of weighing lysimeters and the measuring principle of the weighing lysimeter was described [3-6]. Some advanced data acquisition systems equipped with high-precision load-cell for the weighing lysimeters have been developed [7-9]. Much attention has been paid to the research on the application of the weighing lysimeter. Su Meishuang et al. studied the crop water demands for sprinkler-irrigated winter wheat and sweet corn using a weighing lysimeter and calculated the crop coefficients [10]. Niu Yong et al. employed a large-scale weighing lysimeter to study cucumber transpiration processes in solar greenhouse and established three empirical models for estimation of cucumber transpiration rate [11]. Changming Liu et al. studied the daily evapotranspiration of irrigated winter wheat and maize using a large-scale weighing lysimeter to improve field water utilization efficiency [12].

Because weighing lysimeters provide scientist the basic information for research related to the evapotranspiration, high quality of the collected data from lysimeters is of great significance. However there are many factors that can affect the measurement accuracy of the weighing lysimeter. The error analysis of the collected data from lysimeters indicate that evapotranspiration uncertainty was sensitive to number of rainfall events, the daily percolation quantity, load-cell uncertainty and potentiometer uncertainty [13]. The accuracy of lysimeter was directly proportional to the surface area and the accuracy of the scale and inversely proportional to the lysimeter mass, was also limited by the resolution of the datalogger or data recording system [14]. The systematic measurement error caused by the load-cell and the data acquisition system can be minimized through regular calibration [15-17].

But the error caused by the mechanical vibration and the change of environmental factor such as the wind speed and soil temperature was stochastic, cannot be eliminated through the calibration process. R. W. Malone et al. found that electronic changes can affect the performance of the weighing lysimeters and provide a statistically valid quality control plan using control charts to improve the measurement accuracy of the weighing lysimeters [18]. P.J. Vaughan et al. pointed out that lysimeter and improved the lysimeter data processing to eliminate bad data and minimize variations in the measurements through noise reduction method based on a seven-point Savitsky-Golay filter [19-20]. However, the Savitzky-Golay filter is useful for to eliminate noise with high frequency, but is less successful to detect the high magnitude and low-frequency noise such as sharp pulse noise. In order to improve the measurement accuracy of the weighing lysimeters, it is important to detect the outliers in the measurement data and eliminate them.

In this paper, a data acquisition system based on outlier detection method is developed. The outlier detection method is based on the combination of two digital filters, one is the filter of three sigma rule and another is the median filter. The hardware and software design of the data acquisition system is described, as well as the calibration process of the load-cell sensor. The data processing result is compared with the one based on Savitzky-Golay filter.
## 2 Materials and methods

### 2.1 Description of the system

The weighing lysimeters system is located in National Precision Agriculture Demonstration Station in Xiaotangshang Town of Beijing. The system is consisted of 24 lysimeters with 1.0m*0.75m*2.3m (L*W*H). The machine structure of the lysimeter was counter-balanced and the schematic diagram of the lysimeter is shown in Fig.1.

<figure><img src="image_4.png" /><figcaption>Fig. 1 The schematic diagram of the weighing lysimeter</figcaption></figure>

The load cells used in the system are NS1-3M2-100Kg with a sensitivity of 1.9951mV/V for the lysimeters and the NS6-2-50Kg of 1.9969mV/V for percolation. A four-wire bridge configuration is used for load cells in which 2-wire for the input of the excitation voltage and 2-wire for the output of differential voltage. As the change of the mass of the lysimeter can be measured using the load cell. The measurement data of the load cell can be converted to the evapotranspiration (ET), usually calculated by mm, through dividing the change of mass by the surface area of the lysimeter 0.75m².

A station for observing climatic parameters was establish to collected the data of air temperature, precipitation, solar radiation, humidity and wind velocity/direction. Then the ET could be calculated using the following equation:

$$ET = P + I + Q - \Delta R - \Delta S \quad (1)$$

Where P is the precipitation, mm; I is the irrigation, mm; Q is underground drainage, mm; ΔR is surface runoff, mm; and ΔS is the change of the amount of water stored in a certain period, mm.
## 2.2 Design of the data acquisition system

In the developed data acquisition system, a master/slave structure is adopted. The measurement system consists of communication modules, data acquisition modules, personal computer, RS485-RS232 converter and software. The schematic of the data acquisition system is shown in Fig.2. The data acquisition module is developed based on a 16bit Analog/Digital Convert (ADC) and a 16bit Digital/Analog Converter (DAC). The ADC is used for differential-input of the load cells, and the input range is configured as -15mv to +15mv. The DAC is used to provide +5V excitation voltage with 40mA maximum driven current for the load cells. Each data acquisition module can connect to 3 load cells. There are 24 load cells for the lysimeters and 24 load cells for percolation and total 16 data acquisition modules. Every 4 data acquisition modules are connected to a communication module and the 4 communication modules are connected to a personal computer via RS485 bus and a RS485 to RS232 converter. Software was developed to collect data from the data acquisition modules and a user-defined communication protocol based on Modbus was adopted.

<figure><img src="image_5.png" /><figcaption>Fig.2 The schematic diagram of the data acquisition system</figcaption></figure>

## 2.3 Calibration of the load cells

In order to improve the weighing accuracy, the calibration of the load cells is needed. The calibration can be used to convert the voltage of the load cells into the weight of lysimeters. During calibration procedure, the load cell output was measured every 3 seconds and the average of the 1-min data is calculated. The mass weight is 6.932Kg. Each time a weight was added and the corresponding voltage of the load cell was recorded. Total 6 weights were cumulatively added. The calibration procedure was conducted for all 24 lysimeters and the calibration line is plotted. The data processing was finished using Microsoft Excel 2011.
## 2.4 Outlier detection of the measurement data

The load-cell mass data were recorded every three seconds and the measurement results were saved in txt file for further processing. The data measurement was conducted from 14:42pm of 9th March to 9:39am of 18th March, 2011. For easy analysis, only the data of the No.2 lysimeter collected in 14th March was used for further processing. First, the outlier of the measurement data was detected based on 3-sigma rule. The data out of the range of [mean-3sigma, mean+3sigma] was defined as outlier and eliminated. The data over a 5-min period (100 total values) was used as a group to calculate the mean and the standard deviation. The data from 100 total values after eliminating the outliers were used to calculate the averages. Second, the 5, 7, 11, 15 and 25 points media filters and Savitzky-Golay filters were applied to the averages respectively. The performances of the two-type filters were compared. The mean and standard deviation were used to evaluate the performance. The data processing was finished using Matlab 2011 and Microsoft Excel 2011.

# 3 Results and Discussion

## 3.1 Load cell calibration results

The 24 lysimeters were calibrated respectively. The calibration results of No.1 and No.2 lysimeters were shown in Fig.3. There was a strong linear relationship between the load cell output (mV) and the calibration mass (Kg). The determination coefficients of the No.1 and No.2 lysimeters were 0.9998 and 0.99992 respectively. The calibration equations were established and the calibration coefficients were used for the conversion from the load cell output to the change of mass of the lysimeters.

<figure><img src="image_6.png" /><figcaption>Fig.3 Calibration results of (a) No.1 lysimeter and (b) No.2 lysimeter</figcaption></figure>
## 3.2 The outlier detection results

The 28300 data points collected from No.2 lysimeter were used for outlier detection. The result after the outlier detection using the 3-sigma rule and the averaging of the 100 data points was shown in Fig.4. After the above processing, there were 283 data points used for further processing. It was obvious that there were some outliers in the 283 data points and should be eliminated.

<figure><img src="image_7.png" /><figcaption>Fig.4 Data processing using 3-sigma rule and averaging</figcaption></figure>

The 5, 7, 11, 15 and 25 points media filters and Savitzky-Golay filters were applied to the 283 data points respectively. The mean and standard deviation of the different filters were listed in Table 1. As shown in Table.1, the standard deviation using the media filter reached a minimum when the data point of the filter was 15-point. And the standard deviation of the media filter was smaller than that of the Savitzky-Golay filters, which means that the performance of the median filter was better than the Savitzky-Golay filters.

<table><caption>**Table 1.** The mean and standard deviation of the 5, 7, 11, 15 and 25 points media filters and Savitzky-Golay filters</caption><thead><tr><th></th><th>5-point</th><th>7-point</th><th>11-point</th><th>15-point</th><th>25-point</th></tr></thead><tbody><tr><th colspan="6">Meadian filter</th></tr><tr><th>Mean</th><td>493.814</td><td>493.813</td><td>493.813</td><td>493.811</td><td>493.813</td></tr><tr><th>Standard Deviation</th><td>0.415</td><td>0.414</td><td>0.414</td><td>0.413</td><td>0.414</td></tr><tr><th colspan="6">Savitzky-Golay</th></tr><tr><th>Mean</th><td>493.823</td><td>493.824</td><td>493.824</td><td>493.824</td><td>493.824</td></tr><tr><th>Standard Deviation</th><td>0.432</td><td>0.427</td><td>0.424</td><td>0.422</td><td>0.420</td></tr></tbody></table>

The processing results using 15-point median filter and 15-point Savitzky-Golay filters were shown in Fig.5. The outliers in the Fig.4 were successfully removed after using the 15-point median filtering as shown in Fig.5a. However, the outliers still existed after using the 15-point Savitzky-Golay filtering.
<figure><img src="image_8.png" /><figcaption>Fig.5 Data processing using (a) 15-point median filtering and (b) 15-point Savitzky-Golay filtering</figcaption></figure>

## 4 Conclusion

In this paper, a data acquisition system was developed to collect the data from 24 weighing lysimeters. The calibration process of the load cell was described. An outlier detection method based on the 3-sigma rule and the median filter was proposed to improve the measurement accuracy of the weighing lysimeters. The performance of the proposed method was compared with the method based on Savitzky-Golay filter. Results show that the standard deviations of the 15-point median filter and the 15-point Savitzky-Golay filter applied to the 283 data points were 0.413Kg and 0.422Kg respectively, which means that the performance of the median filter was better than the Savitzky-Golay filter. Moreover the outliers were successfully eliminated using the median filter and were not removed by the Savitzky-Golay filter.

**Acknowledgments.** Our project is supported by National Natural Science Foundation of China (Project No. 31071324) and National Key Technologies R&D Program of China (Project No. 2008BAB38B06).

## References

1. Tim Davie, Fundamentals of hydrology (Routledge, 2003) pg. 35
http://books.google.com/books?id=XAGt03ANojgC&source=gbsNavlinks_s

2. Molden, D., and Oweis. T. Y. Water for food, water for life: A comprehensive assessment of water management in agriculture. Earthscan, London and International Water Management Institute, SriLanka, Colombo, 279-310 (2007)

3. Wu Yunqing, Luo Jin-yao, Wang Fuqing. Development and implementation of the intelligent weighing lysimeter system. Research and exploration in laboratory. Vol.25(4):432-434, 438(2006)

4. Sun Qiang, Zhang Weina, Gao Lifang, Zhang Jianfeng. A new weighing type of measuring and controlling system by lysimeter with high precision. Journal of Xi'an University of Technology. Vol.15(1):56-60(1999)
5. Gholam-Abbas Barani and Mohammad Javad Khanjani. A large electronic weighing lysimeter system design and installation. Journal of the American Water Resources Association, Vol.38(4):1053-1060(2002)

6. Terry A. Howell, Robert L. McCormick, Claude J. Phene. Design and Installation of Large Weighing Lysimeters. Transactions of the ASAE, Vol.28(1):106-112, 117(1985)

7. Sun Qiang, Zhang Jianfeng, Zhang Weina, Gao Lifang. Computer measuring and controlling system of high precision balance lysimeter. Journal of Soil Erosion and Soil and Water Conservation. Vol.5(5):80-84(1999)

8. Yang Xinmin, Shen Bing, Zhang Jianfeng, Liang Yimin. Computer measure and control of a large high precision lysimeter. Transactions of the CSAE. Vol.12(3):72-76(1996)

9. A. Johnson, T. J. Mathews, G. P. Matthews, D. Patel, P. J. Worsfold and K. N. Andrew. High-resolution laboratory lysimeter for automated sampling of tracers through a 0.5m soil block. Journal of Automated Methods & Management in Chemistry, Vol.25(2):43-49(2003)

10. Su Meishua ng, Li Jiushe ng, Rao Minjie. Estimation of crop coefficients for sprinkler- irrigated winter wheat and sweet corn using a weighing lysimeter. Transactions of the CSAE. Vol.21(8):25-29(2005)

11. Niu Yong, Liu Honglu, Wu Wenyong, Yang Shengli. Cucumber transpiration by large-scale weighing lysimeter in solar greenhouse. Transactions of the CSAE. Vol.27(1):52-56(20011)

12. Changming Liu, Xiying Zhang, Yongqiang Zhang. Determination of daily evaporation and evapotranspiration of winter wheat and maize by large-scale weighing lysimeter and micro-lysimeter. Agricultural and Forest Meteorology, Vol.111:109-120(2002)

13. R. W. Malone, J. V. Bonta, D. J. Stewardson, T. Nelsen. Error Analysis and quality improvement of the Coshocton weighing lysimeters. Transactions of the ASAE, Vol.43(2):271-280(2000)

14. Jose O. Payero, Suat Irmak. Construction, Installation, and Performance of Two Repacked Weighing Lysimeters. Irrigation Science 26:191-202(2008)

15. Yan Jingze, Li Yiping, Deng Zuqin, Wang Sheng, Liu Hongyi, Yang Qiguo, Li Xiaoping. Automatic monitoring system of LG-I weighing lysimeter. Transactions of the CSAE. Vol.25(S2):43-48(2009)

16. M. Meshkat, R. C. Warner, L. R. Walton. Lysimeter design, construction and instrumentation for assessing evaporation from a large undisturbed soil monolith. Applied Engineering in Agriculture, Vol.15(4):303-308(1999)

17. T.A. Howell, A.D. Schneider, D.A. Dusek, T.H. Marek, J.L. Steiner. Calibration and scale performance of Bushland weighing lysimeters. Transactions of the ASAE, Vol.38(4):1019-1024(1995)

18. R. W. Malone, D. J. Stewardson, J. V. Bonta, T. Nelsen. Calibration and quality control of the Coshocton weighing lysimeters. Transactions of the ASAE, Vol.42(3):701-712(1999)

19. P.J. Vaughan, T.J. Trout, J.E. Ayars. A processing method for weighing lysimeter data and comparison to micrometeorological ETo predictions. Agricultural water management, Vol.88:141-146(2007)
20. Peter J. Vaughan and James E. Ayars. Noise Reduction Methods for Weighing Lysimeters. Journal of Irrigation and Drainage Engineering, Vol.135(2):235-240(2009)