# An SMS and Email Weather Warning System for Sheep Producers

Anna Weeks, Malcolm Mccaskill, Matthew Cox, Subhash Sharma

► To cite this version:

Anna Weeks, Malcolm Mccaskill, Matthew Cox, Subhash Sharma. An SMS and Email Weather Warning System for Sheep Producers. 11th International Symposium on Environmental Software Systems (ISESS), Mar 2015, Melbourne, Australia. pp.131-140, 10.1007/978-3-319-15994-2_12 . hal-01328537

HAL Id: hal-01328537

https://inria.hal.science/hal-01328537

Submitted on 8 Jun 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An SMS and Email Weather Warning System for Sheep Producers

Anna Weeks¹, Malcolm McCaskill², Matthew Cox³, and Subhash Sharma³

¹,²,³Agriculture Division: Department of Environment and Primary Industries, Rutherglen¹, Hamilton² and Parkville³, Victoria, Australia
{anna.weeks,malcolm.mccaskill,matthew.cox,subhash.sharma}@depi.vic.gov.au

**Abstract.** Sheep are vulnerable to hypothermia shortly after birth and shearing. Since the 1970's sheep weather alerts have been reported at a regional scale by the media up to 24 hours prior to a chill event. The SMS and email weather warning system was designed as an enhanced service to provide sheep producers with advanced warnings of forth-coming chill events, based on local weather forecasts, with personalized chill warnings delivered by SMS and email. A trial was conducted with 30 sheep producers who selected one or more local weather stations and a low, medium or high sensitivity threshold to control the frequency at which messages were sent. Sensitivity thresholds were calculated for each weather station from historical data. Numerical forecast data were sourced from the Bureau of Meteorology, and an email and SMS sent each morning whenever forecast chill exceeded the warning threshold within the 7-day forecast period. Participants were interviewed by telephone after a 2-month trial. The alerts were found to be clear and reasonably accurate, but produced an unexpected high number of false warnings at some sites. The SMS format was well received, and farmers were generally happy to continue the trial. False warnings were attributed to over-prediction of wind speeds at some sites relative to on-ground weather stations, most of which were in northern Victoria.

**Keywords:** Sheep · forecast skill · evaluation · chill index

## 1 Introduction

Sheep in southern Australia are at risk of death through hypothermia shortly after shearing and in the first 3 days after birth. Since the early 1970's the Australian Bureau of Meteorology (BoM) has issued "Sheep Weather Alerts", based on an algorithm that incorporates the effects of wind speed, temperature and rainfall on heat loss from the sheep [2]. These have been broadcast through the mass media up to 24 hours prior to a chill event, but until recently were not issued in Victoria during winter, to avoid "warning fatigue". More recently, the availability of the BoM numerical forecast data has enabled the Department of Environment and Primary Industries (DEPI) to provide seven-day, site-specific forecasts of sheep wind-chill on its website. However, a limitation of a web service is that users are unlikely to check at times when a
stress is not expected. Surprise chill events can occur in summer, such as the chill-
event reported by Bird and Cayley [1] where 100,000 off-shears sheep died in south-
west Victoria in December 1987. To utilize the new weather forecast dissemination
capabilities, DEPI has developed and trialed a subscription service that provides ad-
vance warning of extreme chill events to sheep producers in Victoria via SMS and
email notifications. This paper describes development of the system, the accuracy of
forecasts, and user feedback.

## 2 Methodology

2.1 System Architecture

The SMS and email weather warning system ran daily as an automated service on the
Victorian Resources Online server [5] (Fig. 1). Through a subscription web-page,
subscribers selected from 56 weather stations and specified the sensitivity threshold at
which warnings were sent (low, medium or high). The thresholds were based on a
statistical analysis of historical weather station data, with relationships derived for
each station describing the expected number of warnings per threshold.

<figure><img src="image_2.png" /><figcaption>Fig. 1. System architecture of SMS and email weather warning system</figcaption></figure>

Weather data (Table 1) were downloaded from the BoM Server via an ftp link and
comprised; historical weather station data, seven days of the ~3km resolution Austral-
ian Digital Forecast Data (ADFD) and seven days of the ~40km resolution Access-G
Forecast. The Access-G precipitation forecast was required because the ADFD only
provided a one-day forecast. Inputs were converted to standard units of Co-ordinated
Universal Time (UTC), wind speed (m/s), hourly rainfall (mm/hr) and temperature (°C). Accumulated 3-hourly rainfall was downscaled to an hourly basis and missing data in-filled by linear interpolation.

The service ran a series of scheduled tasks early each morning. Historical and seven-day forecast data were first copied from the BoM server then chill-indices calculated for each weather station. Forecasts of wind-chill were made available on the corporate website through an interactive graphical display then a sub-routine compared the forecast to the subscriber threshold. An email and/or SMS was issued based on the first occurrence and length (hours) of the first forecast event that exceeded the specified threshold.

<table><caption>Table 1. BoM Weather data used for sheep-wind-chill calculations</caption><thead><tr><th>Period</th><th>Source</th><th>Time units</th><th>Element</th><th>Units</th><th>Frequency</th></tr></thead><tbody><tr><td rowspan="3">3-weeks<br>historical<br>data</td><td rowspan="3">BoM Aus-<br>tralian<br>Weather<br>Station Feed</td><td rowspan="3">[Year,<br>month,<br>day, hour]<br>local time</td><td>Wind speed</td><td>m/s</td><td>hourly</td></tr><tr><td>Rainfall since<br>9am</td><td>mm</td><td>hourly</td></tr><tr><td>Temp</td><td>°C</td><td>hourly</td></tr><tr><td rowspan="3">Forecast<br>day 1-7<br>(except<br>for rain)</td><td rowspan="3">Australian<br>Digital Fore-<br>cast Grid<br>(ADFD)<br>~3km grid</td><td rowspan="3">Co-<br>ordinated<br>Universal<br>Time UTC</td><td>Wind speed</td><td>knots</td><td>hourly</td></tr><tr><td>3-hr expected<br>rainfall</td><td>mm/3hr</td><td>3-hourly</td></tr><tr><td>Temp</td><td>°C</td><td>hourly</td></tr><tr><td>Forecast<br>day 2-7</td><td>Access-G<br>(IDY25001)<br>~40km grid</td><td>Seconds<br>since start<br>date UTC</td><td>Accum. rain-<br>fall over mod-<br>el assimilation</td><td>mm/3hr</td><td>3-hourly</td></tr></tbody></table>

## 2.2 Chill Modelling and Calculation of Thresholds

Sheep wind chill was calculated using the chill index of Nixon-Smith [2], which estimates the potential hourly heat loss from new-born lambs ($C_{hr}, kJ/m^2 \cdot hr$) from air temperature ($T, °C$), wind speed at a 0.4m height ($v_{0.4}, m/s$) and rainfall ($R, mm/ 24hr$) accumulated over the previous 24 hours:

$$C_{hr} = (11.7 + 3.1\sqrt{v_{0.4}}(40 - T) + 481 + 418(1 - e^{-0.04R})) \quad (1)$$

The BoM wind speed at 10m ($v_{10}$) was down-scaled to a lamb height of 0.4m using the empirical relationship developed by McCaskill et. al (unpublished).

$$v_{0.4} = \begin{cases} 0.104 + 0.0667v_{10} & v_{10} < 2.682 \\ -0.863 + 0.427v_{10} & v_{10} \ge 2.682 \end{cases} \quad (2)$$

Three thresholds were calculated for each weather station from historical data to give subscribers the option to control the sensitivity of the warning service. The low, medium and high thresholds were set so that approximately 10, 15 and 20 events would be triggered each year, noting that with a seven-day forecast, subscribers could potentially receive up to seven warnings for each event. The annual chill events for a given
threshold value was found to be well approximated by the upper incomplete gamma function ($p(\alpha, x, P)$); shown in Fig. 2 for the Rutherglen weather station.

$$p(\alpha, x, P) = \frac{P}{\Gamma(\alpha)} \int_{x}^{\infty} e^{-t} t^{\alpha-1} dt \text{ where } \Gamma(\alpha) = \int_{0}^{\infty} e^{-t} t^{\alpha-1} dt \text{ and } x = \frac{k-1000}{\theta} \quad (3)$$

Here $\alpha$, $\theta$ and $P$ are fitted constants and the threshold value ($k$) has a lower limit of 1000 kJ/m²/h. The chill relationship of eqn. 1 is presented in Fig. 3 illustrating the wind, rainfall and temperature conditions required to trigger a warning under the low, medium or high threshold values derived for the Rutherglen weather station. Values of $\alpha$, $\theta$ $P$ and $k$ for this and other representative weather stations are presented later in the paper (see Table 3).

<figure><img src="image_3.png" /><figcaption>Fig. 2. The expected annual number of sheep-wind-chill events for wind-chill threshold ($k$), calculated from historical hourly data (1998-2013) for the Rutherglen weather station.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Illustration of the sheep-wind-chill relationship of eqn. 1, showing the combination of wind speed, temperature and rainfall required to trigger thresholds for the Rutherglen station.</figcaption></figure>
## 2.3 Forecast Skill

The forecast skill of each weather station was assessed for forecasts 1 to 7 days ahead over the 77 days between the 20th July to the 4th October 2014. Consistent with the approach used to quantify forecast skill for dichotomous forecasts by the BoM [4], skill was evaluated by means of a contingency table (Table 2) and calculations of bias and Equitable Threat Score (ETS).

<table><caption>**Table 2.** Contingency table to assess the types of errors being made</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="2">Forecast</th><th rowspan="2"></th></tr><tr><th>Yes</th><th>No</th></tr></thead><tbody><tr><th rowspan="2">Observed</th><th>Yes</th><td>Hits</td><td>Misses</td><td>Observed yes</td></tr><tr><th>No</th><td>False alarm</td><td>Correct negative</td><td>Observed no</td></tr><tr><td colspan="2"></td><td>forecast yes</td><td>forecast no</td><td>Total</td></tr></tbody></table>

*Bias* compared the frequency of forecast chill-events to observed events indicating whether the system tended to under-forecast (*Bias*<0) or over-forecast (*Bias*>1).

$$Bias = \frac{\text{hits+false alarms}}{\text{hits+misses}} \quad 0 < Bias < \infty, \text{ Perfect score} = 1 \quad (4)$$

The Equitable Threat Score (ETS) compared the forecast occurrence of events to the observed, adjusting for hits associated with random chance (*h*<sub>r</sub>) to reduce the effect of different climatic conditions (eg. wet or dry).

$$ETS = \frac{\text{hits} - \text{h}_r}{\text{hits+misses+false alarm} - \text{h}_r} - \frac{1}{3} < ETS < 1, \text{ No skill} = 0, \text{ Perfect} = 1 \quad (5)$$

$$\text{where } h_r = \frac{(\text{hits} + \text{misses})(\text{hits} + \text{false alarm})}{\text{total}}$$

The predictive skill of the hourly forecast data was assessed using the Nash-Sutcliff Coefficient of Efficiency (CoE) which ascertains whether the forecast is a better predictor of observed data than the observed mean. The CoE was calculated as

$$CoE = 1 - \frac{\sum_{t=1}^{T} (Q_o^t - Q_f^t)^2}{\sum_{t=1}^{T} (Q_o^t - \bar{Q}_o)^2} \quad \text{No skill} \le 0, \text{ Perfect} = 1 \quad (6)$$

where $Q_o$ is the observed data, $\bar{Q}_o$ the mean observed data, and $Q_f$ the forecast data. The mean bias ($\bar{Q}_f - \bar{Q}_o$) between forecast and observed data was also calculated.

## 2.4 Trial Evaluation

Victorian Sheep producers were recruited for an initial trial of the system at the 'Best Wool/Best Lamb' Conference in Bendigo in late June 2014. The trial evaluation was conducted by an independent reviewer [3] in September and October of 2014 in which 30 subscribers were interviewed by telephone. Participants included large, medium and small-scale sheep farmers drawn from across the state. The interviews adfa, p. 5, 2011.
included both open and closed questions. Findings were organized to answer each of the overarching questions (i) “Were clear, accurate and timely alerts received?”, (ii) “Did alerts lead to reduced lamb losses?”, and (iii) “Did farmers value the alerts?”

# 3 Results

## 3.1 Trial Results

In general, the alerts were found to be clear and reasonably accurate although there were some timing issues possibly attributable to technical glitches. The SMS format was well received. Farmers were generally happy to continue the trial and would potentially be happy to pay a small amount for a commercial service.

The evaluation found that the alerts did not lead to farm responses that saved lives during lambing except for small-scale farming where sheep could be managed individually. This was primarily due to the fact that due to the risks involved in moving ewes during lambing most farmers had already taken the precaution of providing as much protection for their birthing ewes as possible.

However, almost all farmers saw significant potential value in the alerts for shearing when surprise cold can cause significant losses amongst newly shorn sheep. Participants cited greater flexibility to respond using a range of options such as moving sheep into shelter or rescheduling shearing.

The independent reviewer recommended that the trial be continued through shearing with the scope expanded to include sudden changes in weather that might be experienced during the summer months that are potentially dangerous to newly shorn sheep, generally between November and May.

## 3.2 Forecast Skill

The forecast skill of each weather station was assessed for forecast days one to seven, under the low, medium and high thresholds over the 77 days between the 20th July to the 4th October 2014. Key determinants of skill included calculations of bias and ETS which have been presented in Table 3 along with other results for a subset of weather stations.

During the trial period the ETS measure of forecast skill varied with weather station, forecast day and the threshold, ranging from a perfect score of 1.0 at Horsham to 0.2 at Wangaratta (Table 3) for the day-one forecast. Stations with a lower ETS generally predicted actual chill events well however their score was reduced by the frequent occurrence of ‘false alarms’; represented by a bias>1 in Table 3. Across the 56 weather stations there was a significant (P < 0.05) decline in forecast skill from day one to day seven (Fig. 4a). In contrast, there was less differentiation in the forecast skill between the low, medium and high thresholds (Fig. 4b).
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** The ETS range over 56 stations where box-plot represents [99,75,50,25,1] percentile, (*=mean) for (a) forecast day one to day seven, (b) low, med and high threshold</figcaption></figure>

The Coefficient of Efficiency (CoE) was used to evaluate the forecast skill of the hourly sheep-chill-index and underlying temperature, rainfall and wind-speed forecast data. Overall the sheep-chill-index forecast skill was 'good' with over 75% of stations achieving a CoE>0.6 for forecast days one to four (Fig. 5). This could be attributed to forecast skill of temperature which far exceeded that of rainfall and wind-speed. The accumulated rainfall saw a sharp decline in CoE between days one and two which coincided with the transition between the 3km grid ADFD forecast to the 40 km grid Access-G rainfall forecast. The Wind-speed forecast showed less skill with only 25% of stations achieving a CoE>0.6 by day four.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Range of CoE over 56 stations for chill-index, accumulated rain, wind speed and temperature showing decline in predictive skill over seven days. Example response given for Hamilton and Rutherglen stations.</figcaption></figure>
Forecast bias across the 56 stations showed little change with increasing forecast length (Fig. 6), apart from a greater divergence in bias at longer forecast lengths. In general the forecast wind-speed exceeded the observed with over 75% of stations showing a positive mean-bias. This followed through to the calculation of chill-index where the mean forecast chill-index exceeded the observed by more than 10 kJ.m⁻².hr for over 75% of stations. For example at Rutherglen, wind speed was overestimated by 1.1 m/s from the one-day forecast and chill by 22 kJ.m⁻².hr. Overall there was a strong correlation between over-prediction of mean wind-speed and the number of false alarms triggered.

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Range of mean-bias (Δ) over 56 stations for chill-index, accumulated rain, wind speed and temperature. Example mean-bias given for Hamilton and Rutherglen stations.</figcaption></figure>

# 4 Discussion

The forecast skill varied widely from station to station and this was reflected in the mixed comments received through the project evaluation. In general, although there was a decline in performance over successive forecast days, weather stations that had a high forecast skill on day-1 maintained that performance over the seven day forecast. Of the 56 weather stations, 33% performed well with a day-1 forecast skill as measured by the ETS of > 0.6. A further 46% of stations performed moderately-well with a day-1 ETS of between 0.2 and 0.6. Chill events were generally well forecast for these stations but the forecast score was brought down by a high number of false alarms. Significantly, many of the stations that triggered multiple false alarms were located across the Northern region of Victoria (eg. Rutherglen, Wangaratta), where severe frosts often bring hourly temperatures to below zero. Subscribers to these stations commented on the fact that occasionally alerts were “*clearly just for frost*” rather than the “*holy trinity of wind rain and cold*” [3]. It was found that at low tem-adfa, p. 9, 2011.
peratures the wind-chill algorithm is highly sensitive to small changes in wind-speed (Fig. 3) and that the cold temperatures combined with an over-prediction of wind-speed across these locations was enough to trigger a series of false-alarms. Enforcing a precipitation requirement was found to effectively reduce false-alarms for these stations as simply increasing the threshold made little improvement to the ETS.

The difference between the forecast wind and that observed at the weather station highlights not only the difficulty of accurately predicting wind speed but also one of the fundamental assumptions made by this system that the 3 km gridded AFDF data and the 40 km gridded Access-G rainfall provide an adequate representation of the climate at the point of the weather station. Options to correct this include (i) empirical relationships between the station and the numerical forecast, (ii) calculating warning thresholds from historical records of the gridded forecast data, and (iii) improvements to the forecast models that increase their accuracy close to the ground surface.

# 5 Conclusion

The SMS and email weather warning service for sheep producers accurately predicted sheep-wind-chill events, but at some stations generated a high number of false alarms due to the sensitivity of the sheep chill algorithm to wind-speed at low temperatures, and the consistent over-prediction of wind speed in the forecast data. The forecast skill varied widely from station to station and this was reflected in the mixed comments received through the project evaluation. However in general the alerts were found to be clear and reasonably accurate with most farmers happy to continue with the trial.

# References

1. Bird R, Cayley J (1991). Bad weather, shelter and stock losses. Agricultural Science: the journal of the Institute of Agricultural Science 4 (4) 18-19.

2. Nixon-Smith WF (1972). The forecasting of chill risk ratings for new born lambs and off-shears sheep by the use of a cooling factor derived from synoptic data. Working paper No. 150, Bureau of Meteorology, Melbourne.

3. Revium (2014). Department of Environment and Primary Industries Systems for Enhanced Farm Services Program. Spatial Discovery & Visualisation – SDV1: IBAW Trial Participants Survey. Report 1-39

4. Stanski, H.R., L.J. Wilson, and W.R. Burrows (1989). Survey of common verification methods in meteorology. World Weather Watch Tech. Rept. No.8, WMO/TD No.358, WMO, Geneva, 114 pp

5. VRO Victorian Resources Online (2014). Internet Based Agricultural Warnings. Accessed 7th Nov 2014 at http://vro.depi.vic.gov.au/dpi/vro/vrosite.nsf/pages/ibaw