# An evaluation framework to build a cost-efficient crop monitoring system. Experiences from the extension of the European crop monitoring system

Raul Lopez-Lozano, Bettina Baruth

► To cite this version:

Raul Lopez-Lozano, Bettina Baruth. An evaluation framework to build a cost-efficient crop monitor-
ing system. Experiences from the extension of the European crop monitoring system. Agricultural
Systems, 2019, 168, pp.231-246. 10.1016/j.agsy.2018.04.002. hal-02619128

HAL Id: hal-02619128

https://hal.inrae.fr/hal-02619128

Submitted on 25 May 2020

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An evaluation framework to build a cost-efficient crop monitoring system. Experiences from the extension of the European crop monitoring system

<figure><img src="image_2.png" /></figure>

Check for
updates

López-Lozano Raúl*, Baruth Bettina

European Commission, Joint Research Centre (JRC), Directorate for Sustainable Resources, Food Security Unit, Via E. Fermi 2749, I-21027 Ispra, VA, Italy

## ABSTRACT

This paper presents an evaluation framework followed to identify cost-efficient alternatives to extend the MARS Crop Yield Forecasting System (MCYFS), run by the European Commission Joint Research Centre since 1992, to other main producing areas of the world: Eastern European Neighbourhood, Asia, Australia, South America and North America. These new systems would follow the principles and components of the MCYFS Europe: a meteorological data infrastructure, a remote sensing data infrastructure, a crop modelling platform, statistical tools, a team of analysts and a crop area estimation component. The framework designed evaluates the performance of the possible MCYFS-like system realizations against six defined objectives and their costs. Possible monitoring systems are based on a combination of different technical solutions for each of the MCYFS components, and are evaluated through an automatic algorithm that calculates the expected system performance -relying on a priori expert judgement-, the costs, and possible risks to construct some technical solutions, to finally identify the cost-efficient ones. A baseline system, achieving the minimum required performance, was identified as the most efficient starting point for the MCYFS extension in all the geographical areas. Such system would be built upon: (i) near real-time reanalysis meteorological products; (ii) remote sensing data from low-resolution (~1 km) platforms with a long-term product archive; (iii) crop models based on crop-specific model calibration from experimental data published in scientific literature; (iv) statistical methods based on trend and regression analysis applied to national level; (v) a team of analysts with specific technical profiles (on meteorology, remote sensing, and agronomy); and (vi) digital classification of very high resolution imagery supported by non-expensive ground surveys for area estimation. In countries where accessibility to local data and resources is high the baseline system can be upgraded enhancing some of the components: sub-national statistical analysis with additional statistical methods like multiple regression or scenario analysis; recruitment of experts on local agricultural conditions in the team of analysts; local calibration of crop models with experimental data; and exploiting high and low resolution biophysical products from remote sensing for crop monitoring.

## 1. Introduction

High fluctuations of agricultural production during the last decade depicted a scenario of volatility of agricultural prices, leading to food crises and social unrest in different parts of the world. In 2007/2008 the coupled scarcity of goods at world level, consequence of unfavourable conditions during the crop season, and the growing demand of food, resulted in high market tensions and a sharp increase of food prices (von von Braun, 2008). In the 2008/2009 season, abundant agricultural production thanks to favourable weather conditions in the main producing areas reversed the situation, and prices at world level went down, until 2010/2011 when a severe drought in the Black Sea area severely constrained cereal production, increasing international food prices (Zaman et al., 2011).

These events triggered the Directorate-General of Agriculture and Rural Development (DG-Agriculture) of the European Commission (EC) to launch and finance in 2011 the GLOBCAST study to assess the feasibility of extending the European MARS Crop Yield Forecasting System (MCYFS) -run by the EC Joint Research Centre since 1992- to the main grain producing regions of the world (Russia, Ukraine and Kazakhstan; India; China; Australia; North America and South America).

The objective of the GLOBCAST study was to design and develop a global crop monitoring system to provide EC policy-makers with the necessary information to implement effective measures to manage agricultural markets, and contributing to international initiatives, such as the Agricultural Market Information System (AMIS) propelled by G-20 and coordinated by the Food and Agriculture Organization (FAO), to increase transparency in agricultural markets. Independent, evidence-

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: raul.lopez@ec.europa.eu (R. López-Lozano).</footnote>
<table><caption>**Table 1**

Description of objectives and their priority –established by the EC DG-Agriculture– of the crop monitoring system.</caption><thead><tr><td>Objective</td><td>Description</td><td>Priority</td></tr></thead><tbody><tr><td>1. Alert and warning of hazards in crop growth</td><td>Detection of major meteorological constraints or extreme events that can have a significant impact on crop development and growth from sowing/planting to harvest.</td><td>Medium</td></tr><tr><td>2. Qualitative crop growth analysis</td><td>A qualitative analysis of crop growth and development anomalies with a potential impact on crop yields, based on the system indicators: e.g. crop growth above or below the long-term and/or short-term average, crop cycle is advanced as compared to the average year, etc.</td><td>Medium</td></tr><tr><td>3. Quantitative crop yield forecast</td><td>Statistical analysis of the system indicators to produce a quantitative crop forecast. Forecast activities start early in season with a yield trend analysis, and are replaced by more sophisticated statistical methods as the growing season progresses.</td><td>High</td></tr><tr><td>4. Quantitative crop area estimation</td><td>Quantitative crop area figures will be produced that, jointly with crop yield forecasting, will allow calculating crop production. Crop acreage estimation will be provided early and late in the season, based on the use of medium/high resolution remote sensing imagery for digital classification to estimate shifts in crop acreage and spatial distribution from one season to another.</td><td>High</td></tr><tr><td>5. Bulletin production</td><td>Regular analysis will be published in bulleting consisting on: description of the main agro-meteorological event; qualitative analysis of crop conditions; and the quantitative production figures.</td><td>High</td></tr><tr><td>6. Accessibility of information</td><td>The dissemination to the public of the system indicators and outputs –crop modelling indicators, meteorological data and remote sensing products– through dedicated software and/or upon request.</td><td>Low</td></tr></tbody></table>

based and timely qualitative information on crop growth conditions and crop yield forecasting in the main producing areas of the world are critical for policy-making, preventing market disruptions, reducing market speculation and thus contributing to overall food security.

The objective of this paper is to present the conceptual framework that was designed to perform a cost-efficiency analysis of different technical alternatives to build up a MCYFS-like system in the mentioned producing areas. The framework constructed is partially inspired by a similar exercise done some years ago by van Leeuwen et al. (2011) when the authors benchmarked the contribution of new NASA satellite products to improve global production estimates in the FAS (Foreign Agricultural Service of the United States Department of Agriculture) decision support system.

The framework presented comprises the definitions of the objectives to be accomplished by the system, the components of that system –based on the principles of the European MCYFS– and a scale to score the level of attainment for these objectives of the technical solutions proposed to build the components. The team of analysts at MARS evaluated the attainment level of every technical solution, estimated their costs, and established the risk level –mainly a consequence of constraints to access local information and resources in some of the geographical areas covered– to build some of them. Finally, an automatic algorithm was implemented to identify –according to the scores, costs and risks of the technical solutions– the cost-efficient MCYFS-like systems.

The work conducted aims at being an objective a priori analysis of the most suitable realizations of a MCYFS-like crop monitoring system in the main producing areas of the world, responding to the premises and requirements of the EC DG-Agriculture for the management of agricultural markets in a global context, as stipulated in the European Regulation No 1306/2013.

This kind of preliminary assessment, being essential to devise a feasible implementation for such a monitoring system –which requires an important investment from taxpayers– is not frequently made public. The strategy followed in the evaluation framework to address the system's cost-efficiency is a valid example for the reader that illustrates how to translate the functioning of a complex operational system –with several objectives, components, interactions between components, limitations, and expected outputs– in a benchmarking procedure that may help to identify possible vulnerabilities or improvements. All this information is explained in detail in Chapter 3.

The cost-efficient MCYFS-like systems identified by the evaluation framework constitute the main output of the evaluation exercise. In general existing continental/global crop monitoring systems use a wide variety of data sources and methods including meteorological indicators, satellite earth observation (EO) products, crop growth indicators produced by crop growth models, etc. (Kucera and Genovese,

2004; Van Leeuwen et al., 2006; Mueller and Seffrin, 2006; Wu et al., 2014). These methods constitute non-exclusive technical alternatives to build the components of a crop monitoring system. All of them can provide reliable information on crop status, presenting advantages and limitations when used quantitatively as a predictor of crop yield. The results detailed in Chapter 4 show the key components of a MCYFS-like system in order to achieve different performance in crop monitoring, area estimation, and yield forecasting. That analysis is seized to the specific principles and requirements of the MARS monitoring system, but the outputs –particularly on what regards the composition of the baseline cost-efficient system proposed and the roadmap to upgrade it considering their set-up and operational costs– can be of interest in the crop monitoring and yield forecasting community. The availability of local information and the means to collaborate with local authorities are essential when building a crop monitoring system like the MCYFS outside the EU. The impact of these two factors in the possible cost-efficient systems is illustrated through three scenarios (low constraints, medium constraints, and severe constraints) describing the differences found among the geographical areas to be covered in GLOBCAST.

## 2. Objectives and architecture of a MCYFS-like crop monitoring and yield forecasting system for the main grain producing areas of the world

### 2.1. System objectives

A range of objectives for the MCYFS-like monitoring system to be implemented in the main grain producing areas of the world is summarized in Table 1. The first three objectives regard monitoring crop growth and yields, and each one of them represents a different but complementary focus of analysing the effects of weather on crop yield. Objective 1 - Alert and warning of hazards in crop growth- has an anticipatory nature, aiming at identifying possible weather extreme events –even those happening before sowing– that may have a significant impact on final crop yields. Objective 2 - Qualitative crop growth analysis- implies an analysis in near real-time of current crop growth indicators –essentially from crop models and remote sensing– placing the ongoing season below/above a long- or medium-term average that could lead to yield anomalies at the end of the season. Whereas objectives 1 and 2 provide both a qualitative output, objective 3 - Quantitative crop yield forecast- consists in analysing statistically the available indicators –meteorological, crop growth remote sensing– to produce a yield forecast. Although the outputs from all these three objectives are of high value for monitoring agricultural markets, DG-Agriculture attributed a high priority to quantitative yield forecasting (right column of Table 1), as yield figures can be directly used to build agricultural balance sheets, and integrated with other information, they
are an important instrument for policy-making.

Objective 4 is the quantitative crop area estimation, which is not covered by the European MCYFS, where early and late-season area estimates for the Member States are made available by EUROSTAT following a specific calendar (Eurostat, 2015). Outside the EU, crop area estimation is a necessary objective and thus requires a dedicated system component, especially relevant for those countries where inter-annual production variability is largely influenced by sown area.

The last two objectives 5 and 6 relate to the dissemination of the system outputs. Regular bulletins constitute the reference documents containing the results of the monitoring activities: reports on weather and crop growth monitoring, remote sensing analysis, yield forecasts and area estimates. Therefore, the release of regular bulletins has a high priority for stakeholders. Moreover, all the indicators of the system shall be publicly available with an appropriate infrastructure (e.g. internet data portal).

## 2.2. System components and workflow

The monitoring system to be built, following the principles of the European MCYFS, is composed of six components:

A) An infrastructure to acquire and process meteorological data.

B) An infrastructure to process remote sensing data.

C) A software platform to calibrate and run crop growth models.

D) A statistical toolbox to perform yield forecasts, including an archive of crop area and yield statistics.

E) A team of analysts in charge of the quantitative and qualitative analysis of crop growth and yield.

F) A specific component to estimate crop area from remote sensing and field sampling.

A schematic representation of the system and the different components is given in Fig. 1, whereas Fig. 2 describes the data and processes workflow.

The meteorological data infrastructure acquires and processes in

near-real time (NRT) weather data from primary sources: weather stations and atmospheric models. Data processing in this component comprises: error-checking of the original raw data; spatial interpolation of primary weather indicators (e.g. temperature, precipitation, etc.) to a reference system grid, and deriving advanced indicators (e.g. potential evapotranspiration, incoming radiation when not included as primary indicators). The computation of more complex indicators such as the standardized precipitation-evapotranspiration index (SPEI, Vicente-Serrano et al., 2009), the agricultural stress index (ASI, Hoolst et al., 2016), or the global water satisfaction index (GWSI, Nieuwenhuis et al., 2006), is considered as well.

Remote sensing products are used in the system qualitatively, detecting positive or negative anomalies in crop growth, but also quantitatively, as yield predictor. A full description of the use of remote sensing indicators in the MCYFS is given in Royer and Genovese (2004) or Baruth et al. (2006). Overall, the infrastructure for remote sensing relies on medium-low spatial resolution (from 500 to 1 km) imagery to monitor crop growth and development. As an example, in the European MCYFS, this component includes the processing of raw satellite imagery to produce 10-day composites of biophysical products using dedicated algorithms (e.g. Gobron et al., 2002; Weiss et al., 2010), based on SPOT-VEGETATION and METOP-AVHRR. More recently, NDVI and biophysical products -fAPAR, LAI- at 1 km resolution were made available by the land service from the EC Copernicus Global Land Service (Verger et al., 2013, 2015), constituting a seamless historical archive from 1998 up to the present using data of the SPOT-VEGETATION and PROBA-V platforms. This component may also include remote sensing products that can be used in the crop modelling component (Fig. 2), e.g. incoming solar radiation, used as input for crop models (Bojanowski et al., 2013), when these parameters are not reliable enough from weather stations.

The BioMA crop growth modelling platform (http://bioma.jrc.ec.europa.eu/), developed at the JRC, constitutes the engine of the third system component. It uses data from the meteorological data infrastructure to simulate crop growth in NRT. In BioMA several general crop growth models have been implemented: WOFOST (Van Diepen

<figure><img src="image_3.png" /><figcaption>Fig. 1. Overall crop monitoring system structure. Letters A to F identify the six components of the system.</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>Fig. 2. Data and processes workflow of the MCYFS-like crop monitoring and yield forecasting system to be built in the main grain producing areas of the world.</figcaption></figure>

et al., 1989), CropSyst (Stöckle et al., 2003), STICS (Brisson et al., 2009); and the crop-specific models CaneGro (Singels et al., 2008) for sugarcane, and WARM (Confalonieri et al., 2009) for rice. Moreover, this third component includes, in addition to the BioMA platform, a dataset of soil physical properties from existing soil maps –necessary to simulate soil water balance– and crop experimental data (e.g. Boons - Prins, 1993) used for crop model calibration.

The indicators provided by the meteorological, remote sensing and crop modelling components are derived and aggregated at a 25 km resolution reference grid. Then, they are also aggregated at administrative unit level (following the Global Administrative Units Layers, GAUL, from FAO) to produce regional indicators. The aggregation is based on land use -taken from existing land cover maps such as GLOBCOVER (Arino et al., 2008) or GLC 2000 (Bartholomé and Belward, 2005), and crop area regional statistics to weight sub-national administrative units according to crop occurrence. As an example, a detailed description of the aggregation of remote sensing indicators can be found in Genovese et al. (2001). The mentioned three components include a historic archive of grid and regional indicators, where additional ones such as long-term and medium-term averages, quantiles, etc. needed for anomaly detection, are regularly calculated and updated.

Regional indicators produced by the first three components aggregated at different administrative levels are analysed statistically in the fourth component of the MCYFS-like system, the statistical database and toolbox, to produce a yield forecast (Fig. 2). This component has two main elements: a repository of historical time-series of official crop yield and area statistics; and a software called CoBo (Control Board) where different statistical methods are implemented to produce yield forecasts. These methods try to identify the statistical relationship between the different indicators and yields statistics in the past –through the analysis of yield trends, different regression methods, and similarity analysis of time-series– to produce a yield figure. Moreover, CoBo, developed for the European MCYFS, is also an archive of the forecasting exercises and a management tool to guarantee the traceability of the yield figures produced.

The analysis team is the fifth component of the system, and has a central role for the accomplishment of all the system objectives

explained in Section 2.1, with the exception of area estimation. The European MCYFS is a data driven decision-support system, where the role of the analyst is key. The analysts investigate the indicators provided by the rest of the components and identify the ones that better explain actual crop growth and yields. They select the appropriate statistical methods to produce a reliable yield forecast and report the results of the mentioned analysis in the form of regular bulletins (see https://ec.europa.eu/jrc/en/mars/bulletins). Moreover, the analysts in the MCYFS have responsibilities for the system maintenance: highlighting possible caveats and limitations in some of the components; proposing scientific and technical solutions to improve them; but also maintaining links with local institutions of countries to exchange data and resources. The analysis team is multidisciplinary, with professionals having a scientific/technical specialization –e.g. on agro-meteorology, agronomy, crop modelling, remote sensing, statistics…– that favours the development and maintenance of the system components. Each analyst is also specialized in a regional context (countries, group of countries), which permits the analysts to reach an expertise on the local agro-climatic conditions, to better understand how the crop indicators produced by the different system components describe actual yield variations in that regional context, and to consolidate the relationship with local partners. This analysis team approach of the European MCYFS is followed for the expansion of the system outside the EU.

The objective of area estimation (Table 1) –necessary to provide crop production estimates– has a nature very different compared to the other objectives of the system. Area estimation, as it is conceived in the MCYFS-like system, is a task that requires specific skills on spatial sampling and remote sensing imagery classification, quite different from the objectives on crop monitoring and yield forecasting, both driven by expertise on biophysical monitoring and modelling. Therefore, the component on crop area estimation is considered independent from all the other system components, and focused exclusively on achieving objective 4 (Table 1). It is actually a component not existing in the European system, where area estimates are provided during the season by the EU member states. The area estimation component relies on the use of high-resolution remote sensing data –in some cases combined with field surveys, see Section 2.3– as proposed by Gallego
<caption>**Table 2**</caption>

Technical alternatives considered to build up the six components of the crop monitoring system.

<table><thead><tr><th colspan="7">Crop monitoring system components</th></tr><tr><th></th><th></th><th>A. Meteo</th><th>B. Remote Sensing</th><th>C. Crop modelling</th><th>D. Statistical tools</th><th>E. Analysis team</th><th>F. Crop area</th></tr></thead><tbody><tr><th rowspan="3" scope="rowgroup">Technical solution level</th><th scope="row">advanced</th><td>Input: Daily observations from weather stations (high density with local networks)<br>Processing: Error-checking, gap-filling, aggregation to daily values, computation of advanced variables and spatial interpolation</td><td>Input: High resolution (&lt;50 m) multispectral imagery on a subset of relevant agricultural areas<br>Processing: Radiometrically calibrated biophysical products. Fusion of low and high resolution products for crop-specific monitoring.</td><td>Models: General crop growth models (e.g. WOFOST, CropSyst...) plus specific solutions for rice and sugarcane, calibrated using local agronomic experimental data collected from dedicated field studies.<br>Soil Data: Detailed local soil maps to simulate water balance<br>Additional crop management data: fertilization treatments, pest and diseases</td><td>Methods: Forecasts based on the assimilation of remote sensing products into crop growth models<br>Statistical data: official yield and area statistics at sub-national level (GAUL 2)</td><td>Resources: Individual teams with specialized scientific profiles for each geographical area with expertise on local conditions (e.g. networking with local partners)</td><td>Input: Medium/high resolution (&lt;100 m) multispectral imagery<br>Methodology: Digital classification of satellite imagery, supported by ground data collected from optimal sampling protocol</td></tr><tr><th scope="row">intermediate</th><td>Input: Daily observations from weather stations (WMO-GHCN stations)<br>Processing: Error-checking, gap-filling, aggregation to daily values, computation of advanced variables and spatial interpolation</td><td>Input: Low resolution (~1km) platforms<br>Processing: Radiometrically calibrated and interoperable products across platforms (e.g. GIO biophysical products). Long-term archives</td><td>Crop model: General crop growth models (e.g. WOFOST, CropSyst...) plus specific solutions for rice and sugarcane, calibrated using agronomic data collected from literature.<br>Soil Data: Continental soil map to simulate water balance<br>Additional crop management data: distribution of irrigated areas and single/multiple cropping</td><td>Statistical methods: Multiple linear regression and similarity analysis applied at sub-national level<br>Statistical data: official yield and area statistics at sub-national level (GAUL 1)</td><td>Resources: Medium-sized staff group with specialized individual profiles on agrometeorology, remote sensing and crop modelling</td><td>Input: Medium/high resolution (&lt;100 m) multispectral imagery<br>Methodology: Digital classification of satellite imagery, supported by ground data collected from efficient sampling 'along the road'.</td></tr><tr><th scope="row">minimum</th><td>Input: Modelled weather data (e.g. ECMWF) + Satellite-derived incoming radiation (MSG-SEVIRI)<br>Processing: decoding, aggregation to daily indicators, projection/scaling to reference grid</td><td>Input: Low resolution (~1km) platforms<br>Processing: Radiometrically calibrated, sensor-specific products (e.g. Top-of-canopy NDVI).</td><td>Crop model: General vegetation growth models based on default model calibration.<br>Soil data: N/A<br>Additional crop management data: Land cover maps for model run on arable land areas</td><td>Statistical methods: Linear regression and trend analysis applied at national scale<br>Statistical data: official yield and area statistics at national level</td><td>Resources: Small-sized staff group with general scientific profile on agricultural sciences and no high specialization in the different geographical zones</td><td>Input: Medium/high resolution (&lt;100 m) multispectral imagery<br>Methodology: Digital classification of satellite imagery, no ground data.</td></tr></tbody></table>

(2004; Gallego et al. 2010)) and has specific staff for photo-interpretation, digital classification, and statistical methods for crop area survey design and area estimation. The successful implementation of this component of the MCYFS-like system outside the EU requires collaboration with local institutions, especially when planning and conducting field surveys.

## 2.3. Technical alternatives to build up the system components

Several technical options exist to build each of the six components in the MCYFS-like system, having different complexity, costs and impact in achieving the objectives presented in Section 2.1. In this study, these options are summarized in three distinct technical solutions –minimum, intermediate, and advanced– proposed for each of the six system components. For a given component, technical solutions represent an incremental evolution of the component towards a higher degree of complexity, which is assumed to have a positive impact on the system reliability, but also a cost increment. These three technical alternatives –summarized in.

Table 2– are based on state-of-the-art methods in scientific literature, and available resources that –according to the experience of the MARS analysts– can be implemented operationally for large areas.

In the meteorological data infrastructure the minimum solution relies on the use of modelled weather data at global level: reanalysis and forecast weather data from the European Centre of Medium-Range Weather Forecast, (ERA-Interim and HRES products from ECMWF, see Dee et al., 2011), acquired and post-processed (projection, spatial aggregation of indicators) daily in NRT. ECMWF products were preferred to other NRT reanalysis datasets like the JRA-55, from the Japan Meteorological Analysis (Kobayashi et al., 2015) since it permits the use of processing chains already implemented in the European MCYFS, which uses currently ECMWF data. Other alternative reanalysis data like MERRA, from NASA (Rienecker et al., 2011) were discarded as they are

not NRT products. The intermediate and advanced solutions would incorporate additionally data from meteorological weather stations to the ECMWF dataset, having thus two production lines of meteorological indicators based, respectively, on modelled and observed weather data. The use of observed data requires the implementation of a processing chain to pre-process (error checking, computation of daily indicators...) and interpolate spatially weather data (Micale and Genovese, 2004). The intermediate one will rely on available open access weather data, e.g. from the GHCN (Global Historical Climatological Network, see https://www.ncdc.noaa.gov/ghcn-daily-description). In the advanced level, an additional number of observations from local weather station networks (e.g. from national meteorological institutes) are added to the open datasets, increasing the density of observations to reach a density similar to the European MCYFS – > 2500 active stations in the EU territory– thus improving the quality of the interpolated weather indicators.

In the remote sensing component, the minimum technical alternative consists in the use of sensor-specific low-resolution products (e.g. simple NDVI), which are calculated from top-of-canopy reflectance, provided in NRT in the form of 10-day composites. The intermediate alternative is an upgrade of the minimum component using long-term product archives based on inter-operable, multi-platform products. The objective of this alternative is guaranteeing, at least, a product archive of at least 15 years, beyond the end life cycle of remote sensing platforms. This long-term archive is necessary when using the products to detect crop growth anomalies (Baruth et al., 2006) against an average year or as yield predictor (López-Lozano et al., 2015). A product example within this intermediate level are the NRT Copernicus land biophysical products (e.g. fAPAR, LAI, see Verger et al., 2014), which use reflectance data from SPOT-VEGETATION and PROBA-V platforms since 1998 until present. Nevertheless, this level may require an extra cost compared to the basic level, as it comprises the development of inter-calibration algorithms for biophysical products in case of eventual
discontinuities of the existing products (e.g. PROBA-V end on life). The advanced technical solution adds to the intermediate level the use of high-resolution remote sensing products (e.g. Spot-XS, Landsat 8, or Sentinel 2) on a selection of agricultural areas (approximately 150 km per 150 km each area) of high relevance for the agricultural production in every geographical area. No historical archive of high-resolution images will be constructed, and the images will be used for crop-specific monitoring during the growing season. In these selected areas high and low resolution products will be used synergistically to extract valuable crop-specific information such as actual phenology (Zheng et al., 2016; Liu et al., 2016), or crop yield (e.g. Lobell et al., 2003). To achieve this, this advanced solution comprises as well the development of an operational algorithm to estimate biophysical products (LAI, fAPAR) from high-resolution top-of-canopy reflectance images.

The technical solutions for the crop growth modelling component differ in both, the nature of the models used and the quality of the data used to calibrate and run the modelling solutions. The minimum alternative consists in using simple, easy to parameterize vegetation growth models, with parameter values taken from scientific literature. Examples of such simple models can be found in Lee et al. (2003) or Duchemin et al. (2008). These simple models will be run spatially over arable land areas. In the intermediate and advanced alternatives crop growth models, implemented in the BioMA platform (e.g. WOFOST, CropSyst, STICS, WARM, see Section 2.2), will be used to simulate the growth of individual crops. Crop growth models have the advantage against simple vegetation models of a more detailed simulation of the different processes: light interception, carbon assimilation, biomass partitioning, evapotranspiration, etc. In the intermediate implementation the calibration of these models will be performed based on agro-nomic data extracted from scientific literature. Soil properties computed from global soil datasets (e.g. Batjes, 2008) will be used as input for the soil water balance model. Additionally, local agro-management data including irrigated/rainfed arable land and single/multiple cropping distribution are collected enabling models to simulate crop growth under realistic agro-management conditions. In the advanced solution, the calibration of crop models is conducted using field information collected from local experiments, similar to the work of Boons - Prins (1993) for the European MCYFS. Detailed soil datasets will be used as input for soil water balance simulations, and local data on fertilizing practices and possible pressure of pest and diseases will be collected as well to improve the reliability of crop model runs.

In the statistical component, the three technical solutions constitute a gradient in the sophistication of the statistical methods derived for crop yield forecasting, and the quality of the statistical data archive (Table 2). In the minimum solution, the statistical methods used are the simplest ones: trend analysis and linear regressions, applied at national scale. In the intermediate solution, multiple linear correlation with cross-validation and similarity analysis are incorporated to the statistical methods. Moreover, the scale of analysis is sub-national (GAUL 1) and requires a historical archive of sub-national statistics. The European MCYFS currently includes components of these minimum and intermediate solutions (Genovese and Bettio, 2004). The advanced alternative includes collection of statistics at GAUL 2 administrative level (e.g. equivalent to counties in the US or départements in France) and the assimilation of remote sensing data biophysical parameters into crop models (e.g. de Wit and van Diepen, 2007; Dorigo et al., 2007). Although data assimilation is not, properly speaking, a statistical method, it was decided to include them in the statistical tools component as it constitutes a post-processing of data from crop modelling and remote sensing components.

The composition of the analysis team is a crucial element in the crop monitoring system. As mentioned in Section 2.2, the number, profiles and skills of the team members determine the quality and detail of the monitoring analysis; the periodicity of the system outputs (bulletins, meteorological alert reports, ad-hoc analysis, etc.); but also the possible contribution of the team in sharing the outputs from the system

-analysis, forecast, methodologies, etc.- within international initiatives (e.g. AMIS, GEOGLAM,<fnref n="1" /> or MedAmin<fnref n="2" />). In the minimum and intermediate solutions for this component, teams are composed of small/average number of people, with technical profiles but no specialization on local agro-climatic conditions in the GLOBCAST geographical areas. The advanced solution is equivalent to the staff resources in the European MCYFS, with a fully-fledged team including analysts with background and expertise in the countries covered, thus favouring the interaction with local partners from the different geographic areas of the world covered and improving, theoretically, the quality of the system outputs.

The technical alternatives for the area estimation component are established depending on the additional site information that can be collected to correct the bias inherent of area estimates based solely on digital classification (Gallego, 2004). The minimum solution considers only digital classification, with no correction from local sampling. The intermediate alternative includes data from economic efficient sampling surveys “along-the road” as described by Gallego et al. (2014). Finally, in the advanced approach an optimal sampling protocol will be followed to avoid possible over-or under-representation of crops when using an along-the-road sampling.

### 3. Framework to identify the cost-efficient realizations for a MCYFS-like system in the GLOBCAST regions

In this chapter we present the framework to evaluate the cost-efficiency of the possible realizations of the MCYFS-like system, which are the result of the different combinations of technical solutions for the six components of the system. That evaluation is conducted separately for each individual country of the GLOBCAST study: Russia, Belarus, Ukraine, Kazakhstan, India, China, Australia, Argentina, Brazil, the United States of America and Canada. The crops to be forecasted are the most relevant grains -wheat, barley, maize, rice and soybean- and sugarcane, specifically for Brazil.

The efficiency, or performance, of a system is measured as the level of attainment -established through a score- for each of the six objectives explained in Section 2.1. The contribution of every technical solution to the system performance is first established attributing an attainment score to every individual solution. This score is an a priori assessment of the technical solutions, based on expert judgement. Then, the overall performance for any given system is retrieved combining the scores of the individual solutions composing it, according to their actual interaction in the MCYFS, explained in Section 2.2.

An automatic algorithm is then applied to evaluate all the possible realizations of the MCYFS-like systems, to retrieve their performance and costs, to include the effect of risks in implementing some solutions in specific countries, and to identify the cost-efficient options.

#### 3.1. Attainment levels of the system objectives

The definition of the scores ranging from 5 (highest) to 1 (lowest) used to qualify the attainment for each one of the MCYFS-like system objectives is given in Table 3. These scores are attributed to the technical solutions by the team of analysts based on their expert judgement. They represent an a priori indication of their performance, and their definitions have been kept rather general and intuitive.

In Objectives 1 -alert warning- and 2 -qualitative assessment of crop conditions- the scores are mostly describing precision in the identification of meteorological and crop growth anomalies (e.g. identifying major issues, position within quartiles, above or below the average), but also the accuracy of that identification (high or medium reliability). By contrast, Objectives 3 and 4 -crop yield forecasting and

<footnote n="1">¹ See http://geoglam.org/index.php/en/</footnote>

<footnote n="2">² See http://www.med-amin.org/en/</footnote>
<caption>**Table 3**</caption>

Scores used to evaluate the attainment of the different objectives of the monitoring system. Capital letters indicate the components involved in the attainment of each objective: A meteorological data; B remote sensing; C crop modelling platform; D statistical data and tools; E analysis tea; F area estimation.

<table><thead><tr><th>Objectives/ Scores</th><th>1. Alert warning on crop development</th><th>2. Qualitative analysis of crop development</th><th>3. Quantitative forecast of crop yield</th><th>4. Quantitative estimation of crop area</th><th>5. Bulletin production</th><th>6. Accessibility to information for external users</th></tr><tr><th>Components involved</th><th>A, B, C and E</th><th>A, B, C, and E</th><th>A, B, C, D and E</th><th>F</th><th>D</th><th>-</th></tr></thead><tbody><tr><td>5</td><td>Detection of major issues concerning possible crop abiotic stress with high reliability in near-real time.</td><td>Able to rank the current season with high reliability in the actual quartile distribution.</td><td>Yield forecasts with significantly better accuracy than the average inter-annual variability, both at national and sub-national levels.</td><td>Inter-annual changes of crop acreage, with significantly better accuracy than the average inter-annual change, both at national and sub-national levels.</td><td>Detailed crop and region-specific analysis of meteorological conditions and crop conditions + crop yield forecast + crop area estimation + sowing conditions for the new season.</td><td>Distribution to external users of the bulletins along the season and at the end of the season with near-real time access to data from externals through a dedicated web application.</td></tr><tr><td>4</td><td>Detection of major issues concerning possible crop abiotic stress with medium reliability in near-real time.</td><td>Able to rank the current season with medium reliability in the actual quartile distribution.</td><td>Yield forecasts with significantly better accuracy than the average inter-annual variability at national level and with accuracy comparable to the average inter-annual variability at sub-national level for the main production regions.</td><td>Inter-annual changes of crop acreage, with accuracy significantly better than the average inter-annual change at national level and with accuracy comparable to the average inter-annual change at sub-national level for the main production regions.</td><td>Detailed crop and region-specific analysis of overall crop conditions + crop yield forecast + crop area estimation + sowing conditions for the new season.</td><td>Distribution to external users of the bulletins along the season and at the end of the season with off-line external access to data products through a dedicated web application.</td></tr><tr><td>3</td><td>Detection of extreme events that will produce severe damages on crop yield/development with high reliability.</td><td>Able to place the current season with high reliability below or above the actual average season.</td><td>Able to estimate the inter-annual changes of crop yields, with accuracy comparable to the average inter-annual variability at national level. Valid estimations can be produced regarding specific areas where strong variations happen.</td><td>Able to estimate the inter-annual changes of crop acreage, with accuracy comparable to the average inter-annual change at national level. Valid estimations can be produced regarding specific areas where strong changes happen.</td><td>Global analysis of overall crop conditions + crop yield forecast + crop area estimation + sowing conditions for the new season.</td><td>Distribution to external users of the bulletins at the end of the season with external access to data products upon request.</td></tr><tr><td>2</td><td>Detection of only extreme events that would produce severe damages on crop yield/development with medium reliability.</td><td>Able to place the current season with medium reliability below or above the average season.</td><td>Forecasts able to provide indications about major changes in crop yields.</td><td>Crop acreage analysis able to provide indications about major changes in specific areas.</td><td>Global analysis of overall crop conditions + crop yield forecast + crop area estimation.</td><td>Distribution to external users of the bulletins at the end of the season upon request and access to data products.</td></tr><tr><td>1</td><td>No reliable information can be retrieved.</td><td>No reliable information about crop development can be produced.</td><td>No indications can be expected, except if exceptionally strong changes occur.</td><td>No indications can be expected, except if exceptionally strong changes occur.</td><td>Crop yield forecast + crop area estimation.</td><td>No external distribution is possible.</td></tr></tbody></table>

area estimation– are both quantitative and, the scores make reference to the accuracy of the figures produced, considered in relative terms to the inter-annual variability or the inter-annual change of yields and areas. The accuracy for the different scales of analysis is also included in the score, assuming that at national level the accuracy of quantitative figures is higher than at sub-national level, as yield and area variance decreases with spatial scale (Górski and Górska, 2003).

The attainment of Objective 5 (bulletin production) is about the extent and contents of the bulletins to publish and the spatial level of reporting –national or sub-national–, which could be highly relevant in large countries (e.g. Russia, China, India). The periodicity of the bulletins is assumed the same for all scores. Finally, in Objective 6 (accessibility of information to external users) the attainment scores are established based on the IT infrastructure that could give access to the information to external users, ranging from a software application giving access in real-real time to the system indicators to a situation in which no access is given to external users.

## 3.2. Evaluation of the individual technical solutions

The individual contribution of the technical alternatives proposed in Section 2.3 (minimum, intermediate, advanced) to the performance of the overall system with regard to the objectives (Table 1) is evaluated assigning to each one of them an attainment score (Table 3). That contribution is assumed to depend exclusively on the individual performance of the solution and the role of that solution in the MCYFS-like system workflow. Therefore, the score for a technical solution is considered independent of the geographical context where it is implemented.

The scores have been attributed by the team of analysts of the European MCYFS in 2011, based on their expert judgement and

experience contributing to the European Bulletins. When evaluating each solution for a given component, the analysts were asked to attribute the score reflecting the performance of a system where this solution is implemented with the advanced solution the other components. For instance, to evaluate the technical minimum solution of the meteorological data component in the achievement of objective 3 (quantitative yield forecast) the analyst attributes the score based on the question: What is the overall accuracy to forecast yields of a system built with ECMWF weather data in the meteorological component (Table 2) and the advanced solution in all the remaining components?

The components involved in the attainment score of each of the six objectives are given in Table 3. For objectives 1–3 the remote sensing component produces independent indicators to those produced by the interaction of the meteorological and crop modelling components (Fig. 2), and is complementarily used in the MCYFS for qualitative and quantitative analysis. Therefore, the scores for the remote sensing solutions were assigned under the hypothesis that both, meteorological and crop modelling components do not exist in the system. Similarly, when scoring the solutions of the meteorological and crop modelling components, the analysts had to assume that no remote sensing component exists.

The analysts had also to establish a risk level for each technical solution to express concerns about the feasibility to build that solution. These concerns are due to the unavailability of necessary data/re sources in some specific countries, and thus the analysts evaluated the risk of all technical solutions in every country covered by GLOBCAST, according to the following scale:

* No risks. The data/information can be collected without major constraints.

* Warning. Data is not easily available, and acquisition may need
important efforts (e.g. economic).

* High risk. This data is not likely to be available, or is not reliable.

The replies from all the MCYFS Europe analysts on the individual scores and risks were collected, and the most frequent score and risk per component and country was selected.

In the crop modelling and statistical components, the analysts attributed the attainment scores and risks to the sub-components (Table 2): the crop models used –including agronomic data for model calibration–, the soil data available to simulate soil water balance, the additional local crop management data, the statistical methods, and the statistical database. Considering sub-components in these two cases is important since, in some countries a high risk may be attributed only to one sub-component (e.g. experimental field data, or reliable crop statistics, are not accessible), but it can jeopardize the performance of the whole component and eventually the system.

The scores for objectives 1–3 are attributed in different moments of the cropping season (e.g. at sowing, flowering, grain filling, harvest). In the case of area estimation (Objective 4) scores are attributed early in the season and late in the season. The performance of solutions is expected to increase as the crop growing season progresses and, hence, also the attainment score should increase. When evaluating the overall system attainment, only the scores at the latest moment (e.g. harvest, late in the season) are used, but the other scores are shown (see Section 4.1) for informative purposes.

### 3.3. Cost estimation

Costs are estimated for every technical solution (Table 2) under the financial rules governing the European Commission as of 2011. Four categories where considered: data acquisition; staff; software and IT infrastructure; and contracted costs. The costs are established separately for the set-up (una tantum) and operational phases (yearly cost, including maintenance).

Data acquisition costs are calculated directly from public chart prices when available (e.g. subscription to ECMWF forecast data in NRT, acquisition of very high resolution satellite imagery). When price lists are not publicly available (e.g. statistical data for some countries, station weather data in local national networks) these costs are estimated by expert judgement from unit costs (e.g. costs per station, per dataset) remunerated in the operational MCYFS in Europe for data acquisition.

Software (e.g. database server software, remote sensing programs, GIS packages, mathematical applications and IT development environments) and hardware (workstations, FTP servers, etc.) are taken based on the actual market prices. Additionally, information storage and backup devices are estimated from the amount of hard disk space necessary for the input data and output products (meteorological data, satellite imagery, crop modelling products at different spatial units) and assuming a yearly cost per terabyte of information, derived from the actual expenses of the MCYFS system.

The investment needed in staff is based on current fares for the European Commission statutory staff (both permanent and temporary) and external (IT consultants).

Contracted costs are for those tasks that we recommend to outsource to external tenderers: e.g. NRT processing of low-resolution remote sensing products; pre-processing of weather data; or field studies to collect experimental data for the calibration of crop models.

The absolute costs of most technical solutions vary from one country to another, due to the total extension to cover, how the arable land is distributed, the number of crops to monitor, etc. When presenting the results of the cost-efficiency analysis in chapter 4 costs are expressed in relative units (r.u.), and are calculated by dividing absolute costs of a technical solution by the costs of the baseline system set-up (see Section 3.4.2) of the geographical zones. Reporting the costs in r.u. and not in absolute numbers permits, to discuss the cost figures regardless of the

geographical area –relative units tend to be stable from one area to another– while maintaining the proportionality among the technical solutions.

### 3.4. Algorithm for evaluating system performance and selection of the cost-efficient ones

An automatic algorithm has been specifically programmed in Matlab (Mathworks Inc., US) to run the cost-efficiency analysis of the MCYFS-like system for the different countries covered by GLOBCAST. The algorithm first computes the overall attainment score in the six objectives for all the possible systems (a total of 729 possible combinations of the technical solutions for the system components). Then, a set of rules is applied taking into account the system attainment scores and costs to identify the cost-efficient ones.

#### 3.4.1. Computation of system attainment score

The overall attainment score of any given system for a specific objective, $S_o$ where $o$ is the objective number, is computed from the scores $S_o, c=t$ and risks $R_{c=t}$ of the technical solution $t$ adopted for all the components $c$ involved –see Table 3– in objective $o$. Both $S_o, c=t$ and $R_{c=t}$ have been established by expert judgement from the MCYFS analysts in Section 3.2. An overview of the computation of $S_o$ is shown in Fig. 3.

In the first step of the computation process, the initial component scores $S_o, c=t$, are filtered by the risks: when the risk for a solution $R_{c=t}$ has been evaluated high, the score of that component is downgraded to the most advanced solution for that component $R_{c=max(t)}$ where the risk is not high. For instance, if the advanced solution for a component has a high risk, it will be downgraded to the intermediate solution if the risk for that is “warning” or no risk, otherwise it will be downgraded to the minimum solution.

The resulting scores $S_o, c$ for all components of the system are then used to compute the total system score $S_o$ for all the six objectives $o = 1, ..., 6$. The formulae to derive $S_o$ from $S_o, c$ shown in Fig. 3 try to reproduce the actual interaction of the components in the MCYFS workflow (2.2) to the achievable attainment of each objective. In objectives 1, 2 and 3, where more than one objective is involved, the nature of that interaction is twofold: either constraining –identified with a minus sign in Fig. 3– or synergistic –plus sign in Fig. 3.

In a constraining interaction, the component with the minimum score of the interaction acts as a bottleneck for all the others, and the resulting score is the minimum of the components interacting. This happens, for instance, when meteorological data and crop modelling interact in the MCYFS for crop monitoring and yield forecasting. If the meteorological data is not reliable, it will limit the performance given by the crop modelling –using meteorological data as an input– no matter how accurately the crop model has been calibrated and the quality of additional soil and management information, and vice versa. Something similar can be expected when the analysis team and the statistical component interact with the outputs from the meteorological, crop modelling and remote sensing components to produce reliable crop monitoring analysis and yield forecasts (Fig. 3).

A synergy occurs with the result of the interaction between the crop modelling and meteorological components, on one side, and the remote sensing, on the other side (formulae with the plus sign in Fig. 3). As crop modelling and remote sensing are used complementarily in the MCYFS for growth monitoring and yield forecasting (objectives 1–3), the performance of the system is not affected if only one of the two components has a low reliability. Indeed, the resulting score of the interaction may be higher than the two individual scores as a result of the synergy: it will increase to 4 or 5 if both scores are 3 or 4, respectively. For scores below 3 no synergy between components is expected.

The scores of objectives 4 and 5 (area estimation and bulletin production) are directly given by the area estimation and analysis team
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Schema to evaluate the overall system attainment $S_o$ of the objectives $o = 1,..., 5$, from individual score $S_o, c=t$ for solution $t = (\text{minimum, intermediate or advanced})$ of components $c = A, ..., F$, and risks $R_{c=t}$.</figcaption></figure>

performance, not interacting with other components. Finally, objective 6 (accessibility to external users) has been not included in the schema, as the existing tools in the MCYFS Europe (AGRI4CAST toolbox, http://agri4cast.jrc.ec.europa.eu/) are considered sufficient for our purpose to reach the maximum score, and no specific component for this objective has been introduced ad hoc.

### 3.4.2. Selection of cost-efficient systems

For each of the 729 possible systems, costs were calculated summing individual cost estimations per component and technical solution. When a component or sub-component has to be downgraded due to a high risk (see 3.4.1) the costs were adapted accordingly. Three rules were sequentially applied to select the cost-efficient systems:

* The overall attainment score obtained for the objectives 3, 4 and 5 –objectives with the highest priority for DG-Agriculture, as mentioned in Section 2.1– should be at least 3 (see Table 3 for details). If the score is below that threshold, the system is not considered as performing, and thus it is not cost-efficient.

* For those systems with identical scores for the objectives 3, 4 and 5 only the least expensive solution is selected. This rule eliminates systems less efficient from an economical point of view, identifying the ones with the most effective components (those that increase overall score with lower costs).

* From the remaining systems (normally 6–10, depending on the GLOBCAST area considered), three cost-efficient systems were selected from the minimum, median, and maximum sum of the scores for objectives 3, 4 and 5. These three systems are named baseline,
<caption>**Table 4**</caption>

Attainment scores given by the analysts to all the technical solutions of the MCYFS-like system components. The underlined scores (objectives with high priority) were those used in the selection of the cost-efficient systems. Solutions with a high risk in, at least, one country of the geographical areas covered are marked with a red box.

<table><thead><tr><th rowspan="3">MCYFS-like system components</th><th rowspan="3">Objectives</th><th colspan="10">1. Alert warning on crop development</th><th colspan="3">2. Qualitative analysis of crop development</th><th colspan="3">3. Quantitative yield forecast</th><th rowspan="2">4. Crop area</th><th rowspan="2">5. Bulletin production</th><th rowspan="2">6. Acessibility to external users</th></tr><tr><th>start of the season</th><th>sowing/planting</th><th>vegegative development</th><th>reproductive stage</th><th>grain filling</th><th>harvest</th><th>sowing/planting</th><th>vegegative development</th><th>reproductive stage</th><th>grain filling</th><th>harvest</th><th>vegegative development</th><th>reproductive stage</th><th>grain filling</th><th>harvest</th></tr><tr><th>technical solution</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th><th>4</th></tr></thead><tbody><tr><td rowspan="3">A. Meteo data</td><td>min.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td>int.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td>adv.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">B. Remote Sensing</td><td>min.</td><td>1</td><td>2</td><td>3</td><td>3</td><td>3</td><td>3</td><td>1</td><td>2</td><td>3</td><td>3</td><td>3</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>int.</td><td>1</td><td>2</td><td>3</td><td>4</td><td>4</td><td>4</td><td>2</td><td>3</td><td>4</td><td>4</td><td>4</td><td>2</td><td>2</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>adv.</td><td>1</td><td>3</td><td>5</td><td>5</td><td>5</td><td>5</td><td>2</td><td>3</td><td>4</td><td>5</td><td>5</td><td>2</td><td>2</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td rowspan="3">C. Crop modelling</td><td>min.</td><td>1</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>1</td><td>1</td><td>2</td><td>2</td><td>2</td><td>1</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>int.</td><td>2</td><td>3</td><td>3</td><td>3</td><td>3</td><td>4</td><td>1</td><td>1</td><td>2</td><td>3</td><td>4</td><td>4</td><td>2</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>adv.</td><td>2</td><td>3</td><td>4</td><td>4</td><td>4</td><td>4</td><td>1</td><td>3</td><td>4</td><td>5</td><td>5</td><td>2</td><td>3</td><td>4</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">Soil data</td><td>min.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>int.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>3</td><td>3</td><td>4</td><td>4</td><td>4</td><td>2</td><td>3</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>adv.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>3</td><td>4</td><td>5</td><td>5</td><td>5</td><td>3</td><td>4</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">Additional crop management data</td><td>min.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>int.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>2</td><td>3</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>adv.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>2</td><td>4</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">Yield estimation</td><td>min.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>int.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td>adv.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">D. Statistical tools</td><td>min.</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>int.</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>adv.</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td rowspan="3">E. Analysis team</td><td>min.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>int.</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>adv.</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td><td>5</td></tr><tr><td rowspan="3">F. Crop area</td><td>min.</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>int.</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>adv.</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr></tbody></table>

average performing, and high performing systems.

## 4. Results and discussion: technical solutions and expected performances of cost-efficient crop monitoring systems

### 4.1. Contribution of individual technical solutions to objectives attainment and associated risks

Table 4 shows the scores attributed by the analysts of the European MCYFS to the technical solutions proposed for all the six system components. Overall, the analysts are confident with the use of weather data from reanalysis products in crop models, and estimate a low impact on the overall attainment of the system for crop monitoring and yield forecasting (minimum solution of meteorological component has a score of 4 for objectives 1–3). Nevertheless, one of the known main limitations of reanalysis products are the uncertainties in precipitation from convective events (Lorenz and Kunstmann, 2012). This may actually lead to significant differences in the performances of reanalysis products in different areas of the world (e.g. Ceglar et al., 2016; de Leeuw et al., 2015; Peña-Arancibia et al., 2013) where convective precipitation prevails. The incorporation of observed weather data would help to improve the system performance, as both the intermediate and the advanced solution received a score of 5. The analysts considered that incorporating weather observations with open access as such from the GHNC (intermediate technical solution) are already sufficient to reach the maximum attainment score of the component. This is reasonable in many geographical regions of interest for GLOB-CAST like North America, Australia, south of Brazil and Eurasia, where

the number of available stations in the GHNC is rather high. Nevertheless, in some other countries like Argentina, Kazakhstan or China the density of stations included in that archive is lower, and using exclusively GHNC may lead to inaccuracies when describing spatial precipitations fields.

The technical solutions for both the crop modelling and remote sensing components are those, according to the analysts, with the highest impact on the performance of the system for crop monitoring (objectives 1 and 2) and, especially, for yield forecasting (objective 3). Only when calibrated crop models are used –the intermediate solution– the system performance for these objectives reaches the minimum required score of 3, even if models are calibrated based on data from a literature review. For qualitative analysis–objective 1 and 2–upgrading to advanced technical solution results in a performance increase due to the improvement in the model calibration –using local experimental data, in the advanced solution– agreeing with the results of Palosuo et al., (2011). However, to improve crop models performance for quantitative yield forecasting (objective 3) the analysts considered necessary also local information on soil and crop management data, in line with van Ittersum et al. (2013) when reviewing different works on crop models to analyse yield gaps.

The analysts considered that the use of sensor-specific low resolution products–minimum solution– in the remote sensing component are a suitable solution for Objectives 1 and 2, as vegetation indices are, indeed, actual observations of vegetation status permitting to detect, to some extent, crop growth anomalies and extreme events (e.g. Wu et al., 2014; Rojas et al., 2015). However, that solution would not reach a minimum score of 3 for crop yield forecasting. According to the
analysts, cross-platform products with a long-term archive –intermediate technical solution– are necessary to reach a sufficient performance when used as yield predictor. Some studies have proven that these products can be successfully used for crop yield forecasting at regional level (e.g. Balaghi et al., 2008; Becker-Reshef et al., 2010; Kogan et al., 2013 or Johnson, 2014). Nevertheless, biophysical products from remote sensing seem to be reliable as yield predictors in agro-climatic conditions where water availability is the main factor determining inter-annual yield variability, but do not perform well in humid regions where yields are not fully explained by changes in green leaf area (López-Lozano et al., 2015). The use of high and low-resolution sensors synergistically –advanced solution– would increase the attainment score of yield forecasting to 4, as the introduction of high resolution imagery is necessary in highly fragmented agricultural landscapes with small field sizes for crop-specific analysis (Duveiller and Defourny, 2010).

The solutions proposed to build the statistical component do not impose severe constraints to the performance of the system on crop yield forecasting as all the three solutions received a score of, at least, 3. Sub-national yield forecasting is, however, critical in large countries with a wide variability of agro-climatic conditions to produce a reliable analysis and thus the attainment score improves from 3 to 5 (Table 4) if the intermediate solutions are implemented in the component. The MCYFS is a decision support system where the role of the analysis team is central (Section 2.2) in analysing the convergence/divergences of the different indicators, identifying valid evidences, and producing quality outputs. Only with a sufficient number of analysts with specialized profiles (intermediate solution) the reliability of the system outputs, especially on quantitative yield forecast, reaches the minimum required performance. That is essential in large countries, where sub-national analysis is needed but it requires a high volume of work to be conducted in a relatively short period –no > 10 days– as otherwise the analysis would be outdated.

Only those technical solutions of the area estimation component where digital classification is supported by field survey –intermediate and advanced– are able to produce a satisfactory output, in line with recommendations from Gallego et al. (2010). Selecting the advanced technical solution –digital classification assisted by field surveys with an optimal sampling protocol– the component will only improve the attainment score to 4, meaning that area estimates will have an accuracy significantly better than the inter-annual change only at the national level, but will have a lower reliability for sub-national administrative units.

Red boxes in Table 4 highlight those technical solutions evaluated with a high risk in some countries. The advanced solution of the crop modelling component requires ad hoc experimental data to calibrate operationally crop, and local information on agricultural practices and detailed soil data. In some specific countries, this data is not available and, according to the analysts, was extremely difficult to collect. Reliable crop statistics at sub-national level –intermediate and advanced solutions of the statistical component– were not accessible to us in all the countries covered in GLOBCAST. Moreover, the possibility to conduct systematic field campaigns for area estimation in collaboration with local authorities was not always considered feasible, constraining the area component to the basic solution in certain countries. Finally, the advanced solution for the analysis team requires recruiting analysts with an expertise in the local agronomic conditions of the areas covered, which has been judged also not feasible for some particular countries.

## 4.2. Selected cost-efficient MCYFS-like systems and expected performance

The strategy to identify the cost-efficient alternatives to construct the crop monitoring and yield forecasting system presented in Chapter 3 tends to privilege similar technical solutions for the different countries covered by GLOBCAST. This is expected as the solution scores are

fixed for all geographical areas, and the relative costs of the different technical solutions are –as mentioned in Section 3.3– highly stable. Nevertheless, the country-specific risks identified constitute important differential factors that constrain the available technical solutions, and the feasible systems. According to this, the results of the cost-efficient analysis across the GLOBCAST countries can be summarized in three scenarios:

A. Countries with low constraints to access information, however some data is missing or not accessible but this has no major effect on the overall system performance. For these countries, the number of feasible systems is high, and the three systems selected (baseline, average performing and high performing) are purely based on cost-efficiency. This scenario will be used to illustrate the roadmap chosen when upgrading the baseline system to the maximum performing one based on the less expensive solutions.

B. Countries with moderate constraints, preventing some technical solutions to be realized. That reduces the number of different systems that can be actually built, and restricts, especially, the attainment score of the maximum performance system.

C. Countries with severe constraints to build and run system components, as resources needed to build critical technical solutions to guarantee a reliable system are not available. Due to these constraints, there is a very limited number of technical choices to build up the system, and their expected performance is substantially lower than the ones of scenarios A and B.

The composition and objective attainment scores of the cost-efficient systems for these three scenarios are shown in Fig. 4.

### 4.2.1. Roadmap towards a maximum performance system in countries where data availability is high (scenario a)

In the scenario A, only the access to local fertilizing treatments and pest and diseases pressure has been evaluated as high risk (advanced technical solution of the agro-management data sub-component), which do not produce any constraint to the overall system performance.

The baseline system for scenario A –shown in Fig. 4– is the least expensive one giving a score of 3 –lower limit established for a system to be considered as cost-efficient– in objectives 3, 4 and 5. This baseline system is built upon the minimum technical solution in the weather data and statistical components; plus the intermediate solutions for the crop modelling, remote sensing, analysis team, and area estimation components.

The improvement of that baseline system towards more performing ones, with higher attainment scores can be achieved upgrading different components of the system. Fig. 5 illustrates the roadmap to improve from the baseline system (attainment score of 3 in objective 3) to the average and high performing one (scores of 4 and 5, respectively), showing the alternatives existing for every successive upgrade and their cost. The roadmap in Fig. 5 is illustrated with the score of objective 3 – quantitative yield forecasting– as it is the only one involving more than one component among the high priority objectives.

To increase the score of the baseline system on crop yield forecasting and to reach the average performing system an upgrade of the analysis team and the statistical component is required, if not, according to Table 4, any other component upgrade would be inefficient constraining the score to 3. In addition, either the remote sensing or the crop modelling component have to be upgraded to the advanced solution. The cost-efficient choice is upgrading the crop modelling component (Fig. 5). Although the set-up costs of these upgrades are quite similar (about 20 r.u. in both components), the operating costs of upgrading the remote sensing component is much higher than improving the crop model calibration (60versus 0.5 r.u. for four years, respectively, of additional cost to the baseline system). Indeed, improving modelling calibration and soil information affects almost exclusively the costs of the set-up phase, as the additional costs to run in an
Scenario A. low constraints

Technical
solutions

<figure><img src="image_6.png" /></figure>

Systems composition and risks

System
scores

5. Bulletin
production

4. Crop Area
Estimation

3. Quantitative
Yield Forecast

<figure><img src="image_7.png" /></figure>

1. Alert Warning on
Crop Development

2. Qualitative Crop
Growth Analysis

Scenario B. moderate constraints

Technical
solutions

<figure><img src="image_8.png" /></figure>

Systems composition and risks

System
scores

5. Bulletin
production

4. Crop Area
Estimation

3. Quantitative
Yield Forecast

<figure><img src="image_9.png" /></figure>

1. Alert Warning on
Crop Development

2. Qualitative Crop
Growth Analysis

Scenario C. severe constraints

Technical
solutions

<figure><img src="image_10.png" /></figure>

Systems composition and risks

System
scores

5. Bulletin
production

4. Crop Area
Estimation

3. Quantitative
Yield Forecast

<figure><img src="image_11.png" /></figure>

1. Alert Warning on
Crop Development

2. Qualitative Crop
Growth Analysis

Baseline system

Average performing system

High-performing system

(caption on next page)
**Fig. 4.** Composition (left) and attainment scores (right) of the three cost-efficient systems proposed in the scenario A (low constraints to the technical solutions); B (moderate constraints); and C (severe constraints). Red horizontal line and text in the system composition describe a high risk of a given component that is blocking an upgrade to an improved technical solution. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

operational environment a better calibrated model are marginal. By contrast, the acquisition, processing and storage of high resolution imagery increases significantly the budget. In this average performing system the upgrade of the analysis team to the advanced solution–more staff with specialized profiles– implies that the operating costs of this component double (from 105 to 210 r.u. for four years), representing almost 75% of the system costs. This upgraded system, apart from improving the attainment of objective 3, permits to reach also a score of 5 for the objectives 1,2, and 5 (Fig. 4).

An increase in the attainment score from 4–average performing– to 5–high performing system– can be achieved in two different ways: either upgrading the meteorological component (alternative 2.1, in Fig. 5) or the remote sensing component (alternative 2.2). Upgrading the meteorological component to the intermediate solution–individual attainment score of 5, see Table 4– has a cascade effect, increasing the performance of the crop modelling component and, hence, in the absence of bottlenecks, improving the reliability of the system (Fig. 3). This first alternative is, however, very expensive (see Fig. 5), as acquiring, processing, interpolating and storing 30 years of daily weather data to construct the historical archive of weather data makes the budget for the set-up of the system to increase from 59 to 518 r.u. The upgrade of the remote sensing component to the advanced solution–individual score of 4 for objective 3, see Table 4– produces a synergy with the crop modelling component (Fig. 3), increasing the objective 3 score to 5. This second alternative is the cost-efficient one: even if yearly costs of the advanced remote sensing solution are slightly above those of upgrading the meteorological component, the expensive set-up for the latter does not compensate the extra operational cost at short-medium term.

Regarding crop area estimation, to achieve a score of 3 the intermediate solution of the crop area component is needed (Table 4). The maximum performance system must necessarily include the advanced solution to reach a score of 4. The area component is one of the most expensive ones (costs not included in Fig. 5), as it includes the acquisition of very high-resolution satellite imagery, research on efficient digital classification algorithms, and designing and contributing to ground area surveys. The set-up costs for the intermediate solution of this component are about 62 r.u. and 74 r.u. per year in the operational phase which is > 60% of the total baseline system budget. The cost for the advanced solution increases up to 86 r.u. and 98 r.u. per year, respectively, for the set-up and operational phases.

### 4.2.2. Possible cost-efficient system in countries where technical solutions are limited (scenarios B and C)

The baseline system in the scenario B is identical to the one in scenario A (Fig. 4). In the scenario B the constraints to upgrade the technical solutions are mainly affecting the crop model sub-component, as in these countries collecting the necessary local experimental data to improve crop model calibration has been judged not feasible by the analysts. This blocks the crop modelling component to the intermediate solution, and thus, the average performing system can be only achieved upgrading the statistical component, the analysis team, plus the remote sensing component, an upgrade equivalent to the alternative 1 shown in Fig. 5. Any further investment in other components would not produce any increase in the system attainment for objectives 1, 2 or 3. As a consequence, the high performing system in this scenario B is similar to the average performing, but upgrading the area estimation component to the advanced solution, which will increase the attainment score for objective 5.

In the scenario C, the performance of the system is severely

constrained by non-accessibility to reliable crop statistics at the sub-national level. In the MCYFS, the crop indicators are statistically related to official yields to produce the forecasts and, therefore, the accuracy of the crop yield forecasts depends largely on the quality of the statistics. Particularly in large countries with contrasting agro-climatic conditions the crop yield forecasts need to be done at sub-regional level rather than at country level to be reliable and the lack of reliable statistics at sub-national level acts as a bottleneck for the performance of whole system. Moreover, the recruitment of analysts with a high specialization on local conditions in some countries has been judged not feasible, thus limiting the technical solution of the analysis team component to the intermediate. These two constraints make any upgrade of the other system components inefficient, as they will not increase the performance of the system. On what regards area estimation, in scenario C conducting ground area surveys in collaboration with local authorities is not possible, which constraints the solution for this component to the minimum (digital classification not supported by field survey) and would not permit to reach reliable results (attainment score of 2, Table 4). In summary, due to these severe constraints, only a baseline system can be implemented in the scenario C.

The high risks attributed by the analysts to the technical solutions are somehow specific for an international organization like the EC, when aiming to run a crop monitoring system outside the EU. For instance, the feasibility of conducting field surveys for area estimation, acquiring experimental data or recruiting specialized experts would be much higher for a system run by a national institution for its territory. National bodies can more effectively to set-up any kind of operational activity for collecting field data, or involving technical/scientific partners that can help to solve the important challenges that implementing a MCYFS-like system may have. Establishing partnerships and links with national services may constitute an effective strategy to eventually mitigate these limitations, improving the feasibility of the MCYFS in many of the GLOBCAST areas.

## 5. Conclusions

This paper presents an evaluation framework followed to identify cost-efficient alternatives to extend the MARS Crop Yield Forecasting System (MCYFS), running in Europe since 1992, to other main producing areas of the world: Eastern European Neighbourhood, Asia, Australia, South America and North America. The extended systems follow the principles and components of the MCYFS Europe: a data rich system driven by expert knowledge (team of analysts), where indicators are produced by a meteorological data infrastructure, a remote sensing data infrastructure, and a crop modelling platform. They are analysed to report on crop growth and produce crop yield forecasts. An area estimation component –not existing in the European MCYFS– is added to produce independent crop area figures.

The framework designed evaluates the performance of the possible MCYFS-like system realizations against six defined objectives and their costs. Possible monitoring systems are based on a combination of different technical solutions for each of the MCYFS components. The performance is evaluated through a system of scores, representing the attainment level for each objective. An automatic algorithm calculates the attainment scores, the costs, introduces the effect of the risks to construct technical solutions and identifies the least expensive system yielding a satisfactory attainment score. The analysis produced three systems considered cost-efficient: a baseline system, with an adequate attainment score and reduced costs; and an average and a high performing system, which are cost-efficient upgrades from the baseline.
Attainment score
for crop yield
forecasting (obj. 3)

3

Attainment score
for crop yield
forecasting (obj. 3)

4

Attainment score
for crop yield
forecasting (obj. 3)

5

Baseline
system

composition

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

composition

costs (relative units)

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

Alternative 2.1:
Upgrading A

<figure><img src="image_18.png" /></figure>

composition

costs (relative units)

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

Alternative 2.2:
Upgrading B

Alternative 2:
Upgrading C+D+E

Alternative 1:
Upgrading B+D+E

Upgrade

(caption on next page)
**Fig. 5.** Roadmap to upgrade the baseline system (attainment score of 3) towards improved systems with attainment scores of 4 and 5 in the crop yield forecasting objective in the scenario A (low constraints to the technical solutions). Costs are expressed, separately for the set-up and operational (4 years) phases, in relative units to the baseline system set-up costs (100 r.u. = costs of baseline system set-up, including component F on area estimation).

However, the average and high performance system cannot be always achieved, as some technical solutions are not feasible in certain countries. According to an evaluation a priori of the possible risks, some necessary data and resources cannot be easily collected: experimental and agro-management data to improve the calibration and reliability of crop models; reliable sub-national crop statistics; experts on local conditions to recruit; and ground surveys for crop area estimation. These mentioned resources are may not always be accessible for an international organization like the EC, implementing a crop monitoring system outside its territory, but can be, in theory, achieved by national institutions interested in building such systems.

The results of the evaluation framework presented were implemented in successive phases of the GLOBCAST project, and the MCYFS extension to South America, India, China, Turkey, Ukraine, Russia and Kazakhstan started in 2012. In these cases the baseline system was implemented: reanalysis near-real-time products for the meteorological component; a crop modelling platform with crop models calibrated with existing data form scientific literature; global soil and arable land maps to run the model spatially; a remote sensing component based on low resolution biophysical products with a log-term archive; and a statistical component with simple methods -regression, trend analysis- applied at national/sub-national scale; and an analysis team with staff of different backgrounds on agronomy, remote sensing, cop modelling, etc. Only the crop area component has not been implemented for the time being. An overview of the implementation of the MCYFS in those countries and its results can be found in https://ec.europa.eu/agriculture/events/globcast-dissemination_en. The roadmap for system upgrades serves is serving a reference to further improvements in some of the mentioned countries. Bulletins on crop growth monitoring and yield forecasting were released in these countries since the second half of 2014 (see https://ec.europa.eu/jrc/en/mars/bulletins), and once this operational phase is consolidated -e.g. after a minimum of six or seven years- the suitability of the technical solutions adopted will have to be re-evaluated through a quality assessment of the yield forecasts produced.

After 25 years of the MCYFS in Europe the exercise presented in this paper was also a good opportunity to critically evaluate the European system, and study possible cost-efficient upgrades that may help to improve the system performance. The evaluation framework presented here could be adapted to perform that analysis systematically, and to depict a feasible roadmap for a system upgrade that incorporates recent technological developments. For instance, the accessibility to high resolution satellite data, with the open access data policy for Sentinel 2 imagery (https://sentinel.esa.int/web/sentinel/sentinel-data-access); opportunities to develop added-value NRT products that combine different data sources in new big data environments; or the contribution of recent initiatives on crowdsourcing and citizen science (Beza et al., 2017).

This framework can be also of interest to other global or national initiatives on near real time crop monitoring and yield forecasting. The basic principles to evaluate the possible technical solutions and the inclusion of costs when identifying the feasible ones are general ideas that can be applied to other system architectures, e.g. with different objectives, components, or interactions between components.

The results of the analysis presented are, by contrast, specific to the MCYFS architecture, particularly on what regards the contribution of the different technical solutions to the overall system performances. Moreover, the scores to the different technical solutions were given by the current MCYFS analysts based on their own experience working with the European MCYFS, and that introduces some subjectivity in the evaluation. However, there is no easy way to circumvent it in an

analysis a priori when planning a system implementation. Furthermore, the exercise was conducted in 2011 and some technical alternatives and their costs have evolved over time: e.g. open access high resolution imagery, decrease of data processing costs, adoption of transparency and open data access policies from local governmental bodies and research institutions to increase that availability. All this may introduce some differences in the results of the analysis if the exercise was done again nowadays.

## Acknowledgements

The GLOBCAST project preparatory phases 1-3 (2011-2015) were funded by the European Commission's DG Agriculture.

## References

* Arino, O., Gross, D., Ranera, F., Leroy, M., Bicheron, P., Brockman, C., Defourny, P., Vancutsem, C., Achard, F., Durieux, L., Bourg, L., Latham, J., Di Gregorio, A., Witt, R., Herold, M., Sambale, J., Plummer, S., Weber, J.-L., 2008. GlobCover: ESA service for global land cover from MERIS. In: International Geoscience and Remote Sensing Symposium (IGARSS), pp. 2412-2415.

* Balaghi, R., Tychon, B., Eereens, H., Jlibene, M., 2008. Empirical regression models using NDVI, rainfall and temperature data for the early prediction of wheat grain yields in Morocco. Int. J. Appl. Earth Obs. Geoinf. 10, 438-452. http://dx.doi.org/10.1016/j.jag.2006.12.001.

* Bartholomé, E., Belward, A.S., 2005. GLC2000: a new approach to global land cover mapping from Earth observation data. Int. J. Remote Sens. 26, 1959-1977. http://dx.doi.org/10.1080/01431160412331291297.

* Baruth, B., Royer, A., Klisch, A., Genovese, G., 2006. The use of remote sensing within the MARS crop yield monitoring system of the european commission. ISPRS Arch. 36.

* Batjes, N.H., 2008. ISRIC-WISE Harmonized Global Soil Profile Dataset. (Ver. 3.1).

* Becker-Reshef, I., Vermote, E., Lindeman, M., Justice, C., 2010. A generalized regression-based model for forecasting winter wheat yields in Kansas and Ukraine using MODIS data. Remote Sens. Environ. 114, 1312-1323.

* Beza, E., Steinke, J., van Etten, J., Reidsma, P., Fadda, C., Mittra, S., Mathur, P., Kooistra, L., 2017. What are the prospects for citizen science in agriculture? Evidence from three continents on motivation and mobile telephone use of resource-poor farmers. PLoS One 12. http://dx.doi.org/10.1371/journal.pone.0175700.

* Bojanowski, J.S., Vrieling, A., Skidmore, A.K., 2013. Calibration of solar radiation models for Europe using Meteoas second generation and weather station data. Agric. For. Meteorol. 176, 1-9. http://dx.doi.org/10.1016/j.agrformet.2013.03.005.

* Boons - Prins, E.R., 1993. Crop-Specific Simulation Parameters for Yield Forecasting Across the European Community, Simulation Reports/CABOT-TT; no. 32. CABOT-DLO [etc.], Wageningen.

* Brisson, M.L., Mary, Bruno, Nadine, Nicolas Beaudoin, 2009. Conceptual Basis, Formalisations and Parameterization of the Stics Crop Model. (Editions Quae.).

* Ceglar, A., Toreti, A., Balsamo, G., Kobayashi, S., 2016. Precipitation over monsoon Asia: a comparison of reanalyses and observations. J. Clim. 30, 465-476. http://dx.doi.org/10.1175/JCLI-D-16-0227.1.

* Confalonieri, R., Rosenmund, A.S., Baruth, B., 2009. An improved model to simulate rice yield. Agron. Sustain. Dev. 29, 463-474.

* de Leeuw, J., Methven, J., Blackburn, M., 2015. Evaluation of ERA-interim reanalysis precipitation products using England and Wales observations. Q. J. R. Meteorol. Soc. 141, 798-806. http://dx.doi.org/10.1002/qj.2395.

* de Wit, A.J.W., van Diepen, C.A., 2007. Crop model data assimilation with the ensemble Kalman filter for improving regional crop yield forecasts. Agric. For. Meteorol. 146, 38-56.

* Dee, D.P., Uppala, S.M., Simmons, A.J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M.A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A.C.M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A.J., Haimberger, L., Healy, S.B., Hersbach, H., Hólm, E.V., Isaksen, L., Källberg, P., Köhler, M., Matricardi, M., McNally, A.P., Monge-Sanz, B.M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N., Vitart, F., 2011. The ERA-interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137, 553-597. http://dx.doi.org/10.1002/qj.828.

* Dorigo, W.A., Zurita-Milla, R., de Wit, A.J.W., Brazile, J., Singh, R., Schaepman, M.E., 2007. A review on reflective remote sensing and data assimilation techniques for enhanced agroecosystem modeling. Int. J. Appl. Earth Obs. Geoinf. 9, 165-193.

* Duchemin, B., Maisongrande, P., Boulet, G., Benhadj, I., 2008. A simple algorithm for yield estimates: evaluation for semi-arid irrigated winter wheat monitored with green leaf area index. Environ. Model Softw. 23, 876-892. http://dx.doi.org/10.1016/j.envsoft.2007.10.003.

* Duveiller, G., Defourny, P., 2010. A conceptual framework to define the spatial resolution requirements for agricultural monitoring using remote sensing. Remote Sens. Environ. 114, 2637-2650.
Eurostat, 2015. Handbook for Annual Crop Statistics. Crop statistics working group.

Gallego, J., 2004. Remote sensing and land cover area estimation. Int. J. Remote Sens. 25, 3019–3047. http://dx.doi.org/10.1080/01431160310001619607.

Gallego, J., Craig, M., Michaelsen, J., Bossyns, B., Fritz, S., 2010. Best Practices for Crop Area Estimation with Remote Sensing. Publications Office of the European Union, Luxembourg.

Gallego, F.J., Kussul, N., Skakun, S., Kravchenko, O., Shelestov, A., Kussul, O., 2014. Efficiency assessment of using satellite data for crop area estimation in Ukraine. Int. J. Appl. Earth Obs. Geoinf. 29, 22–30. http://dx.doi.org/10.1016/j.jag.2013.12.013.

Methodology of the MARS crop yield forecasting system. In: Genovese, G., Bettio, M. (Eds.), Statistical Data Collection, Processing and Analysis. vol. 4 Office for the Official Publications of the European Communities, Luxembourg.

Genovese, G., Vignolles, C., Nègre, T., Passera, G., 2001. A methodology for a combined use of normalised difference vegetation index and CORINE land cover data for crop yield monitoring and forecasting. A case study on Spain. Agronomie 21, 91–111.

Gobron, N., Pinty, B., Verstraete, M.M., Taberner, M., 2002. VEGETATION - An Optimized FAPAR Algorithm - Theoretical Basis Document.
Górski, T., Górska, K., 2003. The effects of scale on crop yield variability. Agric. Syst. 78, 425–434.

Hoolst, R.V., Eerens, H., Haesen, D., Royer, A., Bydekerke, L., Rojas, O., Li, Y., Racionzer, P., 2016. FAO's AVHRR-based agricultural stress index system (ASIS) for global drought monitoring. Int. J. Remote Sens. 37, 418–439. http://dx.doi.org/10.1080/01431161.2015.1126378.

Johnson, D.M., 2014. An assessment of pre- and within-season remotely sensed variables for forecasting corn and soybean yields in the United States. Remote Sens. Environ. 141, 116–128. http://dx.doi.org/10.1016/j.rse.2013.10.027.

Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H., Miyaoka, K., Takahashi, K., 2015. The JRA-55 reanalysis: general specifications and basic characteristics. J. Meteorol. Soc. Jpn. Ser. II 93, 5–48. http://dx.doi.org/10.2151/imsj.2015.001.

Kogan, F., Kussul, N., Adamenko, T., Skakun, S., Kravchenko, O., Kryvobok, O., Shelestov, A., Kolotii, A., Kussul, O., Lavrenyuk, A., 2013. Winter wheat yield forecasting in Ukraine based on earth observation, meteorological data and biophysical models. Int. J. Appl. Earth Obs. Geoinf. 23, 192–203. http://dx.doi.org/10.1016/j.jag.2013.01.002.

Methodology of the MARS crop yield forecasting system. In: Kucera, L., Genovese, G. (Eds.), Agro-Meteorological Modelling, Processing and Analysis. vol. 2 Office for the Official Publications of the European Communities, Luxembourg.

Lee, J.H., Goudriaan, J., Challa, H., 2003. Using the Explinear growth equation for modelling crop growth in year-round cut chrysanthemum. Ann. Bot. 92, 697–708. http://dx.doi.org/10.1093/aob/mcg195.

Liu, S., Zhao, W., Shen, H., Zhang, L., 2016. Regional-scale winter wheat phenology monitoring using multisensor spatio-temporal fusion in a South Central China growing area. J. Appl. Rem. Sens. 10 (4) JARSC4 10, 046029. https://doi.org/10.1117/1.JRS.10.046029.

Lobell, D.B., Asner, G.P., Ortiz-Monasterio, J.I., Benning, T.L., 2003. Remote sensing of regional crop production in the Yaqui Valley, Mexico: estimates and uncertainties. Agric. Ecosyst. Environ. 94, 205–220.

López-Lozano, R., Duveiller, G., Seguini, L., Meroni, M., García-Condado, S., Hooker, J., Leo, O., Baruth, B., 2015. Towards regional grain yield forecasting with 1km-resolution EO biophysical products: strengths and limitations at pan-European level. Agric. For. Meteorol. 206, 12–32. http://dx.doi.org/10.1016/j.agrformet.2015.02.021.

Lorenz, K., Kunstmann, H., 2012. The hydrological cycle in three state-of-the-art re-analyses: Intercomparison and performance analysis. J. Hydrometeorol. 13, 1397–1420. http://dx.doi.org/10.1175/JHM-D-11-088.1.

Methodology of the MARS crop yield forecasting system. In: Micale, F., Genovese, G. (Eds.), Meteorological Data Collection, Processing and Analysis. vol. 1 Office for the Official Publications of the European Communities, Luxembourg.

Müeller, R., Seffrin, R., 2006. New methods and satellites: a program update on the NASS cropland data layer acreage program. Intl. Arch.Photogramm. Remote Sens. Spat. Inf. Sci. 36, 97–102.

Nieuwenhuis, G.J.A., de Wit, A.J.W., van Kraalingen, D.W.G., van Diepen, C.A., Boogaard, H.L., 2006. Monitoring crop growth conditions using the global water satisfaction index and remote sensing. In: Presented at the ISPRS Commission VII Mid-Term Symposium 2006 "Remote Sensing: From Pixels to Processes", pp. 684–687 (Ensechede).

Palosuo, T., Kersebaum, K.C., Angulo, C., Hlavinka, P., Moriondo, M., Olesen, J.E., Patil,

R.H., Ruget, F., Rumbaur, C., Takáč, J., Trnka, M., Bindi, M., Çaldığ, B., Ewert, F., Ferrise, R., Mirschel, W., Şaylan, L., Šiška, B., Rötter, R., 2011. Simulation of winter wheat yield and its variability in different climates of Europe: a comparison of eight crop growth models. Eur. J. Agron. 35, 103–114. http://dx.doi.org/10.1016/j.eja.2011.05.001.

Peña-Arancibia, J.L., Van, D., Renzullo, L.J., Mulligan, M., 2013. Evaluation of precipitation estimation accuracy in reanalyses, satellite products, and an ensemble method for regions in Australia and south and east Asia. J. Hydrometeorol. 14, 1323–1333. http://dx.doi.org/10.1175/JHM-D-12-0132.1.

Rienecker, M.M., Suarez, M.J., Gelaro, R., Todling, R., Bacmeister, J., Liu, E., Bosilovich, M.G., Schubert, S.D., Takacs, L., Kim, G.-K., Bloom, S., Chen, J., Collins, D., Conaty, A., da Silva, A., Gu, W., Joiner, J., Koster, R.D., Lucchesi, R., Molod, A., Owens, T., Pawson, S., Pegion, P., Redder, C.R., Reichle, R., Robertson, F.R., Ruddick, A.G., Sienkiewicz, M., Woollen, J., 2011. MERRA: NASA's modern-era retrospective analysis for research and applications. J. Clim. 24, 3624–3648. http://dx.doi.org/10.1175/JCLI-D-11-00015.1.

Rojas, O., Li, Y., Cumani, R., 2015. Understanding the Drought Impact of El Nino on the Global Agricultural Areas: An Assessment Using FAO's Agricultural Stress Index (ASI), Environment and Natural Management Series (FAO). Food and Agriculture Organization of the United Nations, Rome. http://dx.doi.org/10.13140/2.1.1868.3687.

Methodology of the MARS crop yield forecasting system. In: Royer, A., Genovese, G. (Eds.), Remote Sensing Information, Data Processing and Analysis. AgriFish Unit. vol. 3 Joint Research Centre of the European Commission, Ispra, Italy.

Singels, A., Jones, M., van den Berg, M., 2008. DSSAT v4.5 - Canegro Sugarcane Plant Module.

Stöckle, C.O., Donatelli, M., Nelson, R., 2003. CropSyst, a cropping systems simulation model. Eur. J. Agron. 18, 289–307.

Van Diepen, C.A., Wolf, J., Van Keulen, H., Rappoldt, C., 1989. WOFOST: a simulation model of crop production. Soil Use Manag. 5, 16–24.

van Ittersum, M.K., Cassman, K.G., Grassini, P., Wolf, J., Tittonell, P., Hochman, Z., 2013. Yield gap analysis with local to global relevance—a review. Field Crop Res. 143, 4–17. http://dx.doi.org/10.1016/j.fcr.2012.09.009.

Van Leeuwen, W.J.D., Hutchinson, C.F., Doorn, B., Sheffner, E., Kaupp, V.H., 2006. Integrated crop production observations and information system. In: International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3506–3508.

van Leeuwen, W., Hutchinson, C., Drake, S., Doorn, B., Kaupp, V., Haithcoat, T., Likholetov, V., Sheffner, E., Tralli, D., 2011. Benchmarking enhancements to a decision support system for global crop production assessments. Expert Syst. Appl. 38, 8054–8065. http://dx.doi.org/10.1016/j.eswa.2010.12.145.

Verger, A., Baret, F., Weiss, M., Kandasamy, S., Vermote, E., 2013. The CACAO method for smoothing, gap filling, and characterizing seasonal anomalies in satellite time series. IEEE Trans. Geosci. Remote Sens. 51, 1963–1972. http://dx.doi.org/10.1109/TGRS.2012.2228653.

Verger, A., Baret, F., Weiss, M., 2014. Near real-time vegetation monitoring at global scale. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 7, 3473–3481. http://dx.doi.org/10.1109/JSTARS.2014.2328632.

Verger, A., Baret, F., Weiss, M., Filella, I., Peñuelas, J., 2015. GEOCLIM: a global climatology of LAI, FAPAR, and FCOVER from VEGETATION observations for 1999–2010. Remote Sens. Environ. 166, 126–137. http://dx.doi.org/10.1016/j.rse.2015.05.027.

Vicente-Serrano, S.M., Beguería, S., López-Moreno, J.I., 2009. A multiscalar drought index sensitive to global warming: the standardized precipitation evapotranspiration index. J. Clim. 23, 1696–1718. http://dx.doi.org/10.1175/2009JCLI2909.1.

von Braun, J., 2008. Food and financial crises: implications for agriculture and the poor (No. 20). In: Food Policy Reports. International Food Policy Research Institute (IFPRI).

Weiss, M., Baret, F., Eerens, H., Swinnen, E., 2010. FAPAR over Europe for the past 29 years: a temporally consistent product derived from AVHRR and VEGETATION sensor. In: Proceeding of the Third AQRS Workshop, pp. 428–433.

Wu, B., Meng, J., Li, Q., Yan, N., Du, X., Zhang, M., 2014. Remote sensing-based global crop monitoring: experiences with China's cropwatch system. Int. J. Digit. Earth 7, 113–137. http://dx.doi.org/10.1080/17538947.2013.821185.

Zaman, H., Ivanic, M., Martin, W., 2011. Estimating the Short-Run Poverty Impacts of the 2010-11 Surge in Food Prices (No. WPS5633). The World Bank.

Zheng, Y., Wu, B., Zhang, M., Zeng, H., 2016. Crop phenology detection using high Spatio-temporal resolution data fused from SPOT5 and MODIS products. Sensors 16. http://dx.doi.org/10.3390/s16122099.