# An AgMIP framework for improved agricultural representation in integrated assessment models

Alex C Ruane, Cynthia Rosenzweig, Senthold Asseng, Kenneth J Boote, Joshua Elliott, Frank Ewert, James W Jones, Pierre Martre, Sonali P Mcdermid, Christoph Müller, et al.

► To cite this version:

Alex C Ruane, Cynthia Rosenzweig, Senthold Asseng, Kenneth J Boote, Joshua Elliott, et al. An AgMIP framework for improved agricultural representation in integrated assessment models. Environmental Research Letters, 2017, 12, 10.1088/1748-9326/aa8da6. hal-02624345

HAL Id: hal-02624345

https://hal.inrae.fr/hal-02624345

Submitted on 26 May 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ACCEPTED MANUSCRIPT • OPEN ACCESS

# An AgMIP framework for improved agricultural representation in IAMs

To cite this article before publication: Alex C Ruane et al 2017 *Environ. Res. Lett.* in press https://doi.org/10.1088/1748-9326/aa8da6

## Manuscript version: Accepted Manuscript

Accepted Manuscript is "the version of the article accepted for publication including all changes made as a result of the peer review process, and which may also include the addition to the article by IOP Publishing of a header, an article ID, a cover sheet and/or an 'Accepted Manuscript' watermark, but excluding any other editing, typesetting or other changes made by IOP Publishing and/or its licensors"

This Accepted Manuscript is © 2017 The Author(s). Published by IOP Publishing Ltd.

As the Version of Record of this article is going to be / has been published on a gold open access basis under a CC BY 3.0 licence, this Accepted Manuscript is available for reuse under a CC BY 3.0 licence immediately.

Everyone is permitted to use all or part of the original content in this article, provided that they adhere to all the terms of the licence https://creativecommons.org/licenses/by/3.0

Although reasonable endeavours have been taken to obtain all necessary permissions from third parties to include their copyrighted content within this article, their full citation and copyright line may not be present in this Accepted Manuscript version. Before using any content from this article, please refer to the Version of Record on IOPscience once published for full citation and copyright details, as permissions may be required. All third party content is fully copyright protected and is not published on a gold open access basis under a CC BY licence, unless that is specifically stated in the figure caption in the Version of Record.

View the article online for updates and enhancements.
LETTER • OPEN ACCESS

# An AgMIP framework for improved agricultural representation in integrated assessment models

To cite this article: Alex C Ruane et al 2017 *Environ. Res. Lett.* **12** 125003

View the [article online](#) for updates and enhancements.

## Related content

- *Implications of climate mitigation for future agricultural production*
Christoph Müller, Joshua Elliott, James Chryssanthacopoulos et al.

- *Assessing inter-sectoral climate change risks: the role of ISIMIP*
Cynthia Rosenzweig, Nigel W Arnell, Kristie L Ebi et al.

- *Simulated vs. empirical weather responsiveness of crop yields: US evidence and implications for the agricultural impacts of climate change*
Malcolm N Mistry, Ian Sue Wing and Enrica De Cian
# An AgMIP framework for improved agricultural representation in IAMs

Alex C. Ruane<fnref n="1" />

Cynthia Rosenzweig<fnref n="1" />

Senthold Asseng<fnref n="2" />

Kenneth J. Boote<fnref n="2" />

Joshua Elliott<fnref n="3" />

Frank Ewert<fnref n="4" />,<fnref n="5" />

James W. Jones<fnref n="2" />,<fnref n="6" />

Pierre Martre<fnref n="7" />

Sonali P. McDermid<fnref n="8" />

Christoph Müller<fnref n="9" />

Abigail Snyder<fnref n="10" />

Peter J. Thorburn<fnref n="11" />

Submitted to *Environmental Research Letters* Special Issue:
Focus on an Inter-method Comparison of Climate Change Impacts on Agriculture
September 6th, 2017

**Tweetable Abstract:** Improved agricultural sector representation within IAMs requires collaborative development and application of crop models across scales.

Corresponding Author:

Alex Ruane

NASA Goddard Institute for Space Studies

2880 Broadway

New York, NY 10025

alexander.c.ruane@nasa.gov

<footnote n="1">¹NASA Goddard Institute for Space Studies, New York, NY, USA</footnote>

<footnote n="2">²University of Florida, Agricultural and Biological Engineering, Gainesville, FL, USA</footnote>

<footnote n="3">³University of Chicago, Computation Institute, Chicago, IL, USA</footnote>

<footnote n="4">⁴University of Bonn, Bonn, Germany</footnote>

<footnote n="5">⁵Leibniz Center of Agricultural landscape Research (ZALF), Muencheberg, Germany</footnote>

<footnote n="6">⁶National Science Foundation, Arlington, VA, USA</footnote>

<footnote n="7">⁷UMR LEPSE, INRA, Montpellier SupAgro, Montpellier, France</footnote>

<footnote n="8">⁸New York University, New York, USA</footnote>

<footnote n="9">⁹Potsdam Institute for Climate Impact Research, Potsdam, Germany</footnote>

<footnote n="10">¹⁰Pacific Northwest National Laboratory, College Park, MD, USA</footnote>

<footnote n="11">¹¹Commonwealth Scientific and Industrial Research Organization, Brisbane, Australia</footnote>
Environmental Research Letters

<figure><img src="image_2.png" /></figure>

CrossMark

OPEN ACCESS

RECEIVED

3 March 2017

REVISED

6 September 2017

ACCEPTED FOR PUBLICATION

19 September 2017

PUBLISHED

24 November 2017

Original content from
this work may be used
under the terms of the
Creative Commons
Attribution 3.0 licence.

Any further distribution
of this work must
maintain attribution to
the author(s) and the
title of the work, journal
citation and DOI.

<figure><img src="image_3.png" /></figure>

LETTER

# An AgMIP framework for improved agricultural representation in integrated assessment models

Alex C Ruane<sup>1,11 Ⓓ</sup>, Cynthia Rosenzweig<sup>1 Ⓒ</sup>, Senthold Asseng<sup>2</sup>, Kenneth J Boote<sup>2</sup>, Joshua Elliott<sup>3</sup>, Frank Ewert<sup>4,5</sup>, James W Jones<sup>2</sup>, Pierre Martre<sup>6</sup>, Sonali P McDermid<sup>7</sup>, Christoph Müller<sup>8 Ⓒ</sup>, Abigail Snyder<sup>9</sup> and Peter J Thorburn<sup>10</sup>

1 NASA Goddard Institute for Space Studies, New York, NY, United States of America

2 University of Florida, Agricultural and Biological Engineering, Gainesville, FL, United States of America

3 University of Chicago, Computation Institute, Chicago, IL, United States of America

4 University of Bonn, Bonn, Germany

5 Leibniz Center of Agricultural landscape Research (ZALF), Muencheberg, Germany

6 UMR LEPSE, INRA, Montpellier SupAgro, Montpellier, France

7 New York University, New York, NY, United States of America

8 Potsdam Institute for Climate Impact Research, Potsdam, Germany

9 Pacific Northwest National Laboratory, College Park, MD, United States of America

10 Commonwealth Scientific and Industrial Research Organization, Brisbane, Australia

11 Author to whom any correspondence should be addressed.

E-mail: alexander.c.ruane@nasa.gov

**Keywords:** AgMIP, climate impacts, crop models, integrated assessment models, agricultural impacts, emulators, model development

## Abstract

Integrated assessment models (IAMs) hold great potential to assess how future agricultural systems will be shaped by socioeconomic development, technological innovation, and changing climate conditions. By coupling with climate and crop model emulators, IAMs have the potential to resolve important agricultural feedback loops and identify unintended consequences of socioeconomic development for agricultural systems. Here we propose a framework to develop robust representation of agricultural system responses within IAMs, linking downstream applications with model development and the coordinated evaluation of key climate responses from local to global scales. We survey the strengths and weaknesses of protocol-based assessments linked to the Agricultural Model Intercomparison and Improvement Project (AgMIP), each utilizing multiple sites and models to evaluate crop response to core climate changes including shifts in carbon dioxide concentration, temperature, and water availability, with some studies further exploring how climate responses are affected by nitrogen levels and adaptation in farm systems. Site-based studies with carefully calibrated models encompass the largest number of activities; however they are limited in their ability to capture the full range of global agricultural system diversity. Representative site networks provide more targeted response information than broadly-sampled networks, with limitations stemming from difficulties in covering the diversity of farming systems. Global gridded crop models provide comprehensive coverage, although with large challenges for calibration and quality control of inputs. Diversity in climate responses underscores that crop model emulators must distinguish between regions and farming system while recognizing model uncertainty. Finally, to bridge the gap between bottom-up and top-down approaches we recommend the deployment of a hybrid climate response system employing a representative network of sites to bias-correct comprehensive gridded simulations, opening the door to accelerated development and a broad range of applications.

## 1. Introduction

Integrated assessment models (IAMs) examine the interactions between human systems and the natural

environment. IAMs thus explore how societal changes, such as global policies, population growth, socioeconomic development, greenhouse gas emissions, and technological advances affect land, air, and water
Abstract

Integrated assessment models (IAMs) hold great potential to assess how future agricultural
systems will be shaped by socioeconomic development, technological innovation, and
changing climate conditions. By coupling with climate and crop model emulators, IAMs
have the potential to resolve important agricultural feedback loops and identify unintended
consequences of socioeconomic development for agricultural systems. Here we propose a
framework to develop robust representation of agricultural system responses within IAMs,
linking downstream applications with model development and the coordinated evaluation
of key climate responses from local to global scales. We survey the strengths and
weaknesses of protocol-based assessments linked to the Agricultural Model
Intercomparison and Improvement Project (AgMIP), each utilizing multiple sites and
models to evaluate crop response to core climate changes including shifts in carbon dioxide
concentration, temperature, and water availability, with some studies further exploring how
climate responses are affected by nitrogen levels and adaptation in farm systems. Site-
based studies with carefully calibrated models encompass the largest number of activities;
however they are limited in their ability to capture the full range of global agricultural
system diversity. Representative site networks provide more targeted response information
than broadly-sampled networks, with limitations stemming from difficulties in covering
the diversity of farming systems. Global gridded crop models provide comprehensive
coverage, although with large challenges for calibration and quality control of inputs.
Diversity in climate responses underscores that crop model emulators must distinguish
between regions and farming system while recognizing model uncertainty. Finally, to
bridge the gap between bottom-up and top-down approaches we recommend the
<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 1.** Overview of aspirational framework and agricultural applications for an IAM with linked climate and crop model emulators. IAMs typically focus on the interplay of socioeconomic development and environmental outcomes, however inclusion of the climate and crop model emulation pathway allows for the resolution of agricultural feedback loops and unintended consequences across scales and sectors. Note that climate emulators have additional applications within IAMs beyond the agricultural sector illustrated here.</figcaption></figure>

resources, as well as repercussions when these natural resources are strained (Füssel et al 2010, Clarke et al 2014). Agriculture has long been central to the relationship between society and natural systems, providing vital foods, fiber, and energy while drawing heavily on land and water resources.

IAMs have traditionally represented agricultural sector changes as exogenous yield changes provided via scenarios aggregated to national or regional level production using current harvested area weights (Müller and Robertson 2014, Nelson et al 2014, Wiebe et al 2015); however these only draw from a small subset of cutting-edge crop model assessments. A more direct coupling of agricultural responses within IAMs is facilitated by the application of crop model emulators, defined here as computationally-efficient representations of crop model results that capture fundamental responses to climate conditions. Crop model emulators may take the form of lookup tables (e.g. based upon response surfaces in Pirttioja et al 2015), simplified response functions (Howden and Crimp 2005, Crimp et al 2008, Ruane et al 2014, Makowski et al 2015), or complex statistical models (Blanc 2017, Ministry et al 2017, Moore et al 2017), each estimating yield as a function of climate variables with varying degrees of non-linearity and detail about the specific crop variety, farm environment, weather extremes, and crop model emulated. As these emulators get more complex the gain in computational efficiency (compared to just using the crop model itself) is reduced, and in the end a crop model emulator is limited by the performance of the crop model or crop model ensemble that it is emulating. Emulators are distinct from statistical crop models, which are trained upon observational data, with one advantage being that

they may use principles of biophysical process response to explore environments that have not been observed (such as future climate and land use change). The exact specifications and desired detail of a crop model emulator depends on the IAM to which it is coupled, the intended applications, and the capabilities and coverage of the underlying crop model assessments.

IAMs have a lot to gain by better incorporating crop responses to changes in carbon dioxide concentration (([CO₂])], temperature, water, nitrogen, and adaptation (CTWNA). CTWNA sensitivity simulations can be more useful than projections driven by global climate models (GCMs) as they provide the information basis to construct crop model emulators for use in IAMs in conjunction with climate emulators (e.g. Meinshausen et al 2011, Castruccio et al 2014, Hartin et al 2015). Figure 1 illustrates how this powerful combination improves agricultural sector representation by allowing IAM land use changes and emissions of greenhouse gases and aerosols to influence regional temperature and precipitation changes (using the climate emulator), affecting crop production and requirements (using the crop model emulator) that feed back into the IAM. This also captures agricultural feedback loops, where societal or environmental changes alter the climate and shift agricultural production in a manner that reinforces or diminishes those changes, and unintended consequences when policies in another sector or region impact distant farming systems (potentially through climate responses or through independent mechanisms such as trade).

The Agricultural Model Intercomparison and Improvement Project (AgMIP; Rosenzweig et al 2013, 2015) was launched in 2010 to provide a common
deployment of a hybrid climate response system employing a representative network of sites to bias-correct comprehensive gridded simulations, opening the door to accelerated development and a broad range of applications.
<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /><figcaption>**Figure 2.** Map of sites and networks for agricultural impacts studies exploring responses to [CO₂], temperature, water, nitrogen, and/or adaptation, and major crops area (%) by Monfreda *et al* (2008), note that studies cover many major production regions, while GGCMI activities simulate the entire land surface.</figcaption></figure>

framework and systematic approach for analysis of agricultural challenges. AgMIP connects climate, crop, livestock, and economic models at local, regional, and global scales, allowing multi-model, multi-discipline, multi-scale assessments of agricultural development and food security (Rosenzweig et al 2016, Antle et al 2015). AgMIP mainly utilizes process-based crop models that represent biophysical processes and their responses to genetics, environment, and management over the course of a growing season, with statistical models also included in some efforts. Integrated assessment modelers examining previous crop modeling studies have been challenged to make sense of differing assumptions, methods, and models in addition to the under-representation of agricultural systems beyond the mid-latitude, high-input breadbaskets (White et al 2011, Challinor et al 2014a). AgMIP facilitates more robust and transferable findings based on common simulation protocols, multi-model ensembles, the tracking of uncertainty, and an emphasis on under-simulated farm systems. Great strides in computational power are opening new doors for agricultural model development and application, raising the ceiling for multi-model analyses, new scales of decision support, and more accurate crop model emulators for IAM applications.

This article takes stock of the methods used by AgMIP to capture the response of agricultural productivity to changing climate conditions, examining the relative strengths and weaknesses of site, network, and gridded modeling approaches to inform IAMs and related crop model emulators. We then provide a framework for coordinated development and application of agricultural responses drawing value from local to global approaches and linking biophysical and integrated assessments. We conclude with recommendations for priority future work and applications.

## 2. Survey of crop model outputs germane to IAM emulators

Although AgMIP conducts more than 30 activities (Rosenzweig et al 2015), here we survey activities that (a) test for sensitivity to some or all of CTWNA factors and utilize (b) multiple agricultural models, (c) multiple sites, and (d) common protocols. These are described in table 1 along with related activities by the MACSUR project (Modelling European Agriculture with Climate Change for Food Security; Ewert et al 2015, Webber et al 2015). Figure 2 presents the geographic coverage of these site, network, and gridded activities.

### 2.1. Site-based approaches

The overwhelming majority of studies in the large literature on crop impacts are site-based studies (White et al 2011, Challinor et al 2014a), but inconsistent protocols, assumptions, geographic sampling, and methods make generalized interpretation of the results difficult. AgMIP's emphasis on model intercomparison and exploration of climate responses drove initial research activities toward species-based assessment at a small number of carefully selected sites. These 'pilot' projects organized around the application of multiple models on high-quality field datasets (Boote et al 2015, Kersebaum et al 2015) to expose differences in model structure, process responses, data requirements, and input/output formats.

The first crop pilot was organized by the AgMIP Wheat Team, in which 27 modeling groups ran historical simulations and 30 year sensitivity tests for [CO₂], temperature, and nitrogen response at sites in the Netherlands, India, Argentina, and Australia (Asseng et al 2013, Martre et al 2015). The Wheat Pilot was open to all interested modeling groups as long as their models were published in peer-reviewed articles.
# 1. Introduction

Integrated assessment models (IAMs) examine the interactions between human systems and the natural environment. IAMs thus explore how societal changes, such as global policies, population growth, socioeconomic development, greenhouse gas emissions, and technological advances affect land, air, and water resources, as well as repercussions when these natural resources are strained (Füssel et al., 2010; Clarke et al., 2014). Agriculture has long been central to the relationship between society and natural systems, providing vital foods, fiber, and energy while drawing heavily on land and water resources.

IAMs have traditionally represented agricultural sector changes as exogenous yield changes provided via scenarios aggregated to national or regional level production using current harvested area weights (Müller and Robertson, 2014; Nelson et al., 2013; Wiebe et al., 2015); however these only draw from a small subset of cutting-edge crop model assessments. A more direct coupling of agricultural responses within IAMs is facilitated by the application of crop model emulators, defined here as computationally-efficient representations of crop model results that capture fundamental responses to climate conditions. Crop model emulators may take the form of lookup tables (e.g., based upon response surfaces in Pirttioja et al., 2015), simplified response functions (Howden and Crimp, 2005; Crimp et al., 2008; Ruane et al., 2014; Makowski et al., 2015), or complex statistical models (Blanc, 2017; Mistry et al., 2017; Moore et al., 2017), each estimating yield as a function of climate variables with varying degrees of non-linearity and detail about the specific crop variety, farm environment, weather extremes, and crop model emulated. As these emulators get more complex the gain in computational efficiency
<figure><img src="image_8.png" /><figcaption>**Table 1.** Overview of multi-model, multi-site, protocol-based research activities sampling the carbon–temperature–water–nitrogen–adaptation (CTWNA) change space by AgMIP and related projects.</figcaption></figure>

<table><thead><tr><th>Research activity</th><th>Scope</th><th>C (ppm)</th><th>T (°C)</th><th>W (%)</th><th>N</th><th>A</th><th>Notes</th></tr></thead><tbody><tr><td>AgMIP-Wheat<br>Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>—</td><td>-50% to<br>+50%</td><td>—</td><td>27 wheat models at each of<br>four sites.<sup>a</sup></td></tr><tr><td>AgMIP-Wheat<br>Phase 2a</td><td>Sites</td><td>—</td><td>+0 to +16</td><td>—</td><td>—</td><td>—</td><td>30 wheat models simulated at<br>two sites.</td></tr><tr><td>AgMIP-Wheat<br>Phase 2b</td><td>Global<br>network</td><td>—</td><td>+0 to +4</td><td>—</td><td>—</td><td>—</td><td>30 wheat models at 30<br>well-watered sites.</td></tr><tr><td>AgMIP-Wheat<br>Phase 3</td><td>Global<br>network</td><td>360 to 550</td><td>+0 to +4</td><td>—</td><td>—</td><td>—</td><td>32 wheat models at 60 sites<br>(water-limited and<br>well-watered sites).</td></tr><tr><td>AgMIP-Maize<br>Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>—</td><td>—</td><td>—</td><td>23 maize models at each of<br>four sites.</td></tr><tr><td>AgMIP-Maize<br>Phase 2</td><td>Sites</td><td>387 and<br>550</td><td>—</td><td>RF and Irr.</td><td>—</td><td>—</td><td>21 maize models for<br>Braunschweig, Germany,<br>FACE site.</td></tr><tr><td>AgMIP-Rice<br>Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>—</td><td>varied N</td><td>—</td><td>13 rice models at each of four<br>sites. Two sites included N<br>treatments ranging from<br>30–150 kg N ha<sup>-1</sup>.</td></tr><tr><td>AgMIP-Rice<br>Phase 2</td><td>Sites</td><td>360 to 720</td><td>—</td><td>—</td><td>varied N</td><td>—</td><td>16 rice models at each of two<br>FACE sites (Japan and<br>China).<sup>b</sup></td></tr><tr><td>AgMIP-Potato</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>-30 to +30</td><td>—</td><td>—</td><td>Nine potato models at each of<br>four sites.</td></tr><tr><td>AgMIP-Canola</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>-25 to +25</td><td>0% to<br>150% of<br>observed</td><td>—</td><td>Eight canola models at each of<br>seven sites.</td></tr><tr><td>AgMIP-Sugarcane</td><td>Sites</td><td>350 to 750</td><td>-3 to +9</td><td>-30 to +30</td><td>—</td><td>—</td><td>Two sugarcane models at<br>seven Brazilian sites.</td></tr><tr><td>AgMIP-Livestock<br>and Grasslands<br>Phase 2</td><td>Sites</td><td>330 to 900</td><td>-1 to +8</td><td>-50 to +50</td><td>—</td><td>—</td><td>Common protocols for single<br>model tests at 14 sites. Seven<br>models contributed yield and<br>GHG balance results.</td></tr><tr><td>AgMIP Regional<br>Integrated<br>Assessments</td><td>Sites</td><td>360 to 720</td><td>-2 to +8</td><td>-75 to<br>+100</td><td>0 to 210 kg<br>N ha<sup>-1</sup></td><td>—</td><td>Two models each for ten sites,<br>multiple crops at many of the<br>sites.</td></tr><tr><td>C3MP</td><td>Global<br>network</td><td>330 to 900</td><td>-1 to +8</td><td>-50 to +50</td><td>—</td><td>—</td><td>1137 simulation sets in 56<br>countries; 18 crop species, 23<br>crop models.</td></tr><tr><td>MACSUR-IRS<br>Phase 1</td><td>Sites</td><td>—</td><td>-2 to +9</td><td>-50 to +50</td><td>—</td><td>—</td><td>26 wheat models at four sites<br>in Europe.</td></tr><tr><td>MACSUR-Crop<br>Rotation</td><td>Sites</td><td>374 and<br>550</td><td>—</td><td>—</td><td>100% and<br>50% of<br>observed</td><td>—</td><td>15 models with and without<br>crop rotations. CN sensitivity<br>tests performed at<br>Braunschweig, Germany, and<br>N sensitivity at Thibie, France.</td></tr><tr><td>GGCMI<br>Phase 2</td><td>Global grid</td><td>360 to 810</td><td>-1 to +6</td><td>-50 to +30<br>plus Irr.</td><td>10 to 200<br>kg N ha<sup>-1</sup></td><td>Fully<br>reverse<br>accelerated<br>maturity</td><td>12 participating models.<sup>b</sup><br>Includes no water stress test<br>and no nitrogen stress test.<br>Adaptation adjusts cultivars to<br>maintain planting to maturity<br>duration.</td></tr></tbody></table>

Notes: RF = rainfed; Irr. = irrigated; GHG = greenhouse gas; varied N = multiple nitrogen treatments at each site.

Similar multi-model crop pilots were conducted across selected sites by AgMIP Maize (CT responses; Bassu et al 2014), AgMIP Rice (CT responses; Li et al 2015), AgMIP Potato (CTW responses; Fleisher et al 2016), AgMIP Sugarcane (CTW responses; Marin et al 2015), and AgMIP Canola (CTWN responses; Wang et al 2017). MACSUR also analyzed TW responses at a transect of four European wheat

sites, providing continuous impacts response surfaces that characterize fundamental crop model properties (Pirttioja et al 2015, Fronzek et al 2017), and examined the CN response of crop rotations (Kollas et al 2015, Yin et al 2017). AgMIP's Livestock and Grasslands Team used individual models at a number of sites to create CTW response surfaces for yield and greenhouse gas balances (Erhardt 2017). Phase 2

<footnote>a Nitrogen tests were only performed for 20 wheat models containing nitrogen dynamics.</footnote>

<footnote>b Ongoing project, final participation may change.</footnote>
(compared to just using the crop model itself) is reduced, and in the end a crop model emulator is limited by the performance of the crop model or crop model ensemble that it is emulating. Emulators are distinct from statistical crop models, which are trained upon observational data, with one advantage being that they may use principles of biophysical process response to explore environments that have not been observed (such as future climate and land use change). The exact specifications and desired detail of a crop model emulator depends on the IAM to which it is coupled, the intended applications, and the capabilities and coverage of the underlying crop model assessments.

IAMs have a lot to gain by better incorporating crop responses to changes in carbon dioxide concentration ([CO₂]), temperature, water, nitrogen, and adaptation (CTWNA). CTWNA sensitivity simulations can be more useful than projections driven by global climate models (GCMs) as they provide the information basis to construct crop model emulators for use in IAMs in conjunction with climate emulators (e.g., Meinshausen et al., 2011; Castruccio et al 2014; Hartin et al., 2015). **Figure 1** illustrates how this powerful combination improves agricultural sector representation by allowing IAM land use changes and emissions of greenhouse gases and aerosols to influence regional temperature and precipitation changes (using the climate emulator), affecting crop production and requirements (using the crop model emulator) that feed back into the IAM. This also captures agricultural feedback loops, where societal or environmental changes alter the climate and shift agricultural production in a manner that reinforces or diminishes those changes, and unintended consequences when policies in another sector or region impact distant farming systems (potentially through climate responses or through independent mechanisms such as trade).
<figure><img src="image_9.png" /><figcaption>**Table 2.** AgMIP-Wheat, AgMIP-Maize, and AgMIP-Rice Team Phase descriptions.</figcaption></figure>

<table><thead><tr><th>Phase (and key references)</th><th>Description</th></tr></thead><tbody><tr><td colspan="2"><strong>AgMIP-Wheat</strong></td></tr><tr><td><strong>Phase 1</strong> (Asseng et al 2013, Martre et al 2015)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>Phase 2a</strong> (Asseng et al 2015a, 2015b)</td><td>Protocol-based multi-model analysis of temperature response at Hot Serial Cereals artificial heating experiment in Arizona and temperature responses in Mexico.</td></tr><tr><td><strong>Phase 2b</strong> (Asseng et al 2015a, 2015b, Liu et al 2016)</td><td>Intercomparison of temperature responses across 30 sites selected as a representative network of well-watered wheat production regions around the world.</td></tr><tr><td><strong>Phase 3</strong> (in progress)</td><td>Intercomparison of temperature responses across 60 sites selected to represent both well-watered and water-limited wheat production regions around the world.</td></tr><tr><td colspan="2"><strong>AgMIP-Maize</strong></td></tr><tr><td><strong>Phase 1</strong> (Bassu et al 2014)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>Phase 2</strong> (Durand et al 2017)</td><td>Protocol-based multi-model intercomparison at Free-Air Carbon Enrichment (FACE) site in Germany.</td></tr><tr><td colspan="2"><strong>AgMIP-Rice</strong></td></tr><tr><td><strong>AgMIP-Rice Phase 1</strong> (Li et al 2015)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>AgMIP-Rice Phase 2</strong> (in progress)</td><td>Protocol-based multi-model intercomparison at FACE Sites in Japan and China.</td></tr></tbody></table>

studies by AgMIP Wheat, Maize, and Rice teams have challenged models with field experiments that gauge climate sensitivity at test sites, utilizing 'Hot Serial Cereal' heat stress experiments for wheat (Asseng et al 2015a, Webber et al 2017) and Free-Air Carbon Enrichment data to explore [CO₂] response in maize (Durand et al 2017) and rice (Hasegawa et al 2017) (table 2). CTWN sensitivity experiments also form a key component of AgMIP's regional integrated assessments at sites across South Asia and Sub-Saharan Africa (Rosenzweig et al 2017).

### 2.1.1. Strengths and weaknesses of site-based approaches

Intensive, multi-model intercomparisons at high-quality pilot field sites are a critical first component of model evaluation, yielding valuable insight into process responses, structural biases, data requirements, and performance across contrasting systems. These analyses are anchored in field data that enable validation of state variables (e.g. leaf-area index; above-ground biomass and N contents; plant-available soil moisture) across a number of phenological stages as well as end-of-season characteristics (e.g. grain yield and protein content, harvest index). This allows evaluation of the mechanisms by which plants respond to environmental changes, highlighting sensitive biophysical processes and growth stages that in turn help focus climate projections on fundamental stresses (e.g. drought in reproductive stages; heat stress at anthesis). Results demonstrate that multi-model ensembles consistently outperform individual models when evaluated across variables and sites (Martre et al 2015, Asseng et al 2013, Bassu et al 2014, Li et al 2015, Fleisher et al 2016), although at any given site a subset of models may be preferred (Castañeda-Vera et al 2015).

Site-based assessments from the initial AgMIP Pilots are limited in their application to IAMs as they cover only a small number of sites and farming systems. As expected, crops responded differently at the selected sites owing to unique soils, weather, cultivars, and farm management. Additional careful sampling of interactions across the broader CTWNA space is needed, as shown by the benefits of elevated [CO₂] for water use efficiency in recent AgMIP crop team activities (Cammarano et al 2016, Deryng et al 2016, Durand et al 2017).

### 2.2. Network-based approaches

As AgMIP protocols were developed and tested on individual sites, the next step scaled up these approaches through larger networks of sites coordinated to ensure adherence to a common protocol that enables direct comparison.

#### 2.2.1. Wide ad hoc network approach

AgMIP launched the Coordinated Climate-Crop Modeling Project (C3MP; Ruane et al 2014, McDermid et al 2015a), to create CTW impact response surfaces at a range of sites around the world. C3MP samples the CTW space projected by GCMs in the 21st century, enabling the fitting of emulators and response surfaces that can be rapidly applied to estimate the agricultural impacts of new climate projections. C3MP created information technology tools and templates to facilitate the process and invited the agricultural modeling community to participate with their own calibrated sites. The resulting archive reflects submissions from 100 crop modelers, with 1137 simulation sets from 55 countries, including results from 19 crop model families and 18 crop species (McDermid et al 2015a).
The Agricultural Model Intercomparison and Improvement Project (AgMIP; Rosenzweig et al., 2013, 2015) was launched in 2010 to provide a common framework and systematic approach for analysis of agricultural challenges. AgMIP connects climate, crop, livestock, and economic models at local, regional, and global scales, allowing multi-model, multi-discipline, multi-scale assessments of agricultural development and food security (Rosenzweig et al., 2016; Antle et al., 2015). AgMIP mainly utilizes process-based crop models that represent biophysical processes and their responses to genetics, environment, and management over the course of a growing season, with statistical models also included in some efforts. Integrated assessment modelers examining previous crop modeling studies have been challenged to make sense of differing assumptions, methods, and models in addition to the under-representation of agricultural systems beyond the mid-latitude, high-input breadbaskets (White et al., 2011; Challinor et al., 2014a). AgMIP facilitates more robust and transferable findings based on common simulation protocols, multi-model ensembles, the tracking of uncertainty, and an emphasis on under-simulated farm systems. Great strides in computational power are opening new doors for agricultural model development and application, raising the ceiling for multi-model analyses, new scales of decision support, and more accurate crop model emulators for IAM applications.

This article takes stock of the methods used by AgMIP to capture the response of agricultural productivity to changing climate conditions, examining the relative strengths and weaknesses of site, network, and gridded modeling approaches to inform IAMs and related crop model emulators. We then provide a framework for coordinated development
<figure><img src="image_10.png" /></figure>

#### 2.2.1.1. Strengths and weaknesses of wide ad hoc networks

C3MP's open call for crop model participation led to an unprecedented number and diversity of contributed simulation sets but also challenges in analyses. The result is a network of voluntary 'crowd-sourced' responses rather than a designed plan of geographic coverage, representative sites, or multi-model analyses. Nevertheless, C3MP's wide ad hoc network covers most major agricultural lands and features models calibrated with site-specific information (figure 2). Sampling across all submitted results for a given category of system (e.g. rainfed maize) provides CTW response surfaces isolating the common yield response across a broad sampling of sites and systems as well as uncertainty stemming from model, soil, baseline climate, cultivar, and farming system differences (McDermid et al 2015a). Recognizing that IAMs typically track major crops (wheat and rice) and commodity groups (e.g. oil seeds, coarse grains, sugar crops, fruits and vegetables), C3MP's relatively large number of crop species also reduces the amount of crop response mapping that is required to represent climate responses across the diversity of agricultural commodities. C3MP is particularly useful in distinguishing responses within a commodity group (for example, differentiating between millet, sorghum, and maize responses for coarse grains).

Aggregation of the C3MP archive to global production responses is challenging given geographic gaps and under-represented systems, and vetting is difficult given its reliance on prior model calibration and a skew toward common crop models (as were also challenges in the Challinor et al 2014a, meta-analysis). We recommend that C3MP analyses do not include simulation sets that use antiquated model versions and a small percentage of flagged sites where low historical yields indicate farming systems that are not presently viable. In some cases, these were conducted as tests of land uses that may become viable in wetter and high-[CO₂] futures, but must be considered distinct from broader CTW analyses. C3MP remains an open process, and each new submission increases the robustness of ensemble statistics and analyses.

#### 2.2.2. Representative network approach

AgMIP Wheat Phase 2b created a global network of 30 well-watered sites selected to represent major wheat systems and regional production areas (irrigated and high-rainfall wheat crops contribute ~70% of global production; see figure 2) (Asseng et al 2015a). 30 wheat models are configured for simulation of CT responses at each site, allowing robust ensemble projections and uncertainty analyses (Wallach et al 2015, 2016).

##### 2.2.2.1. Strengths and weaknesses of representative networks

The AgMIP Wheat Team network is distinct from C3MP's ad hoc network in that its design allows

multi-model assessment on major regional production systems that together generate the large majority of global wheat production (table 2). Simulated relative impacts are applied to recent FAO country production statistics associated with each simulated location to up-scale to global production impacts (Asseng et al 2015a, Liu et al 2016).

Even with 30 sites, the network is limited in its spatial coverage and individual sites may not reflect conditions in the broader production regions they represent. The network is concentrated in high-production zones and is likely to miss important responses in areas that were not simulated (AgMIP-Wheat Phase 3 will fill some of these gaps for water-stressed systems). As a simple metric of comprehensiveness of coverage, figure 3 shows how the rainfed and irrigated wheat networks from C3MP and AgMIP-Wheat Phase 2 cover wheat-growing climate conditions as compared with the global Monthly Irrigated and Rainfed Crop Area (MIRCA) year 2000 dataset (Portmann et al 2010), using AgMERRA climate data (Ruane et al 2015a) and growing seasons from the Global Gridded Crop Model Intercomparison (GGCMI; Elliott et al 2015). Both networks are most dense in climate zones that are prominent for wheat production; however the larger C3MP network also includes less common climates for rainfed wheat and samples more from the tails of the irrigated wheat distribution than does AgMIP-Wheat. By simulating more of the cool and wet tails it is likely that C3MP captures more farms that potentially benefit from increases in temperature or are less vulnerable to decreases in precipitation.

Regions with high levels of diversity are difficult to capture given limitations in representative site networks. Sentinel crop modeling sites are often calibrated with data from field experiment datasets designed to highlight potential genetic, fertilizer, water, or pest control treatments, and therefore may not be representative of prevailing agricultural systems within that production region. These site networks tend to be more useful when examining the percentage yield response to a given climate change; this metric has proven robust even in the face of persistent bias in mean regional yields (Challinor et al 2014b, Asseng et al 2015a).

### 2.3. Global approaches

Advances in high-performance computing have allowed crop models to enter a new phase of development that is nearly unconstrained by computational limitations. While IAMs are typically run on desktop computers or simple clusters, the 18 modeling groups participating in AgMIP's Global Gridded Crop Modeling Intercomparison (GGCMI; table 3) use parallel computing and advanced data processing pipelines to conduct protocol-based simulations on a 0.5° × 0.5° global grid (Rosenzweig et al 2014, Elliott et al 2015), with higher resolution gridded studies in the works. These outputs therefore form a desirable basis for more computationally-efficient IAM application through
and application of agricultural responses drawing value from local to global approaches and linking biophysical and integrated assessments. We conclude with recommendations for priority future work and applications.

## 2. Survey of Crop Model Outputs Germane to IAM Emulators

Although AgMIP conducts more than 30 activities (Rosenzweig et al., 2015), here we survey activities that (a) test for sensitivity to some or all of CTWNA factors and utilize (b) multiple agricultural models, (c) multiple sites, and (d) common protocols. These are described in **Table 1** along with related activities by the MACSUR project (Modelling European Agriculture with Climate Change for Food Security; Ewert et al., 2015). **Figure 2** presents the geographic coverage of these site, network, and gridded activities.

### 2.1 Site-Based Approaches

The overwhelming majority of studies in the large literature on crop impacts are site-based studies (White et al., 2011; Challinor et al., 2014a), but inconsistent protocols, assumptions, geographic sampling, and methods make generalized interpretation of the results difficult. AgMIP's emphasis on model intercomparison and exploration of climate responses drove initial research activities toward species-based assessment at a small number of carefully selected sites. These 'pilot' projects organized around the application of multiple models on high-quality field datasets (Boote et al., 2015; Kersebaum et al., 2015) to expose differences in model structure, process responses, data requirements, and input/output formats.
<figure><img src="image_11.png" /><figcaption>Table 3. Overview of GGCMI phases and model participation.</figcaption></figure>

<table><thead><tr><th>Phase (and key references)</th><th>Description [and # of models participating]</th></tr></thead><tbody><tr><td>Fast Track Rapid Assessment (Rosenzweig et al 2014)</td><td>Conducted for AgMIP/ISIMIP using default versions of global gridded crop models, historical period and future scenarios from downscaled GCMs. Simulated maize, wheat, rice and soybean [seven GGCMs].</td></tr><tr><td>Phase 1 Historical Intercomparison (Elliott et al 2015, Müller et al 2017)</td><td>Default, harmonized, and no nitrogen stress versions of gridded crop models run over historical period using up to nine climate forcing datasets. Simulated maize, wheat, rice and soybean [15 GGCMs].</td></tr><tr><td>Phase 2 CTWNA Sensitivity [results submitted 2016–17]</td><td>Default simulations for historical period and sensitivity tests for [CO<sub>2</sub>], temperature, water, nitrogen, and adaptation for all grid cells and crops. Simulated maize, spring wheat, winter wheat, rice and soybean [~12 GGCMs].</td></tr><tr><td>Phase 3 Future Assessment [planned for 2017–18]</td><td>Conducted for AgMIP/ISIMIP to assess future agricultural production under climate change scenarios [~12–20 GGCMs anticipated].</td></tr></tbody></table>

Note: ISIMIP = the Inter-Sectoral Impacts Model Intercomparison Project (Warszawski et al 2014).

<figure><img src="image_12.png" /></figure>
The first crop pilot was organized by the AgMIP Wheat Team, in which 27 modeling groups ran historical simulations and 30-year sensitivity tests for [CO₂], temperature, and nitrogen (CTN) response at sites in the Netherlands, India, Argentina, and Australia (Asseng et al., 2013; Martre et al., 2015). The Wheat Pilot was open to all interested modeling groups as long as their models were published in peer-reviewed articles.

Similar multi-model crop pilots were conducted across selected sites by AgMIP Maize (CT responses; Bassu et al., 2014), AgMIP Rice (CT responses; Li et al., 2015), AgMIP Potato (CTW responses; Fleisher et al., 2016), AgMIP Sugarcane (CTW responses; Marin et al., 2015), and AgMIP Canola (CTWN responses; Wang et al., personal communication). MACSUR also analyzed TW responses at a transect of four European wheat sites, providing continuous impacts response surfaces that characterize fundamental crop model properties (Pirttioja et al., 2015; Fronzek et al., 2017), and examined the CN response of crop rotations (Kollas et al., 2017; Yin et al., 2017). AgMIP's Livestock and Grasslands Team used individual models at a number of sites to create CTW responses surfaces for yield and greenhouse gas balances (Fiona Erhardt, personal communication). Phase 2 studies by AgMIP Wheat, Maize, and Rice teams have challenged models with field experiments that gauge climate sensitivity at test sites, utilizing "Hot Serial Cereal" heat stress experiments for wheat (Asseng et al., 2015a; Webber et al., 2017) and Free-Air Carbon Enrichment (FACE) data to explore [CO₂] response in maize (Durand et al., 2017) and rice (Hasegawa et al., personal communication) (Table 2). CTWN sensitivity experiments also form a key component of AgMIP's regional integrated assessments at sites across South Asia and Sub-Saharan Africa (Rosenzweig et al., 2017).
<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>**Figure 4.** Example of regional differences in climate response. pDSSAT rainfed maize yield change (%) to a hypothetical 150 ppm increase in [CO₂] and a 2 °C rise in temperature from GGCMI Phase 2 (all regions grown with 200 kg N ha⁻¹ with no adaptation to isolate climate response).</figcaption></figure>

emulators. GGCMI Phase 2 performs a systematic analysis of CTWNA sensitivities for rainfed and irrigated maize, rice, wheat, and soybean with consistent climate information and harmonized planting dates. Adaptation is examined by shifting cultivars to maintain the growing period even as warmer temperatures accelerate phenologic development, thus offsetting some yield losses from climate change.

### 2.3.1. Strengths and weaknesses of global approaches

GGCMI's fast-track results (table 3) provide biophysical impacts across emissions scenarios and five GCMs (Rosenzweig et al 2014), providing applications with ensemble mean impacts and uncertainty information from seven GGCMs for four crop species (maize, wheat, rice, and soybean) across the global grid (Nelson et al 2014, Wiebe et al 2015, Villoria et al 2016). It is difficult for crop model emulators to disentangle fundamental responses from these outputs, however, given the many types of changing and interacting climate conditions (e.g. mean temperatures and rainfall; sub-seasonal variations; extreme events). Emulation is also complicated by the inclusion of responsive adaptations allowing management to evolve with climate change in some participating models (Rosenzweig et al 2014, supplementary).

GGCMI Phase 2 findings indicate considerable spatial variation in CTWNA response across different environments and farm systems, exemplified by the response of rainfed maize to higher [CO₂] and temperature in the parallel-DSSAT crop model (pDSSAT; Elliott et al 2014) (figure 4). These results provide a convenient basis for the construction of crop model emulators, and can also be connected to economic and/or resource availability drivers from IAMs to

dynamically characterize the evolution of socioeconomic yield gap factors such as fertilizer use, irrigation, and adaptation potential.

In contrast to the site networks, GGCMs rely on gridded soil, genetic, management, and weather datasets designed to capture spatially-averaged conditions rather than conditions on a particular farm (Elliott et al 2015). While the 0.5° × 0.5° spatial resolution used within GGCMI is finer than many GCMs, a grid cell on the equator represents >310 000 ha and thus poses a challenge for comprehensive farm system calibration.

GGCM results are often evaluated using regional yield and production reports, with trend adjustment recommended in recognition of technological development and processes that are not explicitly modeled such as pests, diseases, and widespread flooding (Müller et al 2017). Analogously, statistical crop response models are occasionally fitted to similar aggregate yield data that may reflect embedded abiotic factors (e.g. Lobell et al 2011). Bias-adjustment is recommended for GGCM application in IAMs, similar to common practices for climate model output applications (e.g. Wilby et al 2004, Ruane et al 2015b). Overall, GGCMs reflect that there are larger uncertainties in developing country and low-input farming systems, and stand to benefit from improved data collection and sharing in application regions (Kihara et al 2015, McDermid et al 2015b).

## 2.4. Emergent characteristics and opportunities from CTWNA simulations

AgMIP site, network, and gridded results demonstrate that multi-model ensembles outperform individual models when analyzed across multiple sites and evaluation variables (e.g. Asseng et al 2013,
### 2.1.1 Strengths and weaknesses of site-based approaches

Intensive, multi-model intercomparisons at high-quality pilot field sites are a critical first component of model evaluation, yielding valuable insight into process responses, structural biases, data requirements, and performance across contrasting systems. These analyses are anchored in field data that enable validation of state variables (e.g., leaf-area index; above-ground biomass and N contents; plant-available soil moisture) across a number of phenological stages as well as end-of-season characteristics (e.g., grain yield and protein content, harvest index). This allows evaluation of the mechanisms by which plants respond to environmental changes, highlighting sensitive biophysical processes and growth stages that in turn help focus climate projections on fundamental stresses (e.g., drought in reproductive stages; heat stress at anthesis). Results demonstrate that multi-model ensembles consistently outperform individual models when evaluated across variables and sites (Martre et al., 2015; Asseng et al., 2013; Bassu et al., 2014; Li et al., 2015; Fleisher et al., 2016), although at any given site a subset of models may be preferred (Castañeda-Vera et al., 2015).

Site-based assessments from the initial AgMIP Pilots are limited in their application to IAMs as they cover only a small number of sites and farming systems. As expected, crops responded differently at the selected sites owing to unique soils, weather, cultivars, and farm management. Additional careful sampling of interactions across the broader CTWNA space is needed, as shown by the benefits of elevated [CO₂] for water use efficiency in
<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /><figcaption>**Figure 5.** Wheat growing seasons (average planting and harvest date) at each well-watered site in the AgMIP-Wheat network as well as corresponding grid cells in the harmonized GGCMI protocols. Differences for Canada, Turkey, and Ukraine indicate that GGCMI considered spring wheat while the wheat network considered winter wheat, while India and Mexico reflect that there are two wheat-growing seasons in these regions.</figcaption></figure>

Rosenzweig et al 2014, Martre et al 2015, Li et al 2015, Bassu et al 2014, Wallach et al 2015, Ruane et al 2016, Fleisher et al 2016). Liu et al (2016) found relative agreement in wheat response to a 1°C rise in global temperature, with multi-model ensembles in the well-watered AgMIP Wheat network, GGCMI's ISIMIP fast-track, and several statistical model approaches finding 4.1%–6.4% declines in global production.

Uncertainties in input data indicate that there is still room for harmonization that will improve consistency, as illustrated by a comparison of growing seasons at the well-watered AgMIP-Wheat network sites and corresponding GGCMI grid cells (figure 5). Uncertainty owing to model structure and parameters remains substantial, and differences in CTWNA responses by two modelers using the same DSSAT model within the MACSUR IRS and AgMIP-Wheat Phase 1 also highlights the potential role of modeler uncertainty stemming from assumptions and subjective decisions made in the absence of supporting data

(Pirttioja et al 2015, Confalonieri et al 2016). We therefore advise applications to recognize the uncertainty in model-based responses through the use of emulators derived from multiple models or an imposed error term scaled to model-based uncertainty. Evidence across AgMIP activities also recommends avoidance of universal yield functions in favor of yield response functions fitted to broad agro-ecological zones and farming systems (e.g. defined by fertilizer and irrigation inputs).

### 3. AgMIP framework for improved agricultural representation in IAMs

A cascading pathway of development underlies agricultural representation in IAMs, forming a framework that may be used to drive coordinated development of ‘simulation levels’, here defined as common communities of development including site-based crop models, network and gridded models, crop model emulators,
recent AgMIP crop team activities (Cammarano et al., 2016; Deryng et al., 2016; Durand et al., 2017).

## 2.2 Network-based Approaches

As AgMIP protocols were developed and tested on individual sites, the next step scaled up these approaches through larger networks of sites coordinated to ensure adherence to a common protocol that enables direct comparison.

### 2.2.1 Wide *ad hoc* network approach

AgMIP launched the Coordinated Climate-Crop Modeling Project (C3MP; Ruane et al., 2013; McDermid et al., 2015a), to create CTW impact response surfaces at a range of sites around the world. C3MP samples the CTW space projected by GCMs in the 21st century, enabling the fitting of emulators and response surfaces that can be rapidly applied to estimate the agricultural impacts of new climate projections. C3MP created information technology tools and templates to facilitate the process and invited the agricultural modeling community to participate with their own calibrated sites. The resulting archive reflects submissions from 100 crop modelers, with 1137 simulation sets from 55 countries, including results from 19 crop model families and 18 crop species (McDermid et al., 2015a).

#### 2.2.1.1. Strengths and weaknesses of wide *ad hoc* networks

C3MP's open call for crop model participation led to an unprecedented number and diversity of contributed simulation sets but also challenges in analyses. The result is a
<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /><figcaption>**Figure 6.** AgMIP framework for improved agricultural representation in IAMs. The core agricultural response development and application pathway (green arrows) spans several levels of model applications (dark blue boxes) and recognizes that site-based crop models are the backbone of model networks and grids, which feed into IAMs either directly or through crop model emulators built upon a hybrid system blending network and gridded CTWNA responses. Improvement in each level of model development requires access to data for evaluation and configuration (gray boxes) as well as methodological advances (light blue boxes). Agricultural applications also inform development up the framework chain, with IAMs providing critical information about the economic viability of changing land use patterns, emulators helping to isolate aggregate CTWNA responses, and networks and grids testing site-based models in more diverse settings.</figcaption></figure>

and eventual IAM applications (figure 6). Close collaboration and regular updates between site, network, and gridded crop modelers, emulation experts, and IAM groups are needed to keep agricultural impact applications on the cutting edge, to facilitate the use of multiple models, to incorporate understanding from multiple modeling groups, and to avoid the propagation of known biases.

Each simulation level in the AgMIP Framework benefits from improved data access and innovations in core methodologies. Investment in research and development is well served by matching the design, capabilities, and development priorities of models and tools at each level in figure 6. In particular, new biophysical process understanding is best developed within site-based models using field experiment data, particularly for under-sampled agro-ecological zones, crop species, and farming systems under various intensifications (Challinor et al 2015, Maiorano et al 2017).

Networks and gridded models gain from new datasets that allow extensive configuration for many sites and systems, and have tremendous potential to apply advanced bias-correction and aggregation approaches (Challinor et al 2014, van Bussel et al 2016, Hoffmann et al 2015, 2016, Zhao et al 2015, 2016). Crop model emulators are progressed with improved statistical efficiencies and the availability of observed agricultural response data for evaluating strengths and weaknesses. In addition to the potential benefit of adding improved crop model emulators, IAM simulations of long-term shifts in agricultural production are furthered by good data on current systems and advanced representation of the implications of agricultural investment and technological development.

The AgMIP framework for improved agricultural representation in IAMs is non-linear as lower simulation levels build upon advances higher up in the framework and high levels also receive critical
network of voluntary ‘crowd-sourced’ responses rather than a designed plan of geographic coverage, representative sites, or multi-model analyses. Nevertheless, C3MP’s wide *ad hoc* network covers most major agricultural lands and features models calibrated with site-specific information (Fig. 2). Sampling across all submitted results for a given category of system (e.g., rainfed maize) provides CTW response surfaces isolating the common yield response across a broad sampling of sites and systems as well as uncertainty stemming from model, soil, baseline climate, cultivar, and farming system differences (McDermid et al., 2015a). Recognizing that IAMs typically track major crops (wheat and rice) and commodity groups (e.g., oil seeds, coarse grains, sugar crops, fruits & vegetables), C3MP’s relatively large number of crop species also reduces the amount of crop response mapping that is required to represent climate responses across the diversity of agricultural commodities. C3MP is particularly useful in distinguishing responses within a commodity group (for example, differentiating between millet, sorghum, and maize responses for coarse grains).

Aggregation of the C3MP archive to global production responses is challenging given geographic gaps and under-represented systems, and vetting is difficult given its reliance on prior model calibration and a skew toward common crop models (as were also challenges in the Challinor et al., 2014a, meta-analysis). We recommend that C3MP analyses do not include simulation sets that use antiquated model versions and a small percentage of flagged sites where low historical yields indicate farming systems that are not presently viable. In some cases, these were conducted as tests of land uses that may become viable in wetter and high-[CO₂] futures, but must be considered distinct from

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
<figure><img src="image_19.png" /></figure>

feedback from downstream simulation levels. Pathways of upstream improvements include that assessments of improved models on established grids and networks provide vital feedback for site-based model development on diverse sites. Likewise, emulators often spotlight key sensitivities and uncertainties that may spur further site-based model development and the creation of more representative networks. Network and gridded studies examine the biophysical viability of various simulated farm systems to determine land use pressures, but benefit tremendously by incorporating information on economic viability and resource constraints that IAMs can provide. It is also important to note that many of these simulation levels have extensive applications beyond agricultural representation in IAMs, and that the key bottleneck for one applications may differ from another's crucial development priority.

## 4. Priority future development and applications

Analysis of the multi-model, multi-site climate sensitivity datasets reviewed in this study suggest that IAMs and other large-scale applications would be well served by the creation and systematic development of a hybrid CTWNA response system that blends the strengths of network and gridded approaches (as noted in figure 6). This hybrid response system would be rooted in (1) detailed process understanding across a representative network of well-calibrated field sites (ideally using field data from prevailing management systems) combined with (2) comprehensive CTWNA coverage from gridded models. Baseline responses generated by these gridded models could initially be compared against the corresponding representative network simulations to assess methodological uncertainty and calculate bias-correction factors. Bias-corrected gridded results could then provide an information basis for crop model emulators and IAM applications, characterizing different farming systems using nitrogen and water components of the CTWNA analysis.

Table 1 highlights that progress toward the creation of this hybrid response system is most advanced for wheat, given the AgMIP-Wheat Phase 2b representative network and spring and winter wheat simulations within GGCMI Phase 2. In contrast, soybean is simulated in GGCMI but has not yet been the focus of site- or network-based CTWNA analysis, and a number of other important commodities merit inclusion. Coordinated and systematic development of the hybrid response system would foster rapid iterative improvements, as research groups improve the hybrid framework by contributing new process understanding, field sites, model runs, regional configuration information, or statistical approaches. An expanded representative network of models and a fully configured high-resolution gridded (or geo-referenced polygon) model will eventually be interchangeable; however this

hybrid response system provides current state-of-the-art responses and a practical roadmap for applications.

Coordination across AgMIP activities supports the development of linked global and regional assessments to address agricultural sector challenges and food security (Rosenzweig et al 2016). Inclusion of IAMs would bring these to a new level, although it is critical that these account for lingering model uncertainty and data gaps even as these are addressed through the coordinated development of agricultural response in linked models.

## Acknowledgements

We are grateful for the tremendous effort and contributions provided by all modelers and leaders of all projects listed in table 1. This letter was motivated by discussions with members of the Program on Integrated Assessment Model Development, Diagnostics, and Intercomparisons (PIAMDDI) and the Pacific Northwest National Laboratories, and benefited from support by the National Aeronautics and Space Agency Science Mission Directorate (WBS 281945.02.03.06.79), the United Kingdom Department for International Development (GB-1-202108), and the Department of Energy (DE-AC5-76RL01830 and DE-SC0005171) as well as the Joint Research Centre of the European Commission. We appreciate the assistance of Meridel Phillips and Greg Reppucci in figure creation.

## ORCID iDs

Alex C Ruane https://orcid.org/0000-0002-5582-9217

Cynthia Rosenzweig https://orcid.org/0000-0002-7885-671X

Christoph Müller https://orcid.org/0000-0002-9491-3550

## References

Antle J M, Valdivia R O, Boote K, Janssen S, Jones J W, Porter C H, Rosenzweig C, Ruane A C and Thorburn P J 2015 AgMIP's transdisciplinary agricultural systems approach to regional integrated assessment of climate impacts, vulnerability, and adaptation Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1 ICP Series on Climate Change Impacts, Adaptation, and Mitigation ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 27–44

Asseng S et al 2013 Uncertainty in simulating wheat yields under climate change Nat. Clim. Change 3 827–32

Asseng S et al 2015a Rising temperatures reduce global wheat production Nat. Clim. Change 5 143–7

Asseng S et al 2015b Benchmark data set for wheat growth models: field experiments and AgMIP multi-model simulations Open Data J. Agric. Res. 1 1–5

Bassu S et al 2014 Do various maize crop models give the same responses to climate change factors? Glob. Change Biol. 20 2301–20
broader CTW analyses. C3MP remains an open process, and each new submission increases the robustness of ensemble statistics and analyses.

### 2.2.2 Representative network approach

AgMIP Wheat Phase 2b created a global network of 30 well-watered sites selected to represent major wheat systems and regional production areas (irrigated and high-rainfall wheat crops contribute ~70% of global production; see Fig. 2) (Asseng et al., 2015a). 30 wheat models are configured for simulation of CT responses at each site, allowing robust ensemble projections and uncertainty analyses (Wallach et al., 2015; 2016).

#### 2.2.2.1 Strengths and weaknesses of representative networks

The AgMIP Wheat Team network is distinct from C3MP's *ad hoc* network in that its design allows multi-model assessment on major regional production systems that together generate the large majority of global wheat production (Table 2). Simulated relative impacts are applied to recent FAO country production statistics associated with each simulated location to up-scale to global production impacts (Asseng et al., 2015a; Liu et al., 2016).

Even with 30 sites, the network is limited in its spatial coverage and individual sites may not reflect conditions in the broader production regions they represent. The network is concentrated in high-production zones and is likely to miss important responses in areas that were not simulated (AgMIP-Wheat Phase 3 will fill some of these gaps for water-stressed systems). As a simple metric of comprehensiveness of coverage, **Figure 3** shows
<figure><img src="image_20.png" /></figure>

Blanc E. 2017 Statistical emulators of maize, rice, soybean and wheat yields from global gridded crop models *Agric. Forest Meteorol.* **236** 145–61

Boote K J, Porter C H, Jones J W, Thorburn P J, Kersebaum K C, Hoogenboom G, White J W and Hatfield J L. 2015 Sentinel site data for model improvement—definition and characterization *Improving Modeling Tools to Assess Climate Change Effects on Crop Response, Advances in Agricultural Systems Modeling* ed J L Hatfield and D Fleisher (Madison, WI: ASA, CSSA, and SSSA)

Cammarano D *et al* 2016 Uncertainty of wheat water use: Simulated patterns and sensitivity to temperature and CO₂ *Field Crops Res.* **198** 80–92

Castañeda-Vera A, Leffelaar P A, Álvaro-Fuentes J, Cantero-Martínez C and Mínguez M I. 2015 Selecting crop models for decision making in wheat insurance *Eur. J. Agron.* **68** 97–116

Castruccio S, McInerney D J, Stein M L, Crouch F L, Jacob R L and Moyer E J. 2014 Statistical emulation of climate model projections based on precomputed GCM runs *J. Clim.* **27** 1829–44

Challinor A J *et al* 2014a A meta-analysis of crop yield under climate change and adaptation *Nat. Clim. Change* **4** 287–91

Challinor A, Martre P, Asseng S, Thornton P and Ewert F. 2014b Making the most of climate impacts ensembles *Nat. Clim. Change* **4** 77–80

Challinor A, Parkes B and Ramirez-Villegas J. 2015 Crop yield response to climate change varies with cropping intensity *Glob. Change Biol.* **21** 1679–88

Clarke L *et al* 2014 Assessing transformation pathways climate change 2014: mitigation of climate change *Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change* ed O Edenhofer *et al* (Cambridge: Cambridge University Press)

Confalonieri R *et al* 2016 Uncertainty in crop model predictions: what is the role of users? *Environ. Model. Softw.* **81** 165–73

Crimp S, Howden M, Power B, Wang E and deVoil P. 2008 Global climate change impacts on Australia's wheat crops *Garnaut Climate Change report* (Canberra)

Deryng D *et al* 2016 Regional disparities in the beneficial effects of rising CO₂ concentrations on crop water productivity *Nature Clim. Change* **6** 786–90

Durand J L *et al* 2017 How accurately do maize crop models simulate the interactions of atmospheric CO₂ concentration levels with limited water supply on water use and yield? *Eur. J. Agron.* accepted (https://doi.org/10.1016/j.eja.2017.01.002)

Elliott J, Kelly D, Chryssanthacopoulos J, Glotter M, Jhunjhnuwala K, Best N, Wilde M and Foster I. 2014 The parallel system for integrating impact models and sectors (pSIMS) *Environ. Model. Softw.* **62** 509–16

Elliott J *et al* 2015 The global gridded crop model intercomparison: data and modeling protocols for phase 1 (v1.0) *Geosci. Model Dev.* **8** 261–77

Erhardt F. 2017 personal communication

Ewert F *et al* 2015 Crop modelling for integrated assessment of risk to food production from climate change *Environ. Model. Softw.* **72** 287–303

Fleisher D H *et al* 2016 A potato model intercomparison across varying climates and productivity levels *Glob. Change Biol.* **23** 1258–81

Fronzek S *et al* 2017 Classifying multi-model wheat yield impact response surfaces showing sensitivity to temperature and precipitation change *Agric. Syst.* accepted (https://doi.org/10.1016/j.agsy.2017.08.004)

Füssel H-M. 2010 Modeling impacts and adaptation in global IAMs *WIREs Clim. Change* **1** 288–303

Hartin C A, Patel P L, Schwarber A, Link R P and Bond-Lamberty B. 2015 A simple object-oriented and open source model for scientific and policy analyses of the global climate system—Hector v1.0 *Geosci. Model Dev.* **8** 939–55

Hasegawa *et al* 2017 personal communication

Hoffmann H *et al* 2015 Variability of effects of spatial climate data aggregation on regional yield simulation by crop models *Clim. Res.* **65** 53–69

Hoffmann H *et al* 2016 Impact of spatial soil and climate input data aggregation on regional yield simulations *PLoS ONE* **11** e015178

Howden M and Crimp S. 2005 Assessing dangerous climate change impacts on Australia's wheat industry *MODSIM 2005 International Congress on Modeling and Simulation (December 2005)* pp 170–76 (Modelling and Simulation Society of Australia and New Zealand)

Kersebaum K C *et al* 2015 Analysis and classification of data sets for calibration and validation of agro-ecosystem models *Environ. Model. Softw.* **72** 402–17

Kihara J *et al* 2015 Perspectives on climate effects on agriculture: the international efforts of AgMIP in sub-saharan Africa *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 2 ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 3–24

Kollas C *et al* 2015 Crop rotation modelling—a European model intercomparison *Eur. J. Agron.* **70** 98–111

Li T *et al* 2015 Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions *Glob. Change Biol.* **21** 1328–41

Liu B *et al* 2016 Similar negative impacts of temperature on global wheat yield estimated by three independent methods *Nature Clim. Change* **6** 1130–6

Lobell D B, Schlenker W and Costa-Roberts J. 2011 Climate trends and global crop production since 1980 *Science* **333** 616–20

Maiorano A *et al* 2017 Crop model improvement reduces the uncertainty of the response to temperature of multi-model ensembles *Field Crops Res.* **202** 5–20

Makowski D *et al* 2015 Statistical analysis of large simulated yield datasets for studying climate effects *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1 ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 279–98

Marin F R, Thorburn P J, Nassif D S P and Costa L G. 2015 Sugarcane model intercomparison: structural differences and uncertainties under climate change *Environ. Model. Softw.* **72** 372–86

Martre P *et al* 2015 Error of multimodel ensembles of wheat growth: more models are better than one *Glob. Change Biol.* **21** 911–25

McDermid S P *et al* 2015a The agmip coordinated climate-crop modeling project (C3MP): methods and protocols *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 191–220

McDermid S P *et al* 2015b Integrated assessments of the impacts of climate change on agriculture: an overview of AgMIP regional research in South Asia *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 2 ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 201–18

Meinshausen M, Raper S C B and Wigley T M L. 2011 Emulating coupled atmosphere-ocean and carbon cycle models with a simpler model, MAGICC6—Part 1: model description and calibration *Atmos. Chem. Phys.* **11** 1417–56

Mistry M N, Sue Wing I and De Cian E. 2017 Simulated vs. empirical weather responsiveness of crop yields: US evidence and implications for the agricultural impacts of climate change *Environ. Res. Lett.* **12** 075007

Monfreda C, Ramankutty N and Foley J A. 2008 Farming the planet: 2. Geographic distribution of crop areas, yields, physiological types, and net primary production in the year 2000 *Glob. Biogeochem. Cycles* **22** GB1022
how the rainfed and irrigated wheat networks from C3MP and AgMIP-Wheat Phase 2 cover wheat-growing climate conditions as compared with the global Monthly Irrigated and Rainfed Crop Area (MIRCA) year 2000 dataset (Portmann et al., 2010), using AgMERRA climate data (Ruane et al., 2015a) and growing seasons from the Global Gridded Crop Model Intercomparison (GGCMI; Elliott et al., 2015). Both networks are most dense in climate zones that are prominent for wheat production; however the larger C3MP network also includes less common climates for rainfed wheat and samples more from the tails of the irrigated wheat distribution than does AgMIP-Wheat. By simulating more of the cool and wet tails it is likely that C3MP captures more farms that potentially benefit from increases in temperature or are less vulnerable to decreases in precipitation.

Regions with high levels of diversity are difficult to capture given limitations in representative site networks. Sentinel crop modeling sites are often calibrated with data from field experiment datasets designed to highlight potential genetic, fertilizer, water, or pest control treatments, and therefore may not be representative of prevailing agricultural systems within that production region. These site networks tend to be more useful when examining the percentage yield response to a given climate change; this metric has proven robust even in the face of persistent bias in mean regional yields (Challinor et al., 2014b; Asseng et al., 2015a).

## 2.3 Global approaches

Advances in high-performance computing have allowed crop models to enter a new phase of development that is nearly unconstrained by computational limitations. While IAMs are
<figure><img src="image_21.png" /></figure>

Moore F, Baldos U L C and Hertel T 2017 Economic impacts of climate change on agriculture: a comparison of process-based and statistical yield models *Environ. Res. Lett.* 12 065008

Müller C and Robertson R D 2014 Projecting future crop productivity for global economic modeling *Agric. Econ.* 45 37–50

Müller C et al 2017 Global Gridded Crop Model evaluation: Benchmarking, skills, deficiencies and implications *Geosci. Model Dev.* 10 1403–22

Nelson G C et al 2014 Climate change effects on agriculture: economic responses to biophysical shocks *Proc. Natl Acad. Sci.* 111 3274–9

Pirttioja N 2015 A crop model ensemble analysis of temperature and precipitation effects on wheat yield across a European transect using impact response surfaces *Clim. Res.* 65 87–105

Portmann F T, Siebert S and Döll P 2010 MIRCA2000—Global monthly irrigated and rainfed crop areas around the year 2000: a new high-resolution data set for agricultural and hydrological modeling *Glob. Biogeochem. Cycles* 24 GB1011

Ray D K, Gerber J S, MacDonald G K and West P C 2015 Climate variation explains a third of global crop yield variability *Nat. Commun.* 6 5989

Rosenzweig C et al 2013 The agricultural model intercomparison and improvement project (AgMIP): protocols and pilot studies *Agric. Forest Meteorol.* 170 166–82

Rosenzweig C et al 2014 Assessing agricultural risks of climate change in the 21st century in a global gridded crop model intercomparison *Proc. Natl Acad. Sci.* 111 3268–73

Rosenzweig C, Jones J W, Hatfield J L, Antle J M, Ruane A C and Mutter C Z 2015 The agricultural model intercomparison and improvement project: phase I activities by a global community of science *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 3–24

Rosenzweig C, Antle J and Elliott J 2016 Assessing impacts of climate change on food security worldwide *Eos* 97 EO047387

Rosenzweig C et al 2017 Protocols for AgMIP Regional Integrated Assessments (www.agmip.org/regional-integrated-assessments-handbook/)

Ruane A C, McDermid S, Rosenzweig C, Baigorria G A, Jones J W, Romero C C and Cecil L D 2014 Carbon–temperature–water change analysis for peanut production under climate change: a prototype for the AgMIP coordinated climate–crop modeling project (C3MP) *Glob. Change Biol.* 20 394–407

Ruane A C, Goldberg R and Chryssanthacopoulos J 2015a Climate forcing datasets for agricultural modeling: Merged products for gap-filling and historical climate series estimation *Agric. Forest Meteorol.* 200 233–48

Ruane A C, Winter J M, McDermid S P and Hudson N I 2015b Agmip climate datasets and scenarios for integrated assessment *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments*,

*Part 1 ICP Series on Climate Change Impacts Adaptation Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 45–78

Ruane A C et al 2016 Multi-wheat model ensemble responses to interannual climate variability *Environ. Model. Softw.* 81 86–101

van Bussel L et al 2016 Spatial sampling of weather data for regional crop yield simulations *Agric. Forest Meteorol.* 220 101–15

Villoria N et al 2016 Rapid aggregation of global gridded crop model outputs to facilitate cross-disciplinary analysis of climate change impacts in agriculture *Environ. Model. Softw.* 75 193–201

Wallach D, Mearns L O, Rivington M, Antle J M and Ruane A C 2015 Uncertainty in agricultural impact assessment *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) ICP Series on Climate Change Impacts, Adaptation, and Mitigation* ed C Rosenzweig and D Hillel vol 3 (London: Imperial College Press) pp 223–60

Wallach D, Mearns L O, Ruane A C, Rötter R P and Asseng S 2016 Lessons from the climate modeling community on the design and use of ensembles for crop modeling *Clim. Change* 139 551–64

Wang et al 2017 personal communication

Warszawski L, Frieler K, Huber V, Piontek F, Serdeczny O and Schewe J 2014 The Inter-Sectoral Impact Model Intercomparison Project (ISI–MIP): project framework *Proc. Natl Acad. Sci. USA* 111 3228–32

Webber H, Zhao G, Wolf J, Britz W, Vries W D, Gaiser T, Hoffmann H and Ewert F 2015 Climate change impacts on European crop yields: do we need to consider nitrogen limitation? *Eur. J. Agron.* 71 123–34

Webber H et al 2017 Canopy temperature for simulation of heat stress in irrigated wheat in a semi-arid environment: a multi-model comparison *Field Crops Res.* 202 21–35

White J W, Hoogenboom G, Kimball B A and Wall G W 2011 Methodologies for simulating impacts of climate change on crop production *Field Crops Res.* 124 357–68

Wiebe K et al 2015 Climate change impacts on agriculture in 2050 under a range of plausible socioeconomic and emissions scenarios *Environ. Res. Lett.* 10 085010

Wilby R L, Charles S, Zorita E, Timbal B, Whetton P and Mearns L 2004 Guidelines for use of climate scenarios developed from statistical downscaling methods *IPCC Task Group on Data and Scenario Support for Impacts and Climate Analysis* (www.ipcc-data.org/guidelines/dgm_no2_v1_09_2004.pdf) (Accessed: 2 October 2017)

Yin X et al 2017 Multi-model uncertainty analysis in predicting grain N for crop rotations in Europe *Eur. J. Agron.* 84 152–65

Zhao G et al 2015 Effect of weather data aggregation on regional crop simulation for different crops, production conditions, and response

Zhao G et al 2016 Evaluating the precision of eight spatial sampling schemes in estimating regional means of simulated yield for two crops *Environ. Model. Softw.* 80 100–12
typically run on desktop computers or simple clusters, the 18 modeling groups participating in AgMIP's Global Gridded Crop Modeling Intercomparison (GGCMI; Table 3) use parallel computing and advanced data processing pipelines to conduct protocol-based simulations on a 0.5° x 0.5° global grid (Rosenzweig et al., 2014; Elliott et al., 2015), with higher resolution gridded studies in the works. These outputs therefore form a desirable basis for more computationally-efficient IAM application through emulators. GGCMI Phase 2 performs a systematic analysis of CTWNA sensitivities for rainfed and irrigated maize, rice, wheat, and soybean with consistent climate information and harmonized planting dates. Adaptation is examined by shifting cultivars to maintain the growing period even as warmer temperatures accelerate phenologic development, thus offsetting some yield losses from climate change.

### 2.3.1 Strengths and Weaknesses of Global Approaches

GGCMI's fast-track results (Table 3) provide biophysical impacts across emissions scenarios and 5 GCMs (Rosenzweig et al., 2014), providing applications with ensemble mean impacts and uncertainty information from 7 GGCMs for 4 crop species (maize, wheat, rice, and soybean) across the global grid (Nelson et al. 2014; Wiebe et al. 2015; Villoria et al. 2016). It is difficult for crop model emulators to disentangle fundamental responses from these outputs, however, given the many types of changing and interacting climate conditions (e.g., mean temperatures and rainfall; sub-seasonal variations; extreme events). Emulation is also complicated by the inclusion of responsive adaptations allowing management to evolve with climate change in some participating models (Rosenzweig et al., 2014, supplementary).
GGCMI Phase 2 findings indicate considerable spatial variation in CTWNA response across different environments and farm systems, exemplified by the response of rainfed maize to higher [CO₂] and temperature in the parallel-DSSAT crop model (pDSSAT; Elliott et al., 2014) (Figure 4). These results provide a convenient basis for the construction of crop model emulators, and can also be connected to economic and/or resource availability drivers from IAMs to dynamically characterize the evolution of socioeconomic yield gap factors such as fertilizer use, irrigation, and adaptation potential.

In contrast to the site networks, GGCMs rely on gridded soil, genetic, management, and weather datasets designed to capture spatially-averaged conditions rather than conditions on a particular farm (Elliott et al., 2015). While the 0.5° x 0.5° spatial resolution used within GGCMI is finer than many GCMs, a grid cell on the equator represents >310,000 ha and thus poses a challenge for comprehensive farm system calibration.

GGCM results are often evaluated using regional yield and production reports, with trend adjustment recommended in recognition of technological development and processes that are not explicitly modeled such as pests, diseases, and widespread flooding (Müller et al., 2017). Analogously, statistical crop response models are occasionally fitted to similar aggregate yield data that may reflect embedded abiotic factors (e.g., Lobell et al., 2011). Bias-adjustment is a recommended for GGCM application in IAMs, similar to common practices for climate model output applications (e.g., Wilby et al., 2004; Ruane et al., 2015b). Overall, GGCMs reflect that there are larger uncertainties in developing country
and low-input farming systems, and stand to benefit from improved data collection and sharing in application regions (Kihara et al., 2015; McDermid et al., 2015b).

## 2.4. Emergent characteristics and opportunities from CTWNA simulations

AgMIP site, network, and gridded results demonstrate that multi-model ensembles outperform individual models when analyzed across multiple sites and evaluation variables (e.g., Asseng et al., 2013; Rosenzweig et al., 2014; Martre et al., 2015; Li et al., 2014; Bassu et al., 2014; Wallach et al., 2015; Ruane et al., 2016; Fleisher et al., 2016). Liu et al. (2016) found relative agreement in wheat response to a 1 °C rise in global temperature, with multi-model ensembles in the well-watered AgMIP Wheat network, GGCMI's ISIMIP fast-track, and several statistical model approaches finding 4.1-6.4% declines in global production.

Uncertainties in input data indicate that there is still room for harmonization that will improve consistency, as illustrated by a comparison of growing seasons at the well-watered AgMIP-Wheat network sites and corresponding GGCMI grid cells (Figure 5). Uncertainty owing to model structure and parameters remains substantial, and differences in CTWNA responses by two modelers using the same DSSAT model within the MACSUR IRS and AgMIP-Wheat Phase 1 also highlights the potential role of modeler uncertainty stemming from assumptions and subjective decisions made in the absence of supporting data (Pirttioja et al., 2015; Confalonieri et al., 2015). We therefore advise applications to recognize the uncertainty in model-based responses through the use of emulators derived from multiple models or an imposed error term scaled to model-based uncertainty.
Evidence across AgMIP activities also recommends avoidance of universal yield functions
in favor of yield response functions fitted to broad agro-ecological zones and farming
systems (e.g., defined by fertilizer and irrigation inputs).

3. AgMIP framework for improved agricultural representation in IAMs

A cascading pathway of development underlies agricultural representation in IAMs,
forming a framework that may be used to drive coordinated development of “simulation
levels”, here defined as common communities of development including site-based crop
models, network and gridded models, crop model emulators, and eventual IAM
applications (Figure 6). Close collaboration and regular updates between site, network,
and gridded crop modelers, emulation experts, and IAM groups are needed to keep
agricultural impact applications on the cutting edge, to facilitate the use of multiple models,
to incorporate understanding from multiple modeling groups, and to avoid the propagation
of known biases.

Each simulation level in the AgMIP Framework benefits from improved data access and
innovations in core methodologies. Investment in research and development is well served
by matching the design, capabilities, and development priorities of models and tools at
each level in Figure 6. In particular, new biophysical process understanding is best
developed within site-based models using field experiment data, particularly for under-
sampled agro-ecological zones, crop species, and farming systems under various
intensifications (Challinor et al., 2015; Maiorano et al., 2017). Networks and gridded
models gain from new datasets that allow extensive configuration for many sites and
systems, and have tremendous potential to apply advanced bias-correction and aggregation approaches (Challinor et al., 2014; van Bussel et al., 2016; Hoffmann et al., 2015, 2016; Zhao et al., 2015, 2016). Crop model emulators are progressed with improved statistical efficiencies and the availability of observed agricultural response data for evaluating strengths and weaknesses. In addition to the potential benefit of adding improved crop model emulators, IAM simulations of long-term shifts in agricultural production are furthered by good data on current systems and advanced representation of the implications of agricultural investment and technological development.

The AgMIP framework for improved agricultural representation in IAMs is non-linear as lower simulation levels build upon advances higher up in the framework and high levels also receive critical feedback from downstream simulation levels. Pathways of upstream improvements include that assessments of improved models on established grids and networks provide vital feedback for site-based model development on diverse sites. Likewise, emulators often spotlight key sensitivities and uncertainties that may spur further site-based model development and the creation of more representative networks. Network and gridded studies examine the biophysical viability of various simulated farm systems to determine land use pressures, but benefit tremendously by incorporating information on economic viability and resource constraints that IAMs can provide. It is also important to note that many of these simulation levels have extensive applications beyond agricultural representation in IAMs, and that the key bottleneck for one applications may differ from another's crucial development priority.
4. Priority Future Development and Applications

Analysis of the multi-model, multi-site climate sensitivity datasets reviewed in this study suggest that IAMs and other large-scale applications would be well served by the creation and systematic development of a hybrid CTWNA response system that blends the strengths of network and gridded approaches (as noted in Figure 6). This hybrid response system would be rooted in (1) detailed process understanding across a representative network of well-calibrated field sites (ideally using field data from prevailing management systems) combined with (2) comprehensive CTWNA coverage from gridded models. Baseline responses generated by these gridded models could initially be compared against the corresponding representative network simulations to assess methodological uncertainty and calculate bias-correction factors. Bias-corrected gridded results could then provide an information basis for crop model emulators and IAM applications, characterizing different farming systems using nitrogen and water components of the CTWNA analysis.

Table 1 highlights that progress toward the creation of this hybrid response system is most advanced for wheat, given the AgMIP-Wheat Phase 2b representative network and spring and winter wheat simulations within GGCMI Phase 2. In contrast, soybean is simulated in GGCMI but has not yet been the focus of site- or network-based CTWNA analysis, and a number of other important commodities merit inclusion. Coordinated and systematic development of the hybrid response system would foster rapid iterative improvements, as research groups improve the hybrid framework by contributing new process understanding, field sites, model runs, regional configuration information, or statistical approaches. An expanded representative network of models and a fully configured high-resolution gridded
(or geo-referenced polygon) model will eventually be interchangeable; however this hybrid
response system provides current state-of-the-art responses and a practical roadmap for
applications.

Coordination across AgMIP activities supports the development of linked global and
regional assessments to address agricultural sector challenges and food security
(Rosenzweig et al., 2016). Inclusion of IAMs would bring these to a new level, although
it is critical that these account for lingering model uncertainty and data gaps even as these
are addressed through the coordinated development of agricultural response in linked
models.

Acknowledgements

We are grateful for the tremendous effort and contributions provided by all modelers and
leaders of all projects listed in Table 1. This letter was motivated by discussions with
members of the Program on Integrated Assessment Model Development, Diagnostics, and
Intercomparisons (PIAMDDI), and the Pacific Northwest National Laboratories, and
benefitted from support by the National Aeronautics and Space Agency Science Mission
Directorate (WBS 281945.02.03.06.79) and the Department of Energy (DE-AC05-
76RL01830 and DE-SC0005171) as well as the Joint Research Centre of the European
Commission. We appreciate the assistance of Meridel Phillips in figure creation.
References

Antle J M, Valdivia R O, Boote K, Janssen S, Jones J W, Porter C H, Rosenzweig C, Ruane A C, Thorburn P J 2015 AgMIP's transdisciplinary agricultural systems approach to regional integrated assessment of climate impacts, vulnerability, and adaptation. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1. C. Rosenzweig, and D. Hillel, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 27-44, doi:10.1142/9781783265640_0002.

Asseng S et al 2013 Uncertainty in simulating wheat yields under climate change. Nature Clim. Change 3, 827-832. doi: 10.1038/NCLIMATE1916

Asseng S et al 2015a Rising temperatures reduce global wheat production. Nature Clim. Change, 5(2), 143-147, doi:10.1038/nclimate2470.

Asseng S et al 2015b Benchmark data set for wheat growth models: Field experiments and AgMIP multi-model simulations. Open Data J. Agric. Res., 1, 1-5, doi:10.18174/odjar.v1i1.14746.

Bassu S et al 2014 Do various maize crop models give the same responses to climate change factors? Global Change Biology 20, 2301-2320, doi: 10.1111/gcb.12520

Blanc E 2017 Statistical emulators of maize, rice, soybean and wheat yields from global gridded crop models. Agricultural and Forest Meteorology, 236, 145-161. doi:10.1016/j.agrformet.2016.12.022

Boote KJ, Porter C H, Jones J W, Thorburn PJ, Kersebaum KC, Hoogenboom G, White J W and Hatfield J L 2015 Sentinel Site Data for Model Improvement – Definition and Characterization. In: Hatfield, J.L., Fleisher, D. (Eds.), Improving Modeling Tools to Assess Climate Change Effects on Crop Response, Advances in Agricultural Systems Modeling. ASA, CSSA, and SSSA, Madison, WI, USA. doi:10.2134/advagricsystmodel7.2014.0019.

Cammarano D et al 2016 Uncertainty of wheat water use: Simulated patterns and sensitivity to temperature and CO2. Field Crops Res., 198, 80-92, doi:10.1016/j.fcr.2016.08.015.

Castañeda-Vera A, Leffelaar PA, Álvaro-Fuentes J, Cantero-Martínez C, Mínguez M I 2015 Selecting crop models for decision making in wheat insurance. Europ. J. Agronomy 68: 97-116. doi:10.1016/j.eja.2015.04.008

Castruccio S, McInerney DJ, Stein M L, Crouch F L, Jacob RL and Moyer E J 2014 Statistical Emulation of Climate Model Projections Based on Precomputed GCM Runs. Journal of Climate 27(5), 1829-1844.

Challinor A J et al 2014a A meta-analysis of crop yield under climate change and adaptation. Nature Climate Change 4(4), 287-291. doi:10.1038/NCLIMATE2153

Challinor A, Martre P, Asseng S, Thornton P and Ewert F 2014b Making the most of climate impacts ensembles. Nature Climate Change 4: 77-80.

Challinor A, Parkes B, and Ramirez-Villegas J 2015 Crop yield response to climate change varies with cropping intensity. Glob Change Biol, 21: 1679-1688. doi:10.1111/gcb.12808
Clarke L et al 2014 Assessing Transformation Pathways. In: Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change [Edenhofer, O., et al. (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.

Confalonieri R et al 2016 Uncertainty in crop model predictions: What is the role of users? Environmental Modelling and Software 81, 165-173.

Deryng Det al 2016 Regional disparities in the beneficial effects of rising CO2 concentrations on crop water productivity. Nature Clim. Change, 6, no. 8, 786-790, doi:10.1038/nclimate2995.

Durand J L et al 2017 How accurately do maize crop models simulate the interactions of atmospheric CO2 concentration levels with limited water supply on water use and yield? Eur. J. Agron., in press, doi:10.1016/j.eja.2017.01.002.

Elliott J, Kelly D, Chrysanthacopoulos J, Glotter M, Jhunjhnuwala K, Best N, Wilde M and Foster I 2014 The parallel system for integrating impact models and sectors (pSIMS). Environ. Model. Softw., 62, 509-516, doi:10.1016/j.envsoft.2014.04.008.

Elliott J et al 2015 The Global Gridded Crop Model Intercomparison: Data and modeling protocols for Phase 1 (v1.0). Geosci. Model Dev., 8, 261-277, doi:10.5194/gmd-8-261-2015.

Ewert F et al 2015 Crop modelling for integrated assessment of risk to food production from climate change. Environmental Modelling & Software 72: 287-303

Fleisher D H et al 2016 A potato model intercomparison across varying climates and productivity levels. Global Change Biology: 24 pp. doi: 10.1111/gcb.13411

Füssel H-M 2010 Modeling impacts and adaptation in global IAMs. WIREs Clim Chg, 1: 288-303. doi:10.1002/wcc.40

Hartin C A, Patel P L, Schwarber A, Link R P and Bond-Lamberty B 2015 A simple object-oriented and open source model for scientific and policy analyses of the global climate system - Hector v1.0. Geoscientific Model Development 8(4):939-955. doi:10.5194/gmd-8-939-2015

Hoffmann H et al 2015 Variability of effects of spatial climate data aggregation on regional yield simulation by crop models. Climate Research 65, 53-69.

Hoffmann H et al 2016 Impact of spatial soil and climate input data aggregation on regional Yield Simulations. PLoS ONE 11.

Kersebaum K C et al 2015 Analysis and classification of data sets for calibration and validation of agro-ecosystem models. Environmental Modelling & Software, 72, 402-417. DOI: 10.1016/j.envsoft.2015.05.009.

Kihara J et al 2015 Perspectives on climate effects on agriculture: The international efforts of AgMIP in Sub-Saharan Africa. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 2. Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 3-24, doi:10.1142/9781783265640_0013.
Kollas C et al 2015 Crop rotation modelling—A European model intercomparison. European Journal of Agronomy 70: 98-111

Li T et al 2015 Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Glob. Change Biol., 21, no. 3, 1328-1341, doi:10.1111/gcb.12758.

Liu B et al 2016 Similar negative impacts of temperature on global wheat yield estimated by three independent methods. Nature Clim. Change, 6, no. 12, 1130-1136, doi:10.1038/nclimate3115.

Lobell D B, Schlenker W and Costa-Roberts J 2011 Climate trends and global crop production since 1980. Science 333, 616-620.

Maiorano A et al 2017 Crop model improvement reduces the uncertainty of the response to temperature of multi-model ensembles. Field Crops Research 202: 5-20.

Makowski D et al 2015 Statistical analysis of large simulated yield datasets for studying climate effects. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1. C. Rosenzweig, and D. Hillel, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 279-298, doi:10.1142/9781783265640_0011.

Marin F R, Thorburn P J, Nassif D S P and Costa L G 2015 Sugarcane model intercomparison: Structural differences and uncertainties under climate change. Environmental Modelling and Software, 72, 372-386

Martre P et al 2014 Error of multimodel ensembles of wheat growth: more models are better than one. Global Change Biology, 21, 911-925, doi: 10.1111/gcb.12768.

McDermid S P et al 2015a The AgMIP Coordinated Climate-Crop Modeling Project (C3MP): Methods and protocols. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 191-220, doi:10.1142/9781783265640_0008.

McDermid S P et al 2015b Integrated assessments of the impacts of climate change on agriculture: An overview of AgMIP regional research in South Asia. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 2. Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 201-218, doi:10.1142/9781783265640_0018.

Meinshausen M., Raper S C B and Wigley T M L 2011 Emulating coupled atmosphere-ocean and carbon cycle models with a simpler model, MAGICC6—Part 1: Model description and calibration. Atmos Chem Phys 11:1417-1456

Mistry M, Sue Wing I, et al 2017 (article in review in same ERL special issue)

Moore F et al 2017 (article in review in same ERL special issue)
Mueller C and Robertson RD 2014 Projecting future crop productivity for global economic modeling. *Agric. Econ.* 45 37–50

Müller C et al 2017 Global Gridded Crop Model evaluation: Benchmarking, skills, deficiencies and implications. *Geosci. Model Dev.*, submitted, doi:10.5194/gmd-2016-207.

Nelson G C et al 2014 Climate change effects on agriculture: Economic responses to biophysical shocks. *Proceedings of the National Academy of Sciences*, 111, 3274-3279, doi: 10.1073/pnas.1222465110.

Pirttioja N 2015 A crop model ensemble analysis of temperature and precipitation effects on wheat yield across a European transect using impact response surfaces. *Climate Research* 65: 87–105. doi:10.3354/cr01322

Portmann F T, Siebert S and Döll P 2010 MIRCA2000 – Global monthly irrigated and rainfed crop areas around the year 2000: A new high-resolution data set for agricultural and hydrological modeling. *Global Biogeochemical Cycles*, 24, GB 1011, doi:10.1029/2008GB003435. Ray, D.K., J.S. Gerber, G.K. MacDonald, and P.C. West, 2015: Climate variation explains a third of global crop yield variability. *Nat Commun*, 6, doi: 10.1038/ncomms6989.

Rosenzweig C et al 2013 The Agricultural Model Intercomparison and Improvement Project (AgMIP): Protocols and pilot studies. *Agric. Forest Meteorol.* 170, 166-182. doi:10.1016/j.agrformet.2012.09.011

Rosenzweig C et al 2014 Assessing agricultural risks of climate change in the 21st century in a global gridded crop model intercomparison. *Proc. Natl. Acad. Sci.*., 111, 3268-3273, doi:10.1073/pnas.1222463110.

Rosenzweig C, Jones J W, Hatfield J L, Antle J M, Ruane A C and Mutter C Z, 2015: The Agricultural Model Intercomparison and Improvement Project: Phase I activities by a global community of science. In *Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP)*. Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 3-24, doi:10.1142/9781783265640_0001.

Rosenzweig C, Antle J and Elliott J 2016 Assessing impacts of climate change on food security worldwide. *Eos*, 97, no. 8, 11, doi:10.1029/2016EO047387.

Rosenzweig C et al 2017 Protocols for AgMIP Regional Integrated Assessments Version 7.0. available at http://www.agmip.org/regional-integrated-assessments-handbook/

Rötter R P, Carter T R, Olesen J E and Porter J R 2011 Crop-climate models need an overhaul. *Nature Climate Change*, 1(4): 175-177.

Ruane A C, McDermid S, Rosenzweig C, Baigorria G A, Jones J W, Romero C C and Cecil L D 2014 Carbon–Temperature–Water change analysis for peanut production under climate change: a prototype for the AgMIP Coordinated Climate-Crop Modeling Project (C3MP). *Global Change Biology*, 20, 394–407, doi: 10.1111/gcb.12412
Ruane A C, Goldberg R and Chryssanthacopoulos J 2015a Climate forcing datasets for agricultural modeling: Merged products for gap-filling and historical climate series estimation. *Agric. Forest Meteorol.*, 200, 233-248, doi:10.1016/j.agrformet.2014.09.016.

Ruane A C, Winter J M, McDermid S P and Hudson N I 2015b AgMIP climate datasets and scenarios for integrated assessment. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1. Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 45-78, doi:10.1142/9781783265640_0003.

Ruane A C et al 2016 Multi-wheat model ensemble responses to interannual climate variability. *Environ. Model. Softw.*, 81, 86-101, doi:10.1016/j.envsoft.2016.03.008.

Singels A, Jones M, Marin F, Ruane A C and Thorburn P 2013 Predicting climate change impacts on sugarcane production at sites in Australia, Brazil and South Africa using the Canegro model. *Sugar Tech*, doi: 10.1007/s12355-013-0274-1.

van Bussel L et al 2016 Spatial sampling of weather data for regional crop yield simulations. *Agricultural and Forest Meteorology* 220, 101-115.

Villoria N et al 2016 Rapid aggregation of global gridded crop model outputs to facilitate cross-disciplinary analysis of climate change impacts in agriculture. *Environmental Modelling & Software*, 75, 193-201, doi: 10.1016/j.envsoft.2015.10.016.

Wallach D, Mearns L O, Rivington M, Antle J M and Ruane A C 2015 Uncertainty in agricultural impact assessment. In Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). Rosenzweig C and Hillel D, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Imperial College Press, 223-260, doi:10.1142/9781783265640_0009.

Wallach D, Mearns L O, Ruane A C, Rötter R P, and Asseng S 2016 Lessons from the climate modeling community on the design and use of ensembles for crop modeling. *Climatic Change*, 139 (3), 551-564, doi:10.1007/s10584-016-1803-1.

Warszawski L, Frieler K, Huber V, Piontek F, Serdeczny O, Schewe J (2014) The Inter-Sectoral Impact Model Intercomparison Project (ISI–MIP): Project framework, P. Natl. Acad. Sci. USA, 111, 3228–3232, doi:10.1073/pnas.1312330110.

Webber H, Zhao G, Wolf J, Britz W, Vries W D, Gaiser T, Hoffmann H and Ewert F 2015 Climate change impacts on European crop yields: Do we need to consider nitrogen limitation? *European Journal of Agronomy* 71, 123-134.

Webber H, Gaiser T, Oomen R, Teixeira E, Zhao, Wallach D, Zimmermann A and Ewert F 2016 Uncertainty in future irrigation water demand and risk of crop failure for maize in Europe. *Environmental Research Letters* 11.
Webber H et al 2017 Canopy temperature for simulation of heat stress in irrigated wheat in a semi-arid environment: A multi-model comparison. Field Crops Research 202: 21-35.

White J W, Hoogenboom G, Kimball B A and Wall G W 2011 Methodologies for simulating impacts of climate change on crop production. Field Crops Research 124(3):357-368. doi: 10.1016/j.fcr.2011.07.001.

Wiebe K et al 2015 Climate change impacts on agriculture in 2050 under a range of plausible socioeconomic and emissions scenarios, Environmental Research Letters, 10, 085010, doi: 10.1088/1748-9326/10/8/085010.

Wilby R L, Charles S, Zorita E, Timbal B, Whetton P and Mearns L 2004 Guidelines for use of climate scenarios developed from statistical downscaling methods. IPCC Supporting Material, available from the DDC of IPPC TGCIA.

Yin X et al 2017 Multi-model uncertainty analysis in predicting grain N for crop rotations in Europe. European Journal of Agronomy 84: 152-165

Zhao G et al 2015 Effect of weather data aggregation on regional crop simulation for different crops, production conditions, and response

Zhao G et al 2016 Evaluating the precision of eight spatial sampling schemes in estimating regional means of simulated yield for two crops. Environmental Modelling and Software 80, 100-112.
## Tables and Figures

<table><caption>**Table 1:** Overview of multi-model, multi-site, protocol-based research activities sampling the Carbon-Temperature-Water-Nitrogen-Adaptation (CTWNA) change space by AgMIP and related projects.</caption><thead><tr><th>Research Activity</th><th>Scope</th><th>C (ppm)</th><th>T (°C)</th><th>W (%)</th><th>N</th><th>A</th><th>Notes</th></tr></thead><tbody><tr><td>AgMIP-Wheat Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>--</td><td>-50% to +50%</td><td>--</td><td>27 wheat models at each of four sites. #</td></tr><tr><td>AgMIP-Wheat Phase 2a</td><td>Sites</td><td>--</td><td>+0 to +16</td><td>--</td><td>--</td><td>--</td><td>30 wheat models simulated at two sites</td></tr><tr><td>AgMIP-Wheat Phase 2b</td><td>Global Network</td><td>--</td><td>+0 to +4</td><td>--</td><td>--</td><td>--</td><td>30 wheat models at 30 well-watered sites.</td></tr><tr><td>AgMIP-Wheat Phase 3</td><td>Global Network</td><td>360 to 550</td><td>+0 to +4</td><td>--</td><td>--</td><td>--</td><td>32 wheat models at 60 sites (water-limited and well-watered sites).</td></tr><tr><td>AgMIP-Maize Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>--</td><td>--</td><td>--</td><td>23 maize models at each of 4 sites</td></tr><tr><td>AgMIP-Maize Phase 2</td><td>Sites</td><td>387 and 550</td><td>--</td><td>RF and Irr.</td><td>--</td><td>--</td><td>21 maize models for Braunschweig, Germany, FACE site</td></tr><tr><td>AgMIP-Rice Phase 1</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>--</td><td>varied N</td><td>--</td><td>13 rice models at each of 4 sites. Two sites included N treatments ranging from 30-150 kg N/ha</td></tr><tr><td>AgMIP-Rice Phase 2</td><td>Sites</td><td>360 to 720</td><td>--</td><td>--</td><td>varied N</td><td>--</td><td>16 rice models at each of 2 FACE sites (Japan and China).**</td></tr><tr><td>AgMIP-Potato</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>-30 to +30</td><td>--</td><td>--</td><td>9 potato models at each of 4 sites</td></tr><tr><td>AgMIP-Canola</td><td>Sites</td><td>360 to 720</td><td>-3 to +9</td><td>-25 to +25</td><td>0% to 150% of obs</td><td>--</td><td>8 canola models at each of 7 sites</td></tr><tr><td>AgMIP-Sugarcane</td><td>Sites</td><td>350 to 750</td><td>-3 to +9</td><td>-30 to +30</td><td>--</td><td>--</td><td>2 sugarcane models at 7 Brazilian sites.</td></tr><tr><td>AgMIP-Livestock and Grasslands Phase 2</td><td>Sites</td><td>330 to 900</td><td>-1 to +8</td><td>-50 to +50</td><td>--</td><td>--</td><td>Common protocols for single model tests at 14 sites. 7 models contributed yield and GHG balance results.</td></tr><tr><td>AgMIP Regional Integrated Assessments</td><td>Sites</td><td>360 to 720</td><td>-2 to +8</td><td>-75 to +100</td><td>0 to 210</td><td>--</td><td>2 models each for 10 sites, multiple crops at many of the sites.</td></tr></tbody></table>
<table><tr><td></td><td></td><td></td><td></td><td></td><td>kg N/ha</td><td></td><td></td></tr><tr><td>C3MP</td><td>Global Network</td><td>330 to 900</td><td>-1 to +8</td><td>-50 to +50</td><td>--</td><td>--</td><td>1137 simulation sets in 56 countries; 18 crop species, 23 crop models</td></tr><tr><td>MACSUR-IRS Phase 1</td><td>Sites</td><td>--</td><td>-2 to +9</td><td>-50 to +50</td><td>--</td><td>--</td><td>26 wheat models at 4 sites in Europe.</td></tr><tr><td>MACSUR - Crop Rotation</td><td>Sites</td><td>374 and 550</td><td>--</td><td>--</td><td>100% and 50% of obs</td><td>--</td><td>15 models with and without crop rotations. CN sensitivity tests performed at Braunschweig, Germany, and N sensitivity at Thibie, France.</td></tr><tr><td>GGCMI-Phase 2</td><td>Global Grid</td><td>360 to 810</td><td>-1 to +6</td><td>-50 to +30 plus irrigated</td><td>10 to 200 kg N/ha</td><td>Fully reverse accelerated maturity</td><td>12 participating models. ** Includes no water stress test and no nitrogen stress test. Adaptation adjusts cultivars to maintain planting to maturity duration.</td></tr></table>

Notes: RF=Rainfed; Irr.=Irrigated; GHG=Greenhouse Gas; varied N = multiple nitrogen treatments at each site; #= Nitrogen tests were only performed for 20 wheat models containing nitrogen dynamics; ** = ongoing project, final participation may change.
<table><caption>**Table 2:** AgMIP-Wheat, AgMIP-Maize, and AgMIP-Rice Team Phase descriptions.</caption><thead><tr><th>Phase (and key references)</th><th>Description</th></tr></thead><tbody><tr><td><strong>AgMIP-Wheat</strong></td><td></td></tr><tr><td><strong>Phase 1</strong><br>(Asseng et al., 2013;<br>Martre et al., 2015)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>Phase 2a</strong><br>(Asseng et al., 2015a)</td><td>Protocol-based multi-model analysis of temperature response at Hot Serial Cereals artificial heating experiment in Arizona and temperature responses in Mexico.</td></tr><tr><td><strong>Phase 2b</strong><br>(Asseng et al., 2015a;<br>Liu et al., 2016)</td><td>Intercomparison of temperature responses across 30 sites selected as a representative network of well-watered wheat production regions around the world.</td></tr><tr><td><strong>Phase 3</strong></td><td>Intercomparison of temperature responses across 60 sites selected to represent both well-watered and water-limited wheat production regions around the world.</td></tr><tr><td><strong>AgMIP-Maize</strong></td><td></td></tr><tr><td><strong>Phase 1</strong><br>(Bassu et al., 2014)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>Phase 2</strong><br>(Durand et al., 2017)</td><td>Protocol-based multi-model intercomparison at Free-Air Carbon Enrichment (FACE) site in Germany.</td></tr><tr><td><strong>AgMIP-Rice</strong></td><td></td></tr><tr><td><strong>AgMIP-Rice Phase 1</strong><br>(Li et al., 2015)</td><td>Protocol-based multi-model intercomparison at diverse, high-quality sites. Included limited information and full information calibration settings.</td></tr><tr><td><strong>AgMIP-Rice Phase 2</strong></td><td>Protocol-based multi-model intercomparison at FACE Sites in Japan and China.</td></tr></tbody></table>
<table><caption>**Table 3:** Overview of GGCMI phases and model participation.</caption><thead><tr><td>Phase (and key references)</td><td>Description [and # of models participating]</td></tr></thead><tbody><tr><td>Fast Track<br/>Rapid Assessment<br/>(Rosenzweig et al., 2014)</td><td>Conducted for AgMIP/ISIMIP using default versions of global gridded crop models, historical period and future scenarios from downscaled GCMs. Simulated maize, wheat, rice and soybean [7 GGCMs]</td></tr><tr><td>Phase 1<br/>Historical Intercomparison<br/>(Elliott et al., 2015; Müller et al., 2017)</td><td>Default, harmonized, and No Nitrogen Stress versions of gridded crop models run over historical period using up to 9 climate forcing datasets. Simulated maize, wheat, rice and soybean [15 GGCMs]</td></tr><tr><td>Phase 2<br/>CTWNA Sensitivity<br/>[results submitted 2016-17]</td><td>Default simulations for historical period and sensitivity tests for [CO<sub>2</sub>], temperature, water, nitrogen, and adaptation for all grid cells and crops. Simulated maize, spring wheat, winter wheat, rice and soybean [~12 GGCMs]</td></tr><tr><td>Phase 3<br/>Future Assessment<br/>[planned for 2017-18]</td><td>Conducted for AgMIP/ISIMIP to assess future agricultural production under climate change scenarios. [~12-20 GGCMs anticipated]</td></tr></tbody></table>
<figure><img src="image_1.png" /></figure>

**Figure 1:** Overview of aspirational framework and agricultural applications for IAM with linked climate and crop model emulators. IAMs typically focus on the interplay of socioeconomic development and environmental outcomes, however inclusion of the climate and crop model emulation pathway allows for the resolution of agricultural feedback loops and unintended consequences across scales and sectors. Note that climate emulators have additional applications within IAMs beyond the agricultural sector illustrated here.
<figure><img src="image_2.png" /><figcaption>**Figure 2:** Map of sites and networks for agricultural impacts studies exploring responses to [CO₂], temperature, water, nitrogen, and/or adaptation, and major crops area (%) by Monfreda et al. (2007); note that studies cover many major production regions, while GGCMI activities simulate the entire land surface.</figcaption></figure>
<figure><img src="image_3.png" /></figure>

**Figure 3:** Distribution of global (a) rainfed and (b) irrigated wheat area displayed according to growing season mean temperature and precipitation rate (from MIRCA observations, Portmann et al., 2010). Corresponding C3MP and AgMIP-Wheat Phase 2b network sites are presented to show coverage of global wheat-cropping systems. Note that the AgMIP-Wheat 2b network consists entirely of well-watered sites including both irrigated croplands and rainfed areas that rarely experience water stress.
<figure><img src="image_4.png" /><figcaption>**Figure 4:** Example of regional differences in climate response. pDSSAT rainfed maize yield change (%) to a hypothetical 150ppm increase in [CO₂] and a 2 °C rise in temperature from GGCMI Phase 2 (all regions grown with 200 kg N/ha with no adaptation to isolate climate response).</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Figure 5:** Wheat growing seasons (average planting and harvest date) at each well-watered site in the AgMIP-Wheat network as well as corresponding grid cells in the harmonized GGCMI protocols. Differences for Canada, Turkey, and Ukraine indicate that GGCMI considered spring wheat while the wheat network considered winter wheat, while India and Mexico reflect that there are two wheat-growing seasons in these regions.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Figure 6:** AgMIP framework for improved agricultural representation in IAMs. The core agricultural response development and application pathway (green arrows) spans several levels of model applications (dark blue boxes) and recognizes that site-based crop models are the backbone of model networks and grids, which feed into IAMs either directly or through crop model emulators built upon a hybrid system blending network and gridded CTWNA responses. Improvement in each level of model development requires access to data for evaluation and configuration (gray boxes) as well as methodological advances (light blue boxes). Agricultural applications also inform development up the framework chain, with IAMs providing critical information about the economic viability of changing land use patterns, emulators helping to isolate aggregate CTWNA responses, and networks and grids testing site-based models in more diverse settings.</figcaption></figure>