Model-data comparison challenges in paleo-climate analyses: Towards an evaluation toolbox for transient climate model simulations&#160;
Authors: Baudouin Jean-Philippe , Bothe Oliver , Chevalier Manuel , Weitzel Nils , Dallmeyer Anne , Brierley Chris , Rehfeld Kira 
Published: March 2021

&lt;p&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;Modelling studies are evaluated by comparing the simulation outputs to an observational reference. In climate science, the number and complexity of the models and the mass of data have led the community to develop standardised methods and automated tools, such as the Climate Variability Diagnostic Package or the&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;ESMValTool&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;. However, these tools are mostly&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;designed&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;&amp;#160;to evaluate simulations of the instrumental period. Different methods are required to compare paleoclimate simulations to&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;pal&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;a&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;eodata&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;.&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;For example, new variables are being modelled, such as&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;vegetation&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;, ice sheet extent, or isotopic ratio, and are used for the evaluation. Changing boundary conditions in transient simulations&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;further complicate the evaluation process: traditional indices that characterise circulation (e.g. monsoon) or modes of variability (e.g. NAO, ENSO) need to be adapted, while new ones are needed to investigate modes of longer timescale and abrupt events. Finally, t&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;he&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;palaeodata&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;&amp;#160;also present challenges: various type of uncertainty, complex relation to climate variables, and different&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;spatio&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;-temporal representativeness compared to model outputs. Here, we summarise the challenges of model-data comparison&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;in paleo-climate studies. We then&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;r&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;eview some of the different methods and tools&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;already developed by the community&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;, such as biome comparison and Bayesian&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;approaches&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;&amp;#160;to&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;quantify&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;&amp;#160;model-data deviation&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;. We&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;finally&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;discuss&amp;#160;&lt;/span&gt;&lt;/span&gt;&lt;span xml:lang=&quot;EN-GB&quot;&gt;&lt;span&gt;the implementation of an evaluation framework which aims to provide both adaptable tools to the community and automated standardised analyses.&lt;/span&gt;&lt;/span&gt;&lt;/p&gt;