# Improving the usability of climate services for the water sector: The AQUACLEW experience

Rafael Pimentel, Christiana Photiadou, Lorna Little, Andreas Huber, Anthony Lemoine, David Leidinger, Andrea Lira-Loarca, Johannes Lückenkötter, Ernesto Pasten-Zapata

<figure><img src="image_1.png" /></figure>

## To cite this version:

Rafael Pimentel, Christiana Photiadou, Lorna Little, Andreas Huber, Anthony Lemoine, et al. Improving the usability of climate services for the water sector: The AQUACLEW experience. Climate services, 2022, 28, pp.100329. 10.1016/j.cliser.2022.100329. hal-03990745

HAL Id: hal-03990745

https://hal.inrae.fr/hal-03990745

Submitted on 15 Feb 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

Check for
updates

# Improving the usability of climate services for the water sector: The AQUACLEW experience

Rafael Pimentel<sup>a, b, *</sup>, Christiana Photiadou<sup>c, d</sup>, Lorna Little<sup>c</sup>, Andreas Huber<sup>e</sup>, Anthony Lemoine<sup>f</sup>, David Leidinger<sup>g</sup>, Andrea Lira-Loarca<sup>h</sup>, Johannes Lückenkötter<sup>i</sup>, Ernesto Pasten-Zapata<sup>j, k</sup>

## ARTICLE INFO

### Keywords:

- Benchmarking
- Climate projection
- Climate Services
- User Engagement
- Water Sector

## ABSTRACT

In AQUACLEW (Advancing the QUALity of CLimate services for European Water), a project funded by JPI Climate and the ERA-NET Consortium 'European Research Area for Climate Services (ERA4CS)', we examined different ways of improving the usability of existing Climate Services across Europe tackling key aspects in Climate Services improvement: user engagement, lack of resolution, uncertainties, and the need of an evaluation. The rationale of the project is based on an interactive process between service developers and users in seven study cases across Europe assessing the implications of Climate Service's advancement in users' decision-making process. A qualitative evaluation assessment allowed us to identify four pillars when improving the quality of the climate services in the water sector: (1) Robustness, accounting for better quality of the service's information in certain aspects; (2) Recruitment, understood as a need of involving users more actively in CS structures; (3) Reform, highlighting the possible need for changes in both the service structure and users mindsets; and (4) Reflection, as a process of continuous evaluation of the climate service during its life.

## Introduction

### Practical implications

This study presents the experience gained in the AQUACLEW project, which had as main purpose to improve the quality of already existing Climate Services (CS) in the water sector. The need of improving instead of developing new CS has been a recurrent topic in recent years. The enormous effort carried out from different funding agencies,

organisations, and international entities promoting and funding CS has not been equally reflected in their posterior usability. Therefore, why not identify weaknesses on existing CS and improve them? Previous studies pointed out some recurrent issues in CS as the main obstacles in CS' uses: (i) the low level of user engagement in the co-development process, (ii) the low resolution in CS' information and the lack of appropriate method for tailoring, (iii) the uncertainty in climate information cascading in impact assessments, and (iv) the lack of an active evaluation. Using these challenges as the starting point, this work

<footnote><sup>a</sup> Fluvial Dynamics and Hydrology Research Group, Andalusian Institute for Earth System Research, Edificio Leonardo Da Vinci, Área de Ingeniería Hidráulica, Campus de Rabanales, University of Cordoba, 14014 Córdoba, Spain</footnote>

<footnote><sup>b</sup> Department of Agronomy. Unit of Excellence María de Maeztu (DAUCO) University of Córdoba, Spain</footnote>

<footnote><sup>c</sup> Hydrology Research Unit, Swedish Meteorological and Hydrological Institute, Norrköping, Sweden</footnote>

<footnote><sup>d</sup> European Environment Agency, Kongens Nytorv 6, 1050 Copenhagen, Denmark</footnote>

<footnote><sup>e</sup> Unit of Hydraulic Engineering, Institute for Infrastructure Engineering, University of Innsbruck, 6020 Innsbruck, Austria</footnote>

<footnote><sup>f</sup> Université Paris-Saclay, INRAE, UR HYCAR, 92160 Antony, France</footnote>

<footnote><sup>g</sup> Institute of Meteorology and Climatology, University of Natural Resources and Life Sciences (BOKU), Vienna, Austria</footnote>

<footnote><sup>h</sup> Environmental Fluid Dynamics Research Group, Andalusian Institute for Earth System Research, University of Granada, Avda. del Mediterráneo s/n, 18006 Granada, Spain</footnote>

<footnote><sup>i</sup> Institute of Spatial Planning, Faculty of Spatial Planning, TU Dortmund University, Germany</footnote>

<footnote><sup>j</sup> Department of Hydrology, Geological Survey of Denmark and Greenland, Copenhagen, Denmark</footnote>

<footnote><sup>k</sup> Department of Geographical and Historical Studies, University of Eastern Finland, Joensuu, Finland</footnote>

<footnote>* Corresponding author.
E-mail address: rpimentel@uco.es (R. Pimentel).</footnote>
proposes and assesses the applicability of a general framework for improving the quality of already existing CS. The proposed framework is based on the premise that there is room from improvement in some already existing CS with users willing to enhance their involvement in a co-development process. We take advantage of the plethora of CS in the water sector to test the proposed framework, since the services developed in this sector tackle interlinked issues and bring together users of different profiles. Seven study cases in the water sector across Europe with different developers and users were chosen to improve the existing CS through an advanced co-development process and to elaborate on the applicability of the proposed framework.

In terms of practical implications, this study has identified four pillar to tackled when improving CS. We have called them the 4Rs:

1. **Robustness:** understood as the need of accounting for improved methods and data, which can be updated with the most current scientific advancements, hence providing better quality of information. This potentially improves users' trustiness in the CS, especially when the advancement has tackled three of the main challenges: quality of data, uncertainty assessment and improved resolution.

2. **Recruitment:** understood mainly as user active engagement, highlighting the need of involving users more actively in CS structures. In addition, by recruitment we refer to the need of including multiple users who might hold a key position in the governance mechanism of an organisation and play a role in the actual decision-making process

3. **Reform:** understood mainly as the need to adapt the CS structure for a successful implementation of new methodologies and data. The notion could also reflect the need of broadening the mindsets of users and developers to beneficial but difficult to implement changes.

4. **Reflection:** understood as a process of continuous evaluation during the life of the CS, that could help to identify the key aspect that have contributed and limited this improvement.

During the implementation of the seven use cases the proposed framework was useful to improve the usability of already existing CS tackling and overcoming the water issues in decision-making. However, this approach has shown a very demanding interaction with a very exclusive target-oriented user. Hence, despite their potential, it might not be feasible to be widely applied due to the need of knowledge developers for a limited number of users. Therefore, a substantial effort should be made in educational aspects to develop deeper user knowledge and reduce the role of service developers acting as knowledge purveyors.

## Improving the state-of-the-art in climate services

An enormous effort from different funding agencies, organisations, and international entities has been made in the last decades to connect climate and society by promoting and funding Climate Services (CS) (Jones et al., 2014). Originally, CS were intended to provide users with timely and tailored climate-related knowledge and information for decision making processes (Vaughan et al., 2016). Nowadays, despite the goal of CS evolving into a co-creation mechanism, most CS still lack this feature in their development. Therefore, there is an increasing call for co-creation in the CS community, that build the bridge between users' need and climate information and provide data, tools, and analysis to facilitate mitigation and adaptation strategies for different scales and sectors (Street, 2016).

The success of a CS is usually measured by their actual usability (Larosa & Mysiak, 2020). This is still considered insufficient in most of the cases (Hewitson et al., 2017; Klein & Juhola, 2014), particularly for completing 'the last mile' where complex climate science data and information become relevant for local adaptation and planning (Celliers et al., 2021). Several studies have pointed to the main obstacles in these achievements (Brasseur & Gallardo, 2016; Vaughan et al., 2016; Hewitt et al., 2017; Photiadou et al., 2021):

**_The low level of user engagement in the co-development process._** Two factors are usually identified as main causes of a lack of continuous interaction between service developers and users during the co-development of the CS, where collaboration, common understanding, and trust should be established. Firstly, users are not familiar enough with climate information and consequently have a low level of awareness regarding climate change and future scenario changes (Vincent et al., 2017). Then, the difficulties of covering a wide range of users' requirements within the same CS usually dilutes users' demands in favour of a compromise solution and causes a loss of interest in the service by some of the users (Vaughan et al., 2016). For instance, CS under the water sector cover a large range of issues (e.g., water allocation, hydropower production, farmers crop management, biodiversity changes linked with water, or urban flooding) that make tailored CS necessary in order to adequately cover all requirements. Moreover, this heterogeneity can be extrapolated also to users' profiles, who are often connected to different technical standards and directives (Bessembinder et al., 2019). These facts hamper the co-development process and imply the need of having a sufficient heterogeneous and representative group of users (Buontempo et al., 2014) with a continuous dialogue, in which consensus between actors must be achieved (Dessai & Hulme, 2004). This task can be difficult to reach especially when experiencing user fatigue from superficial engagement (Vogel et al., 2019).

**_The low resolution in CS' information and lack of appropriate method for tailoring._** The large spatial resolution developed at global or continental scales and used by CS, makes the climate information provided not immediately applicable for decision-making. Although the spatial resolution of climate models and projections has increased from around 100–50 km to 25–10 km (Jacob et al., 2020) in recent decades, this higher resolution is still insufficient for local applications or is not in the appropriate form. Thus, users are reluctant to justify changes in their decision-making process based on low-resolution information (Dessai et al., 2005). Additionally, in impact modelling in the water sector, hydrological models usually require climate data inputs at a higher resolution than that provided by climate models. Different bias correction and statistical downscaling methods can adjust climate model data to solve this scale issue; however, the methods could introduce changes in the analysed climate signal if the method does not account for that specific signal. Furthermore, these general methodologies are applied without a deep local analysis, which is usually necessary for impact assessments (Maraun et al., 2015).

**_The uncertainty in climate information cascading in impact assessments._** Quality assured climate information tailored to specific needs, sectors and regions is a primary ingredient for successful decision-making (Viel et al., 2016; Webber & Donner, 2017; Ernst et al., 2019; Merks et al., 2020; Photiadou et al., 2021). Service developers often endeavour to reduce the uncertainties in data and increase the robustness of the information. Regarding quality of data, uncertainty emerges as a crucial fact to consider. The climate community uses the spread of ensembles from different simulations to quantify and communicate uncertainty. In impact assessments in the water sector, this methodology is also applied, and relevant impact indicators are usually provided by the CS. When the CS covers a large area, which is common, the hydrological models producing this data usually sacrifice the local data quality in exchange for better performance across a large domain (Donnelly et al., 2016; Arheimer et al., 2020; Merks et al., 2020). This often leads to biases between the hydrological ensemble means and the observations, increasing thus the levels of uncertainty initially provided. It is considered difficult to judge the overall reliability of individual models or projections (and often not scientifically appropriate) but allowing a case-based tailored selection increases the usefulness of the CS in real-life decision-making.

**_The lack of an active evaluation._** Vincent et al., (2018) highlight that the co-development process is lacking a 'golden recipe', specially it does not vouch for an active evaluation when the CS design is concluded. The lack of evaluation metrics or processes that help to assess the success of
<figure><img src="image_5.png" /><figcaption>Fig. 1. Scheme of the interactive process carried out in the AQUACLEW project.</figcaption></figure>

<table><caption>**Table 1**
Summary of the meetings carried out in the step a) Water issue definition.</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th><th>Case 4</th><th>Case 5</th><th>Case 6</th><th>Case 7</th></tr></thead><tbody><tr><td>Participants<br>Organization<br>(no. of<br>participants)</td><td>SMHI (2)/The<br>County<br>Administrative<br>Board of<br>Jönköping (4)</td><td>GEUS (3)/Senior<br>specialists from<br>SEGES and Central<br>Denmark Region (3)</td><td>TUDO (2)/City<br>of Hagen (4)</td><td>INRAE (2)/EDF<br>(4)</td><td>BOKU-University of<br>Innsbruck (3)/Federal<br>government of Upper<br>Austria – Municipality<br>of Schwertberg (2)</td><td>UCO (2)/Coast of<br>Granada<br>Municipalities –<br>Endesa Hydropower –<br>Regional Water<br>Planning Office (5)</td><td>UGR (2)/Andalusian<br>Regional<br>Government –<br>Provincial Coastal<br>Service – Port<br>Authority (4)</td></tr><tr><td>Meetings<br>Number</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Timing*</td><td>Months 1 and 3</td><td>Months 1 and 2</td><td>Months 2 and 3</td><td>Months 2 and 3</td><td>Months 2 and 3</td><td>Months 1 and 4</td><td>Months 1 and 3</td></tr><tr><td>Type</td><td>Online meeting/<br>telephone call</td><td>In person/Online<br>meeting</td><td>In person/<br>Online meeting</td><td>In person/<br>Online meeting</td><td>In person/<br>Online meeting</td><td>In person/Online<br>meeting</td><td>In person/Online<br>meeting</td></tr><tr><td>Interaction<br>Type</td><td>Brainstorming<br>dynamic/<br>Discussion</td><td>Brainstorming/<br>dynamic/<br>Discussion</td><td>Brainstorming<br>dynamic/<br>Discussion</td><td>Brainstorming<br>dynamic/<br>Discussion</td><td>Brainstorming<br>dynamic/Discussion</td><td>Brainstorming<br>dynamic/Discussion</td><td>Brainstorming<br>dynamic/Discussion</td></tr></tbody></table>

an already running CS has been widely discussed in the literature (McNie, 2012; Brooks, 2013; Vogel et al., 2017; Vaughan et al., 2019). An active evaluation will help CS providers to track performance, identify and evaluate processes that need refining, measure impact, set goals, and inform stakeholders (Brooks, 2013). However, their definition faces several problems regarding the qualitative nature of the facts to assess with numerous intangible factors that can influence this success.

All these identified limitations can be considered not only when co-developing new CS, but also, they can be seen as aspects to account for

when improving already existent CS. This paper proposes and assesses a general framework for improving the quality of already designed CS in the water sector through an advanced co-development process between the two main agents involved: developers and users. This was the main goal of AQUACLEW (Advancing the QUALITY of CLimate services for European Water), a project funded by JPI Climate and the ERA-NET Consortium 'European Research Area for Climate Services (ERA4CS).

<table><caption>**Table 2**
Summary of the meetings carried out in the step b) Benchmark selection.</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th><th>Case 4</th><th>Case 5</th><th>Case 6</th><th>Case 7</th></tr></thead><tbody><tr><td>Participants<br>Organization</td><td>The County<br>Administrative<br>Board of Jönköping</td><td>Senior specialists<br>from SEGES and<br>Central Denmark<br>Region</td><td>City of Hagen</td><td>EDF</td><td>Federal government<br>of Upper Austria –<br>Municipality of<br>Schwertberg</td><td>Coast of Granada<br>Municipalities – Endesa<br>Hydropower – Regional<br>Water Planning Office</td><td>Andalusian Regional<br>Government –<br>Provincial Coastal<br>Service – Port<br>Authority</td></tr><tr><td>Interaction<br>Type</td><td>Online<br>questionnaire</td><td>Online<br>questionnaire</td><td>Online<br>questionnaire</td><td>Online<br>questionnaire</td><td>Online questionnaire</td><td>Online questionnaire</td><td>Online questionnaire</td></tr><tr><td>Responses</td><td>1</td><td>1</td><td>1</td><td>1</td><td>2</td><td>3</td><td>3</td></tr><tr><td>Meetings<br>Number</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>Timing*</td><td>Month 4</td><td>Month 4</td><td>Month 4</td><td>Month 4</td><td>Month 5</td><td>Month 5</td><td>Month 5</td></tr><tr><td>Type</td><td>In person</td><td>In person</td><td>In person</td><td>In person</td><td>In person</td><td>In person</td><td>In person</td></tr></tbody></table>

<footnote>*The months are referred to the beginning of the project.</footnote>

<footnote>*The months are referred to the beginning of the project.</footnote>
<table><caption>**Table 3**
Summary of the meetings carried out in the step c) Co-development strategy and d) Analysis of the advancement.</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th><th>Case 4</th><th>Case 5</th><th>Case 6</th><th>Case 7</th></tr></thead><tbody><tr><td>Participants<br>Organization<br>(no. of<br>participants)</td><td>SMHI (2)/The<br>County<br>Administrative<br>Board of Jönköping<br>(4)</td><td>GEUS (3)/Senior<br>specialists from<br>SEGES and<br>Central Denmark<br>Region (3)</td><td>TUDO (2)/City<br>of Hagen (4)</td><td>INRAE (2)/EDF<br>(4)</td><td>BOKU-University of<br>Innsbruck (3)/Federal<br>government of Upper<br>Austria - Municipality<br>of Schwertberg (2)</td><td>UCO (2)/Coast of<br>Granada Municipalities<br>- Endesa Hydropower -<br>Regional Water<br>Planning Office (5)</td><td>UGR (2)/Andalusian<br>Regional<br>Government -<br>Provincial Coastal<br>Service - Port<br>Authority (4)</td></tr><tr><td>Meetings<br>Number<br>Timing*</td><td>3<br>Months 4, 16 and 26</td><td>3<br>Months 5, 20 and<br>28</td><td>4<br>Months 5, 12,<br>23 and 28</td><td>5<br>Months 4, 16,<br>20, 24 and 28</td><td>4<br>Months 5, 18, 23 and<br>27</td><td>4<br>Months 5, 12, 23 and<br>28</td><td>4<br>Months 5, 13, 23 and<br>28</td></tr><tr><td>Type</td><td>In person/Online<br>meeting/Online<br>meeting</td><td>In person/In<br>person/Online<br>meeting</td><td>In person/In<br>person/Online<br>meeting/<br>Online meeting</td><td>In person/In<br>person/Online<br>meeting/Online<br>meeting</td><td>In person/Online<br>meeting/Online<br>meeting/Online<br>meeting</td><td>In person/In person/<br>Online meeting/Online<br>meeting</td><td>In person/In person/<br>Online meeting/Online<br>meeting</td></tr><tr><td>Interaction<br>Type</td><td>Interactive<br>Discussion</td><td>Interactive<br>Discussion</td><td>Interactive<br>Discussion</td><td>Interactive<br>Discussion</td><td>Interactive Discussion</td><td>Interactive Discussion</td><td>Interactive<br>Discussion</td></tr></tbody></table>

## Methodology

### The AQUACLEW rationale

The rationale of the proposed framework is based on the initial premise that there is room for improvement in some already existing CS. To identify these needs users in the water sector, who have previous experience using climate data were chosen as testers of the framework. Seven study cases across Europe were selected. The proposed framework was divided in five steps (Fig. 1):

a) Water issue definition, the identification of an actual problem to be solved or a question to be answered is key for users' involvement. In addition to that, it is also key to understand the limitation of the information they are currently using and to value the advancement in decision making that can be achieved at the end of the process. This actual water issue to be solved was identified by users thanks to two initial meetings between users and developers. The first meeting consisted of a brainstorming of ideas (e.g., How do you used climate information in your decision-making? What can be done for using this information in other aspects of your decision-making process?) The second one in a refinement of these ideas and in the selection of an actual water issue (Table 1).

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Evaluation of the proposed framework assessing: (i) benchmark and advancement in CS, (ii) impact on decision-making and (iii) assessment of change.</figcaption></figure>

<footnote>*The months are referred to the beginning of the project.</footnote>
<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Case studies carried out in the project. Each box summarises the users involved (dark grey, bottom left), resolution required (blue, bottom right), water issue (white, top left) and the targeted advancement (light grey, top right). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

b) **Benchmark selection**, once the water issue was chosen, users were asked to define trough an online questionnaire: the way this issue was currently tackle, the climate information they currently use, the CS that provides this information and the limitations they find to solved it using this information. This information was used to define the CS' current status as a benchmark to start to work with, all possible improvements were compared to this reference (Table 2).

c) **Co-development strategy**, in this stage in an initial meeting, the defined benchmark was finally agreed, and a work plan was designed together, - with the aim of carry out an advancement in the selected benchmark. To guarantee the continuous interaction between service developers and the users along the workplan, continuous meetings were carried out. These meetings were intended to show developers progresses and the level of success achieved from the users' perspective. The number of meetings varied depending on the study case, but they were never lower than 3, including the initial one on this stage, during the implementation of the improvement (Table 3).

d) **Analysis of the advancement**, the quality of the new advancements was assessed comparing them to the benchmark initially defined. This analysis was carried as part of the co-development strategy defined in the previous paragraph. An iterative process was developed until reaching the targeted aim. Through this participatory process, different issues were tackled regarding some key aspects previously identified: user engagement, lack of resolution, uncertainties, and the need of an evaluation.

e) **Decision-making implementation**, this step was intended to occur by the user in their decision-making process. However, this was not feasible within the duration of the project.

## Qualitative evaluation to advance gained knowledge

To assess the proposed framework, for instance the improvements performed in comparison to the initial benchmark and the impact of alternating the decision-making process, we designed a qualitative approach in the questionnaire format. Each use case had to be discussed and answer the questionnaire during an online meeting held between
<caption>**Table 4**</caption>

Summary of answers to the first block of question in the qualitative assessment: Benchmark and advancement in CS.

<table><caption>**Table 4 (continued)**</caption><thead><tr><th></th><th>What was the benchmark in your case study?</th><th>What were the new methods used?</th><th>What were the comparison between existing and new methods?</th><th>Which innovations and advancements were achieved?</th></tr></thead><tbody><tr><td>Case 1</td><td>The clients used the information provided by the national CS (50x50 km CORDEX climate scenarios, downscaled and bias adjusted using DBS), which was used for calculating hydrological indicators for future scenarios using the hydrological model S-HYPE</td><td>The new methods use higher resolution for climate data (11x11 km EURO-CORDEX). In addition, the new methods include the development of a new technique for downscaling and bias adjustment and the use of E-HYPE as hydrological model.</td><td>The main comparison was done between the hydrological indicators calculated by the pan-European model E-HYPE, forced by EURO-CORDEX 11x11km and the local model S-HYPE forced by CORDEX 50x50km</td><td>The comparison demonstrates the areas for potential improvements and allows to do a deeper regional analysis at the county scale, anticipating for local climate regimes, and elaborating on future changes.</td></tr><tr><td>Case 2</td><td>The national CS used by the clients provides information based on three scenarios: wet, dry, and middle. However, it does not provided information on the uncertainty of the projection neither on seasonal changes in extreme values.</td><td>The new methods include the use of bias-corrected climate information using a distribution-based scaling form 16 EURO-CORDEX. This analysis also incorporates an assessment on model uncertainty in three future periods.</td><td>The main comparison was done between previous projection, which were just an average value, and new projection, which includes a band of uncertainty.</td><td>The most important achievement is the use of an ensemble of climate models to assess the climate model uncertainty. In addition to the application of detailed evaluation metrics that were relevant for users' involvement.</td></tr><tr><td>Case 3</td><td>The city of Hagen used climate data from DWD and from 12 weather stations for defining flooding risk maps. They apply a flat 10 % climate change surcharge in heavy precipitation based on an extrapolation of the data of the last 15 years.</td><td>The new methods include the use of an adjusted statistical scaling technique that analyses historical rainfall data on heavy rainfall days. This was applied to derive future precipitation intensities on heavy rainfall days based on temperature scenarios. These maps are the inputs in flood risk map definition.</td><td>The main comparison was done between previous and new flooding maps, with higher similarities in recent future scenarios but with significant changes at the end of the century.</td><td>The most important achievement is the incorporation of future climate data using actual climate change scenarios.</td></tr><tr><td>Case 4</td><td>The CS used by the users provide key information to optimize reservoir</td><td>The new methods have developed an energy indicator based on existing CS and</td><td>The main comparison was done between both old and new indicators.</td><td>The most important achievement is that the new indicator developed</td></tr></tbody></table>

<table><thead><tr><th></th><th>What was the benchmark in your case study?</th><th>What were the new methods used?</th><th>What were the comparison between existing and new methods?</th><th>Which innovations and advancements were achieved?</th></tr></thead><tbody><tr><td></td><td>operations for both hydropower production and water storage management. However, energy aspects are sometimes missing in these assessments.</td><td>including future demand on electricity and prices for facilitating decision-making, foreseeing developments, and adapting the demands at the regional and local levels</td><td>The new indicator allows comparing historic and future periods of hydro-climate conditions.</td><td>considers the potentially available water volumes to reservoir optimization and hydropower production and gives an indication of how risk-based reservoir management rules can potentially be impacted. The most important achievement is the incorporation of climate change data in the definition of pluvial flash flood.</td></tr><tr><td>Case 5</td><td>The government authorities who are working to manage the risks of pluvial flash floods, based their assessment on the national design precipitation values, which does not include projected climate changes.</td><td>The new method includes the use of pan-European CS information in the assessment of potential pluvial flash flood impacts under future climate change applying change rates for sub-daily precipitation and relationships between precipitation and temperature.</td><td>The main comparison was done between previous indicators, which were just an average value, and new projection, which includes a band of uncertainty.</td><td>The most important achievements are the fact of accounting for uncertainty linking pan-European and local scales.</td></tr><tr><td>Case 6</td><td>The different users in the area used the information provided by national CS, which did not include a proper uncertainty quantification over complex terrain led to users being reluctant to use it.</td><td>The new method accounts for uncertainty using stochastic simulations over streamflow projection for assessing the provision of water allocation in the study basin and allowing deeper insight to the potential future seasonal regime.</td><td>The main comparison was done between previous indicators, which were just an average value, and new projection, which includes a band of uncertainty.</td><td>The most important result is the inclusion of climate change aspects which were not being used for any planning strategies by the users in their corresponding management issues.</td></tr><tr><td>Case 7</td><td>The data used were lacking local high-resolution bias-corrected ensemble of climate projections, which implies that future assessments need to be based on predefined storm shapes.</td><td>The new method allows an adequate reproduction of random temporal evolution of storm events, without predefined geometries, for helping in the planning of nourishment projects and design of beach protection infrastructures</td><td></td><td></td></tr></tbody></table>
the developers and the users. The questionnaire was organized around 3 blocks assessing different aspects: (i) Benchmark and advancement in CS (ii) Impact on decision making, (iii) Assessment of change (Fig. 2).

## Results

### Case studies overview

The seven present a large heterogeneity with respect to the water issue they are dealing within. In addition, these issues are representative of the vast variety of issues dependant on water or as broadly accepted the water sector, i.e., biodiversity changes, agricultural production, flooding assessment, hydropower production, water allocation and fluvial-coastal interaction across the continent (Fig. 3). A large miscellany was also shown regarding the user profiles (for instance: municipalities, senior experts, electricity companies or governmental agencies) and spatiotemporal scales required by users.

Details for each case study regarding the water issue, the aimed advancement, the users involved, and the spatiotemporal resolution needed are summarized in Fig. 3.

The differences in study cases have allowed the application of the framework in a more flexible manner, adapting it to each users' needs, helping to improve the quality of the service in decision-support and consequently, increasing user-uptake.

### Qualitative evaluation assessment

### Benchmark and advancement in CS

Table 4 summarizes the answers of the qualitative assessment regarding the section benchmark and advancements in CS carried out in each case study. The mentioned variability among the different water issues in each study case is reflected in the benchmarks initially set (Table 4, first column). They show not only differences in the users' use of the current CS data but also in the limitations they perceived in the way CS was used in their decision-making process. This benchmarking was the starting point for the service developers and users to identify where the focus of the improvement should be targeted (Table 4, second column). The comparison between both is highlighted in Table 4 third column and the final achievement in the fourth column.

When analysing the answers of this block of the assessment three main methodological aspects regarding the improvement were identified as key: i) the need of improved quality of data and methods to be used, ii) a better understanding of uncertainty to trust the information provided by the CS and thus, being able to incorporate it in the decision-making process and iii) an improvement of resolution to be able to apply CS' data at the local scale.

### Improved quality of data and methods
The improvement of data and methods used in the CS was the fundamental principle in the proposed

<table><caption>**Table 5**
Summary of answers to the second block of questions in the qualitative assessment: Impact on decision making.</caption><thead><tr><td></td><td>What do the results from the workflow mean for the water management issue?</td><td>What will be the decisions your results will be used for?</td><td>What will add your results to the decision-making process of the user?</td><td>What obstacles do you expect or have for integrating the improvement in your decision-making process?</td></tr></thead><tbody><tr><td>Case 1</td><td>The results are in line with previous ones; therefore, they serve to increase trust in current management decisions.</td><td>The outcomes from this case study can be used to support further environmental management programmes being for instance considered as part of the coming 5-year planning processes.</td><td>The addition of biodiversity indicators based on different ecological thresholds.</td><td>The main obstacles will appear when users see differences between old and new outcomes. In these cases, users claim a decision cannot be made.</td></tr><tr><td>Case 2</td><td>The inclusion of uncertainty in climate projections helps to perform a more intelligent decision-making process based on science and users' requirements.</td><td>The results will help the decision-making process for future agricultural planning, developing frameworks for the near- and long-term futures.</td><td>The outcomes will help the users to shape up recommendations for the farmers using solid supporting information based on state-of-the-art.</td><td>The main challenge will be the communication with the municipalities. Changing their mindset regarding the use of several models instead the use of one.</td></tr><tr><td>Case 3</td><td>The users appreciated to have a city-wide map of simulated flooding due to a heavy rainfall event, which incorporates a scientifically based method for estimating the impact of climate change on heavy rainfall events.</td><td>The planners intend to use the results for checking the suitability of future residential and commercial areas to be designated in the new land-use main plan, which is currently under preparation.</td><td>The main value highlighted by the users is that the new results are robust enough to withstand possible legal cases.</td><td>The need of a reformulation in how users interact with climate information.</td></tr><tr><td>Case 4</td><td>The indicator proposed addressed users' needs. However, first it is necessary to enhance the robustness of the analysis including some specific constraints that can be different between reservoirs.</td><td>The results obtained by the new developed methodology have the potential to be operationally adopted in a near future.</td><td>The indicator proposed was useful to address the typical operational problem during the periods when tensions might appear, and a lower flexibility of the system might pose a challenge to hydropower production.</td><td>The need of including extra changes that might be feasible under current regulatory frameworks (e.g., the concession regulations for the contracts allowing the management of hydroelectric large reservoir).</td></tr><tr><td>Case 5</td><td>The results present a contribution to broad the current users-knowledge about the impact of climate change in pluvial flash flood.</td><td>This new method provides crucial information for the development of future strategies in the client organization regarding the impact of pluvial flash flood.</td><td>The results indicate that the intensities of convective precipitation events are expected to exhibit a stronger increase with decreasing temperatures.</td><td>The need of change the way users are used to work with climate data.</td></tr><tr><td>Case 6</td><td>The definition of targeted indicators combining pan-European with local information and accounting for uncertainty has increased the users' confidence level.</td><td>The results will be used for planning the future operational rules in the reservoirs of the basin to guarantee both, the future environmental flow levels, and their impact on water allocation.</td><td>The defined indicators have given key information for decision making, accounting for number of days when hydropower cannot produce energy and number of days below a certain threshold.</td><td>The main challenge is connected to the size of the organization, which might difficult the decision of incorporating this new methodology. It will require making contacts with other departments and having a collective agreement of how to incorporate this information.</td></tr><tr><td>Case 7</td><td>The results include climate change aspects which were not used for any planning strategies by the users in their corresponding management issues before.</td><td>The outcomes allow a probabilistic approach towards management decision such as maritime structures planning, coastal retreat adaptation and nourishment works.</td><td>The results are a noticeable upgrade form the current decision-making process as they include the use of sea-level rise and the generation of maritime projections.</td><td>The main obstacles are to train personnel on the methods and the lack of computational power when apply the new method at the users' level.</td></tr></tbody></table>
framework. All case studies developed a new advancement, which to certain extent, impacted the decision-making process, guaranteeing in this way the better use of CS information. For the case study in Austria which tackle pluvial flash floods in the pre-Alpine region a new method was developed for the extrapolation of historical precipitation-temperature relationships to future temperature projections deriving future precipitation intensities on heavy rainfall days and precipitation changing rates (Huber et al., 2020). The same method was successfully applied to the German case study which dealt with urban flooding. The French case study involved the definition of novel indicators for hydropower management based on reservoir filling curves and climate projections (Lemoine et al., 2019), while for the Swedish case study the creation of a protocol to prioritise climate indicators for biodiversity management derived from higher resolution data was deemed necessary. In line with that, in the water allocation study case (Southern Spain), more complex indicators than those provided by the initial CS based on specific allocation demands were also defined (Contreras et al., 2020a). Finally, for the fluvial and coastal interactions in southern Spain the method was refined to use non-stationary mixed distribution models to fit the main maritime variables (Lira Loarca, et al., 2021).

**Better understanding of uncertainty.** Uncertainty in the information provided by the services was identified by the users of the case studies as one of the factors hindering the trust in CS. Although some users were familiar with terminology, its implications and how to use it, others recognized uncertainty as a barrier for the proper use of the information. Among the study cases, some of them directly deal with that issue in their proposed advancement. For instance, the two of the study cases in Denmark and in Southern Spain for coastal flooding, the users at the time of the benchmarking were not considering uncertainty in their process of decision-making for distinct reasons but mostly due to primary lack of information and then lack of capacity to use such information. It was then proposed to use the traditional approach in climate science; that is the use of the spread of an ensemble of climate run projections, to quantifying hydrological uncertainty, in seasonal changes in river discharges and groundwater depth for the Danish basin (Pasten-Zapata et al., 2019) and maritime uncertainty in wave heights for Southern Spain (Lira-Loarca et al., 2021). The Danish case study elaborated more by assessing uncertainty through the evaluation of the simulation skill of climate models in the historical period and the ability of hydrological models to reproduce hydrological processes under changing climate conditions (Pasten-Zapata et al., 2021). Finally, the Spanish case study dealing with water allocation resources proposed a new method for simulating uncertainty in streamflow indicators using a stochastic approach based on the errors of purpose specific metrics quantified in the reference period (Pimentel et al., 2021). It was shown that this approach could lead to a better representation of uncertainty, removing all the noise found when instead using a multimodel ensemble mean for assessing changes over small watershed over complex terrains (Contreras et al., 2020a; b, Pérez-Palazón et al., 2018).

**Improved resolution.** Higher spatial resolution datasets were a common demand in almost all study cases. The resolution of CS' information, usually provided at pan-European scales, was seen as a disadvantage for decision-making at such local scale as the case studies. Dealing with this issue, one study case used a new dataset at higher resolution than the previously used (EURO-CORDEX 11x11 km (EUR-11) vs 50x50 km (EUR-44), (Berg et al., 2019) and a new method for downscaling meteorological forcing that runs through the impact models (Berg et al., 2021; Schmith et al., 2021). In the Spanish case for the water allocation, downscaling was directly performed to streamflow instead of meteorological forcing data at watershed scale lumping the scale effects usually found in small mountain-coastal watersheds. Over these watersheds where snow plays a key role, the traditional bias adjustment techniques, which are carried out independently for precipitation and temperature,

<caption>**Table 6**</caption>

Summary of answers to the third block of questions in the qualitative assessment: Assessment of change.

<table><thead><tr><th></th><th>Successful factors that benefited during the process</th><th>Factors that limited during the process</th><th>What was the main message about advancing CS would you like others to take away?</th></tr></thead><tbody><tr><td>Case 1</td><td>- The use of the same method for bias adjustment.<br>The use of hydrological models already available in house.<br>The use of a defined system for post-processing.</td><td>- The availability of climate information only for three climate models.<br>The different observational datasets and setups in both hydrological models.</td><td>Users can successfully use a pan-European CS for local decision-making. The increased resolution allowed for more detail information at a county level. Moreover, it is important to be sure that the prioritised climate indicators can show the right information.</td></tr><tr><td>Case 2</td><td>- The constant interaction with users.<br>The fact of having the client on board, which guarantee the development of a tailored approach.<br>The previous expertise in CS developing</td><td>- The initial users' reluctance consequence of the limiting information provided by the current CS.<br>The substantial number of resources, human and technical, needed to produce the required data for other study cases.</td><td>Climate science must consider the needs of users in its development. Initially, we were faced with clients that thought that the available CS did not consider their needs. Therefore, it is key to provide useful information for the user based on their needs, but also this information should be scientifically relevant to establishing a fruitful link.</td></tr><tr><td>Case 3</td><td>- The previous working experience with users.<br>The fact that climate change aspects was accepted as a relevant issue to be included in the city planning.</td><td>- The difficulties to stablish a clear comparison with the benchmark due to the different nature of the methodology used.<br>The difficulties to focus the interest of the planners in far future.</td><td>The new method has a scientific background going further than a simple trend extrapolation of historic rainfall data. Therefore, it is considered more robust and suitable for incorporation into planning and decision-making than the previous one.</td></tr><tr><td>Case 4</td><td>- The engagement of stakeholders during the entire process.</td><td>- The numerous challenges of using CS data: downloaded, scaled, bias adjusted, prepared to be used in hydrological modelling</td><td>Useful indicators can be developed based on climate change projection available in different CS.</td></tr><tr><td>Case 5</td><td>- The availability of different data sources in several CS.<br>The good cooperation in the trans-national consortium within the project</td><td></td><td>The inclusion of convection and the use of potential of temperature-based scaling approach are key aspects that permit climate models to improve the estimation of heavy precipitation events.</td></tr><tr><td>Case 6</td><td>- The fact of working hand by hand with the client in the</td><td>- The difficulties linking different spatiotemporal scales</td><td>Users' participation should go further than the initial CS</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 6 (continued)**</caption><thead><tr><th></th><th>Successful factors that benefited during the process</th><th>Factors that limited during the process</th><th>What was the main message about advancing CS would you like others to take away?</th></tr></thead><tbody><tr><td></td><td>definition of indicators that account for water management issues.<br>The fact of having information aggregated at the watershed scale in the CS</td><td>(pan-European vs local).<br>The users' difficulties in understanding CS' information since the language used, English, is different to their mother tongue.<br>The initial users' lack of confidence in climate change data.<br>The lack of knowledge of some climate aspect by the users.</td><td>design, broadens the co-design concept to a co-advance design.</td></tr><tr><td>Case 7</td><td>- The previous expertise of the team working with maritime climate.</td><td>- The lack of maritime climate variables projected at local or regional scales.<br>The computational effort required to obtain the targeted variables.</td><td>It is crucial to account for the random behaviour of storm events without the use of predefined geometries and include the generation and use of maritime climate projections.</td></tr></tbody></table>

usually shift the timing of snowfall and produce significant error in streamflow representation (Pimentel et al., 2021). In the Granada case study and within an inter-university collaboration, the developers used a state-of-the-art ensemble of wave climate projection in the Mediterranean Sea with EUR-11 climate projected data improving the model evaluation for both the past and projections (Lira Loarca, et al., 2021).

Therefore, more robust methods and data are needed when improving the quality of already existing CS, being especially important to tackle the three above mentioned aspects. Therefore, we identify robustness as a first pillar in improving the quality of already existing CS.

### Impact on decision making

Table 5 summarizes the answers of the qualitative assessment regarding the impact on the perception of users in their decision-making process.

In general, users highly valued the achieved improvements and their potential usefulness in their decision-making process (Table 5, columns 1–3). However, they also highlighted that some reformulation in the status of the CS might be needed for a successful implementation of the new advancement. Sometimes, these advancements can imply a substantial change, modification, or inclusion that is difficult to put into practice due to some extend the rigid structure in some of the users’ organizations. They usually claim that CS are focus on delivery, putting the emphasis on products rather than processes, on assumptions that demand, and usually lacking economic evaluation (Findlater et al., 2021).

During the evaluation, some of the case studies reported that dealing with climate change aspect is complex. Even though several users acknowledged the improvement carried out by the new methods, there is still a dissent over who should be responsible for producing, maintaining, and providing the necessary data in the CS and for considering climate aspects and taking the appropriate action (Clifford et al., 2020; Webber, 2019). Although in the new advancements within the project we did not directly address the issue of governance, the users have gained new awareness and understanding for the usefulness of CS and can take this further within their institution and processes.

In addition, developers must be open to substantial changes in the CS' structure, for instance remove old data, include new improved methodologies in data production. CS should not be a rigid structure where changes cannot be done but life “products” in which the advancement in science and users' need goes can be easily included (Mauser et al., 2013).

Therefore, this block of answer suggests that a reform in the CS might be needed for adapting the current CS structure to the new proposed advancement for a successful implementation in the actual decision-making. Reform is identified as the second pillar in improving the quality of already existing CS.

### Assessment of change

The last block of answers deals with the evaluation of change. Simple feedback questions regarding the factors that limited or enhanced the advancement give an overview of the actual change achieved after applying the proposed framework. This approach has drawn several different reflections for both users and developers that on the one hand helped during the implementation of the framework and on the other hand, can provide useful information for other users in the implementation of the framework on their CS (Table 6).

A successful factor highlighter by most study cases is the key role of an active collaboration between providers and users when improving the quality of existing CS. This co-interaction has proven to be crucial for user uptake, understanding, and valuing the information provided by the CS. Moreover, broadening the involvement of users around the CS also helped to discover different perspectives and explored the applicability of the CS creating a solid linkage to provide quality feedback and avoiding user fatigue of a more superficial engagement (Vogel et al., 2019).

Regarding limitations, the collected answers reinforce initial limitations found in previous studies regarding users' knowledge and uptake from CS (Vaughan et al., 2016; Photiadou et al., 2021). In general, during the initial stages, users claimed not to use CS' information in their decision-making processes since neither CS were initially designed for their specific issues nor the indicators provided were adjusted to their needs. Inaccessibility to the CS' information due to the language was also highlighted as a limitation. In general, global, and continental' CS did not cater for languages other than English. Users highlighted the need for an intermediary to translate not only climate data but also the scientific concepts, metadata information or manuals. It was also remarkable the users' eagerness for trustable data but the inability to process this large information by themselves.

Hence, recruitment, understood here as user engagement, has being shown in this third block of the assessment as a crucial need for involving more actively users in improving the quality of existing CS. Recruitment is therefore, the third pillar in the CS improvement process.

Finally, the assessment process itself constitute a fourth pillar. The reflection process carried out can identify the key aspect for evaluating the actual improvement of CS. Here, with reflection we refer to a simpler approach, for instance a simple auto-evaluation after the implementation of the advancement in a periodic way that could help to identify the key aspect that have really contributed and limited this improvement.

### Conclusions

First, The AQUACLEW' proposed framework has proved to improve the usability of already existing CS tackling and overcoming the water issues in decision-making through a lengthy co-development approach through a modelling chain with several steps and an active communication with continuous exchanges between users and developers. However, this approach has shown a very demanding interaction with a very exclusive target-oriented user. Hence, despite their potential, it might not be feasible to be widely applied due to the need of knowledge developers for a limited number of users. Therefore, a deeper effort should be made in educational aspects to develop deeper user
<figure><img src="image_8.png" /><figcaption>**Fig. 4.** The interlinked 4Rs of AQUACLEW 4Rs: Robustness, Recruitment, Reform and Reflection.</figcaption></figure>

knowledge and reduce the role of service developers acting as knowledge purveyors (i.e., Crochemore et al., 2021, Conteras et al., 2020). Moreover, the assessment done has allowed us to identify four pillars to tackle when improving the quality of CS in the water sector (Fig. 4): (1) Robustness, accounting for better quality of the CS' information in certain aspects; (2) Recruitment, understood as a need of involving users more actively in CS structures; (3) Reform, highlighting the possible need of changes in both CS structure and users mindsets; and (4) Reflection, as a process of continuous evaluation during the life of the CS. The 4Rs were the interconnectors in a CS and were regularly visited to improve the quality of the service (Fig. 4). Each R represents a fundamental aspect of our case studies from scientific methods and data to user engagement, decision-making support, and finally an evaluation process.

## CRediT authorship contribution statement

**Rafael Pimentel:** Conceptualization, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing.
**Christiana Photiadou:** Conceptualization, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing.
**Lorna Little:** Conceptualization, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing.
**Andreas Huber:** Investigation, Writing – review & editing.
**Anthony Lemoine:** Investigation, Writing – review & editing.
**David Leidinger:** Investigation, Writing – review & editing.
**Andrea Lira-Loarca:** Investigation, Writing – review & editing.
**Johannes Lückenkötter:** Investigation, Writing – review & editing.
**Ernesto Pasten-Zapata:** Investigation, Writing – review & editing.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Data availability

Data will be made available on request.

## Acknowledgements

R. Pimentel acknowledges fundings by the Juan de la Cierva Incorporación Programme of the Spanish Ministry of Science and Innovation (IJC2018-038093-I). R. Pimentel is a member of DAUCO, Unit of Excellence ref. CEX2019-000968-M, with financial support from the Spanish Ministry of Science and Innovation, the Spanish State Research Agency, through the Severo Ochoa and María de Maeztu Program for Centres and Units of Excellence in R&D.

## Funding

This work was funded by the project AQUACLEW, which is part of ERA4CS, an ERA-NET initiated by JPI Climate, and funded by FORMAS (SE), DLR (DE), BMWFW (AT), IFD (DK), MINECO (ES), ANR (FR) with co-funding by the European Commission [Grant 690462].

## References

Arheimer, B., Pimentel, R., Isberg, K., Crochemore, L., Andersson, J.C.M., Hasan, A., Pineda, L., 2020. Global catchment modelling using World-Wide HYPE (WWH), open data, and stepwise parameter estimation. Hydrol. Earth Syst. Sci. 24 (2), 535–559. https://doi.org/10.5194/hess-24-535-2020.

Berg, P., Christensen, O.B., Klehmet, K., Lenderink, G., Olsson, J., Teichmann, C., Yang, W., 2019. Summertime precipitation extremes in a EURO-CORDEX 0.11° ensemble at an hourly resolution. Nat. Hazards Earth Syst. Sci. 19 (4), 957–971. https://doi.org/10.5194/nhess-19-957-2019.

Berg, P., Bosshard, T., Yang, Wei, & Zimmermann, Klaus. (2021). *MIDAS version 0.1 framtagande och utvärdering av ett nytt verktyg för biasjustering* (Klimatologi Nr 63; p. 37). SMHI. https://www.smhi.se/polopoly_fs/1.173788!/Klimatologi_63%20MIDASv01.pdf.

Besembinder, J., Terrado, M., Hewitt, C., Garrett, N., Kotova, L., Buonocore, M., Groenland, R., 2019. Need for a common typology of climate services. Clim. Serv. 16, 100135. https://doi.org/10.1016/j.cliser.2019.100135.

Brasseur, G.P., Gallardo, L., 2016. Climate services: lessons learned and future prospects. Earth's Future 4 (3), 79–89. https://doi.org/10.1002/2015EF000338.

Brooks, M.S., 2013. ACCELERATING INNOVATION IN CLIMATE SERVICES: the 3 E's for climate service providers. Bull. Am. Meteorol. Soc. 94 (6), 807–819.

Buontempo, C., Hewitt, C.D., Doblas-Reyes, F.J., Dessai, S., 2014. Climate service development, delivery and use in Europe at monthly to inter-annual timescales. Clim. Risk Manage. 6, 1–5. https://doi.org/10.1016/j.crm.2014.10.002.

Celliers, L., Costa, M.M., Williams, D.S., Rosendo, S., 2021. The 'last mile' for climate data supporting local adaptation. Glob. Sustain. 4, e14.

Clifford, K.R., Travis, W.R., Nordgren, L.T., 2020. A climate knowledges approach to climate services. Clim. Serv. 18, 100155 https://doi.org/10.1016/j.cliser.2020.100155.

Contreras, E., Herrero, J., Crochemore, L., Aguilar, C., Polo, M.J., 2020a. Seasonal climate forecast skill assessment for the management of water resources in a run of river hydropower system in the Poqueira River (Southern Spain). Water 12 (8), 2119. https://doi.org/10.3390/w12082119.

Contreras, E., Herrero, J., Crochemore, L., Pechlivanidis, I., Photiadou, C., Aguilar, C., Polo, M.J., 2020b. Advances in the definition of needs and specifications for a climate service tool aimed at small hydropower plants' operation and management. Energies 13 (7), 1827. https://doi.org/10.3390/en13071827.

Crochemore, L., Cantone, C., Pechlivanidis, I.G., Photiadou, C.S., 2021. How does seasonal forecast performance influence decision-making? Insights from a serious game. Bull. Am. Meteorol. Soc. 102 (9), E1682–E1699. https://doi.org/10.1175/BAMS-D-20-0169.1.

Dessai, S., Hulme, M., 2004. Does climate adaptation policy need probabilities? Clim. Policy 4 (2), 107–128. https://doi.org/10.1080/14693062.2004.9685515.

Dessai, S., Lu, X., Risbey, J.S., 2005. On the role of climate scenarios for adaptation planning. Global Environ. Change 15 (2), 87–97. https://doi.org/10.1016/j.gloenvcha.2004.12.004.

Donnelly, C., Andersson, J.C.M., Arheimer, B., 2016. Using flow signatures and catchment similarities to evaluate the E-HYPE multi-basin model across Europe. Hydrol. Sci. J. 61 (2), 255–273. https://doi.org/10.1080/02626667.2015.1027710.

Ernst, K.M., Swartling, A.G., Andre, K., Preston, B.L., Klein, R.J., 2019. Identifying climate service production constraints to adaptation decision-making in Sweden. Environ. Sci. Policy 93, 83–91.

Findlater, K., Webber, S., Milind, K., Donner, S., 2021. Climate services promise better decisions but mainly focus on better data. Nat. Clim. Chan. 11, 731–737. https://doi.org/10.1038/s41558-021-01125-3.

Hewitson, B., Waagsaether, K., Wohland, J., Kloppers, K., Kara, T., 2017. Climate information websites: an evolving landscape. Wiley Interdiscip. Rev. Clim. Change 8 (5). https://ideas.repec.org/a/wly/wirecc/v8y2017i5ne470.html.

Hewitt, C.D., Stone, R.C., Tait, A.B., 2017. Improving the use of climate information in decision-making. Nat. Clim. Change 7 (9), 614–616. https://doi.org/10.1038/nclimate3378.
Huber, A., Lumassegger, S., Leidinger, D., Achleitner, S., Formayer, H., Kohl, B., 2020. Modeling of pluvial flash floods in pre-Alpine regions and assessment of potential climate change impacts. EGU General Assembly Conf. Abstracts 10062.

Jacob, D., Teichmann, C., Sobolowski, S., Katragkou, E., Anders, I., Belda, M., Benestad, R., Boberg, F., Buonomo, E., Cardoso, R.M., Casanueva, A., Christensen, O. B., Christensen, J.H., Coppola, E., De Cruz, L., Davin, E.L., Dobler, A., Domínguez, M., Fealy, R., Wulfmeyer, V., 2020. Regional climate downscaling over Europe: perspectives from the EURO-CORDEX community. Reg. Environ. Change 20 (2), 51. https://doi.org/10.1007/s10113-020-01606-9.

Jones, Patwardhan, A., Cohen, S.J., Dessai, S., Lammel, A., Lempert, R.J., Mirza, M.M.Q., & von Storch, H., (2014). *Foundations for decision making. In: Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change* (pp. 195–228).

Klein, R.J.T., Juhola, S., 2014. A framework for Nordic actor-oriented climate adaptation research. Environ. Sci. Policy 40, 101–115. https://doi.org/10.1016/j.envsci.2014.01.011.

Larosa, F., Mysiak, J., 2020. Business models for climate services: an analysis. Clim. Serv. 17, 100111 https://doi.org/10.1016/j.cliser.2019.100111.

Lemoine, A., Ramos, M.-H., Thirel, G., Andréassian, V., 2019. Using climate services to evaluate projected changes in the management and planning of hydropower production. EGU General Assembly Conf. Abstracts 14887.

Loarca, L., Cobos, M., Besio, G., Baquerizo, A., 2021. Projected wave climate temporal variability due to climate change. Stoch. Environ. Res. Risk Assess. 35 (9), 1741–1757. https://doi.org/10.1007/s00477-020-01946-2.

Maraun, D., Widmann, M., Gutiérrez, J.M., Kotlarski, S., Chandler, R.E., Hertig, E., Wibig, J., Huth, R., Wilcke, R.A.I., 2015. VALUE: A framework to validate downscaling approaches for climate change studies. Earth's Future 3 (1), 1–14. https://doi.org/10.1002/2014EF000259.

Mauser, W., Klepper, G., Rice, M., Schmalzbauer, B.S., Hackmann, H., Leemans, R., Moore, H., 2013. Transdisciplinary global change research: the co-creation of knowledge for sustainability. Curr. Opin. Environ. Sustain. 5 (3), 420–431. https://doi.org/10.1016/j.cosust.2013.07.001.

McNie, E.C., 2012. Delivering climate services: organizational strategies and approaches for producing useful climate-science information. Weather Clim. Soc. 5 (1), 14–26. https://doi.org/10.1175/WCAS-D-11-00034.1.

Merks, J., Photiadou, C., Ludwig, F., Arheimer, B., 2020. Comparison of open access global climate services for hydrological data. Hydrol. Sci. J. 1–17. https://doi.org/10.1080/02626667.2020.1820012.

Pasten-Zapata, E., Sonnenborg, T.O., Refsgaard, J.C. (2019). Climate change: Sources of uncertainty in precipitation and temperature projections for Denmark. *GEUS Bull.* **43. 10.34194/GEUSB-201943-01-02.**

Pasten-Zapata, E., Pimentel, R., Royer-Gaspard, P., Sonnenborg, T.O., Aparicio-Ibañez, J., Lemoine, A., Pérez-Palazón, M.J., Schneider, R., Thirel, G., Photiadou, C., 2021. The impact of weighting hydrological projections based on the transferability of hydrological models under a changing climate. J. Hydrol. under review.

Pérez-Palazón, M.J., Pimentel, R., Polo, M.J., 2018. Climate trends impact on the snowfall regime in Mediterranean Mountain Areas: future scenario assessment in Sierra Nevada (Spain). Water 10, 720. https://doi.org/10.3390/w10060720.

Photiadou, C., Arheimer, B., Bosshard, T., Capell, R., Elenius, M., Gallo, I., Gyllensvärd, F., Klehmet, K., Little, L., Ribeiro, I., Santos, L., Sjökvist, E., 2021. Designing a climate service for planning climate actions in vulnerable countries. Atmosphere 12 (1), 121. https://doi.org/10.3390/atmos12010121.

Pimentel, R., Pérez-Palazón, M.J., Herrera-Grimaldi, P., Aparicio, J., Torralbo, P., Polo, M.J., 2021. Retrieving river flow descriptors for water management under future climate scenarios over complex terrain: combining pan-European and local modelling in Sierra Nevada (Spain). Hydrol. Sci. J. under review.

Schmith, T., Thejll, P., Berg, P., Boberg, F., Christensen, O.B., Christiansen, B., Christensen, J.H., Madsen, M.S., Steger, C., 2021. Identifying robust bias adjustment methods for European extreme precipitation in a multi-model pseudo-reality setting. Hydrol. Earth Sci. 25 (1), 273–290. https://doi.org/10.5194/hess-25-273-2021.

Street, R.B., 2016. Towards a leading role on climate services in Europe: a research and innovation roadmap. Clim. Serv. 1, 2–5. https://doi.org/10.1016/j.cliser.2015.12.001.

Vaughan, C., Buja, L., Kruczkiewicz, A., Goddard, L., 2016. Identifying research priorities to advance climate services. Clim. Serv. 4, 65–74. https://doi.org/10.1016/j.cliser.2016.11.004.

Vaughan, C., Muth, M.F., Brown, D.P., 2019. Evaluation of regional climate services: learning from seasonal-scale examples across the Americas. Clim. Serv. 15, 100104 https://doi.org/10.1016/j.cliser.2019.100104.

Viel, C., Beaulant, A.-L., Soubeyroux, J.-M., Céron, J.-P., 2016. How seasonal forecast could help a decision maker: an example of climate service for water resource management. Adv. Sci. Res. 13, 51–55. https://doi.org/10.5194/asr-13-51-2016.

Vincent, K., Dougill, A.J., Dixon, J.L., Stringer, L.C., Cull, T., 2017. Identifying climate services needs for national planning: insights from Malawi. Climate Policy 17 (2), 189–202. https://doi.org/10.1080/14693062.2015.1075374.

Vincent, K., Daly, M., Scannell, C., Leathes, B., 2018. What can climate services learn from theory and practice of co-production? Clim. Serv. 12, 48–58. https://doi.org/10.1016/j.cliser.2018.11.001.

Vogel, J., Letson, D., Herrick, C., 2017. A framework for climate services evaluation and its application to the Caribbean Agrometeorological Initiative. Clim. Serv. 6, 65–76. https://doi.org/10.1016/j.cliser.2017.07.003.

Vogel, C., Steynor, A., Manyuchi, A., 2019. Climate services in Africa: Re-imagining an inclusive, robust and sustainable service. Clim. Serv. 15, 100107 https://doi.org/10.1016/j.cliser.2019.100107.

Webber, S., 2019. Putting climate services in contexts: advancing multi-disciplinary understandings: introduction to the special issue. Clim. Change 157 (1), 1–8. https://doi.org/10.1007/s10584-019-02600-9.

Webber, S., Donner, S.D., 2017. Climate service warnings: cautions about commercializing climate science for adaptation in the developing world. WIREs Clim. Change 8 (1), e424.