Windbench: Building Trust on Wind Energy Design Tools through Open Science Model Evaluation
Creators: name: Sanz Rodrigo, Javier
name: Gancarski, Pawel
name: Vathsavayi, Sri Harsha
Publisher: name: Zenodo
Year: 2018
Subjects: open science, validation, jupyter hub, EUDAT

description: <strong>Introduction</strong> The Windbench portal has been developed together with the IEA-Wind Task 31 “Wakebench” Wind Energy Model Evaluation Protocol (WEMEP) to promote interdisciplinary research in the development of integrated design tools. Due to the high complexity of the multi-scale wind energy system, a building-block validation process is required to systematically test model adequacy based on fit-to-purpose metrics across a wide range of wind conditions. <strong>Value Proposition</strong> Windbench focuses primarily on model developer needs while also engaging with researchers involved in the design of experiments and generation of validation datasets. End-users from industry use Windbench to judge the adequacy of the models underlying their design tools. The value proposition for these users is: <em>Validate your code together with your peers and share your data safely to contribute to a traceable international model evaluation framework for the development of trustful wind energy engineering tools. </em> This is rolled out into the following features that lead the new design of Windbench: A community platform to map and document the development an international wind energy modeling and evaluation framework. A validation directed research program to establish high-level priorities for experimental analysis, model development and evaluation to meet the requirements of wind energy applications. A guide to adopt formal model evaluation procedures and data standards to improve the traceability and interoperability of the framework. A gateway to open-access resources for modeling, data analysis and validation. A forum to discuss industry challenges and scientific insights that help identify knowledge gaps. <strong>Solution</strong> Windbench will orchestrate general-purpose services around the SeaDataCloud virtual research environment (VRE), a data analytics platform that allows users to run benchmarking activities powered by the EUDAT Collaborative Data Infrastructure. Jupyter notebooks will be integrated in the WEMEP using sphinx-based documentation through Github repositories, published in Zenodo for version control. Benchmark data repositories will be shared and exploited through EUDAT data management services integrated with the SeaDataCloud to adopt policies and standards that promote data interoperability. Potential contributions to the WEMEP will be discussed online through The Wind Vane Blog, a publication hosted by Medium to promote the exchange of scientific insights and challenges between wind energy researchers and industry. <strong>Testing Plans</strong> Windbench is developed alongside the third phase of the IEA Wind Task 31 with support from the H2020-MARINET2 project (grant agreement number 73108). Over the next three years the integration of Windbench in the SeaDataCloud VRE will be enabled and tested with use cases from MARINET2 calls for access to e-infrastructures as well as other parallel projects under the umbrella of the IEA Wind. Solution testing with early adopters will focus on developing critical components that lead to adoption (communities engaged, benchmarks produced, users producing content, etc). This will ultimately depend on an actionable model evaluation protocol that promotes validation repositories that follow FAIR principles on data that are findable, accessible, interoperable and reusable. <strong>Summary</strong> We present a prototype for the new design of Windbench, a service to manage benchmarking activities for verification and validation of wind energy models. Windbench addresses model developers, experimentalists and end-users from industry through an actionable model evaluation protocol that promotes data sharing and trust. This will be powered by a virtual research environment built on the SeaDataCloud to provide data analytics around benchmark repositories managed in the EUDAT data management ecosystem. Key performance indicators will be based on quantifying user adoption as well as assessing the FAIRification of the data as outlined in the GO FAIR initiative.