NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

Technical Report

NREL/TP-5000-72760

January 2019

Blade-Resolved, Single-Turbine 

Simulations Under Atmospheric Flow

Michael Lawson,

1 Jeremy Melvin,2 Shreyas Ananthan,1

Kenny Gruchalla,

1 Jon Rood,1 and Michael Sprague1

1 National Renewable Energy Laboratory

2 University of Texas at Austin
NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

National Renewable Energy Laboratory

15013 Denver West Parkway

Golden, CO 80401

303-275-3000 • www.nrel.gov

Technical Report

NREL/TP-5000-72760

January 2019

Blade-Resolved, Single-Turbine 

Simulations Under Atmospheric Flow

Michael Lawson,

1 Jeremy Melvin,2 Shreyas Ananthan,1

Kenny Gruchalla,

1 Jon Rood,1 and Michael Sprague1

1 National Renewable Energy Laboratory

2 University of Texas at Austin

Suggested Citation

Lawson, Michael, Jeremy Melvin, Shreyas Ananthan, Kenny Gruchalla, Jon Rood, and 

Michael Sprague. 2018. Blade-Resolved, Single-Turbine Simulations Under Atmospheric 

Flow. Golden, CO: National Renewable Energy Laboratory. NREL/TP-5000-72760.

https://www.nrel.gov/docs/fy19osti/72760.pdf.
NOTICE

This work was authored in part by the National Renewable Energy Laboratory, operated by Alliance for Sustainable 

Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding 

provided by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of Energy’s 

Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale 

ecosystem, including software, applications, and hardware technology, to support the nation’s exascale computing 

imperative. The views expressed herein do not necessarily represent the views of the DOE or the U.S. Government.

This report is available at no cost from the National Renewable 

Energy Laboratory (NREL) at www.nrel.gov/publications.

U.S. Department of Energy (DOE) reports produced after 1991 

and a growing number of pre-1991 documents are available 

free via www.OSTI.gov.

Cover Photos by Dennis Schroeder: (clockwise, left to right) NREL 51934, NREL 45897, NREL 42160, NREL 45891, NREL 48097, 

NREL 46526.

NREL prints on paper that contains recycled content.
Executive Summary

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawattscale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD)

simulations for which the mesh resolves the geometry of the turbines and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm

simulations will require exascale-class resources.

The primary code in the ExaWind project is Nalu-Wind, which is an unstructured-grid solver for the acoustically

incompressible Navier-Stokes equations. The model consists of the mass-continuity Poisson-type equation for

pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated

by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem,

the moving meshes greatly affect overall solver costs because reinitialization of matrices and recomputation of

preconditioners is required at every time step.

In this milestone report, we describe efforts to establish a body-resolved model of a megawatt-scale turbine that will

be used for demonstrating and testing our wind farm simulation platform. The team chose the NREL 5-megawatt

(MW) wind turbine, which is a notional turbine that is fully defined in the open domain. Given many proprietary

issues surrounding modern turbines in the field (e.g., turbine control system, blade geometry and materials), a reference turbine such as the NREL 5-MW is ideal for the benchmarking and demonstration studies that will be key to the

success of the ExaWind project. This milestone was designed to demonstrate a working model that will be used for

figure-of-merit measurements and will establish baseline results that will be improved upon as the project progresses.

Two meshes, a 25 million node mesh and a 95 million node mesh, were generated using Pointwise meshing software

that resolved the blades, nacelle, and tower. Rotor motion is handled through a sliding-mesh interface (developed

earlier in the ExaWind project), for which the rotor and near-rotor fluid is embedded in a cylinder that rotates inside

the larger, fixed domain. Two additional meshes were generated by uniformly refining the 95 million node mesh

using Percept. Simulations using the coarsest mesh were run on the National Energy Research Scientific Computing

Center (NERSC) Cori system under uniform inflow and with a wall-adapting local-eddy viscosity (WALE) turbulence model for approximately three rotations. Additionally, turbulent conditions (for inflow boundaries and volume

initialization) were added using data from an external database of homogeneous isotropic turbulence (HIT). The HIT

turbulence fields were derived from the Johns Hopkins Turbulence Database and turbulence intensity was scaled

to 10%, which is representative of atmospheric turbulence encountered by wind turbines in the field. Coarse-mesh

simulations were also run with the generated HIT inflow and initial conditions.

The coarse-mesh WALE simulations with the sliding-mesh interface described here are a step toward predictive

blade-resolved simulations that are the goal of ExaWind. The sliding-mesh interface is well suited for efficient

simulations where the only turbine motion is that of the rotor. In operating turbines, individual blades rotate around

their pitch axis, and the entire nacelle-rotor rotates (yaws) about the tower axis and translates because of tower

deformations. Under these conditions, an overset-mesh approach is seen as necessary. It is understood that the

WALE turbulence model with practical grid-refinement levels is insufficient for predictive simulations, and future

simulations will, therefore, employ hybrid Reynolds-averaged Navier-Stokes (RANS)/large-eddy simulation (LES)

turbulence modeling for which RANS will be employed near blade surfaces and LES will be employed in the farfield wake. Those simulations will also require much more refinement, especially near the blade surface. Finally,

predictive simulations will need to be coupled to the OpenFAST turbine model, which models the nonlinear, largedeformation structural dynamics of the blades, the control system (that controls, e.g., pitch and yaw), and tower

dynamics. These features not exercised in this milestone are all under development and will be demonstrated and

described in the next ExaWind milestone report.

iii

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Acknowledgments

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither

the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or

implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights.

Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer,

or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United

States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily

state or reflect those of the United States Government or any agency thereof.

This research was supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department

of Energy’s Office of Science and National Nuclear Security Administration, responsible for delivering a capable

exascale ecosystem, including software, applications, and hardware technology, to support the nation’s exascale

computing imperative.

This work was authored in part by the National Renewable Energy Laboratory, operated by Alliance for Sustainable

Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308.

This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S.

Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231.

An award of computer time was provided by the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. This research used resources of the Argonne Leadership Computing Facility, which is a

DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357.

iv

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Table of Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Milestone Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Execution Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Overview of milestone completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 5-MW NREL turbine model development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.1 Turbine selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.2 Mesh generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.1 Uniform Inflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.2 Turbulent Inflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5 Concluding remarks and next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.1 Path Forward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

List of Figures

Figure 1. Image of the NREL 5-MW turbine g0 mesh showing the (upper left) domain extents, (lower left)

rotor disk and wake refinement regions, (lower right) hub, nacelle, and blade mesh, and (upper right)

blade tip mesh where the mesh extrusion off the blade surface is visible. . . . . . . . . . . . . . . . . . . 7

Figure 2. Image of the NREL 5-MW turbine and its wake after the turbine has completed 2.3 revolutions

at 14.0 seconds of simulation time. The wake is visualized by showing contours of velocity magnitude of

5.5 m/s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 3. Initial conditions for the turbulent-inflow simulations. Top: Velocity magnitude is shown as a

horizontal slice taken through the center of the hub. Bottom: Velocity magnitude is shown as vertical

slice taken through the center of the turbine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Figure 4. Velocity magnitude contours of 5.5 m/s (blue) and 10.0 m/s (red) after one revolution of the turbine. Left: Uniform initialization. Right: Turbulent initialization with T I = 0.1. . . . . . . . . . . . . . . 13

Figure 5. The velocity magnitude is shown as a horizontal slice through the turbine hub with a 3D contour

of 5.5 m/s after one revolution of the turbine. Top: Uniform initialization. Bottom: Turbulent initialization with T I = 0.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
List of Tables

Table 1. NREL 5-MW Turbine specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Table 2. NREL 5-MW Turbine mesh characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
1 Introduction

The ultimate goal of the ExaWind project is to enable scientific discovery through predictive simulations of wind

farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require

computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines and

captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably

petascale class, multi-turbine wind farm simulations will require exascale-class resources.

The objective of this milestone is to demonstrate a “production run” of a megawatt-scale turbine to exercise and

demonstrate the turbine simulation capabilities that have been developed during the previous two years of the ExaWind project. Demonstrating this simulation capability is the critical first step in enabling Nalu-Wind to be used as

a predictive modeling tool that can advance the scientific understanding of wind turbine performance.

In Section 2 the milestone description is provided. Next, in Section 3, we present the NREL 5-megawatt (MW)

reference turbine that was used to demonstrate Nalu-Wind’s capabilities and describe the four computational meshes

that were generated. In Section 4, we present results from simulations of the turbine operating under uniform inflow

and homogeneous isotropic turbulence (HIT) conditions. Finally, in Section 5, notable achievements, conclusions,

and a path forward are described.

3

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
2 Milestone Description

In this section, we provide the approved milestone description and execution plan followed by a brief description of

how the milestone was completed. Details regarding completion are included in the following sections.

2.1 Description

We will create and demonstrate a simulation capability for a megawatt-scale whole wind turbine operating in turbulent atmospheric flow where the fluid mesh resolves turbine geometry (blades, nacelle, tower). This model will

include the key feature necessary for modeling a full wind farm comprised of many megawatt-scale turbines, including long, flexible blades.

Completion of this milestone will require integrating several modeling and simulation capabilities, including sliding

or overset mesh capabilities to accommodate the rotating blades and mesoscale inflow conditions via a precursor

atmospheric boundary layer (ABL) simulation or a numerical-weather-prediction code. A stretch goal will be to

include fluid-structure-interaction coupling between the CFD code and the OpenFAST wind turbine code appropriate

for flexible blades.

The OpenFAST turbine simulation tool includes nonlinear structural finite-element models and wind turbine controlsystem models. Several of these capabilities are being generated and verified in the U.S. Department of Energy’s

(DOE’s) Office of Energy Efficiency and Renewable Energy’s (EERE’s) Atmosphere to Electrons (A2e) HighFidelity Modeling (HFM) project, which is running concurrently with this Exascale Computing Project (ECP).

2.2 Execution Plan

• Choose a baseline turbine configuration that has as many open source specifications as possible. If there is a

lack of public-domain specifications, choose a reference turbine (e.g., the NREL 5-MW turbine, which is fully

defined in the public domain).

• Create CFD models that resolve the baseline turbine outer geometry and turbine structural models in the

OpenFAST code. Employ turbulence-modeling approaches demonstrated elsewhere in this project or in the

A2e HFM project.

• Create inflow conditions representing relevant turbulent atmospheric flow.

• Create automated tests for full-turbine configuration.

• Perform simulations in relevant atmospheric conditions.

Completion Criteria: Technical report describing the milestone accomplishment as well as a highlight slide summarizing those accomplishments.

2.3 Overview of milestone completion

The following is a concise list of accomplishments that supported completion of this milestone:

• Chose the NREL 5-MW reference turbine as our baseline turbine, which is fully defined in the open domain.

• Created meshes at four refinement levels, ranging in size from 25-million nodes to 6-billion nodes, that resolve

the blades, hub, nacelle, and tower. The meshes described in this report used the sliding-mesh interface developed earlier in the project (see, e.g., fiscal year FY 2018 Q1 milestone and Domino 2018). In ongoing work,

an overset-mesh model is being created; the overset approach will be necessary for simulating an operating

turbine with multiple moving and deforming parts.

• While the intention was to add a coarse-mesh model to the regression-test suite, the model sizes were too large

(e.g., the coarsest mesh was 1.7 GB). However, there is an existing regression test (dgNonConformalThreeBlade)

that effectively tests the sliding-mesh interface used in this report.

4

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
• Performed simulations on the NERSC Cori (Haswell) systems for the coarse mesh with a wall-adapting localeddy viscosity (WALE) turbulence model under uniform inflow and ran simulations for about three rotor

revolutions.

• Performed simulations on the NERSC Cori (Haswell) systems for the coarse mesh (with a WALE turbulence

model) using homogeneous isotropic turbulence (HIT) data for initialization and inflow boundary conditions.

The HIT turbulence fields were derived from the the Johns Hopkins Turbulence Database and turbulence

intensity was scaled to 10%, which is appropriate for wind relevant flows. Simulations were run for about one

full revolution.

• Demonstrated the ability to run the more refined meshes (up to 6 B nodes) on the Cori (Haswell) and Mira

systems.

• Fluid-structure interaction capabilities for body-resolved meshes are under development and will be demonstrated and tested as part of a future milestone.

5

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
3 5-MW NREL turbine model development

3.1 Turbine selection

The NREL 5-MW reference turbine (Jonkman et al. 2009), with the specifications presented in Table 1, was selected

to demonstrate Nalu-Wind’s blade-resolved turbine simulation capabilities for the following reasons:

• It is fully open source and its design and performance characteristics are completely defined in Jonkman et

al. (2009). This will allow the ExaWind team to share scientifically relevant results, as well as the meshes

and input files, from the ExaWind project with the wider wind energy research and development (R&D)

community.

• It is widely used by the R&D community to perform computational and experimental studies on a variety

of topics, including but not limited to turbine wakes (e.g., Troldborg et al. (2012)), wind farm controls (e.g.,

Gebraad et al. (2017)), and offshore systems Lefebvre and Collu (2012). Accordingly, there are publicly

available data sets from other research groups that can be compared to results generated with Nalu-Wind. In

the future, the ExaWind and DOE HFM teams will be able to use the NREL 5-MW turbine model to study

complex physical phenomena at an unprecedented level of detail with the goal of advancing wind turbine and

wind farm technology.

• Fluid-structure interactions between the turbine blades and the atmospheric boundary layer (e.g., blade deformation and bent-twist coupling) affect the turbine performance. Therefore, the NREL 5-MW turbine provides

an excellent test bed to demonstrate Nalu-Wind’s fluid-structure interaction simulation capabilities.

• The turbine size is representative of a modern utility-scale wind turbine and simulations will enable a better

understanding of the physics that govern turbine performance, directly benefiting the wind R&D community.

3.2 Mesh generation

The first step in generating the mesh was developing a CAD model of the turbine using the information presented in

Jonkman et al. (2009). The commercial meshing software Pointwise was then used to create two meshes: a coarse

(gCoarse) mesh consisting of 25 million nodes and higher resolution mesh (g0) consisting of 95 million nodes. The

g0 mesh was then uniformly refined using Percept (part of the Trilinos software stack) to create g1 and g2 meshes.

Table 2 presents the number of nodes and elements in each mesh, and Figure 1 illustrates the g0 mesh.

NREL 5-MW turbine volume meshes are fully unstructured, and consist of hexahedral, tetrahedral, pyramid, and

wedge elements. The blade surface meshes are composed of structured quadrilateral elements. Hub, nacelle, and

tower surface meshes are composed of unstructured quadrilateral-dominant meshes with very few triangular elements. A hexahedral-dominant boundary-layer mesh was extruded from all turbine surfaces (see Figure 1). The

gCoarse mesh has a first cell height of 10−5 m, and the g0 mesh has a first cell height of 10−6 m. The growth rate for

the boundary-layer meshes was 1.15. The boundary-layer mesh for the gCoarse mesh ensured a y+ value of ≤100

Table 1. NREL 5-MW Turbine specifications.

Rating 5 MW

Rotor Orientation, Configuration Upwind, 3 Blades

Control Variable Speed, Collective Pitch

Rotor Diameter 126 m

Hub Height 90 m

Cut-In, Rated, Cut-Out Wind Speed 3 m/s, 11.4 m/s, 25 m/s

Cut-In, Rated Rotor Speed 6.9 rpm, 12.1 rpm

Rated Tip Speed 80 m/s

Overhang, Shaft Tilt, Precone 5 m, 5 degrees, 2.5 degrees

6

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Table 2. NREL 5-MW Turbine mesh characteristics.

Number of nodes (million) Number of elements (million)

gCoarse 25 57

g0 95 176

g1 761 1415

g2 6082 11379

Figure 1. Image of the NREL 5-MW turbine g0 mesh showing the (upper left) domain extents,

(lower left) rotor disk and wake refinement regions, (lower right) hub, nacelle, and blade mesh,

and (upper right) blade tip mesh where the mesh extrusion off the blade surface is visible.

7

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications

for all cells, and the g0, g1, and g2 boundary-layer meshes ensured a y+ value ≤1 for all cells when the turbine

is operating at an oncoming flow velocity of 8.0 m/s. The remainder of the volume mesh consists of tetrahedraldominant isotropic elements.

The computational meshes are composed of two blocks, one containing the rotating blades and rotating portion of

the hub, and another stationary domain that contains all nonrotating turbine components and the inflow and far-wake

regions. This mesh topology enables the use of Nalu-Wind’s nonconformal, discontinuous Galerkin (DG) capability

as described in the FY18 Q1 ExaWind Report, to simulate the motion of the rotating mesh.

8
4 Results

We performed two sets of simulations to demonstrate Nalu-Wind’s simulation capability for a megawatt-scale turbine. First, we simulated the turbine in uniform flow in order to develop the preprocessing (i.e., mesh generation and

refinement), simulation, and postprocessing methodologies on the ANL Mira and NERSC Cori systems. Next, we

performed simulations of the turbine in a HIT inflow. The HIT simulations were performed by adding a turbulent

velocity field to the mean velocity for both volume initialization and the inflow boundary condition. The turbulent

velocity field was taken from external data of a forced homogeneous isotropic simulation run out to a statistically

stationary state. Details on the uniform inflow simulations and the HIT turbulence simulations are described further

in Sections 4.1 and 4.2.

All simulations were performed with the turbine operating in a flow with a mean velocity of 8.0 m/s. At this flow

speed, the NREL 5-MW turbine is operating in “region 2,” where the turbine runs at its design tip-speed ratio

Vblade−tip/|U| = 7.55, where Vblade−tip is the tip speed and U is the mean velocity, and a blade pitch angle of 0 degrees. At this operating point, the rotor rotational speed is 0.95 radians/second and it is operating at peak efficiency.

Today’s utility-scale turbines spend the majority of their operational lives running in “region 2”; therefore, the condition we have simulated is a highly relevant megawatt-scale turbine operating condition.

4.1 Uniform Inflow

The megawatt-scale simulation capability was demonstrated using the gCoarse mesh. Simulations were run on 26

Haswell nodes (832 cores) on the Cori system, with a distribution of 30,000 nodes per MPI rank. All uniform inflow

simulations used the wall-adapting local eddy viscosity (WALE) LES model described in several previous ExaWind

quarterly reports. Simulations were performed using the Trilinos linear-system solver stack (i.e., tpetra, MueLu,

Belos, ifpack2) and the settings are described in Section 5 of the FY18 Q1 ExaWind report.

Seventeen seconds of simulation time, corresponding to 2.85 rotor revolutions, were completed. The simulation

time step was adjusted so that the maximum Courant number was 100 across the entire computational domain, and

after initial startup transients dissipated the time step converged to approximately 10−4seconds. The 17-second

simulation required approximately 336 wall clock hours (14 wall clock days) to complete. Figure 2 presents a visualization of the simulation results. Rotor power and thrust were monitored during the simulation and, at 17 seconds

of simulation time, these quantities were 1.9 MW and 371 kN, respectively. These power and thrust values are within

6% and 20% of the steady-state values calculated using NREL’s FAST reduced-order engineering model (Jonkman

et al. 2009), respectively. While integrated quantities, such as power and thrust, calculated using engineering models

and high-fidelity models typically show good agreement at steady state in uniform inflow, the discrepancies observed could be attributed to the following reasons: (1) The simulation has only completed approximately 3 rotor

revolutions (it typically takes about 5-6 rotor revolutions for the power and thrust to converge to their quasi-steady

values), and (2) The gCoarse mesh has a very coarse resolution of the blade boundary layer, thus it is expected that

the viscous stresses are not adequately resolved for this mesh. The choice of gCoarse mesh was driven by practical

considerations that required a tractable problem for debugging and testing out solver settings as well as developing

the turbulent inflow capability.

We also performed uniform inflow situations on Cori using the g0, g1, and g2 meshes and on ALCF’s Mira system

using the g0 and g1 meshes in order to exercise the megawatt-scale turbine simulation capability on multiple platforms. g0, g1, and g2 simulations were all run for between 100 and 500 time steps and, in the future, these models

can be used to update the figure of merit for the ExaWind project and to generate high-fidelity results that can be

used to study turbine and wake physics at unprecedented levels of detail.

4.2 Turbulent Inflow

Our second test case focuses on establishing a capability to simulate the turbine under turbulent conditions. To

accomplish this, we add a fluctuating field u

0

(x,t) of HIT, taken from external data, to both the initial velocity field

9

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Figure 2. Image of the NREL 5-MW turbine and its wake after the turbine has completed 2.3 revolutions at 14.0

seconds of simulation time. The wake is visualized by showing contours of velocity magnitude of 5.5 m/s.

and the inflow condition at the inlet boundary. Using the same mean field U = (8 m/s,0,0) as in the uniform case

described in Section 4.1, the initial condition for the turbulent inflow is defined as V(x,t) = U+u

0

(x,t).

It must be noted that using HIT to represent the atmospheric boundary layer turbulence is a simplification as it

does not capture the effect of atmospheric shear on the turbulent eddies. However, it is still a relevant step toward

predictive simulations for the following reasons: (1) Synthetic HIT is the de facto standard for loads analysis during

the design of wind turbine blades, and (2) Wind tunnel testing commonly uses grids that generate homogeneous

isotropic turbulence, and many simulations employ synthetic HIT turbulence for studying wake dynamics (MartínezTossas et al. 2018; Kleusberg et al. 2017).

Following the approach described in Martínez-Tossas et al. (2018), we use the John’s Hopkins Turbulence database

http://turbulence.pha.jhu.edu/ (JHTDB) to provide the fluctuating field u

0

(x,t). The JHTDB HIT

case is a 10243triply periodic (2π)

3 box forced to a statistically stationary state. Once it reached the stationary

state, the velocity field was extracted and stored with over 5000 time steps covering approximately five large-eddy

turnover times (Li et al. 2008). This extensive range of data provides a more-than-sufficient tool to create a u

0

(x,t)

field covering the whole domain and an inflow condition over time. Another benefit of the JHTDB is it provides an

easy-to-access web API that can be interfaced with a C++ code to generate the initial and inflow exodus files for

our Nalu-Wind turbulent inflow simulations. The getBoxFilter() function in the database was used with a specified

filter width chosen to match the grid resolution in the mesh onto which the database was mapped, which provides the

added benefit of filtering the HIT data to match the resolution allowed by the physical simulation.

To generate the u

0

(x,t) field from the HIT data, we start by defining a desired turbulent intensity T I, which is then

used to impart a mean velocity of UHIT = uRMS/T I onto the database, where uRMS is the specified root-mean-square

turbulent velocity observed in the database. This defines the velocity scale for the HIT data, whereas the velocity

scale for our physical simulation is the underlying mean flow (8 m/s). Next, we define the length scales, which come

naturally. Our HIT data has a length scale from the box size of LHIT = 2π which will need to be stretched to match

10

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
the length scale in our turbine simulation. Since our turbine domain is rectangular in the y-z plane (4D × 2.71D,

where D is the turbine-rotor diameter), we take the larger of the two dimensions as our physical length scale, L = 4D.

With a length and velocity scale defined, we can then define timescales THIT = LHIT /|UHIT |, T = L/|U|, for both the

database and physical simulation, respectively.

Next, an initial starting time t0 and location x0 are chosen in the database. The velocities in the y − z plane at

(x0,t0) are extracted from the database and stretched by the length scale ratio L/LHIT to represent the initial field

at the inflow boundary x = −5D m in the physical domain. Since L is taken to be the larger of the y and z extents,

this implies that only a subset of the full y − z field is mapped onto the physical domain. This method was chosen to prevent stretching the turbulent field in an anisotropic way. The velocities from the database are scaled by

the velocity scale ratio |U|/|U|HIT to ensure that the desired T I specified in the first step is met. To initialize the

y − z plane at any other x location in the physical domain, the amount of time needed for the flow to reach that

point from the inflow boundary is calculated as t = (x + 5D)/|U|. This is then scaled by the ratio of timescales to

the corresponding database time, tHIT = t ∗ THIT /T. With the associated tHIT , the y − z plane in the database at

(x0 + (x + 5D) ∗ LHIT /L,t0 +tHIT ) is selected. Taking advantage of the periodic nature of the flow in the database,

locations in the database mapped to points outside of the 2π boundaries are represented by their corresponding periodic location inside the domain. This allows us to create an entire initial condition that represents the evolution of

a single slice of the HIT domain, stretched to fit our physical domain and scaled to represent the desired turbulent

intensity T I. The same process can be used to generate the inflow boundary condition, where the x location is chosen

to be at some point upstream of the inflow boundary, associated with some physical time t that would have to elapse

before that point reaches the inflow plane.

Because of the complicated mesh of the physical domain, and to leverage the built-in solution transfer capability

provided by Nalu-Wind, the volume initialization and inflow conditions are stored on an intermediate mesh that has

the same domain as the physical mesh, but has only 128 evenly spaced gridpoints in the y direction, with dx and dz

chosen to be approximately equal to the resulting dy. The initial condition and inflow conditions are then linearly

interpolated onto the actual turbine mesh as part of the external data transfer routine in Nalu-Wind.

In Figure 3, the initial field with the added synthetic HIT is shown for both a horizontal slice through the center

of the hub (left) and a vertical slice at the center of the turbine (right). Qualitatively, there do not appear to be any

negative impacts on the structures from the interpolation, and it can be seen that the turbine will experience turbulent

conditions upon startup of the simulation.

The turbulent inflow simulation was run for a single revolution of the turbine (≈ 6.6 s) and then compared to the

uniform inflow simulation described above. Since this is not a sufficient time to measure quantitative values, we rely

on qualitative comparisons between the two runs. In Figure 4, we show the velocity magnitude contours of 5.5 m/s

(blue) and 10.0 m/s (red). The left plots are from the uniform initialization simulation and the right plots are from the

turbulent initialization with T I = 0.1. Figure 5 shows the velocity magnitude as a horizontal slice through the turbine

hub with a 3D contour of 5.5 m/s. The top plot shows uniform initialization and the bottom plot shows turbulent

initialization with T I = 0.1. Although the simulation has only completed a single revolution under turbulent inflow, it

demonstrates the ability to perform blade-resolved simulations under turbulent inflow conditions with Nalu-Wind.

As expected, a comparison of the uniform inflow and HIT turbulence inflow simulations shows a much more symmetric structure for the uniform case. The turbulent case has a more complex and chaotic wake structure that breaks

down earlier than the wake of the uniform inflow case because of the influence of the HIT turbulence. We note that

with the selected contours of 5.5 m/s and 10.0 m/s, we are just at the edge of the tails for the velocity distribution

after adding in the external HIT field. This allows for the clean view of the turbine and the wake in the turbulent flow

cases shown here.

Additional work related to the turbulent inflow cases will be focused on adding shear to the turbulence to allow for a

more accurate representation of atmospheric turbulence as well as comparing the turbulent flow cases generated here

to those where turbulence from a precursor ABL simulation is used. In addition, we are observing some spurious

oscillations originating from the inflow boundary and propagating along the terrain boundary, which we are currently

investigating.

11

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Figure 3. Initial conditions for the turbulent-inflow simulations. Top: Velocity magnitude is shown as a horizontal slice taken through the center of the hub. Bottom: Velocity magnitude is shown as vertical slice taken through the center of the turbine.

12

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Figure 4. Velocity magnitude contours of 5.5 m/s (blue) and 10.0 m/s (red) after one revolution of the turbine. Left: Uniform initialization. Right: Turbulent initialization with T I = 0.1.

13

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Figure 5. The velocity magnitude is shown as a horizontal slice through

the turbine hub with a 3D contour of 5.5 m/s after one revolution of the turbine. Top: Uniform initialization. Bottom: Turbulent initialization with T I = 0.1.

14

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
5 Concluding remarks and next steps

In this milestone effort, we developed and demonstrated a megawatt-scale turbine simulation capability. The NREL

5-MW reference turbine was selected to demonstrate the capability because it is fully open source and widely used

by the wind turbine R&D community. Other reasons for selecting the NREL 5-MW reference turbine are discussed

in Section 3. Four grids (gCoarse: 25 million nodes, g1: 95 million nodes, g1: 761 million nodes, and g2: 6.082

billion nodes) were generated and used to exercise Nalu-Wind’s per-processing, simulation, and post-processing capabilities on the Cori and Mira systems by running simulations between 50 and 500 timesteps. We used the gCoarse

mesh (25-M nodes) to perform a production simulation of the NREL 5-MW turbine in both uniform inflow and HIT

synthetic turbulence inflow. Approximately 3 turbine revolutions were simulated under uniform inflow and 1 turbine

revolution was simulated under HIT conditions. Although the gCoarse simulations have not yet reached steady state,

the uniform inflow simulation power and thrust values are within 6% and 20%, respectively, of the expected values

that were calculated using NREL’s FAST reduced-order turbine simulation tool (Jonkman et al. 2009).

A comparison of the wake structures between the uniform inflow and HIT synthetic turbulence inflow simulations

shows that wake structures are significantly affected by the presence of turbulence. Specifically, the results presented in Figure 4 show that, as expected, the wake structure is more irregular and breaks down faster under turbulent

conditions. Using Nalu-Wind’s ability to model wake propagation and breakdown under atmospheric conditions,

simulations performed using the more resolved g0-g2 meshes will provide the wind R&D community with an unprecedented level of understanding of turbine and wind farm physics that is required to develop the next generation

of wind turbine and farm technologies.

5.1 Path Forward

The ExaWind team will explore the following paths forward to further develop the Nalu-Wind megawatt-scale

turbine simulation capability and to work toward FY19 ExaWind milestones:

• Run the gCoarse mesh under uniform inflow and HIT synthetic turbulence inflow to steady state to develop

baseline simulation characteristics (e.g., how long it takes integrated quantities, such as power and thrust, to

come to steady state).

• Simulate the NREL 5-MW turbine using the g0 mesh with the k-omega SST-DES model (as developed for the

FY17 Q3 ExaWind milestone) under uniform inflow and atmospheric turbulence conditions. Use the results

to compare Nalu-Wind results to publicly available NREL-5 MW reference turbine performance data to better

understand the physics of turbine performance.

• Perform strong and weak scaling studies on the Mira and Cori systems to understand how Nalu-Wind’s solvers

scale on meshes up to 6 billion nodes.

• Update the ExaWind figure of merit using the g1 mesh on Cori and the g2 mesh on Mira.

• Create a blade-resolved wind farm model by stacking several gCoarse or g0 meshes. Use this mesh to demonstrate Nalu-Wind’s wind farm simulation capabilities and to perform scaling studies using meshes with several

sliding mesh interfaces.

15

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications
Bibliography

Domino, S.P. 2018. “Design-order, non-conformal low-Mach fluid algorithms using a hybrid CVFEM/DG approach”. Journal of Computational Physics 349:331–351.

Gebraad, P., et al. 2017. “Maximization of the annual energy production of wind power plants by optimization of

layout and yaw-based wake control”. Wind Energy 20 (1): 97–107.

Jonkman, J., et al. 2009. “Definition of a 5-MW reference wind turbine for offshore system development”. National

Renewable Energy Laboratory, Golden, CO, Technical Report No. NREL/TP-500-38060.

Kleusberg, E., et al. 2017. “High-Order Numerical Simulations of Wind Turbine Wakes”. Journal of Physics: Conference Series 854:012025.

Lefebvre, S., and M. Collu. 2012. “Preliminary design of a floating support structure for a 5 MW offshore wind

turbine”. Ocean Engineering 40:15–26.

Li, Y., et al. 2008. “A public turbulence database cluster and applications to study Lagrangian evolution of velocity

increments in turbulence”. Journal of Turbulence 9:N31.

Martínez-Tossas, L.A., et al. 2018. “Comparison of four large-eddy simulation research codes and effects of model

coefficient and inflow turbulence in actuator-line-based wind turbine modeling”. Journal of Renewable and Sustainable Energy 10:033301.

Troldborg, N., et al. 2012. “Comparison of the wake of different types of wind turbine CFD models”. In 50th AIAA

Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition, 237.

16

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications