OCS Study

BOEM 2015-049

US Department of the 

Interior

Bureau of Ocean Energy 

Management

Alaska OCS Region

Eastern Research Group, 

Inc.

8950 Cal Center Drive

Suite 325

Sacramento, CA 95826

Ramboll Environ US 

Corporation

19020 33rd Ave. W.

Suite 310

Lynnwood, WA 98036-4754

Arctic Air Quality Modeling Study

Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

[Page Intentionally Left Blank]
OCS Study

BOEM 2015-049

Arctic Air Quality Modeling Study

Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

Prepared by

Ramboll Environ US Corporation

19020 33rd Ave. W., Suite 310

Lynnwood, WA 98036-4754

Prepared for

U.S Department of Interior

Bureau of Ocean Energy Management

3801 Centerpoint Drive, Suite 500

Anchorage, AK 99503-5823

 

US Department of the 

Interior

Bureau of Ocean Energy 

Management

Alaska OCS Region

Eastern Research 

Group, Inc.

8950 Cal Center Drive

Suite 325

Sacramento, CA 95826

Ramboll Environ US 

Corporation

19020 33rd Ave. W.

Suite 310

Lynnwood, WA 98036-4754

January 18, 2016
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

i

Disclaimer

This report has been reviewed by the BOEM and approved for publication. Approval does not 

signify that the contents necessarily reflect the views and policies of the Bureau, nor does 

mention of trade names or commercial products constitute endorsement or recommendation for 

use.

May be cited as: 

Brashers, B.; Knapik, J.; McAlpine, J.D.; Sturtz, T.; Morris, R. 2016. Arctic Air Quality 

Modeling Study – Meteorological Model Performance Evaluation: 2009-2013 BOEM 

Arctic WRF Dataset. Prepared by Ramboll Environ U.S. Corporation for U.S. 

Department of the Interior, Bureau of Ocean Energy Management. OCS Study BOEM 

2015-049, 50 pp.

Authors:

Bart Brashers

Joe Knapik

JD McAlpine

Timothy Sturtz

Ralph Morris
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

ii

METEOROLOGICAL MODEL PERFORMANCE EVALUATION

2009-2013 BOEM Arctic WRF Dataset

Table of Contents

Section Page

METEOROLOGICAL MODEL PERFORMANCE EVALUATION...........................................II

1.0 INTRODUCTION .............................................................................................................. 6

2.0 METEOROLOGICAL MODELING ................................................................................. 6

2.1 Arctic Air Quality Meteorological Modeling ......................................................... 7

3.0 WRF MODELING METHODOLOGY ............................................................................. 8

3.1 Model Inputs and Configuration............................................................................. 8

3.1.1 WRF Version and Options.......................................................................... 9

3.1.2 Domain Configuration .............................................................................. 11

3.1.3 Model Vertical Resolution........................................................................ 12

4.0 WRF MODEL PERFORMANCE EVALUATION RESULTS....................................... 13

4.1 Quantitative Evaluation Using METSTAT .......................................................... 13

4.1.1 Quantitative Statistics............................................................................... 13

4.1.2 Onshore METSTAT Evaluation ............................................................... 14

4.1.3 Offshore METSTAT Evaluation............................................................... 18

4.2 Qualitative Evaluation Using Upper-Air Data...................................................... 21

4.2.1 Point Barrow Soundings........................................................................... 21

4.2.2 Other Upper-Air Datasets......................................................................... 22

4.3 Qualitative Evaluation Using PRISM Precipitation ............................................. 28

4.4 Qualitative Evaluation Using Satellite Cloud Cover Observations...................... 34

5.0 SUMMARY AND CONCLUSIONS ............................................................................... 49

6.0 REFERENCES ................................................................................................................. 49
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

iii

List of Figures

Figure 1. BOEM Arctic WRF Domains, 36km (full plot), 12km (d02) and 4km (d03) grid 

resolution......................................................................................................................................... 8

Figure 2. BOEM Arctic WRF Onshore METSTAT d03 Wind Direction Performance .............. 16

Figure 3. BOEM Arctic WRF Onshore METSTAT d03 Humidity Performance ........................ 16

Figure 4. BOEM Arctic WRF Onshore METSTAT d03 Wind Speed Performance.................... 17

Figure 5. BOEM Arctic WRF Onshore METSTAT d03 Temperature Performance ................... 17

Figure 6. BOEM Arctic WRF Offshore METSTAT d03 Wind Direction Performance.............. 19

Figure 7. BOEM Arctic WRF Offshore METSTAT d03 Humidity Performance ....................... 19

Figure 8. BOEM Arctic WRF Offshore METSTAT d03 Wind Speed Performance ................... 20

Figure 9. BOEM Arctic WRF Offshore METSTAT d03 Temperature Performance .................. 20

Figure 10. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point 

Barrow, AK (PABR) on April 21, 2009 at 00 UTC and July 6, 2009 at 12 UTC........................ 23

Figure 11. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point 

Barrow, AK (PABR) on May 1, 2010 at 00 UTC, and September 5, 2010 at 00 UTC................ 24

Figure 12. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point 

Barrow, AK (PABR) on June 8, 2011 at 12 UTC and October 2, 2011 at 12 UTC..................... 25

Figure 13. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point 

Barrow, AK (PABR) on September 24, 2012 and November 12, 2012 at 00 UTC..................... 26

Figure 14. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point 

Barrow, AK (PABR) on January 4, 2013 at 12 UTC and July 7, 2013 at 12 UTC...................... 27

Figure 15. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for January in the 4 km domain........................... 29

Figure 16. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for February in the 4 km domain......................... 30

Figure 17. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for March in the 4km domain.............................. 31

Figure 18. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for April in the 4km domain................................ 32

Figure 19. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for May in the 4km domain................................. 33

Figure 20. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for June in the 4km domain................................. 35

Figure 21. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for July in the 4km domain ................................. 36

Figure 22. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for August in the 4km domain............................. 37

Figure 23. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for September in the 4km domain....................... 38

Figure 24. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for October in the 4km domain ........................... 39

Figure 25. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for November in the 4km domain ....................... 40

Figure 26. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM precipitation 

average (bottom) monthly precipitation totals for December in the 4km domain........................ 41
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

iv

Figure 27. MISR (left) and WRF (right) CCF, January 2011....................................................... 42

Figure 28. MISR (left) and WRF (right) CCF, March 2011......................................................... 43

Figure 29. MISR (left) and WRF (right) CCF, April 2012........................................................... 43

Figure 30. MISR (left) and WRF (right) CCF, May 2011............................................................ 44

Figure 31. MISR (left) and WRF (right) CCF, June 2009............................................................ 45

Figure 32. MISR (left) and WRF (right) CCF, July 2009 ............................................................ 46

Figure 33. MISR (left) and WRF (right) CCF, August 2013........................................................ 46

Figure 34. MISR (left) and WRF (right) CCF, September 2009.................................................. 47

Figure 35. MISR (left) and WRF (right) CCF, September 2012.................................................. 47

Figure 36. MISR (left) and WRF (right) CCF, October 2012 ...................................................... 48

Figure 37. MISR (left) and WRF (right) CCF, December 2011................................................... 48

List of Tables

Table 1. Physics Options used in the BOEM Arctic WRF Dataset.............................................. 11

Table 2: BOEM Arctic WRF Domain Configuration................................................................... 11

Table 3. BOEM Arctic WRF Dataset Model Levels.................................................................... 12

Table 4. Meteorological model performance benchmarks for simple and complex conditions... 14
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

v

Abbreviations and Acronyms

AMSR Advanced Microwave Scanning Radiometer

AQ Air Quality

ARW Advanced Research WRF

BOEM Bureau of Ocean Energy Management

CCF Cloud Cover Fraction

C-MAN Coastal-Marine Automated Network stations

CMC Canadian Meteorological Centre

ECMWF European Center for Medium-range Weather Forecasting

ERG Eastern Research Group, Inc.

EPA US Environmental Protection Agency

FDDA Four Dimensional Data Assimilation

JAMSTEC Japan Agency for Marine-Earth Science and Technology

LCC Lambert Conformal Conic map projection

LSM Land-Surface Model

MADIS Meteorological Assimilation Data Ingest System observation archive

METSTAT Meteorological /Statistical Program

MISR Multi-angle Imaging SpectroRadiometer

MMM Meso-scale Meteorological Modeling

MPE Model Performance Evaluation

NASA National Aeronautics and Space Administration

NCAR National Center for Atmospheric Research

NCEP National Center for Environmental Prediction

NCDC National Climatic Data Center

NDBC National Data Buoy Center

NMM Nonhydrostatic Mesoscale Model

NOAA National Oceanographic and Atmospheric Administration 

NODC National Oceanographic Data Center

NWS National Weather Service

PABR Meteorological call sign for Point Barrow

PBL planetary boundary layer

PRISM Parameter-elevation Regressions on Independent Slopes Model

RMSE Root Mean Square Error

RRTMG Rapid Radiative Transfer Model for Global Climate Models 

SCAS-OSU Spatial Climate Analysis Service at Oregon State University

SIP State Implementation Plan

SST Sea Surface Temperature

UAF University of Alaska Fairbanks

UTC Universal Time Coordinate

WPS WRF Pre-processing System

WRAP Western Regional Air Partnership

WRF Weather Research and Forecasting model

YSU Yonsei University
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

6

1.0 INTRODUCTION

This report presents a model performance evaluation (MPE) of a Weather Research and 

Forecasting (WRF) model run, simulating the atmosphere of the North Slope of Alaska. The 

results being evaluated represent five years (2009-2013) of WRF meteorological modeling 

performed by Ramboll Environ. The purpose of the modeling is to provide the meteorological 

dataset to the Bureau of Ocean Energy Management (BOEM) for air quality (AQ) modeling in 

the Arctic. The Arctic AQ Modeling Study is being conducted under BOEM Contract Number 

M13PC00014 by the team of Eastern Research Group, Inc. (ERG) and Ramboll Environ US 

Corporation (Ramboll Environ) with assistance from the University of Alaska Fairbanks (UAF).

The BOEM Arctic WRF meteorological dataset was based on a previous EPA/BOEM WRF 

dataset with updates to improve performance in Northern Alaska. This assessment evaluates the 

BOEM Arctic WRF dataset on the bases of METSTAT, vertical profile, precipitation, and cloud 

cover outputs. The use of the METSTAT program and surface meteorological observations was 

used to evaluate WRF for wind speed, wind direction, temperature and mixing ratio. An

evaluation of the vertical profile of WRF temperature and dew point temperature against twice 

daily upper-air observations from Point Barrow, Alaska was also completed. The PRISM model 

output was used to evaluate the spatial distribution and magnitudes of the WRF precipitation 

estimates. Finally, WRF cloud cover estimates were compared to satellite datasets in a 

qualitative manner.

2.0 METEOROLOGICAL MODELING

Over the past decade, emergent requirements for numerical simulation of urban and regional 

scale air quality have led to intensified efforts to construct high-resolution emissions, 

meteorological and air quality data sets. It is now possible, for example, to exercise sophisticated 

mesoscale prognostic meteorological models and Eulerian and Lagrangian 

photochemical/aerosol models for multi-seasonal periods over near-continental scale domains in 

a matter of weeks with the application tailored to a specific air quality modeling project.

The WRF model is the current preferred model for atmospheric research and operational 

forecasting needs at mesoscale resolution (approximately 5 to several hundred km). The model is 

the state-of-the-art atmospheric simulation system, commonly used to drive air quality dispersion 

models on the regional level. 

The operational version of the model is the Nonhydrostatic Mesoscale Model (NMM) WRF core 

version 3, developed and maintained by the National Oceanic and Atmospheric Administration 

(NOAA) and the National Centers for Environmental Prediction (NCEP) (NOAA, 2015). The 

Advanced Research WRF (ARW) core, currently version WRF 3.7.1, is supported by the NCAR 

Mesoscale and Microscale Meteorology Division (NCAR, 2015). Based on extensive experience 

with mesoscale modeling in the Polar Regions by the Polar Meteorology Group of the Byrd 

Polar and Climate Research Center at The Ohio State University, WRF-ARW has been modified 

for use in the Polar Regions (referred to as the Polar WRF or PWRF). The modeling described in 

this report used PWRF version 3.6.1.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

7

The WRF model contains separate modules to compute different physical processes such as 

surface energy budgets and soil interactions, turbulence, cloud microphysics, and atmospheric 

radiation. Within WRF, the user has many options for selecting the different schemes for each 

type of physical process. There is a WRF Pre-processing System (WPS) that generates the initial 

and boundary conditions used by WRF, based on topographic datasets, land use information, and 

larger-scale atmospheric and oceanic models.

2.1 Arctic Air Quality Meteorological Modeling

An evaluation by Ramboll Environ (Brashers et al., 2015) found the existing Arctic 

meteorological datasets to be insufficient for the needs of the Arctic Air Quality Modeling Study 

and recommended upgrading the configuration of the previous EPA/BOEM WRF to create a 

high-resolution, five-year dataset. An optimized version of the EPA/BOEM WRF dataset, known 

as the “BOEM Arctic WRF”, was deemed necessary for use in air quality modeling within the 

Arctic Air Quality Modeling Study domain. This final BOEM Arctic WRF model configuration, 

evaluated in this report, is very similar to the initial EPA/BOEM WRF configuration, but differs 

from it in the following ways:

• Use of Polar WRF version 3.6.1 instead of version 3.4.1 used in the initial EPA/BOEM 

WRF.

• The inner-most 4 km domain was expanded slightly eastward, as shown in Figure 1.

• Use of the UKMO high resolution sea surface temperature (SST) dataset in place of the 

Real Time Global (RTG) dataset.

• Use of the Thompson microphysics scheme over the Morrison scheme.

• Use of the Grell-Freitas cumulus parameterization in the 36 km, 12 km, and 4 km 

domains instead of Kain-Fritsch scheme in the 36 km and 12 km domains.

• Increasing the time between radiation physics calls from 5 min to 20 min in all three 

domains.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

8

Figure 1. BOEM Arctic WRF Domains, 36km (full plot), 12km (d02) and 4km (d03) grid 

resolution

3.0 WRF MODELING METHODOLOGY

This chapter describes the methodology used for conducting the WRF simulation for the January 

2009 through December 2013 modeling period and also describes the model configuration used 

in the final WRF application for the Arctic Air Quality Modeling Study.

3.1 Model Inputs and Configuration

A brief summary of the WRF configuration and input data used for this model performance 

evaluation is provided below.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

9

3.1.1 WRF Version and Options 

Model Selection: The Polar version of WRF version 3.6.1 was used for the BOEM Arctic 

simulation over the North Slope of Alaska. This version of the model was developed to improve 

WRF Arctic modeling capabilities (Hines and Bromwich 2008) (Bromwich, et al., 2009). Their 

modifications have focused on optimizing the surface energy budget and parameterization of seaice and permanent ice surfaces within the Noah land surface model (Chen, Janjic and Mitchell 

1997). The modifications have included implementation of a variable sea-ice and snow thickness 

and seasonally-variable sea-ice albedo. The WPS pre-processor programs including GEOGRID, 

UNGRIB and METGRID were used to develop model inputs.

Topographic Inputs: Topographic information for the WRF was developed using the standard 

WRF terrain databases. The 36 km domain was based on the 10 minute (18 km) global data. The 

12 km domain was based on the 2 minute (~4 km) data. The 4 km domain was based on the 30 

second (~900 m) data.

Vegetation Type and Land Use Inputs: Vegetation type and land use information were

developed using the most recently released WRF databases provided with the WRF distribution. 

Standard WRF surface characteristics corresponding to each land use category were employed.

Atmospheric Data Inputs: WRF relies on other model or re-analysis output to provide initial and 

boundary conditions (IC/BC). Typically, these are available in coarser time- and space-scales 

than WRF produces, e.g. approximately 70 km and 6-hourly for the European Centre for 

Medium-Range Weather Forecasts (ECMWF) interim reanalysis dataset (ERA-Interim). These 

were objectively re-analyzed using traditional observational data (meteorological towers) to the 

higher resolution of each WRF grid, using the OBSGRID program. These fields are used both to 

initialize the model, and to perform analysis nudging to guide the model to best match the 

observations.

Time Integration: Third-order Runge-Kutta integration was used with a variable time step of 120 

to 90 seconds for the 36 km domain, 60 to 30 seconds for the 12 km domain, and 20 to 10 

seconds for the 4 km domain.

Diffusion Options: Horizontal Smagorinsky first-order closure with sixth-order numerical 

diffusion and suppressed up-gradient diffusion was used.

Lateral Boundary Conditions: Lateral boundary conditions were specified from the initialization 

dataset on the 36 km domain with continuous updates nested from the 36 km domain to the 12 

km Alaskan domain and continuous updates nested from the 12 km domain to the 4 km North 

Slope domain.

Top and Bottom Boundary Conditions: The top boundary condition was selected as an implicit 

Rayleigh dampening for the vertical velocity. Consistent with the model application for nonidealized cases, the bottom boundary condition was selected as physical, not free-slip.

Water Temperature Inputs: The UK Met Office’s (UKMO) Operational Sea Surface 

Temperature and Sea Ice Analysis dataset was selected after extensive testing of several SST 

databases. The UKMO’s SST analysis uses a spatial resolution of 0.054 degree (~6 km 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

10

resolution). The UKMO SST dataset was chosen for the lowest bias and error in model 

performance evaluation tests during open water periods and its graphical similarity depicting the 

Beaufort Sea and Mackenzie River discharge in summer. 

Snow Depth/Sea Ice Thickness Inputs: The runs used the 24 km resolution snow/ice data from 

the Canadian Meteorological Centre (CMC) Daily Snow Depth Analysis to supply the input 

snow cover over land. Additionally, the BOEM Arctic WRF runs used three over-ocean datasets 

developed by the Ohio State University Polar Meteorology Group. Dr. Hines, of the Ohio State 

University Polar Meteorology Group, supplied datasets of sea ice thickness, snow-on-ice

thickness, and sea ice/snow albedo based on radiances collected by the Advanced Microwave 

Scanning Radiometer (AMSR). These data were used as input fields for Polar WRF. Although 

WRF writes these fields in its output, they are not predicted fields, only “passed through” from 

the inputs.

FDDA Data Assimilation: The WRF model was run with a combination of analysis and 

observation nudging (i.e., Four Dimensional Data assimilation [FDDA]). Analysis nudging was 

used for winds, temperature, and humidity on the 36 and 12 km domains above the planetary 

boundary layer (PBL) height in both the initial and final WRF simulations. The WRF simulation 

used observation nudging within the 4 km domain for winds, temperature, and humidity. The 

nudging used both surface and aloft nudging, but nudging for temperature and mixing ratio was 

not performed in the lower atmosphere (i.e., within the boundary layer). Observation nudging for 

winds and temperature was performed on the 4 km grid domain using the Meteorological 

Assimilation Data Ingest System (MADIS)1 observation archive. The MADIS archive includes 

the National Climatic Data Center (NCDC)2 observations and the National Data Buoy Center 

(NDBC) Coastal-Marine Automated Network C-MAN3 stations.

Physics Options: The WRF model contains many different physics options. The physics options 

chosen for the BOEM Arctic WRF configuration are summarized in Table 1. See the WRF 

User’s Guide4 or its Physics References page5 for more details on each option.

Application Methodology: The WRF model was executed in 5-day blocks initialized at 12Z 

every 5 days. Model results were output every 60 minutes and output files were split at 12 hour 

intervals. Twelve (12) hours of spin-up were included in each 5-day block before the data were 

used in the subsequent evaluation. The model was run at the 36, 12 and 4 km resolution from 

January 2009 through December 2013.

 1 Meteorological Assimilation Data Ingest System. http://madis.noaa.gov/ 2 National Climatic Data Center. http://lwf.ncdc.noaa.gov/oa/ncdc.html 3 National Data Buoy Center. http://www.ndbc.noaa.gov/cman.php 4 WRF User’s Guide. http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_V3/contents.html 5 WRF Physics References http://www2.mmm.ucar.edu/wrf/users/wrfv3.5/phys_references.html
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

11

Table 1. Physics Options used in the BOEM Arctic WRF Dataset

Physics Parameterization Scheme Description

Long/Shortwave 

Radiation 

Rapid Radiative Transfer Model 

for GCM (RRTMG)6

Scheme with the MCICA method of 

random cloud overlap

Micro-physics Thompson6

Scheme with ice, snow and graupel 

processes suitable for highresolution simulations

Cumulus physics Grell-Freitas6 Scheme that tries to smooth the 

transition to cloud-resolving scales

Planetary boundary 

layer (PBL) Yonsei University (YSU)

6

Scheme with explicit entrainment 

layer and parabolic K profile in 

unstable mixed layer

Land surface model 

(LSM)

NOAH6 with Polar WRF 

modifications

Scheme with soil temperature and 

moisture in four layers, fractional 

snow cover and frozen soil physics

3.1.2 Domain Configuration

The WRF domain configuration is comprised of a system of simultaneous nested grids. All WRF 

grids are defined on a Lambert Conformal Conic (LCC) projection centered at 70°N, 155°W 

with true latitudes at 70°N. The final dataset was produced using three domains as show in

Figure 1: a 4 km resolution domain centered on the Northern Alaska coast, a 12 km domain that 

encompasses most of Alaska and the Beaufort and Chukchi Seas and an extended 36 km domain 

designed to capture synoptic scale features (full plot). Table 2 provides the input configurations 

for the WRF domains used in this project. NX and NY are the number of east-west and northsouth staggered grid points, respectively, in each domain. I-start and J-start indicate the western 

and southern nested grid starting indices with respect to the parent grid. Geographic Resolution 

relates to the geographic datasets employed for each grid, in terms of minutes or seconds of 

degrees.

Table 2. BOEM Arctic WRF Domain Configuration

Grid 

Resolution NX NY I-start J-start Geographic Resolution Coverage

36 km 120 110 1 1 10 minute NW North

America & NE 

Russia

12 km 157 130 33 31 2 minute Alaska

4 km 289 151 36 53 30 second N Alaska

 6 WRF ARW Version 3 Modeling System User’s Guide 

http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap5.htm#
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

12

3.1.3 Model Vertical Resolution 

High vertical resolution enables the model to more accurately capture low level inversions 

frequently present throughout the winter months. The WRF modeling was based on 34 vertical 

layers with levels packed near the surface in an attempt to improve the simulation atmospheric 

boundary layer structure and processes, shown in Table 3.

Table 3. BOEM Arctic WRF Dataset Model Levels

Level eta Pressure 

(mb)

Height 

(m)

Mid Height 

(m)

Thickness

(m)

0 1 1000 0.0

1 0.9985 999 11.6 5.8 11.6

2 0.997 997 23.2 17.4 11.6

3 0.995 996 38.7 30.9 15.5

4 0.992 993 62.0 50.3 23.3

5 0.987 988 100.9 81.4 38.9

6 0.98 982 155.6 128.2 54.7

7 0.97 973 234.3 194.9 78.7

8 0.96 964 313.5 273.9 79.3

9 0.95 955 393.4 353.4 79.9

10 0.94 946 473.8 433.6 80.5

11 0.93 937 554.9 514.4 81.1

12 0.915 924 677.8 616.3 122.8

13 0.90 910 802.1 739.9 124.3

14 0.88 892 970.1 886.1 168.1

15 0.85 865 1227.4 1098.8 257.3

16 0.82 838 1491.3 1359.4 263.9

17 0.78 802 1854.1 1672.7 362.8

18 0.74 766 2230.4 2042.3 376.2

19 0.70 730 2621.2 2425.8 390.8

20 0.64 676 3237.8 2929.5 616.6

21 0.58 622 3895.7 3566.8 657.9

22 0.52 568 4601.6 4248.6 705.9

23 0.455 510 5430.5 5016.0 828.9

24 0.40 460 6194.4 5812.5 763.9

25 0.35 415 6949.1 6571.7 754.6

26 0.30 370 7773.2 7361.1 824.1

27 0.25 325 8682.9 8228.0 909.7

28 0.20 280 9701.0 9191.9 1018.1

29 0.15 235 10861.6 10281.3 1160.6

30 0.10 190 12218.6 11540.1 1357.1

31 0.06 154 13506.9 12862.8 1288.3

32 0.027 124 14768.9 14137.9 1262.0

33 0 100 15998.6 15383.8 1229.7
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

13

4.0 WRF MODEL PERFORMANCE EVALUATION RESULTS

The model performance evaluation using the WRF configuration described herein includes a 

quantitative and qualitative component. The quantitative evaluation compares surface 

meteorological observations with corresponding WRF predictions at specific times and locations. 

The qualitative evaluation performs a larger-scale comparison of spatial fields of meteorological 

observations with gridded WRF fields.

4.1 Quantitative Evaluation Using METSTAT

4.1.1 Quantitative Statistics

A quantitative model performance evaluation of the BOEM Arctic WRF simulation was 

performed using surface meteorological measurements and the publicly-available METSTAT 

evaluation tool. METSTAT calculates statistical performance metrics for bias, error and 

correlation for surface winds, temperature, and mixing ratio (i.e., water vapor or humidity). To 

evaluate the performance of a meteorological model simulation for air quality model 

applications, a number of performance benchmarks for comparison are typically used. Table 4

lists the meteorological model performance benchmarks for simple (Emery et al., 2001) and 

complex (Kemball-Cook et al., 2005) situations. The simple benchmarks were developed by 

analyzing well-performing meteorological model evaluation results for simple, mostly flat terrain 

conditions and simple meteorological conditions (e.g., stationary high pressure) that were mostly 

conducted to support air quality modeling studies (e.g., ozone State Implementation Plan [SIP]

modeling). The complex benchmarks were developed during the Western Regional Air 

Partnership (WRAP) regional haze modeling and are performance benchmarks for more complex 

conditions, such as the complex terrain of the Rocky Mountains and Alaska (Kemball-Cook et 

al., 2005). McNally (2009) analyzed multiple annual runs that included complex terrain 

conditions and suggested an alternative set of benchmarks for temperature and humidity under 

more complex conditions. The purpose of the benchmarks is to understand how good or poor the 

results are relative to other model applications run for the United States. 

In this section, Ramboll Environ compared the WRF meteorological statistics to the benchmarks 

as an indication of the BOEM Arctic WRF model performance. These benchmarks include bias 

and error in temperature, wind direction and mixing ratio as well as the wind speed bias and Root 

Mean Squared Error (RMSE) between the model and observations.

Because air dispersion models are all statistically-based and use meteorology as a statistical 

sample of meteorological conditions, the timing of specific conditions matters less than that a 

condition occurred during the modeling period. The WRF model generally has some error in 

predicting the timing of the passage of surface cold/warm fronts. Errors in frontal passage timing 

can enhance errors in certain statistics, e.g. temperature and wind direction. These errors do not 

necessarily predict poor performance of air dispersion models that use the WRF data, as air 

dispersion models are typically evaluated on longer timescales or using “disaggregated in time” 

statistics. 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

14

Table 4. Meteorological Model Performance Benchmarks for Simple and Complex 

Conditions

Parameter Emery et al. 

(2001)

Kemball-Cook 

et al. (2005) McNally (2009) Resulting Criteria

Conditions Simple Complex Complex Complex

Temperature Bias ≤ ±0.5 K ≤ ±2.0 K ≤ ±1.0 K ≤ ±1.0 K

Temperature Error ≤ 2.0 K ≤ 3.5 K ≤ 3.0 K ≤ 3.0 K

Temperature IOA ≥ 0.8 (not addressed) (not addressed) ≥ 0.8

Humidity Bias ≤ ±1.0 g/kg ≤ ±0.8 g/kg ≤ ±1.0 g/kg ≤ ±1.0 g/kg

Humidity Error ≤ 2.0 g/kg ≤ 2.0 g/kg ≤ 2.0 g/kg ≤ 2.0 g/kg

Humidity IOA ≥ 0.6 (not addressed) (not addressed) ≥ 0.6

Wind Speed Bias ≤ ±0.5 m/s ≤ ±1.5 m/s (not addressed) ≤ ±1.5 m/s

Wind Speed RMSE ≤ 2.0 m/s ≤ 2.5 m/s (not addressed) ≤ 2.5 m/s

Wind Speed IOA ≥ 0.6 (not addressed) (not addressed) ≥ 0.6

Wind Dir. Bias ≤ ±10 degrees (not addressed) (not addressed) ≤ ±10 degrees

Wind Dir. Error ≤ 30 degrees ≤ 55 degrees (not addressed) ≤ 55 degrees

The output from the BOEM Arctic WRF simulation was compared against the global-scale 

NCDC DS-3505 observational data, and against data from the buoys stored in the NOAA 

National Oceanographic Data Center (NODC) database. Typical measurement errors for onshore 

stations can be found in FAA Advisory Circular No: 150/5220-16D7

, and for buoys at the 

NDBC’s website.8 For example, onshore wind speed sensors are accurate to within 1 knot (0.5 

m/s) for ultrasonic anemometers and 2 knots (1.0 m/s) for cup anemometers, with wind direction 

accurate to with 3 to 5 degrees RMSE, respectively. Buoy wind measurements are less accurate, 

at ±1.0 m/s (speed) and ±10 degrees (direction).

A standard set of statistical metrics from the METSTAT package was used. These metrics were 

calculated on hourly, daily and monthly time frames for wind speed, wind direction, temperature, 

and humidity at the surface; using all available observational weather data.

The WRF surface meteorological model performance metrics were compared against the simple 

and complex model performance goals using “soccer plots.” Soccer plots display two WRF 

performance metrics on the x-axis and y-axis (e.g., monthly-averaged temperature bias and error) 

along with the performance benchmarks. Generally, the closer the symbols are to the origin, the 

better the model performance. It is also easy to see when a WRF performance metric falls within 

the benchmark lines (i.e., score a goal). Below we present example monthly surface 

meteorological model performance across the 4 km North Slope Alaska domain (domain d03).

4.1.2 Onshore METSTAT Evaluation

METSTAT was used to evaluate onshore model performance of the BOEM-Arctic WRF dataset

using the National Climate Data Center’s (NCDC) global-scale quality-controlled DS-3505 

integrated surface hourly observational data as verification data (NOAA-NCDC, 2014). Global 

 7 FAA AWOS Advisory http://www.faa.gov/documentLibrary/media/Advisory_Circular/150-5220.16D.pdf 8 NDBC accuracy tables http://www.ndbc.noaa.gov/rsa.shtml
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

15

hourly and synoptic observations are compiled from numerous sources into a single common 

ASCII format and common data model. The DS-3505 database contains records of most official 

surface meteorological stations from airports, military bases, reservoirs/dams, agricultural sites, 

and other sources dating from 1901 to the present. Quality control of the data has corrected well 

over 99% of the errors present in the original data (NOAA-NCDC, 2015).

The onshore METSTAT soccer plot results for BOEM Arctic WRF wind direction performance 

in the 4 km domain are shown in Figure 2. Wind direction performed very well with all months 

meeting the simple conditions benchmark. Figure 3 shows humidity performance in the 4 km 

domain. Again, WRF performed very well with each month falling within the simple conditions 

benchmark with biases ranging from -0.4 to 0.4 g/kg. Wind speed performance for the 4km 

domain is shown in Figure 4. WRF performed well with a small wind speed bias, but a RMSE 

ranging from 1.5 m/s to 2.5 m/s – approximately half of the months fall within the simple 

conditions benchmark and the other half within the complex conditions. The fall and winter 

months generally display a higher wind speed error throughout the year, compared to the openwater periods.

Figure 5 displays soccer plot results for temperature performance for the 4 km domain. BOEM 

Arctic WRF performed reasonably well with a majority of months falling within the complex 

conditions benchmark, but only several months within the simple conditions benchmark. 

Monthly plots display a negative temperature bias overall with increased temperature error 2.0 to 

4.5 K for the winter months (particularly 2010) and decreased temperature error of 1.0 to 2.0 K 

for the open water months. The outliers are winter months from each year and display a very 

negative (cold) temperature bias, outside of the complex condition benchmark. The WRF 

model’s under-prediction of temperatures is likely due to the persistent ice coverage throughout 

the late fall, winter and early spring months.

Overall, WRF performed well compared to onshore surface observations for wind speed, wind 

direction, humidity and temperature for all months in the five-year dataset. For 2012 in 

particular, only one or two month-parameter combinations fell outside the complex condition 

benchmarks.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

16

Figure 2. BOEM Arctic WRF Onshore METSTAT d03 Wind Direction Performance

Figure 3. BOEM Arctic WRF Onshore METSTAT d03 Humidity Performance
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

17

Figure 4. BOEM Arctic WRF Onshore METSTAT d03 Wind Speed Performance

Figure 5. BOEM Arctic WRF Onshore METSTAT d03 Temperature Performance
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

18

4.1.3 Offshore METSTAT Evaluation

METSTAT was used to evaluate WRF performance using observations from meteorological 

buoys stored in the NOAA National Oceanographic Data Center (NODC) database. The 

observation datasets used for the METSTAT analysis included the data collected by shipboard 

and moored meteorological buoy data collected in the Chukchi Sea from NODC Accession 

0093399 (NOAA-NODC, 2014). The originator of each Accession performs QA/QC on the data 

prior to uploading to the NODC database, and the accuracy of the observations may be lower, 

compared to that of a quality-controlled DS3505 database. Note that the buoys deployed in 2012 

and 2013 only reported temperature, not winds or humidity.

Because the WRF output fields related to sea ice are simply “passed through” from the input 

dataset, no assessment of their performance was attempted. A discussion of the uncertainty of the 

satellite AMSR sensor and many of the derived products is available from the National Snow 

and Ice Data Center.9

The offshore METSTAT soccer plot results for BOEM Arctic wind direction performance in the 

4 km domain are shown in Figure 6. A majority of months displayed a slight positive bias with 

an. The average direction error was in the 20-45 degree range, with all months falling within the 

complex conditions benchmark and about half the months falling within the simple conditions 

benchmark. 

The WRF model performed very well for humidity, with all open water months within the simple 

conditions benchmark as shown in Figure 7. In Figure 8, there is overall a small positive wind 

speed bias for the periods of open water. October of 2009, 2010, and 2012 have wind speed 

errors of 2.5 to 3.5 m/s, falling outside the complex conditions threshold indicating that the 

model had difficulties in the transition month. Temperature performance is shown in Figure 9,

with a majority of the open water months within the simple conditions benchmark. Most 

temperature plots display a slight negative bias, suggesting WRF is under-predicting 

temperatures during the warmer months.

Overall, WRF performance is satisfactory when compared to offshore observations. The offshore 

METSTAT performance appears less accurate compared to onshore METSTAT performance. 

This apparent difference is partly a result of the small number of available buoys relative to the 

number of stations onshore, because positive errors from one station and negative errors from 

another station can give average results that appear unbiased. The larger the number of stations 

in an average, the more likely they balance out. There is also a high frequency of missing 

observations in the NODC dataset.This speaks to the inherent difficulty in taking measurements 

in this geographic area, where the sea swell may be much larger than the height of the buoy mast, 

and the wind speed may be very high.

 9 AMSR uncertainties https://nsidc.org/data/amsre/data-quality/data-uncertainty.html
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

19

Figure 6. BOEM Arctic WRF Offshore METSTAT d03 Wind Direction Performance

Figure 7. BOEM Arctic WRF Offshore METSTAT d03 Humidity Performance
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

20

Figure 8. BOEM Arctic WRF Offshore METSTAT d03 Wind Speed Performance

Figure 9. BOEM Arctic WRF Offshore METSTAT d03 Temperature Performance
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

21

4.2 Qualitative Evaluation Using Upper-Air Data

These analyses were conducted using upper air observation datasets within the study domain at 

Point Barrow, Alaska, which have time frames that overlap with the WRF modeling. 

Upper-air data from the Point Barrow radiosonde (PABR) dataset were used to evaluate WRF 

vertical profiles of predicted temperature and moisture above that location. The PABR 

radiosonde dataset is collected by and maintained by the National Weather Service (NWS). 

Radiosondes are launched from PABR twice per day, at approximately 0 and 12 UTC. 

Radiosondes provide high-resolution vertical profiles of temperature, humidity, wind speed, and 

wind direction throughout the troposphere. The data are made publically available by NOAA on 

the Earth System Research Laboratory (ESRL) Radiosonde Observation website 

(www.esrl.noaa.gov/raobs). Ramboll Environ downloaded and stored the radiosonde data from 

2009 to 2013 in FSL output format for use in WRF model dataset comparisons. The advantage of 

traditional radiosonde datasets are the length of the dataset (long periods at the same location) 

and the fact that they are direct measurements (as opposed to remote sensing). Disadvantages 

include the low frequency at only twice per day, and the fact that the lowest reported level comes 

from a different sensor than the rest of the profile and may occasionally lead to un-physical 

profiles. The measurement from the stationary sensor may be warmer than the lowest layer 

reported by the radiosonde, an unstable situation where the air would normally mix rapidly and 

the temperature gradient would be erased. Layers of temperature data showing super-adiabatic 

lapse rates are categorized as valid if the decrease in potential temperature in the layer does not 

exceed 1.0 K (NOAA-OFCM, 1997).

4.2.1 Point Barrow Soundings

Figures 10 through 14 show a selection of upper air profile plots taken from each year of the 

dataset for a sampling of several different atmospheric situations and WRF performance. The 

selection was made to show some of the worst-performing days, some of the best-performing 

days, and some in between. Figure 10 depicts vertical sounding profiles from the BOEM Arctic 

WRF dataset from April 21, 2009 on the left panel and July 6, 2009 on the right panel at Point 

Barrow. An elevated temperature inversion originates from around 350 meters above the surface 

and extends vertically to around 500 meters on the left panel. WRF overestimates the base and 

depth of the inversion. On the right panel, WRF estimates the depth of the surface inversion well, 

which extends to around 300 meters above the surface, but under-forecasts the surface 

temperature by a degree Celsius. 

WRF predicts the temperature and moisture well for September 5, 2010 in Figure 11. WRF 

follows the elevated temperature inversion from 200 meters above the surface to around 400 

meters. In the left panel for May 1, 2010, the model overestimates the surface temperature and 

underestimates the moisture in the upper levels, but follows the temperature closely through the 

lower-levels of the profile. 

On the left panel in Figure 12, WRF represents the strong subsidence inversion well for June 8, 

2011. On the right panel, for October 2, WRF under-predicts the surface temperature by three 

degrees Celsius, but represents the temperature and moisture well through the rest of the vertical 

profile.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

22

Figure 13 displays the sounding from September 24, 2012 on the left panel and from November 

12, 2012 on the right panel. The WRF model depicts the temperature inversion beginning too 

shallow on the left panel, but forecasts the height of the inversion accurately to around 500 

meters from the surface. On the right panel, WRF under-forecasts the height of the subsidence 

inversion by 500 meters.

On the left panel in Figure 14, WRF underestimates the surface temperature by 7 degrees Celsius 

for January 4, 2013 but does represent the very strong inversion that begins at 200 meters above 

the surface and continues to warm nearly 17 degrees Celsius in 200 meters. On the right panel 

for July 7 2013, WRF represents both temperature and moisture well throughout the vertical 

column. Overall, WRF performs well representing temperature and moisture in the vertical 

profiles of the atmosphere below 2,000 feet, including surface and subsidence-type inversions.

4.2.2 Other Upper-Air Datasets

Brashers et al. (2015), used two other sources of upper-air data to assess WRF performance, 

namely the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) cruise 

radiosonde data, and the Endeavor Island microwave thermal profiler data. 

The JAMSTEC cruises are carried out by a Japanese research group to study various subjects 

relating to the world’s oceans and climate. The cruises have been ongoing since 1987. The 

Research Vessel Mirai is a Japanese oceanographic research vessel that has been used for a 

series of JAMSTEC cruises to the Arctic Ocean as part of the JAMSTEC Arctic Ocean Climate 

System Research Project. Many separate legs from 1999 to 2013 were conducted that took the 

Mirai into the Arctic waters. Radiosonde measurements were collected on several of these legs, 

several of which overlap the WRF dataset periods. 

The Endeavor Island upper-air profiler dataset consists of hourly records of temperature in the 

lower layers of the atmosphere collected from May 15, 2010 to February 28, 2013. Endeavor 

Island is a small man-made island located at the Endicott development approximately 15 miles 

east of Prudhoe Bay. A set of meteorological equipment is maintained on the island, including a 

traditional meteorological tower (measuring wind speed and direction, temperature, and 

humidity) and a Kipp & Zonen microwave thermal profiler. 

Brashers et al. (2015) assessed a different, yet related, WRF dataset. The SST inputs used in that 

WRF run were found to have problems related to the Mackenzie River outflow, which caused 

poor WRF performance in the eastern part of the domain. That discovery led to a new WRF run, 

the subject of this report. 

Because the locations of the JAMSTEC cruises and Endeavor Island are far from the areas that 

suffered from poor characterization of the SST, the WRF performance compared to the 

JAMSTEC and profiler datasets is expected to be similar for both the WRF run covered in 

Brashers et al. (2015) and the WRF run covered in this report. The reader is referred to Brashers 

et al. (2015), Section 4.3.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

23

Figure 10. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point Barrow, AK (PABR) on April 

21, 2009 at 00 UTC and July 6, 2009 at 12 UTC
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

24

Figure 11. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point Barrow, AK (PABR) on May 

1, 2010 at 00 UTC, and September 5, 2010 at 00 UTC
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

25

Figure 12. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point Barrow, AK (PABR) on June 

8, 2011 at 12 UTC and October 2, 2011 at 12 UTC
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

26

Figure 13. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point Barrow, AK (PABR) on 

September 24, 2012 and November 12, 2012 at 00 UTC
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

27

Figure 14. Vertical profile soundings comparing BOEM Arctic WRF to upper-air data at Point Barrow, AK (PABR) on 

January 4, 2013 at 12 UTC and July 7, 2013 at 12 UTC
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

28

4.3 Qualitative Evaluation Using PRISM Precipitation

This section presents a qualitative comparison of WRF simulation estimated monthly 

precipitation with monthly “normal” PRISM precipitation estimates. Five-year average (2009-

2013) monthly precipitation plots were constructed from BOEM Arctic WRF output and 

compared to PRISM 30-year average monthly plots. 

The Parameter-elevation Regressions on Independent Slopes Model (PRISM) datasets are spatial 

maps of climate elements across the United States built by the Spatial Climate Analysis Service 

at Oregon State University (SCAS-OSU). Gridded maps of mean monthly and annual 

precipitation and temperature were built using meteorological station measurements and a set of 

statistical weighting procedures and corrections based on distance, elevation, topographic 

orientations and influences, and land-surface type (SCAS-OSU, 2001). The process and results 

have been extensively peer-reviewed and generally accepted by the climatological community as 

state-of-the-art representations. 

The high-resolution Alaska PRISM dataset contains 30-year average (1971-2000) monthly 

precipitation and temperature over the entire onshore areas of Alaska at 400 m resolution. The 

dataset was built using measurements from over 732 surface stations and 1637 upper-air grid 

points from the North American Regional Reanalysis. The data point locations are set in NAD83 

Albers projection. 

Ramboll Environ obtained the dataset from The Climate Source, Inc., which is the commercial 

distribution company formed to distribute SCAS-OSU products, and stored it on its servers for 

the statistical evaluation.

To facilitate the comparison of PRISM to WRF, the PRISM monthly precipitation datasets were 

transformed from their original NAD83 Albers projection to the given BOEM Arctic WRF 

projection for all three domains. Given the different time spans between the PRISM and WRF 

monthly precipitation averages, comparisons of the PRISM plots to the following BOEM Arctic 

WRF plots should be limited to concepts like “is the WRF precipitation consistent with the 

“normal” precipitation for that month?” It is also noteworthy that the PRISM dataset only covers 

mainland Alaska, while the BOEM Arctic WRF dataset encompasses the entire domain. 

Figure 15 through Figure 19 display spatially the five-year WRF precipitation datasets and the 

PRISM data for the months of January through May, respectively, in the 4 km domain d03. It is 

clear that all of these months tend to be quite dry, and both the WRF and PRISM average 

precipitation plots reflect this. Given the two sets of plots, it is evident that WRF tended to 

simulate “pockets” of higher precipitation totals than anywhere in the corresponding PRISM 

datasets. However, WRF also appeared to slightly under-predict winter-spring precipitation totals 

on a larger scale throughout much of the Brooks Mountain Range which forms the southern 

border of the North Slope. However, both of these discrepancies could be due in large part to the 

difference in timescale. The PRISM data was averaged over a much longer span of time, which 

could have the effect of smoothing out the instances of higher precipitation, while accounting for 

some outlier years that include some rainfall in the typically dry areas. 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

29

Figure 15. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for January in the 4 km 

domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

30

Figure 16. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for February in the 4 km 

domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

31

Figure 17. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for March in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

32

Figure 18. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for April in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

33

Figure 19. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for May in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

34

Figures 20 through 22 represent the 4km domain average monthly precipitation plots for WRF 

forecasts and PRISM observations for June through August, respectively. It is apparent that these 

summer months are the rainiest in northern Alaska, but also that the majority of the rain is 

limited to the Brooks Mountain Range on the southern edge of the North Slope. For these three 

months, the WRF five-year average precipitation plots appear very similar to the PRISM 30-year 

data. One difference is that the WRF simulations tend to slightly under-predict precipitation in 

the areas with the highest rainfall.

Figures 23 through 26 show WRF and PRISM average monthly precipitation plots for September 

through December, respectively, in the 4km domain. The fall months show substantial 

discrepency between WRF forecast five-year and PRISM observation 30-year average monthly 

precipitation plots. Particularly in September and October, WRF drastically under-predicts 

precipitation along the Brooks Mountain Range, while appearing to over-allocate precipitation 

along the north coast of Alaska. In November and December, observed precipitation totals begin 

to decrease toward the low winter levels, and these model discepencies become less pronounced.

WRF performed well overall, simulating precipitation over onshore portions of the 4 km domain, 

but tended to under-forecast enhanced rainfall areas over more complex conditons in and around 

the Brooks Mountain Range.

4.4 Qualitative Evaluation Using Satellite Cloud Cover Observations

The dataset selected for this study is the MISR Level 3 Monthly FIRSTLOOK Component cloud 

product derived from the Multi-angle Imaging Spectro-Radiometer (MISR) instrument 

measurements. The MISR is an instrument deployed aboard the NASA Terra satellite. The 

product selected is the derivation of monthly averaged spatial cloud cover fraction on a 0.5° by 

0.5° horizontal resolution. Cloud cover fraction is computed for a set of vertical levels at 500 m 

resolution. The dataset from 2005-2013 was obtained from the online NASA Atmospheric 

Science Data Center, located at (https://eosweb.larc.nasa.gov/project/misr/level3/overview). 

MISR retrievals of cloud cover fraction (CCF) were compared qualitatively to estimates calculated 

from WRF. WRF calculates fractional cloud cover at each level of each grid point (output as 

variable “CLDFRA”). The maximum cloud fraction over the vertical column was identified at each 

grid point and assumed to represent the total cloud cover fraction (cloud cover fraction of an area 

as seen from the surface). This method introduces error because varying degrees of cloudiness at 

different heights could result in a total cloud cover fraction (cloud cover fraction as seen from the

ground) that is greater than, but not less than, the fraction at any given level. Therefore, this 

approach could be expected to produce a negative bias in total cloud cover fraction. 

The mean monthly CCF at each grid point was calculated, resulting in 12 plots per year for each 

domain of the 2009-2013 WRF simulations. The monthly MISR satellite CCF is calculated for 

45 vertical levels at 500 m resolution. The maximum CCF over the vertical column was 

identified at each grid point and assumed to represent the total CCF, as was done to estimate 

WRF cloud fractions. Again, it is expected that such estimates would have a negative bias 

because they do not account for the additive effect of cloud cover at different levels. However, 

since the purpose of the comparison is to evaluate WRF performance and both datasets may 

result in negative bias, the bias is likely inconsequential unless one dataset introduces 

substantially more bias than the other. 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

35

Figure 20. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for June in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

36

Figure 21. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for July in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

37

Figure 22. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for August in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

38

Figure 23. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for September in the 4km 

domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

39

Figure 24. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for October in the 4km domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

40

Figure 25. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for November in the 4km 

domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

41

Figure 26. 5-year (2009-2013) WRF precipitation average (top) and 30-year PRISM 

precipitation average (bottom) monthly precipitation totals for December in the 4km 

domain
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

42

The monthly MISR CCF was re-projected to match the projection and horizontal resolution of 

the WRF domains. Plots of monthly MISR CCF were produced for the 36km (domain 1), 12km

(domain 2), and 4km (domain 3) domains for direct comparison to BOEM Arctic WRF results. 

Also, note that MISR estimates cloud fraction using visible wavelengths. Cloud fraction cannot 

be accurately estimated for a portion of the domain during winter months due to the lack of 

sunlight in the Arctic. MISR also uses a snow/ice mask in regions where ice and snow cover the 

surface. Based on the qualitative examination summarized in this section, it appears that MISR 

CCF estimates may not be reliable during winter months. The influence of surface ice and snow 

and lack of sunlight in the Arctic in winter months introduces error, appearing to result in a 

negative cloud cover bias.

The qualitative comparison of WRF and MISR CCFs was focused on domain 2 of the WRF 

simulations. Domain 1 WRF resolution is the most similar to the resolution of the MISR dataset, 

but domain 2 provided a more focused examination of the area of interest. 

An annual pattern in the MISR data was evident comparing the five years of data. MISR CCF 

distribution was noisy and low during the winter months (November – April). It appears that 

MISR data is likely unreliable as an accurate source of CCF during these months. During these 

periods when average MISR CCF is relatively low (0.3 -0.4) and noisy, WRF average CCF is 

relatively high (0.8 – 0.9) and forms distinct patterns including heavy cloud cover over the North 

Slope and Arctic Ocean. An example of these differences is shown in Figure 27 for January 

2011. 

Figure 27. MISR (left) and WRF (right) CCF, January 2011

In March, the MISR data is punctuated by a region of higher CCF along the central inland north 

slope (with CCF of 0.5-0.6). The regional CCF peak is evident in most March WRF run cases 

also. An example of this feature is shown in Figure 28 for March of 2011. By March, the WRF 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

43

domain is less CCF saturated than the earlier months and distinct patterns are visible. All years 

except for 2012, inland cloud cover becomes less pervasive, dampening from an average of 0.8 -

0.9 in the earlier months to an average in the range of 0.6 – 0.7. CCF over the Arctic Ocean 

dampens also, with the saturated CCF values greater than 0.9 covering less area. 

Figure 28. MISR (left) and WRF (right) CCF, March 2011

By April, the patterns of cloud cover distribution in the MISR data become more defined and 

less noisy. A common feature is the concentration of higher CCF in the range of 0.5 to 0.8 over 

the Chukchi Sea and along the coast of the Beaufort Sea. This pattern is reflected in the WRF 

results but with higher magnitude CCF in the range of 0.8 – 0.95. This is best demonstrated in 

the April 2012 results shown in Figure 29. 

Figure 29. MISR (left) and WRF (right) CCF, April 2012
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

44

By May, the MISR data appear to be much less affected by ice/snow/low-light return problems 

given that the CCF distribution is distinct and less noisy across the domain. All five years of 

April MISR, except 2009, portray a similar pattern characterized by a mass of higher CCF (in the 

range of 0.7 – 0.9) in the Chukchi Sea and Arctic Ocean in the northwest corner of the domain. 

In 2009, the mass of higher CCF concentrates more in the Beaufort Sea and the Arctic Ocean in 

the northeast corner of the domain. This pattern is also evident in the WRF data, characterized by 

a concentration of higher CCF values (0.85 – 1.0) over the Chukchi Sea and Arctic waters in the 

northwest of the domain. The WRF and MISR features most closely match in the May 2011 case 

shown in Figure 30.

For the June cases, the WRF and MISR CCF patterns appear to not agree as well as the May 

cases. The MISR cases are punctuated by bands of high CCF (in the range of 0.9 – 1.0) over the 

Bering Sea all five years. A narrow band of high CCF is also evident along the Beaufort Sea 

coastline all five years. MISR CCF over the Arctic Ocean is widespread, but lower in magnitude 

(0.5 – 0.7 on average) than the WRF results (0.8 – 0.9 on average). Higher values of CCF are 

evident over the inland areas of Alaska in both the MISR and WRF datasets. Bands of higher 

CCF are evident along the Brooks Range of mountains in both MISR and WRF datasets. The 

inland and marine CCF features discussed are evident in the June 2009 cases shown in Figure 31. 

Figure 30. MISR (left) and WRF (right) CCF, May 2011
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

45

Figure 31. MISR (left) and WRF (right) CCF, June 2009

For July, the MISR cases have a similar pattern as seen in June. The Bering Sea is dominated by 

high CCF cover in the range of 0.8 to 1.0. The shape and magnitude of the Bering Sea CCF 

pattern is captured well in the WRF simulations, as seen in Figure 32 for the July 2009 case. 

However, the WRF CCF over the Arctic Ocean is more widespread and higher in magnitude than 

the MISR data. It is unlikely that MISR data suffers from lack of visible light in July, but the 

data may still be biased due to ice and snow cover over the Arctic Ocean. It is unknown to what 

degree the remaining ice cover has on the MISR data. It is possible that the MISR data have low 

bias and WRF highly over-predicts CCF. However, given the increase in MISR CCF in August 

over all five years, we assume that the July MISR data is biased low over the Arctic. 

In August there is a markedly large increase in MISR CCF over the Beaufort and Chukchi Seas. 

MISR continues to measure widespread high CCF values over the Bering Sea and Strait as well. 

WRF under-predicts CCF in the Bering Sea/Strait compared to MISR with average values in the 

range of 0.8 – 0.9 compared to MISR values in the 0.9 – 1.0 range. WRF agrees well with 

patterns and magnitudes of CCF in the MISR dataset over the Chukchi and Beaufort Seas. 

However, WRF predicts widespread CCF near 1.0 across the Arctic Ocean, while MISR predicts 

values in the 0.7 - 0.8 range possibly biased low due to partial ice cover. These patterns are all 

evident in the August 2013 cases, shown in Figure 33. 

WRF and MISR CCF distributions are the most comparable in the September cases compared to 

all other months. This may be due to the fact that MISR negative bias may be minimized since 

ice cover extent is usually at its minimum in September. For all five years, the extent and 

magnitude of WRF CCF matches well to the MISR CCF, particularly over the marine areas. 

WRF tends to predict higher CCF along the north-central portion of the North Slope. September 

2009 and 2012 cases are shown in Figure 34 and Figure 35, respectively, to illustrate these 

features.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

46

Figure 32. MISR (left) and WRF (right) CCF, July 2009

Figure 33. MISR (left) and WRF (right) CCF, August 2013
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

47

Figure 34. MISR (left) and WRF (right) CCF, September 2009

Figure 35. MISR (left) and WRF (right) CCF, September 2012

For the October cases, magnitude and extent of CCF from both MISR and WRF match 

qualitatively well also. The October MISR data are noisier than the September data and there is 

evidence that sea ice expanse and limited light may result in bias, particularly over the northern 

and north-western portions of the domain. Again, WRF tends to be cloudier over land overall. 

The October 2012 cases are shown in Figure 36 to demonstrate these features.

For November and December it is evident that ice cover and lack of visible light limits the ability 

of MISR to measure CCF. The MISR data are noisy and average CCF is unnaturally low. WRF 

CCF coverage is extensive during these winter months. The December 2011 case is shown in

Figure 37.
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

48

In conclusion, MISR data appear to be unreliable as a source of “true” CCF in the Arctic for a 

considerable portion of the year due to negative bias resulting from ice-cover interference and 

insufficient sunlight. MISR data are therefore likely the most reliable in late summer. 

Qualitative comparisons of WRF monthly average CCF to MISR CCF in September and October 

revealed similar spatial distributions and CCF magnitude over much of the domain. WRF CCF 

over land appeared to be biased high by 5-15% on average. 

Figure 36. MISR (left) and WRF (right) CCF, October 2012

Figure 37. MISR (left) and WRF (right) CCF, December 2011
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

49

5.0 SUMMARY AND CONCLUSIONS

The BOEM Arctic January 2009 through December 2013 WRF meteorological model simulation 

reproduced the observed surface meteorological variables reasonably well. The wind direction 

performance is improved over previous meteorological datasets for the same region. Vertical 

profiles also performed well, accurately reproducing observed conditions of coastal interactions 

in the planetary boundary layer. The 5-year average precipitation amounts for each month were 

consistent with the PRISM 30-year normal. Where and when valid satellite cloud retrievals were 

available, WRF-predicted cloud amount compared reasonably well. 

Based on our experience, the BOEM Arctic WRF modeling’s performance provides a sound 

basis for developing meteorological inputs for Arctic Air Quality Study modeling.

6.0 REFERENCES

Brashers et al., 2015. Meteorological Dataset Evaluation – Final Task Report, BOEM Arctic Air 

Quality Impact Assessment Modeling Study. By Ramboll Environ, Lynnwood, WA.

Bromwich, D., Hines, K., and Bai, L.-S. (2009). Development and Testing of Polar WRF, Part 2: 

Arctic Ocean. J. Geophys. Res., D08122.

Chen, F.; Janjic, Z.; Mitchell, K. (1997). Impact of atmospheric surface layer parameterization in 

the new land-surface scheme of the NCEP Mesoscale Eta numerical model. Bound.-Layer 

Meteor. 391-421.

Emery, C.A., E. Tai, and G. Yarwood. 2001. Enhanced Meteorological Modeling and 

Performance Evaluation for Two Texas Ozone Episodes. Prepared for the Texas Natural 

Resource Conservation Commission (now TCEQ), by ENVIRON International Corp, Novato, 

CA. Available at: 

http://www.tceq.texas.gov/assets/public/implementation/air/am/contracts/reports/mm/Enhanced

MetModelingAndPerformanceEvaluation.pdf 

Kemball-Cook, S., Y. Jia, C. Emery, and R. Morris, 2005. Alaska MM5 Modeling for the 2002 

Annual Period to Support Visibility Modeling. Prepared for the Western Regional Air 

Partnership, by ENVIRON International Corp., Novato, CA. Available at: 

http://pah.cert.ucr.edu/aqm/308/docs/alaska/Alaska_MM5_DraftReport_Sept05.pdf 

Hines, K., and Bromwich, D. (2008). Development and testing of Polar WRF, Part 1: Greenland 

ice sheet meteorology. Mon. Wea. Rev., 1971-1989.

McNally, D. E., 2009. 12 km MM5 Performance Goals. Presentation to the Ad-Hoc Meteorology 

Group. June 25, 2009. Available at: http://www.epa.gov/scram001/adhoc/mcnally2009.pdf

NCAR, 2015. National Center for Atmospheric Research. Mesoscale & Microscale 

Meteorological Laboratory. https://www.mmm.ucar.edu/

NOAA (2015). National Weather Service. National Centers for Environmental Prediction. 
Arctic Air Quality Modeling Study- Meteorological Model Performance Evaluation: 

2009-2013 BOEM Arctic WRF Dataset

50

http://www.ncep.noaa.gov/

NOAA-NCDC. (2014). NOAA NCDC Integrated Surface Database. Retrieved from 

www.ncdc.noaa.gov/isd

NOAA-NCDC. (2015). Brief Review of Problems and Issues with Integrated Surface Data 

(ISD). Retrieved from http://www1.ncdc.noaa.gov/pub/data/inventories/ish-qc.pdf

NOAA-OFCM. (1997) Federal Meteorological Handbook No. 3 Rawinsonde and Pibal 

Observations. Office of the Federal Coordinator for Meteorology. Available at 

http://www.ofcm.gov/fmh3/pdf/04-chap4.pdf

NOAA-NODC. (2014). NOAA NODC Ocean Archive System. Retrieved from 

www.nodc.noaa.gov/cgi-bin/OAS/prd/accession/93399

SCAS-OSU, 2001. Climate Mapping with PRISM. Oregon State University (OSU)