NREL is a national laboratory of the U.S. Department of Energy, Office of Energy 

Efficiency & Renewable Energy, operated by the Alliance for Sustainable Energy, LLC.

Contract No. DE-AC36-08GO28308

Case Studies Comparing 

System Advisor Model (SAM) 

Results to Real Performance 

Data

Preprint

N. Blair and A. Dobos

National Renewable Energy Laboratory

N. Sather

Massachusetts Institute of Technology

Presented at the 2012 World Renewable Energy Forum

Denver, Colorado

May 13-17, 2012

Conference Paper

NREL/CP-6A20-54676

June 2012
NOTICE

The submitted manuscript has been offered by an employee of the Alliance for Sustainable Energy, LLC 

(Alliance), a contractor of the US Government under Contract No. DE-AC36-08GO28308. Accordingly, the US 

Government and Alliance retain a nonexclusive royalty-free license to publish or reproduce the published form of 

this contribution, or allow others to do so, for US Government purposes.

This report was prepared as an account of work sponsored by an agency of the United States government. 

Neither the United States government nor any agency thereof, nor any of their employees, makes any warranty, 

express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of 

any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately 

owned rights. Reference herein to any specific commercial product, process, or service by trade name, 

trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, 

or favoring by the United States government or any agency thereof. The views and opinions of authors 

expressed herein do not necessarily state or reflect those of the United States government or any agency thereof.

Available electronically at http://www.osti.gov/bridge

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

phone: 865.576.8401

fax: 865.576.5728

email: mailto:reports@adonis.osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

phone: 800.553.6847

fax: 703.605.6900

email: orders@ntis.fedworld.gov

online ordering: http://www.ntis.gov/help/ordermethods.aspx

Cover Photos: (left to right) PIX 16416, PIX 17423, PIX 16560, PIX 17613, PIX 17436, PIX 17721

Printed on paper containing at least 50% wastepaper, including 10% post consumer waste.
1

WREF 2012: CASE STUDIES COMPARING SYSTEM ADVISOR MODEL (SAM) RESULTS 

TO REAL PERFORMANCE DATA

NREL has completed a series of detailed case studies 

comparing the simulations of the System Advisor Model 

(SAM) and measured performance data or published 

performance expectations. These case studies compare PV 

measured performance data with simulated performance 

data using appropriate weather data. The measured data 

sets were primarily taken from NREL onsite PV systems 

and weather monitoring stations.

ABSTRACT

The results of this case study activity show a good match

(delta of less than 6% for any month) between measured 

and simulated data after adjustments as described in the 

paper. These adjustments include removing bad or missing

data periods from the measured data, removing periods 

with significant snow cover which is difficult to model 

and, in some cases, adjusting the default annual derate 

values. As such, the case studies point out several systemic 

issues with such comparisons. For example, the default 

derate values in SAM are not the best match to the 

performance data generally. The default assumptions 

around snow cover and soiling performance impacts also 

appear to be inappropriate for these data sets.

The case studies highlight modeling techniques in SAM. 

These include modeling systems with components not 

found in current SAM libraries, finding appropriate 

satellite weather data and the use of building load data in 

conjunction with PV performance and complex utility 

rates.

In addition to providing an overview of the individual case 

study reports and the issues they raise, we will also discuss 

the process of determining appropriate weather data and 

financial inputs to enable detailed comparison and 

validation efforts using SAM.

1.

During 2010, the System Advisor Model (SAM) team 

collected data for four PV systems. The goals of this effort

included real-world validation of an entire system (and not 

sub-component validation which is often done) and to 

build examples for SAM users to review in building their 

own systems. We attempted to get a spread of systems 

across a range of sizes and markets. Comparisons were 

performed between the SAM model output and measured 

performance data, reported cost data or with reported 

annual performance values. Four of the system 

comparisons were released with the SAM model release 

version 12.2.2011. The case studies and their associated 

SAM project file are distributed with the SAM download.

INTRODUCTION

1.1

For the four released PV case study systems, some hourly 

performance data were available for all of them and over a 

year of data for all but one of them. The four sites include:

Systems Examined

1.1.1

The James Forrestal Building is the U.S. Department of 

Energy’s (DOE) headquarters in Washington, D.C. The 

205 kW rooftop PV array was installed in 2008 with the 

goal of producing up to 8% of the building’s peak energy 

needs in order to fulfill the Transformational Energy 

Action Management (TEAM) Initiative. The market was 

commercial building.

James Forrestal Building (Forrestal)

Nate Blair

Aron Dobos

National Renewable Energy Laboratory

1617 Cole Boulevard, Golden, CO 80401

nate.blair@nrel.gov

aron.dobos@nrel.gov

Nicolas Sather

Massachusetts Institute of Technology
2

1.1.2

The RSF building is part of NREL’s South Table Mountain 

campus located in Golden, CO. The RSF’s rooftop PV 

array has a nameplate capacity of 449 kW, contributing to 

the building’s net-zero energy standard. The market is 

Commercial PPA (for third party ownership).

NREL Research Support Facility (RSF) Building

1.1.3

The S&TF Building is part of NREL’s South Table 

Mountain campus located in Golden, CO. The S&TF’s 

rooftop PV array has a nameplate capacity of 94.5 kW. The 

market is Commercial PPA (for third party ownership).

NREL Science and Technology Facility (S&TF) 

1.1.4

The Zero-Energy Home in Oklahoma City, Oklahoma 

features a roof-mounted 5.3 kW PV array which serves as 

the energy source for the building. The residential PV 

system began generating energy in November 2005. The 

market is residential.

Oklahoma City Residence (OKCity)

1.2

One of the more difficult tasks in comparing system output 

to simulations is getting the system description data, cost 

data, the "appropriate" local weather data and the system 

output data. To be useful to SAM users, the system 

description data at least had to be public information. In 

the cases above, the performance data was also sharable 

but some of the weather data was only available within 

NREL at this time (becoming public within the next year). 

Note that the use of the OpenPV database to determine 

approximate system cost was necessary, as no public cost 

data were available for any of these cases. The OpenPV 

database allows the user to select the year of installation 

and filter by system size (and location if there are enough 

systems at that granularity to provide a good average).

Data Acquisition

"Appropriate" local weather data is ideally data measured 

at the site for the same time period as the data 

measurements. In many cases this is not possible. Using a 

TMY2 file and comparing to an actual year of measured 

data would be inappropriate. The use of NREL PV and 

weather data systems was invaluable in making it possible 

to gather high-quality, time-syncronous measured data. 

Without this local resource, we would have created 

appropriate datasets from the “Perez Satellite Data” set of 

10k gridded data. This data is publically available via the 

NREL Solar Prospector (http://maps.nrel.gov/prospector) 

from 1998-2005. It will soon be available for more current 

years.

Within each individual case study document, a table has 

also been built to capture which SAM inputs have been 

changed from the default value to a value more appropriate 

for that case.

TABLE 1: 

Site

DESCRIPTION OF THE PRIMARY DATA 

AVAILABLE FOR EACH OF THE CASE STUDIES.

Performance 

Data Source Cost Data Source Weather Data Source

Forrestal

DOE Solar 

Program 

(Dec. 2009 to 

June 2010)

OpenPV.NREL.Gov

1

DOE Solar 

Program 

(measured 

at the 

system)

NREL 

RSF

SunEdison 

Client 

Connect 

Portal 2 OpenPV.NREL.Gov (June 

2010 - May 

2011)

NREL’s 

SRRL site 

data3

NREL 

S&TF

SunEdison 

Client 

Connect 

Portal (Jan. 

2010 – Dec. 

2010.)

OpenPV.NREL.Gov

NREL’s

SRRL site 

data

OKCity

Measured 

Data used in 

prior NREL 

report4

OpenPV.NREL.Gov

Proprietary 

Perez 

Satellite 

data for 

2006

2.

2.1

COMPARISONS WITH MEASURED DATA

In all four cases, the system measured performance data 

did not initially match the simulated data. For example, as 

you can see in Figure 1, there are several months that 

didn’t match output very well (especially January, 

February and March in this case).

Missing Data
3

Fig. 1: Initial Comparison between SAM and Measured 

Data for the NREL S&TF Building

In this case and the other cases, several factors were found 

to be responsible for these disparities. These factors were 

generally consistent across the four case studies. The first 

issue was missing data within the measured performance 

data. This happened in OK City, NREL RSF, and NREL 

S&TF. This was noticeable because the weather file used 

indicated incident radiation but no output was detected in 

the measured data. If both the weather data and 

performance data indicated no radiation and no output, the 

SAM model would have also predicted no system output.

2.2

Another major discrepancy was due to snowfall causing a 

reduction in measured output. This was a major issue in 

comparing to simulated data for all systems. Figure 2 

shows the relationship between snowfall and system output 

for the Forrestal system. The snowfall data was taken from 

the online Farmer's Almanac

Snowfall

5 and Forrestal performance 

data was obtained from DOE. This significantly skews the 

results for each system during months with significant 

snowfall. While some weather data sets include snowfall, 

it’s currently not possible to accurately determine how long 

that snow remains on the panels or how much of the array 

remains covered by snow. While an annual derate 

adjustment can calibrate the annual simulated output, this 

doesn't impact hourly results significantly. Another factor

to remember when using the current version of SAM is that 

if a climate file contains snow data, the ground albedo for 

snowy days increases and actually enhances the simulated 

system performance, when in reality the output should be 

reduced due to snow cover, in most cases. This issue is 

being investigated further.

Fig. 2: Snow depth (red) and DC energy output (blue) for 

each day in February 2010, clearly showing the two 

Mid-Atlantic blizzards (6th and 10th) and the impact 

on solar energy generation.

2.3

The final factor that seems to consistently lead to 

underprediction of performance values by SAM is the 

overall derate value. The SAM default derates (based on 

the derates first presented in PVWatts

Derate

6

), are conservative 

when looking at this very small sample set of relatively 

new systems. As you can see in Figure 3, the overall derate 

(composed of a variety of derate factors such as nameplate 

derate, soiling, wiring losses, mismatch, etc.) should be 

roughly 87% which is a higher than the default values for 

this case. The Forrestal building case study didn’t examine 

modifying the derate and the Oklahoma City case study 

didn’t have a consistent under-prediction (but rather a 

seasonal pattern) so no consistent conclusion should be 

drawn. However, the default derate values for SAM 

should reflect recent technical improvements and improved 

performance precision by the industry and work is being 

initiated in this area. With various subsidy programs basing 

their calculations on values that include these derates, 

accurately updating them is important to do. However, 

users should consider adjusting the derate values to 

accurately match measured data if available.
4

Fig. 3: For the NREL RSF system, a total derate factor in 

SAM of about 87% would minimize the difference 

between SAM's calculated output values and the 

measured values.

Related to the issue of appropriate general derates, the 

Forrestal system raised the issue of the episodic impact of 

heavy soiling on the system. The system on the roof of the 

Forrestal building is flat. The capacity factor for the system 

increases dramatically after each rain event. An example of 

this is shown in Figure 4. This exemplifies the fact that 

significant soiling can occur both on flat systems and 

systems in urban areas (or areas with significant dust). 

SAM has included annual soiling derates (and now 

includes monthly soiling derate factors) to allow users to 

capture the impact of soiling on radiation getting to the 

module, but getting soiling data (or anticipated soiling 

data) is difficult. Research is needed to better capture and 

anticipate soiling impacts. 

Fig. 4: For the Forrestal building, a strong correlation 

between DC capacity factor (green) and total rainfall 

(purple) during March 2010 is shown

3.

3.1

RESULTS

With the corrections to measured data for snowfall, derate 

and missing data, good agreement was found between 

SAM and the measured data for each system. Figure 5

shows this for the S&TF system. After accounting for days 

with snow cover or system malfunction, we calibrated the 

derate value and were able to get within 0.3% of the 

measured annual output and within 3.7% of the measured 

value for every month except November and December 

which had an output error of less than 5%.

S&TF System

Fig. 5: Final comparison graph of the measured data vs. 

the SAM estimates after removing flawed data and 

calibrating the derate factor for the NREL S&TF 

system.

3.2

After adjusting for snow cover in the winter months, the 

SAM model showed good agreement with the measured 

data at the monthly level. Five of the seven months studied 

were within 2.5% of the measured data while the other two 

(December 2009 and January 2010) were within 5.6% of 

the measured DC output values.

Forrestal Building

3.3

After calibrating the derate value to fit the system, SAM 

results were within 1.9% of the measured system output for 

each of the 8 months that did not include flawed data or 

array performance issues.

Oklahoma City Zero Energy Building

3.4

After accounting for days that had snow cover or system 

maintenance and calibrating the derate value, SAM results 

were within 1.5% of the measured energy output for each 

of the 6 months that were studied.

NREL RSF System
5

TABLE 2: 

Metric

STANDARD SAM METRICS TABLE

SAM value

Net Annual Energy 8,758 kWh

LCOE Nominal 31.06 ¢/kWh

LCOE Real 24.73 ¢/kWh

First Year Revenue without System $ -1,180.39

First Year Revenue with System $ -683.00

First Year Net Revenue $ 497.39

After-tax NPV $ -17,313.95

Payback Period 1e+099 years

DC-to-AC Capacity Factor 18.8 %

First year kWhac/kWdc 1,646

System Performance Factor 0.81

Total Land Area 0.02 acres

Additionally, for each of the case studies, the standard 

metrics table from SAM was presented (an example from 

the Oklahoma City Building is shown in Table 2). This 

includes performance values as well as economic values. 

Because the actual cost data used was representative (and 

the actual system costs are proprietary), we can’t 

effectively compare the simulated LCOE (Levelized Cost 

of Energy) values with actual or reported values.

4.

NREL has completed four case studies related to the 

System Advisor Model (SAM). Specifically, we modeled 

four PV systems, making generally minor changes to the 

SAM default values to model the systems (we typically 

had minimal information about the system other than 

component names). These case studies are included in the 

12.2.2011 SAM release. As illustrative case studies, they 

can guide the SAM user in setting up their own systems. 

SUMMARY

In the case of the Oklahoma City system, building load 

data was also available so we were able to demonstrate to 

the SAM user how to include load data and link this to a 

time of use rate.

After adjusting the measured system output for snow days 

and missing data, reasonable agreement was achieved. 

Calibrating the overall SAM derate value to minimize the 

differences further, improved agreement to within 1% for 

all four cases at an annual level.

These case studies have revealed several issues:

(1) It remains difficult to obtain high quality 

measured PV system output and synchronous radiation and 

meteorological data.

(2) The treatment of snowfall and snow cover has a 

major impact on most large systems and isn’t generally 

captured in the models accurately. In fact, snow cover 

raises the ground albedo in SAM simulations, thereby 

increasing system output. 

(3) Evidence suggests that the default values used for 

derates in SAM (and PVWatts from which the data comes) 

should be updated (based on several of these systems 

where an annual derate increase of several percent 

improved the agreement between measured and simulated 

data). With improvements to module information and 

changes in technology, current default values are dated. 

Additionally, specific situations can lead to very large 

soiling derates.

5.

The authors would like to thank SunEdison for 

participating by sharing the data from the systems on 

NREL’s campus as well as the colleagues who worked on 

the Oklahoma City Zero Energy home. We would also like 

to thank our sponsors at the Department of Energy Sunshot 

Program office who sponsor the SAM model and this work 

in particular.

ACKNOWLEDGMENTS

This work was supported by the U.S. Department of 

Energy under Contract No. DE-AC36-08-GO28308 with 

the National Renewable Energy Laboratory.

 

1 PV system cost data available at: http://openpv.nrel.gov/

and then filtered by installation year and system size for 

appropriate but nationally averaged values.

2 Because the system is maintained and owned by 

SunEdison, measured performance data was acquired from 

SunEdison’s Client Connect portal 

(https://my.sunedison.com/). A password is required to 

gain access to the data, which we obtained because NREL 

is the site owner.
6

 

3 This study used climate data collected at NREL’s Solar 

Radiation Research Laboratory (SRRL) located at the 

South Table Mountain site.

4 Barker, G. “Report on PV System Performance.” NREL 

Subcontract # LAX-1-30480-02. 24 April 2006.

5 Historical snowfall data from the Farmer’s almanac: 

http://www.farmersalmanac.com/weather-history/