NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

Indoor and Outdoor 

Spectroradiometer 

Intercomparison for Spectral 

Irradiance Measurement

A. Habte,1 A. Andreas,1 L. Ottoson,1 

C. Gueymard,2 G. Fedor,3 S. Fowler,3 

J. Peterson,4 E. Naranen,5 T. Kobashi,6 

A. Akiyama,6 and S. Takagi6 

1 National Renewable Energy Laboratory

2 Solar Consulting Services

3

 Q-Lab Corporation

4 University of Oregon

5 Atlas Material Testing Technology, LLC

6 EKO Instruments, Inc.

Technical Report

NREL/TP-5D00-61476

May 2014 
NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

National Renewable Energy Laboratory

15013 Denver West Parkway

Golden, CO 80401

303-275-3000 • www.nrel.gov

Indoor and Outdoor 

Spectroradiometer 

Intercomparison for Spectral 

Irradiance Measurement

A. Habte,1 A. Andreas,1 L. Ottoson,1 

C. Gueymard,2 G. Fedor,3 S. Fowler,3 

J. Peterson,4 E. Naranen,5 T. Kobashi,6 

A. Akiyama,6 and S. Takagi6 

1 National Renewable Energy Laboratory

2 Solar Consulting Services

3

 Q-Lab Corporation

4 University of Oregon

5 Atlas Material Testing Technology, LLC

6 EKO Instruments, Inc.

Prepared under Task No. SS13.3513

Technical Report

NREL/TP-5D00-61476

May 2014 
NOTICE

This report was prepared as an account of work sponsored by an agency of the United States government. 

Neither the United States government nor any agency thereof, nor any of their employees, makes any warranty, 

express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of 

any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately 

owned rights. Reference herein to any specific commercial product, process, or service by trade name, 

trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, 

or favoring by the United States government or any agency thereof. The views and opinions of authors 

expressed herein do not necessarily state or reflect those of the United States government or any agency thereof.

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Available electronically at http://www.osti.gov/scitech

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

phone: 865.576.8401

fax: 865.576.5728

email: mailto:reports@adonis.osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

phone: 800.553.6847

fax: 703.605.6900

email: orders@ntis.fedworld.gov

online ordering: http://www.ntis.gov/help/ordermethods.aspx

Cover Photos: (left to right) photo by Pat Corkery, NREL 16416, photo from SunEdison, NREL 17423, photo by Pat Corkery, NREL 

16560, photo by Dennis Schroeder, NREL 17613, photo by Dean Armstrong, NREL 17436, photo by Pat Corkery, NREL 17721.

Printed on paper containing at least 50% wastepaper, including 10% post consumer waste. 
iii

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Acknowledgments

The spectral irradiance intercomparison was a collective effort made by participating laboratories 

EKO instruments, Inc.; Q-Lab Corporation; ATLAS Material Testing Technology, LLC; the 

University of Oregon’s Department of Physics; and Solar Consulting Services. We would like to 

thank Daryl Myers, as well as Peter Gotseff and Manajit Sengupta of the National Renewable 

Energy Laboratory for their critical reviews and invaluable suggestions. 
iv

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

List of Acronyms

ASTM American Society for Testing and Materials

IEC International Electrotechnical Commission

ISO International Organization for Standardization

MBE mean bias error

NIST National Institute of Standards and Technology 

NREL National Renewable Energy Laboratory 

RMSE root mean square error

SMARTS Simple Model of the Atmospheric Radiative 

Transfer of Sunshine 

SRRL Solar Radiation Research Laboratory 
v 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Executive Summary

A global spectral irradiance intercomparison using spectroradiometers was organized by the 

National Renewable Energy Laboratory’s (NREL’s) Solar Radiation Research Laboratory. The 

intercomparison was performed both indoors and outdoors on September 17, 2013. Five 

laboratories participated in the intercomparison using 10 spectroradiometers. A coordinated 

measurement setup and a common platform were employed to compare spectral irradiances 

under both indoor and outdoor conditions. The intercomparison was aimed at understanding the 

performance of the different spectroradiometers and sharing knowledge in making spectral 

irradiance measurements. At NREL’s Optical Metrology Laboratory, the intercomparison is part 

of an internal performance-based quality-control check to monitor the legitimacy of a 

measurement and calibration undertaken by a laboratory to demonstrate compliance with 

International Standards Organization/International Electrotechnical Commission (ISO/IEC) 

17025 accreditation requirements. 

The indoor performance comparison showed that all of the participating spectroradiometers had 

satisfactory statistical results (±1) compared to the NREL reference instrument. However, each 

laboratory’s instruments behaves differently with respect to the statistical limit, and such 

differences could be related to various reasons—for example, differing calibration setups from 

one laboratory to another, differing environmental conditions inside laboratories, whether a 

primary or secondary spectral irradiance calibration lamp was used for the calibration, 

instrument age, and the amount of time since the last calibration. 

The outdoor intercomparison showed up to ±10% deviation relative to the average spectral 

irradiance measured by the participating spectroradiometers. Differing scan rates, sizes of the 

entrance optics, or fast-changing atmospheric conditions could be reasons for such deviations. 

Mean bias error (MBE) and root mean square error (RMSE) were calculated representing 

average differences from the three outdoor runs and results from the aggregation of hundredwavelength bins. Almost all instruments were within +10% MBE and 10% RMSE. 

Simulations using the Simple Model of the Atmospheric Radiative Transfer of Sunshine 

(SMARTS) were applied to the outdoor intercomparison as an explanatory tool and to 

understand how well the SMARTS-modeled spectra compare to various types of 

spectroradiometers considering the model’s spectral resolution compared to the 

spectroradiometers under scrutiny. Running the smoothing postprocessor of the SMARTS model 

was therefore necessary to downgrade the resolution of its spectra and make them match that of 

any specific instrument based on the shape of its passband (e.g., Gaussian), its width (as 

measured by the full width at half maximum), and its wavelength step (e.g., 5 nm). The 

SMARTS model in the outdoor intercomparison provides relevant information when predicting 

clear-sky solar spectral irradiance under varying atmospheric conditions. The output from the 

model compared well to the outdoor spectroradiometers’ spectral irradiance outputs, and the 

differences were within the margin of error.
vi 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Table of Contents

List of Figures ........................................................................................................................................... vii

List of Tables ............................................................................................................................................. vii

1 Introduction ........................................................................................................................................... 1

2 Objective................................................................................................................................................ 2

3 Method and Instrumentation ............................................................................................................... 3

4 Results and Discussions..................................................................................................................... 6

4.1 Indoor Intercomparison.................................................................................................................... 6

4.2 Outdoor Intercomparison................................................................................................................. 9

4.2.1 Experimental Conditions................................................................................................... 9

4.2.2 Experimental Results ...................................................................................................... 11

4.2.3 Modeled Spectra ............................................................................................................. 13

4.2.4 Scanning Speed and Cloud Detection............................................................................. 15

4.2.5 Performance Analysis..................................................................................................... 16

4.2.6 Comparison to a Single Instrument................................................................................. 18

5 Summary ............................................................................................................................................. 19

6 References .......................................................................................................................................... 20
vii 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

List of Figures

Figure 1. Schematic layout of the relative position of each participating spectroradiometer............ 4

Figure 2. Performance statistics results using Equation 1 .................................................................... 7

Figure 3. Plots showing indoor differences/ratios performed using the NIST FEL lamp.................... 8

Figure 4. (Left) Broadband data for global horizontal irradiance, direct normal irradiance, diffuse 

horizontal irradiance, and solar zenith angle (secondary axis). The yellow box indicates the 

outdoor measurement period. (Top right) Picture taken from SRRL showing the frequent 

photochemical smog over Denver, Colorado. Photo by Mike Dooraghi and Tom Stoffel (Bottom 

right) Picture from the SRRL TSI 880 sky imager showing the small and fast cumulus cloud 

that affected measurements.............................................................................................................. 10

Figure 5. One-minute broadband data for global horizontal irradiance and direct normal irradiance 

during the experimental period showing the rapid variations in both signals, the decreasing 

trend in direct normal irradiance after 11:00 LST, and the impact of the passing cumulus cloud 

at solar noon. ...................................................................................................................................... 10

Figure 6. A fast-response instrument (NREL-5) showing a dip in spectral measurement due to a 

small and fast-moving cumulus cloud at approximately 11:55:11 LST. Spectral measurements 

before and after the 11:55:11 measurements do not show the dip............................................... 11

Figure 7. The three outdoor runs, from top to bottom, showing the (left) spectral irradiance plot for 

each participating spectroradiometer and SMARTS model spectral irradiance output and 

(right) ratio using the average spectrum from the participating spectroradiometers ................ 13

Figure 8. Three spectral irradiance results using the SMARTS model. Input values were selected 

using the applicable period of run.................................................................................................... 15

Figure 9. MBE in percent (from Table 2) for various spectral ranges ................................................. 17

Figure 10. RMSE in percent (from Table 2) for various spectral ranges ............................................. 17

Figure 11. Spectral irradiance data differences for the three runs...................................................... 18

List of Tables

Table 1. Participating Spectroradiometer Characteristics...................................................................... 5

Table 2. Average MBE and RMSE in Percent ......................................................................................... 16
1 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

1 Introduction

In September 2013, an intercomparison of spectroradiometers measuring global horizontal 

spectral irradiance measurements was performed. Five calibration laboratories participated in 

this intercomparison, which was held at the National Renewable Energy Laboratory’s (NREL’s) 

Solar Radiation Research Laboratory (SRRL). The intercomparison was performed to understand 

the reliability of the spectroradiometer systems, compare results from different instruments, and 

share knowledge about the process of making spectral irradiance measurements using various 

types of spectroradiometers. The intercomparison is part of an internal performance-based 

quality-control check to monitor the legitimacy of a measurement and calibration undertaken by 

a laboratory (e.g., NREL’s Optical Metrology Laboratory) to demonstrate compliance with

International Standards Organization/International Electrotechnical Commission (ISO/IEC) 

17025 accreditation requirements. Further, participating in an intercomparison provides technical 

competency for accredited laboratories.

Similar intercomparisons of spectroradiometers for solar applications have been conducted in 

other countries (see, for example, Martinez-Lozano et al. 2003 and Galleano et al. 2013), and 

their results have been beneficial to improving laboratories’ specialized spectral measurement

capabilities, reducing metrological sources of errors, and increasing result comparability despite 

differing equipment. In the United States, interagency intercomparisons had been conducted only 

for ultraviolet spectroradiometers (Lantz et al. 2002; Thompson et al. 1997), thus the present 

study was the first of its kind.

NREL’s Optical Metrology Laboratory coordinated the indoor and outdoor comparisons 

presented in this document. In addition to NREL, EKO instruments, Inc.; Q-Lab Corporation; 

ATLAS Material Testing Technology, LLC; and the University of Oregon’s Department of 

Physics participated in the intercomparison. The outdoor intercomparison was conducted on 

September 17, 2013, at approximately solar noon between 11:00 LST and 13:30 LST. That time 

period was selected to avoid large variations in solar zenith angle and irradiance and also to 

reduce cosine effects in global horizontal spectral irradiance measurements due to the 

instruments’ entrance optics. Three runs were made during the specified time period. The solar 

irradiance (sun) was used as the source for the comparison. Additionally, an indoor comparison 

was made at NREL’s Optical Metrology Laboratory using a National Institute of Standards and 

Technology (NIST) FEL lamp.

This report summarizes the comparison results made over the spectral range from 380 nm to

1,100 nm for global spectral irradiance measurements using different spectroradiometers. 

Because the participating spectroradiometers had differing wavelength ranges, we had to limit

the comparison to a common specified wavelength range to obtain valid results overall. 
2 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

2 Objective

An intercomparison is an essential method for evaluating the performance of spectroradiometers, 

comparing results, and objectively verifying the technical competence of laboratories. To ensure 

that all data collection, processing, and analyses are consistent, a common procedure was 

established for this intercomparison. NREL facilitated the comparison as well as the presentation 

of the results and sharing of information among participants. This report provides an unbiased 

assessment of the intercomparison under a mutually agreed framework. The requirements for this 

framework included that (1) the process and analysis of the data from the individual laboratories 

is to be kept confidential; (2) all participating spectroradiometers should have the same 

measurement setup, such as the outdoor measurement height and indoor measurement setup; (3) 

the start and end time of the outdoor measurement should be determined using the slowest 

scanning instrument; and (4) the comparison should be made using the common wavelength 

range from the participating spectroradiometers (380 nm to 1,100 nm).
3 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

3 Method and Instrumentation

The measurement was conducted indoors and outdoors using multiple specroradiometers under 

controlled laboratory conditions and under clear-sky conditions at approximately solar noon, 

respectively. This event took place during a single day, on September 17, 2013, close to the fall 

equinox, and during a period when clear conditions normally prevail at SRRL. Prior to the 

comparison, each instrument was calibrated either by the participant laboratory or by a 

recognized outside laboratory. In this report, no major effort was made to harmonize the inherent 

differences among instruments in terms of differing calibration date, instrument 

integration/measurement time, bandwidth size, or wavelength interval. The postprocessing of the 

indoor and outdoor comparisons was made at 5-nm intervals. Data from instruments with a 

higher resolution (less than 5 nm) were linearly interpolated to 5 nm before the indoor and 

outdoor data sets were compared. 

The instrument characteristics are shown in Table 1. The instruments were operated by personnel 

representing each laboratory for the three outdoor comparison runs. Further, all 

spectroradiometers sat on the same height of measurement to avoid any occlusion of one 

instrument by another from the entire sky vault (180°). This means that all spheres, cosine 

receptors, and diffusers sat 25 cm above the sitting surface. A run is determined by the duration 

of time that the slowest instrument takes to finish its scan. For the outdoor measurements, each 

instrument had a specific time interval within which to finish a measurement scan for the 

specified wavelength range. Therefore, instruments with faster time scans continued to measure 

until the slowest instrument finished its scan. Then, to obtain a comparable data set, the multiple 

files from the fast instruments were averaged to obtain one single result file with which the 

comparison analysis was performed for each run. All of the outdoor measurements were made

within ±1.5 hour from solar noon, thus determining a period during which the sun’s zenith angle 

was lower than 43.5°. 

The indoor comparisons were performed using NREL’s FEL tungsten lamp (F407), which 

NREL does not use to calibrate spectroradiometers. The intent of the indoor comparison was to 

identify subtle shifts and trends that would be clearly seen in each spectroradiometer 

measurement due to factors such as differing calibration laboratories, methods of calibration 

setup, or type of lamp used to calibrate the instruments. Therefore, one spectroradiometer from 

each calibration laboratory was selected. During the indoor comparison, each instrument setup 

was the same, and data collection was made by the owner of each instrument. The NREL-1 

system was considered a reference instrument for the indoor comparison because it is reliable, 

repeatable, and has less uncertainty than the other NREL spectroradiometers that participated in 

this intercomparison. 
4 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 1. Schematic layout of the relative position of each participating spectroradiometer

During the analysis, the participants’ identity remained undisclosed, thus guaranteeing data 

confidentiality. Each organization is identified here using only a generic name (Lab-1, Lab-2, 

etc.). However, each participant was privately given their respective laboratory number so that 

they could access their own results. 
5 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Table 1. Participating Spectroradiometer Characteristics 

Organization Type of 

Spectroradiometer

Wavelength 

Range

(nm)

Entrance 

Optics

Calibration 

Standard/Lamp Detector 

Calibrating 

Laboratory and Date 

of Calibration

NREL (SRRL) OL750 280–2,400 Integrating 

sphere (6-

inch)

NIST FEL Lamp

F655

ASTM G138

Silicon/Ge/Pbs NREL Optical 

Metrology Laboratory

September 16, 2013

EKO

Instruments 

WISER: MS-710/MS712

(Polychromator)

350–1,700 Dome/ 

diffuser 

Optronics FEL 

Lamp

ASTM G138

MS710: Silicon diode 

array/MS712: InGaAs 

diode array

Optronics

December 20, 2012

NREL (SRRL) LI-1800 380–1,100 Cosine 

receptor 

NIST FEL Lamp

F655

ASTM G138

Silicon NREL Optical 

Metrology Laboratory

April 15, 2013

Q-Lab OL750/ Double 

Monochromator

280–1,100 Integrating 

sphere (6-

inch)

Optronics FEL 

Lamp

ASTM G138

Silicon Gooch & Housego,

traceable to NIST

May 20, 2013

NREL (Device 

Performance 

Group)

ASD 350–2,400 Integrating 

sphere (4-

inch, with 

dome)

NIST FEL Lamp

F655

ASTM G138

Silicon diode 

array/InGaAs high-speed 

rotating grating/Extended 

infrared InGaAs highspeed rotating grating

NREL Optical 

Metrology Laboratory

June 11, 2013

NREL (Device 

Performance 

Group)

LI-1800 (with NREL 

temperature controller)

380–1,100 Cosine 

receptor 

NIST FEL Lamp

F655

ASTM G138

Silicon NREL Optical 

Metrology Laboratory

June 13, 2013

NREL (SRRL) Pulse Analysis 

Spectroradiometer 

System (PASS)

280–1,720 Integrating 

sphere (6-

inch)

NIST FEL Lamp

F655

ASTM G138

Silicon/InGaAs NREL Optical 

Metrology Laboratory

September 16, 2013

ATLAS OL770

CCD Array

380–1,100 Integrating 

sphere (2-

inch)

OL752-10 Plug-in 

Standard 

ASTM G138

Silicon diode array ATLAS

September 4, 2013

University of 

Oregon 

LI-1800 (with 

University of Oregon

temperature controller)

380–1,100 Cosine 

receptor 

LI-COR 1800-02 

Optical Radiation 

Calibrator (ORC)

ASTM G138

Silicon LI-COR, traceable to 

NIST 

September 29, 2011
6 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4 Results and Discussions

4.1 Indoor Intercomparison

As mentioned above, the indoor comparison was performed using NREL’s FEL tungsten lamp 

(F407). This lamp was selected for the comparison because it had less than 37 hours of usage, 

which is less than the 50 hours of use recommended by NIST, and also demonstrated good 

repeatability through time. Further, a statistical procedure was conducted to achieve a reasonable 

comparison result. The En performance statistics was selected, according to ISO/IEC 17043:2010 

(Conformity assessment—General requirements for proficiency testing). Participants provided 

the estimated uncertainty value for spectral irradiance measurements using their

spectroradiometer. To simplify the statistical comparison using the performance statistics 

method, the spectral irradiance data from the participating laboratories’ spectroradiometers and 

the NREL reference spectroradiometer were integrated to seven 100-nm bins: 380 nm–400 nm, 

400 nm–500 nm, 500 nm–600 nm, 600 nm–700 nm, 700 nm–800 nm, 800 nm–900 nm, 900 nm–

1,000 nm, 1,000 nm–1,100 nm. 

The performance statistic is defined as

En = 

MLABn+ MREF

��ULABn∗MLABn�

2

+ (UREF∗MREF)2

(1) 

where En is normalized error (unitless) and ULABn and UREF are the reported calibration 

uncertainty in percent for the participating spectroradiometers of each laboratory and the NREL 

reference spectroradiometer, respectively. MLABn represents the integrated measurements from 

participating organizations (LAB1, LAB2, LAB3, etc.) in W/m2

, and MREF is the integrated 

measurement data from the reference instrument (NREL-1) in W/m2.

The performance statistics comparison (based on Equation 1) used integrated values in 100-nm 

bins, i.e., between 380 nm and 1,100 nm. Prior to the integration of the data, all data sets from 

the participating laboratories were interpolated to 5-nm intervals to obtain comparable data sets. 

Typically, the reported uncertainty from each participant is based on a 95% level of confidence. 

The observed deviation of En in Equation 1 involves the establishment of acceptable limits of ±1. 

Each reported limit is then determined to be either satisfactory (i.e., within the designated 

performance limit) or unsatisfactory (outside of the designated limit). 

The results presented in this section contain all the necessary corrections using the calibration 

information for each spectroradiometer. Prior to the intercomparison, each participating 

instrument was calibrated by the participant in their own calibration facility/arrangement. 

Figure 2 and Figure 3 demonstrate the performance of each participating spectroradiometer 

relative to the NREL-1 system using the En statistic from Equation 1. 
7 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 2. Performance statistics results using Equation 1 

All laboratories reported satisfactory results for integrated irradiances for all the 100-nm 

intervals between 380 nm and 1,100 nm (Figure 2). However, it was observed that each 

laboratory’s spectroradiometer showed different signatures in their performance statistic En. This 

could be a result of differences in calibration procedures, differing calibration setups from one 

laboratory to another, differing environmental conditions inside laboratories, whether a primary 

or secondary spectral irradiance calibration lamp was used for the calibrations, instrument age, 

and time elapsed since the last calibration, among others. It is also important to emphasize that 

the calibration uncertainty provided by each participating laboratory affects the determination of 

whether the performance statistic for their instrument is satisfactory (Equation 1). Lower 

reported calibration uncertainties could trigger higher or lower En values (even outside the limit 

of ±1), which would push the participating laboratory into an unsatisfactory category. 
8 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 3. Plots showing indoor differences/ratios performed using the NIST FEL lamp
9 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

The top plot in Figure 3 shows the spectral irradiance in Wm-2nm

-1

 versus wavelength (nm), 

whereas the bottom four plots provide the spectral irradiance ratios relative to the NREL-1 

instrument. These ratios display significant differences from one instrument to another, with 

Lab-3 showing the closest similarity to the NREL-1 measurements and Lab-4 showing 

significant differences at specific wavelengths. Lab-1 values are persistently lower than NREL-1, 

whereas Lab-2 values are persistently higher. These ratios do not take into account the reported 

calibration uncertainty included in the proficiency test calculation of Figure 2. 

4.2 Outdoor Intercomparison

4.2.1 Experimental Conditions

As mentioned above, the selection of the solar noon period has significant advantages because 

there is a reduction in inadvertent sources of errors and irradiance variations are limited during 

the measurements, especially when slow scanning instruments are used. 

During the outdoor event, clear-sky conditions prevailed, except for a very short period (a few 

seconds) at solar noon, during which a small and fast-moving cumulus cloud obscured the sun. 

Further, as is shown in the broadband irradiance records from the main SRRL pyrheliometers 

and pyranometers (Figure 4 and Figure 5), there were slight and rapid irradiance perturbations 

around solar noon. These are typical circumstances at SRRL. They can be related to the location 

of SRRL on top of a mesa, which is thus affected by turbulent upslope flows of pollution 

aerosols and transients during the rapid lifting of the mixing layer during daytime (Gueymard 

2011) and, to some extent, to the frequent photochemical smog conditions over the whole 

Denver agglomeration. 

Rapidly changing atmospheric conditions have a direct effect on the spectral irradiance sensed 

by all spectroradiometers, more particularly those that have a slow scanning mechanism. 

Differing scan rates or sizes of the entrance optics can also impact the instrument-to-instrument 

response variability in fast-changing atmospheric conditions. From the results shown in Figure 5, 

it appears that during the measurement period global horizontal irradiance first increased from 

833 W/m2

 at 11:00 LST to a daily maximum of 863 W/m2 at 12:00 LST then decreased to 784 

W/m2 at 13:30 LST. These variations are normal for that season but may present a challenge for 

measurements carried out with instruments having very slow scanning mechanisms. Using the 

test day conditions, for instance, a natural variation of up to ±5% in global irradiance would 

typically occur during a 45-min scan. This natural variation adds to the uncertainty of the 

measurement itself, which has a similar magnitude. A possible way to limit this issue would be 

to use such instruments during the summer solstice period, when the solar zenith angle, and thus 

global horizontal irradiance, varies less during the noon hours. Clear-sky conditions, however, 

are not as frequent during that period as they are in September. 
10 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

 

Figure 4. (Left) Broadband data for global horizontal irradiance, direct normal irradiance, diffuse 

horizontal irradiance, and solar zenith angle (secondary axis). The yellow box indicates the 

outdoor measurement period. (Top right) Picture taken from SRRL showing the frequent 

photochemical smog over Denver, Colorado. Photo by Mike Dooraghi and Tom Stoffel (Bottom 

right) Picture from the SRRL TSI 880 sky imager showing the small and fast cumulus cloud that 

affected measurements. 

Figure 5. One-minute broadband data for global horizontal irradiance and direct normal irradiance 

during the experimental period showing the rapid variations in both signals, the decreasing trend 

in direct normal irradiance after 11:00 LST, and the impact of the passing cumulus cloud at solar 

noon. 

Photochemical smog over

Small and fast Denver (September 17, 2013)

cumulus cloud

Small and fast 

cumulus 

cloud at 12:00

p.m. It 

obscured the 

sun at 11:56

a.m. 
11 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4.2.2 Experimental Results

Two of the NREL spectroradiometers collected data at 5-nm intervals. Therefore, to obtain 

comparable results, the outdoor data analysis was performed at 5-nm intervals for all 

instruments. A linear interpolation method was applied to data sets obtained from the instruments 

that have a higher resolution. Further, the time used for each measurement run was determined 

by the time taken by the slowest instrument to finish its scan, i.e., approximately 45 minutes. The 

data from the faster instruments was averaged to perform the analysis. These values were 

obtained from the spectral irradiance measurement by the spectroradiometers in the range of 380 

nm to 1,100 nm. An average spectral irradiance was calculated for each wavelength. The 

measured data from each spectroradiometer, as well as the modeled data from the Simple Model 

of the Atmospheric Radiative Transfer of Sunshine (SMARTS) code (see details in Section 

4.2.3), was divided by the average values calculated from all measured data. Aside from spectral 

regions corresponding to sharp absorption bands, a fast-moving cloud (Figure 6), or spikes 

detected in one instrument due to unknown conditions, the relative difference among the 

instruments was less than 10% (Figure 7 (right)). 

Figure 6. A fast-response instrument (NREL-5) showing a dip in spectral measurement due to a 

small and fast-moving cumulus cloud at approximately 11:55:11 LST. Spectral measurements 

before and after the 11:55:11 measurements do not show the dip.

The solar noon for September 17, 2013, occurred at approximately 11:54:30 LST, coinciding 

almost exactly with the passage of the noted cumulus cloud (Figure 4, Figure 5, and Figure 6). 

Except for such anomalous conditions, runs conducted close to solar noon are expected to yield 

lower differences in results than at any other time of the day. Indeed, the second run period, 

which was very close to solar noon, demonstrated relatively smaller differences. That second run 

had comparatively less air mass because of the smallest solar zenith angles of the day, as well as

less variations in global horizontal irradiance during scan times. 
12 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

The 380-nm to 400-nm and 1,000-nm to 1,100-nm spectral bands showed relatively higher 

differences compared to the rest of the spectrum. The relative higher percent difference of the 

former band could be related to the lower performance of most spectroradiometers in the 

ultraviolet region coupled with the higher uncertainty in the output of calibration lamps. The 

latter range (in the near infrared) could be related to the temperature dependence of the silicon 

detectors. In particular, out of the three LI-COR 1800s, one was not equipped with a temperature 

controller during the intercomparison. Moreover, one of the temperature-controlled LI-COR

1800s may have been calibrated at a different temperature than what was used during the 

intercomparison. In any case, the larger uncertainty of that instrument below 400 nm and above 

900 nm has previously been described in the literature (Cachorro et al. 2009; Myers 1989). 
13 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 7. The three outdoor runs, from top to bottom, showing the (left) spectral irradiance plot for 

each participating spectroradiometer and SMARTS model spectral irradiance output and (right) 

ratio using the average spectrum from the participating spectroradiometers

4.2.3 Modeled Spectra

In parallel to the outdoor intercomparison itself, a different experiment is dicussed in this section

to evaluate the additonal potential that may be offered by using modeled spectra. This avenue 

was already explored with success in previous intercomparisons (Martinez-Lozano et al. 2003; 

Galleano et al. 2013). As in these earlier studies, the SMARTS code (Gueymard 1995, 2001) was 

retrospectively used to obtain modeled spectra at the time of the measurements. 

Real Event (Small Cloud briefly covered the sun)

Unknown

Unknown
14 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Comparisons among SMARTS-modeled spectra and actual spectroradiometric measurements at 

SRRL have been conducted in the past, usually to validate the model itself (Gueymard 2002, 

2008, 2011). In the present case, the main intent was to better understand how well the 

SMARTS-modeled spectra compare to various types of spectroradiometers considering that the 

model has a finer resolution than the instruments under scrutiny here. Running the smoothing 

postprocessor of SMARTS was therefore necessary to downgrade the resolution of its spectra 

and make it match that of any specific instrument based on the shape of its passband (e.g., 

Gaussian), its width (as measured by the full width at half maximum), and its wavelength step 

(e.g., 5 nm). 

Another aspect to consider when evaluating the potential of using a model such as SMARTS in 

parallel with spectroradiometric measurements is that scanning instruments (as opposed to solidstate instruments) require more or less significant time to complete their scan during their 

complete spectral range. In contrast, the model can produce results for any instant during that 

period. Using repeated simulations (for example, every 5 minutes), it becomes possible to 

provide useful information about how the actual spectrum changed (at what wavelengths and by 

how much) during a slow scan. This option, however, requires high-frequency collocated 

measurements of aerosol optical depth and precipitable water, in particular, because these are the 

primary—and variable—atmospheric inputs to the model. Such measurements are conducted at 

only a few laboratories in the world, now including SRRL. 

Simulations of global spectral irradiance using SMARTS are shown in Figure 8 for an assumed 

instrument with a Gaussian 5-nm bandpass and 1-nm wavelength step. These spectra correspond 

to the mid-times of the three individual runs (11:30 LST, 12:15 LST, and 13:00 LST), yielding 

air masses of 1.27, 1.27, and 1.32, respectively. The atmospheric conditions did not change much 

throughout the whole period, so that the spectra at 11:30 LST and 12:15 LST were virtually 

identical. In the 13:00 LST spectrum, the slight increase in air mass induced an average decrease 

of approximately 4% in irradiance compared to the two earlier spectra. The SMARTS-modeled 

broadband global horizontal irradiance values for the three times just mentioned (865.4 W/m2

, 

866.7 W/m2

, and 827.0 W/m2, respectively) closely matched the pyranometer measurements 

(855.5 W/m2

, 856.6 W/m2, and 831.4 W/m2, respectively) from SRRL. The following locally 

measured information were used to generate these spectra:

• Station pressure (nearly constant): approximately 816 mb

• Total ozone amount (average for that day): 0.281 atm-cm 

• Total nitrogen dioxide amount (default value): 0.2 matm-cm

• Precipitable water (measured by SRRL’s global positioning system): 1.20 cm to 1.41 cm 

• Aerosol optical depth at 500 nm (average of measured value from two sunphotometers at 

SRRL): 0.06 

• Ångström exponent (derived from the direct spectrum measured with a PGS100 

instrument located at SRRL): 2.0 

• Aerosol single-scattering albedo (derived from sunphotometer data using an inversion 

method from SRRL): 0.98 
15 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

• Aerosol asymmetry factor (derived from sunphotometer data using an inversion method 

from SRRL): 0.71 

• Spectral surface reflectance (assumed; selected from the SMARTS library): dry grass

The SMARTS model outputs were used only for corroborative information, and, as previously 

stated, were not included in the average or difference calculations. 

Figure 8. Three spectral irradiance results using the SMARTS model. Input values were selected 

using the applicable period of run.

4.2.4 Scanning Speed and Cloud Detection

Measuring the spectral irradiance using spectroradiometers involves scanning times from a few 

seconds to multiple minutes. Under fast-changing atmospheric conditions, slow scanning 

instruments could have a problem in correctly detecting any natural temporal variability. This 

may explain some of the largest differences that are apparent in Figure 7. In the present case, the 

intercomparison was performed under relatively stable sky conditions, which means that even 

larger differences could result from intercomparing instruments under more variable conditions. 

As mentioned above, a small and fast-moving cloud briefly obscured the sun during one of the 

outdoor runs. This was detected by some instruments and resulted in distorted spectra, as shown 

in Figure 6 and Figure 7. Generally speaking, fast spectroradiometers assist in detecting a rapid 

temporal variation of spectral irradiance due to fast-moving clouds. However, during the 

intercomparison, not all fast-scanning instruments detected the fast-moving cloud. This could be 

explained by the location of the scanner in a specific wavelength during the passage of the cloud. 

For example, if the scanner of the fast instrument was located in the infrared region of the 

wavelength during the cloud passage, then the change in the measured spectral irradiance would 

be relatively small and relatively unnoticeable. 
16 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4.2.5 Performance Analysis

In this report, we illustrate the comparison results and any presence of systematic (bias) or 

random (scatter) tendencies in the spectral irradiance measurement from each spectroradiometer 

relative to the measured spectral irradiance averaged throughout 100-nm wide spectral bands and 

among all instruments. To perform these calculations, mean bias error (MBE) and root mean 

square error (RMSE) were calculated. The results are shown in percent MBE and RMSE, and 

they are relative to the average reading of the hundred-wavelength bins. Further, the MBE and 

RMSE were averaged for the three outdoor runs, as shown in Table 2. 

The percent MBE was calculated using the following equation: 

MBE(%) = � 1

n ∑n

i=1

(xi−x̅)

x̅ � ∗ 100

Similarly, the percent RMSE was calculated using: 

RMSE(%) = �� 1

n ∑n

i=1 �

xi−x̅

x̅ �

2

� ∗ 100

where xi represents values from the spectral irradiance under each bin and x̅represents the 

average of spectral irradiance within each bin. 

Table 2. Average MBE and RMSE in Percent 

Bins Average of All 

vs. NREL-2 

Average of All 

vs. Lab-1 

Average of All 

vs. Lab-3 

Average of All 

vs. NREL-4 

Average of All 

vs. NREL-5 

Average of All 

vs. Lab-4 

Average of All 

vs. Lab-2 

Average of All 

vs. NREL-1 

Average of All 

vs. NREL-3 

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

MBE 

(%)

RMSE 

(%)

380–400 -0.75 4.85 -8.18 9.77 -1.18 3.76 0.02 7.85 -3.06 5.37 8.18 8.44 6.10 7.11 4.83 6.06 -5.98 6.31

400–500 1.20 3.60 -2.68 4.57 1.34 5.27 -1.90 3.88 -2.92 3.77 8.65 8.97 -4.85 17.52 4.88 5.55 -3.72 4.49

500–600 -0.57 1.52 -1.92 2.04 -0.05 3.34 -2.47 2.54 -3.79 3.81 5.45 5.81 4.24 4.94 2.24 2.29 -3.12 3.30

600–700 -1.43 3.52 -2.42 2.51 -0.05 3.40 -2.57 2.69 -3.32 3.41 5.11 5.17 4.94 5.42 1.90 2.05 -2.16 2.35

700–800 0.14 1.42 -2.92 3.41 0.13 4.25 -3.76 4.85 -2.79 3.27 3.83 4.01 3.16 6.87 2.97 4.26 -0.77 4.35

800–900 0.23 1.18 -3.02 3.15 0.10 3.29 -3.90 3.92 -2.73 2.96 3.05 3.11 4.76 5.27 2.79 2.88 -1.28 1.85

900–1,000 -0.31 2.60 -1.61 2.44 1.10 3.52 -2.64 4.76 -2.56 4.42 1.77 2.78 2.35 10.19 2.96 3.60 -1.05 3.73

1,000–

1,100 -0.41 1.13 -0.96 2.55 -0.63 2.88 -3.06 3.46 -2.31 3.34 -4.55 5.83 5.84 11.04 2.25 2.53 3.82 4.45

Figure 9 and Figure 10 show a visual representation of the Table 2 results. Lab-1 had a larger

negative MBE for the 380-nm to 400-nm bin. Lab-4 had a higher MBE, between 380 nm and 

500 nm.
17 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 9. MBE in percent (from Table 2) for various spectral ranges 

Lab-2 appeared to have a higher RMSE for the 400-nm to 500-nm range (Figure 10). As 

described previously, this is likely related to the fast-moving cloud that obscured the sun during 

the second run and to the unknown experimental condition that occurred during the third outdoor 

run. Another unknown condition is responsible for the poor performance of Lab-2 in the 900-nm 

to 1,100-nm range. 

Figure 10. RMSE in percent (from Table 2) for various spectral ranges 

However, it is important to note that the differences shown in Table 2, Figure 9, and Figure 10

are average differences from the three outdoor runs and result from the aggregation of hundredwavelength bins. Because of this averaging of multiple runs and to the spectral aggregation 

method, relatively low differences were likely obtained because of error cancellations. 
18 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4.2.6 Comparison to a Single Instrument

In previous sections, an average of the measured spectral irradiance data was used to determine 

the differences among spectroradiometers. However, to avoid any bias that might be incurred by 

using instruments with no temperature controller or instruments with calibration problems—

which could eventually affect the averaging of the measured data—in this section, NREL-2 was 

used as a reference instrument to understand the differences of spectral irradiance data. This does 

not mean that the NREL-2 is the best instrument among the participating spectroradiometers for 

outdoor measurements. 

Figure 11 shows the ratio obtained by comparing each measured spectral data from the 

spectroradiometers and the SMARTS model spectral irradiance output to the NREL-2 spectral 

irradiance data set. Overall, the results showed differences within the ±10% limit. Note, 

however, that the ultraviolet and infrared regions edged close to this limit. 

Figure 11. Spectral irradiance data differences for the three runs 

Small and fast-moving 

cloud captured by NREL-2 

O2 band H2O band
19 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

5 Summary

The results from the intercomparison exercise described here provide various benefits, such as a 

better understanding of the performance of the different spectroradiometers under scrutiny, a 

common platform to compare spectral irradiances under both indoor and outdoor conditions, and 

the means to verify the technical competence of laboratories. During the event, there were issues

related to instrument functions and a few problems related to calibration issues when the 

calibration file did not work properly with the setting that we used. Some of the problems were 

solved, and some were not. These types of issues, coupled with the inherent differences in 

instrument design, calibration methods, and age; source of spectral irradiance calibration lamps; 

the amount of time since the last calibration of the instrument; incidence angle; reported 

calibration uncertainty by the laboratories; wavelength shift; environmental conditions; 

interpolation of data during the analysis; or other experimental issues are the primary reasons for 

the differences in spectral irradiance measurements in the analysis. 

The instruments that participated in the indoor tests demonstrated satisfactory comparisons. The 

outdoor comparison differences were within ±10%, but the ultraviolet and near-infrared spectral 

bands showed relatively higher differences than the rest of the spectrum. This is explained in part 

by the usually low performance of most spectroradiometers in the ultraviolet region (a known 

issue)—unless they are specifically designed to sense ultraviolet wavelengths. Moreover, the 

spectral irradiance calibration lamps, whether they are NIST primary FEL lamps or secondary 

lamps, have relatively higher uncertainty in the ultraviolet region due to their low output. Silicon 

detectors that are not stabilized using temperature controllers usually have low performance in 

the near-infrared region. This was indeed observed for some of them during the intercomparison. 

Further, including simulated spectra from the SMARTS model in the outdoor intercomparison

provides relevant information when predicting clear-sky solar spectral irradiance under varying 

atmospheric conditions. The output from the model compares well to the outdoor 

spectroradiometers spectral irradiance output, and the differences were within the margin of 

error.

It is anticipated that similar intercomparisons will be held on a regular annual basis in the future 

and will involve more instruments. The experience gathered during this first event will certainly 

lead to improved and/or expanded experimental protocols during the preparation of future 

editions, with the objective to further reduce inter-laboratory differences and uncertainties.
20 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

6 References

Cachorro, V.E.; Berjon, A.; Toledano, C.; Mogo, S.; De Frutos, A.M.; Vilaplana, J.M.; Sorribas, 

M.; De La Morena, B.A.; Gröbner, J.; Laulainen, N. (2009). “Detailed Aerosol Optical Depth 

Intercomparison between Brewer and Li-Cor 1800 Spectroradiometers and a Cimel Sun 

Photometer.” Journal of Atmospheric and Oceanic Technology (26); pp. 1,558–1,573. 

Galleano, R.; Zaaiman, W.; Virtuani A.; Pavanello, D.; Morabito, P.; Minuto, A.; Spena, A.; 

Bartocci, S.; Fucci, R.; Leanza, G.; Fasanaro, D.; and Catena, M. (2013). “Intercomparison 

Campaign of Spectroradiometers for a Correct Estimation of Solar Spectral Irradiance: Results 

and Potential Impact on Photovoltaic Devices Calibration.” Progress in Photovoltaics: Research 

and Applications. Accessed February 2014: 

http://onlinelibrary.wiley.com/doi/10.1002/pip.2361/full. 

Gueymard, C.A. (1995). SMARTS2: A Simple Model of the Atmospheric Radiative Transfer of 

Sunshine: Algorithms and Performance Assessment. FSEC-PF-270-95. Cocoa, Florida: Florida 

Solar Energy Center. Accessed February 2014: 

http://www.solarconsultingservices.com/SMARTS2_report.pdf. 

Gueymard, C.A. (2001). “Parameterized Transmittance Model for Direct Beam and Circumsolar 

Spectral Irradiance.” Solar Energy (71); pp. 325–346. 

Gueymard, C.A. (2008). “Prediction and Validation of Cloudless Shortwave Solar Spectra 

Incident on Horizontal, Tilted, or Tracking Surfaces.” Solar Energy (82); pp. 260–271. 

Gueymard, C.A. (2011). Validation of Direct Irradiance Spectra Predicted by the SMARTS 

Radiative Code Using NREL’s Radiometric Data. Work performed by Solar Consulting 

Services. Contract No. AGJ-0-40256-01. Golden, CO: National Renewable Energy Laboratory. 

Gueymard, C.A.; Myers, D.; Emery, K. (2002). “Proposed Reference Irradiance Spectra for 

Solar Energy Systems Testing.” Solar Energy (73); pp. 443–467. 

International Organization for Standards. (1997). ISO/IEC 17043:2010—Conformity 

Assessment—General Requirements for Proficiency Testing. Geneva, Switzerland: International 

Organization of Standardization. 

Lantz, K.; Disterhoft, P.; Early, E.; Thompson, A.; DeLuisi, J.; Kiedron, P.; Harrison, L.; Berndt, 

J.; Mou, W.; Erhamjian, T.J.; Cabausua, L.; Robertson, J.; Hayes, D.; Slusser, J.; Bigelow, D.;

Janson, G.; Beaubian, A.; Beaubian, M. (2002). “The 1997 North American Interagency 

Intercomparison of Ultraviolet Monitoring Spectroradiometers.” Journal of Research of NIST 

(National Institute of Standards and Technology) (107); pp. 19–62. 

Martinez-Lozano, J.A.; Utrillas, M.P.; Pedros, R.; Tena, F.; Diaz, J.P.; Exposito, F.J.; Lorente, J.; 

De Cabo, X.; Cachorro, V.; Vergaz, R.; and Carreno, V. (2003). “Intercomparison of 

Spectroradiometers for Global and Direct Solar Irradiance in the Visible Range.” Journal of 

Atmospheric and Oceanic Technology (20); pp. 997–1,010. 
21 

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Myers, D.R. (1989). “Estimates of Uncertainty for Measured Spectra in the SERI Spectral Solar 

Radiation Data Base.” Solar Energy (43); pp. 347–353. 

Thompson, A.; Early, E.A.; DeLuisi, J.J.; Disterhoft, P.; Wardle, D.; Kerr, J.; Rives, J.; Sun, Y.;

Lucas, T.; Mestechkina, T.; Neale, P. “The 1994 North American Interagency Intercomparison 

of Ultraviolet Monitoring Spectroradiometers.” Journal of Research of NIST (National Institute 

of Standards and Technology) (102); pp. 279–322. 

Yoon, H.W.; Gibson, C. (2001). NIST Special Publication 250-89—NIST Measurement Services: 

Spectral Irradiance Calibration. Gaithersburg, MD: National Institute of Standards and 

Technology. Accessed February 2014: http://www.nist.gov/calibrations/upload/sp250-89.pdf.