ARM TR-017

> Total Sky Imager (TSI) Handbook
>
> June 2005
>
> V. R. Morris
>
> Work supported by the U.S. Department of Energy,
>
> Office of Science, Office of Biological and Environmental Research

# Contents {#contents .Heading-Front}

[1. General Overview 1](#general-overview)

[2. Contacts 1](#contacts)

[3. Deployment Locations and History
1](#deployment-locations-and-history)

[4. Near-Real-Time Data Plots 1](#near-real-time-data-plots)

[5. Data Description and Examples 1](#data-description-and-examples)

[6. Data Quality 1](#data-quality)

[7. Instrument Details 1](#instrument-details)

# Tables {#tables .unnumbered}

[1. Current Status and Locations 1](#__RefHeading___Toc105825084)

[2. Primary Variables 1](#__RefHeading___Toc105825085)

[3. Secondary Variables 1](#__RefHeading___Toc105825086)

[4. Diagnostic Variables 1](#__RefHeading___Toc105825087)

[4. Data Quality Flags 1](#__RefHeading___Toc105825088)

[5. Data Quality Thresholds 1](#__RefHeading___Toc105825089)

[6. Dimension Variables 1](#__RefHeading___Toc105825090)

[7. Instrument Specifications 1](#__RefHeading___Toc105825091)

# General Overview

The total sky imager (TSI) provides time series of hemispheric sky
images during daylight hours and retrievals of fractional sky cover for
periods when the solar elevation is greater than 10 degrees.

# Contacts

## Mentor

Victor Morris\
Pacific Northwest National Laboratory\
P.O. Box 999, MS K9-24\
Richland, WA 99352\
Phone: 509-372-6144

Fax: 509-372-6268\
Email: <victor.morris@pnl.gov>

## Instrument Developer

Yankee Environmental Systems (YES), Inc.\
101 Industrial Blvd.\
Turners Falls, MA 01376\
Phone: 413-863-0200, ext. 7201\
Fax: 413-863-0255\
Website: [http://www.yesinc.com](http://www.yesinc.com/)

# Deployment Locations and History 

[]{#__RefHeading___Toc105825084 .anchor}Table 1. Current Status and
Locations

  ------------ -------------- --------------- ----------------- -------------
  **Serial\    **Property\    **Location**    **Installation\   **Status**
  Number**     Number**                       Date**            

  660100       WD41403        PYE/MF1         2005/02/01        operational

  880102       WD30270        TWP/CF1         2003/11/30        operational

  880105       WD30880        TWP/CF3         2002/07/16        operational

  880106       WD30881        SGP/CF1         2000/07/02        operational

  880107       WD30882        TWP/CF2         2002/11/12        operational
  ------------ -------------- --------------- ----------------- -------------

# Near-Real-Time Data Plots

Available at [DQ HandS](http://dq.arm.gov/) (Data Quality Health and
Status).

# Data Description and Examples

See [YES Cloud Cover
Products](http://www.yesinc.com/products/data/tsi880/index.html) and
[YES Imaging Product
Demos](http://www.yesinc.com/products/demos/img.html).

## Data File Contents

The following datastreams produced by the TSI are available from the
[ARM Archive](http://www.archive.arm.gov/):

-   **tsiskycover** -- [fractional sky
    cover](http://www.arm.gov/measurements/measurement.php?id=90) and
    [sun obscuration by
    cloud](http://www.arm.gov/measurements/measurement.php?id=91)

-   **tsiskyimage** -- hemispheric sky image (JPEG)

-   **tsimovie** -- daily movie of hemispheric sky images (MPEG)

-   **tsicldmask** -- processed [fractional sky
    cover](http://www.arm.gov/measurements/measurement.php?id=90) image
    (PNG).

ARM netCDF file header descriptions may be found at [TSI Data Object
Design](http://science.arm.gov/tool/dod/showdod.php?Inst=tsi).

### Primary Variables and Expected Uncertainty

Visual record of sky conditions.

Fractional sky cover (clear, thin, and opaque amounts).

Sun obscuration by cloud (sunshine meter).

[]{#__RefHeading___Toc105825085 .anchor}Table 2. Primary Variables

  ------------------------- -------------------------------- -------------
  **Variable Name**         **Quantity Measured**            **Unit**

  percent.opaque            Percent opaque cloud             percent

  percent.thin              Percentage thin cloud            percent

  sunny                     Sunshine meter                   none
  ------------------------- -------------------------------- -------------

#### Definition of Uncertainty

See ARM Technical Report "[Total Sky Imager Model 880 Status and Testing
Results](http://www.arm.gov/publications/tech_reports/arm-tr-006.pdf)."

### Secondary/Underlying Variables

[]{#__RefHeading___Toc105825086 .anchor}Table 3. Secondary Variables

  ----------------------------- ------------------------------------------- ----------
  **Variable Name**             **Quantity Measured**                       **Unit**

  solar.altitude                Sun altitude above horizon                  degrees

  solar.azimuth                 Solar azimuth angle                         degrees

  region.zenith.count.thin      Pixel count: number thin in zenith circle   pixels

  region.zenith.count.opaque    Pixel count: number opaque in zenith circle pixels

  region.zenith.count           Pixel count: number total in zenith circle  pixels

  region.sun.count.thin         Pixel count: number thin in sun circle      pixels

  region.sun.count.opaque       Pixel count: number opaque in sun circle    pixels

  region.sun.count              Pixel count: number total in sun circle     pixels

  region.horizon.count.thin     Pixel count: number thin in horizon area    pixels

  region.horizon.count.opaque   Pixel count: number opaque in horizon area  pixels

  region.horizon.count          Pixel count: number total in horizon area   pixels

  count.sub.proczen             Pixel count: number total between horizon   pixels
                                and processed circle                        

  count.opaque                  Pixel count: number total opaque            pixels
  ----------------------------- ------------------------------------------- ----------

**Table 3**. (cont'd)

  -------------------- ------------------------------------------- ----------
  **Variable Name**    **Quantity Measured**                       **Unit**

  count.thin           Pixel count: number total thin              pixels

  count.box            Pixel count: box, outside mirror area       pixels

  count.sky            Pixel count: number total in processed      pixels
                       circle                                      

  count.unknown        Pixel count: number total indeterminant     pixels

  count.mask           Pixel count: camera and sun strip mask      pixels

  count.sub.horz       Pixel count: number below horizon in image  pixels
  -------------------- ------------------------------------------- ----------

### Diagnostic Variables

[]{#__RefHeading___Toc105825087 .anchor}Table 4. Diagnostic Variables

### Data Quality Flags

Most fields contain a corresponding, sample-by-sample, automated
quality-check field in the b1 level datastreams. These flags are named
**qc\_\<fieldname\>**. For example, the **percent.opaque** field also
has a companion **qc\_** **percent.opaque** field. Possible values for
each sample of the **qc\_\<fieldname\>** are shown in the table below.

[]{#__RefHeading___Toc105825088 .anchor}Table 4. Data Quality Flags

  ----------- -------------------------------------------------------------
  **Value**   **Definition**

  0           All QC checks passed

  1           Sample contained \'missing data\' value

  2           Sample was less than prescribed minimum value

  3           Sample failed both \'missing data\' and minimum value checks

  4           Sample greater than prescribed maximum value

  5           Sample failed both minimum and maximum value checks (highly
              unlikely)

  7           Sample failed minimum, maximum and missing value checks
              (highly unlikely)

  8           Sample failed delta check (change between this sample and
              previous sample exceeds a prescribed value

  9           Sample failed delta and missing data checks

  10          Sample failed minimum and delta checks

  11          Sample failed minimum, delta and missing value checks

  12          Sample failed maximum and delta checks

  14          Sample failed minimum, maximum and delta checks

  15          Sample failed minimum, maximum, delta and missing value
              checks
  ----------- -------------------------------------------------------------

The following are the current definitions for the minimum and maximum
thresholds:

[]{#__RefHeading___Toc105825089 .anchor}Table 5. Data Quality Thresholds

  ------------------------------------- ------------ --------- ------------
  **Field\                              **Units**    **Min**   **Max**
  Name**                                                       

  percent.opaque                        percent      0         100

  percent.thin                          percent      0         100

  sunny                                 none         0         1

  sun.strength                          none         -100      100

  solar.altitude                        degrees      -90       90

  solar.azimuth                         degrees      0         360

  region.zenith.count.thin              pixels       0         101400

  region.zenith.count.opaque            pixels       0         101400

  region.zenith.count                   pixels       0         101400

  region.sun.count.thin                 pixels       0         101400

  region.sun.count.opaque               pixels       0         101400

  region.sun.count                      pixels       0         101400

  region.horizon.count.thin             pixels       0         101400

  region.horizon.count.opaque           pixels       0         101400

  region.horizon.count                  pixels       0         101400

  count.sub.proczen                     pixels       -1        101400

  count.opaque                          pixels       -1        101400

  count.thin                            pixels       -1        101400

  count.box                             pixels       -1        101400

  count.sky                             pixels       -1        101400

  count.unknown                         pixels       -1        101400

  count.mask                            pixels       -1        101400

  count.sub.horz                        pixels       -1        101400
  ------------------------------------- ------------ --------- ------------

### Dimension Variables

[]{#__RefHeading___Toc105825090 .anchor}Table 6. Dimension Variables

  ----------------- ----------------------- ------------------------------
  **Variable Name** **Quantity Measured**   **Unit**

  base_time         Base time in Epoch      seconds

  lat               north latitude          Degrees

  lon               east longitude          degrees

  alt               altitude                meters above Mean Sea Level
  ----------------- ----------------------- ------------------------------

## Annotated Examples

This section is not applicable to this instrument.

## User Notes and Known Problems

TSI retrievals of fractional sky cover are valid for solar elevation
angles of 10 degrees or greater.

## Frequently Asked Questions

See [YES TSI
FAQs](http://www.yesinc.com/products/data/tsi440/faqindex.html).

# Data Quality

## Data Quality Health and Status

The following links go to current data quality health and status
results:

-   [DQ HandS](http://dq.arm.gov/) (Data Quality Health and Status)

-   [NCVweb](http://dq.arm.gov/ncvweb/ncvweb.cgi) for interactive data
    plotting using.

The tables and graphs shown contain the techniques used by ARM\'s data
quality analysts, instrument mentors, and site scientists to monitor and
diagnose data quality.

## Data Reviews by Instrument Mentor

The system is frequently monitored for continued operation. Sky cover
retrievals are monitored and spot checked by comparison of the sky
images and their corresponding "cloud decision images." This process
cannot be automated. A visual inspection detects any periods that are
not optimal, and these periods are reprocessed. Updated files are sent
to the ARM Archive.

## Data Assessments by Site Scientist/Data Quality Office

All DQ Office and most site scientist techniques for checking have been
incorporated within [DQ HandS](http://dq.arm.gov/) and can be viewed
there.

A comparison between the fractional sky cover data included in the
Shortwave Flux Analysis Value-Added Product (VAP), derived from the
broadband shortwave irradiance, and in the TSI-retrieved total sky cover
is planned.

## Value-Added Procedures and Quality Measurement Experiments

Many of the scientific needs of the ARM Program are met through the
analysisby analyzing and processing of existing data products into
"value-added" products or VAPs. Despite extensive instrumentation
deployed at the ARM CART sites, there will always be quantities of
interest that are either impractical or impossible to measure directly
or routinely. Physical models using ARM instrument data as inputs are
implemented as VAPs and can help fill some of the unmet measurement
needs of the program. Conversely, ARM produces some VAPs not in order to
fill unmet measurement needs, but instead to improve the quality of
existing measurements. In addition, when more than one measurement is
available, ARM also produces "best estimate" VAPs. A special class of
VAP called a Quality Measurement Experiment (QME), which is a special
class of VAP, does not output geophysical parameters of scientific
interest. Rather, a QME adds value to the input datastreams by providing
for continual quality assessments continuous assessment of the quality
of the input data based on internal consistency checks, comparisons
between independent similar measurements, or comparisons between
measurements with modeled results, and so forth. For more information
see, the [VAPs and QMEs web page](http://www.arm.gov/data/vaps.stm).

In addition, VAP information is derived from inferred fractional sky
cover and sunshine duration. A QME comparison with observer reports and
with whole sky imager (WSI) sky cover retrievals are underway as part of
the Southern Great Plains (SGP) Central Facility (CF) system evaluation.

# Instrument Details

## Detailed Description

### List of Components

The YES Total Sky Imager Model TSI-660 is an automatic, full-color sky
imager system that provides real-time processing and display of daytime
sky conditions. An image-processing program running on a PC workstation
captures images via TCP/IP at a 30-sec sampling interval and saves them
to JPEG files that are analyzed to infer both fractional sky cover and
sunshine duration.

For more information, visit [YES Total Sky
Imager](http://www.yesinc.com/products/data/tsi880/index.html).

### System Configuration and Measurement Methods

Images from the sky are captured via a solid-state charge-coupled device
looking downward onto a heated, rotating hemispherical mirror. A
shadowband on the mirror blocks the intense direct-normal light from the
sun, thereby protecting the imager optics. An image-processing algorithm
captures and displays the images.

The TSI is a daytime imager. Once the sun rises above a user-selectable
minimum solar zenith angle, image acquisition begins. The analysis step
first masks out obstructions - the imager, its arm, and the sun-blocking
band. Fractional sky cover is determined by a processing algorithm that
examines the color relationships of the remaining image pixels to infer
whether the pixel represents clear sky or thin or opaque cloud. In
addition, the differential of brightness along the sun blocking band is
analysed to infer if the sun is blocked by cloud or not, i.e., a
sunshine meter.

![](media/image1.png){width="2.7777777777777777in"
height="2.738888888888889in"}

Figure 1. TSI Communications diagram

### Specifications

[]{#__RefHeading___Toc105825091 .anchor}Table 7. Instrument
Specifications

  ---------------------- ------------------------------------------------
  **Parameter**          **Value**

  Image Resolution:      352 x 288 color, 24-bit JPEG format

  Sampling rate:         Variable, with max. of one image every 30 sec

  Operating Temperature: -40° C to +44° C

  Weight/Size:           Approx.70 lbs.(32 kg); dims: 20.83"x18.78";
                         height is 34.19"; mounts on 16.75x12" 1/4-20
                         bolt square

  Power Requirements:    115/230 VAC; mirror heater duty cycle varies
                         with air temperature: 560W with heater on / 60W
                         off

  Software:              Image application supports MS-Windows

  Data Storage:          Local workstation disk

  Communication:         10BaseT/RJ45 (15m)
  ---------------------- ------------------------------------------------

## Theory of Operation

Images from the sky are captured via a solid state CCD imaging camera
that looks downward on a heated hemispherical mirror. The mirror images
the hemisphere over the system into the lens, and has a solar-ephemeris
guided shadowband to block the intense direct-normal radiation from the
sun. An image-processing program running on a user-provided PC
workstation captures images via TCP/IP at a user-defined sampling rate
and saves them to JPEG files for analysis. The analysis software first
masks out known obstructions \-- the camera, its arm, and the
sun-blocking shadowband. The raw color image is analyzed for fractional
cloud cover, and both are stored as files.

## Calibration

### Theory

Sky cover processing limits are set by the instrument mentor, based on
experience and tailored to human observations.

### Procedures

This section is not applicable to this instrument.

### History

This section is not applicable to this instrument.

## Operation and Maintenance

### User Manual

This section is not applicable to this instrument.

### Routine and Corrective Maintenance Documentation

[SGP Preventative Maintenance
Procedure](http://198.124.96.210/pm_proc/tsipm.htm)

[TWP Operating
Procedure](http://www.twppo.lanl.gov/internal/pages/man_operations.html#SectionA22)

### Software Documentation

YES TSI Manager

### Additional Documentation

This section is not applicable to this instrument.

## Glossary

Sky cover - The amount of the hemispheric field-of-view of the sky from
the viewpoint of an observer standing on the surface that contains
"cloud," generally expressed in percent.

Also see [ARM Glossary](http://www.arm.gov/about/glossary.stm).

## Acronyms

JPEG: Joint Photographic Experts Group compressed digital image format\
PNG: Portable Network Graphics digital image format

TXT: ASCII text format

YES: Yankee Environmental Systems

Also see [ARM Acronyms and
Abbreviations](http://www.arm.gov/about/acronyms.stm).

## Citable References

Kassianov, E, and C Long. 2003. "Paired Ground-Based Hemispherical
Observations for Cloud Base Height Estimation." In *Thirteenth
Atmospheric Radiation Measurement (ARM) Program Science Team Meeting*,
Ed. by D Carrothers, U.S. Department of Energy, Richland, WA.

Kassianov, E, C Long, and J Christy. 2004. "ARM Cloudiness
Intercomparison IOP 2003 Analysis: Cloud Base Height." Presented at
Fourteenth ARM Science Team Meeting, Albuquerque, NM.

Kassianov, E, CN Long, M Ovtchinnikov, and J Christy. 2004. "Cloud
properties retrievals from surface hemispherical observations."
Presented at International Radiation Symposium 2004 IRS, Busan, Korea.

Kassianov, E, CN Long, and M Ovtchinnikov. 2005. "Cloud sky cover versus
cloud fraction: whole-sky simulations and observations." *Journal of
Applied Meteorology* 44: 86-98.

Long, CN, and JJ DeLuisi. 1998. "Development of an Automated Hemispheric
Sky Imager for Cloud Fraction Retrievals." In *Proceedings 10th
Symposium on Meteorological Observations and Instrumentation*, January
11-16, 1998, Phoenix, AZ.

Long, CN, DW Slater, and T Tooman. 2001. [Total Sky Imager Model 880
Status and Testing
Results](http://www.arm.gov/publications/tech_reports/arm-tr-006.pdf).
ARM Technical Report ARM TR-006, U.S. Department of Energy, Washington,
D.C.