# Presentation: 268038

## Enhanced AIRNow Data CollectionUsing XML and “Internet” Technologies

- Jeff Smith
- Air Quality Monitoring Manager
- State of Oregon
- Department of Environmental Quality
- &
- Andrew C. Montz
- Owner
- DR DAS

## Background
Motivation for Change
Specifications/Requirements
Solution
Implementation Experiences
Costs
Future Plans

**Motivation for Change**
**Specifications/Requirements**
**Solution**
**Implementation Experiences**
**Costs**
**Future Plans**

## Background - Old PC & Datalogger System

- Personal Computer as central host
- Dataloggers at all sites
- Modem Dial-up access, pulling data
- Not real time, data retrieved twice a day
- Sites accessed serially, one at a time
- DOS based, in service since 1991
- No Web support

- Datalogger

- Modem

- Modem

- Host PC

- Phone line

- Monitor

- Recorder

## Real-Time Monitoring Stations in Oregon

-  35- 45 Sites
-  ~150 parameters

## Motivation for Change

- Wildfires
- Big fires is 2001-2003
- Big demand for data, internal and external
- Cooperative monitoring projects with USFS
- 9 station Forest Health Network

## Motivation for Change

- Air Quality Index
- Hourly Updates
- Local Media
- Clean Air Action Days
- AirNOW Program
- Need for Real-Time data
- National mapping and forecasting
- EPA Funding

## Motivation for Change

- Public Access to Data
- Public Education
- Health Advisories
- Visible Product (tough budget times)
- Internet Widely Available
- Means to provide the data
- Age of existing system

## Specifications/Requirements

- Push data to Central Database
- Data delivered from all sites at top of each hour
- Send 5 minute & hourly averages, calibration data
- Work in rural/remote areas
- Work with current dataloggers
- Use modern database (MS SQL Server)
- Easy to get data to Web pages
- Allow continued remote access to samplers
- Low cost (both to purchase and operate)

## Solution

- Use Internet to transmit data
- Oregon On-Line Services for ISP
- Local access points, higher speed
- XML Pusher software from DRDAS on local PC
- Custom driver for Odessa dataloggers
- Pushes data, calibration results, IP address
- Synchronizes time with central server
- Pass-through feature to access DL directly
- ENVIEW database for central storage (DRDAS)
- Running on MS SQL Server
- Already available of agency server
- Use existing IT resources for Web development

## Solution

- New Data System consists of 3 components
- Remote Station
- XML Pusher running on PC
- Collects data from existing datalogger
- Local access to Internet using modem
- DMZ Computer
- Located outside agency firewall
- Receives XML data packets from remote systems
- Transfers data to database server
- Central Database Server
- ENVIEW database running on SQL Server
- Behind Agency Firewall

## Block Diagram

- Datalogger

- XML Pusher PC

- DMZ  PC hosting IIS & the XML template files

- ISP

- Modem

- Internet

- Outer Firewall

- Inner Firewall

- Local Phone Access

- Database Server with MS SQL Server & ENVIEW database

- Monitor

- Remote Station

- Oregon On-Line Services

- Recorder

## Remote Station

- Surplus PC running Windows 2000/XP
- IIS, MSDE, and XML for SQL Tools installed
- IIS = Internet Info Services, MSDE = SQL Desktop Engine
- Internal modem (56K)
- XML Pusher software
- Configuration Utility (web based – .asp)
- Data display software (web based –.asp)
- Remote control software (VNC)
- UPS for power conditioning and interrupts
- System configured to recover for power interrupt

## DMZ Computer

- Surplus PC running Windows 2000/XP
- IIS and XML for SQL Tools installed
- Fixed IP address, outside Agency firewall
- Data from remote stations posted to specific web page on DMZ PC
- Templates to map data from remote stations into SQL database on server inside firewall
- Template for each type of data
- Using stored procedures in ENVIEW database

## Central Database Server

- Existing Agency server running MS SQL Server 2000
- ENVIEW database copied to server and attached
- Tables in place for data, calibration results, and IP addresses
- Stored procedures for data uploads
- Database user needs Read/Write/Execute permissions

## Block Diagram

- Datalogger

- XML Pusher PC

- DMZ  PC hosting IIS & the XML template files

- ISP

- Modem

- Internet

- Outer Firewall

- Inner Firewall

- Local Phone Access

- Database Server with MS SQL Server & ENVIEW database

- Monitor

- Remote Station

- Oregon On-Line Services

- Recorder

## Implementation - Testing New System

- 3 Phases of Testing
- Basic system operating within Lab network
- No dial-up, ISP
- No firewall
- Single test station using dial-up and ISP
- Using outside phone line
- DMZ PC outside agency firewall
- Deployed at 3 real stations in Portland area
- Real-world test
- Provide real data for Web and software development

## Testing New System

- Basic system running on internal Lab network
- Software installed on remote PC
- DMZ PC setup
- Loaded ENVIEW database on local server
- Had little trouble getting this system to work
- Took a couple of software updates to fix bugs
- SQL Server needs to be using TCPIP protocol
- Issues with oldest dataloggers
- Timeout problems caused by power settings
- Data transfer very quick

## Testing New System

- Test station running on external (ISP) network through Firewall
- Installed internal modem
- Subscribed to Internet Service Provider (ISP)
- Using Oregon State On-Line Services
- Local access numbers for all sites
- Central trouble shooting and administration
- Configured Windows Dial-up Networking
- Using inactivity setting to end connection
- Moved DMZ PC to Agency HQ and within Firewall
- First DMZ PC implemented by Agency
- Initial steep learning curve for IT staff

## Had serious problems with Firewall and access to database server
DMZ PC could not talk to database server at Lab
Firewall would not allow connection
Considered getting outside help (Cisco)
Moved ENVIEW database to HQ server as test
Connection to database now worked
Blessing in disguise
Hardware/software maintenance done by others
Easier access to Agency Web servers

- DMZ PC could not talk to database server at Lab
- Firewall would not allow connection
- Considered getting outside help (Cisco)
- Moved ENVIEW database to HQ server as test
- Connection to database now worked
- Blessing in disguise
- Hardware/software maintenance done by others
- Easier access to Agency Web servers

_**Testing New System**_

## Testing New System

- Still had lots of problems transferring data
- Lots of timeouts
- Odd pattern of failure
- Transfer would eventually complete, but initial attempts would always fail
- Discovered problem to be in “route” tables in outer firewall, too many entries
- Discovered phone billing problem
- Unexpected large phone bills
- Older accounts set up for “metered” service

## Deploying systems to real sites in Network

- Test station working well
- Added more user accounts with ISP
- Installed at 3 sites for final testing
- Cloning PCs to simplify deployment
- Space issues at most sites
- Not arranged for a computer system
- Other considerations
- At the mercy of phone companies and ISP
- Central server operational schedule
- Late night backups caused problems
- Shutdowns for upgrades and maintenance
- Now installed at 30 sites

## Software Development

- Utility to transfer raw data to older system for processing & data management
- New data uploaded to old system every morning
- All editing, validation, and reporting done with old system
- AQI calculations and Web displays updated every hour from new system
- AirNOW uploads every hour from new system
- Custom data displays for agency staff
- 5 minute data and graphs
- Calibration results, IP addresses

## Data and AQI Display

## Costs

| Item | Cost |
| --- | --- |
| Hardware
	PCs for remote sites (12 new)
	Flat panel displays (24)
	UPS, Modems, cables | 
$12,000
$6,000
$4,800 |
| Software - XML Pusher from DRDAS | $7,500 |
| Internet Access – 40 sites (per year)
	Oregon On-Line Services | 
$8,000 |
| Web Pages & Data Utility Development
	Using internal Lab and IT staff | 
$32,000 |
| Total | $70,300 |

## Communications Costs

- Comparison of phone line costs per year
- Old system dialing twice a day, no internet
- Total = ~$1,500
- Old system dialing every hour
- 1 minute per call, $0.06 per minute
- 40 sites, 30 are long distance calls
- Total = ~ $16,000
- New system pushing data every hour
- 40 sites
- All local calls
- ISP charges = $200/year
- Total = $8,000

## Future Plans

- Retire old data system
- Do all data management with new database
- Upgrade from dial-up to high speed internet
- Still not available in most remote locations
- More expensive
- Would allow transfer of current 5 minute data
- Transfer other data from sites
- FRM PM2.5 data records
- Site and instrument records, photos
- Transfer data directly to AirNOW

## Replace old dataloggers
Serial data interfaces
Rethink monitoring station layout
Centered around computer

- Serial data interfaces
- Rethink monitoring station layout
- Centered around computer

_**Future Plans**_

## Questions

- http://www.deq.state.or.us/aq