5. Ground Data System
5.1 JEM/SMILES products
5.2 Operational algorithm andperformance
5.3 System design
5.4 Development schedule
5.5 Summary

JEM/SMILES data sets
Data Type

JEM/SMILES products

Description
Unprocessed mission data at binary packets

Level 0Reconstructed, unprocessed mission data atbinary packets

Level 1b

Calibrated instrument radiances and related
data

Level 2Derived geophysical variables at the same
resolution and location as the Level 1
source data

Level 3

Variables mapped on uniform space-time
grid scales, usually with some completenessand consistency

ProcessingRAWJEM/SMILES products

Products of level 2 data
■Standard Products :- Routine processing- Nonroutine Processing:O3, HCl, ClO, CH3CN, O3 isotopes, HOCl, HNO3:HO2, BrO■ Research Products : (These products are outside of this system)
volcanic SO2, H2O2, UTH, Cirrus Clouds

TypeBand A

Band B

Band CSpecies
retrieved fromsingle-scan data

O3
H37Cl
18OOO
HNO3
CH3CN
HOCl
O17OO

O3
H35Cl
18OOO
O17OO
ClO

O3
18OOO
HNO3Species
retrieved frommulti-scan data(noisy products)

BrO

HO2BrO
HO2

Effects from Other Molecules

JEM/SMILES products

JEM/SMILES data flow
JAXA/TKSC
L1B Data Server

UOA
EOS

SMILES

L1B

Raw data

DPS-L0/L1

JAXA/ISAS

L2 Data ServerUsers
InternetOff-line
InternetDPS-L2
L2DPS-L2

Inversion
Control

UOA: User Operation Area,
EOS: Experiment Operations System,
DPS: Data Processing System

Data transfer

L1B

Downlinked raw data from the
SMILES will be received by the
DPS-L0/L1 at User Operation Area
(UOA) on Tsukuba Space Center(TKSC).
The DPS-L0/L1 processes the raw
data consisting of house keeping(HK) data and mission data tobrightness temperature (level 1Bdata) in near-real-time.
The DPS-L2 produces the verticalprofiles of target species called level2 data in near real time anddistributes the level 2 data to data
users by a Web server.

Data Handling&Visualization

Forward Model
Iteration
L2 Data Processing

L2

Operational algorithm and performance

Level 2 data processing system•DPS-L2
The DPS-L2 produces distributions of the target species concentrations fromcalibrated spectra in near-real-time. The calculation process consists of a forward
model process, which computes the simulated spectra, and an inverse modelprocess, which deduces an atmospheric state.

•

Requirements for the DPS-L2– Accuracy• Forward Model Parameter Error caused by the approximations< 0.01K（1% of the measurement error）– Processing speed• Near real time processing （1 scan: 53 s）– Portability• Work on a general linux system
SystemSoftware

Operating System

Linux (kernel: up to 2.6)

Languagegcc (or Intel C++), Ruby, GNU Octave, java

DatabaseMySQL

Web Application

GFDnavi, ruby on rails

Forward model -flow

Operational algorithm and performance
Function
L1B Data

Tilt of ISS
Antenna pattern(2D)Make 1D Antenna Pattern

Tangent Alt.

Geometry Info.

Ray Tracing
Absorption Coef.

Measured Data
Altitude offset
ProfilesSpectroscopy Data
Frequency Grids

Antenna pat. 1D

Radiative Transfer
FOV Convolution
Doppler shift

Inclination of scan axis
Wind
LOS velocity
AOPT Temperature

Fold Sidebands
Output items:- Spectrum- Weighting function (WF)

Channel Average

Sideband Separator
AOS channels
AOS response func.

Forward model is one of the most important part todetermine the accuracy and its computing costOperational algorithm and performance

Doppler shift & Instrumental functions
Doppler shift・ Velocity of the ISS
・ Rotation of the earth・ WindFOV Convolution

Effects of wind

: 8 km/s: 460 m/s (on the equator): ‫د‬100 m/s.

Fold Sidebands (Single Sideband Separator)

Effects of tilt of antenna scan axis

Channel Average (AOS)Operational algorithm and performance

Fast algorithm

Selection of Absorption Lines• The effect of the line selection. The incremental
retrieved error by this line selection (Green line)
is less than 0.002% in the stratosphere. It isenough small comparing the retrieved precision(Red line).
Selection of Frequency Grids30km

0.002%

60km
・ 0.1MHz equally-spacedfrequency grids (12000 points)

・ Optimized frequency gridsinterpolated cubic spline (~900points)

・ Residual ( 1 – 2 )Operational algorithm and performance

Inversion model•

Retrieval Algorithm: Optimal estimation method (OEM)– Levenberg-Marquardt Method{x i +1 = x i + (S a−1 + K iTS −y1K i + γS a ) −1 K iTS −y1 [y − F(x i )] − S a−1 [x i − x a ]•}Inversion model is implemented by GNU Octave– GNU Octave: High-level language, primarily intended for numericalcomputations. (Mostly compatible with Matlab)– Link ATLAS to speed up for matrices calculation.
– Matrix size
• n : about 200~300(number of elements for a state vector)• m: about 1500*70(number of elements for a measurement vector)• Use n-form of Levenberg-Marquardt Method for calculation speed.

Operational algorithm and performance

Algorithm for noisy products•

To avoid the bias from a priori, we retrieve the multi-scan data simultaneously[Livesey,2004]. i.e. The observation data yi (i=1~N), the weighting function Ki,
the reference spectra fi, and the covariance matrix of the measurements Syi
are represented by :⎛ S y1⎛ K1 ⎞⎛ y1 ⎞⎛ f i1 ⎞⎜⎜⎟⎜ ⎟⎜ ⎟⎜ 0⎜ f2 ⎟⎜K2 ⎟⎜y2 ⎟=y = ⎜ ⎟, f = ⎜ ⎟, K = ⎜,Sy⎜ MMMM ⎟⎜⎜⎟⎜ ⎟⎜ ⎟
⎜y ⎟
⎜f ⎟
⎜K ⎟
⎜ 0
⎝ N⎠
⎝ N⎠
⎝ N⎠
⎝

•

0 ⎞
⎟
0M ⎟O 0 ⎟⎟0 S y1 ⎟⎠L0S y10L

To reduce the load of the system, we calculate below matrices and vectors,
KiTSyiKi and KiTSyi(yi-fi) foe each scan and save, because the size of thesematrices and vectors are small.

[][]xˆ = a + S a−1 + ∑ K Ti S −yi1 K i ⋅ ∑ K Ti S −yi1 [y i − f i (x 0 , b)] − K Ti S −yi1 K i [a − x 0 ]∑ K S K = (K S K + K S K + L + K S K )∑ K S [y − f (x , b)] = (K S [y − f (x , b)] + L + K S [y − f (x , b)])Ti−1yii

Ti−1yii

T1i0−1y1T
21T
1−1y2−1y1TN

2110−1yN

NTN−1yNNN0Reference：N J. Livesey and W. Van Snyder, EOS MLS Retrieval Processes Algorithm Theoretical Basis, 2004

Performance•

Operational algorithm and performance

Test of the processing time by our prototype program– Dual-Core Xeon 3.6 GHz (Use 1 core),– Intel C++ compiler– 64 bit Linux (Debian)■ Processing time for one scan data of band A*1• Forward Model (Spectrum and 11 WFs) *2: 46 s• Inversion: 10 sTotal (no iteration ): 56 s*1: Band A needs the longest processing time, but processing time of band B and band C
are about 30% less than that of band A.
*2: Spectrum and 11 WFs are calculated simultaneously to avoid overlaps.

System configuration

System design

Level2 Data Processing System in JAXA/ISASOperation Server

File Server
RAID5×2 (mirroring)

Web server

Users

Routine processing (include reprocessing):[Quad-Core Intel Xeon×2Core] × 4
MonitorNon-routine processing:[Quad-Core Intel Xeon×2Core] ×1

A Priori / first guess
ObjectO3
O3 isotopes

Reference data
LBLRTM*1, Aura/MLS*2, CCSR/NIES*3O3 references with isotopic abundances for HITRAN

Tracer constituentsHCl

LBLRTM*1, Aura/MLS*2, CCSR/NIES*3ClO

LBLRTM*1, Aura/MLS*2, CCSR/NIES*3CH3CN

UARS/MLS*4 (, Aura/MLS*2)

HOCl

LBLRTM*1, Aura/MLS*2, CCSR/NIES*3HNO3

LBLRTM*1, Aura/MLS*2, CCSR/NIES*3

HO2

Aura/MLS*2, CCSR/NIES*3

BrO

Aura/MLS*2, CCSR/NIES*3

Dynamics

Temperature
H 2O
WindPressure

System design

NASA/GMAO*5*1: Reference profiles used in Line-by-Line Radiative Transfer Model*2: Monthly climatology based on measurements by EOAS-Aura/MLS*3: Monthly climatology based on the CCMVal-REF2 run for 2001-2010 by CCSR/NIES CCM*4: CH3CN reference profiles based on measurements by UARS/MLS*5: Near-realtime analyses produced by NASA/GMAO’s GEOS-5 DAS

Level 2 data distribution system







System design

Level 2 Data Formatz
HDF-EOS (version 5)
Æ Compatible format with Aura/MLS
File components （size: 11MB/day)z
1 file : 1day data for each moleculezRetrieved value / Precision / Quality / Geolocation Info. / Time etc.
Level 2 data distribution systemz
Built on Gfdnavi (Geophysical fluid data navigator), which is based on Ruby on Rails.
Gfdnavi: http://www.gfd-dennou.org/arch/davis/gfdnavi/- a suite of software that facilitates databasing of geophysical fluid data, their analysis, and visualization
Ruby on Rails: http://www.rubyonrails.org/z
Implementation of this sub system has been finished.

Development ScheduleYear

DPS-L2Developm
ent

2007

2008

2009

2010

2011

2012

2013

Algorithm Development
Algorithm Development (revision)
DesignMaintenance / Development
Programming
System test
Computer installation / MaintenanceOperation

Operation
C/O▲launch / 2009.Summer

2014Summary• The launch ready operational algorithm has
been developed, and its performance meetsthe system requirements such as accuracy,processing speed and portability.
• The hardware system for the DPS-L2 iscompact and consists of five linux servers
with two CPU corresponding to Quad-Core
Xeon 3.6GHz.
• The system development is due to becompleted in 2009 early summer.