Regional modelling and forecasting using GNSS techniques.

Z.Bouya, M. Francis, M. Terkildsen
Australian Bureau of Meteorology, IPS Space Weather Services.

www.ips.gov.au

3rd AOSWA workshop , 2-5 March, Fukuoka, Japan.

Products-Flare Forecast-Flare Alert-Coronal Mass Ejection (CME)Warning(Automated Type II, III Detection)-Geomagnetic Storm Warning-Geomagnetic Storm Suddencommencement (SSC) and
Sudden Impulse (SI) Alert-Geomagnetic Storm Alert-Geomagnetically Induced Currents(GICs) in Pipelines-Polar Cap Absorption (PCA) Alert HF COMMS Warning-HF radio ‘Short Wave Fadeout’(SWF) Alert-Total Electron Content (TEC)
Ionospheric Modelling-Scintillation alert

Data for the Services: Space Weather monitoring network (IPS_attached.xls)

3rd party sensors
Magnetometers- U. Newcastle- ICSWSE-MAGDAS-Geoscience Aust
GPS-Geoscience AustSolar
- USAF Learmonth
Antarctica- AAD

Solar Radio SpectrographGeomagnetic

Raw data

Indices – K, A
processing

Dst storm index

Maps of K

HF radio communications

foF2

normaldepressed

SUBJ: IPS HF RADIO COMMUNICATIONS WARNING 10/21 ISSUED AT 23/2354Z
OCTOBER 2010 BY THE AUSTRALIAN SPACE FORECAST CENTRE. DEGRADED HF
PROPAGATION CONDITIONS EXPECTED FOR 24 OCTOBER 2010

Australian Space Forecast Centre (ASFC)
Solar

Geomagnetic

Ionosphere
27 day recurrenceGeomagnetically Induced Currents (GICs) in Pipelines
Threat Level Model from Global network observations[Marshall, Smith, Francis, Waters and Sciffer, Space Weather, 2011]

Geomagnetically Induced Currents (GICs) in Power Networks

GICy-index Occurrences > 50
300

200180160

GICy Index

140200120100

150

80
60100

4020501985

1990
HBTCNB

1995
GNATimeASP

2000

LRM02010

2005
CTAKDU

Sunspot

Sunspot Number250

Australian Region Estimated GIC-Index

Extreme Space Weather (ESW) Model-Most space weather impacts in Australian region associated with extreme events-Event-based analysis-Requirements:
LATENCY-‘Long range’ warning (> 12 hours)Based on solar data only-‘Short range’ alert (~ 1 hour)Based on solar data + ACE
ACCURACY-Long range: Optimise to minimise missed events-Short range: Optimise for forecast accuracy
SIMPLICITY-Design for active use in space weather forecast environment

ESW Model Events

Associated CME

Event- based analysis≥M1 X-ray flare specifies solar
disturbance event
Solar wind shock
IMF Bz south event
Major storm event

Flares / CMEs

solar wind(shocks)

IMF(Bz events)

Dst(storm events)
day of year

ESW Model Parameters
Model covariates (the ‘input data’)
X-RAY FLARE
Solar flare magnitude
Solar flare duration
LOCATION OF SOLAR ACTIVE REGION
Latitude of solar active regionLongitude of solar active region
CME CHARACTERISTICS

Presence of Halo CME (CME width)
CME speed

SOLAR CYCLE–Solar Cycle Phase

SOLAR WIND / IMF– IMF Bz
– Solar windshockESW Model Implementation

Regional Ionospheric specification and forecasting.

Objectives:
Develop an Australian Regional Ionospheric modeling capability-Develop/add value products and services for ionospheric specification and forecasting

IPS products and services are based on:-Regional ionospheric TEC modeling-Real time updating of the regional model: Kalman Filter-Scintillation-Ionospheric short time forecast-Australian regional Disturbance /Activity Index

Regional ionospheric modeling: Spherical Cap Harmonic Analysis (SCHA)
Spherical Harmonic Analysis (SHA)f (θ , λ ) = ∑ ∑ P (cos(θ )[g cos(mλ )K max Kk =0 m =0m+ h sin( mλ )]mkmkkk⇒(Haines,1985)

n k(m)-Mathematical formalism:
Extension of the global SHA.
-Difference between SCHA global SHA:
Basis functions-Wavelength /Resolution
Shorter in SCHA(De Santis et al.,1999)- Model coefficientsfor the same resolution, far lesscoefficients are used in SCHA

Comparison of # Global- and Cap SH coefficients

Nsha / Nscha

1000

100N

1010306090

cap size [deg]120150180

θ= sin (2SCHA02)NSHA

Mathematical formalism
Objective: Create harmonic expansion, using basis functions satisfying the boundary conditions
at edge of cap. ⇒ solving Boundary Value Problem (BVP)

Laplace equation in spherical coordinates:∂2∂∂2 ∂1∂21∆=( 2 +)+ 2(sin θ)+ 2 2r ∂r∂θr sin θ ∂θr sin θ ∂φ 2∂r
Laplace equation can always be separated into three ordinary equations:d  dR ( r ) = kR( r ),rdr dr ddΘ
m1(sin(θ ))−Θ = −λΘ,dθsin(θ ) dθsin (θ )222d Φ+ m Φ = 0.
dϕ222The solution to these differential equations is reduced to solving theeigenvalue/eignfunction problem of equations under boundary conditions.

SCH model for mapping the Australian Regional TEC
VTEC (θ c , λc ) =K max K[∑ ∑Pm) g km cos(mλc )nk ( m ) (cos(θ c⇓+ hkm sin(mλc )]k =0 m =0d=AXn k(m) : Non integer degree (θ 0 ≠ π )Basis functions ( geometry matrix)
0 ≤ λ ≤ 2π :Natural ⇒ eigenfunctions/ eigenvalue: Trigonometric/ integer.
0 ≤ θ ≤ θ 0 : Not natural ⇒ eigenfunctions/eigenvalue: Legendre functions/real.

mgk, hkm :(model coefficients, N=(Kmax +1)2)

Kmax: Maximum degree

Haines (1988)

K max =dw ww . i p s . g o v . a u

pr2θ 0 2π(+ 0.5) − 0.5π

λmin= Gpr

X

The cap and kmax determine theminimum wavelength of the model

SCHA model for the Australian Regional TEC:Benefits?
-Improved spatial resolution.
-Manageable number of coefficients / processing requirements.
-Overcome uneven data distribution.
- Scaling of cap size (“cap zooming”).

Regional TEC map on 02/01/2010 at 04:00UTModeling approach: Combined use of the Principal Component Analysis (PCA) andartificial Neural Network (NN)
Motivation for NN in forecasting – Nonlinearity!

-We want models that are as simple
as possible, but not any simpler.
-Fit the data, as well or better thanLinear Models (LM)s.
-Capture interesting dynamics-Good noise estimates.

The number J of PCs is chosen so as to optimize themodel’s performance.

PC_NN method for TEC Prediction: M PCs extracted, for each PC a NN model trained and next value ofeach PC predicted, finally predicted PCs and EOFs combined for the prediction of the main Map.

Modelling approach : Eignmode Decomposition-Database :z(t,x) Matrix of N Regional TEC maps at P : locations x = (θ , ϕ ) , P=IXJ spatial grids
ij-Spatial structures: Ej(θ,φ) Eigenvectors (Basis Function) (generated directly by themselves)(ranged in descending order according to the proportion of variance explained)-Their time evolution :A (t ) = z ' (t ,θ ,ϕ ) E (θ ,ϕ )Tjj(uncorrelated and carrying information about the variation of TEC along Ej)
The largest components: Capture the slow-varying trends-Seasonal variation and the solar dependence of the background TEC over Australia-Lower component: short-term variation-The meteorology of the regional TEC perturbation

Near-cyclic nature: Predictable "comletely".

-With decreasing variance PCs explainincreasingly complex features

Improve the understanding andprediction of the irregular part.

Irregular structures/ionospheric weather.
Unpredictale "Totally".

Explained variance and the accumulated explained variance for the TEC .

www.ips.gov.auModelling approach : Main Spatial and Temporal Variations:-The largest components

Significance: 69.7%
A1E1 mainly controls the intensity of theIonospheric electron concentration.

Significance: 16.3%
A2E2 : the correction of the main trend.

First two EOFs of the TEC for the year 2010, function of longitude and latitude
A1: Semiannual : 2 maxima(March/April and September/October).

A2: Annual variation

Time series of the first 2 EOFs coefficients shown above (A1, A2) (blue, pink) for the year 2010.

Forecasting approach: Three ionospheric time varying characteristics
Stage 1: 1 day ahead prediction
Eigenvalue decomposition of A1(UT,d): (first layer decomposition coefficient)N

Ak (UT , d ) = ∑ Fk j (UT ) PCkj (d )j(Base function)

F11 (UT )EOF coefficient: (Seasonal, solar cycle)

F21 (UT )+/ morning &evening -/midday(width of the daily peak )A very clear daily signal

The signal has the same shapeand magnitude for all the 4 years

How the daily signal varies through the year.

The signal has the same shapeand magnitude for all the 4 yearsModel construction: Modeling PCj=1,2
Harmonic functions representing: Seasonal ( annual and semi annual)
Solar activity : F10.7 ( 10.7-cm solar radio flux in units of W m-2Hz-1)-A bottom-up strategy: Starting point is a Linear Model-Non linear components are added (only if necessary)(Justify additional efforts necessary for a nonlinear modelcompared to a linear one)

Including geomagnetic indices as input to the neural network was investigated and it was found thatthe performance was not improved.

The tree time varying characteristic of the ionosphere. (daily, yearly and solar cycle)-Low frequency: Easy to predict (harmonics modulated by the solar activity)

Harmonic functions representing: Seasonal ( annual and semi annual)
Solar activity : F10.7 ( 10.7-cm solar radio flux in units of W m-2Hz-1)

EOF (dashed blue line) and Coefficient of second layer decomposition (P(d)) (solid blue line) andcorresponding predictions calculated by the model (solid red line).

Model construction: Stage2: Short term forecast (1hour ahead prediction)( under development)

Nonlinear adjustment

The proposed model predicts (outputs) the TEC at a certain timefrom the Geomagnetic avtivity and its history and the history ofthe TEC.

Higher order components: PCj>2-Present mainly short-term variation and noise-Less correlated with the primary variation in TEC.

-The complexity of the nonlinear part of the models isincreased step- by- step, only if contribute significantly tothe performance of the models.
The number J of PCs is chosen so as to optimize themodel’s performance.
The aim of such a procedure is to keep the models as simple as possible.
NN Improved linear models rather that pure non linear models.

Space Weather Indices and TEC variations: April 2010

Most influential factors
Factors

F10.7

dF10.7

AP

DstCorrelation

0.23

0.38

0.51

0.43

-Delayed perturbation acros
the region-Potential for predictionbasedon earlier space weather
observations

Comparison of daily Dst index (a), change in the geomagnetic field’s horizontal component at
Canberra (b), Ap index (c) and TEC perturbation (d) over the Australian region (April 2010).

Application of the approach (test cases): December 2006 Geomagnetic Storm
Development of the storm:-Solar flares and Sunspots associated with large CMEs
-Change in Dst index (SSCs)-Severity of the disturbed geomagnetic conditions (Ap Index)

Probable CME observed by SOHO satellite
Probable CME (UTC)Class

13 Dec 02:40

X3.4

13 Dec 14:23

C2.2

13 Dec 18:25

C1.7

14 Dec 12:10

C1.0

14 Dec 16:49

C1.2

14 Dec 22:15

X1.5
E3

- Rapid direction changes in the IMF Bz component and the Solar Wind .

Ap:

563104
Delayedperturbation
across the region

Potential forprediction

E12100181695141290108586(e)804275345346347348349

Days(2006)

Storm dynamics:-Three interrelated causal mechanisms: mechanical,electrical and chemical.
-Competition among electric fields as well as
thermospheric dynamics and composition changes.

Complex (Incomplete knowledge)

Simple(a) Solar wind speed, (b) interplanetary magnetic field Bz,(c) AE index, (d) Ap and Dst index and (e) ionosphericperturbation on 13–15 December 2006. The patches
indicate the initial phase of this storm.

Application of the approach (test cases): November 2004 Geomagnetic Storm
5-11 November 2004

E12E3

100%variance

995

790

585

380

1310311312313

314315316Days(2004)-The largest components(PC1 and PC2)
-Capture the slow-varying trends- Primarily deterministic-Lower component (PC3)- short-term variation
Comparison of daily Ap index (top), change in the geomagnetic
field’s horizontal component at Canberra (middle) and TEC
perturbation (bottom) over the Australian region.

The main contributions are:
-The developed SCHA methodology for mapping TEC is the first of its kind for the Australian region
and may be adapted for application in other regions and for other ionospheric parameters.
-We can widen its applicability from regional to local-By including harmonics with higher spatial index to investigate shorter wavelength features-The NN approach for the multicomponents modelling /prediction.
-Reducing the dimension of the prediction problem and the associated processing overhead.
-Gap Filling in Geophysical data.
Fact Sheet for Needs-Seeds Matching
Name of Organization: Australian Bureau of Meteorology , Space Weather Services.

SEEDSNEEDS

Data& Instrument See attached
OtherTraining CourcesHigh rate, near real time GPS and ISM data from
Data& Instrument PNG/Indonesia and South East Asia.
Ionosonde data from PNG and Indonesia .

Model

WANTS

Oblique ionospheric sounding (OIS).
Regional scintillation forecast model.

Data &Instrument Plasma bubles observations