Data Assimilation- A03. A Toy Model Shunji Kotsuki
Environmental Prediction Science Laboratory
Center for Environmental Remote Sensing (CEReS), Chiba University(shunji.kotsuki@chiba-u.jp)

DA Lectures A (Basic Course)‣ (1) Introduction and NWP‣ (2) Deterministic Chaos and Lorenz-96 model‣ (3) A toy model and Bayesian estimation‣ (4) Kalman Filter (KF)
‣ (5) 3D Variational Method (3DVAR)‣ (6) Ensemble Kalman Filter (PO method)‣ (7) Serial Ens. Square Root Filter (Serial EnSRF)‣ (8) Local Ens. Transform Kalman Filter (LETKF)‣ (9) Innovation Statistics‣ (10) Adaptive Inflations‣ (11) 4D Variational Method (4DVAR)Today’s Goal
‣ To understand minimum variance estimation‣ To understand maximum likelihood estimation‣ To understand assumptions in these estimations‣ To understand Bayesian estimation

Data Assimilation- a toy model -Two Major Streams of DA‣ Minimum variance estimation‣Kalman filter (KF)
‣ensemble Kalman filter (EnKF)‣ Maximum likelihood estimation‣3D variational (3DVAR)‣4D variational (4DVAR)‣particle filter (PF)ଵ =20℃

A simple example: two thermometersଶ = 30℃(less reliable) (more reliable)

Minimum Variance
Estimation(最小分散推定)Minimum Variance Estimation: truth: random error

forecast
observation<・>: expectation

Assumption (1) : unbiased error

Assumption (2) : uncorrelated error

Minimum Variance Estimation
forecast(1) unbias

observation(2) uncorr.
& minimize variance of analysis (a)definition of variance: standard deviation: varianceMinimum Variance Estimation
forecast(1) unbias

observation(2) uncorr.

weighted average by variance (σ2; =accuracy)

first guessincrement

Minimum Variance Estimation
Kalman filter
Predicted state
estimate

M( ): nonlinear model
M: Tangent Linear model)

H( ): nonlinear obs. operator
H: Jacobian of H()

Predicted estimate
covariance

P: error covariance

OptimalKalman gain

Q: model error covariance

Update estimate
covariance

R: obs. error covariance

Update state
estimate

K: Kalman gain

b: backgroundo: observation
a: analysis
analysis equation

first guessincrementMaximum Likelihood
Estimation(最尤推定)

Maximum Likelihood Estimation
forecast(1) unbias

observation(2) uncorr.
Likelihood

Prior (uniform, i.e., no prior info)

Bayesian Estimates
Posteriorconstant (since they are given)to maximize likelihood

Maximum Likelihood Estimation
forecast(1) unbias

observation(2) uncorr.

Suppose x1 & x2 follow
Gaussian PDF N(x, σ)Maximum Likelihood Estimation
forecast(1) unbias

observation(2) uncorr.

analysis of maximum likelihood estimatesSummary
weighted average by variance (σ2; =accuracy)(1) unbias(2) uncorr.
minimum variance estimates maximum likelihood estimates(3) Gaussian error PDF

Summary

Extension to multi-dims problems
The two thermometers’ example

weighted average஺ =20℃

A simple example: two thermometers஻ = 30℃(less reliable) (more reliable)

Extension to multi-dims problems
The two thermometers’ example

weighted average
Uncertainty (reliability) Uncertainty (reliability)of model forecastsof observations

x: state
P: state error covariancey: observation R: obs. error covariance
H: obs. operator

b: backgrounda: analysis
o: observation

Kalman Filter
Prediction (state))

Prediction (error covariance)

Prior Error Cov. Pb(flow-dependent!)

Kalman gain

Analysis (state)

Analysis (error covariance)
obs error cov: RSummary‣ Minimum variance estimation suppose‣ unbias‣ uncorrelated error‣ Maximum likelihood estimation suppose‣ unbias‣ uncorrelated error‣ Gaussian error PDF‣ These solutions are identical‣ when errors are Gaussian‣ Namely, minimum variance estimation gives optimal
analysis following Bayesian theory w/ Gaussian errors‣ (細かいが大事) 最小分散推定 (KF & EnKF)は、誤差のガウス分布性を仮定しない。我々が信頼するのは最尤推定で、最小分散推定と最尤推定は、誤差がガウス分布の時に一致する。
だから、誤差のガウス分布性はKF & EnKFにも望ましいのだ。

何故、観測を真としないか?
‣ データ同化のコンセプト‣ 観測よりも真値に近い値を求まる‣ よくある疑問‣ 観測ってかなり精度が高いハズ‣ なんで観測にも誤差があると考えるのか?

‣ (1) 劣決定問題 (観測 << モデルの変数)

e.g. 水位観測‣ 一意に解が求まらない‣  L2正則化で (a) 解を一意に、(b) 自信過剰の抑制‣ L2正則化と、「観測も誤差がある」は、数学的に同値‣ 伊藤・藤井 (2020; ながれ)‣ (2) 観測誤差 ≠ 計測誤差

‣ (a) 計測誤差: あらゆる計測は誤差を含む‣ (b) モデル表現性誤差: 自然現象の非完全な数値モデル化‣ (c) 代表性誤差: 或る観測の空間代表制‣ データ同化は数値モデルの予測が改善することを目的とする

Bayesian Estimation

Bayesian Theorem
Posterior

Likelihood

Prior

Obs

Whole EventP(A)P(B)

Bayesian Theorem: an example
An example: virus infection (e.g. COVID19) and inspection
・virus rate is 0.005 (0.5 %)・inspection to people with virus  gives positive (+) w/ 80 %・inspection to healthy people  gives negative (-) w/ 90 %Now, you have a positive result by the inspection!!!
 The percentage of having virus is only about 3.9 %.

Interpretation by cases
Total
10000

Virus Rate50(virus)
9950(healthy)

Jointx0.8 = 40x0.2 = 10x0.9 = 8955
x0.1 = 995

Inspection Result
Positive (correct)
Negative (incorrect)
Negative (correct)
Positive (incorrect)

Intuitive Interpretation

w/o virus(9950)

incorrect positive (偽陽性) 10%  995

w/ virus(50)

correct positive (陽性) 80%  40

Bayesian Theorem: an example
An example: virus infection (e.g. COVID19) and inspection
・virus rate is 0.005 (0.5 %)・inspection to people with virus  gives positive (+) w/ 80 %・inspection to healthy people  gives negative (-) w/ 90 %Now, you have a positive result by the inspection!!!
 The percentage of having virus is only about 3.9 %.
Posterior

Likelihood

PriorObs

Prior: Prob. of virus
Obs: Prob. of positive
Likelihood: Prob. of positive given virus
Posterior: Prob. of virus given positive

Then,,,, so what?
Bayesian Theorem: forward: backward (結果原因)

陰性(negative)

backward causality !!!!

陽性(positive)

Bayesian Estimation
Bayesian Theorem (discrete)x
uniform distribution

Bayesian Estimation
Bayesian Theorem (discrete)x
uniform distribution

We would like to findthat maximizes

Bayesian Estimation
Bayesian Theorem (discrete)

Bayesian Theorem (general)

Posterior

We would like to findthat maximizes

Likelihood

Prior (uniform)
constant(i.e., not afunc. of x)

Maximum Likelihood Estimation
forecast(1) unbias

observation(2) uncorr.
Likelihood

Prior (uniform, i.e., no prior info)

Bayesian Estimates
Posteriorconstant (since they are given)to maximize likelihood

Recommendations (Jpn)

Thank you for your attention!
Presented by Shunji Kotsuki(shunji.kotsuki@chiba-u.jp)
Further information is available athttps://kotsuki-lab.com/