Analysing the 21cm signal
with Artificial Neural Networks
Hayato Shimabukuro [島袋隼士] (Observatoire de Paris)
Based on Shimabukuro & Semelin(astro-ph/1701.07026, Accepted in MNRAS)

1Contents• Introduction• Cosmological 21cm signal• Artificial neural networks (ANN)• Result• Summary2Introduction

3

Who am I?
•Okinawa-Sendai-Nagoya-Kumamoto-Paris•Ph.D at Nagoya university (03/2016)•Post-doc at Observatoire de Paris with Benoit Semelin (04/2016~)

Meridian(子午線)Introduction○z >30・・Dark age：Star formation has not started yet.
○15<z<30・・Cosmic dawn：After star formation, astrophysical effects
become effective.
○7<z<15・・Epoch of Reionization(EoR)：Hydrogen is ionized by UVphotons emitted from stars,galaxies.

5

Epoch of Reionization

Blue：Neutral IGM

Red：HII region

Yellow：Ionizing source (galaxy)Pre-overlap : HII regions grow in relative isolation
Overlap : Once galaxies become common, HII regions intersect.
Post-overlap : Ionising of IGM advances sufficiently.

Observational constraints on EoR† The polarisation of CMB photons by free electron.

zr = 9.9+1.8
1.6(Assuming instantaneous reionization model)(Planck collaboration 2015)†High-z QSO absorption spectra

xHI >⇠ 10 4 (z > 6)

(Put constraints on neutral fraction)(Fan et al 2006)† The luminosity function of high-z LAE (Lyman-alpha emitter galaxies)

xHI = 0.3
(Konno et al 2014)

0.8 (z = 7.3)

(Put constraints on neutral fraction)

Beyond current constraints
Current observations can constrain state of the IGM at late stage of the EoR.
We would like to observe the higher redshift universe more directly !
21cm line signal from neutral hydrogen in the IGM
Simulated 21cm map

z=12.1

Neutral(Furlanetto & Briggs 2004)

z=9.2

z=7.6

IonizedCosmological 21cm signal

921cm signal○21cm line radiation : Neutral hydrogen emits the radiation due to the hyper
fine structure.

Spin temperature

n1= 3 expn0✓h⌫21kTS◆○Collision with CMB photons○Collision with Ly-α photons○Collision with atoms
10Brightness temperature
We actually observe brightness temperature instead of spin temperature○Brightness temperature

Red：astrophysics.
Blue：cosmology

Pritchard & Loeb (2011)1121cm power spectrum
We often evaluate statistical property of the 21cm fluctuations by power spectrum.

21cm power spectrum (PS)00h Tb (k) Tb (k )i = (2⇡)3 (k + k )P21

Scale dependence

Redshift dependence

EoR

Pober et al (2014)

WF
effectX-ray
heatingz

Observations
Some on-going telescopes have started observation (MWA, LOFAR,PAPER).
Future observations are now planning (SKA,HERA) on 2020’s.

MWA (Australia)

LOFAR (Netherland)

PAPER(South Africa)

Commercial Message
Collaboration with SKA-JAPAN has started (Kumamoto, Nagoya, ICRR).
Cross-correlation between 21cm - LAE (Kubota et al, in preparation)
13

Upper limits on 21cm PS
z=8.4 (Ali et al 2015)

Current upper limit by PAPER•1-2 magnitude of order higher thantheoretical prediction.
•But, we believe 21cm signal isdetectable in near future.
After we actually detect 21cm signal,
How do we extract astrophysicalinformation from 21cm PS ?

Theoretical prediction

Artificial neural network

Artificial neural networks

15Motivation•We would like to extract the EoR information from 21cm PS.

•To determine EoR parameters helps this purpose.

•How precisely can we determine EoR model parameters?
(ex.MCMC, Greig et al 2015).

•We train neural network architecture to learn association
between 21cm PS and EoR parameters.

•Once we train ANN architecture, we can apply this tounknown data.

16

What is artificial neural network (ANN)?
ANN is one of the methods inspired by brain neural network which
is used to establish approximate function between input andoutput data.
STEP
1.Prepare known data set (training data)(~xdata , ~ydata )2. Train architecture of neural network by training data.

y = f (x)3. Apply trained network to unknown (test data) and can obtain expected
output data.

yANN = f (xtest )17Flowchart
Training set①
Machine learning algorithm②③
Test data

Trained neural network④

Predicted output

18

What is ANN?
Neurons

Hidden layer

x1x2

Linear partsi =x3nX
j=1

wij xjActivatetiti = (si )xnNeuronwij :Weight:Activation function
1.At input layer, we first calculate linear combination of input data with weightand transfer them to hidden layer.
2.Next, we activate that linear combination with activation function.
19

What is ANN?
Neuronst1
t2

Output layer
Linearyi =tkkXyi

wij tjj=1

Neuronwij :weight between hidden layer
and output layer

At output layer, we consider linear combination of neurons at hidden layer.
Unlike hidden layer, we do not need to activate linear combination at outputlayer.
20

What is ANN?
We have to determine the weight to construct architecture of neural network.

How?
AnswerWe adjust the weight to satisfy minimizing the cost function E which isdifference between true value and output value. This procedure is called“Training”

cost functionNX
1
E=(ydata,i2 iyANN )2

N: the number of training datasets

We use “Back propagation algorithm” to determine weight. (Rumelhart etal 1986)
21

Back-propagation algorithm
We update the weights by gradient descentof cost function until they converge.

w(t + 1) = w(t) +Forwardw(t)with(l)wij =⌘@E(l)@wij=⌘NXtrain@En(l)n=1 @wij

Back-forward
We calculate the derivative of cost function starting from output
layer toward input layer.
22

Dataset~d~ = [P (k), ✓]✓EoR = f (P21 )EoR parameter (output)

21cm power spectrum (input) (0.04Mpc 1  k  1.4Mpc 1 )(14 bins)
EoR Parameter⇣: the ionizing efficiency.

Tvir : the minimum viral temperature of halos producing ionizing
photons

Rmfp : the mean free path of ionizing photons through the IGM(Maximum HII bubble size)
23Condition•

We used 70 training datasets and 50 test datasets.

•We calculate the 21cm PS by 21cmFAST•

We choose EoR parameters and 21cm PS datasets atz=9,12 for single z (and z=9,10,11 for multiple z).

•We train network with/without thermal noise and cosmicvariance.

•N_{input}=14,N_{hidden}=14,N_{output}=3 for single z.

24Result25

ConvergenceWe perform

10

6 iterations for back-propagation algorithm to see convergence.

0.5

vuu
RMSE = t

Rmfp
ζTvir

0.4

RMSE

0.3

0.2

1
NtrainX = (✓ANNNXtrain

X2i=1✓data )/✓data0.1(θ is each EoR parameter.)
00100000

200000300000400000

5000006000007000008000009000001x106

Iteration

More than 200000 iterations, all values of RMSE converge.
26

The number of neurons
0.1

Rmfp
ζTvir

0.08

•Fix 100000 iterations• Change the number ofneurons at hidden layer.

RMSE

0.06

0.04

•Plot RMSE as function of
the number of neurons.

0.02

0810

12

14161820

The number of neurons(Note)
14 neurons at input layer.

Week dependence on the number ofneurons.
27

Results
z=12,PS without any noise
14 neurons, 100000 iterations
6060

Rmfp

403020403020

10

z=1210203040
Rmfp,true[Mpc]5060

z=12102030405060• True value .vs. Reconstructed valueTvir•The scatter of R_mfp is large.

10110
ζtrue

100Tvir,ANN[K/103]

⇣50

ζANN

Rmfp,ANN[Mpc]

50

z=121

10Tvir,true[K/103]

100•Other reconstructed parameters matchtrue one relatively well.
28

Results
z=9,PS without any noise

14 neurons, 100000 iterations

6060

Rmfp

40
30
20403020

10z=9102030

40
Rmfp,true[Mpc]5060

10z=9102030

405060ζtrue

100Tvir,ANN[K/103]⇣50
ζANNRmfp,ANN[Mpc]50•Compared with z=12, reconstructed
R_{mfp} match true value better.

Tvir
101z=91

10Tvir,true[K/103]

100•Because R_{mfp} expresses maximum
size of HII bubble, it affects lower redshift
when reionization advances.

Noise & Cosmic variance
In order to evaluate the effect of noise, we include both thermal noise andcosmic variance for test data.
However,
We include only cosmic variance for training data.
How?
•We calculate comic variance from 10 realizations of initial condition.
•Produce 50 noise realizations for each parameter set by adding randomgaussian noise .
We calculate thermal noise assuming SKA observation based on (Morales et al2005).
30

Results
z=9, PS including thermal noise and cosmic variance

Rmfp

⇣ζANN

Rmfp,ANN[Mpc]

14 neurons, 100000 iterations10
z=9,with noise
z=9,without noise

1010

z=9,with noise
z=9,without noise

Red :with noiseBlue :
without noise

10

Rmfp,true[Mpc]ζtrue

Tvir,ANN[K/103]

100

Tvir
10

1

z=9,with noise
z=9,without noise
1

10Tvir,true[K/103]

100

Accuracy is better when noises are notincluded.
31

Results
60

60

505040ζANN

Rmfp,ANN[Mpc]z=9, 10, 11. PS including thermal noise and cosmic variance

3020Rmfp

10102030405040
30
2060

10

⇣
10Rmfp,true[Mpc]
Tvir,ANN[K/103]

30

40

5060

ζtrue

42 neurons,100000 iterations

100

Red : z=9,10,11
Blue : z=910

1

20

Tvir
1103Tvir,true[K/10 ]100

Visually, it is difficult to see whether theaccuracy of parameter estimation isimproved by taking redshift evolution intoaccount.
32

Results
single zmultiple z

Reduce the number of training datasets (N=20).
Estimate of the EoR parameter is improved by•Taking redshift evolution into account.
• Increasing the number of training datasets
33

Compare with MCMC
ANN

MCMC

•For comparison with MCMC, we also show 1sigma error obtained by MCMC.
•We compare the RMSE obtained by ANN with 1 sigma error obtained by
MCMC in the both case of HERA and SKA observation.
• EoR parameters obtained by ANN have similar error level to those by MCMC.
34

Summary•

Artificial neural network (ANN) is one of the machine learningtechniques based on brain architecture model.

•We applied the ANN to 21cm signal in order to extract EoR
information.

•EoR parameters produced by ANN were good agreement with truevalues.

•Multiple redshift data improved accuracy.

•We’re now trying to analyse the epoch of cosmic dawn with moreparameters (with A.Fialkov).

35

BackupThermal history
Mesinger et al 2010

Blue：CMB temperature/ (1 + z)
Green：kinetic temperature/ (1 + z)2
Red : spin temperature

37

Thermal history
Mesinger et al 2010

X-ray heating
collision coupling

collision decoupling
WF eﬀectSpin temperature couples to kinetic temperature via Ly-αphotons.
38Constraints
The observation of the CMB polarization→the optical depth of Thomson scattering

Optical depth

Z 0dt⌧e /ne (z) dzdzzr⌧e = 0.078 ± 0.0019
zr = 9.9+1.8
1.6(Planck collaboration 2015)

Constraints
high-z QSO absorption→ constraint on the epoch where the EoR ﬁnishes.
Gun-Peterson test
If the neutral hydrogen exists, it
absorbs the Ly-alpha photons.
No emission line !

We can know the epoch where EoR
ﬁnished via the QSO spectrum.

Sensitivity
Sensitivity

Results6060505040

ζANN

Rmfp,ANN[Mpc]

Reduce the number of training data. (N=20)3020102030

4050

10Tvir,ANN[K/103]

10z=9,10,11,Ntrain=20
10

32030

ζtrue

100

1z=9,10,11,Ntrain=20

1060

Rmfp,true[Mpc]130
20z=9.10,11,Ntrain=20

1040

100

Tvir,true[K/10 ]
4240

5060

MCMC

The probability distribution function of each parameter.

Activation function