AKARI IRC Data User Manualfor Post-Helium (Phase 3) Mission
Version 1.1
Takashi Onaka1, Rosario Lorente2, Yoshifusa Ita3, Youichi Ohyama4,
Toshihiko Tanabé1, and Chris Pearson5
with contributions from:

Martin Cohen6 , Daisuke Ishihara1 , Itsuki Sakon1 , Takehiko Wada7 ,Issei Yamamura7 , and Hideo Matsuhara7
1 The University of Tokyo, Japan
2 European Space Astronomy Centre (ESAC), ESA
3 National Astronomical Observatory, Japan (NAOJ)
4 Academia Sinica Institute of Astronomy & Astrophysics, Taiwan (ASIAA)
5 Rutherford Appleton Laboratory (RAL), U.K.
6 University of California, Berkeley, U.S.A.
7 Institute of Space and Astronautical Science (ISAS), JAXA, Japan

August 28, 2009

Date
30 January 20097 March 200924 August 2009

Revision
Version 0.9
Version 1.0
Version 1.1

Comments
copy from IDUM 1.4
Revisions completed
Table 4.6.4 corrected

Contents
1 Introduction
1.1 Purpose of this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Relevant information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
1
1

2 Instrument overview
2.1 Focal plane arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 Near-InfraRed Camera: NIR . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.2 Arrays operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Instrument AOTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 IRCZ0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 IRCZ2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3 IRCZ3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.4 IRCZ4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 In-orbit sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3
4
4
7
7
8
8
8
8
8

3 Distributed Data Products
3.1 IRC FITS ﬁle naming rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Raw data description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Raw data naming convention . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 AKARI FITS Primary HDU (common information) rawdata header . . .
3.2.3 Raw data dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Quick look data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 IRC imaging IRCZ0, IRCZ2 and IRCZ3 . . . . . . . . . . . . . . . . . . .
3.3.2 IRC spectroscopy: IRCZ4 . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 IRCZ4 image orientation and dispersion direction . . . . . . . . . . . . . . . . . .

11
11
11
11
12
15
16
16
16
17

4 IRC Calibration and Accuracy
19
4.1 Dark image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 Flatﬁeld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2.1 Flatﬁleds for imaging data . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2.2 Flats for spectroscopy images . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3 Instrument linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.4 Instrument Point Spread Function . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.5 RSRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.6 Flux calibration for point sources . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.6.1 Estimation of the in-band ﬂux . . . . . . . . . . . . . . . . . . . . . . . . 22
4.6.2 Absolute calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.6.3 Stability of the sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.7 Flux calibration for extended sources . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.8 Color correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24i

ii

IRC Data User Manual for Post-Helium Mission (Phase 3)
4.9 Pixel scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.10 Astrometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.11 Arrays anomalies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.12 General concerns on slit-less spectroscopy data . . . . . . . . . . . . . . . . . . .

28
28
28
28

5 Imaging toolkit cookbook
32
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2 General overview of the toolkit processing . . . . . . . . . . . . . . . . . . . . . . 32
5.3 Expected Data Processing Rate (minimum expectation) . . . . . . . . . . . . . . 33
5.4 How to install the IRC imaging Toolkit . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.2 Install IRAF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.3 Download IRC imaging data reduction software package . . . . . . . . . . 34
5.4.4 Unpack irc p3.tgz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.5 Make irc binaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.6 Run ”setpath.pl” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4.7 Perl path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.4.8 Add IRC P3 entry to IRAF . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.5 How to UPGRADE the version of IRC P3 imaging toolkit . . . . . . . . . . . . . 36
5.6 Setting up your toolkit environment and running the pipeline . . . . . . . . . . . 36
5.6.1 Creating the directory structure . . . . . . . . . . . . . . . . . . . . . . . 36
5.6.2 Launch IRAF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.6.3 Load the IRC P3 package . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.7 The pre-pipeline processor (Red-Box) . . . . . . . . . . . . . . . . . . . . . . . . 39
5.7.1 Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.7.2 Running the prepipeline processor . . . . . . . . . . . . . . . . . . . . . . 39
5.8 Before runnning the pipeline processor . . . . . . . . . . . . . . . . . . . . . . . . 42
5.9 The pipeline processor (Green-Box) . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.9.1 Conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.9.2 Running Green-Box pipeline module . . . . . . . . . . . . . . . . . . . . . 45
5.10 The pipeline processor (Blue-Box): bluebox.coadd wrapper Module . . . . . . . . 49
5.10.1 The Blue-Box Co-Add Wrapper . . . . . . . . . . . . . . . . . . . . . . . . 49
5.10.2 Log ﬁles produced from the toolkit . . . . . . . . . . . . . . . . . . . . . . 53
5.11 Toolkit structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.12 Working on the output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.13 Limitations of the functionalities in the current version of the imaging toolkit . . 55
5.13.1 Instrumental characteristics and artefacts in the data . . . . . . . . . . . 55
5.13.2 Toolkit Limitations and Caveats . . . . . . . . . . . . . . . . . . . . . . . 56
5.14 Error messages when running the pipeline and Troubleshooting . . . . . . . . . . 56
6 Spectroscopy pipeline cookbook
59
6.1 General overview of the pipeline processing . . . . . . . . . . . . . . . . . . . . . 59
6.1.1 Dark subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.1.2 Linearity correction/Saturation Masking . . . . . . . . . . . . . . . . . . . 60
6.1.3 (monochromatic) Flat-ﬁelding . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.1.4 Background subtraction (from individual sub-frames) . . . . . . . . . . . 61
6.1.5 Image screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.1.6 Image registration (among sub-frames) . . . . . . . . . . . . . . . . . . . . 61
6.1.7 Imaging stacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.1.8 Target detection/position measurement . . . . . . . . . . . . . . . . . . . 62

Version 1.1 (August 28, 2009)

6.2
6.3
6.4

6.5

6.6

6.7

6.8
6.9

6.1.9 Background subtraction (from stacked image) . . . . . . . . . . . . . . . .
6.1.10 Extracting 2D spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.11 Wavelength calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.12 Flat color-term correction . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.13 (local) Background subtraction (from extracted 2D spectra) . . . . . . . .
6.1.14 Spectral tilt correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.15 Spectral response calibration . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.16 Notes on slit spectroscopy . . . . . . . . . . . . . . . . . . . . . . . . . . .
How to install and to set-up the IRC spectroscopy pipeline . . . . . . . . . . . .
6.2.1 Data preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Calibration data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3.1 Calibration ﬁles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Running the pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.1 Data reduction order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.2 Running the pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.3 Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.4 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.5 Summary of interactive operations within the pipeline . . . . . . . . . . .
6.4.6 Warning messages of the pipeline . . . . . . . . . . . . . . . . . . . . . . .
Working on the pipeline output . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5.1 Displaying the whole images on ds9 . . . . . . . . . . . . . . . . . . . . .
6.5.2 Displaying the extracted images on ATV . . . . . . . . . . . . . . . . . .
6.5.3 Checking for wavelength zero reference point with the zero-th order lightimage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5.4 Spectral plotting tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5.5 Working on saved data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
(Hopefully) Useful Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.6.1 Getting best S/N spectra in slit (Ns, Nh) or Np spectroscopy data . . . . .
6.6.2 Tackling narrow spikes seen in NIR spectra (especially in slit spectra) . . .
6.6.3 Examining strange/fake features in NP spectra, especially around 2.4-3.5
μm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.6.4 Examining ﬂux level (in)consistency among diﬀerent disperser data . . . .
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.7.1 Variable name conventions . . . . . . . . . . . . . . . . . . . . . . . . . .
6.7.2 Summary of Commands and their Options . . . . . . . . . . . . . . . . .
Error messages when running the pipeline and Troubleshooting . . . . . . . . . .
Tips for the Phase 3 data reduction . . . . . . . . . . . . . . . . . . . . . . . . .
6.9.1 AOT IRCZ4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.9.2 Spectroscopic performance . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.9.3 Dark current, linearity, and hot pixels . . . . . . . . . . . . . . . . . . . .
6.9.4 Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.9.5 New options for the phase 3 data reduction . . . . . . . . . . . . . . . . .
6.9.6 Installation and usage of the toolkit for phase 3 . . . . . . . . . . . . . . .
6.9.7 Known problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii
62
62
63
64
64
65
65
65
65
66
66
66
69
69
69
71
72
74
74
76
76
76
77
78
80
80
80
80
81
81
82
82
83
85
87
87
87
87
87
88
88
89

iv

IRC Data User Manual for Post-Helium Mission (Phase 3)

Chapter 1

Introduction
1.1

Purpose of this document

This document is intended to provide a comprehensive guide to data taken during the postHelium (hereafter phase 3) mission with the Infrared Camera (IRC) onboard AKARI for observers to get started quickly. It includes a brief summary of the instrumentation, a summaryof the data products, and the pipeline software overview together with the calibration of the
instrument. Part of the data and information given here will be updated as the data reduction
and calibration will get improved. The latest information may be posted on the web till therevised version will be prepared. A comprehensive overview of the instrumentation and operation is described in the AKARI IRC Data User Manual and is given in the AKARI ﬁrst result
volume (Publications of the Japanease Astronomical Society volume 59, No. SP2). Also see the
AKARI Observer’s Manual for the Post-Helium (Phase 3) Mission.

1.2

Relevant information

AKARI Observer’s Web
The ISAS Web page contains the most up to date information:
URL:http://www.ir.isas.jaxa.jp/AKARI/Observation/
The ESAC page also includes up to date information:
URL:http://akari.esac.esa.int/observers/
Helpdesk
Any questions and comments on AKARI observations and user support are addressed to
the AKARI Helpdesks:
iris help@ir.isas.jaxa.jp
http://akari.esac.esa.int/esupport/

1

2

IRC Data User Manual for Post-Helium Mission (Phase 3)

Chapter 2

Instrument overview
The Infrared Camera (IRC) onboard AKARI was originally designed to make wide-ﬁeld deep
imaging and low-resolution spectroscopic observations in the pointing mode of the AKARI satellite. Its unique wide ﬁeld coverage of 10 × 10 arcmin2 is ideal for survey-type observations or
multi-object spectroscopic programs.
The capability for the use in the survey mode has later been explored and now is also being
used to carry out mid-infrared all-sky survey observations.
Each channel has a ﬁlter wheel, on which medium band ﬁlters, dispersive elements, and ablank window as a shutter are installed. Table 2.0.1 summarizes the parameters of the IRC
ﬁlters and dispersion elements. The NIR camera covers three independent wavelength bands
that very roughly correspond to the well known K, L, and M bands. Each of the two MIR
cameras have two narrow ﬁlters which cover the shorter and longer half of the wavelength rangeof the cameras, and a wide ﬁlter that overlaps the two narrow ﬁlters. The S9W and L18W bands
are used for the all-sky survey observations,
In the phase 3 mission, only the NIR camera is working and the MIR cameras are turned
oﬀ. In the followings however, some description of the MIR cameras is also given.
The pixel scale and the imaging area of the focal plane arrays are summarized in Table 2.0.2.
The imaging area is the rectangular area excluding the slit region.
The dispersion elements of the IRC are set into the ﬁlter wheel so that all the light in the
FoV is dispersed. A spectrum is obtained in the direction parallel to the scan path (in-scandirection). Slits are provided for each camera (Figure 2.0.2) in order to avoid contamination
by nearby sources / diﬀuse radiation. The two dispersion elements of the NIR camera provide
diﬀerent spectral resolutions over a similar wavelength range.
The slits are primarily designed for extended sources and it should not be assumed thatthey can be used to guide a point source into the slit, except for the NIR camera which has an
aperture for point sources. The current design is the following:• The slit for the NIR camera consists of three parts of diﬀerent widths. The left most(closest to the imaging area) has a 5 arcsec width and will be mainly used for simultaneous
observations of diﬀuse light with the MIR-S camera. This slit position is labeled as ‘Ns’ for
IRC04 AOT observation parameter. Both the NP (low-resolution prism) and NG (highresolution grism) will be used with this slit. The middle 1 × 1 square part, referred to as‘Np’ is for spectroscopy of point sources. The aperture is large compared to the absolute
pointing accuracy of the satellite (designed to be better than 30 arcsec) to ensure that thetarget can be accurately guided into the area. Note that for observations of faint sources,
confusion due to galaxies may be a serious problem. The NG (grism) is assumed to be
used with this aperture. The rightmost (outer) part (‘Nh’) has a 3 arcsec width and isused for the highest resolution spectroscopy of diﬀuse radiation with the NG (grism).

3

IRC Data User Manual for Post-Helium Mission (Phase 3)

4

MIR-L

50mm

folding mirror
unit (NIR)

NIR

filter unit
(MIR-S)
detector unit
(MIR-S)

MIR-S

Figure 2.0.1: Bird’s-eye view of the IRC camera.

2.1

Focal plane arrays

The IRC comprises three channels (Fig 2.0.1). The near-infrared (NIR) channel operates in 2-5
μm, the mid-infrared short (MIR-S) channel covers 5-12 μm, and the mid-infrared long (MIR-L)
channel works in 12-26 μm. The NIR uses 512 × 412 format InSb array and MIR-S and MIR-L
both employ 256 × 256 format Si:As impurity band conduction arrays. The three channels
have a ﬁeld of view of about 10’ × 10’ and the NIR and MIR-S share the same ﬁeld of view by
the beam splitter, whereas the MIR-L observes a sky about 25’ away from the NIR/MIR-S sky(Figure 2.0.2).

2.1.1

Near-InfraRed Camera: NIR

Figure 2.1.3 shows the side view of the NIR channel. The light from the telescope is split bya germanium beam splitter.and the near-infrared radiation is introduced to the NIR channel.
The NIR consists of silicon and germanium lenses. There are color aberrations among N2, N3,

Version 1.1 (August 28, 2009)

5

Scan Direction

Slit (MIR-L): 7"x0.4'
10.7'

Telescope Axis
5.0'

MIR-L
4.9'

10.2'

0.9'

NIR
& MIR-S

10.0'

5.0'
Slit (NIR): 3"x1'

0.4'

Slit (NIR): 1'x1'9.1'-9.5'*
25.0'

Slit (NIR/MIR-S): 5"x0.8'
0.5'** 9.3' (0.7') for N2, 9.5'(0.5') for N3 & N4
9.1' for MIR-S (all bands)

Figure 2.0.2: Field of view location of the three channels of the IRC. The vertical arrow indicatesthe scan direction in the survey mode. The NIR and MIR-S share the same ﬁeld of view bymeans of a beam splitter. See text for the usage of the slits.

Folding mirror
Si

Detector module
Si
Si
Ge

Telescope axis

50mm

Figure 2.1.3: Side-view of the NIR channel.
and N4 and the telescope focus is adjusted in between them. Faint ghosts of bright sources arealso present, which come from internal reﬂections in the beam splitter. The brightness is about
0.7% of the true source and the position is well determined and in a good agreement with the
ray-tracing simulation. There are also ghosts that seem to come from the internal scattering inIRC Data User Manual for Post-Helium Mission (Phase 3)

6

Table 2.0.1: IRC Filters and Dispersion Elements(1)
Channel

(2)
Name

(3)

(4)
λref
(μm)
2.4
3.2
4.1

(5)
Wavelength(μm)
1.9–2.8
2.7–3.8
3.6–5.3
1.8–5.2
2.5–5.0
5.9–8.4
6.7–11.6
8.5–13.1
5.4–8.4
7.5–12.9
12.6–19.4
13.9–25.6
20.3–26.5
17.5–25.7

(6)
λc
(μm)
2.34
3.19
4.33
—
—
7.12
8.61
10.45
—
—
15.58
18.39
22.89
—

(7)
Δλ
(μm/pix)
0.71
0.87
1.53
—
—
1.75
4.10
4.12
—
—
5.98
9.97
5.34
—

(8)
Dispersion(μm/pix)———
0.06 @3.5 μm
0.0097
—
—
—
0.057
0.099
—
—
—
0.175

N2
ﬁlter
N3
ﬁlter
NIR
N4
ﬁlter
NP
prism
NG
grism
S7
ﬁlter
7.0
S9W
ﬁlter
9.0
MIR-S
S11
ﬁlter
11.0
SG1
grism
SG2
grism
L15
ﬁlter
15.0
1
L18W
ﬁlter
18.0
MIR-L
L24
ﬁlter
24.0
LG2
grism
(4) Reference wavelength.
(5) Deﬁned as where the responsivity for a given energy is larger than 1/e of the peak.
(6) Isophotal wavelength of the ﬁlter band.
(7) Eﬀective bandwidth.
(8) Dispersion power of NP depends on wavelength.
The quoted value corresponds to 3.5μm
1 Renamed from L20W. No change of the wavelength proﬁle itself.

Table 2.0.2: General characteristics of the IRC focal plane arrays
Channel
NIR
Detector
InSb (SBRC-189)
Format
512 × 512
Pixel scale (”)
1.46 × 1.46
Imaging area (pixel2 ) 391 × 412
Readout noise (e− )30.6a−Dark current (e /s)
0.2
Operability(%)c99.9
a Estimated by Fowler 1 samplingb Estimated by Fowler 4 samplingc Pre-launch performance

MIR-S
Si:As (CRC-744)
256 × 2562.34 × 2.34
233 × 256
14.2b
26
100

MIR-L
Si:As (CRC-744)
256 × 2562.51 × 2.39
246 × 239
same as MIR-S
same as MIR-S
100

the NIR optics. Details are under investigation.
NIR and MIR-S channels acts as a folding mirror for the MIR-S.
astigmatism. Thus the pixel scale of the MIR-L channel is not the same for the X and Y
directions.

Version 1.1 (August 28, 2009)

2.1.2

7

Arrays operation

The focal plane arrays are operated in a synchronized manner. The array operation is madein a unit frame, which consists of short and long exposures. The short exposure is intended toincrease the dynamic range by about 8 to 10 times. However, data taken in the short exposure
indicate anomalous behavior and at present the short-exposure data are not able to be calibrated.
The clock pattern of one frame is the same for all AOTs. One frame is about 63 sec, in which
NIR has one short and one long exposures. The short exposure is made with the Fowler 1
sampling scheme.

2.2

Instrument AOTs

In a pointed observation, the ﬁlter and dithering combinations for IRC have been ﬁxed to a few
patterns, which are called Astronomical Observation Template (AOT). The duration of a pointed
observation and the frame time are well determined. Each AOT consists of a combination offrames of the IRC operation with dithering and ﬁlter wheel rotation operations, which is wellﬁxed in a pointed observation. The AOTs in phase 3 are designated by AOTZ#. Figure 2.2.4
shows an illustration of the pre-ﬁxed sequences in each AOT. In all the AOTs, except for
AOT11, a dark frame is carried out before and after the observation (pre-dark and post-dark
observations).
In a pointed staring observation, the IRC observation is started once it receives the notiﬁcation of the stabilization of the attitude from the attitude and orbital controlling system. The
IRC observation continues till the angle between the telescope axis and the earth rim becomesless than a certain value. Thus the last image in a pointed observation may be taken during the
maneuver and cannot be used for astronomical observations. This will be correctly treated in
the pipeline software.
ManeuverIRCZ0

Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)(filter 1)

IRCZ2

Exposure Cycle(filter 1)

IRCZ3

Exposure Cycle(filter 1)

IRCZ4

Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle(disperser 1)(disperser 1)(disperser 1)(disperser 1)

W

W

Exposure Cycle(filter 2)

Exposure Cycle(filter 2)

W

M

Exposure Cycle(filter 2)

Exposure Cycle(filter 3)

M

W

Exposure Cycle(filter 1)

Exposure Cycle(filter 3)

W

Exposure Cycle
(N3)

W

M

Exposure Cycle(filter 1)

Exposure Cycle(filter 2)

W

W

W

Exposure Cycle(filter 1)

Exposure Cycle(filter 2)

M

M

Exposure Cycle(filter 1)

Exposure Cycle(filter 1)

W

Exposure Cycle(filter 2)

Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle Exposure Cycle(disperser 1)(disperser 1)(disperser 1)(disperser 1)(disperser 1)

Figure 2.2.4: Observation sequences of the AOT IRCZ0, Z2, Z3, and Z4. Yellow boxes labeled as“Exposure cycle” indicate exposure frames. Orange boxes with “M” are Micro-Scan operations
including stabilization, and light-blue boxes with “W” are Filter Wheel rotations. Dead time
for a Filter Wheel change depends on the relative position of the elements. The Green area on
the right side is the extra observation time which is not guaranteed.

IRC Data User Manual for Post-Helium Mission (Phase 3)

8

Table 2.3.3: IRC sensitivity and image quality

5-σ sensitivity (μJy)
AOTZ0
AOTZ2
AOTZ3

2.2.1

N2

N3

N4

18

18
31
39

18
31
38

39

IRCZ0

The IRCZ0 mode was designed for deep imaging observations.

2.2.2

IRCZ2

The IRC02 mode was designed for general purpose imaging observations that take images with
two ﬁxed ﬁlters in a pointed observation. It provides at least three images for a ﬁlter with
dithering operations.

2.2.3

IRCZ3

The IRCZ3 mode was designed for general purpose imaging observations that take images withthree ﬁlters in a pointed observation. For each ﬁlter two imaging observations are made with
dithering operations.

2.2.4

IRCZ4

The IRC04 mode was designed for general purpose spectroscopic observations. This mode
does not have dithering operations. It has an imaging observation sandwiched by spectroscopic
observations (observations with the dispersive elements) of 4 frames. The imaging observation
will be used to determine the wavelength reference point for slit-less spectroscopy.

2.3

In-orbit sensitivity

Optical thougputs of all IRC channels are conﬁrmed to be as expected by observations of standard stars.
The sensitivity values for the imaging AOTs are collected in Table 2.3.3.
In the case of IRCZ4, Figures 2.3.5 and 2.3.6 show the 5σ sensitivity for point and diﬀuse
sources, respectively, as a function of λ.

Version 1.1 (August 28, 2009)9

5σ Detection Limit (mJy)

100
NP
NG
10

1

0.1

2

2.5

3

3.5

4

4.5

Wavelength (μm)

55.5Figure 2.3.5: 5σ detection limits for point sources with NP or NG in IRCZ4 with the Ns slit.

IRC Data User Manual for Post-Helium Mission (Phase 3)

5σ Detection Limit (MJy sr-1)

10

100
NP
NG

For Ns

10

1

0.1

2

2.5

3

3.5

4

4.5

Wavelength (μm)

5Figure 2.3.6: 5σ detection limits for diﬀuse sources with NP or NG in IRCZ4

5.5

Chapter 3

Distributed Data Products
Two levels of data products are delivered to the users: raw data and processed data. The
raw data are intended to be used to run the pipelines described in Chapters 5 and 6 to getinteractively the best science out of them. The quick look data are merely a rough approximationof what the user will get from his observations. They are the result of running the pipeline
automatically and with a default setting. An interactive and ﬁne-tuned run of the pipeline isrecommended in order to improve the quality of the results.
The quick look data header contain the version of the pipeline used. Since it is continously
evolving it could be diﬀerent from the version made available to the users.

3.1

IRC FITS ﬁle naming rule

Pointed observation data are distributed to the users as a tar+gz package format (hearafter’data package’) per observation. A data package contains FITS format observation data and a
Readme ﬁle describing the contents etc.
During the prioritized data use period data packages are encrypted and compressed by gpg(GnuPG) program.
The naming convention for the IRC package is:
AKARI IRC TargetID SubID.tar.gpg
where TargetID is a 7 digits number and SubID is a 3 digits number given by the observation
database.
Combination of TargetID and SubID give a unique identiﬁcation of the observation.
When extracting an IRC data package a directory named AKARI IRC TargetID SubID/
is created. Two subdirectories, rawdata/ and irc ql/ contain the raw FITS data ﬁles and theprocessed result ﬁles, respectively.

3.2

Raw data description

3.2.1

Raw data naming convention

The naming convention for the IRC raw data ﬁles is common for all the IRC AOTs ans it is thefollowing:
FVVxxxxxx [N—M].ﬁts
where:• F is a ﬁxed character• VVVxxxxxx: Extended frame counter (decimal degits). This is a unique identiﬁer of the
exposure.
11

IRC Data User Manual for Post-Helium Mission (Phase 3)

12• xxxxxx: frame conter in the telemetry ﬁle.
• VVV: maintained by the FITS creation program. Incremented when xxxxxx is reset to 0.
N—M : NIR / MIR (Scan mode data may have extra characters to this).
NIR data is in a separate ﬁle, while MIR-S and MIR-L are stored in the same FITS ﬁle.
The README ﬁle describes:• ﬁle list in the data package• observation summary extracted from the FITS ﬁle• comments speciﬁc for the observation

3.2.2

AKARI FITS Primary HDU (common information) rawdata header

The contents of the AKARI FITS Primary HDU of all raw data, both IRC and FIS, is as follows:
SIMPLEBITPIXNAXISNAXIS1NAXIS2NAXIS3
EXTENDFMTTYPEFTYPEVER
CNTTYPEDATECREATORCRTRVERPIPELINEDATASTATORIGINTELESCOP
INSTRUME
DETECTOROBSERVERPROPOSALOBS-CATPNTNG-ID
TARGETIDSUBIDOBJECT
OBJ-RA
OBJ-DECAOTAOTPARAMINSTMODETIMESYS

=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=

T
16
3
412
512
2
T
’ASTRO-F IMAGE IRC ’
4’IRC NIR ’’2006-09-25T09:45:24’’TBD’’1.0 ’’ircpl ver. 1.0’
’GOOD ’’ISAS/JAXA’’AKARI ’’IRC ’’NIR ’’PI Name ’’PRPID ’’OT ’
123456712345671’target ’320.5533-23.3325’IRC03 ’’8;0.5;70’
TBD’UTC ’

/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/

Standard FITS format
number of bits per data pixel
Number of axes
Image dimension
Image dimension
Image frame
Extension may be present
Type of File Format in FITS ﬁleVersion of FMTTYPE
Type of data content
File Creation Date
Data generator program nameVersion of CREATOR
Data Processing Pipeline name
Data status
Organization creating FITS ﬁle
AKARI mission
Identiﬁer of the instrument
Detector name
PI Name (Observer’s ID)
Proposal ID
Observation Category
Pointing ID
Target ID
Sub ID
Object name[degree] Target position[degree] Target position
Observation AOT
AOT Parameter
Instrument operation mode
Explicit time scale speciﬁcation

Version 1.1 (August 28, 2009)
DATE-OBS
DATE-END
DATE-REF
AFTM-OBS
AFTM-END
AFTM-REF
PIMTIOBSPIMTIENDPIMTIREF
EQUINOXRADEC
ROLL
AA-SOLAA-EAR
AA-LUN
TM-SAA
SAT-POSX
SAT-POSY
SAT-POSZDAYNIGHT
STTA-NUM
STTB-NUM
STTA-MODSTTB-MODCOMMENTHISTORY
END

=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=

YYYY-MM-DDTHH:MM:SSYYYY-MM-DDTHH:MM:SSYYYY-MM-DDTHH:MM:SS[double][double][double]’0xXXXXX ’’0xXXXXX ’’0xXXXXX ’
2000.0000
320.5533-23.3325-30.553
90.0021180.2083
210.6821
1829.
2903.5528
1704.3092
1968.4286’DAY ’
4
4’TRK ’’TRK ’

13
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/

Observation start date+time
Observation end date+time
Reference time in the Observation
DATE-OBS in ASTRO-F Time
DATE-END in ASTRO-F Time
DATE-REF in ASTRO-F Time
DATE-OBS in PIM-TI (36 bits DHUTI)DATE-END in PIM-TI (36 bits DHUTI)DATE-REF in PIM-TI (36 bits DHUTI)
Epoch of Coordinate[degree] Target position at DATE-REF[degree] Target position at DATE-REF[degree] Roll Angle at DATE-REF[degree] Solar avoidance Angle at DATE-REF[degree] Earth avoidance Angle at DATE-REF[degree] Lunar avoidance Angle at DATE-REF[sec] Duration since last SAA at DATE-REF[km] Satellite position at DATE-REF[km] Satellite position at DATE-REF[km] Satellite position at DATE-REF
Day/night status at DATE-REF
number of tracked stars in STT-A at DATE-REF
number of tracked stars in STT-B at DATE-REF
STT-A Mode status at DATE-REF
STT-B Mode status at DATE-REF
Any strings
Any strings

The header keywords are sorted as follows:• FITS basic information, data size information– SIMPLE refers to the standard FITS format– BITPIX is the number of bits per data pixel, and equal to 16 in the case of IRC– NAXIS is the number of axes, equal to 3 for IRC– NAXIS1 and NAXIS2 are the image dimensions– NAXIS3 is the number of image frames– EXTEND refers to the presence of extensions in the FITS ﬁle• Data type, creation and processing information– FMTTYPE is the type of File Format in FITS ﬁle. It can be ’ASTRO-F IMAGE
IRC’ or ’ASTRO-F SCAN IRC’
– FTYPEVER is the version of the ﬁle format described in FMTTYPE– CNTTYPE is the type of data content. It can be ’IRC NIR’or ’IRC MIR’
– DATE is the ﬁle Creation Date
– CREATOR is the data generator program name.
– CRTRVER is the version of CREATOR. Contents TBD

IRC Data User Manual for Post-Helium Mission (Phase 3)

14– PIPELINE is the Data Processing Pipeline name and version– DATASTAT is the Data status. It describes data status mainly from completeness of telemetry data. This does not tell detailed scientiﬁc quality of the data All
appropriate error status are listed, otherwise GOOD is given GOOD: No problem
INCOMPLETE: (Scientiﬁc data) incomplete due to telemetry loss etc. NOHK: HK
Status not available NOADS: Attitude information not available STTINI: STT did
not work properly. More status may be added as analysis progresses. Data otherthan GOOD may not be in the archive at the ﬁrst stage.
• instrument information– ORIGIN is the organization creating FITS ﬁle– TELESCOP is the AKARI mission [Satellite Name]– INSTRUME is the Identiﬁer of the instrument– DETECTOR is the detector name, either ’NIR’ or ’MIR’• observation details– OBSERVER is the PI Name (Observer’s ID)– PROPOSAL is the Proposal ID– OBS-CAT is Observation Category, either ’LS’, ’MP’, ’OT’, ’DT’, ’CAL’ or ’ENG’– PNTNG-ID is the Pointing ID. Usually it is identical with the Target ID, but is
diﬀerent for ’parallel mode’ observations.
– TARGETID is the Target ID– SUBID is the Target Sub ID– OBJECT is the Object name
– OBJ-RA is the RA Target position in degrees recorded in the database (double precision)– OBJ-DEC is the DEC Target position in degrees recorded in the database (double
precision)– AOT is the Observation AOT– AOTPARAM is the AOT Parameters set– INSTMODE is the Instrument operation mode. Contents TBD– TIMESYS is the Time system used in this ﬁle– DATE-OBS is the Observation start date+time, with format
YYYY-MM-DDTHH:MM:SS– DATE-END is the Observation end date+time. Same format than above– DATE-REF is the Reference time in the Observation. Same format than above– AFTM-OBS is the DATE-OBS in ASTRO-F Time
– AFTM-END is the DATE-END in ASTRO-F Time
– AFTM-REF is the DATE-REF in ASTRO-F Time
– PIMTIOBS is the DATE-OBS in PIM-TI (36 bits DHUTI) , format ’0xXXXXX ’– PIMTIEND is the DATE-END in PIM-TI (36 bits DHUTI), same format than above

Version 1.1 (August 28, 2009)15– PIMTIREF is the DATE-REF in PIM-TI (36 bits DHUTI) , same format than above???? OBS and ???? END are identical with ???? REF for convenience. ???? REF is
Reference time from TI sampled during each exposure cycle. This is due to the factthat IRC only sample time information once per exposure cycle. Note: PIMTI is theprimary information directly from telemetry. AFTI- and DATE- are from the timing
correction based on PIMTI.
• attitude information– EQUINOX is the Epoch of Coordinate– RA is the Target position at DATE-REF in degrees– DEC is the Target position at DATE-REF in degrees– ROLL is the Roll Angle at DATE-OBS– AA-SOL is the Solar avoidance Angle at DATE-REF in degrees– AA-EAR is the Earth avoidance Angle at DATE-REF in degrees– AA-LUN is the Lunar avoidance Angle at DATE-REF in degrees– TM-SAA is the duration in seconds since last SAA at DATE-REF. Deﬁnition of
SAA is diﬀerent for diﬀerent detectors. SAA region is deﬁned by the glitch rate
map observed by the Star Tracker with arbitrary threshold level. IRC follows thisthreshold. Shifts of 30 and 60 seconds are applied to FIS SW and LW, respectively.
– SAT-POSX is the Satellite position at DATE-REF in km– SAT-POSY is the Satellite position at DATE-REF in km– SAT-POSZ is the Satellite position at DATE-REF in km– DAYNIGHT is the / Day/night status at DATE-REF These ﬁelds are updated as
pointing analysis goes on from On-board AOCS to G-ADS ( Pointing reconstruction
for Survey mode)– STTA-NUM is the number of tracked stars in STT-A at DATE-REF– STTB-NUM is the number of tracked stars in STT-B at DATE-REF– STTA-MOD is the STT-A Mode status at DATE-REF, either ’TRK’, ’ACQ’, ’STB’,’INI R’ or ’INI N’– STTB-MOD is the STT-B Mode status at DATE-REF, same as above– COMMENT: Any strings– HISTORY: Any strings

3.2.3

Raw data dimensions

FITS dimension of the NIR frames is 412 (x) × 512 (y) × 2 (short+long exposure sub-frames).
With the AOTZ4, one will obtain 11 or 12 sets of frames:• 5 sets of pre-dark frame• 8 or 9 sets of spectroscopy exposure frames• 1 set of reference imaging exposure frame• 5 sets set of post-dark frame

IRC Data User Manual for Post-Helium Mission (Phase 3)

16

The dark frames will be taken, during pre- and post-satellite maneuvering period, with the
shutter closed (the ﬁlter wheel at so-called ”CAL” position).
The spectroscopy frames are taken with dispersers inserted along the optical path, by rotating
the ﬁlter wheel.
The direct imaging frame is taken with the same ways as for the normal imaging observations(with AOTZ0, Z2, and Z3). The image taken during the spectroscopy mode (AOTZ4) is called
as the ”reference image”. The N3 wide-band ﬁlters will be used as the reference images of NIR.
Other combinations of the broad-band ﬁlters and dispersers are not available.
The satellite starts to maneuver back to the survey mode according to the timer, irrespectiveof the IRC operation. The last exposure should be discarded since part of the exposure could
be made during the maneuver. The present FITS header does not record the status of thesatellite attitude control. In the imaging data pipeline the removal of the last exposure is madeautomatically by checking the source intensity. Such an automatic procedure is not included in
the current spectroscopy data reduction software: users need to do it by themselves (see Chapter
6). Note, however, that the automatic removal is not always perfect and users may have to do
it manually in some cases even for the imaging data.

3.3

Quick look data

Together with the rawdata, quick look products are distributed in the archive for both imaging
and spectroscopy IRC observations. These ﬁles are not intended for science and the users arestrongly encouraged to run the correponding pipelines in order to obtain the best out of the raw
data.

3.3.1

IRC imaging IRCZ0, IRCZ2 and IRCZ3As explained in the README ﬁle distributed with the data in the archive, the irc ql subdirectory contains, together with a set of log ﬁles, a main quick look ﬁts image per detector, withthe following naming convention. Its name is efmslCnDaFVVVxxxxxx [N]yyy.ﬁts. The ﬁts preﬁxes refer to the performed steps during the processing: DARK correction (D), linearity (ln),distortion correction (d) and ﬂatﬁedling (f). [NSL] refer to each detector. Under the directoryof stacked IM/ a stacked image of TARGENAME FILTER EXPOSURE.ﬁts is also provided.
The irclog ﬁle contains a brief description of each ﬁle from the data reduction pipeline. Please
consult the README ﬁle in the delivery package for more information.

3.3.2

IRC spectroscopy: IRCZ4

The quick look products distributed in the case of spectroscopy observations (IRC04) are thefollowing:• TARGETID.SUBID.FILTER DISPERSER.reﬁmage bg.ﬁts: dark, linearity and ﬂatﬁled
corrected reference image.
• TARGETID.SUBID.FILTER DISPERSER.specimage bg.ﬁts: dark, linearity and ﬂatﬁeld
corrected spectroscopy image. This quick look image is usable only for slit-less spectroscopy. For slit spectroscopy further processing with the spectroscopy pipeline is needed.
Auxiliary ﬁles are described in the README ﬁle distributed in the archive.

Version 1.1 (August 28, 2009)

3.4

17

IRCZ4 image orientation and dispersion direction

Since raw NIR images are rotated for technical reasons of data handling in the IRC electronics
onboard the satellite, NIR images (NP, NG and N3) will be rotated by 90 deg counterclockwise
at the very ﬁrst stage of the data reduction for convenience in the spectroscopy pipeline.
This is the original orientation in raw images (without rotation for NIR):• NP: longer wavelength comes at right side (toward positive X).
• NG: longer wavelength comes at left side (toward negative X).
The IRCZ4 data reduction pipeline makes the rotation of the NIR images at the ﬁrst step.
After the NIR rotation, the dispersion directions are the same for all the dispersers in spectroscopy images. Note, however, that NP shows positive dispersions (longer wavelength comesat higher Y), whereas NG shows negative dispersions (longer wavelength comes at lower Y).
At the same stage, the orientation of all NIR images is also set right, i.e., the image is neither
ﬂipped nor mirrored. Thus one can match the IRC images with other WCS-correct images, suchas 2MASS images, only by shifting and/or rotating the images. Since the satellite is designed
to scan the sky along the ecliptic latitude on the sky, and the X axis of the IRC array is aligned
perpendicular to the scan direction, the Y axis is aligned closely with the ecliptic latitude while
the X axis is aligned with the ecliptic longitude.

18

IRC Data User Manual for Post-Helium Mission (Phase 3)

Chapter 4

IRC Calibration and Accuracy
This chapter oﬀers an overview of the main issues related to the IRC calibration and in-orbir
performance. The instrument calibration is addressed in an standard way: dark level, ﬂat-ﬁelds,
linearity, point spread function, absolute ﬂux calibration and distortion correction. Spectroscopy
and imaging are addressed at the same time. Speciﬁc topics for each of them are explicitely
indicated. Please be aware that the calibration for the phase 3 period has not been completed
and the followings are preliminary results.

4.1

Dark image

Five dark images are taken before and after the target observation in a pointing observation forphase 3. The self-dark image is created from these 10 dark frames. This is completely diﬀerent
from the phase 2 observations, where the superdark is used instead of the self-dark for the NIR
channel.

4.2

Flatﬁeld

4.2.1

Flatﬁleds for imaging data

N2

N3

N4

Figure 4.2.1: Flatﬁelds for N2, N3, and N4.
The IRC super ﬂats for the NIR bands were derived from all the imaging observations so
far taken during the phase 3. Unfortunately, stray light is present and an unexpected highbackground is seen. This is due to Earth rim light, which is uniform over the Field of View.
Therefore, observations of the diﬀuse background may be severely aﬀected.
19

IRC Data User Manual for Post-Helium Mission (Phase 3)

20

The S/N of the superﬂat is estimated to be 100 for N2 and N3, and about 400 for N4. The
superﬂats will be updated in future.

4.2.2

Flats for spectroscopy images

The spectroscopy ﬂatﬁelds are images made by a large number of blank sky spectroscopy images
combined and normalized so that any faint object spectra are removed by clipping averaging
techniques. Super-ﬂats are prepared for NP and NG. As will be described at the end of thischapter, these ﬂats show spectral features, which are not due to the sensitivity variation. Therefore object spectra should not follow the ’ﬂat’ pattern. However, this ’super-ﬂat’ with spectralfeatures is used in the data reduction because the color-term correction actually cancel out the
spectral feature correction.
For ﬂat ﬁelding slit spectra, including NG with the point source aperture (Np), the superﬂats are typical for conventional slit spectroscopy. These ﬂats are also made by combining alarge number of blank sky spectra. Preliminary ﬂats are prepared for the Ns and Nh slits. They
have low S/N ratios and they could degrade the data quality when applied. Better ﬂats arebeing prepared.

4.3

Instrument linearity

The detector operation parameters have been adjusted to accommodate the increase of the darkcurrent in phase 3. This reduced the detector well capacity signiﬁcantly and the saturationlevel changed from 12000 ADU in phase 2 to 2000 ADU in phase 3. Accordingly the linearity
was changed. The data taken with the calibration lamp is used for phase 3 data instead of thelaboratory measurement to derive the linearity correction formula. The data taken with constantillumination of the calibration lamp are ﬁtted with a polynomial. The correction accuracy isestimated to be about 5% for the range up to 2000 ADU. Fig. 4.3.2 shows the raw signal (green)
and the signal after linearity correction (red) in ADU.
20

15

10

Deviation [%]

5

0

-5

-10

-15

-20
0

500

1000

1500

2000

2500

FIT [ADU]

Figure 4.3.2: Raw signals (green) and signals after linearity correction (red) in units of ADU.

4.4

Instrument Point Spread Function

Table 4.4.1 shows the FWHM of the in ﬂight PSF in the imaging mode, based on observationsof standard stars, performed in October 2007 after the telescope focus adjustment. It is slightly

Version 1.1 (August 28, 2009)21

degraded from the phase 2 performance due to the degradation of the telescope image quality
around 40K (Onaka et al. 2008, Proc. of SPIE, 7010, 70102X).
N2
3.2

N3
3.2

N4
3.2

Table 4.4.1: In orbit PSF (in pixels).
The PSFs are not spherical. Therefore the users are recommended to average individual
images in the coadding process, if photometric accuracy is concerned. In some cases, mainly
in deep survey, the median ﬁltered image will lose some signal if the images are rotated withrespect to each other. To avoid this problem users should use “average’ instead of “medianﬁlter” when combining individual images.
The PSF in the spectroscopic mode is worse by one or two pixels compared to imaging data.
No signiﬁcant wavelength dependence is seen along dispersers.

4.5

RSRF

Figures 4.5.3 and 4.5.4 show the relative spectral response functions for the NIR ﬁlters anddispersion elements. No change has been recognized from phase 2 and the same RSR data canbe used for the phase 3 data reduction.

Relative Response

1

N4

N3

0.8

N2

0.6
0.4
0.2
0
1

2

3

4

Wavelength (μm)

56Figure 4.5.3: The Relative Spectral Response Function of the IRC/NIR Camera for Fλ .

4.6

Flux calibration for point sources

Observations of the same standard stars employed in phase 2 are being executed. At this stage
only tentative absolute calibration results are available.
The raw data were reduced with the IRC imaging pipeline. The pipeline produces one
coadded image for each band/exposure conﬁguration, using ”median” as the combine mode.
Each conﬁguration image corresponds to the exposure unit, listed in Table 4.6.2.The actual

IRC Data User Manual for Post-Helium Mission (Phase 3)

22

Relative Response

1
0.8

NP
NG

0.6
0.4
0.2
0
1

2

3

4

5

Wavelength (μm)

6Figure 4.5.4: The Relative Spectral Response Function of the IRC/NIR dispersion elements per
photon.
exposure time is the unit number × the unit exposure time, tunit , which is approximately 0.5844
s.
Table 4.6.2: Exposure time of each band/exposure conﬁguration.
Band Exposure unit number∗
NIR
short
8
NIR
long
76
∗ One unit time corresponds to about 0.5844 s.

4.6.1

Estimation of the in-band ﬂux

The in-band ﬂux density of each band at the nominal wavelength, fλquoted (λi ), was calculatedby the following equation: νie
Ri (ν)fν (ν) dνhννisquoted
(4.6.1)(νi ) =  νie  fν
νi Ri (ν)
dν
ν
hν
νis
or
 λie

Ri (λ) λ fλ (λ) dλλfλquoted (λi ) =  λis 
ie
λis

λi
λ



(4.6.2)Ri (λ) λ dλwhere fλ (λ) is the ﬂux density of a standard star (Cohen template) and Ri (λ) is the spectral response (the transmission of the optics and the response of the detector, unit: electron photon−1 )Version 1.1 (August 28, 2009)23of the band i. Here fν ∝ ν −1 or fλ ∝ λ−1 is assumed. The adopted normal wavelengths of eachband, λi , are listed in Table 4.6.3 along with the range of the integration (λis , λie ).
Table 4.6.3: The normal wavelength λi and the range of integration, λis and λie .
band
N2
N3
N4

4.6.2

λi
2.40
3.20
4.10

λis
1.60
1.60
1.60

λie
5.770
5.770
5.770

Absolute calibration

The conversion factors for phase 3 imaging observations were derived based on observations ofstandard stars performed until March 2009. The results are summarized in Table 4.6.4. Notethat the ratio of the conversion factors between the long and short exposures in the phase is not
the one expected from the exposure time. The reason is not known at present. The calibration
for phase 3 spectroscopy has not been completed. We simply apply a factor of 0.7 reduction in
the sensitivity in the spectroscopy pipeline toolkit at the present.
Table 4.6.4: Conversion factors (ADU exposure−1 to Jy).
Band
N2
N3
N4

4.6.3

short exposure
fs
5.50 × 10−6
3.89 × 10−6
2.78 × 10−6

error (%)
1.85
1.96
2.13

long exposurefl6.60 × 10−7
4.95 × 10−7
3.62 × 10−7

error (%)
2.16
2.37
2.28

Stability of the sensitivity

Although the temperature of the IRC is gradually increasing, we do not so far see any systematicchange in the sensitivity at the time when this document is being prepared. Updated informationon the stability will be posted on the web.

4.7

Flux calibration for extended sources

A correction factor should be applied to convert the point sources ﬂux calibration into extended
sources ﬂux. The absolute calibration for diﬀuse sources is being planned, but is not available
at present.

IRC Data User Manual for Post-Helium Mission (Phase 3)

24

4.8

Color corrections mentioned in previous section, the quoted values of the IRC ﬂux calibration assume fλ ∝ λ−1 .
For cases of other incident spetra, the color correction is required.
The quoted value at a reference frequency is given byRi (ν)fν (ν)dνhν
quoted=f νi ν  R (ν) ,
i
i
dν
ν
hν

where R is the response in units of electron per photon. The correction factor K is calculatedas
=fνquotediRi (ν) fν (ν)
Ri (ν) fν (ν)
Ri (ν)dνdν(ν)fν (ν)dν
actualhνfhν fν (νi )hνactual  =    νifνactual=≡ K(λi )fνactual ν  R (ν) fνiii
νi Ri (ν)νi Ri (ν)
i
i
dν
dν
dν
ν
hν
ν
hν
ν
hν

Thus, K is given byRi (ν) fν (ν)
dνν f (ν )K(λi ) ≡    ν i
νi Ri (ν)
dνννWe calculate the correction factors for the following incident spetrum cases:fν (λ) = τ0

λ
λ0

α 

2hc
λ3


exp

1
 hc 
λkT−1(4.8.3)

with α = 0, −1, −2.
The results are shown in the following tables. The reference wavelengths are ﬁxed as: λ1 =2.4μm, λ2 = 3.2μm, and λ3 = 4.1μm.

Version 1.1 (August 28, 2009)

Table 4.8.5: Color Correction factors for NIR channela – Black Body (α = 0)
Intrinsic
NIR/N2
NIR/N3
NIR/N4Temperature (K) K(2.4μm) K(3.2μm) K(4.1μm)
40
–
–
–
50
–
–
–
60
–
–
–
70
–
–
–
80
–
–
–
90
–
–
396.131100––
164.506110––
81.786120–873.058
46.429130–297.140
29.139140–
121.81419.760150–
58.29714.242160–
31.767
10.776170–
19.2918.482
180–
12.811
6.895
190
–
9.149
5.756
200
–
6.928
4.914
210
–
5.496
4.275
220
582.603
4.526
3.778
230
316.052
3.839
3.385
240
181.324
3.336
3.069
250
109.374
2.955
2.810
300
16.057
1.952
2.024
350
5.065
1.543
1.643
400
2.631
1.333
1.428
450
1.818
1.212
1.294
500
1.463
1.136
1.204
600
1.167
1.052
1.095
700
1.051
1.012
1.033
800
0.997
0.992
0.994
900
0.970
0.982
0.969
1000
0.957
0.977
0.951
1500
0.963
0.981
0.910
2000
0.990
0.992
0.896
2500
1.013
1.001
0.890
3000
1.031
1.008
0.886
3500
1.045
1.013
0.884
4000
1.056
1.017
0.883
4500
1.064
1.020
0.882
5000
1.071
1.022
0.881
6000
1.081
1.026
0.880
7000
1.088
1.029
0.880
8000
1.094
1.031
0.879
9000
1.098
1.032
0.879
10000
1.101
1.034
0.879
20000
1.116
1.039
0.878
30000
1.121
1.041
0.877
40000
1.123
1.042
0.877
50000
1.125
1.043
0.877
60000
1.126
1.043
0.877
a

Values are calculated for λ1 = 2.4μm, λ2 = 3.2μm, λ3 = 4.1μm.

25

26

IRC Data User Manual for Post-Helium Mission (Phase 3)

Table 4.8.6: Color Correction factors for NIR channela – Gray Body (α = −1)
Intrinsic
NIR/N2
NIR/N3
NIR/N4Temperature (K) K(2.4μm) K(3.2μm) K(4.1μm)
40
–
–
–
50
–
–
–
60
–
–
–
70
–
–
–
80
–
–
938.23390––
305.576100––
127.958110––
64.122120–541.92036.679130–187.774
23.187140–78.79615.834150–
38.79411.488160–
21.8228.749
170–
13.696
6.929
180
–
9.390
5.665
190
–
6.906
4.757
200
–
5.367
4.083
210
546.949
4.356
3.571
220
280.566
3.657
3.172
230
153.521
3.154
2.856
240
89.003
2.779
2.601
250
54.374
2.492
2.393
300
8.881
1.715
1.757
350
3.276
1.390
1.449
400
1.952
1.223
1.276
450
1.477
1.127
1.168
500
1.257
1.069
1.097
600
1.064
1.007
1.011
700
0.989
0.981
0.963
800
0.957
0.971
0.934
900
0.945
0.968
0.915
1000
0.943
0.970
0.902
1500
0.982
0.992
0.875
2000
1.027
1.012
0.868
2500
1.061
1.027
0.866
3000
1.087
1.037
0.865
3500
1.106
1.045
0.865
4000
1.121
1.051
0.865
4500
1.132
1.056
0.865
5000
1.142
1.059
0.865
6000
1.156
1.065
0.866
7000
1.166
1.069
0.866
8000
1.173
1.072
0.866
9000
1.179
1.074
0.866
10000
1.183
1.076
0.867
20000
1.203
1.084
0.868
30000
1.210
1.087
0.868
40000
1.213
1.089
0.868
50000
1.215
1.089
0.868
60000
1.216
1.090
0.868
a

Values are calculated for λ1 = 2.4μm, λ2 = 3.2μm, λ3 = 4.1μm.

Version 1.1 (August 28, 2009)

Table 4.8.7: Color Correction factors for NIR channela – Gray Body (α = −2)
Intrinsic
NIR/N2
NIR/N3
NIR/N4Temperature (K) K(2.4μm) K(3.2μm) K(4.1μm)
40
–
–
–
50
–
–
–
60
–
–
–
70
–
–
–
80
–
–
719.02090––
236.320100––
99.825110––50.445120–339.050
29.088130–120.115
18.531140–
51.86412.748150–
26.4079.316
160–
15.3977.143
170–
10.012
5.694
180
–
7.092
4.686
190
–
5.367
3.959
200
549.483
4.275
3.418
210
264.183
3.541
3.006
220
136.893
3.025
2.685
230
75.855
2.647
2.430
240
44.673
2.361
2.224
250
27.825
2.140
2.055
300
5.291
1.528
1.541
350
2.329
1.268
1.293
400
1.566
1.135
1.154
450
1.270
1.061
1.068
500
1.125
1.017
1.012
600
0.996
0.976
0.946
700
0.949
0.963
0.910
800
0.935
0.962
0.889
900
0.936
0.968
0.876
1000
0.945
0.975
0.868
1500
1.019
1.016
0.855
2000
1.084
1.047
0.855
2500
1.132
1.068
0.857
3000
1.166
1.084
0.860
3500
1.192
1.095
0.862
4000
1.212
1.103
0.864
4500
1.227
1.110
0.866
5000
1.239
1.115
0.867
6000
1.258
1.123
0.869
7000
1.271
1.129
0.871
8000
1.281
1.133
0.873
9000
1.288
1.137
0.874
10000
1.294
1.139
0.874
20000
1.320
1.151
0.879
30000
1.329
1.155
0.880
40000
1.333
1.157
0.881
50000
1.335
1.158
0.881
60000
1.337
1.159
0.881
a

Values are calculated for λ1 = 2.4μm, λ2 = 3.2μm, λ3 = 4.1μm.

27IRC Data User Manual for Post-Helium Mission (Phase 3)

28

4.9

Pixel scale

The pixel scale of the NIR image is determined based on observations of globular clusters andGalactic center with the IRC. In these images we matched the 2MASS corrdinates of detectedstars with the detector xy coordinates by ﬁtting polynomials and determined the pixel scale.
The pipeline software regrids the image with the pixel scale of 1.46 . The deviation from an
ideal grid square are up to 2 pixels at the edge. No appreciable change is noticed from phase 2.
Refer to the IRC Data Users’s manual for details of the distorion. In summary, the accuracy isabout 0.1 pixel (N4 may be slightly worse).

4.10

Astrometry

A corrected and ﬁnal astrometry can be achieved at the end of the pipeline cross-correlatingwith the 2MASS catalogue. This option is turned oﬀ by default. This is because the softwareto retrieve the 2MASS catalog may be platform-dependent. To enable this option, “curl” has to
be installed in the system. The instructions for it are beyond this manual. Ask your computer
administrator for the installation.
The current pipeline has a capability to ﬁrst include the pointing information from theattitude and orbital control system (AOCS) directly in the WCS format. If matching withthe 2MASS data is successful, then the pointing information will be replaced by the matchingdata. The parameter “WCSROOT” in the FITS header indicates which information (AOCS or
2MASS) is used to determine the coordinates.

4.11

Arrays anomalies

NIR anomalies are shown in Figures 4.11.5 and 4.11.6. Most of them also aﬀects the Spitzer
IRAC instrument.
• Muxbleed• Muxstripes• Column pulldown• Banding

4.12

General concerns on slit-less spectroscopy data

This section is dealing mainly with problems related to the slit-less spectroscopy data. Issues
for the slit spectroscopy matters are described separately.
• Wavelength reference point: In the slit-less spectroscopy mode, the wavelength reference
point depends on the location of objects within the FOV. Therefore, the determinationof the source posistions on the reference image is very important. Errors in the source
positions leads to errors in the asigned wavelengths and, hence the ﬂux calibration.
• Contamination by nearby sources: Spectra of more than two objects aligned along with
the Y axis could overlap on the same pixels with diﬀerent wavelengths. It is impossible
to separate the spectral overlap on the observed image, without knowing the spectrum ofeach object.

Version 1.1 (August 28, 2009)29

Figure 4.11.5: Banding in NIR array• Contamination by zero-th/2nd order light: For the grisms, images of the zero-th and the
2nd order light can be seen as well as the 1st order image (our prime target for data
reduction), although the eﬃciency for the zero-th and the 2nd order light is very low. All
these images contaminate other spectra aligned along with the columns of pixels where
the objects exit. These contaminations can be ignored in most cases, and cautions areneeded actually only around very bright sources. For the NG spectroscopy with point
source aperture (Np), there is little chance for this kind of contamination to occur since
the aperture size is much smaller than the size of the dispersed spectroscopy image along
the Y or the dispersion direction.
• Spectral smearing for extended sources: In the slit-less spectroscopy mode, the spectral
resolution is determined by the size of the sources (or the PSF structure plus satellite
attitude drift). Thus the spectral resolution is lower for extended objects. If an extended
object shows resolved structures, the interpretation of the 2D spectra becomes very diﬃcult
because of the convolution of the spectrum over the object spatial structure.
• Background spectral features: Although the background sky is essentially ”ﬂat” in theimaging mode, it is not true for spectroscopy images.
Note that these features would not be observed with the point source aperture (Np) andother narrow slits, since the aperture size is much smaller than the size of the dispersed
spectroscopy image along the Y or the dispersion direction.
• Faint background areas near the Y=0 or Y=Ymax edge of FOV: For the slit area, thebackground is dominated by the zodiacal light. The spectral-response curve shows a simple
pattern with the decreasing sensitivity at the highest and lowest wavelength ends of thedisperser’s spectral coverage. Thus one will see a background pattern which becomes

30

IRC Data User Manual for Post-Helium Mission (Phase 3)

Figure 4.11.6: NIR array anomalies: Muxbleed, Muxstripes and column pulldown

faint at the top and bottom of the image in untis of ADU. For the slit-less area, both
the wavelength and the spatial axis Y go along with the same direction (Y), and thus
the observed background image is a spatially convolved background spectrum. Since thelength of the spectrum along Y (50-70 pix) is much smaller than the aperture size (256or 412 pix), the resultant background spectrum is almost constant across Y after being
convolved spatially by the large aperture. The regions around Y=0 and Y=Ymax are
exceptions, where the edge of the aperture prevents full convolution of the background
spectrum along the Y direction and, hence, the background signal becomes faint near
Y=0 and Y=Ymax is seen. Note that this kind of pattern at the very Y edge of the FOV
does occur only for the background light, but not for object spectra.
• Jump of the background (ridge) near the center of FOV: Another background feature isa ridge seen near the Y center, stretching along the X axis, seen in grism spectroscopy
images. This is caused by the zero-th order light of the grisms. Since the zero-th order
light image forms at an oﬀset position from the reference image, only about a half of the
FOV suﬀers from its contamination.

Version 1.1 (August 28, 2009)31• Satellite attitude stability/instability: Position shifts among sub-frames are frequently
observed due to the satellite attitude drift in the pointing attitude control mode. The
drift is as large as several pixels in the worst cases. To correct the drift, one needs toregister images among spectroscopy sub-frames and between spectroscopy and reference
images. The second correction is very important to determine the wavelength reference
point. Note that it takes about 30 sec to switch the spectroscopy mode to/from the imaging
mode for the ﬁlter wheel rotation. Thus the time interval between the last spectroscopy
sub-frame and the ﬁrst imaging sub-frame is longer by this period than the interval between
taking spectroscopy sub-frames. To measure the image drift among spectroscopy images,
we use cross-correlation image matching technique. To measure the image drift between
spectroscopy and reference images, we could use the same cross-correlation technique.
In this cross-correlation we use a template spectrum that was extracted from the datawith negligible drifts. Then by comparing the template with spectra extracted from thespectroscopy data in question, we estimate the relative drift.
For spectra of diﬀuse sources at the narrow slits (Nh, and Ns), we do not have to takeaccount of the satellite attitude stability, if the size of the drift is much smaller than thatof the objects.
• Ghosts in Np observations: Ghosts relating the Np slit have also been recognized. Leftpanel of Fig 4.12.7 shows an example of the imaging data. There are sources seen on theslit mask region, which must be ghosts from the sources in the Np slit. Right panel ofFig 4.12.7 shows the corresponding spectral data, in which ghost spectra overlap with thesource spectrum. No deﬁnite origin for these ghosts has been elucidated and thus no clear
recipe has been prepared to correct for them at present.

Figure 4.12.7: Ghosts generated in Np observations. Left panel: Imaging data. Right panel:
spectroscopic dataChapter 5

Imaging toolkit cookbook
5.1

Introduction

The IRC imaging data reduction toolkit is developed to address and correct any IRC Instrument
features, such as linearity, ﬂat ﬁelding, distortion etc., and converts the raw ADU signal tophysically meaningful units (i.e. ﬂux). However, in the current version, the data number ofresultant image is ADU per frame, and they should be multiplied by a certain constant toconvert it to a physical unit such as Jy. We will provide a table of the conversion of ADU/s to
Jy for point sources in addition to the integration time for each frame. The in-orbit calibration ismade based on aperture photometry with an aperture radius of 10 pixels for the NIR. For otheraperture sizes, observers have to make an aperture correction, which is currently not known
owing to the unknown PSF (not stable and not well deﬁned). Similar to the Spitzer IRAC
arrays, reﬂection of the light within the arrays exists also for the IRC arrays. The calibration
for the diﬀuse light is currently not available.
The toolkit (including the entire source code) will be released, and be progressively updated
reﬂecting the user’s feedback. The user will receive the raw data plus automatic toolkit (to makebasic science data). The user may customize the toolkit at his/her own risk.
We provide super-ﬂats, which will be updated periodically with the version number. e.g.,
Super-darkYYMMDD.ﬁts
The toolkit for phase 3 data reduction is released separately from that for phase 2 data
reduction. The user interface is basically the same, but users should not use it for phase 2 data
reduction. It can be installed simultaneously with that for the phase 2 data reduction toolkit as
described below. For users who are familiar with the phase 2 data reduction, the usage of thetoolkit is completely the same although the treatment of dark frames and the linearity correction
are diﬀerent.

5.2

General overview of the toolkit processing

The toolkit runs on IRAF, and it consists of two main steps plus an optional step.
At present the toolkit concept consists of 3 ”boxes” currently in a state of development.
• pre-pipeline processor (Red-Box):• pipeline processor (Green-Box):• post-pipeline processor (Blue-Box):

32

Version 1.1 (August 28, 2009)33• Pre-pipeline Processor (Red Box) : Produces Basic Data
– redbox.ircslice: header formatting, slice IRC 3D images into usual 2D ones
– redbox.mkirclog: making the observing log ﬁle “irclog”• Pipeline (Green Box) : Produces Basic Science (Calibrated) Data
– greenbox.anomalous pix: mask Bad/Dead pixels– greenbox.wraparound: wrap around correction– greenbox.dark: subtract dark current– greenbox.ircnorm: Normalize sampling and compression bit shift– greenbox.scatt light: Subtract scattered light pattern– greenbox.cosmic ray: remove cosmic rays
– greenbox.linearity: correct linearity of the detector response– greenbox.saturation: mask saturated pixels– greenbox.slit mask: mask slit area on detector– greenbox.ﬂat: ﬂatten the pixels– greenbox.aspect ratio: Distortion Correction (Aspect ratio resampling)• Post-Pipeline Processor (Blue Box) (This is a separated package, but called from pipeline
processor by default)– bluebox.coadd: to form a co-added image∗ source extract: Perform source extraction∗ calcshift: Calculate shift&rotation between images∗ adjust sky: adjust sky level before stacking∗ irc stack: stack individual frames to form a co-added image
The pipeline can be run automatically or interactively step by step.

5.3

Expected Data Processing Rate (minimum expectation)

NIR = 412*512[pixels] * 16[bit/pixel] / 8[bit/byte] = 412 Kbyte/frame• 1 NIR ﬁts = 412 Kbyte/frame * 2 frame/ﬁts = 824 Kbyte/ﬁts ﬁle• AOTZ0: 6 exposure cycles/pointing * (1 NIR ﬁts)/exposure cycle ∼ 10.8 Mbyte/pointing• AOTZ2: 6 exposure cycles/pointing * (1 NIR ﬁts)/exposure cycle ∼ 10.8 Mbyte/pointing• AOTZ3: 6 exposure cycles/pointing * (1 NIR ﬁts)/exposure cycle ∼ 10.8 Mbyte/pointing
The toolkit requires 3 or 4 times of disk space compared to the data.

IRC Data User Manual for Post-Helium Mission (Phase 3)

34

5.4

How to install the IRC imaging Toolkit

5.4.1

Requirements

The toolkit is developed in a Linux PC environment and has also been successfully run in the
OSX Unix environment. The toolkit requires the following environment for its full function:• unix (Solaris, MacOS X, Linux, BSD)
• IRAF 2.12.2a or later• gcc 3.0 or later• perl• curl (for WCS calculation on Solaris machines)
We notice that the toolkit does not work properly under IRAF 2.14 and it is not fully tested
with later versions of IRAF 2.14.

5.4.2

Install IRAF

• http://iraf.noao.edu/

5.4.3

Download IRC imaging data reduction software package

The latest version of the toolkit with installation and operating instructions for the phase 3
data reduction can be obtained from the AKARI Observer’s web site (see section 1.2). It can beinstalled simultaneously with that for the phase 2 data reduction toolkit, but must be installed
in a diﬀerent directory.

5.4.4

Unpack irc p3.tgz• mv ircYYMMDD.tgz /where/you/install/irc p3• cd /where/install/irc p3• tar xvzf ircYYMMDD p3.tgz

5.4.5

Make irc binaries• cd /where/you/install/irc p3/src• makeThis will create binary ﬁles in /where/you/want/to/install/irc/bin/

5.4.6

Run ”setpath.pl”• cd /where/you/install/irc p3/lib• perl setpath.pl
This will rewrite the ”setpath.dat”. Please check the following line in ”setpath.dat” is thefollowing format
set irchome = ”/where/you/install/irc p3/”
This line should indicate the location where you installed the irc p3 package.

Version 1.1 (August 28, 2009)

5.4.7

35

Perl path

Typing which perl in the unix environment indicates where the perl is installed in your system.
The toolkit assumes that the perl is installed in/usr/local/bin/perl
Therefore, those who have the perl binary elsewhere, should do as follows:• If you know the root password, please create a symbolic link in /usr/local/bin by typing:– cd /usr/local/bin– ln -s ‘which perl‘ perl
It will create a symbolic link perl in /usr/local/bin.
• If you do not know the root password, please modify the ﬁrst line of every perl script ﬁlein /where/you/want/to/install/irc/perl. For example, if you have perl in /usr/bin/perl,
please modify it as;– #!/usr/local/bin/perl– −>– #!/usr/bin/perl
As long as you have perl in /usr/bin/perl, you can also achieve the same result by runnig ascript• cd /where/you/want/to/install/irc/perl• perlpath.shwhich will create a directory temp containing perl ﬁles whose ﬁrst lines are modiﬁed as above.
Then you can overwrite these new perl ﬁles by typing;• mv ./temp/*.pl .
Those who do not have perl in /usr/bin/perl, please modify perlpath.sh and indicate thelocation of your perl.

5.4.8

Add IRC P3 entry to IRAF

• If you know root password and you installed IRAF package into for example /iraf,– cd /iraf/iraf/unix/hlib/– edit the ﬁle extern.pkg
Please add the following 2 lines to A /iraf/iraf/unix/hlib/extern.pkg,
reset irc p3 = /where/you/install/irc p3/
task $irc p3.pkg = irc p3$irc p3.cl
DO NOT FORGET THE ”$” BEFORE THE IRC P3.PKG!! DO NOT FORGET THE
LAST SLASH!!
• If you do not know the root password then you should launch iraf from your home directory
every time you want to use the irc p3 package.
– cd (change directory to your home directory)– mkiraf (only when using IRAF for the very ﬁrst time)IRC Data User Manual for Post-Helium Mission (Phase 3)

36

This will create the ”login.cl” ﬁle in your home directory. Then edit ”login.cl” and add 2lines:
reset irc p3 = /where/you/install/irc p3/
task $irc p3.pkg = irc p3$irc p3.cl
DO NOT FORGET THE ”$” BEFORE THE IRC P3.PKG!! DO NOT FORGET THE
LAST SLASH!!
the toolkit is now installed and is ready to be run !!

5.5

How to UPGRADE the version of IRC P3 imaging toolkit

Upgrades to the IRC imaging toolkit is a progressive and ongoing process and new versions of
the toolkit will be available to users throughout the course of the mission. On receiving a newversion of toolkit package (e.g., irc p3YYMMDD.tgz), the following commands are required to
be input;• mv irc p3YYMMDD.tgz /where/you/installed/previous/version/• cd /where/you/installed/previous/version/• tar xvzf irc p3YYMMDD.tgz
These commands will OVERWRITE any old ﬁles/directories. Then, follow the original
procedure described in 5.4.5 and 5.4.7. Finally, please type following commands when you use
a new version of the toolkit for the ﬁrst time.
• launch IRAF
• load the irc p3package by typing “irc p3”• type “unlearn all”
• load the irc tool package by typing “irc tool”

5.6

Setting up your toolkit environment and running the pipeline

The steps to reduce the raw data are outlined below.

5.6.1

Creating the directory structure

The toolkit assumes the following directory structure (see Fig.5.6.1);∼/anyname +——–rawdata (must be this name)——–anyname (your working directory)e.g.
∼/data/rawdata/∼/data/working/
The toolkit should be run in the working directory. Thus you may create sets of these
directory structures for diﬀerent sets of IRC data.

Version 1.1 (August 28, 2009)37

Figure 5.6.1: Example directory structure for IRC toolkit.

5.6.2

Launch IRAF

Launch IRAF in your home directory and check that you have an entry for the irc p3 packagein the list of the IRAF packages. If you install both the toolkits for phase 2 and phase3, both
irc and irc p3 should appear in the list of the IRAF packages (see Fig.5.6.2).

Here
Figure 5.6.2: Start up screen for IRAF showing the IRC package visible in the lsit of IRAF
packages.
Then you can move to your working directory before starting to run the diﬀerent tools by
typing:• cd to your working directory
where to your working directory should follow the rules explained in Section 5.6.1.

5.6.3

Load the IRC P3 package

The irc p3 package can be loaded by typing “irc p3”. You should then see the IRC pipelinsplash screen (welcome message) and a list of the available IRC packages (see Fig.5.6.3). The

38

IRC Data User Manual for Post-Helium Mission (Phase 3)

screen shows the version of toolkit, super-ﬂat, super-dark, linearity, and distortion. Please let
us know these numbers if you send any email to the help desk iris help@ir.isas.jaxa.jp to inquire
any trouble with the toolkit.

Figure 5.6.3: Start up screen for the IRC package showing the available modules and tools. The
start up splash screen gives the current version of the toolkit, and current versions of ﬂat, dark,
linearity, distortion ﬁles etc.

Version 1.1 (August 28, 2009)

5.7

39

The pre-pipeline processor (Red-Box)

Note that prepipeline processor is a one-time-only process. Do not run prepipeline twice.

5.7.1

Conﬁguration

Before running prepipeline, you can conﬁgure the parameter for prepipeline by typing “eparprepipeline”. This displays the parameter screen shown in Figure 5.7.4

Here

Figure 5.7.4: parameter screen for the Red-Box showing the available parameters.
Please do not change the parameter ”irccons”. Prepipeline has only one changeable parameter, ”verbose”. If you want to see verbose progress messages, change it to ”yes”. The default
is ”no”.

5.7.2

Running the prepipeline processor

The prepipeline processor consists of two functions.
• redbox.ircslice: slice IRC 3D images into usual 2D ones
• redbox.mkirclog: making the observing log ﬁle ”irclog”
The prepipeline processor is run by entering ”prepipeline” at the IRAF command prompt.
Alternatively, the individual Red-Box modules can be run by entering ”redbox” at the IRAF
command prompt. The present conﬁguration of the pre-pipeline modules is shown in Fig 5.7.5:

Figure 5.7.5: Present conﬁguration of the Red-Box pre-pipeline modules.
along with the current ﬂat, dark, distortion, and linearization versions, e.g. ;IRC Data User Manual for Post-Helium Mission (Phase 3)40
1. redbox.ircslice

The number of FITS ﬁles produced for any given single pointing will depend on the AOT.
Each AOT comprises of a combination of exposure cycles (EC), Filter Wheel changes(W) and Dither Maneuvers (M) (see Fig.2.2.4). The IRC FITS data is not a usual 2D
one. A raw data FITS ﬁle is created for each Exposure Cycle during a pointing for the
NIR channel. The ﬁlename format is given as F***** N.ﬁts, where ***** is a distinct
incremental reference number. A NIR raw ﬁts data ﬁle is a data cube containing 2 frames
within it, corresponding to one short and one long exposure.
In addition to the Exposure Cycles, 5 Dark frames are taken at the beginning and the endof the operation (i.e. extra 10 FITS ﬁles). Therefore, as shown in Figure 2.2.4 for AOT
IRCZ2, for a single pointing, the maximum number of raw data ﬁles will comprise of (1NIR/EC × 7 (EC) + 10(Dark) = 17 FITS ﬁles for one pointing. Consequently, for AOT
IRCZ3, for a single pointing, the raw data ﬁles will comprise of (1 NIR)/EC × 8(EC) +10(Dark) = 18 FITS ﬁles for one pointing. In addition to the raw FITS ﬁles, a text ﬁle isalso included giving details of the original target list for the observation.
The ”redbox.ircslice” module slices each raw FITS ﬁle into the individual frames (separates
all individual IRC frames for one Exposure Cycle). Thus for every NIR raw FITS ﬁle(including the Dark), redbox.ircslice will create 2 FITS ﬁles (corresponding to the shortand long exposure). Therefore after running ”redbox.ircslice” on a single pointing for AOT
IRCZ3 you can expect as many as 50 individual FITS ﬁles. The ﬁlename format takes theoriginal format with an extension deﬁning the channel (N) and frame number (001-002).
Fig. 5.7.7 shows the images of IRC raw data for the NIR bands. The orientation of theimages are such that the NIR is rotated by 90 degrees. Please note the dark areas ineach images. These areas are reserved for slit-spectroscopy, and they are supposed not toreceive any light. Therefore, these regions are useful to monitor the dark level. Hereafter,
we refer to this this dark area as the ”slit area” and the other as the ”imaging area”.
2. redbox.mkirclog
The ”redbox.mkirclog” creates the observing log ﬁle ”irclog”. This is a text ﬁle, with
content shown in Table 5.8.1. The contents of the irclog ﬁle summarize the nature of theprocessed ﬁles in the working directory. The table entries correspond;• FRAME: The ﬁlename corresponding to the ”sliced frame” (e.g. following the format
such as F23340 N001.ﬁts as described above)• OBJECT: Target name taken from the original target list
• NAXIS: Number of pixels in cross-scan direction (256 for MIR and 412 for NIR)
• FILTER: Filter name (i.e. N2,N3,N4 or DARK)• RA-SET: Right ascension coordinates• DEC-SET: Declination coordinates• AOT: AOT type (e.g. IRCZ2, IRCZ3 etc) item EXPID: Sequential frame number
during an exposure cycle (1 or 2)• IDNUM: Pointing ID• SUBID: sub-Pointing ID (greater than 1 for multi-pointing observations)Version 1.1 (August 28, 2009)41

1 NIR FITS image

N1: NIR short exposure frame
N2: NIR long exposure frame

Figure 5.7.6: IRC/NIR raw FITS ﬁle data structure.

Figure 5.7.7: Image of NIR raw data. The NIR raw FITS ﬁle is 2 frame deep (corresponding to
1 short and 1 long exposure).

IRC Data User Manual for Post-Helium Mission (Phase 3)

42

5.8

Before runnning the pipeline processor

After runnning ”prepipeline”, you will get 2 text ﬁles, namely ”irclog” and ”darklist.before”.
• irclog
The irclog ﬁle is used to select the data to be reduced and any unnecessary entries should
be removed from the irclog ﬁle, editing it by hand. Alternatively entries and unnecessarylines can be commented out by adding # to the head of the entries.
NOTE: the DARK frames and any grism/prism images need to be removed/commentedout before running the ”Green-Box” pipeline. This is automatically done by prepipeline.
In general, bad frames may cause a failure in the pipeline at a later stage downstream,
the best strategy may be an iterative run where, if a crash occurs, the individual frames
can be examined and the culprits excluded from future runs of the Green-Box. The data
frame taken during the maneuver can be also automatically discarded, so you need not
examine each frame and edit irclog.
• darklist.before
In a pointing observation, 5 dark frames are taken at the beginning and the end of theoperation in the phase 3. In the phase 3 data reduction, the self-dark created from thesedark frames is always used to estimate the dark current.

Table 5.8.1: Sample of the irclog ﬁleFRAME#F011131797 N001
#F011131797 N002
#F011131798 N001
#F011131798 N002
#F011131799 N001
#F011131799 N002
#F011131800 N001
#F011131800 N002
#F011131801 N001
#F011131801 N002
#F011131802 N001
#F011131802 N002
F011131803 N001
F011131803 N002
F011131804 N001
F011131804 N002
#F011131805 N001
#F011131805 N002
#F011131806 N001
#F011131806 N002
F011131807 N001
F011131807 N002
F011131808 N001
F011131808 N002
#F011131809 N001
#F011131809 N002
#F011131810 N001
#F011131810 N002
#F011131811 N001
#F011131811 N002
#F011131812 N001
#F011131812 N002
#F011131813 N001
#F011131813 N002
#F011131814 N001
#F011131814 N002
#F011131815 N001
#F011131815 N002

OBJECT”LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135””LMC P3 2C135”

NAXIS1
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412
412

FILTER
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
NP
NP
N3
N3
N3
N3
NP
NP
NP
NP
N3
N3
N3
N3
NP
NP
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK
DARK

RA-SET
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75252
82.75364
82.75364
82.75364
82.75364
82.74459
82.74459
82.74459
82.74459
82.73914
82.73914
82.73914
82.73914
39.74704
39.74704
34.73183
34.73183
31.62331
31.62331
28.98775
28.98775
26.45258
26.45258
24.12861
24.12861

DEC-SET-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.88891-65.8848-65.8848-65.8848-65.8848-65.88335-65.88335-65.88335-65.88335-65.88638-65.88638-65.88638-65.88638-48.87008-48.87008-43.88028-43.88028-40.16951-40.16951-36.5962-36.5962-32.7553-32.7553-28.86755-28.86755

AOT
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2
IRCZ2

EXPID
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2

IDNUM
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359
1900359

SUBID
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

Version 1.1 (August 28, 2009)

5.9

The pipeline processor (Green-Box)

5.9.1

Conﬁguration

43

Before running the pipeline, you can conﬁgure the parameters for the pipeline by typing “eparpipeline”

Figure 5.9.8: Screen for parameter conﬁguration for pipeline.
The pipeline has several parameters.
• irccons (string, default=constants.database):
NEVER CHANGE THIS. This is the irc constant ﬁle name.
• com mod (string, average/median, default=”median”):
Method of combination of frame images.
– average:– median:• com area (int, 1 or 2, default=2):
Coadded image area of stacked frames used for extraction (see Fig. 5.9.9, NOTE: althoughstrictly speaking a Blue-Box process but is run from the Green-Box Pipeline script).
– THIS PARAMETER IS NOT SUPPORTED YET, AND YOU WILLGET THE WHOLE AREA IMAGE EVEN IF YOU CHOOSE 1 TO GETTHE COMMON AREA.
– 1: Only the common area (red region) is extracted.
– 2: The whole area (areas within green dashed line) is extracted.
• sky are (int, 1 or 2, default=2):
Area of sky to be used for statistics to adjust sky level before coadding frames.

IRC Data User Manual for Post-Helium Mission (Phase 3)44– 1: Only the common area is used.
– 2: The whole area is used.
• det sig (real, min=1.0, max=1000.0, default=4.0):
Source detection threshold in sigmas.
• sig rej (real, min=0.0, max=100.0, default=3.0):
Rejection unit in sigmas.
• max itr (int, min=1, max=10, default=10):
Maximum number of iterations in statistical process.
• obslog (string, default=irclog):
NEVER CHANGE THIS. Output ﬁlename of redbox.mkirclog• wcs (boolean, yes or no, default=no):Try to match the 2MASS sources to calculate wcs? To use this function, you shouldhave internet connection to automatically download 2mass catalog from the internet. This
process will not be executed as default. Note; although the toolkit was checked on Solaris
machines also. Solaris does not have curl in the original applications. You needto install it to run WCS.
• selfdar (boolean, yes or no, default=no):
Use an alternative dark to the super-dark provided?
• coaddsh (boolean, yes or no, default=no):
Coadd short exposure frames as well as long ones?
• interac (boolean, yes or no, default=no):
option to run pipeline interactively
With this option selected, the pipeline can be run step by step, e.g. asGreenbox.anomalous pix
Greenbox.dark, etc
or alternatively by typing Greenbox then running from inside the Green-Box asanomalous pix
dark, etc.
• deltemp (boolean, yes or no, default=yes):delete any temporary created ﬁles?
• rejecti (string, noneminmaxccdclipcrrejectsigclipavsigclippclip, default=sigclip):Type of rejection operation performed in coadding.
– none : No rejection– minmax : Reject the nlow and nhigh pixels– ccdclip : Reject pixels using CCD noise parameters– crreject : Reject only positive pixels using CCD noise parameters– sigclip : Reject pixels using a sigma clipping algorithm– avsigclip : Reject pixels using an averaged sigma clipping algorithm– pclip : Reject pixels using sigma based on percentiles

Version 1.1 (August 28, 2009)45• rej sky (boolean, yes or no, default=yes):
activate if you want to reject any images which have outlier sky level from the coaddition
process. Some frames may be taken during maneuver, and the pipeline will detect suchframes by watchig their sky level.
• submeds (boolean, yes or no, default=no):
activate if you want to make a median ﬁltered sky image and subtract it from each image
before coadding. This option is useful for deep-imaging observation where the sky has no
structure.
• coadd (boolean, yes or no, default=yes):de-activate if you do not want to try to coadd images. This option is useful for users who
have their own coaddition strategy and software.

• default (boolean, yes or no, default=yes):This parameter deﬁnes which ﬂat is used in the toolkit. De-activate if you want to use yourown ﬂat images instead of using the toolkit-default ﬂats. In that case, you have to prepare
ﬂat images for three ﬁlters (N2,N3,and N4) and put them in /where/you/installed/irc/lib/ﬂat/user/.
Their ﬁle names should be named as those of the ﬂat images in /where/you/installed/irc/lib/ﬂat/soramamari/.
• verbose (boolean, yes or no, default=no):
activate if you want to print verbose progress messages

Figure 5.9.9: Area utilized for the Co-added images depends upon the parameter com area. The
2 options are common area (red region) and whole area (within green dashed region)

5.9.2

Running Green-Box pipeline module

The Green-Box pipeline processor can be run by entering ”pipeline” at the IRAF commandprompt. The Green-Box pipeline produces the basic calibrated data and currently consists ofthe following steps• greenbox.anomalous pix: Mask Bad/Dead pixels• greenbox.dark: Subtract dark current• greenbox.ircnorm: Normalization for sampling and data compression

46

IRC Data User Manual for Post-Helium Mission (Phase 3)• greenbox.cosmic ray: Detect and replace cosmic rays in MIR images.
• greenbox.linearity: Correct linearity of the detector response• greenbox.saturation: Mask saturated pixels• greenbox.slit mask: Masks the slit area of IRC ﬁeld of view• greenbox.ﬂat: Flatten the pixels• greenbox.aspect ratio: Distortion Correction (Aspect ratio resampling)

Alternatively, the individual Green-Box modules can be run interactively by 3 possible methods;
1. By setting the interac parameter in the parameter list
2. By running the pipeline with the command pipeline interactive=yes
3. By running the pipeline with the command pipeline interactive+
When running the pipeline interactively, individual steps may be entered asGreenbox.anomalous pix
Greenbox.dark, etc
or alternatively by typing Greenbox then running from inside the Green-Box asanomalous pix
dark, etc
When running interactively, there are options to perform/skip/stop each process. The present
conﬁguration of the Green-Box pipeline modules are shown in Fig 5.9.10:
Starting from the original input FITS ﬁle (e.g. F23342 N004.ﬁts). At each step of the GreenBox pipeline processing a qualiﬁer is added as a preﬁx to the original ﬁlename (note the original
FITS ﬁles are preserved). These preﬁxes build up over each processing step. The preﬁxes are
summarized in Table 5.9.2.
1. Mask bad or dead pixels: (greenbox.anomalous pix)
Bad or dead pixels were identiﬁed by using pre-ﬂight laboratory data. Mask ﬁles are storedin ircroot/lib/anomalous pix. They are shown in Fig. 5.9.11. After masking bad/hot pixels , an ”a” will be added as a preﬁx to the original ﬁlename such that F23342 S004.fits
becomes aF23342 N004.fits.

2. Dark Subtraction (greenbox.dark)
In a pointing observation, 5 dark frames are taken at the beginning (pre-dark) and theend of the operation. In phase 3 data reduction, the selfdark data created from these darkframes are always used because of the increase of the dark current and the number of hot
pixels.
Due to various reasons (1st frame eﬀect, and so on), the dark level may vary during apointing. Therefore we monitor the slit-area as a reference of the dark level, and we shift
the self-dark by adding/subtraction a constant to them and then subtracting this fromeach image to account for the changes in the dark level.
After the dark subtraction, a ”D” will be added such that aF23342 N004.fits becomes
DaF23342 N004.fits

Version 1.1 (August 28, 2009)47

greenbox.anomalous_pix
mask Bad/Dead pixels
greenbox.dark
subtract dark current
greenbox.ircnorm
normalization of sampling after data compression
greenbox.cosmic_ray
Detect and correct for cosmic rays
greenbox.linearity
Linearize detector response
greenbox.saturation
mask saturated pixels
greenbox.slitmask
Mask slit area of detectors
greenbox.flat
Flatten the pixels
greenbox.aspect_ratio
Aspect ration resampling

Figure 5.9.10: Present conﬁguration of the Green-Box pipeline modules.

48

IRC Data User Manual for Post-Helium Mission (Phase 3)

Figure 5.9.11: Mask ﬁle. Outlier pixels have a value of 0 (white) and others 1 (black). Anyoutlier pixels will be masked out by mutipling the mask ﬁle to images of concern.
3. Normalization: (greenbox.ircnorm)
To reduce the readout noise of the detectors the IRC data is read with the Fowler sampling
method (non-destructively reading and summing the array multiple times (determined by
the Fowler number) then dividing this resulting sum by the Fowler number. However,when the data is transmitted to the ground, for the purpose of data compression, theleast signiﬁcant bit is dropped. The normalization process accounts for this bit loss bycorrecting the data value by
Acorrected =Auncorrected ∗ 2bitshift
Fowler number(5.9.1)

Where the bitshif t represents the number of bit shifts emloyed.
After the normalization process a ”n” will be added such that DwaF23342 N004.fits
becomes nDwaF23342 N004.fits
nDaF23342 S004.fits becomes cnDaF23342 S004.fits
4. linearity correction: (greenbox.linearity)
The linearity correction is made as explained in section 4.3.
After the linearity correction, a ”l” will be added such that CnDaF23342 N004.fits becomes lCnDaF23342 N004.fits.
5. Saturation: (greenbox.saturation)
Any pixels that are marked as saturated are masked. Physical detector saturation occurs
around 2000 ADU (after ircnorm) for the NIR detector in the phase 3. Therefore, any
pixels with values greater than the ”scaled values” in the short exposure frame are pickedout and the corresponding pixels in the long exposure frames will be masked out to be 0.
The ”scaled values” in the exposure frames are calculated by
Sshort exposure in ADU = Sphysical saturation in ADU ∗Exposure Timeshort frame(5.9.2)
Exposure Timelong frameThis is done because the IRC uses Correlated Double Sampling and with the current IRC
operating clock, we cannot tell which pixels are saturated by long exposure frames alone.
After masking saturated pixels, a ”s” will be added such that lCnDaF23342 N004.fits
becomes slCnDaF23342 N004.fits

Version 1.1 (August 28, 2009)49

Table 5.9.2: Preﬁxes added during Green-Box Pipeline Processing
Preﬁx
a
D
n
C
l
s
m
f
e

modulegreenbox.anomalous pix
greenbox.dark
greenbox.norm
greenbox.cosmic ray
greenbox.linearity
greenbox.saturation
greenbox.slit mask
greenbox.ﬂat
greenbox.aspect ratio

actionMask Bad or Dead Pixels
Dark Subtraction
Normalization
Detect and Replace Cosmic Rays
Linearity Correction
Mask Saturated Pixels
Slit Mask
Flat Fielding
Aspect Ratio Resampling

6. Slit Mask: (greenbox.slit mask)
The slit mask processing masks the slit area in the IRC Field of View.
After the slit mask processing, a ”m” will be added such that slCnDaF23342 S004.fits
becomes mslCnDaF23342 N004.fits
7. Flat Fielding: (greenbox.ﬂat)
The ﬂat ﬁelding is made using the imaging super ﬂats, described in section 4.2. In thenear future, we plan to attach the most suitable ﬂat images to every distributed data set.
These ﬂats can then be selected by putting them in /where/you/installed/irc/lib/ﬂat/user
and deactivating the “default ﬂat” parameter.
After the ﬂat ﬁelding process, a ”f ” will be added such that mslCnDaF23342 N004.fits
becomes fmslCnDaF23342 N004.fits
8. Aspect ratio resampling: (greenbox.aspect ratio)
This is a distortion correction. The module the corrects aspect ratios of pixels to be square
i.e. Aspect ratio is corrected to 1 to 1 by resampling the image. .Thus the toolkit corrects
the linear distortion only. Non-linear distortion is not corrected for at present since it isassumed to be negligible.
After the resampling, a ”e” will be added such that fmslCnDaF23342 N004.fits becomes
efmslCnDaF23342 N004.fits
Therefore at the end of the Green-Box, correctly processed frame ﬁles should have aefmslCcnDwa
preﬁx before the original ﬁlename (i.e., F23342 N004.fits >>>efmslCcnDwaF23342 N004.fits).
The resulting data is the basic processed data with individual frames (not per pointing) corrected
for detector and instrumental eﬀects.

5.10

The pipeline processor (Blue-Box): bluebox.coadd wrapper Module

5.10.1The Blue-Box Co-Add Wrapper

It has not yet been decided whether an additional post-pipeline processing module will be
released to the users. The pre-requisite for the IRC data reduction team is at present solely todeliver basic science grade processed data to the users. It is not the responsibility of the teamto provide modules to produce mosaiced images or to provide tools for photometry, etc..
The output of the green box produces the basic processed data for individual frames. Additional processing is required to produce the basic science grade processed data. There exists

50

IRC Data User Manual for Post-Helium Mission (Phase 3)

a jitter between frames in the IRC images causing frames to become unaligned with each other(note this is not an intentional dither which is a separate procedure). The attitude of theseframes must be matched (for any shift and rotation in position) and stacked. This processing
is carried out by the modules called from the Blue-Box pipeline (Note this is not the same as
actually running the Blue-Box Post-Pipeline processor). In reality, this process occurs automatically at the end of the Green-Box pipeline processor. In order to do this, bright sources (stars)
are extracted from each frame as source lists. These source lists are then used to calculate theshift and rotation for each frame looking at a particular ﬁeld-of-view on the sky using the ﬁrst
frame as a reference.
The processing currently consists of the following steps almost entirely included in the bluebox co − add wrapper module;• Co-add individual frames: (bluebox.coadd)
This module calls further individual modules to coadd the image frames together. There isalso an alternative module that may be called to coadd the image frames for MIR-L using
the information from the MIR-S channel in the case that there are not enough bright stars
for an independent calculation of the shift and rotation of each frame.
– bluebox.source extract: Extracts bright reference sources– bluebox.calshift: Calculate shift and rotation value between image frames– bluebox.adjust sky: Adjusts sky level between individual frames– bluebox.irc stack: Stacks the IRC images• bluebox.putwcs: To add WCS information to an image FITS ﬁle by matching 2MASS
catalog coordinates.
The present conﬁguration of the Blue-Box co − add wrapper module is shown in Fig 5.10.12.

Figure 5.10.12: Present conﬁguration of the Blue-Box co − add wrapper module.
The starting point for the Blue-Box Co-Add wrapper are the ﬁles created from the ﬁnal stepin the Green-Box pipeline Processor efmslCcnDwaF23342 S004.fits. The Co-Add module thenperforms the following tasks:

Version 1.1 (August 28, 2009)51

1. Extract bright reference sources: (bluebox.source extract)
This module extracts individual bright sources from individual frames. Before extraction, original 412x512 NIR images are paseted to larger 1024x1024 images to prepare
for the xy shift and rotation. During source extraction, a ”S” will be added such that
fdmslCnDaF23342 N004.fits becomes SfdmslCnDaF23342 N004.fits. Bright sources (stars)
are extracted from each frame as source lists.
2. Calculate Shift & Rotation to match frames: (bluebox.calshift)
The bright reference star source lists are used to calculate the relative shift and rotation
values between individual image frames looking at a particular ﬁeld-of-view on the sky using the ﬁrst frame as a reference. After this shift and rotation process, the original frame
ﬁle names will receive additional ”R” preﬁxes such that SfdmslCnDaF23342 N004.fits becomes RSfdmslCnDaF23342 N004.fits. There is a parameter minmatch that determines
the minimum number of stars required for matching between 2 frames. If there are notenough stars then a warning will be given and a log ﬁle coadd.failure created containing
the names of the processed frames that could not be combined. A log ﬁle calcshift.log
is also produced listing the number of stars matched between and the corresponding x &y oﬀset for included frames.
3. : Adjust Sky Level: (bluebox.adjust sky)
The bluebox.adjust sky module collects all frames looking at a given area on the sky
and calculates the median sky from each image. Each calculated median value is thensubtracted from each individual corresponding frame. From these median values an averagesky brightness is calculated that is then re-added to all the frames. A log ﬁle is produced
during this process with a name skypair0002 N2.list long or similar. This ﬁle lists theframes looking at a given area on the sky with the corresponding mean, median and modesky brightness and 1σ standard deviation. Those who want to subtract median box car
ﬁltered image instead of adjusting sky level should conﬁgure the following parametes bytyping ”epar adjust sky” before running pipeline.
• submeds (boolean, yes or no, default=no):
Subtract median ﬁltered sky?
• rmmedsk (boolean, yes or no, default=yes):
Remove median ﬁltered image? Those who want to check box car median ﬁlterd
image, set this parameter no.
• x box: (integer, min=1, max=100, default=21):
x box car size
• y box: (integer, min=1, max=100, default=21):
y box car size
The parameters x box and y box change the size of the median kernel.
After adjusting the sky level, the ”R” preﬁx is removed and an ”A” is added, such that
RSfdmslCnDaF23342 N004.fits becomes ARSfdmslCnDaF23342 N004.fits.
4. Image Stacking (Co-adding): (bluebox.irc stack)Once every frame pointing at the same area of sky in the irclog list has been correctly
matched and the sky brightness adjusted, every frame is stacked to produce the ﬁnal coadded image. The FITS images and ﬁles created from the stacking process can be foundwithin a new directory stacked IM. The stacking process creates 3 ﬁles for any given ﬁlter
position on the sky;52

IRC Data User Manual for Post-Helium Mission (Phase 3)• A co-added image ﬁle (e.g. 1757132 N2 long.fits).
• A noise map (e.g. sigma1757132 N2 long.fits)• Summary File (e.g. pl1757132 N2 long.fits.pl)

Figure 5.10.13: Example of ﬁnal science grade data produced by the Green-Box and Blue-Box
Co-Add Wrapper. lef t is the stacked image map. right is the corresponding noise map.
5. Convert to WCS coordinates: (bluebox.putwcs)
After the Co-add wrapper has been completed, the module bluebox.putwcs adds WCS
information to the FITS images by matching with 2MASS catalog coordinates using thefollowing procedure;(a) source extraction of stars from the IRC image(b) download 2MASS catalog for the observed area(c) convert 2MASS RA DEC to the x’y’ image coodinates(d) match 2MASS x’y’ coordinates with image xy coordinates
At this point there will be a list for extracted IRC stars corresponding tostar1 x1’ y2’ ra1 dec1 x1 y2
star2 x2’ y3’ ra2 dec2 x2 y2.... .... .... ....
starN xN’ yN’ raN decN xN yN
for the matched stars. The module then calculates a transformation matrix (Nthorder polynomial) that will convert xy to RA and DEC. The module attempts to ﬁnd
the best ﬁtting solution by increasing the order of plynomial from 2 to 4.Usually, thebest result should be 2nd order, as long as the image has been corrected for distortion.
(e) (if matched) put wcs in the header
Note that it is possible to shorten the time to download the relevant 2MASS catalog
by changing the download mirror in the toolkit ﬁle irc/perl/get2mass.pl . Find theline that determines the data server where the 2MASS catalog will be downloaded. The
default is Tokyo, Japan. To choose the nearest server, un-comment the relevant U RL andcomment out T okyoJapan.

Version 1.1 (August 28, 2009)53

Note for multiple pointings and deep imaging data, many pointings of data may have to
be co-added. In such cases, all the images must be rotated, and the toolkit will have totake care of this before co-adding.
The ﬁnal result is the basic science grade images (see Figure 5.10.13).

5.10.2

Log ﬁles produced from the toolkit

In addition, there are various log ﬁles produced by the toolkit which are described below• coo Files The .coo ﬁles e.g. SfdmslCnDaF23342 N004.fits.coo.1 are the results forrobust source extraction that extracts x and y coordinates & robust photometry. One.coo ﬁle is created for each individual exposure frame. These ﬁles can then be used to
calculate the shift and rotation of frames that are being co-added by matching bright stars
in the extracted source lists.
• Pair Files The pair ﬁles collect together and list all (long and short) frames that point thesame area of sky.
Taking the N3 band as an example the following ﬁles may be seen,
pair0001 N3.list
pair0001 N3.list short
pair0001 N3.list long
pair0001 N3.list long.shift
pair0001 N3.list long.shift.0
The pair0001 N3.list ﬁle lists all (long and short) frames that point at the same area ofsky. The contents of the ﬁle are the original frame names (e.g. F23343 N001), IRC channel
and the coordinates. The ﬁles pair0001 N3.list short & pair0001 N3.list long are
extracted and segregated from F23343 N001 on the basis of the exposure time (long or
short exposure frames). The ﬁle pair0001 N3.list long.shift lists the result of thecoordinate matching before coadding. Each line entry in the ﬁle consists of a ﬁlename, xshift(pix), y-shift(pix), rotation(deg) and the number of stars used for the shift & rotation
angle calculation. For example;
fdmslnDwaF01001 N003.fits.coo.1 0.0 0.0 0.0 0
fdmslnDwaF01007 N003.fits.coo.1 19.25942609 -0.00474931 359.99288567 126
The values are calcualted relative to the ﬁrst frame listed in the ﬁle, such that the ﬁrst
entry is always filename 0.0 0.0 0.0 0.
There may also be an additional pair ﬁle pair0001 N3.list long.shift.0 which is almost
the same as the pair0001 N3.list long.shift, but it also contains detector information,
as ﬁlename, detectorID, x-shift(pix), y-shift(pix), rotation(deg) and the number of stars
used for the shift & rotation angle calculation, i.e.
fdmslnDwaF01001 N003.fits.coo.1 1 0.0 0.0 0.0 0
fdmslnDwaF01007 N003.fits.coo.1 1 19.25942609 -0.00474931 359.99288567 126• exp.input ﬁles These ﬁles contain a list of all the pair ﬁles for all ﬁlters pointing at one
position on the sky. There may be ﬁles for both the long long exp.input and short
short exp.inputexposure frames.
• Skypair Files These log ﬁle can be found inside the separate directory logs. This log ﬁleis produced during the adjustment of the sky level between frames (bluebox.adjust sky)

IRC Data User Manual for Post-Helium Mission (Phase 3)54

with a name skypair0002 N2.list long or similar. This ﬁle lists the frames looking at agiven area on the sky with the corresponding mean, median and mode sky brightness and1σ standard deviation.
• calcshift.log This log ﬁle can be found inside the separate directory logs. The calcshift.log
ﬁle is the log ﬁle for the calculation of shift and rotation in the bluebox.calshift process.
Relative to the ﬁrst frame in the pair ﬁle described above the log ﬁle gives the number ofstars matched between 2 .coo ﬁles and the corresponding x & y oﬀset for these frames.
The calcshift.log ﬁle will also record any matching failures.
• darklist.before

5.11

Toolkit structure

Version 1.1 (August 28, 2009)

5.12

55

Working on the output

Users will receive both the original raw data and processed data up to, but not including theco-add stage for the toolkit. The data is is basic, processed with the default parameters of thetoolkit and users are advised to re-process using the latest version of the IRC toolkit.

5.13

Limitations of the functionalities in the current version of
the imaging toolkit

5.13.1

Instrumental characteristics and artefacts in the data

• First frame eﬀect: The ﬁrst frame of the dark current prior to the pointing observationshows larger values than others. This eﬀect could be related to the detector temperature.
• Increase of hot pixels: The number of hot pixels is increasing along the mission. They
should be corrected when coadding individual images or correcting the image by its own
dark image.
• Memory eﬀects: No noticeable eﬀects have been reported• Dark Level: The dark level is not stable during the pointing observations. The NIR darklevel shows no correlation with temperature.
• Eﬀects of high energy particle hits: The toolkit does not perform any deglitching.
Glitches should disappear when coadding individual images (median ﬁlter). Even so, theireﬀect on the responsivity are not yet well investigated. Future versions of the toolkit will
involve a more careful treatment of cosmic ray hits.
• Ghosts: Ghosts originating from the beam splitter are recognized. The positions, which
depend on the real source positions, are well determined for all detectors and also depends
on the ﬁlter. The intensity of the ghosts is well determined (a few %).
• Background Level: The background level changes during the pointing observations inall bands. It is due to the Earth shine reﬂection, therefore it depends on the angle between
satellite and Earth and the epoch of the observation. The eﬀect from the Earthshine isworse at northern ecliptic latitudes during June, July & August. The Background level is
diﬀerent at the begining and at the end of the observation (see Fig 5.13.14), at diﬀerent
pointings (see Fig 5.13.15) and in diﬀerernt seasons (strongest in solstices). Note that thelatter ﬁgures are in the MIR, but similar variations should be present in the NIR.
• Earthshine in imaging observations: Data taken in May, June and July may suﬀer
from the Earthshine problem. The following recipe should be used:– Check the drift of the background level during a pointing and comment out any highbackground frames from “irclog” and run the pipeline again.
– use rej sky=yes option in order to ignore the bad frames:cl> pipeline rej sky=yes– use submedsky option to remove the diﬀuse background:cl> pipeline submedsky=yes # use for point sources only...
– adjust the median kernel size in the submedsky option:
bluebox> epar adjust sky(x box = 20) If submedsky=yes, x box car size

IRC Data User Manual for Post-Helium Mission (Phase 3)

56(y box = 20) If submedsky=yes, y box car size
• Problem with short exposure frames with IRC: At present no reliable estimate of
the sensitivity in the short-exposure frame data is obtained. They can be used to checkthe saturation in the long-exposure frame data, but should not be used to derive the ﬂux
of bright objects.

Figure 5.13.14: Change in the background level in a pointing observation

5.13.2

Toolkit Limitations and Caveats

5.14

Error messages when running the pipeline and Troubleshooting

1. The pre-pipeline stage sometimes give an error message such as ls: ../rawdata: Nosuch file or directory or cat: slice tmp0: No such file or directory, etc, although the pre-pipeline seems to run correctly. Typing unlearn all should clear theseerror messages.

Version 1.1 (August 28, 2009)57

Figure 5.13.15: Background levels
2. The distortion processing (green box processing) can give the following error message,
causing the pipeline crash:### DISTORTION ###Making the input file list...
Correcting distortion...
#This may take a while...
ERROR on line 56: Cannot open file(∼/iraf/irc/lib/distortion/DARK distortion database.dat)
distortion (ircconst=constants.database, logfile=irclog, prefixs=mslnDwa,
verbose=no)pipeline ()This is because the DARK entries should be removed from the irc log ﬁle.
3. The coadd stage can result in the following error:### COADD ###Making the input file list...
Extracting sources...
Calculating XY shift...
Adjusting sky level...
Coadding images...

58

IRC Data User Manual for Post-Helium Mission (Phase 3)
ERROR on line 148: parameter ‘direction’ not foundThis problem may be caused when using versions of IRAF earlier than IRAF 2.12.2. The
problem is with geoxytran task. eparimages.immatch.geoxytran can be typed in the
IRAF shell to see if the parameter exists. Even after installing the newer version of IRAF
it is necessary to type unlearn geoxytran before the toolkit runs correctly.
4. The pre-pipeline run can apparently miss some perl scripts and end with an empty or non
existing irclog ﬁle.
### MKIRCLOG ###Making the file list...
Reading header...
#This may take a while...
tcsh: ∼/iraf/irc/perl/formatlog.pl: Command not found.
tcsh: ∼/iraf/irc/perl/checkname.pl: Command not found.
### MKIRCLOG finished!! ###Primarily, it has to be checked that the toolkit is being run from the working directoryand not, for example in ∼ /iraf /. If the problem persists then it may be because perl
is in the wrong place! P erl should be in usr/local/bin/perl, therefore a symbolic linkshould be set up from the perl library to usr/local/bin/perl (type whichperl to ﬁnd
out where perl is currently hiding).
5. If a new version of the toolkit crashes, as a ﬁrst ﬁx, unlearn all should be typed within
irc.
6. The IRC toolkit does not run on the new Intel Mac machines, due to diﬀerences in the
IRAF build in the intel binaries. Replacing the intel binaries with the original binaries
causes iraf to run under emulation but solves the problem. IRAF 2.13 has been successfully
run on the intel machines.
7. The coaddition process or the coaddLusingS function, can cause the crash of the toolkit
with the error:
ERROR on line 128: Attempt to access undefined local variable ‘filter’.
This error will occur if the pre-pipeline is attempted to run twice on a given data set. As acheck the ﬁles listed in the pair ﬁle should be examined to see if they have the appropriate”FILTER” keywords in their header.
8. The “Matching failed” error message that could appear in the putwcs process is not an
intrinsic pipeline error. It is due to the fact that the 2MASS data format changes fromtime to time. Appropriate updates or patches are made to the pipeline. The updates will
be recorded in the Observer’s page.

Chapter 6

Spectroscopy pipeline cookbook
The spectroscopy pipeline is being developed and maintained by the IRC spectroscopy data
reduction team. It is mostly written in IDL, and uses the IDL ASTRO library (maintained at
the GSFC) among others. Although it is developed under Linux environment, it may be portableto other platforms after some modiﬁcations (though we have no plan to increase the exportable
platforms by ourselves). The toolkit also uses the ds9 FITS viewer, and the xpa package forcommunication between the ds9 and the IDL main program.
The spectroscopy data reduction requires a calibration database (FITS images and ascii
tables) distributed and updated by the IRC spectroscopy data reduction team. The databaseis based on observations of calibration objects and calibration frames taken during PV/DT
phase observations, as well as pre-launch calibration experiments in our laboratory. Therefore
observers do not have to make their own calibration observations.
The reduction of the data of phase 3 observations requires special care and tips. The
new pipeline software (IRC SPEC TOOLKIT P3) has been developed for the phase 3 data reduction, which is designed to work also for the phase 2 data. However, at the present, theveriﬁcation of IRC SPEC TOOLKIT P3 has not yet been completed. It is recommended thatIRC SPEC TOOLKIT P3 should be used exclusively for phase 3 data, whereas for phase 2 data
the previous IRC SPEC TOOLKIT be employed. It should also be noted that the basic usage of
IRC SPEC TOOLKIT P3 is exatly the same as IRC SPEC TOOLKIT. We provide special commands
in IRC SPEC TOOLKIT P3 to work with phase 3 data. Therefore, in the followings the chaptersfor the phase 2 data reduction remain, which include those not relevant to the phase 3 data
reduction, such as MIR-S and MIR-L data reduction. Users who are interested only in thephase 3 data can skip those descriptions. In addition, a chapter special for the phase 3 data
reduction is added at the end (section 6.9). Users who work on the phase 3 spectroscopic data
should read this chapter carefully.

6.1

General overview of the pipeline processing

The main pipeline processing consists of several well deﬁned steps, which are explained in thefollowing sections.
In general terms, the IRC slitless spectroscopy diﬀers from more standard slit spectroscopy
in that in the latter case, the pixel position and the wavelength are closely correlated. The IRC
spectroscopy does not follow this relation and needs to be applied in three steps:• Many spectroscopic ﬂats have been combined to form a ”super-ﬂat” which can be used toremove the pixel to pixel variation of the monochromatic response of the detector.
• After subtracting the background, extracting the sources and performing the wavelength
calibration, the wavelength ﬂat can be used. The wavelength dependency of the ﬂats
59

IRC Data User Manual for Post-Helium Mission (Phase 3)

60

is due to the same mechanism to that of imaging mode, and is corrected by interpolating/extrapolating the imaging super-ﬂat taken with two diﬀerent ﬁlters for every pixel andthe wavelength (colour correction).
• Finally, the spectral response speciﬁc to the spectroscopic mode observation is correctedby the spectral response function obtained from the observations of standard stars. Moredetailed explanations can be found in following subsections.

6.1.1

Dark subtraction

Scaled super-dark images are subtracted from the raw data. For the scaling, dark count oﬀset
is calculated within the pipeline, by comparing average counts at the slit-mask-covered portionof the pre-dark image, and the corresponding area on the super-dark image.
Additionally the average dark counts of the mask-covered portion of each observed sub-frame
is measured, except for LG2 for which there is no good dark area on the spectroscopy images formeasuring the dark level. The averaged oﬀset is added to (or subtracted from) the super-dark
to make the scaled super-dark.

6.1.2

Linearity correction/Saturation Masking

Linearity correction is made following the method adopted for imaging data reduction.
The saturation masking is also performed as in the imaging pipeline: the toolkit ﬁrst checks
the short-exposure images, and ﬁnd regions where expected counts in longer-exposure frame
exceeds the saturation limit based on the exposure time ratio of short-/long-exposure frames(see table 4.6.2). Then, the regions are masked in the longer-exposure frames. The user should
check the masking in the edges of the saturated regions, due to the drift between short and longexposures pointing.

6.1.3(monochromatic) Flat-ﬁelding

In the case of the slit-less spectroscopy, the entire FOV is the aperture for the backgroundsky, but the object image itself is the aperture for the object. This aperture size diﬀerence
makes diﬃculties in ﬂat-ﬁelding the slit-less spectroscopy images since, unlike a conventional
long-slit spectrograph, a given pixel can be illuminated by background photons with a rangeof wavelengths within the disperser’s passband, and the mixing fraction of photons of diﬀerent
wavelengths depending on the spatial distributions and spectra of sources. We need 3D ﬂat
information (X, Y along space, plus Z along the wavelength) for full ﬂat calibration. However it
is impossible to obtain such a detailed ﬂat since there are no good monochromatic ﬂat light nor aseries of narrow-band ﬁlters covering the passband of the disperser. Because of these diﬃculties,
we will make a ﬂat calibration in two steps, starting with the monochromatic correction.
To correct for the pixel-to-pixel variation of the monochromatic response (response or ﬂatthat does not depend on wavelength), the whole image is divided by the ’super-ﬂat’ . The
super-ﬂats are made by combining a larger number of background spectroscopy images. Notethat the background sky image is not ﬂat along Y or the wavelength axis, i.e., there are several
spectral features on the ’super-ﬂats’ due to, e.g., contributions of zero-th or higher-order lightin cases of grism spectroscopy images. Although the features should be removed to correct forthe pixel-to-pixel variation of the response, we ﬁrst divide the dark-subtracted images by thesuper-ﬂat with the spectral features. As shown below in equation 6.1.1, this process makes the
background ﬂat, facilitating the sky subtraction.
Assume that the dark-subtracted images are represented by the following equation:
obs = sky × F1 (x, y) × spectral f eature(x, y) + obj(λ) × F2 (x, y, λ) × R(λ)(6.1.1)Version 1.1 (August 28, 2009)61

where F1 (x, y) is the ﬂat in which the background spectral features are not present, F1 (x, y) ×
spectral f eature(x, y) is the ’super-ﬂat’, F2 (x, y, λ) is the wavelength-sensitive ﬂat (see below
for color-term correction), and R(λ) is the spectral response. Since the super-ﬂat is only afunction of the position in the detector, this step is called monochromatic ﬂat-ﬁelding. If we
divide the observation by the ’super-ﬂat’, we obtain:
obs = sky + obj(λ) × F2 (x, y, λ) × R(λ)/[F1 (x, y) × spectral f eature(x, y)]

6.1.4(6.1.2)

Background subtraction (from individual sub-frames)

After monochromatic ﬂat-ﬁelding, the background should be ﬂat over the aperture. However,in real data, this is not occurring in most of the cases. For NP, NG, and SG1 , outlier-tolerant
low-order polynomial surface ﬁtting is performed for each sub-frame, and is subtracted oﬀ fromthe ﬂat-ﬁelded images. For SG2 and LG2, background suﬀers from straylight (from the Earthshine) contamination, and the ﬂat-ﬁelded images are much more structured. To make good sky
subtraction, a median running-box ﬁlter is applied to estimate the local sky, and is subtracted
oﬀ from the ’stray light-covered’ background. Such background should be removed from individual sub-frames, since the contribution of the Earthshine depends on the telescope pointing
angle from the earth direction, and the amount of straylight changes within a single pointing
maneuver (or within a single AOT operation).
In the ﬁrst pass of the processing within the pipeline, source masking will not be applied indetermining the background, because no source position information is available at the stage. In
the second pass, after processing through section 6.1.9 once, the background is measured again
while considering the source masks.
The image obtained after background subtraction has the following expression:obs = obj(λ) × F2 (x, y, λ) × R(λ)/[F1 (x, y) × spectral f eature(x, y)]

6.1.5(6.1.3)

Image screening

Some sub-frames may be severely damaged by cosmic-ray shower, a satellite passing in front of
the telescope, etc. If this is the case, one can interactively set the ﬂag, by using ds9, for any
sub-frames to be discarded in the following processes.

6.1.6

Image registration (among sub-frames)

Relative image shift due to satellite attitude drift among spectroscopy sub-frames is measured
by means of cross-correlation technique. This can be made only with NP and NG due to thenumber of spectra present in NIR images. For SG1, SG2, and LG2, pixel oﬀsets measured with
NP/NG are used for matching, rather than measuring their own shifts, since all cameras takeimages simultaneously.
Similar shift-and-add technique is also needed for the reference sub-frames, except for N3,
where there is only a single reference sub-frame. To ﬁnd the shift values for L18W, we ﬁrst detect
point sources in S9W , measure their positions, and calculate the shift by using the list of target
coordinates, and shift both S9W and L18W images.

6.1.7

Imaging stacking

After registering all the sub-frames, all the selected (screened) sub-frames are combined to makehigher S/N stacked images. For spectroscopy images, a three-sigma clipping averaging method

62

IRC Data User Manual for Post-Helium Mission (Phase 3)

is used. For reference images, a simple median averaging is used due to the small number ofsub-frames.
Some sub-frames are removed due to satellite maneuver. When the toolkit recognizes that thelast exposure frame (cycle) was taken during this maneuvering period, the frame is automaticallyremoved from the stacking list. Note that, although the observing manual says that 8 sub-frames
will be taken for spectroscopy in AOT04 operation, it turned out that one additional frame (9thframe) can often be taken without suﬀering from satellite maneuvering. You may add the frame
for stacking.

6.1.8

Target detection/position measurement

Target positions can be provided by the user in the form of a source table (see below), or
automatically computed within the toolkit, using the daofind method for target detection. Even
if the target positions are set by users, the toolkit performs re-centering of the source positions
by means of Gaussian peak search. (This functionality can be disabled with the command line
option.) This process is coupled with the following step (background subtraction).

6.1.9

Background subtraction (from stacked image)

Although background subtraction has been already applied before stacking images, any remaining background is removed at this step.
The background subtraction and target detection/position measurement (explained in theprevious section) are made iteratively in the following way. First, the target is tentatively detected by the automatic source detection program (not based on the user-supplied source table),and source masks will be created for all detectable sources. Then the background is measured
while considering the source masks, and is subtracted oﬀ from the original stacked image. The
background-subtracted image is used for better source detection/position measurement, andbetter source masks will be created for better background subtraction.
When sources are detected, source masks are created for better background subtraction fromindividual sub-frames (section 6.1.4) in the second pass of the pipeline processing.

6.1.10Extracting 2D spectra

By using the reference image positions and pre-deﬁned coordinates oﬀsets in the calibration
database, rectangle areas around the source spectra are extracted on the spectroscopy images.
For NP, spectral distortion (see below) is taken into account along the dispersion direction toﬁnd best Y oﬀset when extracting 2D spectra.
X-oﬀset adjustment in spectroscopy image extraction
In real data, one need to further adjust the oﬀset of the source extraction boxes. One may sometimes ﬁnd 2D spectra slightly away from the expected position (at the center of the extractionbox) along X axis. This kind of shift cannot be corrected in the previous image registration
processes, and the correction is made at this stage. The center position of the 2D spectra ismeasured for each extracted 2D spectra, and the mean X oﬀset from the center of the extraction
box is calculated. If the pipeline successfully ﬁnds the shift value, the shift will be applied in
extracting the 2D spectra. However, for some cases where only very faint objects were detected,
calculating this additional shift may fail (and no further shift is applied).
Measuring the X oﬀset and making good source masks (as explained in the previous section)
are closely related to each other during the pipeline processing. In the ﬁrst pass of the source
detection, the automatic source detection sub-program works even if the target table is supplied

Version 1.1 (August 28, 2009)63

in the pipeline command line. The X oﬀset is measured with the tentatively detected sourcesat this stage. In the second pass, measured X oﬀset (as well as Y oﬀset found in adjusting
the wavelength zero reference point; see the next section) between spectroscopy and reference
images is taken into account in extracting the 2D spectra. Only the sources speciﬁed in thesource table, or sources detected with the sub-program, will be processed (extracted) in thesecond pass. This two-stage process ensures good source masking for better sky subtraction andgood oﬀset measurement with brighter and larger number of sources, even for the cases in which
the observer is only interested in small number of faint sources.
Measurement of the center position of the 2D spectra can be made easily for NP, NG, and SG1
spectra for which larger number of brighter sources can be detected even in blank sky regions.
Therefore, the toolkit measures the X oﬀset at NP and NG for correcting both NP and NG, andat SG1 for correcting SG1, SG2, and LG2. Therefore, the user must start pipeline processing for
NP/NG ﬁrst, then SG1, and ﬁnally SG2 and LG2. See section 6.4.1 for more explanations.

6.1.11

Wavelength calibration

In the case of grism spectra, the dispersion equation is almost linear theoretically and a linear
equation for expressing the wavelength is assumed. Therefore, once the 2D spectra extraction
is made with suﬃcient accuracy, it is straightforward to make a wavelength calibration.
For prism (NP) spectra, since the dispersion equation is highly non-linear one especially atshorter wavelength end, it is better not to transform images to avoid introducing extra uncertainties in the image interpolating/extrapolating processes. Rather, a separate array (whose lengthis equal to the Y length of the extracted 2D spectra) will be created to store the wavelength
values for each Y pixel. One wavelength array is applicable for all the extracted spectra within
the FOV. This kind of wavelength array is also used for grism spectra (NG/SG1/SG2/LG2).
Finding wavelength zero reference point
Some problems arise for accurate wavelength determination. One is the satellite attitude stability
problem. If the satellite pointing moves between reference and spectroscopy images, the reference
image does not provide good wavelength zero reference point any more.
The NP spectrum (before ﬂux calibration) shows a notable peak around 2.4 μm, and there arelots of fairly bright stars with almost identical spectra, regardless of the type of stars within asingle FOV. Therefore it is possible to ﬁnd the best oﬀset of the wavelength zero reference point,
by measuring the NP peak positions with respect to the spectral template that was taken when
the satellite attitude stability was good enough, by averaging the oﬀsets measured for manystars. Once the oﬀset is measured, extraction of the 2D spectra is made again, after considering
the Y oﬀset (as well as X oﬀset as explained earlier).
The zero reference point is much diﬃcult to be found in NG spectra, due to lack of notable spectral features before applying the ﬂux calibration. Therefore, although the toolkit
can estimate the reference point by examining the observed spectroscopy images, it is strongly
recommended to check the zero-th order light images for more accurate wavelength calibration.
The chance of detecting zero-th order light image at signiﬁcant level for SG1, SG2, and LG2
is not so large. Therefore the drift of the wavelength zero reference point is calculated by using
the drift measured in NP or NG for MIR-S/L grisms, after correcting the pixel scale diﬀerence.
Another issue related to measuring the wavelength zero reference point is the ﬁnite pixel
resolution. Although the source positions can be measured with an accuracy of less than one
pixel size unit on the reference image, the extraction of 2D spectroscopy images can only bemade on integer pixel number to avoid erroneous image interpolation. This means that a pixel
error as large as ±0.5 could be introduced in the wavelength calibration process if not corrected,
and is not so small comparing with the full length of the dispersed spectroscopy images (∼ 50

IRC Data User Manual for Post-Helium Mission (Phase 3)

64

pixel). As a ﬁrst-order correction, we shift both the wavelength array and spectral response
curve, both of which should show rather smooth change along Y (or wavelength) axis, and not
perform sub-pixel shifting of the images. As a result, since object positions change slightly
among diﬀerent pointing observations, the wavelength at the same Y pixel of the extracted 2D
spectra (or the wavelength array) also changes with diﬀerent pointing observations.

6.1.12

Flat color-term correction

The presence of signiﬁcant color variation in the ﬂat images can be found in the ratio images ofthe broad-band ﬂats (e.g., S7 ﬂat / S15 ﬂat). Therefore, although monochromatic ﬂat-ﬁelding
can ﬂat the background, the object spectrum is aﬀected by the color term of the sensitivity. The
correction for this could be done after extracting the 2D spectra for each target and applying the
wavelength calibration. However, since we have only two broad-band ﬁlters for MIR-S/MIR-L(S7 and S15 for MIR-S, and L15 and L24 for MIR-L) and three for NIR (N2, N3, and N4), we
can derive only global trends of wavelength-dependence of the ﬂats. (Note that the wide-band
ﬁlters, S9W and L18W, are not suitable for deriving the color dependence of the ﬂat within the
spectral coverage of the channel). Two broad-band ﬂat images are interpolated to estimate theﬂats for a given wavelength in the following way:F2 (x, y, λ) = [F (x, y, λ2 ) − F (x, y, λ1 )]/(λ2 − λ1 ) × (λ − λ1 ) + F (x, y, λ1 )(6.1.4)

where λ2 and λ1 are eﬀective wavelength for broad-band ﬁlters.
Therefore the color term in equation 6.1.3 is expressed as:

F2 (x, y, λ)=spectral f eature(x, y) × F1 (x, y)

F (x,y,λ2 )−F (x,y,λ1 )× (λ − λ1 ) + F (x, y, λ1 )λ2 −λ1spectral f eature(x, y) × F1 (x, y)(6.1.5)

Therefore the color-term correction is calculated by two broad-band super-ﬂats and one spectroscopy super-ﬂat. Note that the product F1 (x, y) × spectral f eature(x, y) always appears
together, i.e., we do not have to separate spectral f eature term from the ’super-ﬂat’.
After the color-term correction the image is as follows:obs = obj(λ) × R(λ)(6.1.6)

For the NG spectra with the point source aperture (Np), ﬂat-ﬁelding will be made in a similar
way to the slit spectroscopy of diﬀuse sources, since the aperture size is much smaller than thesize of dispersed spectroscopy images. For spectroscopy with NG at Np and other slit spectroscopy
data calibration, the super-ﬂat is normalized to be unity at each wavelength (or Y) pixel, andthere are no spectral features in the ﬂats. On the other hand, NG spectra at Nc will be reducedin the same way as for other slit-less spectroscopy.

6.1.13(local) Background subtraction (from extracted 2D spectra)

Although the background has been subtracted and it should be close to zero at this stage, any
remaining background on the extracted 2D spectroscopy images is subtracted at this stage. Herewe consider source masks for better determination of the local background. The background ismeasured and subtracted oﬀ, assuming a constant value across the images for all dispersers.

Version 1.1 (August 28, 2009)

6.1.1465

Spectral tilt correction

Although we do not reform the image ’shape’ along the wavelength direction during the courseof wavelength calibration because of simple linear grism dispersion, we do reform along X (orspace) direction to correct spectral tilt. The tilt occurs due to misalignment of grism insertion
direction with respect to chip orientation. The tilt is notably seen in NG images in which a pixel
shift as large as ΔX 2 is observed over the longer dispersed image length (ΔY 250 pixel).
Similar tilt is also observed for other prism/grisms (NP, SG1, SG2, and LG2), although the tilt
looks very small due to short dispersed image length. However, a tilt correction will be madefor spectra of all types of dispersers.

6.1.15Spectral response calibration

The spectral response calibration R(λ) depends exclusively on the wavelength. Therefore theresponse calibration table is a 1D vector (lambda vs. ADU/s/mJy). This means that response
variations of individual pixels have been removed beforehand by ﬂat-ﬁelding processes. Being
diﬀerent from the spatial ﬂat correction, the spectral response varies signiﬁcantly with wavelength due to change in quantum eﬃciency of chip and change in optical transmission along thecamera optical trains, including the disperser.
The observed wavelength-calibrated 1D spectrum in ADU is just divided by the spectral
response table. Since the wavelength for a given pixel in the observed spectrum is slightly
diﬀerent from that in the response table, simple interpolation will be performed to match thewavelength of the observed wavelength-calibrated spectrum with that in the response table.
The spectral response function used in the pipeline was deﬁned at the centre of the ﬁeld ofview and checked at the periphery of the ﬁeld for consistency.

6.1.16

Notes on slit spectroscopy

The same pipeline can be used for both slit-less and slit spectroscopy data reduction. Although
the reduction of the slit spectroscopy data can be made in a similar way as for the slit-less
spectroscopy data, there are some diﬀerences in some processing steps between the two. Here
we summarize these diﬀerences.
• The slit spectroscopy ﬂat-ﬁelding consists of a normalized feature-less ﬂat image. Therefore
no color-term correction is necessary.
• Background subtraction is made on extracted 2D spectra in the case of point sources slit
spectroscopy (Np), after stacking sub-frames. If dealing with slit spectroscopy for diﬀuse
sources, the background subtraction is not performed since entire slit is expected to be
ﬁlled with the object that is diﬀuse and there is no pure background area in the image.
• Shift-and-add feature of sub-frames will be disabled. Therefore any spectral change along
the slit could not be related to a real change in the spatial dimensions of the extended
source in case of large telescope jitter motion is observed. To measure such motion, run thepipeline for slit-less mode (without any slit-processing option in the pipeline command).
• In the wavelength calibration, measurement of the reference image positions will not be
made, and pre-deﬁned slit positions will be used as a wavelength zero reference point.

6.2

How to install and to set-up the IRC spectroscopy pipeline

The package has been tested in a Linux environment. It seems to run on the MAC OS-X
environment, though it is not oﬃcially supported.

IRC Data User Manual for Post-Helium Mission (Phase 3)

66• Ask you local computer administrator for the IDL installation. There are no special requests in the IDL installation.
• Ask you local computer administrator for the ds9/xpa installation. There are no special
requests in the ds9/xpa installation.
• Get the irc specred package from the AKARI Observer’s web site (see section 1.2).
Extract and store it under your favorite directory.
• Set the following environment variables in your command-line shell. Below is an example
for csh:setenv IRC SPECRED ROOT <somewhere>
setenv IRC SPECRED LIB ${IRC SPECRED ROOT}/LIB/
setenv IRC SPECRED HOME ${IRC SPECRED ROOT}/ASTRO-F/
setenv IRC SPECRED CALIBDIR ${IRC SPECRED HOME}/IRC SPECRED/CALIBDIR/
setenv IRC SPECRED DATADIR <anotherwhere>
setenv IDL PATH +<IDL system path>:+${IRC SPECRED HOME}:+${IRC SPECRED LIB}IRC SPECRED ROOT, IRC SPECRED DATADIR, and IDL PATH, have to be modiﬁed by substituting <somewhere>, <anotherwhere>, and <IDL system path>, according to your local
system settings 1 .

6.2.1

Data preparation

As explained in Chapter 3, when de-packing the data from the archive under IRC SPECRED DATADIR,the raw data will be stored in IRC SPECRED DATADIR/<targetid>.<targetsubid>/rawdata.
IRC SPECRED DATADIR, <targetid>, and <targetsubid> will be used in the command line ofthe pipeline command.
All the reduced data and related information will be stored in a separate directory called<root dir>/<targetid>.<targetsubid>/irc specred out. When the directory is missing,
the toolkit will create it.

6.3

Calibration data

When de-packing the toolkit, the following calibration directories are created:

6.3.1

Calibration ﬁles

• ${IRC SPECRED CALIBDIR}/DARK contains dark images with high S/N, combined with precompiled dark images, which can be applied for all the observations, after performing asmall correction in the count oﬀset. There are several ’super-darks’ for NIR/MIR-S/MIR-L(as speciﬁed in the following list ﬁles).
– DARK/DARK NIR long.lst, DARK/DARK NIR short.lst– DARK/DARK MIRS long.lst, DARK/DARK MIRS short.lst– DARK/DARK MIRL long.lst, DARK/DARK MIRL short.lst• ${IRC SPECRED CALIBDIR}/FLAT contains two types of ﬂats: super-ﬂats for spectroscopy
and reference images. These super-ﬂats are made by combining a large number of blanksky images:
1

In the following, contents within the parentheses <something> should be changed according to user’s interests.

Version 1.1 (August 28, 2009)67

For slit-less spectroscopy:– FLAT/SPEC2DFLAT NP.lst– FLAT/SPEC2DFLAT NG.lst– FLAT/SPEC2DFLAT SG1.lst– FLAT/SPEC2DFLAT SG2.lst– FLAT/SPEC2DFLAT LG2.lst
For slit spectroscopy:– FLAT/SPEC2DFLAT NP slit.lst
For reference images and color-term correction of the ﬂats:– FLAT/IMAG2DFLAT N3.lst– FLAT/IMAG2DFLAT N4.lst– FLAT/IMAG2DFLAT S7.lst– FLAT/IMAG2DFLAT S9W.lst– FLAT/IMAG2DFLAT S11.lst– FLAT/IMAG2DFLAT L15.lst– FLAT/IMAG2DFLAT L18W.lst– FLAT/IMAG2DFLAT L24.lst• ${IRC SPECRED CALIBDIR}/MASK contains images of known bad pixels (hot pixels, cold
pixels, etc.). Masked area are marked by NaN (Not a Number) in the mask images.
– MASK/SLITMASK NIR.lst– MASK/SLITMASK MIRS.lst– MASK/SLITMASK MIRL.lst– MASK/OUTLIERMASK NIR.lst– MASK/OUTLIERMASK MIRS.lst– MASK/OUTLIERMASK MIRL.lst• ${IRC SPECRED CALIBDIR}/COORDOFFSET contains a table of coordinate oﬀsets in pixels(dX, dY) used to extract spectroscopy images for each object based on target positions
on the reference image for all dispersers. The table also includes sizes of source extraction
boxes (ΔX, ΔY) on reference and spectroscopy images, and oﬀsets (dX, dY) for zero-thorder light image position.
– COORDOFFSET/IRCCOORDOFFSETPAR.dat• ${IRC SPECRED CALIBDIR}/WAVEPAR contains the wavelength calibration tables.
For grisms (NG/SG1/SG2/LG2), the wavelength (μm) is expressed by a 1st order polynomial(linear) equation, and parameters are (1) dispersion (dλ/dY), (2) wavelength at origin (λ0 ),and (3) position of the origin (Y0 ). In the toolkit, the position of the origin is ﬁxed at thecenter of the extracted spectroscopy image. Therefore, only dispersion and wavelength atthe origin are set in the calibration database ﬁles. The parameters can be applied for all

68

IRC Data User Manual for Post-Helium Mission (Phase 3)
the spectra within the FOV (i.e., the parameters are constant across the FOV). Eﬀective
wavelength ranges for each disperser are also set in the table (there are two kinds of range
deﬁnitions: ’sensitive’ and ’reliable’ ranges).
For NP, the pixel position for a given λ is expressed in a 2nd-order polynomial equation,deﬁned through three parameters (0th, 1st, and 2nd-order coeﬃcients). Note that, for data
analysis convenience, the equation is in pixel=function(λ) form, being in inverse form forthe grisms. Eﬀective wavelength ranges are also set in the table.
Only for NP, signiﬁcant spectroscopy distortion exists, i.e., reference positions on the spectroscopy images cannot be represented by a constant pixel shift (dX, dY) from reference
image positions. The deviation of pixel shift along wavelength axis (or Y axis) from thecase of constant pixel shift is expressed in 3rd-order polynomial equation of reference position (dYdistortion =function(Xref , Yref )). Polynomial coeﬃcients are tabulated in a separate
ascii table, and are stored under the same directory. Note that spectroscopy distortion
along X (dXdistortion ) is not so signiﬁcant, and constant pixel shift works rather well for
NP, as in the same way for NG/SG1/SG2/LG2.
– WAVEPAR/IRCWAVEPAR COMMON.dat– WAVEPAR/IRCWAVEPAR NP.dat– WAVEPAR/IRCWAVEPAR NG.dat– WAVEPAR/IRCWAVEPAR SG1.dat– WAVEPAR/IRCWAVEPAR SG2.dat– WAVEPAR/IRCWAVEPAR LG2.dat– IRCSPECBOXDYPAR NP.dat• ${IRC SPECRED CALIBDIR}/RESPONSE contains ascii ﬁles tabulating λ vs. ADU/long exposure time in seconds/mJy for each disperser. There are ﬁve such tables (NP/NG/SG1/SG2/LG2).
The spectral response is measured by observing ﬂux standard stars with known ﬂux energy
distribution. This directory also contains a NP template spectrum (wavelength-calibrated
spectrum, before calibrating spectral response) that will be used to ﬁnd relative image
shift along dispersion direction (or Y) between spectroscopy and reference images.
– RESPONSE/RESPONSE NP.lst– RESPONSE/RESPONSE NG.lst– RESPONSE/RESPONSE SG1.lst– RESPONSE/RESPONSE SG2.lst– RESPONSE/RESPONSE LG2.lst• ${IRC SPECRED CALIBDIR}/DISTPAR contains ascii tables containing spectral tilt information of the grism insertion angle.
– DISTPAR/IRCDISTPAR NP.dat– DISTPAR/IRCDISTPAR NG.dat– DISTPAR/IRCDISTPAR SG1.dat– DISTPAR/IRCDISTPAR SG2.dat– DISTPAR/IRCDISTPAR LG2.dat• ${IRC SPECRED CALIBDIR}/APCOR contains the spectral aperture correction tables.

Version 1.1 (August 28, 2009)69– APCOR/APCOR NP.dat– APCOR/APCOR NG.dat– APCOR/APCOR SG1.dat– APCOR/APCOR SG2.dat– APCOR/APCOR LG2.dat

6.4

Running the pipeline

6.4.1

Data reduction order

The pipeline has to be run more than once, for NP (AOT0a) or NG (AOT0b) ﬁrst, then SG1 andSG2, and ﬁnally LG2. We need to start processing the shorter wavelength cameras (NIR and
MIR-S), where larger number of brighter sources are expected to be observed within the FOV as
position reference sources. The information derived there will be used for registering the longer
wavelength cameras (MIR-S and MIR-L).
• to measure relative X and Y shift among spectroscopy sub-frames of NP/NG for registering
spectroscopy sub-frames of NP/NG/SG1/SG2/LG2.
• to measure relative X and Y shift among reference sub-frames of S9W (with larger numberof brighter sources) for registering imaging sub-frames of S9W and L18W.
• to measure relative X and Y shift of the spectroscopy image with respect to the reference
image of NP or NG for registering reference and spectroscopy images of NP/NG/SG1/SG2/LG2.
Therefore the data reduction order should be as follows:• First run: run the pipeline for NP (AOT04a) or NG (AOT04b), without a target table forenabling automatic target detection sub-program.
For NG data with point source aperture (Np), one needs to run the pipeline ﬁrst without/Np spec option. For other types of slit spectroscopy, one may skip this run because
shift-and-add feature is disabled for this observing mode.
• Second run: run the pipeline for your desired targets. Make target tables for N3, S9W, andL18W on the raw images (at this stage only N3 image has been processed). Run the pipeline
again for NP/NG/SG1/SG2/LG2 with target lists. Image shift database created previouslywith NP/NG will be used for the 2nd run. If your target is compact and bright enough to
be detected by the source detection sub-program, you do not have to perform this 2nd
run. For the 2nd run, begin processing for NP/NG ﬁrst, then SG1, and ﬁnally LG2.
• Third run: run the pipeline again if you want to revise the target table. This is optional.
Now you got processed reference images for revising the target tables. Run the pipeline
again with the updated target tables to see ﬁnal results.

6.4.2

Running the pipeline

Type the following in the IDL command line (a command in a single line):
irc specred, <targetid>, <targetsubid>, <ext source table>, <refimage list>,<specimage list>, <filter spec>

IRC Data User Manual for Post-Helium Mission (Phase 3)

70
Mandatory arguments:• targetid: ID of the pointing observation. The information will be provided with the data
distribution.
Example: 1331048• targetsubid: Sub-ID of the pointing observation. The information will be provided withthe data distribution.
Example: 1• ext source table: An ascii list describing source position (X, Y) in pixel, or the target
table.
Example: myobjects.tbl
cat myobjects.tbl
100.0 100.0
200.5 200.5
150.0 150.5
This will extract spectra of sources located at (100.0, 100.0), (200.5, 200.5), and (300.0,
300.5) in pixel on the reference image.
If ext source table is set to ”” (null string), then the pipeline activates its sub-programfor automatic source detection.
There are some important tips in preparing the source table:– Coordinates counts from 1, not 0, i.e., coordinates of the lower left corner of the imageis (1, 1), not (0, 0).
– For NIR the toolkit interprets by default the pixel coordinates as measured in the
raw image (before the image rotation). When one measures the source positions in
the processed images (after the image rotation), set the /rotated NIR source tableoption at the pipeline command line.
– For MIR-L, one can use Y range of either 257-512 (for images before detaching
MIR-S/L) or 1-256 (for images after detaching MIR-S/L) to set Y position of thetargets. The pipeline subtracts 256 from the Y input if Y>256.
• refimage list: An ascii ﬁle listing FITS ﬁle names of reference image.
Example: N3.lst
cat N3.lst
F000001 N.ﬁts
A default list is provided with the data distribution, and is found in /rawdata directory.
• specimage list: an ascii ﬁle listing FITS ﬁle names of spectroscopy images.
Example: NP.lst
cat NP.lst
F000002 N.ﬁts
F000004 N.ﬁts
F000006 N.ﬁts
A default list is provided with the data distribution, and is found in /rawdata directory.
Even if you ﬁnd some images being damaged severely and you do not want them to be
included in the pipeline processing, you must list all the images in the input list. Then you

Version 1.1 (August 28, 2009)71

should specify the sub-frames to be removed on ds9 within the pipeline. This is because
the ﬁle names and their order in the list are used to relate FITS ﬁles to the exposure
timing along the AOT operation sequence.
• filter spec: a string specifying a disperser for the processing. Set one of the following:
N3 NP, N3 NG, S9W SG1, S9W SG2, L18W LG2

6.4.3

Options• root dir: a string specifying a directory in which a set of data is located. If set, thisoverrides the setting found in the environment variable IRC SPECRED DATADIR.
Example: root dir=’/DATA/ASTRO-F/IRC/SPEC/’• /Nh spec, /Ns spec, /Ls spec: ﬂags for slit spectroscopy data reduction.
• /Np spec: a ﬂag for Np spectroscopy data reduction.
See section 6.1.16 for more information of slit spectroscopy data reduction.
• /no tune sourcepos: a ﬂag for disabling source position tuning subprogram within thepipeline. By default, irc specred tries to measure accurate source positions by searching
Gaussian peak around the coordinates set in the target table. This ﬂag disables thisfunctionality.
• /use short refimage: use a short exposure image to measure source positions. By default, irc specred uses the long exposure frame for measuring source positions in thereference image. With this ﬂag set, the toolkit uses the short exposure frame for measuring the source positions. Note that the data reduction will be made for both short andlong exposure frames even with this ﬂag set. This ﬂag could be useful for measuring source
positions of bright and saturated objects in the long exposure frame.
• savefile: set this option to a named variable that will contain the ﬁle name of the
IDL save ﬁle (section 6.5.5). The simplest way to restore the IDL save ﬁle is to run
irc specred with the savefile=savefile option, and issue restore,savefile command
after irc specred ﬁnished and before, e.g., plotting the spectra.
• /no slit flat: a ﬂag for disabling ﬂat ﬁelding process for slit spectroscopy at NIR. By default, irc specred performs spectroscopy ﬂat ﬁelding with the super-ﬂat image. However,as the natural background is faint in the NIR, the quality of the ﬂat can not be improved
so much. As a result, unfortunately, the quality of the processed spectra is limited by
the quality of the ﬂat, not by the dark current nor photon noise. With this option set,
irc specred skips performing the ﬂat ﬁelding. Users are recommended to compare thespectra processed with and without the ﬂat ﬁelding process. We expect no big diﬀerencesamong the two spectra, except for random pixel-to-pixel variation of the ﬂux, or some narrow (as narrow as one-pixel size) spike features, meanwhile spectra processed without theﬂat ﬁelding could look better in terms of S/N. However, it seems a good idea to compare
the results processed with and without the ﬂat ﬁelding. If you ﬁnd that this is also thecase for your data, you may adopt the data processed with the option /no slit flat.
• /nir column pulldown correction: a ﬂag for enabling masking column pulled-down regions in NIR images. With this option set, irc specred searches for any pulled-downcolumns in each sub-frame, by examining dark area pattern at Nh slit area where background is minimum, and the pipeline masks out the columns. Furthermore, the pipeline
also masks the same region in the next sub-frames in which damage by the column pulldown is still visible for deepest pulled-down columns.

IRC Data User Manual for Post-Helium Mission (Phase 3)

72

6.4.4

Outputs• Whole image products:– Processed and stacked reference image (and corresponding mask and residual (reference image × mask) images). Dark subtraction, ﬂat-ﬁelding, and background subtraction are made:<targetid>.<targetsubid>.<filter spec>.refimage bg.fits<targetid>.<targetsubid>.<filter spec>.refimage mask.fits<targetid>.<targetsubid>.<filter spec>.residual refimage bg.fits– Processed and stacked spectroscopy image (and corresponding mask and residual(spectroscopy × mask) images). Dark subtraction, ﬂat-ﬁelding, and background subtraction are made:<targetid>.<targetsubid>.<filter spec>.specimage bg.fits<targetid>.<targetsubid>.<filter spec>.specimage mask.fits<targetid>.<targetsubid>.<filter spec>.residual specimage bg.fits

These images are in 3D, being the third (Z) dimension for the short (Z=0) and long (Z=1)
exposure frames.
NaN (Not a Number) represents masked pixel/area.
• Extracted image products:– Extracted reference images for individual targets:<targetid>.<targetsubid>.<filter spec>.refimage bg indiv.fits– Extracted spectroscopy images for individual targets (and corresponding mask images). There are two kinds of images. One is wavelength calibrated ( WC) image forwhich ﬂat color-term correction and wavelength calibration were applied. The otherare ﬂux calibrated ( FC) images for which ﬂux calibration was applied as well as ﬂatcolor-term correction and wavelength calibration:<targetid>.<targetsubid>.<filter spec>.specimage wc indiv.fits<targetid>.<targetsubid>.<filter spec>.specimage mask indiv.fits<targetid>.<targetsubid>.<filter spec>.specimage fc indiv.fits<targetid>.<targetsubid>.<filter spec>.specimage noisemap indiv.f its
Note that our spectral plotting tool (see below) uses WC image as an input, not the
FC image, since the tool does on-the-ﬂy ﬂux calibration after considering various
plotting conditions/options. Since the default FC images does not care these, westrongly recommend to use our plotting tool to review the spectra.
In wc indiv.fits and fc indiv.fits ﬁles, wavelength information is set in the
FITS headers with header keywords: CRVAL, CDELT, CRPIX, and CD matrix. Users canplot the spectra with, e.g., the standard spectral plotting tool SPLOT on IRAF.
Note, however, that there are no such FITS header information for prism NP spectra
because of their non-linear wavelength solution.
Here is an example to plot the SG1 spectra with IRAF/ SPLOT task.
splot xxxxxxx.1.S9W SG1.specimage fc indiv.fits

Version 1.1 (August 28, 2009)73

Image line/aperture to plot (0:) (1): 10
Image band to plot (1:) (1): 2
In this example, the plot shows ﬂux-calibrated spectrum of the 2nd object along X(space) =10 pixel. Note that 2nd object means source ID 1 on the IRC SPECRED
since IDL counts from zero while IRAF does from one. Note also that no aperture
correction can be applied for the spectral plot with the SPLOT task.
These images are in 3D, being the third (Z) dimension for source id. NaN (Not a
Number) represents the masked pixel/area.
• Others:– Target information table: a table of target information, such as positions (after source
position tuning), ﬂux, and size (Gaussian FWHM).
<targetid>.<targetsubid>.<filter spec>.target table.tbl
Its format is the following:ID, image x, image y, image mask dx, image mask dy, spec x, spec y, spec mask dx,
spec mask dy, ﬂux, image FWHM, spec bgnoise ADU, spec x pos,spec x FWHM,
bad sourcepos ﬂag– Target table: If the source detection sub-program is used within the pipeline, a target
table will be written. The ﬁle format is similar to the input target table. When source
position tuning option is on, this ﬁle contains the updated source coordinates.
<targetid>.<targetsubid>.<filter spec>.source table.tbl– IDL save ﬁle: The output of the pipeline will also be stored in the IDL save ﬁle, which
is basically a dump ﬁle of the IDL memory image at the end of the pipeline processing.
See also 6.5.5 to see how to work on the save ﬁle.
<targetid>.<targetsubid>.<filter spec>.IRC SPECRED OUT.sav– Log of the toolkit processing: a copy of the irc specred logger is saved as an asciiﬁle.
<targetid>.<targetsubid>.<filter spec>.log– DS9 region ﬁles: DS9 region ﬁles that have been used to locate targets on reference
and spectroscopy images on ds9.
<targetid>.<targetsubid>.<filter spec>.refimage.reg<targetid>.<targetsubid>.<filter spec>.specimage.reg
The region ﬁles and saved FITS images of the whole image products can be used toreview the targets on ds9 manually.
• Pipeline work ﬁles: The following ﬁles will be created by the pipeline. The ﬁles will beoverwritten without notice. User are asked not to delete any of these ﬁles.
– NP SHIFT XY.dat: for registering NP/SG1/SG2/LG2 sub-frames in AOT04a– NG SHIFT XY.dat: for registering NP/SG1/SG2/LG2 sub-frames in AOT04b– S9W SHIFT XY.dat: for registering S9W and L18W sub-frames– NP SPECBOX SHIFT X.dat: for shifting SG2/LG2 along X in AOT04a

IRC Data User Manual for Post-Helium Mission (Phase 3)

74– NP SPECBOX SHIFT Y.dat: for shifting the wavelength zero-reference point for NP/SG1/SG2/LG2 in AOT04a– NG SPECBOX SHIFT X.dat: for shifting SG2/LG2 along X in AOT04b– NG SPECBOX SHIFT Y.dat: for shifting the wavelength zero-reference points of NG/SG1/SG2/LG2 in AOT04b

6.4.5

Summary of interactive operations within the pipeline

Although the pipeline program works as a ’pipeline’ (i.e., without interactive operation by users),
some operations in the irc specred do require interactive operation.
• Removing one or more sub-frames before combining the sub-frames: In some sub-frames,
one may ﬁnd severe damage by cosmic-ray shower, nearby satellite passing in front of thetelescope, etc. To check the sub-frames and remove from the sub-frame combination list,
all the dark-subtracted/ﬂat-ﬁelded/sky-subtracted sub-frames will be shown on a ds9 forreviewing. To remove some sub-frames, use the Frame -> show/hide frames pull-downmenu of ds9 to hide the frame(s) you need to remove. The toolkit checks the hidden
sub-frames and remove them from the internal sub-frame combination list.
• Retrying automatic source detection with modiﬁed detection parameters: If you use automatic source detection sub-program within the pipeline, you can retry source detection
after tweaking the detection parameters. You will see the reference image on the ds9 with
detected sources marked, and you are asked to answer if the detection is satisfactory ornot. If the answer is no, then another dialog with several detection parameter entries pops
up. You can edit the parameters, and press OK to re-ﬁnd the sources. You can repeat
this detection process as many times as you like until you get satisfactory source list. The
sub-program uses daofind algorithm, with the following parameters:– threshold: Threshold intensity for a point source - should generally be 3 or 4 sigma
above background.
– fwhm: FWHM to be used in the convolve ﬁlter.
– sharplim: 2 element vector giving low and high cutoﬀ for the sharpness statistic(Default: [0.2,1.0] ). Change this default only if the stars have signiﬁcantly larger orsmaller concentration than a Gaussian.
– roundlim: 2 element vector giving low and high cutoﬀ for the roundness statistic(Default: [-1.0,1.0] ). Change this default only if the stars are signiﬁcantly elongated.
By default, threshold is three times the measured background noise. When too muchor too less sources are found, threshold change would give better results, and it is notnecessary to take care of sharpness/roundness parameters in most of the cases.

6.4.6

Warning messages of the pipeline

Some common and frequently appearing warning messages are listed below.
• Warning (irc specred):
guments, and try again.

Wrong number of arguments! Check the command line ar-• Warning: No <..> was found/set...Stop! Some ﬁles that should be provided forthe pipeline seem missing. Check if list ﬁles are properly set in the command line. Thiswarning will also appear if you run SG1/SG2/LG2 data reduction before performing NP or

Version 1.1 (August 28, 2009)75

NG data processing, and some mandatory database ﬁles, e.g., describing relative image
shifts, are missing.
• Warning: Offset search box is too small!: When the telescope drifts too much,the image matching sub-program fails to ﬁnd the best shift values among sub-frames.
Normally the sub-program resumes searching the shift with larger search box. Normally,
this message is not so serious.
• Warning: No data are available for measurement... Returning 0! or Warning:
Data seems too noisy (sigma=<sigma>)... Returning 0!: These messages will appear if, e.g., detected sources are too faint and their positions could not be measured withgood enough accuracy. Check the source detection parameters for better source detections.
This message is not so serious in most of the cases.
• Warning: sigma of specbox Y shift measurement seems too large!: This warning appears when the toolkit fails to measure the relative Y shift of NP spectra withrespect to N3 reference image. Check the reference image on ds9 to know if source detection could be made successfully. Sometimes the program might detect only cosmic-ray
hits, not stars, if the parameters are not optimized.
• Warning: sky level is larger than 1.5 times sky sigma!: After extracting spectroscopy images, remaining sky will be subtracted oﬀ locally around the source. If everything works ﬁne, the average of the remaining sky should be very small, and is typicallyless than 1.5 times sky rms ﬂuctuation. This warning message will appear if sky level islarger than the typical value. When this warning appears, check the ds9 image to see if
there remains large scale sky level variation. Presence of such sky often indicates failure inmonochromatic ﬂat-ﬁelding and/or global sky subtraction, or any big unexpected debris.
Note that this warning will not appear for slit spectroscopy (including NG at Np).
• Warning: Available sky area is too small!: Due to clouded source masking, theremaining sky area is too small for measuring sky. Check the source detection parameters
for fewer source detection. Usually, this message is not so serious.
• Warning: Cross-correlation peak is weak: The cross-correlation function will becalculated while registering sub-frames and/or short and long exposure frames. The peakvalue of the cross-correlation function is a good measure of the frame registration. The
value is 1 for ideal image matching, and is typically 0.6 or larger for real data with noise,but peak of 0.3 is still good. When this warning appears, check the quality of the registeredand stacked images. So far registration of the short exposure frame can not be made withgood enough accuracy, and one should ignore this warning for short exposure frames. Thismessage is not so serious in most of the cases.
• Warning: ROB MAPFIT: Too few points!: This message comes from the ﬁtting routine
ROB MAPFIT called from some of the toolkit programs. The routine tries to ﬁt the image
region robustly (e.g., excluding outliers, iteratively, etc.), but it sometimes fails when theﬁtting area is heavily masked out for some reason. However, it somehow return the result(even if the results are not reliable in terms of statistics). Users may ignore the messages.
• Warning: strange spec file list... Stop!: The irc specred reads list of FITS
ﬁles to be processed (NP.lst/NG.lst/SG1.lst/SG2.lst/LG2.lst). When something wrong
happens during data acquisition (i.g., too much time before stabilizing the telescope pointing), telemetry downlink (e.g., loss of telemetry), and/or data handling at ground (e.g.,
disk storage failure), the listed available FITS ﬁles are not in a proper order. Observers

IRC Data User Manual for Post-Helium Mission (Phase 3)

76

may notice that the list misses one or more FITS ﬁles due to some of these reasons. The
toolkit analyze the ﬁle list before actually processing the data, and warns the uses if it
detects something strange. Upon this warning messages, users are requested to checkthe input ﬁle lists or contents of distributed FITS ﬁles, and report any problems to the
AKARI/IRC Helpdesk for more support. Although the irc specred skips this list checking routine if a ﬂag /no database check is set to the toolkit, use of this option without
noting problems is dangerous, and usually is not recommended.

6.5

Working on the pipeline output

6.5.1

Displaying the whole images on ds9

Here we explain the region marks on the ds9 that are shown in the processed whole image
products, e.g., spectroscopy and reference images before source extraction.
In the spectroscopy image side (left),• The blue rectangle shows spectroscopy image extraction area.
• The green rectangle shows area reserved for the object (or masked area for other objects).
• The red rectangle shows plotting area as deﬁned by the spectral plotting tool.
• The yellow circle shows the expected positions of the zero-th order light.
In the reference image side (right), marked circles correspond to source positions (withdiameter of the source FWHM).
• Green circles show sources with good position measurements.
• Red circles show sources with less accurate position measurements.
Source identiﬁcation numbers are also indicated by the region marks.
For clear view of the image under complicated region marks, check out the region -> show
regions check box in the ds9 pull-down menu.
To display these images manually, issue
show aperture on ds9, <image>, source table (,/imag) 2
where <image> could be refimage ff[*,*,1] or specimage bg, etc. on the IDL commandline. When /imag option is set, the image is shown on the right side of the ds9, with region
marks for reference image. The default (no option) is for spectroscopy image. See appendix for
more on array name conventions.

6.5.2

Displaying the extracted images on ATV

ATV is a general-purpose interactive array image-displaying tool. The array should be in 2D.
See ATV web page (http://www.physics.uci.edu/ barth/atv/) for more information on the
program 3 .
Command line syntax:
atv,array(,/block)
Here are some ATV tips:
2

Items in parentheses () are optional.
In some systems, /block option may be required for interactive operation on the ATV. The conditions forwhich /block requirement seems to depend on local Linux or X11 system settings. The author does not have any
good ideas on the correct usage of this option.
3

Version 1.1 (August 28, 2009)77• To display whole images, issue
atv,<array>[*,*,1]
1 means long exposure frame (and 0 for short). Here, <array> should be something like
refimage bg or specimage bg.
• To display extracted images of your desired source id, use
atv,<array>[*,*,source id]
Here, <array> should be something like specimage n bg. To check mask area, use
atv,<array>[*,*,source id]*mask[*,*,source id]• When color table looks abnormal, type set color at the IDL prompt before launching the
atv.

6.5.3

Checking for wavelength zero reference point with the zero-th order
light image

The toolkit calculates the wavelength zero reference point based primary on positions of theobjects on the reference image. It also takes into account a wavelength zero-point drift due
to satellite attitude drift, as well as coordinate rounding eﬀect when extracting 2D spectra.
In most cases, these methods work well to estimate the wavelength zero reference point withaccuracy of 0.5 pixel or less. Since positions of the zero-th order light image are also calculated
in the same way as for the wavelength zero reference point, one may check the wavelength
zero reference point accuracy by comparing positions of expected zero-th order light image andactually observed ones. Unfortunately, this check can be made only when bright sources withmeasurable zero-th order light images are observed at some part of the FOV by chance.
When a shift is found, the change wave offset command can correct for the estimatedwavelength shift in pixel. The shift is calculated from the current position. This commandchanges the internal variable that records the wavelength oﬀset, updates an oﬀset database on
disk, and re-draws the images on ds9 with updated zero-th order light marks. This command canbe used as many times as needed until getting satisfactory result. Then one can use the spectral
plot tool for reviewing spectra with updated wavelength and spectral-response calibration. See
below for the plotting tools.
Command line syntax (a command in a single line):change wave offset,<wave offset>,source table,
space profile,spec profile,wave array,specimage fc 1d,
refimage bg,specimage bg,
refimage n bg,refimage n bg short,specimage n sff,specimage n sff short,specimage n wc,specimage n wc short,specimage n fc,specimage n fc short,
res refimage bg,res specimage bg,
mask refimage,mask specimage,mask specimage n,
imagheader,specheader,imagheader extract,specheader extract,noisemap,noisemap n,noisemap short n,noise mask,
dir=!IRC SPECRED DIR.workdir,savefile=savefile,/write indiv spec
where <wave offset> is a pixel number (e.g., 1.0, 0.5, -0.5) to be shifted. 44Since this is a very long command, it seems better to copy the command that is written in the${IRC SPECRED HOME}/IRC SPECRED/irc specred.pro program.

IRC Data User Manual for Post-Helium Mission (Phase 3)

78

6.5.4

Spectral plotting tool

The toolkit’s spectral plotting tool can handle many IRC-spectroscopy-speciﬁc features, and werecommend to use it for creating spectral plots. These are some of the functionalities of thetool:• source masks can be considered. (see also /no mask option)• error bars (statistical plus systematical ones) can be plotted.
• simple image ﬁlters for removing isolated ’spike’ pixels can be applied.
• aperture stacking along spatial direction and smoothing along wavelength direction canbe applied for higher S/N spectra.
• shifting aperture positions along spatial direction (X axis) can be made.
• aperture correction.
Command line syntax (a command in a single line):plot spec with image,wave array,specimage n wc,mask specimage n,source table,<source id>
and for short exposures:plot spec with image,wave array,specimage n wc short,mask specimage n,source table,<source id>,/shortwith the following options:• nsum: Number of pixels along X axis combined for plotting. Default is 3. For slitless
spectroscopy, larger nsum (wider aperture) collects more photons from the object, but thisalso collects more background noise. Therefore, the best nsum for highest S/N is typically
2 or 3 for point sources (corresponding typical full-width of image PSF). In the case of slit
spectroscopy, the range to plot the spectrum is:– NP/NG: up to the edge of the extracted region– SG1: nsum≤ 10– SG2: nsum≤ 8– LG2: nsum≤ 8
If space shift= 0 (see below) then the range should be smaller in order to prevent fromextracting the spectrum out of the slit area.
• smooth: Boxcar smoothing width (in pixel) along wavelength direction. Default is smooth=0(no smoothing). For spectra with higher S/N without loosing spectral resolution, smoothshould be 3 (corresponding typical full-width of image PSF). Larger boxcar smoothing window will loose the spectral resolution. Note that when even number is set in the smooth(e.g., smooth=2), the actual smoothing box size would be smooth+1 (e.g., smooth=3).
• /sigma filter: This option enables sigma ﬁlter operation (at 3 sigma signiﬁcance level)
over 2D spectra to remove spatially isolated high or low count pixels. Default is oﬀ. If
you ﬁnd too narrow emitting/absorbing features in your plots, try this option to see if
this is a real feature or not. Note that, if there is a cosmic ray hit at the position in
one of the sub-frames, such count should be removed as an outlier while combining subframes with sigma-clipping averaging method. However, if there are temporal hot pixels

Version 1.1 (August 28, 2009)79

or weaker hot/cold pixels that are not shown in the hot pixel mask database, you will ﬁnd
outliers (hot or cold pixels) even after sub-frame combination. Users should take care notto remove real narrow features.
• space shift: shift of the plot extracting box along X (spatial) axis in pixel. Default
is zero. Although the pipeline adjusts the X coordinates of spectrum extraction box bymeasuring positions of spectra, one may ﬁnd remaining X oﬀset in some cases. Change
this oﬀset to ﬁnd peak position of the ﬂux. If one changes nsum and space shift, issue
the display command to see the modiﬁed spectrum extraction boxes graphically on the
spectroscopy images over the ds9.
show aperture on ds9,specimage bg,source table,
space shift=<space shift>,nsum=<nsum>(a command in a single line)• /no mask: by default, plots are shown after applying spectral overlapping masks by nearby
sources. The /no mask option disables this masking functionality and plot spectra regardless of the possible source overlapping. Since masks are created without examining thesource brightness and/or spectral shape (e.g., line emitters, continuum emitters with break,etc.), one may often ﬁnd the situation where no signiﬁcant change in plots is found whencomparing plots with and without the masking. If this is the case, you can just disable
the masking by setting /no mask option.
This option is especially relevant when the source of interest is much brighter that theoverlapping ones.
• ps, png, ascii: Plots will be recorded on the ﬁles, not on the IDL plot window.
– ps=<filename> creates a postscript ﬁle (filename.ps) of the plot in the working
output directory (under /irc specred out).
– png=<filename> creates a png image (filename.png) of the plot in the working
output directory.
– ascii=<filename> creates an ascii ﬁle (λ vs ﬂux and ﬂux errors) (filename.spc) of
the plot in the working output directory. This ﬁle can be used for further analysis,for example using gnuplot program, with the command of plot ’filename.spc’
with yerrorbar.
• /with image, tvbottom=<tvbottom>, tvtop=<tvtop>: The 2D image will be shown
on TV (an IDL graphic window) below the spectrum plot. Extraction box for plotting
1D spectrum is overlaid. The default is with image=0 (i.e., no TV display). The top
and bottom ADU counts for displaying image can be speciﬁed with tvtop and tvbottom.
These options are active only when /with image is set.
• /diffuse: For slit spectroscopy data, spectra will be shown in units of MJy/str, rather
than mJy with this option. The diﬀuse ﬂux calibration is not yet fully established. Before
this ﬂux scale conversion, both nsum and space shift parameters are taken into accountto specify the spectral extraction aperture to create the spectrum for plotting.
• /no aperture correction: Aperture correction for point-like sources will be applied automatically within the plotting tool. The aperture size, set by nsum, is taken into accountto ﬁnd the correction factor. When the /no aperture correction option is set, the correction will be disabled. To perform good aperture correction, the aperture is centeredon the source, and nsum should be 3 or more. Note that the aperture can be shifted by
space shift parameter.

IRC Data User Manual for Post-Helium Mission (Phase 3)

80• Other generic plot options useful in the plotting tool: The plotting tool accepts any kindof IDL-generic plot options for spectral plotting. See plot manual in the IDL documents
for full information.
Following are some frequently used options.
– xrange: plot range along the X (wavelength) axis. By default, plots are shownwithin the wavelength range set in the wavelength calibration database ﬁle along the
X axis. If set explicitly in the plotting tool command line, the default settings will
be overridden.
– yrange: plot range along the Y (ﬂux) axis. By default, plots are shown in auto-scalemode along Y. One can limit/ﬁx the plot range by setting yrange.
– /xlog, /ylog: plotting in log scale along x or y.

6.5.5

Working on saved data

All the processed data are saved as an IDL save ﬁle, as well as FITS output ﬁles. The IDL
save ﬁle is actually a dump record of the IDL memory image at the end of the data processing. Therefore, one can recall the pipeline results by simply issuing the following command:
restore,<savefile>. Here the <savefile> is a string of the save ﬁle name. To recall othersave ﬁle image, issue .reset session ﬁrst to clear the current IDL memory contents, and thenissue another restore command with another save ﬁle name. See IDL manual to know more on
the IDL save ﬁle, restore, and .reset session commands.
After restoring the IDL memory, one can use, for example, the following irc specred commands to review the results.
• show aperture on ds9, specimage bg,source table• show aperture on ds9, refimage bg,source table,/imag• plot spec with image,wave array,specimage n wc,mask specimage n,source table,<source id>

6.6(Hopefully) Useful Tips

6.6.1

Getting best S/N spectra in slit (Ns, Nh) or Np spectroscopy data

Due to the higher dispersion of NG spectra, through narrower slit area (Np, Ns, Nh), at nearinfrared where the natural background light is faintest, the S/N of the NG slit ﬂat image is ratherpoor, and this often limits the quality of the processed spectra. There is an option to disable
the slit ﬂat correction (/no slit flat option of irc specred). We recommend processing thesame data, with (default) and without (with the option) ﬂat correction, for comparison. If one
ﬁnds little noticeable diﬀerences in spectral features, (except for quality of the spectra), as we
expect, one may accept the spectra processed without ﬂat correction as a ﬁnal calibrated data.
See also section 6.4.3.

6.6.2

Tackling narrow spikes seen in NIR spectra (especially in slit spectra)

Even if one is interested in the slit area, any anonymous bright stars that happened to be within
the big aperture may cause the column pull-down eﬀect of the array, damaging even the slit
spectroscopy data. To check this possibility, examine not only the 1D-extracted spectra but also

Version 1.1 (August 28, 2009)81

the entire 2D-images, even outside of the slit area, so as to locate damaged columns (or raws,
since NIR images are rotated during the toolkit processing).
With the toolkit version 2, new option /nir column pulldown correction was introduced,
and the pulled-down columns will be masked out. See also section 6.4.3.

6.6.3

Examining strange/fake features in NP spectra, especially around 2.43.5 µm

Sometimes an error in wavelength calibration leads to errors in the spectral response calibration,
creating funny/fake spectral features in NP. In this case, the features resemble the response curve(or more exactly, its change per pixel along wavelength). Because of lowest spectral resolution
and rather structured response-curve shape, this eﬀect is most clearly seen in NP spectra. Notethat even small wavelength oﬀset (say, a mere 0.5 pixel) would produce noticeable pseudo spectralfeatures, especially around 2.4-3.5 μm. We recommend to check the spectra of nearby ﬁeld stars
as well as your object to see if the funny features are common to all objects. If they look similar,
the features are likely to be fake.
To see how the features change with wavelength oﬀset, try the procedure described in section 6.5.3. Although there are no zero-th order light images available for NP, the commandchange wave offset works even for NP for testing how the spectral features change with theoﬀset. Note that, for NP, there are no guidelines, such as zero-th order light, to ﬁnd the best
wavelength zero point in general. Therefore the user should justify the amount of oﬀset enteredwith this command. Remember that nominal error of the wavelength zero-point calculation is
1.0 pixel or less.

6.6.4

Examining ﬂux level (in)consistency among diﬀerent disperser data

Sometimes one may ﬁnd jumps in ﬂux level between SG1 and SG2 spectra (or other pairs ofadjacent wavelength) for a single source. There are possible reasons for the jump, and tips tosolve the problem.
• Source contamination: Degree of contamination changes with wavelength and, hence, withdispersers, causing ﬂux change across the dispersers. User needs to check the reference
image as well as spectroscopy images to ﬁnd possible contaminating sources. There are nogood recipes to remove the contamination.
• Aperture centering: Check the aperture position (along X or space) to see if the aperture isreally at the peak of the dispersed images. The toolkit tries to put it at the right position
by tracking the telescope jittering motion, but it is not always perfect. Use space shiftoption of the plotting tool along with /with image option to ﬁnd the best position. See
section 6.5.4 for actual operation.
• Aperture correction uncertainties: Aperture correction is not always perfect because oneneeds to adopt rather narrow aperture (nsum=3 or so) for achieving the best S/N. If thisis the case, even a small shift of the aperture position on sky (even at a scale of subpixel, although space shift should be given in an integer pixel unit) gives rather large
uncertainties in the ﬂux aperture correction. Note that spectroscopy data relies on 1D
spatial PSF of the source, and the pixel scale is not high enough to sample the PSF atgood enough accuracy. So, with these limitations in mind, one needs to check the ﬂux
level, by changing the nsum and/or space shift parameters, for your own source and its
brighter neighbours, to ﬁnd the accuracy of the absolute ﬂux calibration. See section 6.5.4
for actual operation.

IRC Data User Manual for Post-Helium Mission (Phase 3)82• Wavelength oﬀset error: Because of the very low spectral resolution, the spectral response
curves show rather sharp change near the band edge. If the wavelength oﬀset is notproperly set, a systematic error in the ﬂux level is expected near the band edge due towrong wavelength assignment in the spectral ﬂux calibration process. If this happens, thespectrum looks brighter and fainter (or fainter and brighter) at the shortest and longest
wavelength coverage of the disperser in a systematic manner. The way the shape change
depends on the sign of error in wavelength oﬀset (plus or minus). See section 6.5.3 forchecking the wavelength oﬀset error.

6.7

Appendix

6.7.1

Variable name conventions

As a general rule, the extension of the IDL variables ( ff, bg, wc, fc) indicates the ﬁnallyprocessed calibration status. Examples:• ff means that the data is processed all the way to ﬂat-ﬁelding.
• bg means that the data is processed all the way to background subtraction, i.e., ﬂat-ﬁeldedand background subtracted.
• wc means that the data is processed all the way to wavelength calibration, i.e., ﬂat-ﬁelded,
background subtracted, and wavelength calibrated.
• fc means that the data is processed all the way to ﬂux calibration, i.e., ﬂat-ﬁelded,
background subtracted, color-term corrected, wavelength calibrated, and ﬂux calibrated.
Whole image products: ff and bg are in 3D format (X, Y, short/long ID). Short exposure
frame is in Z=0, and long exposure frame is in Z=1.
Extracted image products: n images are in 3D. The third dimension is for indicating thesource id. The following extensions refer to diﬀerent processing stages:• n bg: background subtracted on extracted images.
• n wc: wavelength calibrated, after color-term corrected, residual background subtracted.
• n fc: ﬂux calibrated, after color-term corrected, residual background subtracted, andwavelength calibrated.
Finally, these are the most frequently referred arrays:• refimage bg: Flat-ﬁelded, background-subtracted whole reference image in 3D (X, Y,
short/long ID)• specimage bg: Flat-ﬁeld, background-subtracted whole spectroscopy image in 3D (X, Y,
short/long ID)• specimage n wc: Wavelength calibrated extracted 2D spectra in 3D (X, Y, source id)• specimage n fc: ﬂux calibrated extracted 2D spectra in 3D (X, Y, source id)

Version 1.1 (August 28, 2009)

6.7.2

83

Summary of Commands and their Options

This is a broad outline of the main commands related to the pipeline together with their generalsyntax. For arguments encompassed by <> and <>, you need to specify your own processing
parameters. Options starting with / are ﬂags or switches to enable some functionality of theroutine. For other options, you can set your parameters in numerals or ascii characters (thatshould be encompassed with single quotations).
• Running the pipeline
Usage:
irc specred, <targetid>, <targetsubid>, <ext source table>, <refimage list>,<specimage list>, <filter spec>
Options– <targetid>: pointing id of your observation. Typically, ’1234567’. Check your data
distribution.
– <targetsubid>: pointing sub-id of your observation. Typically, ’1’. Check yourdata distribution.
– <ext source table>: an ascii ﬁle that speciﬁes the source coordinates in X and Y
in units of pixels. If this is set to be ” (null), then automatic source detection program
will be enabled within the pipeline.
– <refimage list>: an ascii ﬁle that speciﬁes the FITS ﬁle names of the reference
images to be processed. Typically, either ’N3.lst’, ’S9W.lst’, or ’L18W.lst’– <specimage list>: an ascii ﬁle that speciﬁes the FITS ﬁle names of the spectroscopy images to be processed. Typically, either ’NP.lst’, ’NG.lst’, ’SG1.lst’,’SG2.lst’, or ’LG2.lst’– <filter spec>: an code to specify the data processing mode, or a pair of ﬁlter anddisperser names for the processing. This should be one of the following: N3 NP, N3 NG,
S9W SG1, S9W SG2, or L18W LG2.
– /Nh spec, /Ns spec, /Np spec, /Ls spec: ﬂags to specify the slit spectroscopy
data reduction.
– /no tune sourcepos: a ﬂag to disable source centering functionality (ﬁne tuning ofsource positions with local Gaussian ﬁtting around the user-speciﬁed source locations)when the toolkit is run with user-speciﬁed source table.
– /use short refimage: a ﬂag to use short reference image, rather than long reference
image, for source detection. This is good for bright, saturated sources.
– /no slit flat: a ﬂag to disable ﬂat-ﬁeld correction for slit spectroscopy data (forNp spec, Ns spec, Nh spec, Lc spec). Use of this switch is recommended for slit
spectroscopy with the current version of the toolkit/calibration database.
– /nir columnpulldown correction: a switch to enable correction for column pulldown eﬀect for Np spectra.
– savefile: an ascii variable that store the ﬁle name of the IDL save ﬁle that records
the irc specred output.
• Displaying whole image products
Usage:
show aperture on ds9,specimage bg,source table

IRC Data User Manual for Post-Helium Mission (Phase 3)

84
orshow aperture on ds9,refimage bg,source table,/imag
options– nsum: width of source extraction apertures in units of pixel, corresponding to thewidth of red rectangles shown on ds9.
– space shift: number of pixels by which to shift the source extraction apertures
along spatial direction.
– /imag: a ﬂag to display reference images. Without this ﬂag, the procedure assumes
that the data to be displayed are spectroscopy images.
• Plot spectra
Usage:
plot spec with image,wave array,specimage n wc,mask specimage n,source table,<source id>
options previously:– <source id>: a number that specify the source to be plotted. You can ﬁnd thenumber by looking at the ds9 window that displays whole image products.
– /short: a ﬂag to display short-exposure spectra. One needs to set specimage wc short,instead of specimagen n wc in the plot spec with image command.
– nsum: number of pixels along X axis combined for plotting. This nsum parameter also
works for /diffuse option.
– smooth: boxcar smoothing width (in pixel) along wavelength direction.
– /sigma filter: a ﬂag to enable sigma ﬁlter operation (at 3 sigma signiﬁcance level)
over 2D spectra to remove spatially isolated high or low count pixels.
– space shift: shift of the plot extracting box along X (spatial) axis in pixel.
– /no mask: a ﬂag to disable masking functionality, i.e., plotting spectra regardless ofthe possible source overlapping.
– ps: root ﬁlename of the PS graphical output (<ps>.ps) of the spectral plot.
– png: root ﬁlename of the PNG graphical output (<png>.png) of the spectral plot.
– ascii: root ﬁlename of the ASCII numerical output (<ascii>.spc) of the spectra.
– /with image: a ﬂag to display 2D image below the spectral plot.
– /diffuse: a ﬂag to convert ﬂux scale to MJy/str for slit spectra.
– /no aperture correction: a ﬂag to disable aperture correction factor in plotting
the spectra.
– xrange, yrange: parameters to specify plot ranges in X and Y axes, respectively.
– /xlog, /ylog: a ﬂag to plot spectra in logarithmic scale along X and Y axes, respectively.
• Tweak wavelength zero-point manually
Usage:
change wave offset,<wave offset>,source table,
space profile,spec profile,wave array,specimage fc 1d,

Version 1.1 (August 28, 2009)85

refimage bg,specimage bg,
refimage n bg,refimage n bg short,specimage n sff,specimage n sff short,specimage n wc,specimage n wc short,specimage n fc,specimage n fc short,
res refimage bg,res specimage bg,
mask refimage,mask specimage,mask specimage n,
imagheader,specheader,imagheader extract,specheader extract,noisemap,noisemap n,noisemap short n,noise mask,
dir=!IRC SPECRED DIR.workdir,savefile=savefile,/write indiv spec
– <wave oﬀset>: oﬀset along wavelength direction in pixel units.

6.8

Error messages when running the pipeline and Troubleshooting• In general double quotation mark (“) can cause problems when calling the toolkit
in IDL. For example:
restore,"12345678 some more information.sav"
orirc specred,"12345678 some more information.tbl",’SG1.lst’,’SG2.lst’,S9W SG 1
give IDL syntax error, We suggest to use ’ (single quotation mark), rather than ” (double
quotation mark) anytime when you work with the IRC SPECRED toolkit. For more, checkthe IDL manual (search ’quotation’).
• When one of the input parameters of the toolkit is a null string, no space between
single quotation marks should be given. For example:
irc specred,’ ’,’SG1.lst’,’SG2.lst’,’S9W SG1’
will give an error message, because the toolkit tries to ﬁnd the source table whose nameis ’ ’ (blank space), and it fails. The correct syntax is:
irc specred,’’,’SG1.lst’,’SG2.lst’,’S9W SG1’
• If IDL complains about some undeﬁned procedures/functions, probably something is wrong in the setting:– Check .cshrc carefully, and/or.
– After launching the IDL, issue print,!path on the IDL command line. This command shows all the paths for the IDL to search the libraries. See if all the TOOLKIT
directories are included in !path.
• If after running the pipeline several times successfully, it fails with some unknown reasons,
try resetting the IDL session, by issuing .reset command at the IDL command line. This
resets everything stored in the IDL memory, including IDL common and environmental
variables that are referred/updated during the toolkit session. Some old settings mightcause the trouble. We recommend to reset every time the toolkit is restarted after apipeline processing failure to avoid any confusion regarding old IDL variables. Remember,
however, that the .reset command will destroy even the main IDL memory contents that
contain information for, e.g., plotting the spectra. Of course, users can restore the IDL
save ﬁle contents back to the main IDL memory to resume the data analysis.

86

IRC Data User Manual for Post-Helium Mission (Phase 3)• The ﬁlename where the toolkit saves the information is always<targetid>.<targetsubid>.<filter spec>.IRC SPECRED OUT.sav.
Therefore the savefile is a string variable output that records the save ﬁle name, not an
input to specify the save ﬁle name. If the pipeline is run more than once with diﬀerent
options, the saveﬁle should be renamed before the second run in order to be able to recover
the previous results. You can recall the record even if you rename the save ﬁle name, by
issuing:
restore,<renamed save file>.
• If you want to see the results of previous toolkit run, recalling the save ﬁle, and you
do not remember the ID of your source, issue:
show aperture on ds9, <image>, source table, (/imag) as instructed at section 6.5.1.
This launches a ds9 and displays image and region marks on it. Do not forget to include
source table after <image>.
• There are no good ways to run more than two pipeline processes at a same time
on a single computer as a single user, because more than two toolkit processes share somesame dS9 windows. Please use other computers or other user account.
• There are no good tools to coadd data taken over multiple pointings. This is beyond the purpose of the toolkit right now. The simplest way is to average/combine extracted 1D spectra after dumping the results on ASCII ﬁles, with the ASCII option of theplot spec with image command and coadd them with any available tool. Note that the
wavelength for a given pixel is diﬀerent from pointing to pointing for slit-less spectroscopy
data, even if the same target was observed with the same pointing coordinates. One mayneed to rebin the spectra for a common grid for proper stacking.
• There are no good tools for plotting spectra for a given source taken with diﬀerent
dispersers at once. One needs to dump the spectra with the ASCII option when plotting
the individual spectra, and then use your favorite plotting software for plotting multiple
spectra once.
• If the spacecraft attitude shifts after the reference frames have been taken, theeﬀect will be apparent in the wavelength calibration. A shift of more than 1-2 pixels canhappen. For NP only, the signal at the peak of the RSF is used to correct the wavelengthreference by default. The result will be inserted into the log ﬁle which will be further used
to correct the drift in SG1/2 and LG2. Therefore the NIR data should be processed ﬁrst.
Similarly, for NG in the point source aperture (Np), the NG slitless spectroscopy ﬁeld should
be processed ﬁrst. The actual correction is made by shifting the wavelength array rather
than the image array.
• Strange features found in the NG spectra taken at Np aperture can be due either
to ﬂat-ﬁeld problem or column pull-down. See section 6.6.1 or 6.6.2.
• Strange features in the NP spectra taken with the big aperture (for Nc pointing)will be due to wrong calibration only if the features are seen not only in your target object
but also in nearby ﬁeld stars. See section 6.6.3 for more information.
• There are several kinds of diﬃculties in matching the ﬂux level of the same source
across the dispersers, especially in slit-less spectroscopy mode, and jumps in ﬂux levelcan be found between spectra from diﬀerent dispersers in the same source. See section 6.6.4
for more details.

Version 1.1 (August 28, 2009)87• If unexpected narrow emission-line objects are found, it could be due to zero-thorder light images of nearby brighter stars. See the ds9 window carefully since it showsthe expected location of such features with yellow circles.

6.9

Tips for the Phase 3 data reduction

The evaluation of the phase 3 data is ongoing. Here we summarize the caveats and tips of thephase 3 data reduction. In the phase 3 data reduction special options are prepared to deal withthe saturation of the reference image.

6.9.1

AOT IRCZ4In phase 3, 5 dark frames are taken before and after the observations of your target. The
observation sequence is the same as in phase 2: 4 spectroscopic observations, one reference
imaging with N3, and 4 spectroscopic observations.
In phase 3, the number of hot pixels increases signiﬁcantly. To secure redundancy, automatic dithering scheme is implemented for multi-pointing observations both for slit-less and slit
observations. For Np observations, the pointing position is shifted by 3 arcsec in the X or Y
direction, whereas 6 arcsec shift is executed for Nc. For Ns and Nh slit observations, a shit of 4
arcsec in the X direction is executed in multi-pointing observations.

6.9.2

Spectroscopic performance

The telescope image quality is degraded in phase 3 by 10% (Onaka et al. 2008). This leads toa degradation in the spectral resolution of the slit-less, including Np, spectroscopy as well as
the revision of the aperture correction. In most case, the degradation in the resolution is not
signiﬁcant except for the region shorter than 2.5 μm with NP, where the calibration becomes
more diﬃcult. However, no change in the spectral response and spectral dispersion is noticed.
According to the change of the point spread function, a new aperture correction table isprovided for phase 3.

6.9.3

Dark current, linearity, and hot pixels

In phase 3, the dark current is increased and the number of hot pixels is also increased significantly. To accommodate these changes, the array operation was adjusted, which reduces thesaturation level signiﬁcantly (see the Imaging cookbook). A new linearity correction, which isthe same as for imaging observations, is applied to phase 3 data.
The decrease of the saturation level makes observations at Nc with NP at high backgroundregions (low ecliptic latitude regions) very diﬃcult. The background level becomes very close to
the saturation and only a small amount of the dynamic range for point sources is left.
Because of the increase of hot pixels, dark images created from the 10 dark observations
before and after the target observations in the same pointing is applied in the phase 3 data
reduction. The toolkit for phase 3 have several options to deal with this situation (see section 6.9.5), such as the usage of short-exposure frames to detect point sources in the slit-lessmodes (Nc and Np).

6.9.4

Response

No change is noticed in the relative spectral response in phase 3. However, the absolute sensitivity is estimated to be 70% of the phase 2 based on observations of the standard stars. This

IRC Data User Manual for Post-Helium Mission (Phase 3)

88

is compatible with the result of imaging observations. The 0.7 factor is included in the phase 3
toolkit.
On the other hand, there is a problem unclear at present in the absolute calibration in
short-exposure frame data. Therefore short-exposure data cannot be used to estimate the ﬂux
accurately.

6.9.5

New options for the phase 3 data reduction

In irc specred we have the following new options.
• /use short refimage: This option allows the program to use the short-exposure image
for the detection of point sources when they are saturated in the long-exposure image.
• /no long saturation mask: This option dismisses the saturation mask for long-exposure
frames. This option is useful for high-background observations, where the frame-to-frame
shift measurement fails with the mask. Try this option if you found unrealistic values in
the X or Y shifts between frames.
• /use short for wave offset: This option allows the program to determine the zero-pointof the wavelength in the short-exposure frame. This option works in a similar way for the
spectroscopic images to /use short refimage for the reference image data.
Try this option if the software cannot measure the zero-point of the wavelength.
• /short saturation mask: With this option, the saturation mask of short-exposure framesof spectroscopy data is used to calculate the wavelength oﬀset. This is applied only forthe spectroscopic data.
For the phase 2 toolkit, the saturation mask is applied only for long-exposure frames.
Similar saturation masks cannot be applied to the short-exposure reference image because
saturated pixels are treated as hot pixels and corrected by interpolation. In the present
software there is only one mask data, which include hot pixels and saturated pixels.
• /use short for calc x shift: This option allows the software to determine the shift
between the spectroscopic images and the reference images from the short-exposure frames.
With too many hot pixels, it becomes diﬃcult to determine the shift and this option may
be useful.
For plot spec with image we have the following option for the phase 3 data reduction.
• median = 3: Whereas the shift-and-add process reduces the number of masked pixels in
slit-less observations, the reduction of masked pixels is not expected for slit observations(Ns and Nh) and hot-pixel masked pixels are often seen in the ﬁnal stacked image. With
the option median = 3, the data are median-averaged in the spatial direction to makeup the masked pixels. This will be useful for observations of extended sources, but it isintroduced only to make the spectrum look better. For scientiﬁc purposes, users should
apply more sophisticated methods.

6.9.6

Installation and usage of the toolkit for phase 3

The installation of the toolkit is the same as that for phase 2 except that the software will be
installed in the directory irc spec toolkit p3. You should set the path to this directory. To
start the toolkit, it is the same as for phase 2 for the observation of 12345678.1 (obsID.obs#) as
IDL>irc specred,12345678,9,"N3.lst","NG.lst","N3 NG", root dir="dir/"Version 1.1 (August 28, 2009)89

No change is needed also for plot spec with image.
The toolkit automatically recognizes AOTZ4 data from the FITS headers.
The software irc specred should be executed without any new options ﬁrst and follow thesame sequence as for the phase 2 data.
If your sources are too bright and saturated in long-exposure reference image, then use theoption /use short refimage.
If you ﬁnd that a large fraction of the area is saturated in the long-exposure frames ofthe spectroscopic images, try the options /use short refimage, /no long saturation mask,/short saturation mask, /use short for wave offset, /use short for calc x shift.

6.9.7

Known problems

Some of the problems described below should hold also for phase 2 data.
• As describe above, the absolute ﬂux calibration for short-exposure frames is still underinvestigation. It is diﬀerent from that can be expected from the diﬀerence in the integration
time from long-exposure data. At present the cause is not yet fully understood.
• New ﬂats for spectroscopic image data of phase 3 are not yet ready. New ﬂats will be
provided in the future. For slit observations (Ns and Nh), it is still recommended to use
the option /no slit flat, which give better S/N spectra in most cases.
• Although we conﬁrmed the spectral performance (dispersion) is not changed in phase 3
and no change has been applied in the relative spectral response, the calibration of NP
data shorter than 2.5 μm may be changed with a large uncertainty because of the worse
PSF of the telescope in phase 3. Users should use NP spectra shorter than 3 μm withspecial caution.
• The wavelength calibration for the shortest end of NG (2.5 − 2.6 μm) has a systematic
error of a pixel (∼ 0.05 μm). For longer wavelengths no systematic deviation is seen so far.
This problem is under investigation.
• The calibration of NG for wavelengths longer than 4.9 μm has an error due to the overlapof the second-order light. In the present calibration, the data of blue standard stars areused. Therefore, it is underestimated and red sources often show a decline at the end of
the NG spectrum. This is not a true feature. Since this is a problem originating from thesecond-order light, there will be no recipe to correct for it for objects that have diﬀerent
colors from standard stars.