IRC Data Reduction Workshop
9/18-19/2007

The IRC Spectroscopy:
Data and their Calibration
Youichi Ohyama
AKARI Project Researcher/ISAS/JAXA
On behalf of the IRC spectroscopy datareduction teamReferences• Documents on AKARI web site
– AKARI/IRC observing manual– AKARI/IRC Data User Manual– Documents for today’s workshop• Ita et al. for imaging (previous presentation)• Ohyama et al. for spectroscopy (this presentation)• Papers– The Infrared Camera (IRC) for AKARI --- Design andImaging Performance• Onaka et al. 2007, PASJ, in press, or astro-ph/07054144– Near-infrared and Mid-infrared Spectroscopy with theInfrared Camera (IRC) for AKARI• Ohyama et al. 2007, PASJ, in press, or astro-ph/07084290– Properties of UIR Bands in NGC 6946 Based on Mid-Infrared
Imaging and Spectroscopy with IRC on Board AKARI• Sakon et al. 2007, PASJ, in press

Quick Review of the IRC
Spectroscopy Mode

Infra-Red Camera (IRC) as aSpectrograph• Telescope/Satellite operation– same ones that for imaging mode (pointed attitude)• Main Optics (Collimator/Camera)– same ones that for imaging mode.

• Array and its operation (clock/exposure time)– same ones that for imaging mode.

• Slit– Wider aperture for imaging + slit areas– For slit-less and slit spectroscopy• Disperser– prism or grism, mounted on filter wheel

Ita san’stalkThe Slit-less Mode:
Very Basic Ideas for Slit-less Spectroscopy• Reference (direct) image is taken.
– Source locations will be measured on the reference image.
– The source coordinates are used as…
• Origin of the spectral image extraction.
• Origin of wavelength calibration.

• Spectroscopy image is reduced…
– First in a similar way as for conventional direct imaging.
– Then, after spectral image extraction, in a similar way asfor conventional slit spectroscopy.

The “Slit-less” Concept
direct image spectral image

YY,λX

X
84

LG1 L24
12 CAL

L15 0

L20W LG2
1620

Selectable filter wheel

The (conventional) Slit Mode:• Observation (Satellite/Array/Filter wheel controls) will
be made in a same way as for the slit-less mode.
– Targets are placed on some slit. That’s it!
– Single AOT works for both slit-less and slit modes.
• You will obtain similar kinds of data set.

• Data reduction will be made in a very similar way as
for the slit-less mode:– The same toolkit works on both slit/slit-less data…
• With only minor changes:– Reference image is taken, but it is not essential to locate the slit.
– Shift-and-coadding of individual exposure frames is disabled.
– No local sky-subtraction is possible.

• The toolkit accepts the option for slit-mode data reduction.

Basic Characteristics of IRC
Spectroscopy Data

Review of Raw/Calibration Data
Rawdata in ‘/rawdata’• Raw images (F*_[M/N].fits)• List files (*.lst)• Information files (README, etc.)

AOT04a Raw Data (NIR)
NP

NP

NPNP

refimage

NP

NP

NP

NP

wavelength

Rotated (bypre dark

NP90deg)maneuver post darktime

AOT04a Raw Data (MIR-S/L)
LG2

MIR-L

SG1

SG1

SG1

MIR-S

LG2

L18W

LG2

LG2

SG1

S9W

SG2

SG2

LG2

LG2

SG2

SG2

LG2
wavelength

wavelength

LG2maneuver

SG1/2

LG2

time

Calibration Data in‘/CALIBDIR/*’• DARK/• LINEAR/• MASK/• FLAT/• RESPONSE/• APCOR/• COORDOFFSET/• DISTPAR/• WAVEPAR/

super-darks
linearity correction tables
mask images
super-flats (imag/spec)
spectral response tables
aperture correction tables
coordinate offset tables
spectral tilt correction tables
wavelength calibration tables

Spectroscopy Flats
NPNG

SG2

SG1

Fade near Y=Ymax

LG2
Fade near Y=0

Ridge due tozero-th order lightThroughput& Spectral Response

0.5
NP pseudo-peakat 2.4um

0• Throughput: ratio of incident/detected photon number• Spectral Response: data number (DN) per frame per unit incident
light energy (Jy) on a pixel– NP spectral response has a notable peak where throughput shows local
maximum, due to lower dispersion (wavelength per pixel).

Calibrating the Data

coord. trans.

A(Xs, Ys)

B

B

Dark sub.
Flat for sky
Sky sub.

B

JyADU’ADU

pixel

wave. calib.
pos./λ-dependent flatfor obj. A
λone-dim. cut

λSens. response corr.
(pos. independent)

JyADU’ADU

B

wave. calib.
pos./λ-dependent flatfor obj. BpixelλABsourceextraction

AOT4 input dataA

cut along disp.

(Xi, Yi)sourcedetection

ASourceoverlapping?
Is aperturesize OK?

cut along disp.

AdispersedSpec.

Ref. imagλProcesses in Reducing Spectroscopy Data•

Processes common to conventional imaging with large-format arrays,or IRC Imaging data processing––––––•

Dark subtraction (hotpix subtraction)
Linearity correction
Flat fielding
Sky subtraction
Shift & add-ing individual frames
Source detection

Processes common to conventional spectroscopy– Wavelength calibration– Flux calibration– Extracting 1D spectra•Processes that are unique to the IRC spectroscopy––––

Measuring shift among subframes
Spectral image extraction
Flat fielding/Color correction for slit-less spectroscopy
Wavelength calibration for slit-less spectroscopyCalibration

START

Flowchart

To be continued.

From previous page

Multiple pointing data
not covered by the IRC_SPECRED

Overview of Basic Calibration
Processes• [REF]: procedure of reference images• [SPEC]: procedure of spectroscopy images• [REF/SPEC]: procedure of both• [WHOLE]: procedures of whole image,before source extraction• [EXTRAC]: procedures of extractedimages.

Dark Subtraction[REF/SPEC/WHOLE]• We subtract scaled-superdark fromobserved images.
– Superdarks are in the CALIBDIR.
– Scaling is done by measuring dark level ofsuperdark/pre-dark/observed images.
• At the slit-masked area.

Applying bad-pixel/slit Masks[REF/SPEC/WHOLE]• Bad-pixel masks are provided in the CAL.
– The same ones for the imaging data processing.

• Slit masks will be also applied for slit-less
spectroscopy data.
– This process will be skipped for slit data, of course.

Flat Fielding[REF/WHOLE]• Dividing the dark-subtracted images withthe super-flat for flat fielding

Sky Subtraction[REF/WHOLE]• Sky is subtracted from each subframe.
– By globally fitting the sky over the FOV,
– With object rejecting algorithm,– With masks in the second path.

Frame Stacking[REF/WHOLE]• NIR: no image stacking is possible.
• MIR-S/L: shift-and-coadd three subframeswith median combine mode.
– To remove cosmic rays

Object Detection[REF/WHOLE]• Objects will be detected on the reference
images with DAOFIND (IDL version).
• The detection parameters can be changed interactively
and iteratively within the toolkit.
– noise level, detection threshold over the noise level, andsource size

• The toolkit accepts a user-created source
table as an input, if supplied.
– If the list is set, the toolkit skips the object
detection procedure.

Flat Fielding[SPEC/WHOLE]• Divide each subframe by spec-flat images.
– This would create globally-flat background.

Sky Subtraction[SPEC/WHOLE]• Sky is subtracted from each subframe.
– By globally fitting the sky over the FOV,– With object rejecting algorithm,– With masks in the second path.

Frame Stacking[SPEC/WHOLE]• Shift-and-adding sub-frames– While removing cosmic-ray events.

• Shift value is measured on the NIR– Even for MIR-S/L– Based on pseudo features in NP or NG spectra.

Spectral Extraction[SPEC/WHOLE]• Spectral images of each source are extracted– The toolkit converts source coordinates on theREFIMAGE into that in the SPECIMAGE.
• based on the CAL information.

– Extraction boxes include surrounding sky area.

• Spectral masks will be also created.
– To find possible source overlapping.

Wavelength Calibration[SPEC/EXTRC]• Basically no image transformation is made, rather• Wavelength array will be created to relate Y pixel
and wavelength in um.
– For grisms…
• Wavelength=linear_function(dY, d_lambda, lambda0)– For prism…
• Wavelength=3rd-order-poly(Y)• Single wavelength array for all extracted spectralimages.

Sky Subtraction[EXTRAC/SPEC]• Any remaining sky is subtracted locally.
– Note: The sky is close to zero for most cases.
– Sky is an average of surrounding sky.

Color-term Correction[SPEC/EXTRAC]• Ideally, flat response is a function of both– Pixel (X and Y) and Wavelength (lambda)• But, spectral flat-fielding applied over thewhole image was a function of– Pixel, but not Wavelength.

• We need somehow correct colordependence of the flat response.
– After calibrating wavelength.

Flux Calibration[SPEC/EXTRAC]
Or spectral response correction• Flux(mJy, lambda)=count(ADU,lambda)/response(lambda)• Response is a 1D function, but actual flux
calibration is made on wavelength-calibrated
2D images.
– Then, extract 1D spectra in the plotting program.

The IRC_SPECRED
Data Reduction Toolkit• The toolkit is called ‘IRC_SPECRED’.
– The toolkit can be used for both reducing andreviewing the data
– The toolkit needs some interactive operations.

• A single toolkit works on both slit-less and slitmodes.
– Processing of the slit mode data requires a subset of theprocedures for processing the slit-less mode data.

• In the followings, I mainly describe the toolkit forthe slit-less mode.

Computer Environment for theToolKit• The toolkit is written with IDL.
• It also requires– DS9 FITS viewer– XPA program for communication between IDL and DS9.

• It is developed with IDL ver 6.1 & 6.2 on Linux PCs,but…
– IDL ver 6.0 or later should be fine.
– Solaris/other UNIX platform should be fine.
– Mac OS-X seems OK.

• The toolkit requires the ASTROLIB IDL library at
GFSC and others.

Review of Input/Output
Params/Products of the ToolKit[INPUTs]• File lists of raw data to be processed (mandatory):– Lists of a reference image and spectroscopy images.

• Target table (optional):– If users want to create their own target list with theirfavorite source detection programs, a target tableshould be specified as a toolkit option.

[OUTPUTs]
Main outputs:• Object catalogue, or table of object information– target location, brightness, size, coordinates of thespectroscopy apertures, etc.

• Processed WHOLE reference/spectroscopy
images:– dark-subtracted, flat-fielded, background-subtracted,stacked.

• Processed EXTRACTED reference/spectroscopy
images of each object:– Wavelength calibrated/color corrected/flux calibrated.

• IDL save files

Auxiliary outputs:• Masks– WHOLE mask images:• Images showing the object occupation onreference/spectroscopy images.

– EXTRACTED mask images for each object:• Images showing location of object overlapping, area of lost
information either due to out-of-chip or bad pixels.

• Residual images (after masking detected objects):– Combined images masked for the detected objects.
– The images could be useful to examine object detection
completeness, background subtraction quality, totalnoise quality, etc., of the toolkit.

• DS9 region files– for identifying extraction area, zero-th order lightoccupation, etc.

The IRC_SPECRED Directory<!--%;$(!>&<%"!=+%;($>/<%"!=+%;,0*($>&!"#$"%"&!"#'$"%"'#())'*+',%-!+$'.+!+/,-0!1+' %"*)+' ,.-0)$' *+' ,%-!+$' .+!+/(!1;,3+1!+$;-0%&%.+'(!1;,3+1!+$'3!-$01%,'#())'*+',%-!+$'.+!+/%23&
4.(,'(,'50,%'"'%+23-!"!6'$(!+1%-!6/4.+'1-7%+7%,'1"7'*+'!+2-8+$'(9'6-0')(:+/

Processing ‘Filter-Grism’ OrderNIR data provide some basic information forprocessing MIR-S/L data.
1.
2.
3.
4.
5.

NP or NG, without source table.
(NP or NG, with source table)
SG1 with or without source table.
(SG2 with or without source table.)
LG2 with or without source table.

Examples:••If you want to reduce MIR-S data, first reduce NIR, thenSG1/2.
If you want to reduce MIR-L data, first reduce NIR andSG1, and then MIR-L.

Preview of the Toolkit Operation• Input list files for this operation example
ohyama@cava: ls *lst
L18W.lst N3.lst NP.lst SG1.lst LG2.lst NG.lst S9W.lst SG2.lst DARK_NIR.lst
DARK_MIR.lst
ohyama@cava: ls *tbl
target_MIRS.tbl target_MIRL.tbl MYOBJECTS.tbl
ohyama@cava: cat N3.lst
F54919_N.fitsohyama@cava: cat NP.lst F54911_N.fits
F54913_N.fits
F54915_N.fits
F54917_N.fits
F54921_N.fits
F54923_N.fits
F54925_N.fits
F54927_N.fits

The default list files work just fine for most cases.

Example ofIRC_SPECRED Commands• Slit-less‘’=Null string– irc_specred,5020056,1,’’,’N3.lst’,’NP.lst’,’N3_NP’• Slit-less + target tablesingle quotation, not double!

– irc_specred,5020056,1,’MYOBJECT.tbl',’S9W.lst’,’SG1.lst’,’S9W_SG1’,root_dir=’/data/IRC/TEST'• Slit– irc_specred,1400043,1,’’,’N3.lst’,’NG.lst’,’N3_NG’,/Ns_spec,savefile=savefile
One command per line!

Step 1: The case of NP/no source
Issue the ‘irc_specred’ command
The LOG window
The IDL command line••

Left: Issue a toolkit command at the IDL command line.
Right: A log window will show up.

Step 2: Screen sub-frames

1st subframe

2nd subframe

3rd subframe

4th subframe• All sub-frames are shown on ds9 for eye-screening.
– Note: Typical AOT04a gives 8 or 9 sub-frames.

Step 3: Tweak ‘find’ parameters
Stackedspectroscopy image

Stackedreference image

Step 4: ‘All done’Step 5: Examine Spectrawith plot_spec_with_image• Restore the results before plotting by issuingrestore,savefileContents of ‘irc_specred_out’ Dir.

• Processed FITS images (*.fits)– <target_id>.<target_subid>.<filter-grism>.<image type>.fits••••

DS9 region files (*.reg)
Source tables (*.tbl)
Processing logs (*.log)
Database for next processing (*.dat)

Review of FITS Output
SPECIMAGE• Whole:REFIMAGE• Whole:– Refimage_bg– Refimage_mask– Residual_refimage_bg• Extracted:– Refimage_bg_indiv– Refimage_mask_indiv• Region file– Refimage.reg– Specimage_bg– Specimage_mask– Residual_specimage_bg• Extracted:– Specimage_wc_indiv– Specimage_fc_indiv– Specimage_mask_indiv• Region file
– Specimage.reg

Others
Source table.tbl
Processing log.log

[WHOLE IMAGE: SG1]

zoomedSlit mask (NaN)

specimagerefimage[WHOLE IMAGE: NG]wavelength

Slit mask (NaN)

specimagerefimage[WHOLE IMAGE: SG1 Masks]
refimage
specimage

image

masks

residual image

[EXTRACTED: SG1]
2D spectra of wavelengthcalibrated extracted sources(_WC)
30 objects in this example.

Extracted specimage: _wc vs. _fc
WC=Wavelength
Calibrated

FC=FluxCalibrated

Example of the Spectra
source_id=16

source_id=7

source_id=18

Demonstration