ALMA Science Archive (ASA) End-to-End Archive Primer 2023
ToshikiToshikiSaito
Saito- -NAOJ
NAOJALMA
ALMAProject
Project2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

This talk will…
- make new users feel comfortable to start archive-based science.
- cover from downloading data from the ALMA archive to moment map creation step-by-step.
- contain all the relevant information (e.g., references, documents) as much as possible.
- target new students who just start archive science, but without PC skills and knowledge of interferometer.
- assume data reduction happen in the ADC MDAS server.
- 新しいALMAユーザーがALMAアーカイブを触るハードルを下げたい。
- アーカイブからデータをダウンロードするところから、モーメントマップの作成までをステップ毎に紹介。
- アーカイブサイエンスを進めるにあたって、必要情報が載っている書類や論文などを全てカバーする。
- アーカイブサイエンスを始めたばかりの学生を想定。PCスキルや干渉計の知識がなくても理解できる入門書。
。。。がこのスライドの目標です。。。

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

2

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

3

Step 0: Get to the Archive
Step 0a: Google it※1.

Step 0b: Click this website.

Step 0c: Mouse over the “Data” tab, then click “Archive”.

This is the archive home page.

Step 0a’: Or click the link in the lower-right corner of this slide.

※1: if you are in East Asia, archive mirror in NAOJ should be fast in theory.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

https://almascience.nao.ac.jp/aq/

Toshiki Saito (toshiki.saito@nao.ac.jp)

4Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

5Step 1: Checking data quality and usefulness quickly- Let’s search for two nearby galaxies “NGC 1068”, and a single molecular line N2H+(J=1-0).
- This time, you want data with spatial resolution better than ~2 arcsec.
- NGC 1068 is at z = 0.00379.
- According to Splatalogue※1, the observed (redshifted) frequency is 92.82160 GHz@NGC 1068.

Step 1a: Mouse over the “Search” bar in the top-left corner and specify some boxes※2.

※1: a spectral line database which can deal with redshift and others (https://splatalogue.online//).
※2: if a box becomes green, it’s ok. If it becomes red, something is wrong.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

6Step 1: Checking data quality and usefulness quickly- OK, seven observations were done for the NGC 1068’s N2H+(1-0) line.

Step 1b: Check the details of the observations※1.

Step 1c: Check the observing positions and areas.
In this speci c case, NGC1068 has new SN in the northern arm. Projects observing
this SN may not cover the full disk of NGC1068. This can be graphically checkedby mousing over one project row. Then, a yellow circle on the Figure will behighlighted interactively.
This time, I go for 2019.1.00130.S, because it seems to cover the major part of thedisk, spatial resolution < 2.0 arcsec※2, and newest project satisfying these criteria.
You can check published papers utilizing this data in “Publication” column.
※1: You can scroll to the right for more info.
※2: Numbers shown in this list are just to give you ideas of the data quality. Not accurate sometimes. You need to check the actual numbers later.

fi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)Toshiki Saito (toshiki.saito@nao.ac.jp)

7Step 1: Checking data quality and usefulness quickly
Step 1d: Mouse over this icon.

Step 1e: It shows thumbnails of the FITS
products and data info※1. Click the CARTAlogo.
The second ts cube (i.e., second spw) contains theN2H+ line (at 92.82160 GHz).
Probably, it’s good to remember the freq. resolution.
(But I don’t believe the line sensitivities listed here.)

Step 1f: Click the CARTA logo.
※1: This contains continuum map, line peak map, and a spectrum toward the peak position of the spw. If you click the left panel, you can see a more detailed plots.

fi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

8

Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④

①

③ ②

① draw an aperture.
Product cubes are corrected for primary beam. So,
we measure the rms level around the center of linefree channels.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

9Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④

①

③ ②

② measure rms.
What we are interested in is “RMS”. The achieved
rms level is ~0.29 mJy/beam at 1.9 MHz resolution(= 1.9/92000*300000 ~ 6 km/s resolution).
This is an useful number to assess the quality.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

10Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④①③ ②③ Check spectrum.
You can draw a spectrum for an aperture youde ned. Now you already put an aperture, so you
only need to push.
The N2H+ line is strongly detected around 92.82GHz!

fi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

11Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④①③ ②③ Check spatial distribution.
If you drag the red vertical line, the cube imagechanges accordingly.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

12Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④

①

③ ②

③ Quick look moment-0 map.
Click2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

button, change parameters. Then “Generate”.

Toshiki Saito (toshiki.saito@nao.ac.jp)

13

Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④①③ ②③ Quick look moment-0 map.
Clearly detected!

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

14

Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④①③ ②④ Check the header. Click “View” tab, thenclick “File header”.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

15

Step 1: Checking data quality and usefulness quickly
Step 1g: Check some stats, spectra, moment-0 map of the FITS cube with 4 steps.
④

①

③ ②

④ Remember angular resolution, etc.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

16

Step 1: Checking data quality and usefulness quickly
Step 1h: Is this project su cient for your science?
- We found seven observations for the NGC 1068’s N2H+(1-0) line, and selected one project (2019.1.00130.S).
- The achieved rms level for the N2H+(1-0) line is ~0.29 mJy/beam at ~6 km/s resolution.
- The beam size is 2.6 arcsec x 2.3 arcsec.
- N2H+(1-0) line is clearly detected, but there is no clear detection of the other lines in this spw.
- If you need more sensitivity, angular resolution, spatial coverage, etc, digging into the archive! You can combine multiple ALMA projects!

ffi

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

17

Step 1: Checking data quality and usefulness quickly
Step 1i: Archive check sheet. This may contain everything what your supervisor wants to know at rst…
SheetExample

Project code

Project code

2100.1.00001.S

PIPI

Taro Yamada

Project title

Project title

Dense gas in NGC 1068

BandBand3

Spatial resolution (“)

Spatial resolution (“)~2.6x~2.3

Velocity resolution (km/s)

Velocity resolution (km/s)~6

Sensitivity (mJy/beam)Sensitivity (mJy/beam)

0.29

Frequency coverage

91.0~93.0 GHz; 93.0~95.0 GHz;
106.0~108.0 GHz; 108.0~110.0 GHz

Notes (e.g., detected lines)

fi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Notes (e.g., detected lines)

N2H+ (con rmed)
13CO, C18O covered in other spws

Toshiki Saito (toshiki.saito@nao.ac.jp)
fiFrequency coverage

18

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

19Step 2: Reviewing data quality a bit more seriously
Step 2a: Download the QA2 report. This is really informative.
Go back to the thumbnails of the FITS data in the archive query. There is actually “QA2 report” tab on top of the page.
If you click it, QA2 report PDF will be downloaded.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

20

Step 2: Reviewing data quality a bit more seriously
Step 2b: Go through the QA2 report.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

21

Step 2: Reviewing data quality a bit more seriously
Step 2c: Check the Final QA2 comment.
This “Final QA2 comment” is lled by the Data Reducer, who was in charge of the quality assurance of this data.
This should contain all the Data Reducer’s concerns and actions (e.g., agging).

Toshiki Saito (toshiki.saito@nao.ac.jp)
flfi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

22

Step 2: Reviewing data quality a bit more seriously
Step 2d: Check the achieved rms of the data cube.
In addition, this report contains data quality summarized by the Data Reducer.
Remember our assessment for the cube containing N2H+(1-0). Our quick assessment was actually good!
Our rms value: ~0.29 mJy/beam at ~6 km/s resolution.
Our beam size: 2.6 arcsec x 2.3 arcsec.
Note that N2H+ is in spw29. The cube info shown in the QA2 report is for the representative spw. In this case, spw25.
Thus, some values can be di erent.

ff

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

23

Step 2: Reviewing data quality a bit more seriously
Step 2e: Check requested and achieved rms levels.
If “RMS at the PI requested spectral resolution” is worse than the “sensitivity goal”, you may want to gure out the reason(because something should’ve happened in the data). In this speci c case, achieved rms < requested rms as expected.

fi

Toshiki Saito (toshiki.saito@nao.ac.jp)fi2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

24

Step 2: Reviewing data quality a bit more seriously
Step 2f: Check the CASA and pipeline version.
Memorize it somewhere.

↑ a hint for processing time2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

25

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data → See Pei-Ying’s talk!
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

26

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

27

Step 4: Running the calibration & imaging pipeline
Step 4a: Go to the CASA webpage.

Step 4b: Download CASA+pipeline with appropriate version (e.g., OS). In this case, CASA 5.6.1-8.
Step 4c: Click to download.

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

28

Step 4: Running the calibration & imaging pipeline
Step 4d: Upload the tarball to the MDAS server, and untar it.
In this case, I created “casa” directory in the home directory of the MDAS server.
$ cd Downloads/$ rsync -av --inplace --progress --partial —append casa-pipeline-release-5.6.1-8.el7.tar.gz
saitodesu@kaim02.ana.nao.ac.jp:/home02/saitodesu/casa$ ssh saitodesu@kaim02.ana.nao.ac.jp -XYC
$ cd casa/$ tar zxvf casa-pipeline-release-5.6.1-8.el7.tar.gz$ ls
casa-pipeline-release-5.6.1-8.el7 casa-pipeline-release-5.6.1-8.el7.tar.gz$ rm -rf casa-pipeline-release-5.6.1-8.el7.tar.gz # if you don’t need the tarball.

Step 4e: Upload the download script for the ALMA data, and download (this takes time).
In this case, I created “ngc1068_n2hp10” directory under /lfs02/saitodesu/.
$ cd Downloads/$ rsync -av --inplace --progress --partial —append download-*-files.shsaitodesu@kaim02.ana.nao.ac.jp:/lfs02/saitodesu/ngc1068_n2hp10/$ ssh saitodesu@kaim02.ana.nao.ac.jp -XYC
$ cd /lfs02/saitodesu/ngc1068_n2hp10/$ sh download-*-files.sh # after dowloading, you will be asked if you want to untar the data.
$ rm -rf ???*.tar # if you don’t need the tarballs.
2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

29

Step 4: Running the calibration & imaging pipeline
Step 4f: run the pipeline (this takes time).
$ ssh saitodesu@kaim02.ana.nao.ac.jp -XYC
$ cd /lfs02/saitodesu/ngc1068_n2hp10/science_goal.*/group.*/member.*/$ cd scripts/$ /home02/saitodesu/casa/casa-pipeline-release-5.6.1-8.el7/bin/casa --pipeline
CASA <1>: execfile(‘member.*.scriptForPI.py’)
If you are familiar with background processing with nohup/screen commands, it’s useful because you want to keepprocesses running even after ending ssh session (pipeline sometimes takes ~days).
If not, you need to keep the ssh session, e.g., over night. Don’t forget to charge and properly setup your PC, then you
can turn o the screen but keep PC running.

ff

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

30

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

31

Quickly reviewing weblog
There are some useful introductions and primers available.
- “Reviewing the WebLog”, George Bendo (UK ALMA Regional Centre Node)- “I-TRAIN #4: ALMA WebLog inspection”, George Bendo (UK ALMA Regional Centre Node)- presentation slides, YouTube video, and weblog examples are available in the website.
- “I-TRAIN #18: The ALMA Single-Dish Pipeline”, Daniel Tafoya (Chalmers University)
If you want to know more… (also this is the ALMA pipeline paper you want to cite in your paper)- “The ALMA Interferometric Pipeline Heuristics”, Todd Hunter et al. 2023, PASP, 135, 4501

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

32

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

33Checking imaging
There are some useful introductions and primers available.
- “I-TRAIN #1: Imaging with the ALMA Pipeline”, Daniel Tafoya (Chalmers University)- “I-TRAIN #17: Introduction to tclean”, Ashley Bemis (Leiden University)

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

34

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

35Come soon… meanwhile your supervisor can help…

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

36

Steps1. Checking data quality and usefulness quickly
2. Reviewing data quality a bit more seriously
3. Downloading data
4. Running the calibration & imaging pipeline
5. Checking calibration
6. Checking imaging
7. Re-doing imaging with own recipes
8. Creating moment maps

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)

37Creating moment maps
There are some useful introductions and primers available.
- “MAPS TO SCIENCE: 近傍銀河”, EA-ARC Support (in Japanese)- “MAPS TO SCIENCE: 遠方銀河”, EA-ARC Support (in Japanese)- “TWHydraBand7 Imaging 4.2”, CASA Guides

2023-12-22 (ALMA/45m/ASTE Users Meeting 2023)

Toshiki Saito (toshiki.saito@nao.ac.jp)38