Doc 9.10, ver. 1.0 | Oct 2022

ALMA QA2 Data Products for Cycle 9

ALMA is a partnership of ESO (representing its member states), NSF (USA), and NINS (Japan), together with
NRC (Canada), NSC and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile.
The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ.

For further information or to comment on this document, please contact your regional Helpdesk through the
ALMA User Portal at www.almascience.org. Helpdesk tickets will be directed to the appropriate ALMA Regional
Center at ESO, NAOJ or NRAO.

Version

Date

Editors

0.1-0.4

January 2014

Dirk Petry

0.5, 1.0

February 2014

Dirk Petry

2.0, 2.1

September 2014

Dirk Petry

2.2

February 2015

Dirk Petry

3.0

November 2015

Dirk Petry

5.0

March 2018

Dirk Petry, Arielle Moullet, Mark Lacy

5.1

May 2018

Dirk Petry, Mark Lacy

7.12

February 2021

Luciano Cerrigone, Dirk Petry

8.2

February 2022

Fabrizia Guglielmetti

9.10

September 2022

Fabrizia Guglielmetti

Fabrizia Guglielmetti (ESO), Dirk Petry (ESO), Brian Mason (NRAO), Luciano Cerrigone (JAO), Harold
Francke (JAO), Misato Fukagawa (NAOJ), Andrés Guzmán (JAO), John Hibbard (NRAO), Erica Keller(NRAO), Mark Lacy (NRAO), Arielle Moulet (NRAO), Hiroshi Nagai (NAOJ), Kouichiro Nakanishi (NAOJ),
Theodoros Nakos (JAO), Tony Remijan (NRAO), Thomas Stanke (ESO), Catarina Ubach (NRAO) andMartin Zwaan (ESO)

In publications, please refer to this document as:
Guglielmetti, F. et al., 2022, ALMA QA2 Data Products for Cycle 9, Version 1.0, ALMATable of contents
1

Introduction .................................................................................................................................. 32

Group ObsUnitSet, Member ObsUnitSet, and Scheduling Block ................................................ 4

3

Overview ....................................................................................................................................... 5

4

Data Delivery Products ................................................................................................................. 8
4.1 README ................................................................................................................................. 8
4.2 Data Reduction Scripts (directory “script”) .......................................................................... 9
4.3 Imaging Products (directory “product”) ............................................................................. 10
4.4 Calibration Tables (directory “calibration”) ....................................................................... 12
4.5 QA Documentation (directory “qa”) ................................................................................... 12

5

Generating the calibrated visibilities ......................................................................................... 13
5.1 Running the scriptForPI.py in the case of manually calibrated data ................................. 14
5.2 Running the scriptForPI.py in the case of pipeline-calibrated data .................................. 18
5.3 Running casa_pipescript.py ................................................................................................ 21
5.4 Saving disk space during and after the execution of the scriptForPI.py ........................... 21
5.5 Splitting out the calibrated data at the end of running the scriptForPI.py ....................... 22
5.6 Generating the pipeline continuum subtraction products ................................................ 22
5.7 Manually calibrated TP data ............................................................................................... 22

6

Relevant documentation and help ............................................................................................. 231

Acronyms
In the following is a list of acronyms frequently appearing in the text:
ARC
ASDM
CASA
DMG
DP
EB
FOV
GOUS
JAO
MOUS
MS
PB
PI
QA
QA0
QA2
SB
TP
UID

ALMA Regional Center
ALMA Science Data Model
Common Astronomy Software Applications package
Data Management Group
Data Processing
Execution Block
Field Of View
Group ObsUnitSet
Joint ALMA Observatory
Member ObsUnitSet
Measurement Set
Primary Beam
Principal Investigator
Quality Assurance
First level of QA
Second level of QA
Scheduling Block
Total Power
Unique Identification number

2

1 Introduction
The goal of ALMA Quality Assurance (QA) is to ensure that a reliable final data product is delivered
to the PIs and that this has reached the desired control parameters outlined in the science goals,
meaning that it is calibrated to the desired accuracy and contains no significant calibration or
imaging artefacts. For Cycle 9, as for all previous Cycles, the production of imaging products during
QA will be done on a “best effort” basis.
The first level of QA (QA0) is determined shortly after observations, based on metadata and quicklook analysis. The second level of Quality Assurance (QA2) deals with QA at the level of data
reduction, where data processing (DP) experts at JAO, the ARCs and/or the ARC-nodes inspect the
products created by the ALMA pipeline or semi-interactively. This document describes the final
science data products that are archived and delivered to the principle investigators (PIs) at the endof the QA2 process.
The DP experts perform for each ALMA science dataset a detailed analysis, to confirm that theobservations have achieved the science goals requested by the PI. In particular, frequency setup,spatial setup, angular resolution, and continuum and/or line detection sensitivity are verified.
If the requirements are met within the cycle-dependent tolerances (see chapter 11 of the ALMA
Technical Handbook), the data are declared “QA2-pass”, packaged in a standardized way, anddelivered to the PI.
If, on the other hand, the requirements are not met, but additional observations of the scheduling
block are no longer possible within the cycle time allotted to the project, the data are declared“QA2-semipass” and are also delivered soon after it has become clear that re-observation is notpossible (the observing unit has “timed-out”). Data may also be delivered as QA2-semipass if it isdetermined that the data quality cannot be improved by further observations (for instance, in thecase of dynamic-range limited noise), or if all the datasets taken are classified as QA0-semipass andthe project has timed out. In the latter case, only the raw data are delivered.
The delivery is made of two parts:a) the science products and supporting material,b) the raw data.
Part b, the raw data, consists of visibility datasets in ALMA native format, i.e. the ALMA Science
Data Model (ASDM), which are needed if the PI would like to perform custom data calibration. On
the other hand, part a of the delivery, the “QA2 Products”, consists of the QA2 data reduction
scripts, log files, calibration and flagging tables, and the imaging products on which the quality
assurance decision was based. This document describes the contents of the Products section of thedelivery in detail and is aimed at ALMA users.
Until September 2014, QA2 work was entirely performed manually using the QA2 calibration script
generator tool1. We refer to data processed in this way as “manually calibrated”. In September
2014, the first official version of the ALMA Science Pipeline became available and has since thenbeen upgraded several times. It is now used to calibrate most of the data. We refer to dataprocessed by the Science Pipeline as “pipeline calibrated”. In addition to the content in the QA2
1

The calibration and imaging script generators are CASA scripts which create a template script fordata reduction, which is then edited by the data reducer. See Petry, D. et al. 2014, Proc. SPIE, 9152, 91520J

3

report, the user can tell whether the data are pipeline calibrated by the presence of a “*PPR*.xml”
or a “*pprequest.xml” file in the “script” subdirectory of the delivery package. See Section 4 formore details.
At the beginning of Cycle 4, the Science Pipeline was enabled to also perform imaging. This
increased the processing capacity significantly. We refer to data processed by the imaging sectionof the Science Pipeline as “pipeline imaged”. If the data are imaged by a data reducer using CASA
without the help of the pipeline (typically using templates or the imaging script generator tool 1), wecall them as “manually imaged”. The approval of the imaging products remains in the hands of thedata reducers and Data Reduction Managers. Manual imaging can be necessary even when thepipeline was used for calibration, depending on the specifics of the dataset (e.g. for full polarization
data sets, the pipeline performs the intensity calibration but full polarization calibration andimaging are following manual procedures). Sometimes manual imaging is performed in addition to
pipeline imaging, for example to show the PI that the science goal can be reached just by tweaking
one of the imaging parameters. In this case, the QA2 report will mention the delivery of
supplemental manual imaging. For more details, see Chapter 11 of the Cycle 9 ALMA TechnicalHandbook and https://almascience.org/processing/science-pipeline.

2 Group ObsUnitSet, Member ObsUnitSet, and Scheduling Block
By design, the smallest scheduling entity which is processed and delivered individually is theMember ObsUnitSet (MOUS). QA2 therefore is carried out on individual MOUSes. Following theprevious Cycles, each MOUS contains only one Scheduling Block (SB) even in the case of Total
Power (TP) Array observations.
Hence also in Cycle 9, QA2 continues to be carried out together on all the Execution Blocks (EBs) ofa single SB corresponding to one MOUS. The products for each MOUS are delivered separately, andthe delivery of the products for each MOUS is labelled with the MOUS UID, the unique hexadecimalname providing access to the ALMA archive.
The MOUSes are organized into Group ObsUnitSets (GOUSes). In the trivial case, there is only one
GOUS for each Science Goal. In Cycle 9, the following types of MOUSes can in principle be in one
GOUS:a) The 12 m Array MOUS(s),b) The 7 m Array MOUS,c) The TP observation MOUS,d) The TP calibration MOUS (may be omitted).
The delivery packaging results in three tar files named as follows:<Project ID>_<MOUS_UID>_001_of_001.tar<Project ID>_<MOUS_UID>_auxiliary.tar
member_<MOUS_UID>_README.txt.tar

4

The archive offers the possibility to download individual product files. The auxiliary tar file contains
calibration tables, logs, scripts and quality assurance reports, including the weblog. The tar filemarked with 001_of_001 contains the FITS product files of the target and possibly the calibrators.
The README.txt.tar file contains a description of the main actions, categories and files of the ALMA
data at the MOUS level. Before Cycle 4, the README file was providing information on what isdelivered and how to work with it. This information has been transferred to the QA2 Report since
Cycle 5.
Optionally, if the user would like to recreate the calibrated data, the user will need to download
also the uncalibrated raw data and these will appear as tar files named<Project ID>_<EB_UID>_asdm.sdm.tar
where EB_UID indicates the unique ID that identifies the execution block, since one tar file will beavailable per execution. By untarring any of the tar files in a delivery package, the user will find thata directory tree is built as described in the next Section.

3 Overview
An ALMA data delivery in Cycle 9 consists of the seven items shown in the directory tree below.
Items 1 to 6 are the main delivery package (part a), which should be downloaded first. Item 7 (partb) is only required if the users would like to work with the raw visibility data. Once unpacked, alldata fall into a standardized directory structure:|-- project_id/| |-- science_goal.ouss_id/| | |-- group.ouss_id/| | | |-- member.ouss_id/| | | | |-- README(1)(member. MOUS_UID.README.txt)| | | | |-- product/(2)| | | | |-- calibration/ (3)| | | | |-- qa/(4)| | | | |-- script/(5)| | | | |-- log/
(6)
| | | | |-- raw/
(7)
(only present when part b is unpacked)
The directory content is summarized in the following. When an item is marked as “best effort”, thismeans that it is generated by the data reducers only if sufficient resources are available and it iscompatible with the science goal.
1) README
A text file explaining where to find information about the QA2 results. Please note that the main
source of information on the data processing is the “weblog” included in the subdirectory “qa”: formore details see Section 4), i.e. QA documentation in subdirectory “qa”.
52) Imaging Products (in FITS format) in subdirectory “product”a) for pipeline imaged data:— A set of continuum and line images for all science spectral windows, covering as large as
possible a part of the spectral range. This set is as complete as possible given the computing
resources of the processing pipeline, i.e. for datasets with very high spatial and/or spectral
resolution some parts of the set may be automatically omitted by the pipeline, to keep thesize of the product package within reasonable limits. This is what is called “mitigation” ofthe data products. See the pipeline documentation athttps://almascience.org/processing/science-pipeline.
b) for pipeline calibrated data:— Continuum images of the bandpass and phase calibrators and (when present) the checksource(s), for each of the science spectral windows.
c) manually imaged line cubes:— For the representative spectral window: One spectral cube of “representative channels” forat least the representative target in the MOUS, made at the requested spectral resolution
considering the bandwidth specified for sensitivity. If the achieved spatial resolution is higher
than requested, tapering may be applied if needed, to reach the science goal. Coarse
continuum subtraction is applied for sources with bright continuum.
— (best effort) For each source and each spectral window: cubes of all species specifically listed
in the proposal, made at an angular resolution as close as possible to the requested one andincluding line-free channels on either side. Continuum subtraction is applied for sources withbright continuum. A note may be added if additional species are found in the data, but imaging
may not be carried out.
— (best effort) Continuum image of all non-edge, non-line channels.
d) manually imaged continuum and or aggregate bandwidth images:— One image of all continuum spectral windows, all non-edge channels, for at least the representative target in the MOUS.
— (best effort) One image of all continuum spectral windows, all non-line channels.
3) CASA Tables (in CASA Table format) in subdirectory “calibration”a) for pipeline calibrated data
— Calibration tables: including Tsys, WVR, Bandpass, Gain, Amplitude (*caltables.tgz)— List of the applycal commands for the delivered calibration table (*calapply.txt)— Flagversions tables (*flagversions.tgz)— Auxilliary products (*auxproducts.tgz):— Calibrator fluxes from the ALMA calibration database (flux.csv)
6

— Calibration table antenna positions (antennapos.csv)— Continuum frequency ranges (cont.dat)— Flagging commands added manually by the pipeline operator (*flagtemplate.txt,*flagtargetstemplate.txt, *flagtsystemplate.txt).
See https://almascience.org/processing/science-pipeline for more information on the role ofthese files.
b) for manually calibrated data
— Calibration tables: including Tsys, WVR, Bandpass, Gain, Amplitude.
— Flagversions tables.
— Calibration plots and wvrgcal diagnostic output.

4) QA documentation in subdirectory “qa”a) for pipeline-calibrated and/or imaged data
— The Pipeline Weblog, i.e. a system of webpages containing all the diagnostic plots andother information generated by the pipeline. This is the main source of information on allproperties of the data and the steps taken to calibrate/image and flag them. The weblog
also contains the summary of the commands executed by the pipeline, the casa log file foreach stage and its QA scoring.
— A QA0 Report for each EB UID in pdf format. This document is a summary of what
occurred during data acquisition. It contains information about the system and receivertemperature in every antenna, the amount of time spent on calibrators and target sources,
and a first quality evaluation of the data.
— A QA2 Report for the MOUS UID in pdf format. This report contains information about
the quality assessment performed after processing the data and the comments from thereviewer who decided to deliver the dataset.
b) for manually calibrated/imaged data
— QA Reports for each EB of the imaged data (png and txt files).
— The Manual Weblog is provided in html format. The Manual Weblog contains the information of the CASA reduction scripts and related images: CASA reduction scripts and diagnostic
plots are displayed, fits images are listed.

75) Data Reduction Scripts (ASCII files) in subdirectory “script”a) for pipeline calibrated and imaged data
— The Python scripts needed to restore the calibrated MS(s) (*scriptForPI.py and*casa_piperestorescript.py) or rerun the entire pipeline (*casa_pipescript.py).
— The pipeline processing request file (PPR*.xml or *pprequest.xml).
b) for pipeline calibrated and manually imaged data
— (in addition to a)): The imaging preparation script (if necessary) and the imaging script.
— Optionally also the *scriptForPolCalibration.pyc) for manually calibrated and imaged data
— The Python scripts needed to restore the calibrated MS(s) (*scriptForPI.py,*scriptForCalibration.py, and optionally scriptForImagingPrep.py, scriptForPolCalibration.py)— Imaging script, and the script to set up and run the calibration.
6) calibration and imaging log files in subdirectory “log”— CASA log files from the QA2 processing in case of manually calibrated or manually imaged
datasets.
— CASA commands log from the QA2 processing in case of pipeline reduced datasets.
7) Datasets (in ASDM format) in subdirectory “raw” (if downloaded)— (for optional download) The EBs that were used in the imaging/QA2 assessment.
— (for optional download if applicable) Additional datasets that were NOT used in the data
reduction but may still contain valuable scientific information.

4 Data Delivery Products
4.1 README
The complete name of this file in normal deliveries is member.<MOUS_UID>.README.txt. Until theend of Cycle 4, this was a text file summarizing the QA2 results and explaining the structure of thetarball and file naming conventions. Since Cycle 5 this README file has normally been just areference to where one can find additional information. Summary of the QA2 results, explanationof the tarball structure and file naming convention are provided by the QA2 report, located in the“qa” directory,8

4.2 Data Reduction Scripts (directory “script”)
In the subdirectory “script”, the user finds the CASA data reduction scripts (Python) which wereused to calibrate and image the data. Depending on whether the Science Pipeline was used or thedata were calibrated manually, the content of this directory varies:
1) uid....ms.scriptForCalibration.py: one calibration script for each EB of manually calibrated
interferometric data.
2) uid....ms.scriptForSDCalibration.py: one calibration script for each EB of manually calibrated
single-dish data.
3) scriptForImagingPrep.py (for manually imaged datasets, if needed): this script contains allnecessary steps to prepare imaging that needs to be performed on all calibrated EBs
together (therefore cannot be performed by the calibration script for the individual EBs).
These steps may include alignment of frequency grids before concatenation, theconcatenation into one MeasurementSet (MS), adjustments to the flux calibration etc. The
result is a MS or a set of MSs that are ready for the final imaging. If present, this script is run
automatically as the last step by the scriptForPI.py (see below).
4) scriptForPolCalibration.py (for manually calibrated full-polarization data): special calibration
routine for full-polarization data that is executed after the per-EB-calibration scripts. If
present, it will be run automatically by scriptForPI.py (see below).
5) scriptForImaging.py (for interferometric data): this script generates the same imaging
products (see Section 5 below) as those that are stored in the “product” subdirectory of thedelivery. Where convenient for the data reducer, the scriptForImaging.py may also containelements that could in principle be put into a scriptForImagingPrep.py. In that case, ascriptForImagingPrep.py is not created.
6) scriptForSDImaging.py (for manually imaged single-dish data): this script generates thesame imaging products (see Section 5 below) as those that are stored in the “product”
subdirectory of the delivery.
7) scriptForPI.py: in all cases, whether pipeline or manually calibrated, run this script in the“script” directory in order to regenerate the calibrated MS of the delivered data, after
downloading the raw ASDMs. The README file and the QA2 report contain instructions (orpoints to them) on how to do this. Running the scriptForPI.py requires that you havedownloaded the ASDM datasets and unpacked them, so that they reside in the “raw”
directory (created during unpacking). See next Section for more details.
Until mid-Cycle 5, the only type of dataset that was not delivered with a scriptForPI.py was
the manually calibrated TP data. See Section 5 below.
8) casa_piperestorescript.py: this script is used by scriptForPI.py in case of pipeline-calibrated
data to restore the calibrated MS(s) from the raw ASDMs by applying the necessary
calibration tables. If it is not provided or it has been removed, the scriptForPI.py will run thecasa_pipescript.py instead.
9) casa_pipescript.py: this script performs the calibration from scratch using Pipeline tasks. See
the ALMA Science Pipeline User’s Guide documentation (Section 6) for more details.

910) The pipeline processing request (PPR): an XML file that was used to drive the Pipeline
calibration and/or imaging. See https://almascience.org/processing/science-pipeline
Furthermore, there may be the following script file present in case of solar observations:
sun_reduction_util.py (contains helper routines specific to solar data analysis).
4.3 Imaging Products (directory “product”)
The “product” subdirectory of the delivery contains the imaging products that were generated
either by the imaging section of the Science Pipeline or by a data reducer with the script
scriptForImaging.py. These imaging products were created from the calibrated dataset produced
when the scriptForPI.py is run. For pipeline calibrated interferometric data, this directory also
contains calibrator images generated by the Pipeline.
A coherent file naming scheme was introduced for all imaging products:<MOUS>.<target>_<target type>.spw<SPWs>.<imagetype>.<stokes>.<comment>.<x>.<ext>
Where<MOUS><target><target type><SPWs><image type><stokes><comment>
<x>

<ext>

Is the MOUS UID (for example,
uid://A001/X23a/X2c).
Is the name of the imaged field (it can be thename of a calibrator or of the scientific target).
Is “sci” for science target, “ph” for phasecalibrator, “bp” for bandpass calibrator, “amp”
for flux calibrator (this will not be present if the
bandpass calibrator is also used as flux
calibrator), “chk” for check source, “pol_leak”
for the polarization calibrator.
Are the imaged spectral window IDs.
Is “mfs” for aggregate bandwidth image, “cube”
for a spectrally resolved image, “repBW” forcube at representative bandwidth, “cont” for animage with the line channels excluded (inpipeline calibrated/imaged data, this will haveall spectral windows merged).
The name of the image Stokes parameter (e.g.,“I”).
Can be an explanatory text, such as a moleculartransition.
Is “pbcor” for the primary beam (PB) corrected
image, “pb” for the PB itself, “mask” for themask image, “sd” for single dish.
Is the file name extension describing the file
type, e.g. “fits”.

10Note that the <comment> field typically contains the word “manual” if the image was obtained bymanual imaging. Information about the ALMA naming scheme can also be found in the following
knowledgebase article
https://help.almascience.org/kb/articles/what-calibration-and-imaging-products-will-bedelivered-to-me
The goal of ALMA QA2 in Cycle 9 remains that the calibration is reliable and "science ready", but
the imaging products may not be. A sufficient fraction of the possible imaging products are createdto verify that the data meet the science goals set by the PI (resolution and sensitivity), within sometolerance, but the imaging may be incomplete. In general, investigators should expect to need to
re-image their data using the provided scripts (for manual imaging) or the pipeline documentation(for pipeline imaged data) as a guideline. In particular, care should be taken to optimize continuum
subtraction (if relevant) and fine-tune the image deconvolution parameters (e.g. clean masks andthresholds). See “ALMA Imaging Pipeline Reprocessing” in CASA guides:
https://casaguides.nrao.edu/index.php?title=ALMA_Imaging_Pipeline_Reprocessing
Care has been taken that the images conform to the FITS standard 3.0.
Where necessary (i.e. where a clean mask was actually applied), the imaging masks are provided as
CASA images. These represent the final mask used in the last cleaning iteration. The masks used inprevious iterations may have been different (smaller in area).
The science images included in deliveries of interferometric data are corrected for the PB, i.e. thedependence of the instrument sensitivity on direction within the field of view (FOV). For each
image, two files are delivered:a) The PB-corrected image (file name ending in ".pbcor.fits").
b) The image of the PB used for the correction ("*.pb.fits" or “*.flux.fits”, possibly gzipped).
The image noise was measured in the uncorrected image. The corrected image (a) was thenobtained by dividing the uncorrected image by the PB image (b). The uncorrected image can berecovered using the CASA task “impbcor” in mode "m":impbcor(imagename='image.pbcor.fits',pbimage='image.flux.fits', mode='m',outfile='image.recovered')

For TP data, images are delivered accounting for the convolution of the sky intensity distribution
with the PB of the antenna, implying no need for a PB correction. In early cycles and high frequency
regular observations, additional amplitude calibrator observation were taken whose images were
not included.
If the data reducer deemed it necessary, in the case of pipeline imaged data, supplemental manual
imaging may be added to the delivery package. The user will then find in the product directory afile named manual_imaging.tgz containing the CASA log files and the scripForImaging.py used toperform such supplemental imaging. The user should take care not to mix up thisscriptForImaging.py with that in the script directory, which is related to the pipeline run. The

11

supplemental products will contain the comment manual in their names and can be found either
inside the manual_imaging.tgz or simply in the products directory along with the pipeline products.
4.4 Calibration Tables (directory “calibration”)
In order to further document the process of data calibration, all calibration tables generated by thecalibration process are stored in the “calibration” subdirectory of the delivery.
In the case of manually calibrated data, information on how each table was created can be found in
the scriptForCalibration.py and the scriptForImagingPrep.py scripts. CASA provides functionality toview these tables. Please refer to the CASA documentation (see section 6, in particular the ALMA
Science Pipeline User’s Guide) on how to do that. However, many of the plots of interest in thiscontext are also available in the QA documentation (see Section 4.5 below).
In the case of pipeline-calibrated data, the files in this directory are needed to restore thecalibrated MS with “casa_piperestorescript.py” which is called by “scriptForPI.py”. For details on
how they were generated, please refer to https://almascience.org/processing/science-pipeline.
All diagnostic plots are also available in the Pipeline Weblog, which can be found in the directory“qa” (see Section 4.5 below).
4.5 QA Documentation (directory “qa”)
In the subdirectory “qa” of the delivery, the user finds diagnostic information obtained duringcalibration and imaging.
In the case of pipeline calibrated data, all diagnostic information is assembled in a system of html
pages which is called the Weblog. To access the Weblog, run
tar xzf *hifa*weblog.tar.gz

in the “qa” directory and then navigate to the root of the untarred weblog directory e.g. pipeline20220914T170517 and run h_weblog() within a CASA session containing pipeline tasks. It is
recommended to set your default browser to Firefox to view the weblog, although since the 2021
pipeline release there has been limited support for Chrome and Safari; for more information refer
to https://help.almascience.org/kb/articles/what-is-the-best-way-to-view-the-weblog.
For more information on the contents of the Weblog, please refer to the Pipeline documentation athttps://almascience.org/processing/science-pipeline.
In the case of manually calibrated data, there are several png images and a textfile for each EB,labelled with the EB UID. The png files can be viewed with standard system tools such as “eog”(under Linux). They show a set of diagnostic plots describing the following aspects of the data:
1) Observing Schedule (observation intent vs. time).
2) Mosaic Pointing Configuration.
3) Antenna Array Configuration.
4) Effectiveness of the WVR correction for each antenna (for 12M observations).
5) Temporal gain calibration solutions for each antenna.
6) Temporal phase calibration solutions for each antenna.
7) Average bandpass solution for each spectral window.

128) System Temperature vs. frequency for each antenna and each spectral window.
9) Phase calibrator amplitude and phase vs. frequency for each spectral window andpolarization.
10) Flux calibration model and data (visibility amplitude vs. UV distance).
11) Target visibility amplitude vs. UV distance for each spectral window and polarization.
12) Phase calibrator amplitude and phase vs. frequency for each spectral window andpolarization.
13) Target field UV coverage.
14) Test image of the target.
15) Target imaging synthesized beam (PSF).
The content of the uid*textfile.txt file is mostly self-explanatory. Of particular interest is the “Checkof a target image and sensitivity”, which mentions approximately the achieved spatial resolution
and sensitivity. Note that the finally achieved resolution and sensitivity could be different, due toadditional procedures in imaging.
From Cycle 7, all available png images and scripts related to the manual data reduction areassembled in the manual Weblog, a system of html pages. To access the Weblog, run
tar xzf *manual*weblog.tar.gz

in the “qa” directory and then use your favourite web browser to open the resulting file:manual*/html/index.html

In case of manual calibration, all EBs are expandable links with the related diagnostic information.
In case of manual imaging procedure, the imaging tab provides the related imaging scripts and listof plots. Each script is equipped with a link to visualize the content.
Based on the calibration tables in the “calibration” directory, the user can produce additional
diagnostic plots in CASA using the tasks plotms, plotcal, and plotbandpass. See the CASA
documentation for more details.
If additional diagnostic plots on the visibility data themselves are required, one needs first to
generate the calibrated MS from the raw data (see Section 5 below). Once this is done, the CASA
task plotms can be used to generate the plots.

5 Generating the calibrated visibilities
In ALMA Cycle 0 and early Cycle 1, the visibility data were delivered with all other products inready-to-use MS format. Since the middle of Cycle 1, the products are separated from the raw
visibility data (as described in the introduction); downloads of the latter are now optional. To
minimize download time, the raw visibilities are delivered in the native ALMA format, the ASDM.
If the user would like to only modify or extend the imaging of the dataset, but accepts theobservatory calibration of the data, they need to download the raw data and apply the calibration
via the scripts provided with the delivery as described in the following. Otherwise, the user can
download the raw data and use the calibration scripts or pipeline documentation as a guidance todevelop a custom calibration. Moreover, each ARC provides calibrated MeasurementSets directly:

13https://help.almascience.org/kb/articles/how-do-i-obtain-a-file-of-calibrated-visibilitiesmeasurement-set-for-alma-data
The download of the raw data depends on the details of where the data were staged for delivery.
The user should refer to the information in the delivery email as to how exactly the raw data should
be obtained. Unless there is a technical problem, this will work via the ALMA Archive Request
Handler. See http://almascience.org/alma-data.
5.1 Running the scriptForPI.py in the case of manually calibrated data
For manually calibrated data, the scriptForPI will find the scripts named scriptForCalibration.py andscriptForImagingPrep.py (if present), set up every necessary item for their execution, and then run
them to obtain the MS ready for imaging.
Once the data-products tar file is downloaded and unpacked, the user finds a certain directory
structure on disk (see the description in Section 3). The following directory tree shows an example
of manually calibrated (rather than pipeline calibrated) data. However, also for pipeline calibrated
data, the procedure for running “scriptForPI.py” is the same and has essentially the same final
product, although the naming differs.
2019.1.01482.T└── science_goal.uid___A001_X1465_Xb46└── group.uid___A001_X1465_Xb4a└── member.uid___A001_X1465_Xb4b├── calibration│ ├── member.uid___A001_X1465_Xb4b.concat_S1.ms.TDM.Df0.plots.tgz│ ├── uid___A002_Xe5aacf_X29ee.calibration.plots.tgz│ ├── uid___A002_Xe5aacf_X29ee.calibration.tgz│ ├── uid___A002_Xe5aacf_X29ee.ms.wvrgcal.txt│ ├── uid___A002_Xe5aacf_X2d6f.calibration.plots.tgz│ ├── uid___A002_Xe5aacf_X2d6f.calibration.tgz│ ├── uid___A002_Xe5aacf_X2d6f.ms.wvrgcal.txt│ ├── uid___A002_Xe5aacf_X3155.calibration.plots.tgz│ ├── uid___A002_Xe5aacf_X3155.calibration.tgz│ └── uid___A002_Xe5aacf_X3155.ms.wvrgcal.txt├── log│ ├── member.uid___A001_X1465_Xb4b.analysis.log.tgz│ ├── member.uid___A001_X1465_Xb4b.calibrated.log.tgz│ ├── uid___A002_Xe5aacf_X29ee.log.tgz│ ├── uid___A002_Xe5aacf_X2d6f.log.tgz│ └── uid___A002_Xe5aacf_X3155.log.tgz├── member.uid___A001_X1465_Xb4b.README.txt├── product│ ├── member..concat_S1.ms.TDM_img1.mask.tgz│ ├── member..GRB_pol1_sci.spw0_1_2_3.mfs.A.manual.pbcor.fits│ ├── member..GRB_pol1_sci.spw0_1_2_3.mfs.IQUV.manual.mask.tgz

14│ ├── member..GRB_pol1_sci.spw0_1_2_3.mfs.IQUV.manual.pbcor.fits│ ├── member..GRB_pol1_sci.spw0_1_2_3.mfs.IQUV.manual.pb.fits.gz│ ├── member..GRB_pol1_sci.spw0_1_2_3.mfs.P.manual.pbcor.fits│ ├── member..J1058p0133_polleak.spw0_1_2_3.mfs.A.manual.pbcor.fits│ ├── member..J1058p0133_polleak.spw0_1_2_3.mfs.IQUV.manual.mask.tgz│ ├── member..J1058p0133_polleak.spw0_1_2_3.mfs.IQUV.manual.pbcor.fits│ ├── member..J1058p0133_polleak.spw0_1_2_3.mfs.IQUV.manual.pb.fits.gz│ ├── member..J1058p0133_polleak.spw0_1_2_3.mfs.P.manual.pbcor.fits│ ├── member..target_check.mask.tgz│ ├── uid___A002_Xe5aacf_X29ee__target_check.mask.tgz│ ├── uid___A002_Xe5aacf_X2d6f__target_check.mask.tgz│ └── uid___A002_Xe5aacf_X3155__target_check.mask.tgz├── qa│ ├── member..concat_S1.ms.TDM.Dterm_sky_imag_spw0-1.png│ ├── member..concat_S1.ms.TDM.Dterm_sky_imag_spw2-3.png│ ├── member..concat_S1.ms.TDM.Dterm_sky_real_spw0-1.png│ ├── member..concat_S1.ms.TDM.Dterm_sky_real_spw2-3.png│ ├── member..concat_S1.ms.TDM.GainRatiosPol.png│ ├── member..concat_S1.ms.TDM.Gxyamp.png│ ├── member..concat_S1.ms.TDM.GxyampRatio.png│ ├── member..concat_S1.ms.TDM.XY0_Amb-NoAmb.png│ ├── member..concat_S1.ms.TDM.XY-CrossPhase.png│ ├── member.uid___A001_X1465_Xb4b.qa2_report.html│ ├── member.uid___A001_X1465_Xb4b.qa2_report.pdf│ ├── member.uid___A001_X1465_Xb4b.weblog.pdf│ ├── uid___A002_Xe5aacf_X29ee.qa0_report.pdf│ ├── uid___A002_Xe5aacf_X29ee__qa2_part1.png│ ├── uid___A002_Xe5aacf_X29ee__qa2_part2.png│ ├── uid___A002_Xe5aacf_X29ee__qa2_part3.png│ ├── uid___A002_Xe5aacf_X29ee__textfile.txt│ ├── uid___A002_Xe5aacf_X2d6f.qa0_report.pdf│ ├── uid___A002_Xe5aacf_X2d6f__qa2_part1.png│ ├── uid___A002_Xe5aacf_X2d6f__qa2_part2.png│ ├── uid___A002_Xe5aacf_X2d6f__qa2_part3.png│ ├── uid___A002_Xe5aacf_X2d6f__textfile.txt│ ├── uid___A002_Xe5aacf_X3155.qa0_report.pdf│ ├── uid___A002_Xe5aacf_X3155__qa2_part1.png│ ├── uid___A002_Xe5aacf_X3155__qa2_part2.png│ ├── uid___A002_Xe5aacf_X3155__qa2_part3.png│ └── uid___A002_Xe5aacf_X3155__textfile.txt└── script├── member.uid___A001_X1465_Xb4b.calimage.product_rename.txt

15├── member.uid___A001_X1465_Xb4b.scriptForImaging.py├── member.uid___A001_X1465_Xb4b.scriptForPI.py├── member.uid___A001_X1465_Xb4b.scriptForPolCalibration.py├── uid___A002_Xe5aacf_X29ee.ms.scriptForCalibration.py├── uid___A002_Xe5aacf_X2d6f.ms.scriptForCalibration.py└── uid___A002_Xe5aacf_X3155.ms.scriptForCalibration.py

Note that for simplicity, the names of the files in the “product” and “qa” directory were shortened
here by removing the MOUS UID, which in a real delivery would be contained within the double
dots in the names of our example. In the example above, the “qa” directory contains a qa2 reportin html format, that has been deprecated during Cycle 7. From Cycle 8, the manual weblog isprovided in html format and compressed in a tgz file.
This dataset is from a polarization observation and has three EBs with UIDs
uid___A002_Xe5aacf_X29ee, uid___A002_Xe5aacf_X2d6f, and uid___A002_Xe5aacf_X3155. The
directory “script” therefore has three scripts for calibration:├── uid___A002_Xe5aacf_X29ee.ms.scriptForCalibration.py├── uid___A002_Xe5aacf_X2d6f.ms.scriptForCalibration.py└── uid___A002_Xe5aacf_X3155.ms.scriptForCalibration.py
and, since additional commands are necessary to perform the polarization calibration, a script forthis aim is also present:├── member.uid___A001_X1465_Xb4b.scriptForPolCalibration.py
Furthermore, like in every delivery, the directory “script”“*scriptForImaging.py” and “*scriptForPI.py”.
├── member.uid___A001_X1465_Xb4b.scriptForImaging.py├── member.uid___A001_X1465_Xb4b.scriptForPI.py

contains

thescripts

The user could have also found a scriptForImagingPrep.py, which is an optional script sometimes
used by a data reducer to perform further adjustments (if deemed necessary) after the calibration,for example flux equalization.
The user is encouraged to inspect all these scripts, as they may also contain helpful comments on
special properties of the data.
Unpacking the raw data after downloading should be done at the top level (in the directory
containing the directory 2019.1.01482.T in this example). It should result in an additional directory“raw” at the level of the directory “script”:| | | | |-- raw/16

| | | | | |-- uid___A002_Xe5aacf_X29ee.asdm.sdm| | | | | |-- uid___A002_Xe5aacf_X2d6f.asdm.sdm| | | | | |-- uid___A002_Xe5aacf_X3155.asdm.sdm
In order to just reproduce the observatory calibration, the user can then run the “scriptForPI.py” by
typing at the shell prompt of the operating system:
cd script
casa -c "execfile('member.uid___A001_X1465_Xb4b.scriptForPI.py')"

Note that the same version of CASA should be used as that of the original processing.
Executing scriptForPI will first perform various tests on the directory structure and the presence ofthe necessary files and then run the corresponding scriptForCalibration on each of the ASDMs. If
there is more than one ASDM and further steps were necessary in the QA2 process to prepare thedataset for imaging, the scriptForImagingPrep will be present and will be run as well.
The results will be placed in a new directory called “calibrated” at the same level as the directories“raw” and “script”:| | | | |-- calibrated| | | | | |-- concat_S1.ms| | | | | |-- uid___A002_Xe5aacf_X29ee.calibration| | | | | |-- uid___A002_Xe5aacf_X29ee.ms.split.cal| | | | | |-- uid___A002_Xe5aacf_X2d6f.calibration| | | | | |-- uid___A002_Xe5aacf_X2d6f.ms.split.cal| | | | | |-- uid___A002_Xe5aacf_X3155.calibration| | | | | |-- uid___A002_Xe5aacf_X3155.ms.split.cal
If the data reducer deems it necessary, the final calibrated visibility data combined for all EBs canbe found in a MS called “calibrated.ms” or, like in our example, “concat_S1.ms”, where S1 is due tothe polarization nature of the dataset and indicates Session 1. The concatenated MS can also not
be present, if the data reducer does not deem it necessary to perform a concatenation, and theimaging will operate directly on the MSs for each EB. The intermediate output of CASA to achieve
the calibration of each EB is stored in the calibration working directories:| | | | | |-- uid___A002_Xe5aacf_X29ee.calibration| | | | | |-- uid___A002_Xe5aacf_X2d6f.calibration| | | | | |-- uid___A002_Xe5aacf_X3155.calibration
while the calibrated visibilities for each EB individually are stored in the MSs with names ending in“.ms.split.cal” (if there is only one EB, this MS will take the role of calibrated.ms).
The execution of scriptForPI.py will produce MSs that are ready for imaging, but will not perform
any imaging on them: it will not run scriptForImaging.py.

17

5.2 Running the scriptForPI.py in the case of pipeline-calibrated data
The following shows the contents of the directory tree for a dataset that was calibrated and imaged
using the ALMA pipeline. This can easily be seen from the fact that there is a file “*pprequest.xml”
present in the “script” directory. Like in the case of manually calibrated data, the “scriptForPI.py” isfound in the “script” directory.
2019.1.01056.S└── science_goal.uid___A001_X1465_X1b46└── group.uid___A001_X1465_X1b47└── member.uid___A001_X1465_X1b48├── calibration│ ├── member..hifa_calimage*.auxproducts.tgz│ ├── member.uid___A001_X1465_X1b48.session_1.auxcaltables.tgz│ ├── member.uid___A001_X1465_X1b48.session_1.caltables.tgz│ ├── member.uid___A001_X1465_X1b48.session_2.auxcaltables.tgz│ ├── member.uid___A001_X1465_X1b48.session_2.caltables.tgz│ ├── uid___A002_Xe1baa0_X5b8c.ms.calapply.txt│ ├── uid___A002_Xe1baa0_X5b8c.ms.flagversions.tgz│ ├── uid___A002_Xe1baa0_X5b8c_target.ms.auxcalapply.txt│ ├── uid___A002_Xe1d2cb_X31f.ms.calapply.txt│ ├── uid___A002_Xe1d2cb_X31f.ms.flagversions.tgz│ └── uid___A002_Xe1d2cb_X31f_target.ms.auxcalapply.txt├── product│ ├── member.._G24.73p0.69__sci.spw16_18_20_22.cont.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw16_18_20_22.cont.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw16_18_20_22.cont.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw16.cube.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw16.cube.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw16.cube.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw16.mfs.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw16.mfs.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw16.mfs.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw18.cube.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw18.cube.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw18.mfs.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw18.mfs.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw18.mfs.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw20.cube.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw20.cube.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw20.cube.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw20.mfs.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw20.mfs.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw20.mfs.I.pb.fits.gz
18│ ├── member.._G24.73p0.69__sci.spw22.cube.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw22.cube.I.pb.fits.gz│ ├── member.._G24.73p0.69__sci.spw22.mfs.I.mask.fits.gz│ ├── member.._G24.73p0.69__sci.spw22.mfs.I.pbcor.fits│ ├── member.._G24.73p0.69__sci.spw22.mfs.I.pb.fits.gz│ ├── member..J1832-1035_ph.spw16.mfs.I.mask.fits.gz│ ├── member..J1832-1035_ph.spw16.mfs.I.pbcor.fits│ ├── member..J1832-1035_ph.spw16.mfs.I.pb.fits.gz│ ├── member..J1832-1035_ph.spw18.mfs.I.mask.fits.gz│ ├── member..J1832-1035_ph.spw18.mfs.I.pbcor.fits│ ├── member..J1832-1035_ph.spw18.mfs.I.pb.fits.gz│ ├── member..J1832-1035_ph.spw20.mfs.I.pbcor.fits│ ├── member..J1832-1035_ph.spw20.mfs.I.pb.fits.gz│ ├── member..J1832-1035_ph.spw22.mfs.I.mask.fits.gz│ ├── member..J1832-1035_ph.spw22.mfs.I.pbcor.fits│ ├── member..J1832-1035_ph.spw22.mfs.I.pb.fits.gz│ ├── member..J1912-0804_ph.spw16.mfs.I.mask.fits.gz│ ├── member..J1912-0804_ph.spw16.mfs.I.pbcor.fits│ ├── member..J1912-0804_ph.spw16.mfs.I.pb.fits.gz│ ├── member..J1912-0804_ph.spw18.mfs.I.mask.fits.gz│ ├── member..J1912-0804_ph.spw18.mfs.I.pbcor.fits│ ├── member..J1912-0804_ph.spw18.mfs.I.pb.fits.gz│ ├── member..J1912-0804_ph.spw20.mfs.I.mask.fits.gz│ ├── member..J1912-0804_ph.spw20.mfs.I.pbcor.fits│ ├── member..J1912-0804_ph.spw20.mfs.I.pb.fits.gz│ ├── member..J1912-0804_ph.spw22.mfs.I.mask.fits.gz│ ├── member..J1912-0804_ph.spw22.mfs.I.pbcor.fits│ ├── member..J1912-0804_ph.spw22.mfs.I.pb.fits.gz│ ├── member..J1924-2914_bp.spw16.mfs.I.pbcor.fits│ ├── member..J1924-2914_bp.spw16.mfs.I.pb.fits.gz│ ├── member..J1924-2914_bp.spw18.mfs.I.pbcor.fits│ ├── member..J1924-2914_bp.spw18.mfs.I.pb.fits.gz│ ├── member..J1924-2914_bp.spw20.mfs.I.pbcor.fits│ ├── member..J1924-2914_bp.spw20.mfs.I.pb.fits.gz│ ├── member..J1924-2914_bp.spw22.mfs.I.pbcor.fits│ └── member..J1924-2914_bp.spw22.mfs.I.pb.fits.gz├── qa│ ├── member..hifa_calimage*.weblog.tgz│ ├── member..qa2_report.html│ ├── member..qa2_report.pdf│ ├── uid___A002_Xe1baa0_X5b8c.qa0_report.pdf│ └── uid___A002_Xe1d2cb_X31f.qa0_report.pdf

19├── raw│ ├── uid___A002_Xe1baa0_X5b8c.asdm.sdm│ └── uid___A002_Xe1d2cb_X31f.asdm.sdm└── script├── member..calimage*.product_rename.txt├── member..hifa_calimage*.casa_commands.log├── member..hifa_calimage*.casa_piperestorescript.py├── member..hifa_calimage*.casa_pipescript.py├── member..hifa_calimage*.pipeline_manifest.xml├── member..hifa_calimage*.pprequest.xml└── member..scriptForPI.py
The file names in the directories “product”, “qa”, and “script” were shortened by not showing the
MOUS UID, which is indicated by two consecutive dots in the file name. Note the additional
calibrator images created by the Science Pipeline.
Also here, as for a manually calibrated dataset, you will first need to download and unpack the raw
data, if you want to perform further imaging or re-calibrate. This will create the additional
subdirectory “raw” containing the ASDM(s). In order to then run “scriptForPI.py”, you will need toverify that you have installed the CASA version indicated in the QA2 Report or Weblog. Then start
up CASA in the working directory “script” usingcasa --pipeline

At the CASA prompt then type
execfile('member.uid___A001_X1465_X1b48.scriptForPI.py')

Unlike for manually calibrated data, where scriptForPI.py re-runs the calibration by default, forpipeline calibrated data, scriptForPI.py will look for casa_piperestorescript.py and execute it, if
found. This will load the flagging and calibration tables included in the archival package and apply
them to the raw MSs. The final calibrated MSs will be placed in the directory “calibrated”.
The casa_piperestorescript.py can also be run stand-alone, without the scriptForPI. See the pipeline
documentation for more details.
The user needs to be aware that if supplemental manual imaging was performed, like for any
other imaging task, this will not be run by scriptForPI.py, which only deals with the calibration of
the dataset. If the user wants to execute the supplemental imaging with different parameters,
he/she will need to edit the scriptForImaging.py found inside the manual_imaging.tgz file locatedin the products directory. Since this is supplemental imaging, it is assumed that the imaging
pipeline has been executed, therefore this scriptForImaging.py runs on the *_target.ms created by
the imaging pipeline and cannot be executed right away after scriptForPI.py has finished. The*_target.ms contains two columns of data: in the DATA column one finds the calibrated data of thescience target and in the CORRECTED column the continuum-subtracted calibrated data. To be able
to run scriptForImaging.py in the case of supplemental imaging, the user will then need to restore

20

the continuum subtraction and create the *_target.ms, which can be done following the
instructions at this link
https://casaguides.nrao.edu/index.php?title=ALMA_Cycle_9_Imaging_Pipeline_Reprocessing
See section Restore Pipeline Continuum Subtraction and Manually Make Image Products. Once*_target.ms has been created, scriptForImaging.py can be run from within CASA. Since the data
reducer may have used pipeline tasks to create the supplemental products, it is always better toload CASA with the --pipeline option, to execute scriptForImaging.py.

5.3 Running casa_pipescript.py
In the case of pipeline calibrated data, there is an alternative way to generate the calibrated
visibilities. It uses the script “casa_pipescript.py” that is provided in the “script” directory. This
contains the same tasks and parameters that were used when the Data Management Group (DMG)
calibrated the data. In principle, the user can modify this script to adjust the calibration, but it isrecommended to try the delivered version first. The details of the usage of this script are describedin the Pipeline documentation (see Section 6 below).
The script casa_pipescript.py can also be run via the scriptForPI. Simply move thecasa_piperestorescript.py out of the “script” directory: scriptForPI will then use casa_pipescript.py.
For some pipeline-calibrated data (e.g. TP data), the script casa_piperestorescript.py may not havebeen provided. In that case, the scriptForPI will run the casa_pipescript.py by default.

5.4 Saving disk space during and after the execution of the scriptForPI.py
The ALMA datasets are increasingly large. If the user would like to save disk space and only workwith the calibrated data once these were regenerated with “scriptForPI.py”, they can either delete
all calibration directories (cd calibrated; rm -rf *.calibration) or, in case of manually calibrated data,
only delete some or all of the intermediate MSs contained in the calibration directories, using
cd calibrated; rm -rf *.calibration/*.ms; rm -rf *.calibration/*.ms.split

Since the middle of Cycle 1, the scriptForPI.py has also offered the “SPACESAVING” option to limit
the disk space usage during and after the run its execution. In order to make use of this, the Python
global variable SPACESAVING needs to be set before starting the script, e.g. using
cd script
casa -c "global SPACESAVING; SPACESAVING=N; execfile('scriptForPI.py')"where N is an integer from 0 to 3 with the following meaning:
SPACESAVING

=0
=1
=2
>= 3
= -1

same as not set (all intermediate MSs are kept)
do not keep intermediate MSs named *.ms.split
do not keep intermediate MSs named *.ms and *.ms.split
do not keep intermediate MSs named *.ms, *.ms.split,
and *.ms.split.cal (if possible)
do not check disk space21

With SPACESAVING=0, the required additional diskspace is up to 14 times as large as the delivered
data (products and raw data), while with SPACESAVING=3 (maximum savings), it is up to 6 times aslarge. The script will estimate the required disk space and will not execute if there is not sufficient
free space available. You can disable the disk-space checking by setting the value to -1.

5.5 Splitting out the calibrated data at the end of running the scriptForPI.py
For manually calibrated data, it is standard practice to conclude the calibration by copying thecalibrated visibilities into a new MS at the end of calibration. This is done using the CASA tasks
mstransform or split and is called “splitting out” the calibrated data. The final calibrated MSs are
named uid*.ms.split.cal. They contain only the science spectral windows.
For pipeline calibrated and imaged data, the calibrated visibilities for the science target(s) are split
to a new MS with *target.ms to facilitate the continuation of the processing with the imaging
section of the Science Pipeline. The original MS is called uid*.ms and contains all original spectral
windows and both the calibrated and uncalibrated data. The calibrated data can be found in the
MS table column “CORRECTED_DATA”. The splitted MS, called uid*target.ms, has the calibrated
data (continuum + line) in the MS table column DATA which is automatically picked up by the CASA
imaging tasks.
If you want to force the scriptForPI.py to perform the “splitting out”, i.e. to create uid*.ms.split.cal
for each EB, you can set the variable DOSPLIT to True before starting the script:
casa -c “global SPACESAVING; global DOSPLIT; SPACESAVING=N; DOSPLIT=True;
execfile(‘scriptForPI.py’)”
5.6 Generating the pipeline continuum subtraction products
Starting with data from Cycle 8, it is possible to make the scriptForPI also trigger the generation of
the continuum subtracted MSs. This is controlled by the variable “DOCONTSUB”. Just like withDOSPLIT above, you need to set the variable to True before running the scriptForPI using execfile.

5.7 Manually calibrated TP data
If your TP data were reduced manually, you will find scripts “*.scriptForSDCalibration.py” in thescript folder. If the scriptForPI.py is also present, you can proceed by running it as for other data.
For details on manual TP data calibration, please see the official CASA guide available athttps://casaguides.nrao.edu/index.php?title=M100_Band3 .
In older deliveries, the scriptForPI.py was not present. In that case, you need to download the raw
data (ASDMs) from the ALMA archive, remove the '.asdm.sdm' extensions, and then run the scripts('*.scriptForSDcalibration.py' from the scripts folder) in the same folder with the ASDM.
For the imaging, we recommend you use the provided script, as it contains certain values (e.g. Jy/K
conversion factors) that must be set to a precise value. You could run the script step-by-step, as forthe calibration script. Before doing that, you may need to update the paths to the calibrated data,in the msNames variable near the top of the script.

22

6 Relevant documentation and help
ALMA Quality Assurance is described in chapter 11 of the ALMA Technical Handbook
https://almascience.org/documents-and-tools/cycle9/alma-technical-handbook
For more information on the usage of CASA, please refer to the CASA home page
http://casa.nrao.edu/
Detailed examples are given in the ALMA guides
http://casaguides.nrao.edu/index.php?title=ALMAguides
and the documentation of the ALMA Science Pipeline is available fromhttps://almascience.org/processing/science-pipeline
If you have any problems with your ALMA data, please contact the ALMA helpdesk athttps://help.almascience.org/23

24