Pipeline Tasks Reference Manual
Release 2023.1.0

pipeline teamOct 1, 2023

CONTENTS

1

Generic
1.1 h_init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 h_resume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 h_save . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 h_tsyscal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 h_weblog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
1
2
3
3
4

2

Interferometry Generic
2.1 hif_analyzealpha . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 hif_antpos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 hif_applycal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 hif_bandpass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 hif_checkproductsize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6 hif_correctedampflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 hif_editimlist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.8 hif_findcont . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9 hif_gaincal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.10 hif_lowgainflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.11 hif_makecutoutimages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.12 hif_makeimages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.13 hif_makeimlist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.14 hif_makermsimages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.15 hif_mstransform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.16 hif_rawflagchans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.17 hif_refant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.18 hif_selfcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.19 hif_setjy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.20 hif_setmodels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.21 hif_transformimagedata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.22 hif_uvcontsub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5
5
6
8
10
12
14
16
19
20
23
25
26
28
33
33
35
37
39
41
44
45
47

3

Interferometry ALMA
3.1 hifa_antpos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 hifa_bandpass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 hifa_bandpassflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 hifa_bpsolint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 hifa_exportdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 hifa_flagdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7 hifa_flagtargets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 hifa_fluxcalflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49
50
52
56
61
64
66
69
70

i

3.9
3.10
3.11
3.12
3.13
3.14
3.15
3.16
3.17
3.18
3.19
3.20
3.21
3.22
3.23
3.24
3.25
3.26

hifa_gaincalsnr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
hifa_gfluxscale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
hifa_gfluxscaleflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
hifa_imageprecheck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
hifa_importdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
hifa_lock_refant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
hifa_polcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
hifa_polcalflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
hifa_renorm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
hifa_restoredata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
hifa_session_refant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
hifa_spwphaseup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
hifa_targetflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
hifa_timegaincal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
hifa_tsysflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
hifa_unlock_refant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
hifa_wvrgcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
hifa_wvrgcalflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4Interferometry ALMA SRDP112
4.1 hifas_imageprecheck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

5Interferometry VLA
5.1 hifv_analyzestokescubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 hifv_applycals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 hifv_checkflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 hifv_circfeedpolcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 hifv_exportdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.6 hifv_exportvlassdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.7 hifv_finalcals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.8 hifv_fixpointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.9 hifv_flagcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.10 hifv_flagdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.11 hifv_flagtargetsdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.12 hifv_fluxboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.13 hifv_hanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.14 hifv_importdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.15 hifv_pbcor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.16 hifv_plotsummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.17 hifv_priorcals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.18 hifv_restoredata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.19 hifv_restorepims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.20 hifv_selfcal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.21 hifv_semiFinalBPdcals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.22 hifv_solint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.23 hifv_statwt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.24 hifv_syspower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.25 hifv_testBPdcals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.26 hifv_vlasetjy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.27 hifv_vlassmasking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6Single Dish144
6.1 hsd_applycal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.2 hsd_atmcor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

114
115
115
117
119
120
122
122
123
124
124
126
127
128
129
131
132
132
133
135
136
137
138
138
139
140
141
142

ii

6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
6.12
7

hsd_baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_blflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_exportdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_flagdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_importdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_k2jycal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_restoredata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_skycal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
hsd_tsysflag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

148
153
156
157
159
160
163
165
167
169

Nobeyama
7.1 hsdn_exportdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 hsdn_importdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 hsdn_restoredata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

173
173
174
176

Note: The dryrun and acceptresults parameters for each task are no longer supported and may be removed in a future
release. As such, they are excluded from this manual.

iii

CHAPTERONE

GENERIC

5 tasks available.
task namedescription

h_inith_resumeh_saveh_tsyscal
h_weblog

Initialize the pipeline
Restore a save pipeline state
Save the pipeline state to disk
Derive a Tsys calibration table
Open the pipeline weblog in a browser

1.1 h_init
1.1.1 Task description
The h_init task initializes the pipeline.
h_init must be called before any other pipeline task. The pipeline can be initialized in one of two ways: by creating a
new pipeline state (h_init) or be loading a saved pipeline state (h_resume).
h_init creates an empty pipeline context but does not load visibility data into the context.
hsd_importdata can be used to load data.

hif_importdata or

The pipeline context is returned.

1.1.2 Parameter List
parameter nameloglevelplotlevel

description
Log level for pipeline messages. Log messages below this threshold will
not be displayed.
Toggle generation of detail plots in the web log. A level of ‘all’ generates
all plots; ‘summary’ omits detail plots; ‘default’ generates all plotsapart from for the hif_applycal task.
continues on next page1

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 1 – continued from previous page
parameter nameweblogdescription
Generate the web log

1.1.3 Examples
1. Create the pipeline context>>> h_init()

1.2 h_resume
1.2.1 Task description
h_resume restores a name pipeline state from disk, allowing a suspended pipeline reduction session to be resumed.

1.2.2 Parameter List
parameter namefilename

description
Name of the saved pipeline state. Setting filename to ‘last’ restores themost recently saved pipeline state whose namebegins with ‘context*’.

1.2.3 Examples
1. Resume the last saved session>>> h_resume()
2. Resume the named saved session>>> h_resume(filename='context.s3.2012-02-13T10:49:11')

1.2. h_resume2

Pipeline Tasks Reference Manual, Release 2023.1.0

1.3 h_save
1.3.1 Task description
h_save saves the current pipeline state to disk using a unique filename. If no name is supplied one is generated automatically from a combination of the root name, ‘context’, the current stage number, and a time stamp.

1.3.2 Parameter List
parameter namefilename

description
Name of the saved pipeline state. If filename is ‘’ then a unique name will
be generated computed several components: theroot, ‘context’, the current stage number, and the time stamp.

1.3.3 Examples
1. Save the current state in the default file>>> h_save()
2. Save the current state to a file called ‘savestate_1’>>> h_save(filename='savestate_1')

1.4 h_tsyscal
1.4.1 Task description
Derive the Tsys calibration for list of ALMA MeasurementSets.
Output:results – The results object for the pipeline task is returned.

1.4.2 Parameter List
parameter namevis
caltable

description
List of input visibility files. example: vis=[‘ngc5921.ms’]
Name of output gain calibration tables. example: caltable=’ngc5921.gcal’continues on next page

1.3. h_save

3

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter namechantol

description
The tolerance in channels for mapping atmospheric calibration windows(TDM) to science windows (FDM or TDM).
example: chantol=5

1.4.3 Examples
1. Standard call>>> h_tsyscal()

1.5 h_weblog
1.5.1 Task description
h_weblog opens the weblog in a new browser tab or window.

1.5.2 Parameter List
parameter namerelpathdescription
Relative path to the weblog index file. This file must be locatedin a child directory of the CASA working directory. If relpath
is left unspecified, the most recent weblog will be located anddisplayed.

1.5.3 Examples
1. Open pipeline weblog in a browser:>>> h_weblog()

1.5. h_weblog

4CHAPTERTWOINTERFEROMETRY GENERIC

22 tasks available.
task namedescription

hif_analyzealphahif_antpos
hif_applycal
hif_bandpass
hif_checkproductsize
hif_correctedampflag
hif_editimlist
hif_findcont
hif_gaincal
hif_lowgainflag
hif_makecutoutimages
hif_makeimages
hif_makeimlist
hif_makermsimages
hif_mstransform
hif_rawflagchans
hif_refant
hif_selfcal
hif_setjy
hif_setmodels
hif_transformimagedata

Extract spectral index from intensity peak in VLA/VLASS images
Derive an antenna position calibration table
Apply the calibration(s) to the data
Compute bandpass calibration solutions
Check imaging product size
Flag corrected - model amplitudes based on calibrators.
Add to a list of images to be produced with hif_makeimages()
Find continuum frequency ranges
Determine temporal gains from calibrator observations
Flag antennas with low or high gain
Cutout central 1 sq. degree from VLASS QL, SE, and Coarse Cube images
Compute clean map
Compute list of clean images to be produced
Create RMS images for VLASS data.
Create new MeasurementSets for science target imaging
Flag deviant baseline/channels in raw data
Select the best reference antennas
Determine and apply self-calibration with the science target data
Fill the model column with calibrated visibilities
Set calibrator source models
Extract fields for the desired VLASS image to a new MS and reset weights
if desired
Fit and subtract continuum from the data

hif_uvcontsub

2.1 hif_analyzealpha2.1.1 Task description
Extract spectral index from intensity peak in VLA/VLASS images
The results object for the pipeline task is returned.

5

Pipeline Tasks Reference Manual, Release 2023.1.0

2.1.2 Parameter List
parameter name
vis

image
alphafilealphaerrorfile

description
List of visisbility data files. These may be ASDMs, tar files of ASDMs,
MSs, or tar files of MSs, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
Restored subimage
Input spectral index map
Input spectral index error map

2.1.3 Examples
1. Basic analyzealpha task>>> hif_analyzealpha()

2.2 hif_antpos
2.2.1 Task description
Derive the antenna position calibration for list of MeasurementSets.
The hif_antpos task corrects the antenna positions recorded in the ASDMs using updated antenna position calibration
information determined after the observation was taken.
Corrections can be input by hand, read from a file on disk, or in the future by querying an ALMA database service.
The antenna positions file is in ‘csv’ format containing 6 comma-delimited columns as shown below. The default nameof this file is ‘antennapos.csv’.
Contents of example ‘antennapos.csv’ file:ms,antenna,xoffset,yoffset,zoffset,comment uid___A002_X30a93d_X43e.ms,DV11,0.000,0.010,0.000,”Noment” uid___A002_X30a93d_X43e.dup.ms,DV11,0.000,-0.010,0.000,”No comment”

com-

The corrections are used to generate a calibration table which is recorded in the pipeline context and applied to theraw visibility data, on the fly to generate other calibration tables, or permanently to generate calibrated visibilities forimaging.
Outputresults – The results object for the pipeline task is returned.

2.2. hif_antpos

6

Pipeline Tasks Reference Manual, Release 2023.1.0

2.2.2 Parameter List
parameter namevis
caltable
hm_antpos

antenna

offsets

antposfile

description
List of input visibility files. example: [‘ngc5921.ms’]
Name of output gain calibration tables. example: [‘ngc5921.gcal’]Heuristics method for retrieving the antenna position corrections. The
options are ‘online’ (not yet implemented), ‘manual’,and ‘file’.
The list of antennas for which the positions are to be corrected. Available
when hm_antpos=’manual’.
example: antenna=’DV05,DV07’
The list of antenna offsets for each antenna in ‘antennas’. Each offset is aset of 3 floating point numbers separated by commas, specified
in the ITRF frame. Available when hm_antpos=’manual’.
example: offsets=[0.01, 0.02, 0.03, 0.03, 0.02, 0.01]
The file(s) containing the antenna offsets. Used if hm_antpos is ‘file’.
example: ‘antennapos.csv’

2.2.3 Examples
1. Correct the position of antenna 5 for all the visibility files in a single pipeline run:>>> hif_antpos(antenna='DV05', offsets=[0.01, 0.02, 0.03])
2. Correct the position of antennas for all the visibility files in a single pipeline run using antenna positions files on
disk. These files are assumed to conform to a default naming scheme if antposfile is unspecified by the user:>>> hif_antpos(hm_antpos='file', antposfile='myantposfile.csv')

2.2. hif_antpos

7

Pipeline Tasks Reference Manual, Release 2023.1.0

2.3 hif_applycal
2.3.1 Task description
Apply precomputed calibrations to the data.
hif_applycal applies the precomputed calibration tables stored in the pipeline context to the set of visibility files using
predetermined field and spectral window maps and default values for the interpolation schemes.
Users can interact with the pipeline calibration state using the tasks h_export_calstate and h_import_calstate.
Output:results – The results object for the pipeline task is returned

2.3.2 Parameter List
parameter name
vis

field

intentspw

antennaparangdescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
in the pipeline context.
example: [‘X227.ms’]
A string containing the list of field names or field ids to which thecalibration will be applied. Defaults to all fields in the pipeline
context.
example: ‘3C279’, ‘3C279, M82’
A string containing the list of intents against which the selected fields will
be matched. Defaults to all supported intents
in the pipeline context.
example: ‘*TARGET*’The list of spectral windows and channels to which the calibration will beapplied. Defaults to all science windows in the pipeline
context.
example: ‘17’, ‘11, 15’
The selection of antennas to which the calibration will be applied. Defaults
to all antennas. Not currently supported.
Apply parallactic angle correction
continues on next page2.3. hif_applycal

8Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter nameapplymode

calwt
flagbackup
flagsum
flagdetailedsum

paralleldescription
Calibration apply mode ‘calflag’: calibrate data and apply flags fromsolutions‘calflagstrict’: (default) same as above except flag spws for whichcalibration isunavailable in one or more tables (instead of allowing them to pass
uncalibrated and unflagged)‘trial’: report on flags from solutions, dataset entirely unchanged‘flagonly’: apply flags from solutions only, data not calibrated‘flagonlystrict’: same as above except flag spws for which calibration isunavailable in one or more tables‘calonly’: calibrate data only, flags from solutions NOT applied
Calibrate the weights as well as the data
Backup the flags before the apply
Compute before and after flagging summary statistics
Compute detailed before and after flagging statistics summaries. Parameter
available only when if flagsum is True.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to False)

2.3.3 Examples
1. Apply the calibration to the target data>>> hif_applycal (intent='TARGET')

2.3. hif_applycal

9Pipeline Tasks Reference Manual, Release 2023.1.0

2.4 hif_bandpass
2.4.1 Task description
Compute amplitude and phase as a function of frequency for each spectral window in each MeasurementSet.
Previous calibration can be applied on the fly.
hif_bandpass computes a bandpass solution for every specified science spectral window. By default a ‘phaseup’ precalibration is performed and applied on the fly to the data, before the bandpass is computed.
The hif_refant task may be used to precompute a prioritized list of reference antennas.
Outputresults – The results object for the pipeline task is returned.

2.4.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

caltable

The list of output calibration tables. Defaults to the standard pipelinenaming convention.
Example: caltable=[‘M82.gcal’, ‘M82B.gcal’]field

The list of field names or field ids for which bandpasses are computed.
Defaults to all fields.
Examples: field=’3C279’, field=’3C279, M82’

intentA string containing a comma delimited list of intents against which theselected fields are matched. Defaults to all datawith bandpass intent.
Example: intent=’*PHASE*’
continues on next page2.4. hif_bandpass

10

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter namespw

description
The list of spectral windows and channels for which bandpasses arecomputed. Defaults to all science spectral windows.
Example: spw=’11,13,15,17’

antennaphaseupphaseupsolint

Set of data selection antenna IDs
Do a phaseup on the data before computing the bandpass solution.
The phase correction solution interval in CASA syntax. Used whenphaseup is True.
Example: phaseupsolint=300

phaseupbw

Bandwidth to be used for phaseup. Defaults to 500MHz. Used whenphaseup is True.
Examples: phaseupbw=” to use entire bandpass
phaseupbw=’500MHz’ to use central 500MHz

solintcombineTime and channel solution intervals in CASA syntax.
Examples: solint=’inf,10ch’, ‘inf’
Data axes to combine for solving. Axes are ‘’, ‘scan’, ‘spw’, ‘field’ or any
comma-separated combination.
Example: combine=’scan,field’refantReference antenna names. Defaults to the value(s) stored in the pipeline
context. If undefined in the pipeline context
defaults to the CASA reference antenna naming scheme.
Examples: refant=’DV01’, refant=’DV06,DV07’

solnorm

Normalise the bandpass solution
continues on next page2.4. hif_bandpass

11

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter nameminblperant

minsnrdescription
Minimum number of baselines required per antenna for each solve.
Antennas with fewer baselines are excluded fromsolutions.
Reject solutions below this SNR

2.4.3 Examples
1. Compute a channel bandpass for all visibility files in the pipeline context using the CASA reference antenna determination scheme:>>> hif_bandpass()
2. Same as the above but precompute a prioritized reference antenna list:>>> hif_refant()>>> hif_bandpass()

2.5 hif_checkproductsize
2.5.1 Task description
Check interferometry imaging product size and try to mitigate to maximum allowed values. The task implements a
mitigation cascade computing the largest cube size and tries to reduce it below a given limit by adjusting the nbins,
hm_imsize and hm_cell parameters. If this step succeeds, it also checks the overall imaging product size and if
necessary reduces the number of fields to be imaged.
Alternatively, if maximsize is set, the image product pixel count is mitigated by trying to adjust hm_cell parameter.
If the pixel count is still greater than maximsize at hm_cell of 4ppb, then this value is kept and the image field istruncated around the phase center by forcing hm_imsize = maximsize.
Note that mitigation for image pixel count and for the product size currently are mutually exclusive, with maximsize
taking precedence if set.
Output:results – The results object for the pipeline task is returned.

2.5. hif_checkproductsize

12Pipeline Tasks Reference Manual, Release 2023.1.0

2.5.2 Parameter List
parameter namevis

maxcubesize

maxcubelimit

maxproductsize

maximsize

calcsbparalleldescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
Maximum allowed cube size in gigabytes (mitigation goal) -1: automaticfrom performance parameters
Maximum allowed cube limit in gigabytes (mitigation failure limit)-1: automatic from performance parameters
Maximum allowed product size in gigabytes (mitigation goal and failure
limit)-1: automatic from performance parameters
Maximum allowed image count size (mitigation goal and hard maximum).
Parameter maximsize must be even and divisible by 2,3,5,7 only.
Note that maximsize is disabled by default and cannot be set atthe same time as maxcubesize, maxcubelimit and maxproductsize!
-1: disables mitigation for this parameter
Force (re-)calculation of sensitivities and beams
Use MPI cluster where possible

2.5.3 Examples
1. Basic call to check the product sizes using internal defaults>>> hif_checkproductsize()
2. Typical ALMA call>>> hif_checkproductsize(maxcubesize=40.0, maxcubelimit=60.0, maxproductsize=350.0)

2.5. hif_checkproductsize

13

Pipeline Tasks Reference Manual, Release 2023.1.0

2.6 hif_correctedampflag
2.6.1 Task description
Flag corrected - model amplitudes based on calibrators.
This task computes the flagging heuristics on a calibrator by calling hif_correctedampflag which looks for outlier
visibility points by statistically examining the scalar difference of corrected amplitudes minus model amplitudes, andflags those outliers. The philosophy is that only outlier data points that have remained outliers after calibration will
be flagged. The heuristic works equally well on resolved calibrators and point sources because it is not performing avector difference, and thus is not sensitive to nulls in the flux density vs. uvdistance domain. Note that the phase of thedata is not assessed.
Outputresults – The results object for the pipeline task is returned.

2.6.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

intentA string containing a comma delimited list of intents against which theselected fields are matched. If undefined (default),
it will select all data with the BANDPASS intent.
Example: intent=’*PHASE*’

field

The list of field names or field ids for which bandpasses are computed. If
undefined (default), it will select all fields.
Examples: field=’3C279’, ‘3C279, M82’

spw

The list of spectral windows and channels for which bandpasses arecomputed. If undefined (default), it will select allscience spectral windows.
Example: spw=’11,13,15,17’
continues on next page2.6. hif_correctedampflag

14

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 6 – continued from previous page
parameter nameantnegsig

antpossig

tmantint

tmint

tmbl

antblnegsig

antblpossig

relaxed_factor

niter

description
Lower sigma threshold for identifying outliers as a result of bad antennas
within individual timestamps
Upper sigma threshold for identifying outliers as a result of bad antennas
within individual timestamps
Threshold for maximum fraction of timestamps that are allowed to containoutliers
Initial threshold for maximum fraction of “outlier timestamps” over “total
timestamps” that a baseline may be a part ofInitial threshold for maximum fraction of “bad baselines” over “alltimestamps” that an antenna may be a part ofLower sigma threshold for identifying outliers as a result of “bad baselines”
and/or “bad antennas” within baselines (across all timestamps)
Upper sigma threshold for identifying outliers as a result of “bad baselines”
and/or “bad antennas” within baselines (across all timestamps)
Relaxed value to set the threshold scaling factor to under certain conditions(see task description)
Maximum number of times to iterate on evaluation of flagging heuristics.
If an iteration results in no new flags, thensubsequent iterations are skipped.

2.6.3 Examples
Run default flagging on bandpass calibrator with recommended settings:>>> hif_correctedampflag()

2.6. hif_correctedampflag

15

Pipeline Tasks Reference Manual, Release 2023.1.0

2.7 hif_editimlist
2.7.1 Task description
Add to a list of images to be produced with hif_makeimages(), which uses hif_tclean() to invoke CASA tclean. Manyof the hif_editimlist() inputs map directly to tclean parameters.
The results object for the pipeline task is returned.

2.7.2 Parameter List
parameter nameimagenamesearch_radius_arcsec
cell

cfcacheconjbeams
cyclefactor
cycleniterdatatype

description
Prefix for output image names.
Size of the field finding beam search radius in arcsec.
Image X and Y cell size(s) with units or pixels per beam. Single value
same for both. ‘<number>ppb’ for pixels per beam.
Compute cell size based on the UV coverage of all the fields
to be imaged and use a 5 pix per beam sampling.
The pix per beam specification uses the above default cell size(‘5ppb’) and scales it accordingly.
example: [‘0.5arcsec’, ‘0.5arcsec’] ‘3ppb’
Convolution function cache directory name
Use conjugate frequency in tclean for wideband A-terms.
Controls the depth of clean in minor cycles based on PSF.
Controls max number of minor cycle iterations in a single major cycle.
Data type(s) to image. The default ‘’ selects the best available data type(e.g. selfcal over regcal) with
an automatic fallback to the next available data type.
With the datatype parameter of ‘regcal’ or ‘selfcal’, one
can force the use of only given data type(s).
Note that this parameter is only for non-VLASS data when the datacolumn
is not explictly set by user or imaging heuristics.
continues on next page2.7. hif_editimlist

16

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 7 – continued from previous page
parameter namedatacolumn

deconvolver
editmode
field
imaging_mode

imsizeintentgriddermask
pbmask

nbin
nchan
niter
ntermsparameter_file

description
Data column to image; this will take precedence over the datatypeparameter.
Minor cycle algorithm (multiscale or mtmfs)
The edit mode of the task (‘add’ or ‘replace’). Defaults to ‘add’.
Set of data selection field names or ids.
Identity of product type (e.g. VLASS quick look) desired. This will
determine the heuristics used.
Image X and Y size(s) in pixels or PB level (single fields), ‘’ for default.
Single value same for both. ‘<number>pb’ for PB level.
Set of data selection intents
Name of the gridder to use with tclean
Used to declare whether to use a predefined mask for tclean.
Used to declare primary beam gain level for cleaning with primary beammask (usemask=’pb’), used only for VLASS-SE-CONT imaging mode.
Channel binning factor.
Number of channels, -1 = all
The max total number of minor cycle iterations allowed for tclean
Number of Taylor coefficients in the spectral model
keyword=value text file as alternative method of input parameters
continues on next page2.7. hif_editimlist

17

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 7 – continued from previous page
parameter namepblimitphasecenter

reffreq
restfreqrobustscalesspecmode
spw
start

stokessensitivity
thresholdnsigmauvtaperuvrange

description
PB gain level at which to cut off normalizations
The default phase center is set to the mean of the field directions of all
fields that are to be image together.
example: 0, ‘J2000 19h30m00 -40d00m00’
Reference frequency of the output image coordinate system
List of rest frequencies or a rest frequency in a string for output image.
Briggs robustness parameter for tclean
The scales for multi-scale imaging.
Spectral gridding type (mfs, cont, cube, ‘’ for default)
Set of data selection spectral window/channels, ‘’ for all
First channel for frequency mode images. Starts at first input channel of the
spw.
example: ‘22.3GHz’
Stokes Planes to make
Theoretical sensitivity (override internal calculation)
Stopping threshold (number in units of Jy, or string)Multiplicative factor for rms-based threshold stopping
Used to set a uv-taper during clean.
Set of data selection uv ranges, ‘’ for all.
continues on next page2.7. hif_editimlist

18

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 7 – continued from previous page
parameter namewidth

description
Channel width

2.8 hif_findcont
2.8.1 Task description
Find continuum frequency ranges for a list of specified targets.
If the cont.dat file is not already present in the working directory, then dirty image cubes are created for each spectral
window of each science target at the native channel resolution unless the nbins parameter was used in the preceding
hif_makeimlist stage. Robust=1 Briggs weighting is used for optimal line sensitivity, even if a different robust had beenchosen in hifa_imageprecheck to match the PI requested angular resolution. Using moment0 and moment8 images ofeach cube, SNR-based masks are created, and the mean spectrum of the joint mask is computed and evaluated withextensive heuristics to find the channel ranges that are likely to be free of line emission. Warnings are generated if thechannel ranges contain a small fraction of the bandwidth, or sample only a limited extent of the spectrum.
If the cont.dat file already exists in the working directory before this task is executed, then it will first examine thecontents. For any spw that already has frequency ranges defined in this file, it will not perform the analysis describedabove in favor of the a priori ranges. For spws not listed in a pre-existing file, it will analyze them as normal and update
the file. In either case, the cont.dat file is used by the subsequent hif_uvcontsub and hif_makeimages stages.
results – The results object for the pipeline task is returned.

2.8.2 Parameter List
parameter namevistarget_list
hm_mosweighthm_perchanweightdensity

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
Dictionary specifying targets to be imaged; blank will read list from context
Mosaic weighting Defaults to ‘’ to enable the automatic heuristics
calculation.
Can be set to True or False manually.
Calculate the weight density for each channel independently Defaults to ‘’
to enable the automatic heuristics calculation.
Can be set to True or False manually.
continues on next page2.8. hif_findcont

19

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 8 – continued from previous page
parameter namehm_weighting

datacolumn

paralleldescription
Weighting scheme (natural,uniform,briggs,briggsabs[experimental],briggsbwtaper[experimental])
Data column to image. Only to be used for manual overriding when theautomatic choice by data type is not appropriate.
Use MPI cluster where possible

2.8.3 Examples
1. Perform continuum frequency range detection for all science targets and spws:>>> hif_findcont()

2.9 hif_gaincal
2.9.1 Task description
The complex gains are derived from the data column (raw data) divided by the model column (usually set with hif_setjy).
The gains are obtained for a specified solution interval, spw combination and field combination.
Good candidate reference antennas can be determined using the hif_refant task.
Previous calibrations that have been stored in the pipeline context are applied on the fly. Users can interact with thesecalibrations via the h_export_calstate and h_import_calstate tasks.
Outputresults – The results object for the pipeline task is returned.

2.9.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
continues on next page2.9. hif_gaincal

20

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namecaltable

description
The list of output calibration tables. Defaults to the standard pipelinenaming convention.
Example: caltable=[‘M82.gcal’, ‘M82B.gcal’]field

The list of field names or field ids for which gain solutions are to becomputed. Defaults to all fields with the standard
intent.
Example: field=’3C279’, field=’3C279, M82’

intentA string containing a comma delimited list of intents against which theselected fields are matched. Defaults to *PHASE*.
Examples: intent=”, intent=’*AMP*,*PHASE*’

spw

The list of spectral windows and channels for which gain solutions arecomputed. Defaults to all science spectral
windows.
Examples: spw=’21’, spw=’21, 23’antenna
hm_gaintype

calmodeSet of data selection antenna ids
The type of gain calibration. The options are ‘gtype’ and ‘gspline’ for
CASA gain types = ‘G’ and ‘GSPLINE’ respectively.
Type of solution. The options are ‘ap’ (amp and phase), ‘p’ (phase only)and ‘a’ (amp only).
Examples: calmode=’p’, calmode=’a’, calmode=’ap’

solintTime solution intervals in CASA syntax. Works for hm_gaintype=’gtype’
only.
Examples: solint=’inf’, solint=’int’, solint=’100sec’
continues on next page2.9. hif_gaincal

21

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namecombinerefantdescription
Data axes to combine for solving. Options are ‘’, ‘scan’, ‘spw’, ‘field’ or
any comma-separated combination. Works for
hm_gaintype=’gtype’ only.
Reference antenna name(s) in priority order. Defaults to most recent values
set in the pipeline context. If no reference
antenna is defined in the pipeline context use the CASA
defaults.
Examples: refant=’DV01’, refant=’DV05,DV07’

refantmodeControls how the refant is applied. Currently available choices are ‘flex’,‘strict’, and the default value of ‘’.
Setting to ‘’ allows the pipeline to select the appropriate
mode based on the state of the reference antenna list.
Examples: refantmode=’strict’, refantmode=”

solnorm
minblperant

minsnrsmodelNormalize average solution amplitudes to 1.0
Minimum number of baselines required per antenna for each solve.
Antennas with fewer baselines are excluded fromsolutions. Works for hm_gaintype=’gtype’ only.
Solutions below this SNR are rejected. Works for hm_gaintype=’channel’
only.
Point source Stokes parameters for source model (experimental). Defaults
to using standard MODEL_DATA column data.
Example: smodel=[1,0,0,0] - (I=1, unpolarized)

splinetime

npointaver

Spline timescale (sec). Used for hm_gaintype=’gspline’. Typicalsplinetime should cover about 3 to 5 calibrator scans.
Tune phase-unwrapping algorithm. Used for hm_gaintype=’gspline’. Keepat default value.
continues on next page2.9. hif_gaincal

22

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namephasewrap

description
Wrap the phase for changes larger than this amount (degrees). Used for
hm_gaintype=’gspline’. Keep at default value.

2.9.3 Examples
Compute standard per scan gain solutions that will be used to calibrate the target:>>> hif_gaincal()

2.10 hif_lowgainflag
2.10.1 Task description
Deviant antennas are detected by outlier analysis of a view showing their amplitude gains, pre-applying a temporary
bandpass and phase solution. This view is a list of 2D images with axes ‘Scan’ and ‘Antenna’; there is one image foreach spectral window and intent. A flagcmd to flag all data for an antenna will be generated by any gain that is outside
the range [fnm_lo_limit * median, fnm_hi_limit * median].
Output:results – The results object for the pipeline task is returned.

2.10.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

intentA string containing the list of intents to be checked forantennas with deviant gains. The default is blank, whichcauses the task to select the ‘BANDPASS’ intent.
continues on next page2.10. hif_lowgainflag

23

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 10 – continued from previous page
parameter namespw

description
The list of spectral windows and channels to which thecalibration will be applied. Defaults to all science windows
in the pipeline context.
Examples: spw=’17’, spw=’11, 15’

refantA string containing a prioritized list of reference antenna
name(s) to be used to produce the gain table. Defaults to thevalue(s) stored in the pipeline context. If undefined in thepipeline context defaults to the CASA reference antenna naming
scheme.
Examples: refant=’DV01’, refant=’DV06,DV07’

flag_nmedian

fnm_lo_limit
fnm_hi_limit
niter

tmef1_limit

2.10. hif_lowgainflag

Whether to flag figures of merit greater than
fnm_hi_limit * median or lower than fnm_lo_limit * median.
(default: True)
Flag values lower than fnm_lo_limit * median (default: 0.5)
Flag values higher than fnm_hi_limit * median (default: 1.5)
The maximum number of iterations to run of the sequence:solve for amplitude gains, assess statistics, flag spw/antenna
combinations that are outliers (default: 2)
Threshold for “too many entirely flagged” the critical fraction of antennas whose solutions are entirely
flagged in the flagging view of a spw for this stage:if the fraction is equal or greater than this value, then flag
the visibility data from all antennas in this spw(default: 0.666)

24

Pipeline Tasks Reference Manual, Release 2023.1.0

2.10.3 Examples
1. Flag antennas with low or high gain using recommended thresholds:>>> hif_lowgainflag()2.11 hif_makecutoutimages
2.11.1 Task description
Cutout central 1 sq. degree from VLASS QL, SE, and Coarse Cube images
Output:results – The results object for the pipeline task is returned.

2.11.2 Parameter List
parameter namevis

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSs,
or tar files of MSs.
If ASDM files are specified, they will be converted to
MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]

offsetblc
offsettrc-x and -y offsets to the bottom lower corner (blc) in arcseconds+x and +y offsets to the top right corner (trc) in arcseconds

2.11.3 Examples
1. Basic makecutoutimages task>>> hif_makecutoutimages()

2.11. hif_makecutoutimages

25

Pipeline Tasks Reference Manual, Release 2023.1.0

2.12 hif_makeimages
2.12.1 Task description
Compute clean results from a list of specified targets.
Output:results – The results object for the pipeline task is returned.

2.12.2 Parameter List
parameter namevistarget_list
hm_masking

hm_sidelobethreshold
hm_noisethreshold
hm_lownoisethreshold
hm_negativethreshold
hm_minbeamfrac
hm_growiterations
hm_dogrowprune

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
Dictionary specifying targets to be imaged; blank will read list from context
Clean masking mode. Options are ‘centralregion’, ‘auto’, ‘manual’ and‘none’
sidelobethreshold * the max sidelobe level
noisethreshold * rms in residual image
lownoisethreshold * rms in residual image
negativethreshold * rms in residual image
Minimum beam fraction for pruning
Number of binary dilation iterations for growing the mask
Do pruning on the grow mask Defaults to ‘’ to enable the automaticheuristics calculation.
Can be set to True or False manually.
continues on next page2.12. hif_makeimages

26

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 12 – continued from previous page
parameter namehm_minpercentchange
hm_fastnoise

hm_nsigma
hm_perchanweightdensity

hm_npixels
hm_cyclefactor

hm_minpsffraction
hm_maxpsffraction
hm_weighting

hm_cleaning
tlimitdrcorretionmasklimit
cleancontranges

description
Mask size change threshold
Faster noise calculation for automask or nsigma stopping Defaults to ‘’ toenable the automatic heuristics calculation.
Can be set to True or False manually.
Multiplicative factor for rms-based threshold stopping
Calculate the weight density for each channel independently Defaults to ‘’
to enable the automatic heuristics calculation.
Can be set to True or False manually.
Number of pixels to determine uv-cell size for super-uniform weighting
Scaling on PSF sidelobe level to compute the minor-cycle stopping
thresholdPSF fraction that marks the max depth of cleaning in the minor cycle
PSF fraction that marks the minimum depth of cleaning in the minor cycle
Weighting scheme (natural,uniform,briggs,briggsabs[experimental],briggsbwtaper[experimental])
Pipeline cleaning mode
Times the sensitivity limit for cleaning
Override the default heuristics-based DR correction (for ALMA data only)
Times good mask pixels for cleaning
Clean continuum frequency ranges in cubes
continues on next page2.12. hif_makeimages

27

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 12 – continued from previous page
parameter namecalcsbhm_mosweight

overwrite_on_export

paralleldescription
Force (re-)calculation of sensitivities and beams
Mosaic weighting Defaults to ‘’ to enable the automatic heuristics
calculation.
Can be set to True or False manually.
Replace existing image products when h/hifa/hifv_exportdata is called.
If False, images that would have the same FITS name on export,
are amended to include a version number. For example, if
oussid.J1248-4559_ph.spw21.mfs.I.pbcor.fits would already be
exported by a previous call to hif_makeimags, then‘oussid.J1248-4559_ph.spw21.mfs.I.pbcor.v2.fits’ would also beexported to the products/ directory. The first exported
product retains the same name. Additional products start
counting with ‘v2’, ‘v3’, etc.
Clean images using MPI cluster

2.12.3 Examples
1. Compute clean results for all imaging targets defined in a previous hif_makeimlist or hif_editimlist call:>>> hif_makeimages()
2. Compute clean results overriding automatic masking choice:>>> hif_makeimages(hm_masking='centralregion')2.13 hif_makeimlist
2.13.1 Task description
Generate a list of images to be cleaned. By default, the list will include one image per science target per spw. Calibrator
targets can be selected by setting appropriate values for intent.
By default, the output image cell size is set to the minimum cell size consistent with the UV coverage.
By default, the image size in pixels is set to values determined by the cell size and the primary beam size. If a calibrator
is being imaged (intents ‘PHASE’, ‘BANDPASS’, ‘FLUX’ or ‘AMPLITUDE’) then the image dimensions are limited
to ‘calmaxpix’ pixels.
By default, science target images are cubes and calibrator target images are mfs. Science target images may be mosaics
or single fields.
2.13. hif_makeimlist

28

Pipeline Tasks Reference Manual, Release 2023.1.0

Outputresults – The results object for the pipeline task is returned.

2.13.2 Parameter List
parameter namevis

imagenameintentfield

spw

contfilelinesfileuvrange

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
“”: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
Prefix for output image names, “” for automatic.
Select intents for which associated fields will be imaged. Possible choices
are PHASE, BANDPASS, AMPLITUDE, CHECK and
TARGET or combinations thereof.
Examples: ‘PHASE,BANDPASS’, ‘TARGET’
Select fields to image. Use field name(s) NOT id(s). Mosaics are assumed
to have common source / field names. If intent is
specified only fields with data matching the intent will beselected. The fields will be selected from MeasurementSets in“vis”.
“” Fields matching intent, one image per target source.
Select spectral windows to image. “”: Images will be computed for all
science spectral windows.
Name of file with frequency ranges to use for continuum images.
Name of file with line frequency ranges to exclude for continuum images.
Select a set of uv ranges to image. “”: All uv data is included
Examples: ‘0~1000klambda’, [‘0~100klambda’, 100~1000klambda]
continues on next page

2.13. hif_makeimlist

29

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 13 – continued from previous page
parameter namespecmode

outframe
hm_imsize

hm_cellcalmaxpix

description
Frequency imaging mode, ‘mfs’, ‘cont’, ‘cube’, ‘repBW’. ‘’ defaults to‘cube’ if intent parameter includes ‘TARGET’otherwise ‘mfs’.
specmode=’mfs’ produce one image per source and spw
specmode=’cont’ produce one image per source and aggregate
over all specified spws
specmode=’cube’ produce an LSRK frequency cube, channels are
specified in frequency
specmode=’repBW’ produce an LSRK frequency cube atrepresentative channel width
velocity frame of output image (LSRK, ‘’ for automatic) (not implemented)
Image X and Y size in pixels or PB level for single fields. The explicit sizes
must be even and divisible by 2,3,5,7 only.
The default values are derived as follows:
1. Determine phase center and spread of field centersaround it.
2. Set the size of the image to cover the spread of field
centers plus a border of width 0.75 * beam radius, tofirst null.
3. Divide X and Y extents by cell size to arrive at the
number of pixels required.
The PB level setting for single fields leads to an imsize
extending to the specified level plus 5% padding in all
directions.
Examples: ‘0.3pb’, [120, 120]
Image X and Y cell sizes. “” computes the cell size based on the UV
coverage of all the fields to be imaged and uses a
5 pix per beam sampling.
The pix per beam specification (‘<number>ppb’) uses the above
default cell size (‘5ppb’) and scales it accordingly.
The cells can also be specified as explicit measures.
Examples: ‘3ppb’, [‘0.5arcsec’, ‘0.5arcsec’]Maximum image X or Y size in pixels if a calibrator is being imaged(‘PHASE’, ‘BANDPASS’, ‘AMPLITUDE’ or ‘FLUX’ intent).
continues on next page

2.13. hif_makeimlist30

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 13 – continued from previous page
parameter namephasecenter

nchan

start

width

nbins

robustuvtaperclearlist
per_ebper_session
calcsbdescription
Direction measure or field id of the image center. The default phase centeris set to the mean of the field
directions of all fields that are to be image together.
Examples: ‘J2000 19h30m00 -40d00m00’, 0
Total number of channels in the output image(s) -1 selects enough channels
to cover the data selected by
spw consistent with start and width.
Start of image frequency axis as frequency or velocity. “” selects startfrequency automatically.
Output channel width. Difference in frequency between 2 selected channels
forfrequency mode images.
‘pilotimage’ for 15 MHz / 8 channel heuristic
Channel binning factors for each spw. Format: ‘spw1:nb1,spw2:nb2,. . . ’with optional wildcards: ‘*:nb’Examples: ‘9:2,11:4,13:2,15:8’, ‘*:2’Briggs robustness parameter Values range from -2.0 (uniform) to 2.0(natural)uv-taper on outer baselines
Clear any existing target list
Make an image target per EB
Make an image target per session
Force (re-)calculation of sensitivities and beams
continues on next page

2.13. hif_makeimlist

31

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 13 – continued from previous page
parameter namedatatype

datacolumn

paralleldescription
Data type(s) to image. The default ‘’ selects the best available data type(e.g. selfcal over regcal) with
an automatic fallback to the next available data type.
With the datatype parameter one can force the use of only
given data type(s) without a fallback. The data type(s) are
specified as comma separated string of keywords. Acceptedvalues are the standard data types such as‘REGCAL_CONTLINE_ALL’, ‘REGCAL_CONTLINE_SCIENCE’,‘SELFCAL_CONTLINE_SCIENCE’, ‘REGCAL_LINE_SCIENCE’,‘SELFCAL_LINE_SCIENCE’. The shortcuts ‘regcal’ and ‘selfcal’are also accepted. They are expanded into the full data types
using the specmode parameter and the available data types forthe given MSes. In addition the strings ‘best’ and ‘all’ areaccepted, where ‘best’ means the above mentioned automatic
mode and ‘all’ means all available data types for a given
specmode. The data type strings are case insensitive.
Examples: ‘selfcal’, ‘regcal, ‘selfcal,regcal’,‘REGCAL_LINE_SCIENCE,selfcal_line_science’
Data column to image. Only to be used for manual overriding when theautomatic choice by data type is not appropriate.
Use MPI cluster where possible

2.13.3 Examples
1. Make a list of science target images to be cleaned, one image per science spw.

>>> hif_makeimlist()
2. Make a list of PHASE and BANDPASS calibrator targets to be imaged, one image per science spw.

>>> hif_makeimlist(intent='PHASE,BANDPASS')
3. Make a list of PHASE calibrator images observed in spw 1, images limited to 50 pixels on a side.

>>> hif_makeimlist(intent='PHASE',spw='1',calmaxpix=50)

2.13. hif_makeimlist

32

Pipeline Tasks Reference Manual, Release 2023.1.0

2.14 hif_makermsimages
2.14.1 Task description
Create RMS images for VLASS data.
Output:results – The results object for the pipeline task is returned.

2.14.2 Parameter List
parameter namevis

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSs, or tar files of MSs, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]

2.14.3 Examples
1. Basic makermsimages task>>> hif_makermsimages()

2.15 hif_mstransform
2.15.1 Task description
Create new MeasurementSets for imaging from the corrected column of the input MeasurementSet via a single call to
mstransform with all data selection parameters. By default, all science target data is copied to the new MS. The new
MeasurementSet is not re-indexed to the selected data and the new MS will have the same source, field, and spw names
and ids as it does in the parent MS.
Outputresults – The results object for the pipeline task is returned.

2.14. hif_makermsimages

33

Pipeline Tasks Reference Manual, Release 2023.1.0

2.15.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

outputvis

The list of output transformed MeasurementSets to be used for imaging.
The output list must be the same length as the input
list. The default output name defaults to<msrootname>_targets.ms
Examples: ‘ngc5921.ms’,[‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

field

Select fields name(s) or id(s) to transform. Only fields with data matching
the intent will be selected.
Examples: ‘3C279’, ‘Centaurus*’, ‘3C279,J1427-421’

intentSelect intents for which associated fields will be imaged. By default only
TARGET data is selected.
Examples: ‘PHASE,BANDPASS’

spw

chanbintimebin2.15. hif_mstransform

Select spectral window/channels to image. By default all science spws forwhich the specified intent is valid are
selected .
Width (bin) of input channels to average to form an output channel. If
chanbin > 1 then chanaverage is automaticallyswitched to True.
Bin width for time averaging. If timebin > 0s then timeaverage is
automatically switched to True.

34

Pipeline Tasks Reference Manual, Release 2023.1.0

2.15.3 Examples
1. Create a science target MS from the corrected column in the input MS.

>>> hif_mstransform()
2. Make a phase and bandpass calibrator targets MS from the corrected column in the input MS.

>>> hif_mstransform(intent='PHASE,BANDPASS')

2.16 hif_rawflagchans
2.16.1 Task description
hif_rawflagchans flags deviant baseline/channels in the raw data.
The flagging views used are derived from the raw data for the specified intent - default is BANDPASS.
Bad baseline/channels are flagged for all intents, not just the one that is the basis of the flagging views.
For each spectral window the flagging view is a 2d image with axes ‘channel’ and ‘baseline’. The pixel for each channel,
baseline is the time average of the underlying unflagged raw data.
The baseline axis is labeled by numbers of form id1.id2 where id1 and id2 are the IDs of the baseline antennas. Bothid1 and id2 run over all antenna IDs in the observation. This means that each baseline is shown twice but has the benefit
that ‘bad’ antennas are easily identified by eye.
Three flagging methods are available:
If parameter flag_hilo is set True then outliers from the median of each flagging view will be flagged.
If parameter flag_bad_quadrant is set True then a simple 2 part test is used to check for bad antenna quadrants and/or
bad baseline quadrants. Here a ‘quadrant’ is defined simply as one quarter of the channel axis. The first part of thetest is to note as ‘suspect’ those points further from the view median than fbq_hilo_limit * MAD. The second partis to flag entire antenna/quadrants if their fraction of suspect points exceeds fbq_antenna_frac_limit. Failing that,entire baseline/quadrants may be flagged if their fraction of suspect points exceeds fbq_baseline_frac_limit. Suspect
points are not flagged unless as part of a bad antenna or baseline quadrant.
Outputresults – The results object for the pipeline task is returned.

2.16.2 Parameter List
parameter namevis

description
List of input MeasurementSets.
default: [] - Use the MeasurementSets currently known to the pipeline
context.
continues on next page2.16. hif_rawflagchans35

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 16 – continued from previous page
parameter namespw

description
The list of spectral windows and channels to which the calibration will beapplied. Defaults to all science windows in the pipeline
context.
example: spw=’17’, spw=’11, 15’intentA string containing the list of intents to be checked for antennas withdeviant gains. The default is blank, which causes the task to selectthe ‘BANDPASS’ intent.
example: intent=’*BANDPASS*’

flag_hilo

fhl_limit

fhl_minsample

flag_bad_quadrant

fbq_hilo_limit

fbq_antenna_frac_limit

True to flag channel/baseline data further from the view median than
fhl_limit * MAD.
If flag_hilo is True then flag channel/baseline data further from the view
median than fhl_limit * MAD.
Do no flagging if the view median and MAD are derived from fewer than
fhl_minsample view pixels.
True to search for and flag bad antenna quadrants and baseline quadrants.
Here a /’quadrant/’ is one
quarter of the channel axis.
If flag_bad_quadrant is True then channel/baselines further from the view
median than fbq_hilo_limit * MAD will be noted as‘suspect’. If there are enough of them to indicate that an antenna orbaseline quadrant is bad then all channel/baselines in that quadrant will
be flagged.
If flag_bad_quadrant is True and the fraction of suspect channel/baselines
in a particular antenna/quadrant exceeds
fbq_antenna_frac_limit then all data for that antenna/quadrant will
be flagged.
continues on next page2.16. hif_rawflagchans36

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 16 – continued from previous page
parameter namefbq_baseline_frac_limit

description
If flag_bad_quadrant is True and the fraction of suspect channel/baselines
in a particular baseline/quadrant exceeds
fbq_baseline_frac_limit then all data for that baseline/quadrant will
be flagged.

2.16.3 Examples
1. Flag bad quadrants and wild outliers, default method:>>> hif_rawflagchans()
equivalent to:>>> hif_rawflagchans(flag_hilo=True, fhl_limit=20, flag_bad_quadrant=True, fbq_hilo_limit=8,
fbq_antenna_frac_limit=0.2, fbq_baseline_frac_limit=1.0)

2.17 hif_refant
2.17.1 Task description
The hif_refant task selects a list of reference antennas and stores them in the pipeline context in priority order.
The priority order is determined by a weighted combination of scores derived by the antenna selection heuristics. In
manual mode the reference antennas can be set by hand.
Output:results – The results object for the pipeline task is returned.

2.17.2 Parameter List
parameter name
vis

field

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
in the pipeline context.
example: [‘M31.ms’]
The comma delimited list of field names or field ids for which flagging
scores are computed if hm_refant=’automatic’ and flagging = True
example: ‘’ (Default to fields with the specified intents), ‘3C279’,‘3C279,M82’
continues on next page

2.17. hif_refant37

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 17 – continued from previous page
parameter namespw

intenthm_refantrefantgeometry

flagging

parallelrefantignore

2.17. hif_refant

description
A string containing the comma delimited list of spectral window ids forwhich flagging scores are computed if hm_refant=’automatic’ and flagging= True.
example: ‘’ (all spws observed with the specified intents), ‘11,13,15,17’
A string containing a comma delimited list of intents against which theselected fields are matched. Defaults to all supported
intents.
example: ‘BANDPASS’,‘AMPLITUDE,BANDPASS,PHASE,POLARIZATION’
The heuristics method or mode for selection the reference antenna. The
options are ‘manual’ and ‘automatic. In manualmode a user supplied reference antenna refant is supplied.
In ‘automatic’ mode the antennas are selected automatically.
The user supplied reference antenna for hm_refant=’manual’. If no antenna
list is supplied an empty list is returned.
example: ‘DV05’
Score antenna by proximity to the center of the array. This option is quick
as only the ANTENNA table must be read.
Parameter is available when ``hm_refant``=’automatic’.
Score antennas by percentage of unflagged data. This option requires
computing flagging statistics.
Parameter is available when ``hm_refant``=’automatic’.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to False)
string list to be ignored as reference antennas. example:
refantignore=’ea02,ea03’38

Pipeline Tasks Reference Manual, Release 2023.1.0

2.17.3 Examples
1. Compute the references antennas to be used for bandpass and gain calibration.

>>> hif_refant()

2.18 hif_selfcal
2.18.1 Task description
Determine and apply self-calibration with the science target data

2.18.2 Parameter List
parameter name
vis

field

spw

contfile

apply

recal

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
default = “”: use all MeasurementSets in the context
Select fields to image. Use field name(s) NOT id(s). Mosaics are assumed
to have common source / field names. If intent is
specified only fields with data matching the intent will beselected. The fields will be selected from MeasurementSets in“vis”.
default= “” Fields matching intent, one image per target source.
Select spectral windows to image. “”: Images will be computed for all
science spectral windows.
Name of file to specify line-free frequency ranges for selfcal continuum
imaging. default=”cont.dat”
Apply final selfcal solutions back to the input MeasurementSets. default =True
Always re-do self-calibration even solutions/caltables are found in the
Pipeline context or json restore file.
default = False
continues on next page

2.18. hif_selfcal

39

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 18 – continued from previous page
parameter namerefantignore

restore_resources

n_solints
amplitude_selfcal

gaincal_minsnr
minsnr_to_proceeddelta_beam_thresh

apply_cal_mode_default

rel_thresh_scaling

description
string list to be ignored as reference antennas. example:
refantignore=’ea02,ea03’
Path to the restore resources from a standard run of hif_selfcal. hif_selfcal
will automatically do an exhaustive search to lookup/extract/verify
the selfcal restore resources, i.e., selfcal.json and all selfcal-caltablereferredin selfcal.json, starting from working/, to products/ and rawdata/.
If restore_resources is specified, this file path will be evaluated first
before the pre-defined exhaustive search list.
The value can be the file path of *auxproducts.tgz file or *selfcal.json file.
number of solution intervals to attempt for self-calibration. default: 4
Attempt amplitude self-calibration following phase-only self-calibration; if
median time between scans of a given target is < 150s,
solution intervals of 300s and inf will be attempted, otherwise justinf will be attempted.
default = False
Minimum S/N for a solution to not be flagged by gaincal. default = 2.0
Minimum estimated S/N on a per antenna basis to attempt self-calibrationof a source.
default = 3.0
Allowed fractional change in beam size for selfcalibration to accept resultsof a solution interval.
default = 0.05
Apply mode to use for applycal task during self-calibration; same options
as applycal.
default = ‘calflag’
Scaling type to determine how clean thresholds per solution interval should
be determined going from the starting
clean threshold to 3.0 * RMS for the final solution interval.
default=’log10’, options: ‘linear’, ‘log10’, or ‘loge’ (natural log)
continues on next page

2.18. hif_selfcal

40

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 18 – continued from previous page
parameter name
dividing_factor

check_all_spws

inf_EB_gaincal_combine

description
Scaling factor to determine clean threshold for first self-calibration solution
interval.
Equivalent to (Peak S/N / dividing_factor) *RMS = First clean threshold;however, if (Peak S/N / dividing_factor) *RMS is < 5.0; a value of 5.0
is used for the first clean threshold.
default = 40 for < 8 GHz; 15 for > 8 GHz
If True, the S/N of mfs images created on a per-spectral-window basis will
be compared at the initial stages final self-calibration.
default=Falsechange gain solution combination parameters for the inf_EB solution
interval. if True, the gaincal combine parameter will be set to ‘scan,spw’; if
False,
the gaincal combine parameter will be set to ‘scan’.
default=False

2.18.3 Examples
1. Run self-calibration and apply solutions to all science targets and spws>>> hif_selfcal()
2. Run self-calibration and apply solutions to a single science target>>> hif_selfcal(field="3C279")
3. Run self-calibration with a more relaxed allowed fractional change in the beam size for a solution interval to be
successful>>> hif_selfcal(delta_beam_thresh=0.15)

2.19 hif_setjy
2.19.1 Task description
Fills the model column with the model visibilities.
Outputresults – The results object for the pipeline task is returned.

2.19. hif_setjy

41

Pipeline Tasks Reference Manual, Release 2023.1.0

2.19.2 Parameter List
parameter name
vis

field

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
defined in the pipeline context.
The list of field names or field ids for which the models are to be set.
Defaults to all fields with intent ‘*AMPLITUDE*’.
example: field=’3C279’, field=’3C279, M82’

intentA string containing a comma delimited list of intents against which theselected fields are matched. Defaults to all datawith amplitude intent.
example: intent=’*AMPLITUDE*’

spw

The list of spectral windows and channels for which bandpasses arecomputed. Defaults to all science spectral windows.
example: spw=’11,13,15,17’

model

Model image for setting model visibilities. Not fully supported.
example: see details in help for CASA setjy task

reffilePath to a file containing flux densities for calibrators unknown to CASA.
Values given in this file take precedence over the CASA-derived
values for all calibrators except solar system calibrators. By default thepath is set to the CSV file created by h_importdata, consisting of
catalogue fluxes extracted from the ASDM.
example: reffile=”, reffile=’working/flux.csv’

normfluxes

Normalize lookup fluxes.
continues on next page

2.19. hif_setjy

42

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 19 – continued from previous page
parameter namereffreq

description
The reference frequency for spix, given with units. Provided to avoid
division by zero. If the flux density is being scaled by spectral
index, then reffreq must be set to whatever reference frequency is correct
for the given fluxdensity and spix. It cannot be determined from vis. On
the other hand, if spix is 0, then any positive frequency can be used andwill be ignored.
example: reffreq=’86.0GHz’, reffreq=’4.65e9Hz’

fluxdensity

Specified flux density [I,Q,U,V] in Jy. Uses [1,0,0,0] flux density forunrecognized sources, and standard flux densities forones recognized by ‘standard’, including 3C286, 3C48, 3C147, and several
planets, moons, and asteroids.
example: [3.06,0.0,0.0,0.0]

spix

scalebychan

standard

Spectral index for fluxdensity S = fluxdensity * (freq/reffreq)**spix Only
used if fluxdensity is being used. If fluxdensity is positive, andspix is nonzero, then reffreq must be set too. It is applied in the same
way to all polarizations, and does not account for Faraday rotation or
depolarization.
This determines whether the fluxdensity set in the model is calculated on a
per channel basis. If False then only one fluxdensity
value is calculated per spw.
Flux density standard, used if fluxdensity[0] less than 0.0. The options are:‘Baars’,’Perley 90’,’Perley-Taylor 95’, ‘Perley-Taylor 99’,
‘Perley-Butler 2010’ and ‘Butler-JPL-Horizons 2010’.
default: ‘Butler-JPL-Horizons 2012’ for solar system object‘Perley-Butler 2010’ otherwise

2.19. hif_setjy

43

Pipeline Tasks Reference Manual, Release 2023.1.0

2.19.3 Examples
1. Set the model flux densities for all the amplitude calibrators:>>> hif_setjy()

2.20 hif_setmodels
2.20.1 Task description
Set model fluxes values for calibrator reference and transfer sources using lookup values. By default the reference
sources are the flux calibrators and the transfer sources are the bandpass, phase, and check source calibrators. Reference
sources which are also in the transfer source list are removed from the transfer source list.
Built-in lookup tables are used to compute models for solar system object calibrators. Point source models are used forother calibrators with flux densities provided in the reference file. Normalized fluxes are computed for transfer sources
if the normfluxes parameter is set to True.
The default reference file is ‘flux.csv’ in the current working directory. This file is usually created in the importdata
stage. The file is in ‘csv’ format and contains the following comma delimited columns.
vis,fieldid,spwid,I,Q,U,V,pix,comment
Output:results – The results object for the pipeline task is returned

2.20.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the pipeline context.
example: [‘M32A.ms’, ‘M32B.ms’]

reference

A string containing a comma delimited list of field names defining thereference calibrators. Defaults to field names withintent ‘AMPLITUDE’.
example: ‘M82,3C273’

refintent

A string containing a comma delimited list of intents used to select thereference calibrators. Defaults to ‘AMPLITUDE’.
example: ‘BANDPASS’continues on next page

2.20. hif_setmodels

44

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 20 – continued from previous page
parameter nametransfer

description
A string containing a comma delimited list of field names defining thetransfer calibrators. Defaults to field names withintent ‘’.
example: ‘J1328+041,J1206+30’

transintent

A string containing a comma delimited list of intents defining the transfer
calibrators. Defaults to ‘BANDPASS,PHASE,CHECK’.
‘’ stands for no transfer sources.
example: ‘PHASE’

reffilenormfluxes
scalebychan

The reference file containing a lookup table of point source models This
file currently defaults to ‘flux.csv’ in the working directory. This
file must conform to the standard pipeline ‘flux.csv’ format
example: ‘myfluxes.csv’
Normalize the transfer source flux densities.
Scale the flux density on a per channel basis or else on a per spw basis

2.20.3 Examples
1. Set model fluxes for the flux, bandpass, phase, and check sources.

>>> hif_setmodels()

2.21 hif_transformimagedata
2.21.1 Task description
Extract fields for the desired VLASS image to a new MS and reset weights if desired
Output:results – The results object for the pipeline task is returned.

2.21. hif_transformimagedata45

Pipeline Tasks Reference Manual, Release 2023.1.0

2.21.2 Parameter List
parameter namevis

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSs, or tar files of MSs, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]

outputvis
field
intentspw
datacolumn

The output MeasurementSet.
Set of data selection field names or ids, ‘’ for all.
Set of data selection intents, ‘’ for all.
Set of data selection spectral window ids ‘’ for all.
Select spectral windows to split. The standard CASA options are supported
example: ‘data’, ‘model’chanbintimebin
replaceclear_pointing
modify_weights
wtmode2.21. hif_transformimagedata

Bin width for channel averaging.
Bin width for time averaging.
If a split was performed delete the parent MS and remove it from thecontext.
example: True or False
Clear the pointing table.
Re-initialize the weights.
optional weight initialization mode when modify_weights=True

46

Pipeline Tasks Reference Manual, Release 2023.1.0

2.21.3 Examples
1. Basic transformimagedata task>>> hif_transformimagedata()

2.22 hif_uvcontsub
2.22.1 Task description
hif_uvcontsub fits the continuum for the frequency ranges given in the cont.dat file, subtracts that fit from the uv dataand generates a new set of MSes containing the continuum subtracted (i.e. line) data. The fit is attempted for all sciencetargets and spws. If a fit is impossible, the corresponding data selection is not written to the output line MS.
results – The results object for the pipeline task is returned

2.22.2 Parameter List
parameter name
vis

field

intentspw

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
The list of field names or field ids for which UV continuum fits arecomputed. Defaults to all fields.
Examples: ‘3C279’, ‘3C279,M82’
A string containing a comma delimited list of intents against which theselected fields are matched.
‘’: Defaults to all data with TARGET intent.
The list of spectral windows and channels for which uv continuum fits arecomputed.
‘’, Defaults to all science spectral windows.
Example: ‘11,13,15,17’
continues on next page

2.22. hif_uvcontsub

47

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 22 – continued from previous page
parameter name
fitorderdescription
Polynomial order for the continuum fits per source and spw. Defaults to {}which means fit order 1 for all sources andspws. If an explicit dictionary is given then all unspecified
selections still default to 1.
Example: {‘3C279’: {‘15’: 1, ‘17’: 2}, ‘M82’: {‘13’: 2}}

2.22.3 Examples
1. Fit and subtract continuum for all science targets and spws>>> hif_uvcontsub()
2. Fit and subtract continuum only for a subsect of fields>>> hif_uvcontsub(field='3C279,M82'
3. Fit and subtract continuum only for a subsect of spws>>> hif_uvcontsub(spw='11,13')
4. Override automatic fit order choice>>> hif_uvcontsub(fitorder={'3C279': {'15': 1, '17': 2}, 'M82': {'13': 2}})2.22. hif_uvcontsub

48

CHAPTER

THREE

INTERFEROMETRY ALMA

26 tasks available.
task namedescription

hifa_antpos
hifa_bandpass
hifa_bandpassflag
hifa_bpsolint
hifa_exportdata
hifa_flagdata
hifa_flagtargets
hifa_fluxcalflag
hifa_gaincalsnr
hifa_gfluxscale
hifa_gfluxscaleflag
hifa_imageprecheck

Derive an antenna position calibration table
Compute bandpass calibration solutions
Bandpass calibration flagging
Compute optimal bandpass calibration solution intervals
Prepare interferometry data for export
Do metadata based flagging of a list of MeasurementSets.
Do science target flagging
Locate and flag line regions in solar system flux calibrators
Compute gaincal signal-to-noise ratios per spw
Derive flux density scales from standard calibrators
Flag the phase, pol, flux calibrators
Calculates the best Briggs robust parameter to achieve sensitivity and angular resolution goals.
Imports data into the interferometry pipeline
Lock reference antenna list
Derive instrumental polarization calibration for ALMA.
Flag polarization calibrators
ALMA renormalization task
Restore flagged and calibration interferometry data from a pipeline run
Select best reference antenna for session(s)Compute phase calibration spw map and per spw phase offsets
Flag target source outliers
Determine temporal gains from calibrator observations
Flag deviant system temperatures for ALMA interferometry measurements.
Unlock reference antenna list

hifa_importdata
hifa_lock_refanthifa_polcal
hifa_polcalflag
hifa_renorm
hifa_restoredata
hifa_session_refanthifa_spwphaseup
hifa_targetflag
hifa_timegaincal
hifa_tsysflag
hifa_unlock_refanthifa_wvrgcal

Generate a gain table based on Water Vapor Radiometer data, and calculate
a QA score based on its effect on the interferometric data.
hifa_wvrgcalflag
Generate a gain table based on Water Vapor Radiometer data, interpolating
overantennas with bad radiometers.

49

Pipeline Tasks Reference Manual, Release 2023.1.0

3.1 hifa_antpos
3.1.1 Task description
The hifa_antpos task corrects the antenna positions recorded in the ASDMs using updated antenna position calibration
information determined after the observation was taken.
Corrections can be input by hand, read from a file on disk, or in the future by querying an ALMA database service.
The antenna positions file is in ‘csv’ format containing 6 comma-delimited columns as shown below. This file should
not include blank lines, including after the end of the last entry. The default name of this file is ‘antennapos.csv’.
Example of contents for an ‘antennapos.csv’ file:ms,antenna,xoffset,yoffset,zoffset,comment
uid___A002_X30a93d_X43e.ms,DV11,0.000,0.010,0.000,”No comment”
uid___A002_X30a93d_X43e.dup.ms,DV11,0.000,-0.010,0.000,”No comment”
The offset values in this file are in meters.
The corrections are used to generate a calibration table which is recorded in the pipeline context and applied to theraw visibility data, on the fly to generate other calibration tables, or permanently to generate calibrated visibilities forimaging.
Note: the hm_antpos ‘online’ option will be implemented when the observing system provides an antenna position
determination service.
Output:results – The results object for the pipeline task is returned.

3.1.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

caltable

List of names for the output calibration tables. Defaults
to the standard pipeline naming convention.
Example: caltable=[‘ngc5921.gcal’]continues on next page

3.1. hifa_antpos

50

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 1 – continued from previous page
parameter namehm_antpos

description
Heuristics method for retrieving the antenna position
corrections. The options are ‘online’ (not yet implemented),‘manual’, and ‘file’.
Example: hm_antpos=’manual’antenna

The list of antennas for which the positions are to be corrected
if hm_antpos is ‘manual’.
Example: antenna=’DV05,DV07’

offsets

The list of antenna offsets for each antenna in ‘antennas’.
Each offset is a set of 3 floating point numbers separated bycommas, specified in the ITRF frame.
Example: offsets=[0.01, 0.02, 0.03, 0.03, 0.02, 0.01]

antposfile

threshold

The file(s) containing the antenna offsets. Used if
hm_antpos is ‘file’.
Highlight antenna position offsets greater than this value inthe weblog. Units are wavelengths and the default is 1.0.
Example: threshold=1.0

3.1.3 Examples
1. Correct the position of antenna ‘DV05’ for all the visibility files in a single pipeline run:>>> hifa_antpos(antenna='DV05', offsets=[0.01, 0.02, 0.03])
2. Correct the position of antennas for all the visibility files in a single pipeline run using antenna positions files on
disk. These files are assumed to conform to a default naming scheme if antposfile is unspecified by the user:>>> hifa_antpos(hm_antpos='file', antposfile='myantposfile.csv')

3.1. hifa_antpos

51

Pipeline Tasks Reference Manual, Release 2023.1.0

3.2 hifa_bandpass
3.2.1 Task description
The hifa_bandpass task computes a bandpass solution for every specified science spectral window. By default, a‘phaseup’ pre-calibration is performed and applied on the fly to the data, before the bandpass is computed.
The hif_refant task may be used to pre-compute a prioritized list of reference antennas.
Output:results – The results object for the pipeline task is returned.

3.2.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

caltable

List of names for the output calibration tables. Defaults
to the standard pipeline naming convention.
Example: caltable=[‘ngc5921.gcal’]field

The list of field names or field ids for whichbandpasses are computed. Set to field=” by default,
which means the task will select all fields.
Example: field=’3C279’, field=’3C279,M82’

intentA string containing a comma delimited list of intents
against which the selected fields are matched. Set tointent=” by default, which means the task will selectall data with the BANDPASS intent.
Example: intent=’*PHASE*’
continues on next page3.2. hifa_bandpass

52

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter namespw

description
The list of spectral windows and channels for whichbandpasses are computed. Set to spw=” by default, whichmeans the task will select all science spectral windows.
Example: spw=’11,13,15,17’

antennahm_phaseup

Set of data selection antenna IDs
The pre-bandpass solution phaseup gain heuristics. The
options are:‘snr’: compute solution required to achieve the specified SNR‘manual’: use manual solution parameters‘’: skip phaseup
Example: hm_phaseup=’manual’phaseupsolint

The phase correction solution interval in CASA syntax.
Used when hm_phaseup = ‘manual’ or as a default if
the hm_phaseup = ‘snr’ heuristic computation fails.
Example: phaseupsolint=’300s’

phaseupbw

Bandwidth to be used for phaseup. Used whenhm_phaseup = ‘manual’.
Example:phaseupbw=” to use entire bandpass
phaseupbw=’500MHz’ to use central 500MHz

phaseupsnr

The required SNR for the phaseup solution. Used to calculate
the phaseup time solint, and only ifhm_phaseup = ‘snr’.
Example: phaseupsnr=10.0

phaseupnsols

The minimum number of phaseup gain solutions. Used only
if hm_phaseup = ‘snr’.
Example: phaseupnsols=4
continues on next page3.2. hifa_bandpass

53

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter namehm_bandpass

solintmaxchannels

evenbpints

bpsnr

minbpsnr

bpnsols

description
The bandpass solution heuristics. The options are:‘snr’: compute the solution required to achieve the specified SNR‘smoothed’: simple ‘smoothing’ i.e. spectral solint>1chan‘fixed’: use the user defined parameters for all spws
Example: hm_bandpass=’snr’
Time and channel solution intervals in CASA syntax.
Default is solint=’inf’, which is used whenhm_bandpass = ‘fixed’.
If hm_bandpass = ‘snr’, then the task will attempt
to compute and use an optimal SNR-based solint (and warn
if this solint is not good enough).
If hm_bandpass = ‘smoothed’, the task will override
the spectral solint with bandwidth/maxchannels.
Example: solint=’int’
The bandpass solution ‘smoothing’ factor in channels,
i.e. spectral solint will be set to bandwidth / maxchannels
Set to 0 for no smoothing. Used if
hm_bandpass = ‘smoothed’.
Example: maxchannels=240
Force the per spw frequency solint to be evenly divisible
into the spw bandpass if hm_bandpass = ‘snr’.
Example: evenbpints=False
The required SNR for the bandpass solution. Used only if
hm_bandpass = ‘snr’.
Example: bpsnr=30.0
The minimum required SNR for the bandpass solution
when strong atmospheric lines exist in Tsys spectra.
Used only if hm_bandpass=’snr’.
Example: minbpsnr=10.0
The minimum number of bandpass solutions. Used only if
hm_bandpass = ‘snr’.
Example: bpnsols=8
continues on next page3.2. hifa_bandpass

54

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter namecombinerefantsolnorm

minblperant

minsnrunregister_existing

fillgaps

3.2. hifa_bandpass

description
Data axes to combine for solving. Axes are ‘’, ‘scan’,‘spw’, ‘field’ or any comma-separated combination.
Example: combine=’scan,field’
List of reference antenna names. Defaults to thevalue(s) stored in the pipeline context. If undefined in
the pipeline context defaults to the CASA reference
antenna naming scheme.
Example: refant=’DV06,DV07’
Normalise the bandpass solution; defaults to True.
Example: solnorm=False
Minimum number of baselines required per antenna foreach solve. Antennas with fewer baselines are excluded
from solutions.
Example: minblperant=4
Solutions below this SNR are rejected in the phaseup andbandpass solves.
Example: minsnr=3.0
Unregister all bandpass calibrations from the pipeline
context before registering the new bandpass calibrations
from this task. Defaults to False.
Example: unregister_existing=True
Fill flagged solution channels by interpolation.
Example: fillgaps=1055

Pipeline Tasks Reference Manual, Release 2023.1.0

3.2.3 Examples
1. Compute a channel bandpass for all visibility files in the pipeline context using the CASA reference antenna determination scheme:>>> hifa_bandpass()
2. Same as the above but precompute a prioritized reference antenna list:>>> hif_refant()>>> hifa_bandpass()

3.3 hifa_bandpassflag
3.3.1 Task description
This task performs a preliminary phased-up bandpass solution and temporarily applies it, then computes the flagging
heuristics by calling hif_correctedampflag which looks for outlier visibility points by statistically examining the scalardifference of the corrected amplitudes minus model amplitudes, and then flags those outliers. The philosophy is that
only outlier data points that have remained outliers after calibration will be flagged. Note that the phase of the data is
not assessed.
Plots are generated at two points in this workflow: after bandpass calibration but before flagging heuristics are run,and after flagging heuristics have been run and applied. If no points were flagged, the ‘after’ plots are not generated ordisplayed.
Output:results – The results object for the pipeline task is returned.

3.3.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

caltable

List of names for the output calibration tables. Defaults
to the standard pipeline naming convention.
Example: caltable=[‘ngc5921.gcal’]continues on next page3.3. hifa_bandpassflag

56

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter nameintentdescription
A string containing a comma delimited list of intents against
which the selected fields are matched. Set to intent=” by default, whichmeans the task will select all data with the BANDPASS intent.
Example: intent=’*PHASE*’

field

The list of field names or field ids for which bandpasses arecomputed. Set to field=” by default, which means the task
will select all fields.
Example: field=’3C279’, field=’3C279,M82’

spw

The list of spectral windows and channels for which bandpasses arecomputed. Set to spw=” by default, which means the task will select allscience spectral windows.
Example: spw=’11,13,15,17’

antennahm_phaseup

Set of data selection antenna IDs
The pre-bandpass solution phaseup gain heuristics. The options are:‘snr’: compute solution required to achieve the specified SNR‘manual’: use manual solution parameters‘’: skip phaseup
Example: hm_phaseup=’manual’phaseupsolint

The phase correction solution interval in CASA syntax.
Used when hm_phaseup = ‘manual’ or as a default if thehm_phaseup = ‘snr’ heuristic computation fails.
Example: phaseupsolint=’300s’

phaseupbw

Bandwidth to be used for phaseup. Used whenhm_phaseup = ‘manual’.
Example:phaseupbw=” to use entire bandpass
phaseupbw=’500MHz’ to use central 500MHz
continues on next page3.3. hifa_bandpassflag

57

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namephaseupsnr

description
The required SNR for the phaseup solution. Used only ifhm_phaseup=’snr’.
Example: phaseupsnr=10.0

phaseupnsols

The minimum number of phaseup gain solutions. Used only ifhm_phaseup=’snr’.
Example: phaseupnsols=4

hm_bandpass

solintmaxchannels

evenbpints

The bandpass solution heuristics. The options are:‘snr’: compute the solution required to achieve the specified SNR‘smoothed’: simple ‘smoothing’ i.e. spectral solint>1chan‘fixed’: use the user defined parameters for all spws
Time and channel solution intervals in CASA syntax.
Default is solint=’inf’, which is used whenhm_bandpass = ‘fixed’.
If hm_bandpass = ‘snr’, then the task will attempt tocompute and use an optimal SNR-based solint (and warn if thissolint is not good enough).
If hm_bandpass = ‘smoothed’, the task will override thespectral solint with bandwidth/maxchannels.
The bandpass solution ‘smoothing’ factor in channels, i.e.
spectral solint will be set to bandwidth/maxchannels
Set to 0 for no smoothing.
Used if hm_bandpass = ‘smoothed’.
Example: maxchannels=240
Force the per spw frequency solint to be evenly divisible
into the spw bandpass if hm_bandpass = ‘snr’.
Example: evenbpints=False

bpsnr

The required SNR for the bandpass solution. Used only if
hm_bandpass = ‘snr’.
Example: bpsnr=30.0
continues on next page3.3. hifa_bandpassflag

58

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter nameminbpsnr

description
The minimum required SNR for the bandpass solution
when strong atmospheric lines exist in Tsys spectra.
Used only if hm_bandpass = ‘snr’.
Example: minbpsnr=10.0

bpnsolscombine

The minimum number of bandpass solutions. Used only if
hm_bandpass = ‘snr’.
Example: bpnsols=8
Data axes to combine for solving. Axes are ‘’, ‘scan’, ‘spw’,‘field’ or any comma-separated combination.
Example: combine=’scan,field’refant

List of reference antenna names. Defaults to the value(s) stored in thepipeline context. If undefined in the pipeline context defaults to
the CASA reference antenna naming scheme.
Example: refant=’DV06,DV07’

minblperant

minsnrsolnorm
antnegsig

antpossig

Minimum number of baselines required per antenna for each solve.
Antennas with fewer baselines are excluded from solutions.
Example: minblperant=4
Solutions below this SNR are rejected.
Example: minsnr=3.0
Normalise the bandpass solution; defaults to True.
Lower sigma threshold for identifying outliers as a result of
bad antennas within individual timestamps.
Example: antnegsig=4.0
Upper sigma threshold for identifying outliers as a result of
bad antennas within individual timestamps.
Example: antpossig=4.6
continues on next page3.3. hifa_bandpassflag

59

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter nametmantint

tmint

tmbl

antblnegsig

antblpossig

relaxed_factor

niter3.3. hifa_bandpassflag

description
Threshold for maximum fraction of timestamps that are allowed
to contain outliers.
Example: tmantint=0.063
Initial threshold for maximum fraction of ‘outlier timestamps’
over ‘total timestamps’ that a baseline may be a part of.
Example: tmint=0.085
Initial threshold for maximum fraction of ‘bad baselines’ over‘all baselines’ that an antenna may be a part of.
Example: tmbl=0.175
Lower sigma threshold for identifying outliers as a result of‘bad baselines’ and/or ‘bad antennas’ within baselines (across alltimestamps).
Example: antblnegsig=3.4
Upper sigma threshold for identifying outliers as a result of‘bad baselines’ and/or ‘bad antennas’ within baselines (across alltimestamps).
Example: antblpossig=3.2
Relaxed value to set the threshold scaling factor to undercertain conditions (see documentation of the underlying correctedampflagtask).
Example: relaxed_factor=2.0
Maximum number of times to iterate on evaluation of flagging
heuristics. If an iteration results in no new flags, then subsequent
iterations are skipped.
Example: niter=260

Pipeline Tasks Reference Manual, Release 2023.1.0

3.3.3 Examples
1. run with recommended settings to create bandpass solution with flagging using recommended thresholds:>>> hifa_bandpassflag()

3.4 hifa_bpsolint
3.4.1 Task description
The optimal bandpass phaseup time and frequency solution intervals required to achieve the required signal-to-noise
ratio is estimated based on nominal ALMA array characteristics the metadata associated with the observation.
The phaseup gain time and bandpass frequency intervals are determined as follows:• For each data set the list of source(s) to use for bandpass solution signal-to-noise estimation is compiled basedon the values of the field, intent, and spw parameters.
• Source fluxes are determined for each spw and source combination.
• Fluxes in Jy are derived from the pipeline context.
• Pipeline context fluxes are derived from the online flux calibrator catalog, the ASDM, or the user via the flux.csv
file.
• If no fluxes are available the task terminates.
• Atmospheric calibration and observations scans are determined for each spw and source combination.
• If intent is set to ‘PHASE’ are there are no atmospheric scans associated with the ‘PHASE’ calibrator, ‘TARGET’
atmospheric scans will be used instead.
• If atmospheric scans cannot be associated with any of the spw and source combinations the task terminates.
• Science spws are mapped to atmospheric spws for each science spw and source combination.
• If mappings cannot be determined for any of the spws the task terminates.
• The median Tsys value for each atmospheric spw and source combination is determined from the SYSCAL table.
Medians are computed first by channel, then by antenna, in order to reduce sensitivity to deviant values.
• The science spw parameters, exposure time(s), and integration time(s) are determined.
• The phase-up time interval, in time units and number of integrations required to meet the phaseupsnr are computed, along with the sensitivity in mJy and the signal-to-noise per integration. Nominal Tsys and sensitivity
values per receiver band provided by the ALMA project are used for this estimate.
• Warnings are issued if estimated phase-up gain time solution would contain fewer than minphaseupints solutions.
• The frequency interval, in MHz and number of channels required to meet the bpsnr are computed, along withthe per channel sensitivity in mJy and the per channel signal-to-noise. Nominal Tsys and sensitivity values perreceiver band provided by the ALMA project are used for this estimate.
• Warnings are issued if estimated bandpass solution would contain fewer than minbpnchan solutions.
• If strong atmospheric features are detected in the Tsys spectrum for a given spw, the frequency interval of bandpass solution is recalculated to meet the lower threshold, minbpsnr - i.e. a lower snr is tolerated in order topreserve enough frequency intervals to capture the atmospheric line.
Output:3.4. hifa_bpsolint

61

Pipeline Tasks Reference Manual, Release 2023.1.0

results – The results object for the pipeline task is returned.

3.4.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
example: vis=[‘M82A.ms’, ‘M82B.ms’]

field

The list of field names of sources to be used forsignal-to-noise estimation. Defaults to all fields with thestandard intent.
example: field=’3C279’

intentA string containing a comma delimited list of intents against
which the selected fields are matched. Defaults to‘BANDPASS’.
example: intent=’PHASE’spw

The list of spectral windows and channels for which gain
solutions are computed. Defaults to all the science spectral
windows for which there are both ‘intent’ and TARGET intents.
example: spw=’13,15’

phaseupsnr

The required phase-up gain time interval solution
signal-to-noise.
example: phaseupsnr=10.0

minphaseupints

The minimum number of time intervals in the phase-up gain
solution.
example: minphaseupints=4
continues on next page

3.4. hifa_bpsolint

62

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter nameevenbpints

bpsnr

description
Use a bandpass frequency solint that is an integer divisor of
the spw bandwidth, to prevent the occurrence of one narrower
fractional frequency interval.
The required bandpass frequency interval solution
signal-to-noise.
example: bpsnr=30.0

minbpsnr

The minimum required bandpass frequency interval solution
signal-to-noise when strong atmospheric lines exist in Tsys
spectra.
example: minbpsnr=10.0

minbpnchan

The minimum number of frequency intervals in the bandpass
solution.
example: minbpnchan=16

hm_nantennas

The heuristics for determines the number of antennas to use
in the signal-to-noise estimate. The options are ‘all’ and‘unflagged’. The ‘unflagged’ options is not currently
supported.
example: hm_nantennas=’unflagged’

maxfracflagged

The maximum fraction of an antenna that can be flagged before
it is excluded from the signal-to-noise estimate.
example: maxfracflagged=0.80

3.4. hifa_bpsolint

63

Pipeline Tasks Reference Manual, Release 2023.1.0

3.4.3 Examples
1. Estimate the phaseup gain time interval and the bandpass frequency interval required to match the desired signalto-noise for bandpass solutions:>>> hifa_bpsolint()

3.5 hifa_exportdata
3.5.1 Task description
Prepare interferometry data for export.
The hifa_exportdata task for ALMA CASA pipeline exports the data defined in the pipeline context and exports it to
the data products directory, converting and or packing it as necessary.
The current version of the task exports the following products• an XML file containing the pipeline processing request• a tar file per ASDM / MS containing the final flags version• a text file per ASDM / MS containing the final calibration apply list
• a FITS image for each selected calibrator source image• a FITS image for each selected science target source image• a tar file per session containing the caltables for that session• a tar file containing the file web log• a text file containing the final list of CASA commands• an XML “manifest” file listing the products• an XML “aquareport” file listing the QA scores and sub-scores, image sensitivities, and other numerical information
Output:results – The results object for the pipeline task is returned.

3.5.2 Parameter List
parameter namevis

description
List of visibility data files for which flagging and calibration
information will be exported. Defaults to the list maintained in thepipeline context.
Example: vis=[‘X227.ms’, ‘X228.ms’]
continues on next page

3.5. hifa_exportdata

64

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 5 – continued from previous page
parameter namesessionimaging_products_only
exportmses

pprfilecalintents

calimages

targetimages

products_dir

3.5. hifa_exportdata

description
List of sessions one per visibility file. Currently defaults
to a single virtual session containing all the visibility files in vis.
In the future, this will default to the set of observing sessions definedin the context.
Example: session=[‘session1’, ‘session2’]
Export science target imaging products only
Export the final MeasurementSets instead of the final flags,
calibration tables, and calibration instructions.
Name of the pipeline processing request to be exported. Defaults
to a file matching the template ‘PPR_*.xml’.
Example: pprfile=[‘PPR_GRB021004.xml’]
List of calibrator image types to be exported. Defaults toall standard calibrator intents, ‘BANDPASS’, ‘PHASE’, ‘FLUX’.
Example: ‘PHASE’
List of calibrator images to be exported. Defaults to all
calibrator images recorded in the pipeline context.
Example: calimages=[‘3C454.3.bandpass’, ‘3C279.phase’]
List of science target images to be exported. Defaults to all
science target images recorded in the pipeline context.
Example: targetimages=[‘NGC3256.band3’, ‘NGC3256.band6’]
Name of the data products subdirectory. Defaults to ‘./’.
Example: products_dir=’../products’65Pipeline Tasks Reference Manual, Release 2023.1.0

3.5.3 Examples
1. Export the pipeline results for a single session to the data products directory:>>> !mkdir ../products>>> hif_exportdata(products_dir='../products')
2. Export the pipeline results to the data products directory specify that only the gain calibrator images be saved:>>> !mkdir ../products>>> hif_exportdata(products_dir='../products', calintents='*PHASE*')

3.6 hifa_flagdata
3.6.1 Task description
The hifa_flagdata data performs basic flagging operations on a list of measurements including:• applying online flags• applying a flagging template• partial polarization flagging• autocorrelation data flagging• shadowed antenna data flagging• scan-based flagging by intent or scan number• edge channel flagging, as needed• low atmospheric transmission flagging
About the spectral window edge channel flagging:• For TDM spectral windows, a number of edge channels are always flagged, according to the fracspw and fracspwfps parameters (the latter operates only on spectral windows with 62, 124, or 248 channels). With the default
setting of fracspw, the number of channels flagged on each edge is 2, 4, or 8 for 64, 128, or 256-channel spectral
windows, respectively.
• For most FDM spectral windows, no edge flagging is done. The only exceptions are ACA spectral windows thatencroach too close to the baseband edge. Channels that lie closer to the baseband edge than the following values
are flagged: 62.5, 40, 20, 10, and 5 MHz for spectral windows with bandwidths of 1000, 500, 250, 125, and 62.5
MHz, respectively. A warning is generated in the weblog if flagging occurs due to proximity to the basebandedge. By definition, 2000 MHz spectral windows always encroach the baseband edge on both sides of the spectral
window, and thus are always flagged on both sides in order to achieve 1875 MHz bandwidth (in effect, they are
flagged by 62.5 MHz on each side), and thus no warning is generated.
Output:results – The results object for the pipeline task is returned.

3.6. hifa_flagdata66

Pipeline Tasks Reference Manual, Release 2023.1.0

3.6.2 Parameter List
parameter namevis

autocorrshadowtolerancescanscannumber

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets defined in the pipeline context.
Flag autocorrelation data.
Flag shadowed antennas.
Amount of antenna shadowing tolerated, in meters. A positive number
allows antennas to overlap in projection. A negative number forces
antennas
apart in projection. Zero implies a distance of radius_1+radius_2 between
antenna centers.
Flag a list of specified scans.
A string containing a comma delimited list of scans to be
flagged.
Example: scannumber=’3,5,6’

intents

A string containing a comma delimited list of intents against
which the scans to be flagged are matched.
Example: intents=’*BANDPASS*’

edgespw
fracspw
fracspwfps

onlineFlag the edge spectral window channels.
Fraction of channels to flag at both edges of TDM spectral windows.
Fraction of channels to flag at both edges of ACA TDM
spectral windows that were created with the earlier (original)
implementation of the frequency profile synthesis (FPS) algorithm.
Apply the online flags.
continues on next page

3.6. hifa_flagdata67

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 6 – continued from previous page
parameter namepartialpol

lowtrans

mintransnonrepspws

mintransrepspw

fileonline

template
filetemplate

hm_tbuff

tbuff

description
Identify integrations in multi-polarisation data where partof the polarization products are already flagged, and flag the otherpolarization products in those integrations.
Flag spectral windows for which a significant fraction of
the channels have atmospheric transmission below thethreshold (mintransrepspw, mintransnonrepspws).
This atmospheric transmissivity threshold is used to flaga non-representative science spectral window when more than 60% of
its channels have a transmissivity below this level.
This atmospheric transmissivity threshold is used to flag therepresentative science spectral window when more than 60% of its channels
have a transmissivity below this level.
File containing the online flags. These are computed by theh_init or hif_importdata data tasks. If the online flags files
are undefined a name of the form ‘msname.flagonline.txt’ is assumed.
Apply flagging templates
The name of a text file that contains the flagging templatefor RFI, birdies, telluric lines, etc. If the template flags files
is undefined a name of the form ‘msname.flagtemplate.txt’ is assumed.
The heuristic for computing the default time interval padding
parameter. The options are ‘halfint’ and ‘manual’. In ‘halfint’ mode tbuff
is set to half the maximum of the median integration time of the scienceand calibrator target observations. The value of 0.048 seconds is
subtracted from the lower time limit to accommodate the behavior of the
ALMA Control system.
The time in seconds used to pad flagging command timeintervals if hm_tbuff = ‘manual’. The default inmanual mode is no flagging.
continues on next page

3.6. hifa_flagdata

68

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 6 – continued from previous page
parameter nameqa0

description
QA0 flags.

qa2

QA2 flags.

flagbackup

Back up any pre-existing flags.

3.6.3 Examples
1. Do basic flagging on a MeasurementSet:>>> hifa_flagdata()
2. Do basic flagging on a MeasurementSet flagging additional scans selected by number as well:>>> hifa_flagdata(scannumber='13,18')

3.7 hifa_flagtargets
3.7.1 Task description
The hifa_flagtargets task performs basic flagging operations on a list of science target MeasurementSets, including:• applying a flagging template
Output:results – The results object for the pipeline task is returned.

3.7.2 Parameter List
parameter namevis

template

description
The list of input MeasurementSets. Defaults to the listof MeasurementSets defined in the pipeline context.
Apply flagging templates; defaults to True.
continues on next page

3.7. hifa_flagtargets

69

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 7 – continued from previous page
parameter namefiletemplateflagbackup

description
The name of a text file that contains the flagging
template for issues with the science target data etc.
If the template flags files is undefined a name of theform ‘msname_flagtargetstemplate.txt’ is assumed.
Back up any pre-existing flags; defaults to False.

3.7.3 Examples
1. Do basic flagging on a science target MeasurementSet:>>> hifa_flagtargets()

3.8 hifa_fluxcalflag
3.8.1 Task description
Search the built-in solar system flux calibrator line catalog for overlaps with the science spectral windows. Generate alist of line overlap regions and flagging commands.
Output:results – The results object for the pipeline task is returned.

3.8.2 Parameter List
parameter name
vis

field

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets defined in the pipeline context.
The list of field names or field ids for which the models areto be set. Defaults to all fields with intent ‘AMPLITUDE’.
Example: field=’3C279’, field=’3C279, M82’
continues on next page3.8. hifa_fluxcalflag

70

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 8 – continued from previous page
parameter nameintentdescription
A string containing a comma delimited list of intents against
which the selected fields are matched. Defaults to all datawith amplitude intent.
Example: intent=’AMPLITUDE’

spw

Spectral windows and channels for which bandpasses arecomputed. Defaults to all science spectral windows.
Example: spw=’11,13,15,17’

thresholdappendlines
linesfileapplyflags

If the fraction of a spectral window occupied by line regions
is greater than this threshold value, then flag the entire
spectral window.
Append user defined line regions to the line dictionary.
Read in a file containing lines regions and append it to thebuiltin dictionary. Blank lines and comments beginning with #are skipped. The data is contained in 4 whitespace delimited
fields containing the solar system object field name, e.g.
‘Callisto’, the molecular species name, e.g. ‘13CO’, and thestarting and ending frequency in GHz.
Boolean for whether to apply the generated flag commands. (default True)

3.8.3 Examples
1. Locate known lines in any solar system object flux calibrators:>>> hifa_fluxcalflag()

3.8. hifa_fluxcalflag

71

Pipeline Tasks Reference Manual, Release 2023.1.0

3.9 hifa_gaincalsnr
3.9.1 Task description
The gaincal solution signal-to-noise is determined as follows:• For each data set the list of source(s) to use for the per-scan gaincal solution signal-to-noise estimation is compiled
based on the values of the field, intent, and spw parameters.
• Source fluxes are determined for each spw and source combination.
• Fluxes in Jy are derived from the pipeline context.
• Pipeline context fluxes are derived from the online flux calibrator catalog, the ASDM, or the user via the flux.csv
file.
• If no fluxes are available the task terminates.
• Atmospheric calibration and observations scans are determined for each spw and source combination.
• If intent is set to ‘PHASE’ are there are no atmospheric scans associated with the ‘PHASE’ calibrator, ‘TARGET’
atmospheric scans will be used instead.
• If atmospheric scans cannot be associated with any of the spw and source combinations the task terminates.
• Science spws are mapped to atmospheric spws for each science spw and source combinations.
• If mappings cannot be determined for any of the spws the task terminates.
• The median Tsys value for each atmospheric spw and source combination is determined from the SYSCAL table.
Medians are computed first by channel, then by antenna, in order to reduce sensitivity to deviant values.
• The science spw parameters, exposure time(s), and integration time(s) are determined.
• The per scan sensitivity and signal-to-noise estimates are computed per science spectral window. Nominal Tsysand sensitivity values per receiver band provide by the ALMA project are used for this estimate.
• The QA score is based on how many signal-to-noise estimates greater than the requested signal-to-noise ratio
can be computed.
Output:results – The results object for the pipeline task is returned.

3.9.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘M82A.ms’, ‘M82B.ms’]
continues on next page

3.9. hifa_gaincalsnr

72

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namefield

description
The list of field names of sources to be used for signal to noise
estimation. Defaults to all fields with the standard intent.
Example: field=’3C279’

intentA string containing a comma delimited list of intents against which
the selected fields are matched. Defaults to ‘PHASE’.
Example: intent=’BANDPASS’spw

The list of spectral windows and channels for which gain solutions arecomputed. Defaults to all the science spectral windows for which there areboth ‘intent’ and TARGET intents.
Example: spw=’13,15’

phasesnr

The required gaincal solution signal to noise.
Example: phasesnr=20.0

bwedgefrac

The fraction of the bandwidth edges that is flagged.
Example: bwedgefrac=0.0

hm_nantennas

The heuristics for determines the number of antennas to use
in the signal to noise estimate. The options are ‘all’ and ‘unflagged’.
The ‘unflagged’ options is not currently supported.
Example: hm_nantennas=’unflagged’

maxfracflagged

The maximum fraction of an antenna that can be flagged
before it is excluded from the signal to noise estimate.
Example: maxfracflagged=0.80

3.9. hifa_gaincalsnr

73

Pipeline Tasks Reference Manual, Release 2023.1.0

3.9.3 Examples
1. Estimate the per scan gaincal solution sensitivities and signal to noise ratios for all the science spectral windows:>>> hifa_gaincalsnr()

3.10 hifa_gfluxscale
3.10.1 Task description
Derive flux densities for point source transfer calibrators using flux models for reference calibrators.
Flux values are determined by:• computing phase only solutions for all the science spectral windows using the calibrator data selected by thereference and refintent parameters and the transfer and transintent parameters, the value of the phaseupsolint
parameter, and any spw combination determined in hifa_spwphaseup.
• computing complex amplitude only solutions for all the science spectral windows using calibrator data selectedwith reference and refintent parameters and the transfer and transintent parameters, the value of the solint
parameter.
• transferring the flux scale from the reference calibrators to the transfer calibrators using refspwmap for windows
without data in the reference calibrators.
• inserting the computed flux density values into the MODEL_DATA column.
Resolved calibrators are handled via antenna selection either automatically (hm_resolvedcals = ‘automatic’) or manually (hm_resolvedcals = ‘manual’). In the former case, antennas closer to the reference antenna than the uv distance
where visibilities fall to peak_fraction of the peak are used. In manual mode, the antennas specified in antenna areused.
Note that the flux corrected calibration table computed internally is not currently used in later pipeline apply calibration
steps.
Output:results – The results object for the pipeline task is returned.

3.10.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: [‘M32A.ms’, ‘M32B.ms’]
continues on next page

3.10. hifa_gfluxscale74

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 10 – continued from previous page
parameter namereferencedescription
A string containing a comma delimited list of field names
defining the reference calibrators. Defaults to field names withintent ‘*AMP*’.
Example: reference=’M82,3C273’

transfer

A string containing a comma delimited list of field names
defining the transfer calibrators. Defaults to field names withintent ‘*PHASE*’.
Example: transfer=’J1328+041,J1206+30’

refintent

A string containing a comma delimited list of intents
used to select the reference calibrators. Defaults to ‘AMPLITUDE’.
Example: refintent=”, refintent=’AMPLITUDE’

transintent

A string containing a comma delimited list of intents
defining the transfer calibrators. Defaults to‘PHASE,BANDPASS,CHECK,POLARIZATION,POLANGLE,POLLEAKAGE’.
Example: transintent=”, transintent=’PHASE,BANDPASS’

refspwmap

Vector of spectral window ids enabling scaling acrossspectral windows. Defaults to no scaling.
Example: refspwmap=[1,1,3,3] - (4 spws, reference fields in 1 and 3,
transferfields in 0,1,2,3

reffilePath to a file containing flux densities for calibrators.
Setjy will be run for any that have both reference and transfer intents.
Values given in this file will take precedence over MODEL column values
set by previous tasks. By default, the path is set to the CSV file createdby hifa_importdata, consisting of catalogue fluxes extracted from the
ASDM
and / or edited by the user.
example: reffile=”, reffile=’working/flux.csv’continues on next page

3.10. hifa_gfluxscale

75

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 10 – continued from previous page
parameter namephaseupsolint

description
Time solution intervals in CASA syntax for the phase solution.
example: phaseupsolint=’inf’, phaseupsolint=’int’, phaseupsolint=’100sec’

solintTime solution intervals in CASA syntax for the amplitude solution.
example: solint=’inf’, solint=’int’, solint=’100sec’

minsnr

Minimum signal-to-noise ratio for gain calibration solutions.
example: minsnr=1.5, minsnr=0.0

refantA string specifying the reference antenna(s). By default,
this is read from the context.
Example: refant=’DV05’

hm_resolvedcals

antennaHeuristics method for handling resolved calibrators. The
options are ‘automatic’ and ‘manual’. In automatic mode,
antennas closer to the reference antenna than the uv
distance where visibilities fall to peak_fraction of thepeak are used. In manual mode, the antennas specified inantenna are used.
A comma delimited string specifying the antenna names or ids
to be used for the fluxscale determination. Used in
hm_resolvedcals = ‘manual’ mode.
Example: antenna=’DV16,DV07,DA12,DA08’

peak_fraction

3.10. hifa_gfluxscale

The limiting UV distance from the reference antenna forantennas to be included in the flux calibration. Defined as
the point where the calibrator visibilities have fallen topeak_fraction of the peak value.

76

Pipeline Tasks Reference Manual, Release 2023.1.0

3.10.3 Examples
1. Compute flux values for the phase calibrator using model data from the amplitude calibrator:>>> hifa_gfluxscale()

3.11 hifa_gfluxscaleflag
3.11.1 Task description
This task computes the flagging heuristics on the phase calibrator and flux calibrator by calling hif_correctedampflag
which looks for outlier visibility points by statistically examining the scalar difference of corrected amplitudes minus
model amplitudes, and flags those outliers. The philosophy is that only outlier data points that have remained outliers
after calibration will be flagged. The heuristic works equally well on resolved calibrators and point sources because it
is not performing a vector difference, and thus is not sensitive to nulls in the flux density vs. uvdistance domain. Notethat the phase of the data is not assessed.
In further detail, the workflow is as follows: a snapshot of the flagging state is preserved at the start, a preliminary
phase and amplitude gaincal solution is solved and applied, the flagging heuristics are run and any outliers are marked
for flagging, the flagging state is restored from the snapshot. If any outliers were found, then these are flagged. Plotsare generated at two points in this workflow: after preliminary phase and amplitude calibration but before flagging
heuristics are run, and after flagging heuristics have been run and applied. If no points were flagged, the ‘after’ plots
are not generated or displayed. The score for this stage is the standard data flagging score, which depends on the fractionof data flagged.
Output:results – The results object for the pipeline task is returned.

3.11.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘M51.ms’]

intentA string containing a comma delimited list of intents against
which the selected fields are matched. If undefined (default), it
will select all data with the AMPLITUDE, PHASE, and CHECK intents,
except for one case: if one of the AMPLITUDE intent fields was also
used for BANDPASS, then this task will select only data with PHASE
and CHECK intents.
Example: intent=’*PHASE*’
continues on next page

3.11. hifa_gfluxscaleflag

77

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 11 – continued from previous page
parameter namephaseupsolint

description
The phase correction solution interval in CASA syntax.
Example: phaseupsolint=’300s’

solintTime and channel solution intervals in CASA syntax.
Example: solint=’inf,10ch’, solint=’inf’

minsnrrefantSolutions below this SNR are rejected.
Reference antenna names. Defaults to the value(s) stored in thepipeline context. If undefined in the pipeline context defaults to
the CASA reference antenna naming scheme.
Example: refant=’DV01’, refant=’DV06,DV07’

antnegsig

antpossig

tmantint

tmint

tmbl

Lower sigma threshold for identifying outliers as a result of
bad antennas within individual timestamps.
Example: antnegsig=4.0
Upper sigma threshold for identifying outliers as a result of
bad antennas within individual timestamps.
Example: antpossig=4.6
Threshold for maximum fraction of timestamps that are allowed tocontain outliers.
Example: tmantint=0.063
Threshold for maximum fraction of “outlier timestamps” over“total timestamps” that a baseline may be a part of.
Example: tmint=0.085
Initial threshold for maximum fraction of “bad baselines” over “allbaselines” that an antenna may be a part of.
Example: tmbl=0.175
continues on next page

3.11. hifa_gfluxscaleflag

78

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 11 – continued from previous page
parameter nameantblnegsig

antblpossig

relaxed_factor

niter

description
Lower sigma threshold for identifying outliers as a result of“bad baselines” and/or “bad antennas” within baselines, across alltimestamps.
Example: antblnegsig=3.4
Threshold for identifying outliers as a result of“bad baselines” and/or “bad antennas” within baselines, across alltimestamps.
Example: antblpossig=3.2
Relaxed value to set the threshold scaling factor to undercertain conditions (see task description).
Example: relaxed_factor=2.0
Maximum number of times to iterate on evaluation of flagging
heuristics. If an iteration results in no new flags, then subsequent
iterations are skipped.
Example: niter=2

3.11.3 Examples
1. run with recommended settings to create flux scale calibration with flagging using recommended thresholds:>>> hifa_gfluxscaleflag()3.12 hifa_imageprecheck3.12.1 Task description
In this task, the representative source and the spw containing the representative frequency selected by the PI in the
OT are used to calculate the synthesized beam and to make sensitivity estimates for the aggregate bandwidth andrepresentative bandwidth for several values of the Briggs robust parameter. This information is reported in a table in
the weblog. If no representative target/frequency information is available, it defaults to the first target and center of first
spw in the data (e.g. pre-Cycle 5 data does not have this information available). The best Briggs robust parameter toachieve the PI’s desired angular resolution is chosen automatically. See the User’s guide for further details.
Output:results – The results object for the pipeline task is returned.

3.12. hifa_imageprecheck

79

Pipeline Tasks Reference Manual, Release 2023.1.0

3.12.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context.
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

calcsbparallelForce (re-)calculation of sensitivities and beams; defaults toFalse.
Use MPI cluster where possible

3.12.3 Examples
1. run with recommended settings to perform checks prior to imaging:>>> hifa_imageprecheck()
2. run to perform checks prior to imaging and force the re-calculation of sensitivities and beams:>>> hifa_imageprecheck(calcsb=True)

3.13 hifa_importdata
3.13.1 Task description
The hifa_importdata task loads the specified visibility data into the pipeline context unpacking and / or converting it asnecessary.
If the overwrite input parameter is set to False and the task is asked to convert an input ASDM input to an MS, thenwhen the output MS already exists in the output directory, the importasdm conversion step is skipped, and the existing
MS will be imported instead.
Output:results – The results object for the pipeline task is returned.

3.13. hifa_importdata

80

Pipeline Tasks Reference Manual, Release 2023.1.0

3.13.2 Parameter List
parameter namevis

sessionasis

process_caldevice
overwrite

nocopybdfflags

description
List of visibility data files. These may be ASDMs, tar
files of ASDMs, MSes, or tar files of MSes. If ASDM files
are specified, they will be converted to MS format.
Example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
List of session names, one for each visibility dataset,
used to group the MSes into sessions.
Example: session=[‘session_1’, ‘session_2’]
Creates verbatim copies of the ASDM tables in the output MS.
The value given to this option must be a list of table names
separated by space characters.
Import the caldevice table from the ASDM.
Overwrite existing files on import; defaults to False.
When converting ASDM to MS, if overwrite=False and the MS
already exists in the output directory, then this existing
MS dataset will be used instead.
Example: overwrite=True
Disable copying of MS to working directory; defaults toFalse.
Example: nocopy=True
Apply BDF flags on import.
continues on next page

3.13. hifa_importdata

81

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 13 – continued from previous page
parameter namedatacolumns

description
Dictionary defining the data types of existing columns.
The format is:{‘data’: ‘data type 1’}or{‘data’: ‘data type 1’, ‘corrected’: ‘data type 2’}
For ASDMs the data type can only be RAW and one
can only specify it for the data column.
For MSes one can define two different data types
for the DATA and CORRECTED_DATA columns and they
can be any of the known data types (RAW,REGCAL_CONTLINE_ALL, REGCAL_CONTLINE_SCIENCE,SELFCAL_CONTLINE_SCIENCE, REGCAL_LINE_SCIENCE,SELFCAL_LINE_SCIENCE, BASELINED, ATMCORR). The
intent selection strings _ALL or _SCIENCE can beskipped. In that case the task determines thisautomatically by inspecting the existing intents
in the dataset.
Usually, a single datacolumns dictionary is used
for all datasets. If necessary, one can define alist of dictionaries, one for each EB, withdifferent setups per EB.
If no types are specified,{‘data’:’raw’,’corrected’:’regcal_contline’}
or {‘data’:’raw’} will be assumed, depending onwhether the corrected column exists or not.

lazy
dbserviceocorr_mode
createmms

Use the lazy filler import.
Use the online flux catalog.
ALMA default set to ca.
Create an MMS.
continues on next page

3.13. hifa_importdata82

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 13 – continued from previous page
parameter nameminparang

description
Minimum required parallactic angle range for polarisation
calibrator, in degrees. The default of 0.0 is used for
non-polarisation processing.

3.13.3 Examples
1. Load an ASDM list in the ../rawdata subdirectory into the context:>>> hifa_importdata(vis=['../rawdata/uid___A002_X30a93d_X43e', '../rawdata/uid_A002_
˓→x30a93d_X44e'])2. Load an MS in the current directory into the context:>>> hifa_importdata(vis=['uid___A002_X30a93d_X43e.ms'])
3. Load a tarred ASDM in ../rawdata into the context:>>> hifa_importdata(vis=['../rawdata/uid___A002_X30a93d_X43e.tar.gz'])
4. Import a list of MeasurementSets:>>> myvislist = ['uid___A002_X30a93d_X43e.ms', 'uid_A002_x30a93d_X44e.ms']>>> hifa_importdata(vis=myvislist)
5. Run with explicit setting of data column types:>>> hifa_importdata(vis=['uid___A002_X30a93d_X43e_targets.ms'], datacolumns={'data': 'regcal_
˓→contline'})>>> hifa_importdata(vis=['uid___A002_X30a93d_X43e_targets_line.ms'], datacolumns={'data':
˓→'regcal_line', 'corrected': 'selfcal_line'})3.14 hifa_lock_refant
3.14.1 Task description
hifa_lock_refant marks the reference antenna list as “locked” for specified measurement sets, preventing modificationof the refant list by subsequent tasks.
After executing hifa_lock_refant, all subsequent gaincal calls will by default be executed with refantmode=’strict’.
The refant list can be unlocked with the hifa_unlock_refant task.
Output:results – The results object for the pipeline task is returned.

3.14. hifa_lock_refant83

Pipeline Tasks Reference Manual, Release 2023.1.0

3.14.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

3.14.3 Examples
1. Lock the refant list for all MSes in pipeline context:>>> hifa_lock_refant()

3.15 hifa_polcal
3.15.1 Task description
Derive the instrumental polarization calibrations for ALMA using the polarization calibrators.
Output:results – The results object for the pipeline task is returned.

3.15.2 Parameter List
parameter namevis

solint_chavg

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: [‘M32A.ms’, ‘M32B.ms’]
Channel averaging to include in solint for gaincal steps
producing cross-hand delay, cross-hand phase, and leakage(D-terms) solutions.
Default: ‘5MHz’continues on next page

3.15. hifa_polcal

84

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 15 – continued from previous page
parameter namevs_stats

vs_thresh

description
List of visstat statistics to use for diagnostic comparison
between the concatenated session MS and individual MSes inthat session after applying polarization calibration tables.
Default: [‘min’,’max’,’mean’]Threshold to use in diagnostic comparison of visstat
statistics; relative differences larger than this threshold
are reported in the CASA log.
Default: 1e-3

3.15.3 Examples
1. Compute the polarization calibrations:>>> hifa_polcal()

3.16 hifa_polcalflag
3.16.1 Task description
This task flags corrected visibility outliers in the polarization calibrator data using the hif_correctedampflag heuristics.
Output:results – The results object for the pipeline task is returned.

3.16.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

3.16. hifa_polcalflag

85

Pipeline Tasks Reference Manual, Release 2023.1.0

3.16.3 Examples
1. Run with recommended settings to flag visibility outliers in the polarization calibrator data:>>> hifa_polcalflag()3.17 hifa_renorm
3.17.1 Task description
This task makes an assessment, and optionally applies a correction, to data suffering from incorrect amplitude normalization caused by bright astronomical lines detected in the autocorrelations of some target sources.
For a full description of the effects of bright emission lines and the correction heuristics used in this task, please see
the Pipeline User Guide.
Output:results – The results object for the pipeline task is returned.

3.17.2 Parameter List
parameter name
vis

apply

thresholdcorrectATM

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]
Boolean to select whether to apply the renormalization
correction (True), or only run the assessment (False,
default).
Example: apply=True
Apply correction if max correction is above this thresholdvalue and apply = True. Default is 1.02 (i.e. 2%).
Example: threshold=1.02
Use the ATM model transmission profiles to try and correct
for any ATM residual features that get into the scaling
spectra.
continues on next page

3.17. hifa_renorm

86

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 17 – continued from previous page
parameter namespw

excludechan

atm_auto_exclude

bwthreshspw

3.17. hifa_renorm

description
The list of real (not virtual - i.e. the actual spwIDs in
the MS) spectral windows to evaluate.
Set to spw=” by default, which means the task will selectall relevant (science FDM) spectral windows.
Note that for data with multiple MSs, a list with thecorrect spectral window selection for each MS can beprovided.
Examples:spw=”11,13,15,17”
spw=[“11,13,15,17”, “5,7,11,13”]
Channels to exclude in either channel or frequency space(TOPO, GHz), specifying the real (not virtual) spectral
window per selection.
Note that for data with multiple MSs, a list of
dictionaries with the correct selection for each MS can beprovided.
Examples:excludechan={‘22’:’100~150;800~850’, ‘24’:’100~200’}
excludechan={‘22’:’230.1GHz~230.2GHz’}
excludechan=[{‘22’:’100~150’}, {‘15’:’100~150’}]
Automatically find and exclude regions with atmosphericfeatures. Default is False
Bandwidth beyond which a SPW is split into chunks to fit
separately. The default value for all SPWs is 120e6, andthis parameter allows one to override it for specific SPWs,
due to needing potentially various ‘nsegments’ when EBs
have very different SPW bandwidths.
Example: bwthreshspw={‘16: 64e6, ‘22: 64e6}

87Pipeline Tasks Reference Manual, Release 2023.1.0

3.17.3 Examples
1. Run with recommended settings to assess the need for an ALMA amplitude renormalization correction.

>>> hifa_renorm()
2. Run to assess the necessary ALMA amplitude renormalization correction, and apply this correction if it exceeds athreshold of 3% (1.03).

>>> hifa_renorm(apply=True, threshold=1.03)

3.18 hifa_restoredata
3.18.1 Task description
The hifa_restoredata task restores flagged and calibrated MeasurementSets from archived ASDMs and pipeline flagging
and calibration date products.
The hifa_restoredata restores flagged and calibrated data from archived ASDMs and pipeline flagging and calibration
data products. Pending archive retrieval support hifa_restoredata assumes that the required products are available in
the rawdata_dir in the format produced by the hifa_exportdata task.
hifa_restoredata assumes that the following entities are available in the raw data directory:• the ASDMs to be restored• for each ASDM in the input list:– a compressed tar file of the final flagversions file, e.g. uid___A002_X30a93d_X43e.ms.flagversions.tar.gz– a text file containing the applycal instructions, e.g. uid___A002_X30a93d_X43e.ms.calapply.txt– a compressed tar file containing the
uid___A001_X74_X29.session_3.caltables.tar.gz

caltables

for

the

parentsession,e.g.

hifa_restoredata performs the following operations:• imports the ASDM(s)• removes the default MS.flagversions directory created by the filler• restores the final MS.flagversions directory stored by the pipeline• restores the final set of pipeline flags to the MS• restores the final calibration state of the MS• restores the final calibration tables for each MS• applies the calibration tables to each MS
When importing the ASDM and converting it to a Measurement Set (MS), if the output MS already exists in the output
directory, then the importasdm conversion step is skipped, and instead the existing MS will be imported.
Output:results – The results object for the pipeline task is returned.

3.18. hifa_restoredata88

Pipeline Tasks Reference Manual, Release 2023.1.0

3.18.2 Parameter List
parameter namevis

sessiondescription
List of raw visibility data files to be restored.
Assumed to be in the directory specified by rawdata_dir.
Example: vis=[‘uid___A002_X30a93d_X43e’]
List of sessions one per visibility file.
Example: session=[‘session_3’]

products_dir

Name of the data products directory to copy calibration
products from. The parameter is effective only when
copytoraw = True. When copytoraw = False, calibration
products in rawdata_dir will be used.
Example: products_dir=’myproductspath’

copytoraw

Copy calibration and flagging tables from products_dir to
rawdata_dir directory.
Example: copytoraw=False

rawdata_dir

Name of the rawdata subdirectory.
Example: rawdata_dir=’myrawdatapath’lazy

Use the lazy filler option.
Example: lazy=Truebdfflags

Set the BDF flags.
Example: bdfflags=False

ocorr_mode

Set ocorr_mode.
Example: ocorr_mode=’ca’
continues on next page

3.18. hifa_restoredata

89

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 18 – continued from previous page
parameter nameasis

description
Creates verbatim copies of the ASDM tables in the output MS.
The value given to this option must be a string containing alist of table names separated by whitespace characters.
Example: asis=’Source Receiver’

3.18.3 Examples
1. Restore the pipeline results for a single ASDM in a single session:>>> hifa_restoredata(vis=['uid___A002_X30a93d_X43e'], session=['session_1'], ocorr_mode='ca')

3.19 hifa_session_refant
3.19.1 Task description
This task re-evaluates the reference antenna lists from all measurement sets within a session and combines these to
select a single common reference antenna (per session) that is to be used by any subsequent pipeline stages.
Output:results – The results object for the pipeline task is returned.

3.19.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

phase_threshold

Threshold (in degrees) used to identify absolute phasesolution outliers in caltables.
Example: phase_threshold=0.005

3.19. hifa_session_refant90Pipeline Tasks Reference Manual, Release 2023.1.0

3.19.3 Examples
1. Compute a single common reference antenna per session:>>> hifa_session_refant()

3.20 hifa_spwphaseup
3.20.1 Task description
The spw map for phase calibration is computed. Phase offsets as a function of spectral window are computed using
high signal-to-noise calibration observations.
Previous calibrations are applied on the fly.
hifa_spwphaseup performs two functions:• Determines the spectral window mapping or combination mode, for each phase and check source, to use whensolving for phase as a function of time (gfluxscale and timegaincal), and when applying those solutions to targets.
• Computes the per-spectral-window phase offset table that will be applied to the data to remove mean phasedifferences between the spectral windows.
If hm_spwmapmode = ‘auto’, then the spectral window map is computed for each SpectralSpec and each source withphase or check intent, using the following algorithm:• Estimate the per-spectral-window (spw) per-scan signal-to-noise ratio for each phase and check source based on
catalog flux densities, Tsys, number of antennas, and integration scan time.
• If the signal-to-noise of all spws is greater than phasesnr, then hm_spwmapmode = ‘default’ mapping is used
in which each spw is used to calibrate itself.
• If the signal-to-noise of only some spws are greater than the value of phasesnr, then each lower-SNR spw ismapped to the highest SNR spw in the same SpectralSpec.
• If all spws have low SNR, or SNR cannot be computed for any reason (for example, there is no flux information),then hm_spwmapmode = ‘combine’.
If hm_spwmapmode = ‘combine’, hifa_spwphaseup maps all the science windows to a single science spectral window.
For example, if the list of science spectral windows is [9, 11, 13, 15] then all the science spectral windows in the data
will be combined and mapped to the science window 9 in the combined phase vs time calibration table.
If hm_spwmapmode = ‘simple’, a mapping from narrow science to wider science spectral windows is computed using
the following algorithm:• Construct a list of the bandwidths of all the science spectral windows.
• Determine the maximum bandwidth in this list as ‘maxbandwidth’• For each science spectral window with bandwidth less than ‘maxbandwidth’ construct a list of spectral windows
with bandwidths greater than minfracmaxbw * ‘maxbandwidth’, then select the spectral window in this list
whose band center most closely matches the band center of the narrow spectral window, and preferentially matchwithin the same baseband if samebb = True.
If hm_spwmapmode = ‘default’ the spw mapping is assumed to be one to one.
Phase offsets per spectral window are determined by computing a phase only gain calibration on the selected data,normally the high signal-to-noise bandpass calibrator observations, using the solution interval ‘inf’.

3.20. hifa_spwphaseup

91

Pipeline Tasks Reference Manual, Release 2023.1.0

At the end of the task the spectral window map and the phase offset calibration table in the pipeline are stored in thecontext for use by later tasks.
Finally, the SNR of the calibration solutions are inspected and if the median value on a per-spw basis does not reachspecific thresholds, then issue a warning and reduced QA score, with thresholds at phasesnr *0.75 (blue), *0.5 (yellow)
and *0.33 (red).
Output:results – The results object for the pipeline task is returned.

3.20.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘M82A.ms’, ‘M82B.ms’]

caltable

The list of output calibration tables. Defaults to the standard
pipeline naming convention.
Example: caltable=[‘M82.gcal’, ‘M82B.gcal’]field

The list of field names or field ids for which phase offset solutions
are to be computed. Defaults to all fields with the default intent.
Example: field=’3C279’, field=’3C279, M82’

intentA string containing a comma delimited list of intents against
which the selected fields are matched. Defaults to the BANDPASS
observations.
Example: intent=’PHASE’spw

The list of spectral windows and channels for which gain solutions arecomputed. Defaults to all the science spectral windows.
Example: spw=’13,15’
continues on next page

3.20. hifa_spwphaseup

92

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 20 – continued from previous page
parameter namehm_spwmapmodedescription
The spectral window mapping mode. The options are: ‘auto’,‘combine’, ‘simple’, and ‘default’. In ‘auto’ mode hifa_spwphaseup
estimates the SNR of the phase calibrator observations and uses theseestimates to choose between ‘combine’ mode (low SNR) and ‘default’mode(high SNR). In combine mode all spectral windows are combined andmapped to
one spectral window. In ‘simple’ mode narrow spectral windows are
mapped towider ones using an algorithm defined by ‘maxnarrowbw’,‘minfracmaxbw’, and‘samebb’. In ‘default’ mode the spectral window map defaults to thestandard one to one mapping.
Example: hm_spwmapmode=’combine’

maxnarrowbw

The maximum bandwidth defining narrow spectral windows. Values
must be in CASA compatible frequency units.
Example: maxnarrowbw=”

minfracmaxbw

The minimum fraction of the maximum bandwidth in the set ofspws to use for matching.
Example: minfracmaxbw=0.75

samebbMatch within the same baseband if possible.
Example: samebb=False

phasesnr

The required gaincal solution signal-to-noise.
Example: phaseupsnr=20.0

bwedgefrac

The fraction of the bandwidth edges that is flagged.
Example: bwedgefrac=0.0
continues on next page

3.20. hifa_spwphaseup

93

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 20 – continued from previous page
parameter namehm_nantennas

description
The heuristics for determines the number of antennas to use
in the signal-to-noise estimate. The options are ‘all’ and ‘unflagged’.
The ‘unflagged’ options is not currently supported.
Example: hm_nantennas=’unflagged’

maxfracflagged

The maximum fraction of an antenna that can be flagged
before it is excluded from the signal-to-noise estimate.
Example: maxfracflagged=0.80

combine

Data axes to combine for solving. Options are ‘’, ‘scan’, ‘spw’,‘field’ or any comma-separated combination.
Example: combine=”

refantReference antenna name(s) in priority order. Defaults to most recent
values set in the pipeline context. If no reference antenna is defined in
the pipeline context the CASA defaults are used.
Example: refant=’DV01’, refant=’DV05,DV07’

minblperant

Minimum number of baselines required per antenna for each solve.
Antennas with fewer baselines are excluded from solutions.
Example: minblperant=2

minsnrunregister_existing

3.20. hifa_spwphaseup

Solutions below this SNR are rejected.
Unregister previous spwphaseup calibrations from the pipeline context
before registering the new calibrations from this task.

94

Pipeline Tasks Reference Manual, Release 2023.1.0

3.20.3 Examples
1. Compute the default spectral window map and the per spectral window phase offsets:>>> hifa_spwphaseup()
2. Compute the default spectral window map and the per spectral window phase offsets set the spectral window mapping
mode to ‘simple’:>>> hifa_spwphaseup(hm_spwmapmode='simple')

3.21 hifa_targetflag
3.21.1 Task description
This task flags very obvious target source outliers. The calibration tables and flags accumulated in the cal library up
to this point are pre-applied, then hif_correctedampflag is called for just the TARGET intent. Any resulting flags areapplied and the calibration library is restored to the state before calling this task.
Because science targets are generally not point sources, the flagging algorithm needs to be more clever than for point
source calibrators. The algorithm identifies outliers by examining statistics within successive overlapping radial uv
bins, allowing it to adapt to an arbitrary uv structure. Outliers must appear to be a potential outlier in two bins in order
to be declared an outlier. To further avoid overflagging of good data, only the highest threshold levels are used (+12/-13
sigma). This stage does can add significant processing time, particularly in making the plots. So to save time, the amp
vs. time plots are created only if flags are generated, and the amp vs. uv distance plots are made for only those spws
that generated flags. Also, to avoid confusion in mosaics and single field surveys, the amp vs. uv distance plots only
show field IDs with new flags.
Output:results – The results object for the pipeline task is returned.

3.21.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]

3.21. hifa_targetflag

95

Pipeline Tasks Reference Manual, Release 2023.1.0

3.21.3 Examples
1. Run with recommended settings to flag outliers in science target(s):>>> hifa_targetflag()3.22 hifa_timegaincal
3.22.1 Task description
The time-dependent complex gains for each antenna/spwid are determined from the raw data (DATA column) divided
by the model (MODEL column), for the specified fields. The gains are computed according to the spw-combination
model determined in hifa_spwphaseup.
Previous calibrations are applied on the fly.
The complex gains are derived from the data column (raw data) divided by the model column (usually set with hif_setjy).
The gains are obtained for the specified solution intervals, spw combination and field combination. One gain solution
is computed for the science targets and one for the calibrator targets.
Good candidate reference antennas can be determined using the hif_refant task.
Previous calibrations that have been stored in the pipeline context are applied on the fly. Users can interact with thesecalibrations via the h_export_calstate and h_import_calstate tasks.
Output:results – The results object for the pipeline task is returned.

3.22.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘M82A.ms’, ‘M82B.ms’]calamptable

The list of output diagnostic calibration amplitude tables forthe calibration targets. Defaults to the standard pipeline naming
convention.
Example: calamptable=[‘M82.gacal’, ‘M82B.gacal’]continues on next page

3.22. hifa_timegaincal

96

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 22 – continued from previous page
parameter nameoffsetstable

description
The list of output diagnostic phase offset tables for thecalibration targets. Defaults to the standard pipeline naming convention.
Example: offsetstable=[‘M82.offsets.gacal’, ‘M82B.offsets.gacal’]calphasetable

The list of output calibration phase tables for thecalibration targets. Defaults to the standard pipeline naming convention.
Example: calphasetable=[‘M82.gpcal’, ‘M82B.gpcal’]targetphasetable

The list of output phase calibration tables for the sciencetargets. Defaults to the standard pipeline naming convention.
Example: targetphasetable=[‘M82.gpcal’, ‘M82B.gpcal’]amptable

The list of output calibration amplitude tables for thecalibration and science targets.
Defaults to the standard pipeline naming convention.
Example: amptable=[‘M82.gacal’, ‘M82B.gacal’]field

The list of field names or field ids for which gain solutions are to
be computed. Defaults to all fields with the standard intent.
Example: field=’3C279’, field=’3C279, M82’

spw

The list of spectral windows and channels for which gain solutions arecomputed. Defaults to all science spectral windows.
Example: spw=’11’, spw=’11, 13’antennacalsolint

The selection of antennas for which gains are computed. Defaults to all.
Time solution interval in CASA syntax for calibrator source
solutions.
Example: calsolint=’inf’, calsolint=’int’, calsolint=’100sec’
continues on next page

3.22. hifa_timegaincal

97

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 22 – continued from previous page
parameter nametargetsolint

description
Time solution interval in CASA syntax for target source
solutions.
Example: targetsolint=’inf’, targetsolint=’int’, targetsolint=’100sec’refantReference antenna name(s) in priority order. Defaults to most recent
values set in the pipeline context. If no reference antenna is defined in
the pipeline context use the CASA defaults.
example: refant=’DV01’, refant=’DV05,DV07’

refantmodeControls how the refant is applied. Currently available
choices are ‘flex’, ‘strict’, and the default value of ‘’.
Setting to ‘’ allows the pipeline to select the appropriate
mode based on the state of the reference antenna list.
Examples: refantmode=’strict’, refantmode=”

solnorm
minblperant

Normalise the gain solutions.
Minimum number of baselines required per antenna for each solve.
Antennas with fewer baselines are excluded from solutions.
Example: minblperant=2

calminsnr
targetminsnr

smodelSolutions below this SNR are rejected for calibrator solutions.
Solutions below this SNR are rejected for science target
solutions.
Point source Stokes parameters for source model (experimental)
Defaults to using standard MODEL_DATA column data.
Example: smodel=[1,0,0,0] - (I=1, unpolarized)

3.22. hifa_timegaincal98

Pipeline Tasks Reference Manual, Release 2023.1.0

3.22.3 Examples
1. Compute standard per scan gain solutions that will be used to calibrate the target:>>> hifa_timegaincal()3.23 hifa_tsysflag
3.23.1 Task description
This task flags all deviant system temperature measurements in the system temperature calibration table by running asequence of flagging tests, each designed to look for a different type of possible error.
If a file with manual Tsys flags is provided with the filetemplate parameter, then these flags are applied prior to theevaluation of the flagging heuristics listed below.
The tests are:
1. Flag Tsys spectra with high median values
2. Flag Tsys spectra with high median derivatives. This is meant to spot spectra that are ‘ringing’.
3. Flag the edge channels of the Tsys spectra in each SpW.
4. Flag Tsys spectra whose shape is different from that associated with the BANDPASS intent.
5. Flag ‘birdies’.
6. Flag the Tsys spectra of all antennas in a timestamp and spw if proportion of antennas already flagged in thistimestamp and spw exceeds a threshold, and flag Tsys spectra for all antennas and all timestamps in a spw, if
proportion of antennas that are already entirely flagged in all timestamps exceeds a threshold.
Output:results – The results object for the pipeline task is returned.

3.23.2 Parameter List
parameter namevis
caltable

description
List of input MeasurementSets (Not used).
List of input Tsys calibration tables.
Default: [] - Use the table currently stored in the pipeline context.
Example: caltable=[‘X132.ms.tsys.s2.tbl’]

flag_nmedian

True to flag Tsys spectra with high median value.
continues on next page

3.23. hifa_tsysflag

99

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 23 – continued from previous page
parameter namefnm_limit

fnm_byfield
flag_derivative
fd_max_limit

flag_edgechansfe_edge_limit

flag_fieldshape

ff_refintent

ff_max_limit

flag_birdies
fb_sharps_limit

flag_toomany

description
Flag spectra with median value higher than fnm_limit * medianof this measure over all spectra.
Evaluate the nmedian metric separately for each field.
True to flag Tsys spectra with high median derivative.
Flag spectra with median derivative higher than
fd_max_limit * median of this measure over all spectra.
True to flag edges of Tsys spectra.
Flag channels whose channel to channel difference >fe_edge_limit * median across spectrum.
True to flag Tsys spectra with a radically different
shape to those of the ff_refintent.
Data intent that provides the reference shape for‘flag_fieldshape’.
Flag Tsys spectra with ‘fieldshape’ metric values >ff_max_limit.
True to flag channels covering sharp spectral features.
Flag channels bracketing a channel to channel
difference > fb_sharps_limit.
True to flag Tsys spectra for which a proportion ofantennas for given timestamp and/or proportion of antennas that are
entirely flagged in all timestamps exceeds their respective thresholds.
continues on next page

3.23. hifa_tsysflag

100

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 23 – continued from previous page
parameter nametmf1_limit

tmef1_limit

metric_ordernormalize_tsys

filetemplatedescription
Flag Tsys spectra for all antennas in a timestamp and spw if
proportion of antennas already flagged in this timestamp and spw exceeds
tmf1_limit.
Flag Tsys spectra for all antennas and all timestamps
in a spw, if proportion of antennas that are already entirely flagged
in all timestamps exceeds tmef1_limit.
Order in which to evaluate the flagging metrics that are
enabled. Disabled metrics are skipped.
True to create a normalized Tsys table that is used toevaluate the Tsys flagging metrics. All newly found flags are also applied
to the original Tsys caltable that continues to be used for subsequentcalibration.
The name of a text file that contains the manual Tsys flagging
template. If the template flags file is undefined, a name of the form
‘msname.flagtsystemplate.txt’ is assumed.

3.23.3 Examples
1. Flag Tsys measurements using currently recommended tests:>>> hifa_tsysflag()
2. Flag Tsys measurements using all recommended tests apart from that using the ‘fieldshape’ metric:>>> hifa_tsysflag(flag_fieldshape=False)

3.24 hifa_unlock_refant
3.24.1 Task description
hifa_unlock_refant marks the reference antenna list as “unlocked” for specified measurement sets, allowing the list to
be modified by subsequent tasks.
After executing hifa_unlock_refant, all subsequent gaincal calls will by default be executed with refantmode=’flex’.
The refant list can be locked with the hifa_lock_refant task.
Output:3.24. hifa_unlock_refant101

Pipeline Tasks Reference Manual, Release 2023.1.0

results – The results object for the pipeline task is returned.

3.24.2 Parameter List
parameter namevis

description
List of input MeasurementSets. Defaults to the list of
MeasurementSets specified in the pipeline context.
Example: vis=[‘ngc5921.ms’]

3.24.3 Examples
1. Unlock the refant list for all MSes in pipeline context:>>> hifa_unlock_refant()

3.25 hifa_wvrgcal
3.25.1 Task description
Generate a gain table based on the Water Vapor Radiometer data in each vis file. By applying the wvr calibration to
the data specified by qa_intent and qa_spw, calculate a QA score to indicate its effect on interferometric data; ascore > 1 implies that the phase noise is improved, a score < 1 implies that it is made worse. If the score is less than
accept_threshold then the wvr gain table is not accepted into the context for subsequent use.
Output:results – The results object for the pipeline task is returned.

3.25.2 Parameter List
parameter namevis

description
List of input visibility files.
Default: none, in which case the vis files to be usedwill be read from the context.
Example: vis=[‘ngc5921.ms’]
continues on next page

3.25. hifa_wvrgcal

102

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 25 – continued from previous page
parameter namecaltable

description
List of output gain calibration tables.
Default: none, in which case the names of the caltables
will be generated automatically.
Example: caltable=’ngc5921.wvr’offsetstable

List of input temperature offsets table files to subtractfrom WVR measurements before calculating phase corrections.
Default: none, in which case no offsets are applied.
Example: offsetstable=[‘ngc5921.cloud_offsets’]hm_toffset

toffsetsegsource

sourceflag

If ‘manual’, set the toffset parameter to the user-specified value.
If ‘automatic’, set the toffset parameter according to thedate of the MeasurementSet; toffset=-1 if before 2013-01-21T00:00:00
toffset=0 otherwise.
Time offset (sec) between interferometric and WVR data.
If True calculate new atmospheric phase correction coefficientsfor each source, subject to the constraints of the tie parameter.
‘segsource’ is forced to be True if the tie parameter is set to a
non-empty value by the user or by the automatic heuristic.
Flag the WVR data for these source(s) as bad and do not
produce corrections for it. Requires segsource = True.
Example: [‘3C273’]

hm_tieIf ‘manual’, set the tie parameter to theuser-specified value. If ‘automatic’, set the tie
parameter to include with the target all calibratorsthat are within 15 degrees of it: if no calibrators arethat close then tie is left empty.
continues on next page

3.25. hifa_wvrgcal

103

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 25 – continued from previous page
parameter name
tie

nsol

disperse
wvrflag

description
Use the same atmospheric phase correction coefficients
when calculating the WVR correction for all sources inthe tie. If tie is not empty then segsource
is forced to be True. Ignored unless hm_tie = ‘manual’.
Example: tie=[‘3C273,NGC253’, ‘IC433,3C279’]
Number of solutions for phase correction coefficients during thisobservation, evenly distributed in time throughout the observation. It
is used only if segsource=False because if segsource=True then thecoefficients are recomputed whenever the telescope moves to a newsource (within the limits imposed by ‘tie’).
Apply correction for dispersion. (Deprecated; will be removed)
Flag the WVR data for the listed antennas as bad and replace
their data with values interpolated from the 3 nearest antennas withunflagged data.
Example: [‘DV03’,’DA05’,’PM02’]

hm_smoothsmoothscale
maxdistm

If ‘manual’ set the smooth parameter to the user-specified value.
If ‘automatic’, run the wvrgcal task with the range of smooth parameters
required to match the integration time of the wvr data to that of theinterferometric data in each spectral window.
Smooth WVR data on this timescale before calculating the correction.
Ignored unless hm_smooth=’manual’.
Scale the entire phase correction by this factor.
Maximum distance in meters of an antenna used for interpolationfrom a flagged antenna.
Default: -1 (automatically set to 100m if >50% ofantennas are 7m antennas without WVR and otherwise set to500m).
Example: maxdistm=550
continues on next page

3.25. hifa_wvrgcal

104

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 25 – continued from previous page
parameter nameminnumants

description
Minimum number of nearby antennas (up to 3) used forinterpolation from a flagged antenna.
Example: minnumants=3

mingoodfrac
refantMinimum fraction of good data per antenna.
Ranked comma delimited list of reference antennas.
Example: refant=’DV01,DV02’

qa_intent

The list of data intents on which the wvr correction is to betried as a means of estimating its effectiveness.
A QA ‘view’ will be calculated for each specified intent, in each spectral
window in each vis file.
Each QA ‘view’ will consist of a pair of 2-d images with dimensions[‘ANTENNA’, ‘TIME’], one showing the data phase-noise before the
wvr application, the second showing the phase noise after (both ‘before’and ‘after’ images have a bandpass calibration applied as well).
An overall QA score is calculated for each vis file, by dividing the‘before’ images by the ‘after’ and taking the median of the result. An
overall score of 1 would correspond to no change in the phase noise,a score > 1 implies an improvement.
If the overall score for a vis file is less than the value in‘accept_threshold’ then the wvr calibration file is not made available
for merging into the context for use in the subsequent reduction.
If you do not want any QA calculations then set qa_intent=”.
example: qa_intent=’PHASE’

qa_bandpass_intent

The data intent to use for the bandpass calibration in
the qa calculation. The default is blank to allow the underlying bandpass
task to select a sensible intent if the dataset lacks BANDPASS data.
continues on next page

3.25. hifa_wvrgcal

105

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 25 – continued from previous page
parameter nameqa_spw

accept_thresholddescription
The SpW(s) to use for the qa calculation, in the order that theyshould be tried. Input as a comma-separated list. The default is blank, inwhich case the task will try SpWs in order of decreasing median sky
opacity.
The phase-rms improvement ratio(rms without wvr / rms with wvr) above which the wrvg file will be
accepted into the context for subsequent application.

3.25.3 Examples
1. Compute the WVR calibration for all the MeasurementSets:>>> hifa_wvrgcal(hm_tie='automatic')

3.26 hifa_wvrgcalflag
3.26.1 Task description
This task will first identify for each vis whether it includes at least 3 antennas with Water Vapor Radiometer (WVR)
data, and that the fraction of WVR antennas / all antennas exceeds the minimum threshold (ants_with_wvr_thresh).
If there are not enough WVR antennas by number and/or fraction, then no WVR caltable is created and no WVR
calibration will be applied to the corresponding vis. If there are enough WVR antennas, then the task proceeds asfollows for each valid vis:
First, generate a gain table based on the Water Vapor Radiometer data for each vis.
Second, apply the WVR calibration to the data specified by ‘flag_intent’, calculate flagging ‘views’ showing the ratio‘phase-rms with WVR / phase-rms without WVR’ for each scan. A ratio < 1 implies that the phase noise is improved,a ratio > 1 implies that it is made worse.
Third, search the flagging views for antennas with anomalous high values. If any are found then recalculate the WVR
calibration with the ‘wvrflag’ parameter set to ignore their data and interpolate results from other antennas according
to ‘maxdistm’ and ‘minnumants’.
Fourth, after flagging, if the remaining unflagged antennas with WVR number fewer than 3, or represent a smaller
fraction of antennas than the minimum threshold (ants_with_wvr_thresh), then the WVR calibration file is rejected
and will not be merged into the context, i.e. not be used in subsequent calibration.
Fifth, if the overall QA score for the final WVR correction of a vis file is greater than the value in ‘accept_threshold’
then make available the wvr calibration file for merging into the context and use in the subsequent reduction.
Output:results – The results object for the pipeline task is returned.

3.26. hifa_wvrgcalflag

106

Pipeline Tasks Reference Manual, Release 2023.1.0

3.26.2 Parameter List
parameter namevis

caltable

offsetstable

hm_toffset

toffsetsegsource

sourceflag

description
List of input visibility files.
Default: none, in which case the vis files to be usedwill be read from the context.
Example: vis=[‘ngc5921.ms’]
List of output gain calibration tables.
Default: none, in which case the names of the caltables
will be generated automatically.
Example: caltable=’ngc5921.wvr’
List of input temperature offsets table files tosubtract from WVR measurements before calculating phasecorrections.
Default: none, in which case no offsets are applied.
Example: offsetstable=[‘ngc5921.cloud_offsets’]If ‘manual’, set the ‘toffset’ parameter to the user-specified
value. If ‘automatic’, set the ‘toffset’ parameter according to thedate of the MeasurementSet; toffset=-1 if before 2013-01-21T00:00:00
toffset=0 otherwise.
Time offset (sec) between interferometric and WVR data.
If True calculate new atmospheric phase correction
coefficients for each source, subject to theconstraints of the tie parameter. segsource isforced to be True if the tie parameter is set to a
non-empty value by the user or by the automaticheuristic.
Flag the WVR data for these source(s) as bad and do not
produce corrections for it. Requires
segsource = True.
Example: sourceflag=[‘3C273’]
continues on next page

3.26. hifa_wvrgcalflag

107

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 26 – continued from previous page
parameter namehm_tietie

description
If ‘manual’, set the tie parameter to theuser-specified value. If ‘automatic’, set the tie
parameter to include with the target all calibratorsthat are within 15 degrees of it: if no calibrators arethat close then tie is left empty.
Use the same atmospheric phase correction coefficients
when calculating the WVR correction for all sources inthe tie. If tie is not empty then segsource
is forced to be True. Ignored unless hm_tie = ‘manual’.
Example: tie=[‘3C273,NGC253’, ‘IC433,3C279’]

nsol

disperse

wvrflag

Number of solutions for phase correction coefficients during thisobservation, evenly distributed in time throughout the observation. It
is used only if segsource=False because if segsource=True then thecoefficients are recomputed whenever the telescope moves to a newsource (within the limits imposed by ‘tie’).
Apply correction for dispersion.
(Deprecated; will be removed)
Flag the WVR data for these antenna(s) as bad and replace its datawith interpolated values.
Example: wvrflag=[‘DV03’,’DA05’,’PM02’]

hm_smoothsmoothscale

If ‘manual’ set the ‘smooth’ parameter to the user-specified value.
If ‘automatic’, run the wvrgcal task with the range of ‘smooth’ parameters
required to match the integration time of the WVR data to that of theinterferometric data in each spectral window.
Smooth WVR data on this timescale before calculating the correction.
Ignored unless hm_smooth=’manual’.
Scale the entire phase correction by this factor.
continues on next page

3.26. hifa_wvrgcalflag

108

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 26 – continued from previous page
parameter namemaxdistm

description
Distance in meters of an antenna used for interpolationfrom a flagged antenna.
Default: -1 (automatically set to 100m if >50% ofantennas are 7m antennas without WVR and otherwise set
to 500m).
Example: maxdistm=550

minnumants

Minimum number of nearby antennas (up to 3) used forinterpolation from a flagged antenna.
Example: minnumants=3

mingoodfrac

Minimum fraction of good data per antenna.
Example: mingoodfrac=0.7

refantRanked comma delimited list of reference antennas.
Example: refant=’DV02,DV06’

flag_intent

The data intent(s) on whose WVR correction results thesearch for bad WVR antennas is to be based.
A ‘flagging view’ will be calculated for each specified
intent, in each spectral window in each vis file.
Each ‘flagging view’ will consist of a 2-d image with dimensions[‘ANTENNA’, ‘TIME’], showing the phase noise after the WVR
correction has been applied.
If flag_intent is left blank, the default, the flagging views will bederived from data with the default bandpass calibration intent i.e.
the first in the list BANDPASS, PHASE, AMPLITUDE for which the
MeasurementSet has data.
continues on next page

3.26. hifa_wvrgcalflag

109

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 26 – continued from previous page
parameter nameqa_intent

description
The list of data intents on which the WVR correction is to betried as a means of estimating its effectiveness.
A QA ‘view’ will be calculated for each specified intent, in each spectral
window in each vis file.
Each QA ‘view’ will consist of a pair of 2-d images with dimensions[‘ANTENNA’, ‘TIME’], one showing the data phase-noise before the
WVR application, the second showing the phase noise after (both ‘before’and ‘after’ images have a bandpass calibration applied as well).
An overall QA score is calculated for each vis file, by dividing the‘before’ images by the ‘after’ and taking the median of the result. An
overall score of 1 would correspond to no change in the phase noise,a score > 1 implies an improvement.
If the overall score for a vis file is less than the value in‘accept_threshold’ then the WVR calibration file is not made available formerging into the context for use in the subsequent reduction.

qa_bandpass_intent

accept_threshold

flag_hifhi_limit
fhi_minsample

The data intent to use for the bandpass calibration in
the qa calculation. The default is blank to allow the underlying bandpass
task to select a sensible intent if the dataset lacks BANDPASS data.
The phase-rms improvement ratio(rms without WVR / rms with WVR) above which the wrvg file will be
accepted into the context for subsequent application.
True to flag high figure of merit outliers.
Flag figure of merit values higher than limit * MAD.
Minimum number of samples for valid MAD estimate.
continues on next page

3.26. hifa_wvrgcalflag

110

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 26 – continued from previous page
parameter nameants_with_wvr_thresh

description
This threshold sets the minimum fraction of antennas
that should have WVR data for WVR calibration and flagging to proceed;thesame threshold is used to determine, after flagging, whether there remainenough unflagged antennas with WVR data for the WVR calibration to be
applied.
Example: ants_with_wvr_thresh=0.5

3.26.3 Examples
1. Compute the WVR calibration for all the MeasurementSets:>>> hifa_wvrgcalflag(hm_tie='automatic')

3.26. hifa_wvrgcalflag

111

CHAPTER

FOUR

INTERFEROMETRY ALMA SRDP

1 tasks available.
task namedescription

hifas_imageprecheck

Calculates the best robust value and Briggs weighting parameter to achievesensitivity and angular resolution goals.

4.1 hifas_imageprecheck4.1.1 Task description
In this task, the representative source and the spw containing the representative frequency selected by the PI in the
OT are used to calculate the synthesized beam and to make sensitivity estimates for the aggregate bandwidth andrepresentative bandwidth for several values of the robust parameter. This information is reported in a table in the
weblog. If no representative target/frequency information is available, it defaults to the first target and center of first
spw in the data (i.e. pre-Cycle 5 data does not have this information available). The best Briggs weighting parameterto achieve the PI’s desired angular resolution is chosen automatically. See the User’s guide for further details.
results – The results object for the pipeline task is returned.

4.1.2 Parameter List
parameter namevis

desired_angular_resolution

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hif_importdata task.
‘’: use all MeasurementSets in the context
Examples: ‘ngc5921.ms’, [‘ngc5921a.ms’, ngc5921b.ms’, ‘ngc5921c.ms’]
User specified angular resolution goal string. ‘’: automatic fromperformance parameters (default)
Example: ‘1.0arcsec’
continues on next page

112

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 1 – continued from previous page
parameter namecalcsbparalleldescription
Force (re-)calculation of sensitivities and beams
Use MPI cluster where possible

4.1.3 Examples
1. run with recommended settings to perform checks prior to imaging:>>> hifas_imageprecheck()
2. run to perform checks prior to imaging and force the re-calculation of sensitivities and beams:>>> hifas_imageprecheck(calcsb=True)

4.1. hifas_imageprecheck113

CHAPTERFIVE

INTERFEROMETRY VLA

27 tasks available.
task namedescription

hifv_analyzestokescubes

Characterize stokes IQUV flux densities as a function of frequency for
VLASS coarse cube images
Apply calibration tables to measurement set
Run RFI flagging using flagdata in various modes
Perform polarization calibration for VLA circular feeds.
Prepare and export interferometry and imaging data
Export Image data from QL, SE, and Coarse Cube modes of VLASS Survey
Compute final gain calibration tables
Base fixpointing task
Flagcal task
Do basic deterministic flagging of a list of MeasurementSets
Apply a flagtemplate to target data prior to running imaging pipeline tasks
Fluxboot
Hanning smoothing on a dataset
Imports data into the VLA pipeline
Apply primary beam correction to VLA and VLASS images
Create pipeline summary plots
Runs gaincurves, opacities, requantizer gains, antenna position corrections,tec_maps, switched power.
Restore flagged and calibration interferometry data from a pipeline run
Restore VLASS SE per-image measurement set data, resetting flagging,weights, and applying self-calibration.
Perform phase-only self-calibration, per scan row, on VLASS SE images
Runs a second delay and bandpass calibration and applies to calibrators tosetup for RFI flagging
Determines different solution intervals
Compute statistical weights and write them to measurement set
Determine amount of gain compression affecting VLA data below Ku-band
Runs initial delay and bandpass calibration to setup for RFI flagging
Sets flux density scale and fills calibrator model to measurement set
Create clean masks for VLASS Single Epoch (SE) images

hifv_applycals
hifv_checkflag
hifv_circfeedpolcal
hifv_exportdata
hifv_exportvlassdata
hifv_finalcals
hifv_fixpointing
hifv_flagcal
hifv_flagdata
hifv_flagtargetsdata
hifv_fluxboot
hifv_hanning
hifv_importdata
hifv_pbcor
hifv_plotsummary
hifv_priorcals
hifv_restoredata
hifv_restorepims
hifv_selfcal
hifv_semiFinalBPdcals
hifv_solint
hifv_statwt
hifv_syspower
hifv_testBPdcals
hifv_vlasetjy
hifv_vlassmasking

114

Pipeline Tasks Reference Manual, Release 2023.1.0

5.1 hifv_analyzestokescubes
5.1.1 Task description
Characterize stokes IQUV flux densities as a function of frequency for VLASS Coarse Cube (CC) images
Output:results – The results object for the pipeline task is returned.

5.1.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.

5.1.3 Examples
1. Basic analyzestokescubes task>>> hifv_analyzestokescubes()

5.2 hifv_applycals
5.2.1 Task description
hifv_applycals applies the precomputed calibration tables stored in the pipeline context to the set of visibility files using
predetermined field and spectral window maps and default values for the interpolation schemes.
Users can interact with the pipeline calibration state using the tasks h_export_calstate and h_import_calstate.
Output:results – The results object for the pipeline task is returne

5.2.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
continues on next page

5.1. hifv_analyzestokescubes

115

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter namefield

intentspw

antennaapplymode

flagbackup
flagsum
flagdetailedsum
gainmap

5.2. hifv_applycals

description
A string containing the list of field names or field ids to which thecalibration will be applied. Defaults to all fields in the pipeline
context.
example: ‘3C279’, ‘3C279, M82’
A string containing the list of intents against which the selected fields will
be matched. Defaults to all supported intents
in the pipeline context.
example: ‘*TARGET*’The list of spectral windows and channels to which the calibration will beapplied. Defaults to all science windows in the pipeline.
example: ‘17’, ‘11, 15’
The selection of antennas to which the calibration will be applied. Defaults
to all antennas. Not currently supported.
Calibration apply mode ‘calflag’: calibrate data and apply flags fromsolutions‘calflagstrict’: same as above except flag spws for which calibration isunavailable in one or more tables (instead of allowing them to pass
uncalibrated and unflagged)‘trial’: report on flags from solutions, dataset entirely unchanged‘flagonly’: apply flags from solutions only, data not calibrated‘flagonlystrict’: same as above except flag spws for which calibration isunavailable in one or more tables‘calonly’: calibrate data only, flags from solutions NOT applied
Backup the flags before the apply.
Compute before and after flagging summary statistics
Compute detailed flagging statistics
Mode to map gainfields to scans.

116

Pipeline Tasks Reference Manual, Release 2023.1.0

5.2.3 Examples
1. Run the final applycals stage of the VLA CASA pipeline.

>>> hifv_applycals()

5.3 hifv_checkflag
5.3.1 Task description
Run RFI flagging using flagdata in various modes
Output:results – The results object for the pipeline task is returned.

5.3.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
continues on next page

5.3. hifv_checkflag

117

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namecheckflagmode

growflags

overwrite_modelcol

5.3. hifv_checkflag

description– Standard VLA modes with improved RFI flagging heuristics: ‘bpd-vla’,‘allcals-vla’, ‘target-vla’ – blank string default use of rflag on bandpass anddelay calibrators– use string ‘semi’ after hifv_semiFinalBPdcals() for executing rflag on
calibrators– use string ‘bpd’, for the bandpass and delay calibrators:
execute rflag on all calibrated cross-hand corrected data;
extend flags to all correlations
execute rflag on all calibrated parallel-hand residual data;
extend flags to all correlations
execute tfcrop on all calibrated cross-hand corrected data,
per visibility; extend flags to all correlations
execute tfcrop on all calibrated parallel-hand corrected data,
per visibility; extend flags to all correlations– use string ‘allcals’, for all the other calibrators, with delays and BPcal
applied:similar procedure as ‘bpd’ mode, but uses corrected data throughout– use string ‘target’, for the target data:
similar procedure as ‘allcals’ mode, but with a higher SNR cutoff
for rflag to avoid flagging data due to source structure, andwith an additional series of tfcrop executions to make up forthe higher SNR cutoff in rflag– VLASS specific modes include ‘bpd-vlass’, ‘allcals-vlass’, and‘target-vlass’
which calculate thresholds to use per spw/field/scan (action=’calculate’,then,per baseband/field/scan, replace all spw thresholds above the median withthe median,
before re-running rflag with the new thresholds. This has the effect oflowering the thresholds for spws with RFI to be closer to the RFI-freethresholds, and catches more of the RFI.
– Mode ‘vlass-imaging’ is similar to ‘target-vlass’, except that it executeson the split off target
data, intent=’*TARGET’, datacolumn=’data’ and uses a timedevscale of
4.0.
Grow flags in time at the end of the following checkflagmodes:
default=True, for ‘bpd-vla’, ‘allcals-vla’, ‘bpd’, and ‘allcals’default=False, for ‘’ and ‘semi’Always write the model column, even if it already exists

118

Pipeline Tasks Reference Manual, Release 2023.1.0

5.3.3 Examples
1. Run RFLAG with associated heuristics in the VLA CASA pipeline.

>>> hifv_checkflag()

5.4 hifv_circfeedpolcal
5.4.1 Task description
Perform polarization calibration for VLA circular feeds.
Only validated for VLA sky survey data in S-band continuum mode with 3C138 or 3C286 as polarization angle. Requires that all polarization intents are properly set during observation.
Output:results – The results object for the pipeline task is returned.

5.4.2 Parameter List
parameter namevis
Dterm_solint
refantignore
leakage_poltype

mbdkcrossclipminmax

5.4. hifv_circfeedpolcal

description
List of input visibility data
D-terms spectral averaging. Example: refantignore=’ea02,ea03’
Help appears with Dterm_solint parameter
poltype to use in first polcal execution - blank string means use default
heuristics
Run gaincal KCROSS grouped by baseband
Acceptable range for leakage amplitudes, values outside will be flagged.

119Pipeline Tasks Reference Manual, Release 2023.1.0

5.4.3 Examples
1. Basic circfeedpolcal task>>> hifv_circfeedpolcal()

5.5 hifv_exportdata
5.5.1 Task description
The hifv_exportdata task for the VLA CASA pipeline exports the data defined in the pipeline context and exports it to
the data products directory, converting and or packing it as necessary.
The current version of the task exports the following products• an XML file containing the pipeline processing request• a tar file per ASDM / MS containing the final flags version OR the MS if tarms is False• a text file per ASDM / MS containing the final calibration apply list
• a FITS image for each selected calibrator source image• a FITS image for each selected science target source image• a tar file per session containing the caltables for that session• a tar file containing the file web log• a text file containing the final list of CASA commands
Output:results – The results object for the pipeline task is returned.

5.5.2 Parameter List
parameter namevis

description
List of visibility data files for which flagging and calibration information
will be exported. Defaults to the list maintained in thepipeline context.
example: vis=[‘X227.ms’, ‘X228.ms’]
continues on next page

5.5. hifv_exportdata

120

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 5 – continued from previous page
parameter namesessionimaging_products_only
exportmses

tarms
exportcalprods

pprfilecalintents

calimages

targetimages

products_dir

gainmap

5.5. hifv_exportdata

description
List of sessions one per visibility file. Currently defaults to a single virtual
session containing all the visibility files in vis.
In the future, this will default to the set of observing sessions definedin the context.
example: session=[‘session1’, ‘session2’]
Export science target imaging products only
Export the final MeasurementSets instead of the final flags, calibrationtables, and calibration instructions.
Tar final MeasurementSets
Export flags and caltables in addition to MeasurementSets. this parameteris only valid when exportmses = True.
Name of the pipeline processing request to be exported. Defaults to a file
matching the template ‘PPR_*.xml’.
example: pprfile=[‘PPR_GRB021004.xml’]
List of calibrator image types to be exported. Defaults to all standard
calibrator intents, ‘BANDPASS’, ‘PHASE’, ‘FLUX’.
example: ‘PHASE’
List of calibrator images to be exported. Defaults to all calibrator images
recorded in the pipeline context.
example: calimages=[‘3C454.3.bandpass’, ‘3C279.phase’]
List of science target images to be exported. Defaults to all science target
images recorded in the pipeline context.
example: targetimages=[‘NGC3256.band3’, ‘NGC3256.band6’]
Name of the data products subdirectory. Defaults to ‘./’ example:‘../products’
The value of gainmap parameter in hifv_restoredata task put in
casa_piperestorescript.py

121

Pipeline Tasks Reference Manual, Release 2023.1.0

5.5.3 Examples
1. Export the pipeline results for a single session to the data products directory>>> !mkdir ../products>>> hifv_exportdata (products_dir='../products')
2. Export the pipeline results to the data products directory specify that only the gain calibrator images be saved.

>>> !mkdir ../products>>> hifv_exportdata (products_dir='../products', calintents='*PHASE*')

5.6 hifv_exportvlassdata
5.6.1 Task description
Export Image data from QL, SE, and Coarse Cube modes of VLASS Survey
Output: results – The results object for the pipeline task is returned.

5.6.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.

5.6.3 Examples
1. Basic exportvlassdata task>>> hifv_exportvlassdata()

5.7 hifv_finalcals
5.7.1 Task description
Compute final gain calibration tables
Output:results – The results object for the pipeline task is returned.

5.6. hifv_exportvlassdata

122

Pipeline Tasks Reference Manual, Release 2023.1.0

5.7.2 Parameter List
parameter nameweakbprefantignore

description
Activate weak bandpass heuristics
String list of antennas to ignore

5.7.3 Examples
1. Create the final calibration tables to be applied to the data in the VLA CASA pipeline.

>>> hifv_finalcals()

5.8 hifv_fixpointing
5.8.1 Task description
The hifv_fixpointing task
Output:results – The results object for the pipeline task is returned.

5.8.2 Parameter List
parameter namevis

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.

5.8.3 Examples
1. Basic fixpointing task>>> hifv_fixpointing()

5.8. hifv_fixpointing

123Pipeline Tasks Reference Manual, Release 2023.1.0

5.9 hifv_flagcal
5.9.1 Task description
Flagcal task
Output:results – The results object for the pipeline task is returned.

5.9.2 Parameter List
parameter namevis
caltable
clipminmax

description
List of input visibility data
String name of the caltable
Range to use for clipping

5.9.3 Examples
1. Flag existing caltable>>> hifv_flagcal()

5.10 hifv_flagdata
5.10.1 Task description
The hifv_flagdata task performs basic flagging operations on a list of MeasurementSets including:• autocorrelation data flagging• shadowed antenna data flagging• scan based flagging• edge channel flagging• baseband edge flagging• applying online flags• applying a flagging template• quack, shadow, and basebands• Antenna not-on-source (ANOS)

5.9. hifv_flagcal

124

Pipeline Tasks Reference Manual, Release 2023.1.0

Output:results – The results object for the pipeline task is returned.

5.10.2 Parameter List
parameter namevis

autocorrshadowscan
scannumber

quack
clip
basebandintentsedgespw
fracspw
onlinedescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
Flag autocorrelation data
Flag shadowed antennas
Flag specified scans
A string containing a comma delimited list of scans to be flagged. example:‘3,5,6’
Quack scans
Clip mode
Flag 20MHz of each edge of basebands
A string containing a comma delimited list of intents against which thescans to be flagged are matched.
example: ‘*BANDPASS*’
Fraction of the baseline correlator TDM edge channels to be flagged.
Fraction of baseline correlator edge channels to be flagged
Apply the online flags
continues on next page

5.10. hifv_flagdata125

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 10 – continued from previous page
parameter namefileonline

template
filetemplate

hm_tbufftbuff
flagbackup

description
File containing the online flags. These are computed by the h_init or
hif_importdata data tasks. If the online flags files
are undefined a name of the form ‘msname.flagonline.txt’ is assumed.
Apply a flagging template
The name of a text file that contains the flagging template for RFI, birdies,
telluric lines, etc. If the template flags files
is undefined a name of the form ‘msname.flagtemplate.txt’ is assumed.
The time buffer computation heuristic
List of time buffers (sec) to pad timerange in flag commands
Backup pre-existing flags before applying new ones.

5.10.3 Examples
1. Do basic flagging on a MeasurementSet>>> hifv_flagdata()
2. Do basic flagging on a MeasurementSet as well as flag pointing and atmosphere data>>> hifv_flagdata(scan=True intent='*BANDPASS*')

5.11 hifv_flagtargetsdata
5.11.1 Task description
Apply a flagtemplate to target data prior to running imaging pipeline tasks
Output:results – The results object for the pipeline task is returned.

5.11. hifv_flagtargetsdata126

Pipeline Tasks Reference Manual, Release 2023.1.0

5.11.2 Parameter List
parameter namevis

template
filetemplateflagbackup

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
defined in the pipeline context.
Apply flagging templates.
The name of a text file that contains the flagging template for issues withthe science target data etc.
If the template flags files is undefined a name of theform ‘msname_flagtargetstemplate.txt’ is assumed.
Back up any pre-existing flags.

5.11.3 Examples
1. Basic flagtargetsdata task>>> hifv_flagtargetsdata()

5.12 hifv_fluxboot
5.12.1 Task description
Determine flux density bootstrapping for gain calibrators relative to flux calibrator.
Output:results – The results object for the pipeline task is returned.

5.12.2 Parameter List
parameter namevis

caltable

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
String name of the flagged caltable
continues on next page

5.12. hifv_fluxboot

127

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 12 – continued from previous page
parameter name
fitorderrefantignore

description
Polynomial order of the spectral fitting for valid flux densities with multiple
spws. The default value of -1 means that the heuristics determine the fit
order based on
fractional bandwidth and receiver bands present in the observation.
An override value of 1,2,3 or 4 may be specified by the user.
Spectral index (1) and, if applicable, curvature (2) are reported in the
weblog.
If no determination can be made by the heuristics, a fitorder of 1 will be
used.
String list of antennas to ignore Example: refantignore=’ea02,ea03’

5.12.3 Examples
1. VLA CASA pipeline flux density bootstrapping.

>>> hifv_fluxboot()

5.13 hifv_hanning
5.13.1 Task description
The hifv_hanning task will hanning smooth a VLA dataset
Output:results – The results object for the pipeline task is returned.

5.13.2 Parameter List
parameter namevis5.13. hifv_hanningdescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.

128

Pipeline Tasks Reference Manual, Release 2023.1.0

5.13.3 Examples
1. Run the task to execute hanning smoothing on a VLA CASA pipeline loaded MeasurementSet.

>>> hifv_hanning()

5.14 hifv_importdata
5.14.1 Task description
The hifv_importdata task loads the specified visibility data into the pipeline context unpacking and / or converting it
as necessary.
Output:results – The results object for the pipeline task is returned.

5.14.2 Parameter List
parameter namevis

sessionasis

overwrite
nocopydescription
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSes, or tar files of MSes, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
List of sessions to which the visibility files belong. Defaults to a single
session containing all the visibility files, otherwise
a session must be assigned to each vis file.
example: session=[‘Session_1’, ‘Sessions_2’]
Creates verbatim copies of the ASDM tables in the output MS. The value
given to this option must be a list of table names
separated by space characters.
examples: ‘Receiver CalAtmosphere’‘Receiver’, ‘’Overwrite existing files on import.
When importing an MS, disable copying of the MS to the working
directory.
continues on next page

5.14. hifv_importdata

129

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 14 – continued from previous page
parameter namecreatemms

ocorr_mode

datacolumns

description
Create a multi-MeasurementSet (‘true’) ready for parallel processing, or astandard MeasurementSet (‘false’). The default setting(‘automatic’) creates an MMS if running in a cluster environment.
Read in cross- and auto-correlation data(ca), cross- correlation data only(co), or autocorrelation data only (ao).
Dictionary defining the data types of existing columns. The format is:{‘data’: ‘data type 1’}or{‘data’: ‘data type 1’, ‘corrected’: ‘data type 2’}
For ASDMs the data type can only be RAW and one
can only specify it for the data column.
For MSes one can define two different data types
for the DATA and CORRECTED_DATA columns and they
can be any of the known data types (RAW,REGCAL_CONTLINE_ALL, REGCAL_CONTLINE_SCIENCE,SELFCAL_CONTLINE_SCIENCE, REGCAL_LINE_SCIENCE,SELFCAL_LINE_SCIENCE, BASELINED, ATMCORR). The
intent selection strings _ALL or _SCIENCE can beskipped. In that case the task determines thisautomatically by inspecting the existing intents
in the dataset.
Usually, a single datacolumns dictionary is used
for all datasets. If necessary, one can define alist of dictionaries, one for each EB, withdifferent setups per EB.
If no types are specified,{‘data’:’raw’,’corrected’:’regcal_contline’}
or {‘data’:’raw’} will be assumed, depending onwhether the corrected column exists or not.

5.14. hifv_importdata

130

Pipeline Tasks Reference Manual, Release 2023.1.0

5.14.3 Examples
1. Load an ASDM list in the ../rawdata subdirectory into the context.

>>> hifv_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e', '../rawdata/uid_A002_
˓→x30a93d_X44e'])2. Load an MS in the current directory into the context.

>>> hifv_importdata (vis=[uid___A002_X30a93d_X43e.ms])
3. Load a tarred ASDM in ../rawdata into the context.

>>> hifv_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e.tar.gz'])
4. Check the hifv_importdata inputs, then import the data>>> myvislist = ['uid___A002_X30a93d_X43e.ms', 'uid_A002_x30a93d_X44e.ms']>>> hifv_importdata(vis=myvislist)
5. Run with explicit setting of data column types:>>> hifv_importdata(vis=['uid___A002_X30a93d_X43e_targets.ms'], datacolumns={'data': 'regcal_
˓→contline'})>>> hifv_importdata(vis=['uid___A002_X30a93d_X43e_targets_line.ms'], datacolumns={'data':
˓→'regcal_line', 'corrected': 'selfcal_line'})5.15 hifv_pbcor
5.15.1 Task description
Apply primary beam correction to VLA and VLASS images
Output:results – The results object for the pipeline task is returned.

5.15.2 Parameter List
parameter namevis5.15. hifv_pbcor

description
List of input visibility data131

Pipeline Tasks Reference Manual, Release 2023.1.0

5.15.3 Examples
1. Basic pbcor task>>> hifv_pbcor()

5.16 hifv_plotsummary
5.16.1 Task description
Create pipeline summary plots
Output:results – The results object for the pipeline task is returned.

5.16.2 Parameter List
parameter namevis

description
List of input visibility data

5.16.3 Examples
1. Execute the pipeline plotting task.

>>> hifv_plotsummary()

5.17 hifv_priorcals
5.17.1 Task description
Runs gaincurves, opacities, requantizer gains, antenna position corrections, tec_maps, switched power.
Output:results – The results object for the pipeline task is returned.

5.16. hifv_plotsummary132

Pipeline Tasks Reference Manual, Release 2023.1.0

5.17.2 Parameter List
parameter namevis

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSes, or tar files of MSes, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]

show_tec_maps
apply_tec_correction
swpow_spw

Plot tec maps
Apply tec correction
Spectral-window(s) for plotting: “” ==>all, spw=”6,14”

5.17.3 Examples
1. Run gaincurves, opacities, requantizer gains and antenna position corrections.

>>> hifv_priorcals()

5.18 hifv_restoredata
5.18.1 Task description
The hifv_restoredata restores flagged and calibrated data from archived ASDMs and pipeline flagging and calibration
data products. Pending archive retrieval support hifv_restoredata assumes that the required products are available in
the rawdata_dir in the format produced by the hifv_exportdata task.
hifv_restoredata assumes that the following entities are available in the raw data directory• the ASDMs to be restored• for each ASDM in the input list:– a compressed tar file of the final flagversions file, e.g. uid___A002_X30a93d_X43e.ms.flagversions.tar.gz– a text file containing the applycal instructions, e.g. uid___A002_X30a93d_X43e.ms.calapply.txt– a compressed tar file containing the
uid___A001_X74_X29.session_3.caltables.tar.gz

caltables

for

the

parentsession,e.g.

hifv_restoredata performs the following operations• imports the ASDM(s))• removes the default MS.flagversions directory created by the filler• restores the final MS.flagversions directory stored by the pipeline

5.18. hifv_restoredata133

Pipeline Tasks Reference Manual, Release 2023.1.0• restores the final set of pipeline flags to the MS• restores the final calibration state of the MS• restores the final calibration tables for each MS• applies the calibration tables to each MS
Output:results – The results object for the pipeline task is returned.

5.18.2 Parameter List
parameter namevis

sessionproducts_dir

copytoraw

rawdata_dir
lazybdfflags
ocorr_mode
gainmap

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSes, or tar files of MSes, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
List of sessions one per visibility file. Example: session=[‘session_3’]
Name of the data products directory to copy calibration products from. The
parameter is effective only when copytoraw = True
When copytoraw = False, calibration products in rawdata_dir will be used.
example: products_dir=’myproductspath’
Copy calibration and flagging tables from products_dir to rawdata_dir
directory.
Example: copytoraw=False.
The rawdata directory. Example: rawdata_dir=’myrawdatapath’
Use the lazy filler option.
Set the BDF flags.
Correlation import mode
If True, map gainfields to a particular list of scans when applying
calibration tables.
continues on next page

5.18. hifv_restoredata134

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 18 – continued from previous page
parameter nameasis

description
List of tables to import asis.

5.18.3 Examples
1. Restore the pipeline results for a single ASDM in a single session>>> hifv_restoredata (vis=['myVLAsdm'], session=['session_1'], ocorr_mode='ca')

5.19 hifv_restorepims
5.19.1 Task description
Restore VLASS SE per-image measurement set data, resetting flagging, weights, and applying self-calibration.
Output:results – The results object for the pipeline task is returned.

5.19.2 Parameter List
parameter namevis
reimaging_resources

description
List of input visibility data
file path of reimaging_resources.tgz from the SE imaging product5.19.3 Examples
1. Basic restorepims task>>> hifv_restorepims()

5.19. hifv_restorepims

135

Pipeline Tasks Reference Manual, Release 2023.1.0

5.20 hifv_selfcal
5.20.1 Task description
Perform phase-only self-calibration, per scan row, on VLASS SE images
Output: results – The results object for the pipeline task is returned.

5.20.2 Parameter List
parameter namevis

refantignore
combinedescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
String list of antennas to ignore
Data axes which to combine for solve Options: ‘’,’obs’,’scan’,’spw’,field’, oranycomma-separated combination in a single string
Example: combine=’scan,spw’ - Extend solutions
over scan boundaries (up to the solint), andcombine spws for solving.
In selfcalmode=’VLASS-SE’ use the default value.

selfcalmoderefantmodeoverwrite_modelcol

5.20. hifv_selfcal

Heuristics mode selection. Known modes are ‘VLASS’ and ‘VLASS-SE’.
Default value is ‘VLASS’.
Reference antenna mode
Always write the model column, even if it already exists

136

Pipeline Tasks Reference Manual, Release 2023.1.0

5.20.3 Examples
1. Basic selfcal task>>> hifv_selfcal()
2. VLASS-SE selfcal usage>>> hifv_selfcal(selfcalmode='VLASS-SE', combine='field,spw')

5.21 hifv_semiFinalBPdcals
5.21.1 Task description
Runs a second delay and bandpass calibration and applies to calibrators to setup for RFI flagging
Output:results – The results object for the pipeline task is returned.

5.21.2 Parameter List
parameter namevis

weakbprefantignore

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
Activate weak bandpass heuristics
String list of antennas to ignore

5.21.3 Examples
1. Heuristic flagging>>> hifv_semiFinalBPdcals()

5.21. hifv_semiFinalBPdcals

137

Pipeline Tasks Reference Manual, Release 2023.1.0

5.22 hifv_solint
5.22.1 Task description
The hifv_solint task determines different solution intervals. Note that the short solint value is switched to ‘int’ whenthe minimum solution interval corresponds to one integration.
Output:results – The results object for the pipeline task is returned.

5.22.2 Parameter List
parameter namevis

limit_short_solint

refantignore

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
Keyword argument in units of seconds to limit the short solution interval.
Can be a string or float numerical value in units of seconds of ‘0.45’ or
0.45.
Can be set to a string value of ‘int’.
String list of antennas to ignore Example: refantignore=’ea02,ea03’

5.22.3 Examples
1. Determines different solution intervals:>>> hifv_solint()

5.23 hifv_statwt
5.23.1 Task description
Compute statistical weights and write them to measurement set
Output:results – The results object for the pipeline task is returned.

5.22. hifv_solint

138

Pipeline Tasks Reference Manual, Release 2023.1.0

5.23.2 Parameter List
parameter namevis

datacolumn

overwrite_modelcol
statwtmodedescription
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
Data column used to compute weights. Supported values are “data”,“corrected”, “residual”, and “residual_data”(case insensitive, minimum match supported).
Always write the model column, even if it already exists
Sets the weighting parameters for general VLA (‘VLA’) or VLASS Single
Epoch (‘VLASS-SE’) use case. Note that the ‘VLASS-SE’mode is meant to be used with datacolumn=’residual_data’.
Default is ‘VLA’.

5.23.3 Examples
1. Statistical weighting of the visibilities:>>> hifv_statwt()
2. Statistical weighting of the visibilities in the Very Large Array Sky Survey Single Epoch use case:>>> hifv_statwt(mode='vlass-se', datacolumn='residual_data')

5.24 hifv_syspower
5.24.1 Task description
Determine amount of gain compression affecting VLA data below Ku-band
Output:results – The results object for the pipeline task is returned.

5.24. hifv_syspower139

Pipeline Tasks Reference Manual, Release 2023.1.0

5.24.2 Parameter List
parameter namevis
clip_sp_templateantexclude

apply
do_not_apply

description
List of input visibility data
Acceptable range for Pdiff data; data are clipped outside this range andflaggeddictionary in the format of: {‘L’: {‘ea02’: {‘usemedian’: True}, ‘ea03’:{‘usemedian’: False}},‘X’: {‘ea02’: {‘usemedian’: True}, ‘ea03’: {‘usemedian’: False}},‘S’: {‘ea12’: {‘usemedian’: False}, ‘ea22’: {‘usemedian’: False}}}If antexclude is specified with ‘usemedian’: False, the template values are
replaced with 1.0.
If ‘usemedian’: True, the template values are replaced with the median of
the good antennas.
Apply task results to RQ tablecsv string of band names to not apply. Example: ‘L,X,S’

5.24.3 Examples
1. Basic syspower task>>> hifv_syspower()

5.25 hifv_testBPdcals
5.25.1 Task description
Runs initial delay and bandpass calibration to setup for RFI flagging
Output:results – The results object for the pipeline task is returned.

5.25. hifv_testBPdcals

140

Pipeline Tasks Reference Manual, Release 2023.1.0

5.25.2 Parameter List
parameter namevis

weakbprefantignore
doflagundernspwlimit

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
Activate weak bandpass heuristics
String list of antennas to ignore Example: refantignore=’ea02,ea03’
If the number of bad spws is greater than zero, and the keyword is True,then spws are flagged individually.

5.25.3 Examples
1. Initial delay calibration to set up heuristic flagging.

>>> hifv_testBPdcals()

5.26 hifv_vlasetjy
5.26.1 Task description
The hifv_vlasetjy task does an initial run of setjy on the vis
Output:results – The results object for the pipeline task is returned.
standard – Flux density standard default: ‘’

5.26.2 Parameter List
parameter name
vis

field

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
List of field names or ids.
continues on next page

5.26. hifv_vlasetjy

141

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 26 – continued from previous page
parameter nameintentspw
model
reffilefluxdensity
spix
reffreq
scalebychan
standard

description
Observing intent of flux calibrators.
List of spectral window ids.
File location for field model.
Path to file with fluxes for non-solar system calibrators.
Specified flux density [I,Q,U,V]; -1 will lookup values
Spectral index of fluxdensity. Can be set when fluxdensity is not -1
Reference frequency for spix. Can be set when fluxdensity is not -1
Scale the flux density on a per channel basis or else on a per spw basis
Flux density standard

5.26.3 Examples
1. Initial run of setjy:>>> hifv_vlasetjy()

5.27 hifv_vlassmasking
5.27.1 Task description
Create clean masks for VLASS SE images
Output:results – The results object for the pipeline task is returned.

5.27. hifv_vlassmasking

142

Pipeline Tasks Reference Manual, Release 2023.1.0

5.27.2 Parameter List
parameter namevis

vlass_ql_databasemaskingmodecatalog_search_size

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
specified in the h_init or hifv_importdata task.
vlass_ql_database - usage in Socorro: /home/vlass/packages/VLASS1Q.fits
maskingmode options are vlass-se-tier-1 or vlass-se-tier-2
catalog_search_size in units of degrees

5.27.3 Examples
1. Basic vlassmasking task>>> hifv_vlassmasking()

5.27. hifv_vlassmasking

143CHAPTERSIX

SINGLE DISH

12 tasks available.
task namedescription

hsd_applycal
hsd_atmcor
hsd_baseline

Apply the calibration(s) to the data
Apply offline ATM correction to the data.
Detect and validate spectral lines, subtract baseline by masking detectedlines
Flag spectra based on predefined criteria of single dish pipeline
Prepare single dish data for export
Do basic flagging of a list of MeasurementSets
Generate single dish images
Imports data into the single dish pipeline
Derive Kelvin to Jy calibration tables
Restore flagged and calibration single dish data from a pipeline run
Calibrate data
Flag deviant system temperature measurements

hsd_blflag
hsd_exportdata
hsd_flagdata
hsd_imaging
hsd_importdata
hsd_k2jycal
hsd_restoredata
hsd_skycal
hsd_tsysflag

6.1 hsd_applycal
6.1.1 Task description
Apply the calibration to the data.
hsd_applycal applies the precomputed calibration tables stored in the pipeline context to the set of visibility files using
predetermined field and spectral window maps and default values for the interpolation schemes.
Users can interact with the pipeline calibration state using the tasks h_export_calstate and h_import_calstate.
Output:results – The results object for the pipeline task is returned

144

Pipeline Tasks Reference Manual, Release 2023.1.0

6.1.2 Parameter List
parameter name
vis

field

intentspw

antennaapplymode

calwt
flagbackup

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
in the pipeline context.
example: [‘X227.ms’]
A string containing the list of field names or field ids to which thecalibration will be applied. Defaults to all fields in the pipeline context.
example: ‘3C279’, ‘3C279, M82’
A string containing the list of intents against which the selected fields will
be matched. Defaults to all supported intents
in the pipeline context.
example: ‘*TARGET*’The list of spectral windows and channels to which the calibration will beapplied. Defaults to all science windows in the pipeline context.
example: ‘17’, ‘11, 15’
The selection of antennas to which the calibration will be applied. Defaults
to all antennas. Not currently supported.
Calibration apply mode ‘calflag’: calibrate data and apply flags fromsolutions‘calflagstrict’: same as above except flag spws for which calibration isunavailable in one or more tables (instead of allowing them to pass
uncalibrated and unflagged)‘trial’: report on flags from solutions, dataset entirely unchanged‘flagonly’: apply flags from solutions only, data not calibrated‘flagonlystrict’: same as above except flag spws for which calibration isunavailable in one or more tables‘calonly’: calibrate data only, flags from solutions NOT applied
Calibrate the weights as well as the data.
Backup the flags before the apply.
continues on next page

6.1. hsd_applycal

145

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 1 – continued from previous page
parameter nameparalleldescription
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)

6.1.3 Examples
1. Apply the calibration to the target data>>> hsd_applycal (intent='TARGET')
2. Specify fields and spectral windows>>> hsd_applycal(field='3C279, M82', spw='17', intent='TARGET')

6.2 hsd_atmcor
6.2.1 Task description
The hsd_atmcor task provides the capability of offline correction of residual atmospheric features in the calibrated
single-dish spectra originated from incomplete calibration mainly due to a difference of elevation angles between
ON_SOURCE and OFF_SOURCE measurements.
Optimal atmospheric model is automatically determined by default (atmtype = ‘auto’). You may specify desired atmospheric model by giving either single integer (apply to all EBs) or a list of integers (models per EB) to atmtype
parameter. Please see parameter description for the meanings of integer values.

6.2.2 Parameter List

6.2. hsd_atmcor

146

Pipeline Tasks Reference Manual, Release 2023.1.0

parameter nameatmtypedtem_dhh0

infilesantenna

description
Type of atmospheric transmission model represented as an integer.
Available options are as follows. Integer values can be given as
either integer or string, i.e. both 1 and ‘1’ are acceptable.
‘auto’: perform heuristics to choose best model (default)
1: tropical
2: mid latitude summer
3: mid latitude winter
4: subarctic summer
5: subarctic winterIf list of integer is given, it also performs heuristics using theprovided values instead of default, [1, 2, 3, 4], which is used
when ‘auto’ is provided. List input should not contain ‘auto’.
Default: ‘auto’Temperature gradient [K/km], e.g. -5.6. (”” = Tool default) The value isdirectly passed to initialization method for ATM model.
Float and string types are acceptable. Float value is interpreted as
the value in K/km. String value should be the numeric value with unit
such as ‘-5.6K/km’. When list of values are given, it will
trigger heuristics to choose best model from the provided value.
Default: ‘’ (tool default, -5.6K/km, is used)
Scale height for water [km], e.g. 2.0. (”” = Tool default) The value isdirectly passed to initialization method for ATM model.
Float and string types are acceptable. Float value is interpreted as
the value in kilometer. String value should be the numeric value withunit compatible with length, such as ‘2km’ or ‘2000m’.
When list of values are given, it will trigger heuristics tochoose best model from the provided value.
Default: ‘’ (tool default, 2.0km, is used)
ASDM or MS files to be processed. This parameter behaves as data
selection parameter. The name specified by
infiles must be registered to context before you run
hsd_atmcor.
Data selection by antenna names or ids. example: ‘PM03,PM04’‘’ (all antennas)
continues on next page

6.2. hsd_atmcor

147

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter name
field

spw

pol

description
Data selection by field names or ids. example: ‘*Sgr*,M100’‘’ (all fields)
Data selection by spw ids. example: ‘3,4’ (spw 3 and 4)‘’ (all spws)
Data selection by polarizations. example: ‘XX,YY’ (correlation XX andYY)‘’ (all polarizations)

6.2.3 Examples
1. Basic usage>>> hsd_atmcor()
2. Specify atmospheric model and data selection>>> hsd_atmcor(atmtype=1, antenna='PM03,PM04', field='*Sgr*,M100')
3. Specify atmospheric model per EB (atmtype 1 for 1st EB, 2 for 2nd EB)>>> hsd_atmcor(atmtype=[1, 2])

6.3 hsd_baseline
6.3.1 Task description
The hsd_baseline task subtracts baseline from calibrated spectra. By default, the task tries to find spectral line featureusing line detection and validation algorithms. Then, the task puts a mask on detected lines and perform baseline
subtraction. The user is able to turn off automatic line masking by setting linewindow parameter, which specifies
pre-defined line window.
Fitting order is automatically determined by default. It can be disabled by specifying fitorder as non-negative value. In
this case, the value specified by fitorder will be used.
*WARNING* Currently, hsd_baseline overwrites the result obtained by the previous run. Due to this behavior, users
need to be careful about an order of the task execution when they run hsd_baseline multiple times with different data
selection. Suppose there are two spectral windows (0 and 1) and hsd_baseline is executed separately for each spw as
below,>>> hsd_baseline(spw='0')>>> hsd_baseline(spw='1')>>> hsd_blflag()>>> hsd_imaging()
6.3. hsd_baseline148

Pipeline Tasks Reference Manual, Release 2023.1.0

Since the second run of hsd_baseline overwrites the result for spw 0 with the data before baseline subtraction, this will
not produce correct result for spw 0. Proper sequence for this use case is to process each spw to the imaging stage
separately, which looks like as follows:>>> hsd_baseline(spw='0')>>> hsd_blflag(spw='0')>>> hsd_imaging(spw='0'))>>> hsd_baseline(spw='1')>>> hsd_blflag(spw='1')>>> hsd_imaging(spw='1')
Output: results – The results object for the pipeline task is returned.

6.3.2 Parameter List
parameter namefitfunc

fitorderswitchpoly

description
fitting function for baseline subtraction. You can only choose cubic spline(‘spline’ or ‘cspline’)Fitting order for polynomial. For cubic spline, it is used to determine howmuch the spectrum is segmented into.
Default (-1) is to determine the order automatically.
If True, switch to 1st or 2nd order polynomial fit when large mask exists atedge regardless of whatever fitfunc
or fitorder are specified. Condition for switching is asfollows:
if nmask > nchan/2 => 1st order polynomial
else if nmask > nchan/4 => 2nd order polynomial
else => use fitfunc and fitorderwhere nmask is a number of channels for mask at edge whilenchan is a number of channels of entire spectral window.
continues on next page

6.3. hsd_baseline149

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namelinewindow

description
Pre-defined line window. If this is set, specified line windows are used as aline mask for baseline subtraction
instead to determine masks based on line detection andvalidation stage. Several types of format are acceptable.
One is channel-based window,[min_chan, max_chan]
where min_chan and max_chan should be an integer. For
multiple windows, nested list is also acceptable,[[min_chan0, max_chan0], [min_chan1, max_chan1], . . . ]Another way is frequency-based window,[min_freq, max_freq]
where min_freq and max_freq should be either a float ora string. If float value is given, it is interpreted asa frequency in Hz. String should be a quantity consistingof “value” and “unit”,
e.g., ‘100GHz’. Multiple windows are also supported.
[[min_freq0, max_freq0], [min_freq1, max_freq1], . . . ]Note that the specified frequencies are assumed to be
the value in LSRK frame. Note also that there is alimitation when multiple MSes are processed.
If native frequency frame of the data is not LSRK
(e.g. TOPO), frequencies need to be converted to thatframe. As a result, corresponding channel range may vary
between MSes. However, current implementation is notable to handle such case. Frequencies are converted todesired frame using representative MS (time, position,
direction).
In the above cases, specified line windows are applied
to all science spws. In case when line windows vary withspw, line windows can be specified by a dictionary whose
key is spw id while value is line window.
For example, the following dictionary gives different
line windows to spws 17 and 19. Other spws, if available,
will have an empty line window.
{17: [[100, 200], [1200, 1400]], 19: [‘112115MHz’, ‘112116MHz’]}
Furthermore, linewindow accepts MS selection string.
The following string gives [[100,200],[1200,1400]] forcontinues on next page

6.3. hsd_baseline150Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namedescription
spw 17 while [1000,1500] for spw 21.
“17:100~200;1200~1400,21:1000~1500”
The string also accepts frequency with units. Note,however, that frequency reference frame in this case is
not fixed to LSRK. Instead, the frame will be taken from
the MS (typically TOPO for ALMA).
Thus, the following two frequency-based line windows
result different channel selections.
{19: [‘112115MHz’, ‘112116MHz’]} # frequency frame is LSRK
“19:11215MHz~11216MHz” # frequency frame is taken from the data
# (TOPO for ALMA)
None is allowed as a value of dictionary input to indicate thatno line detection/validation is required even if manually specified
line window does not exist. When None is given as a value and if
linewindowmode is ‘replace’, line detection/validation is not performed
for the corresponding spw. For example, suppose the following parameters
are given for the data with four science spws, 17, 19, 21, and 23.
linewindow={17: [112.1e9, 112.2e9], 19: [113.1e9, 113.15e9], 21: None}
linewindowmode=’replace’
The task will use given line window for 17 and 19 while the task performs
line deteciton/validation for spw 23 because no line window is set.
On the other hand, line detection/validation is skipped for spw 21 due tothe effect of None.
example: [100,200] (channel), [115e9, 115.1e9] (frequency in Hz)[‘115GHz’, ‘115.1GHz’], see above for more examples

linewindowmodeMerge or replace given manual line window with line detection/validation
result. If ‘replace’ is given, line
detection and validation will not be performed.
On the other hand, when ‘merge’ is specified, line
detection/validation will be performed and manuallyspecified line windows are added to the result.
Note that this has no effect when linewindow for target
spw is an empty list. In that case, line detection/validation
will be performed regardless of the value of linewindowmode.
In case if no linewindow nor line detection/validation
are necessary, you should set linewindowmode to ‘replace’and specify None as a value of the linewindow dictionary
for the spw to apply. See parameter description of linewindow
for detail.
continues on next page

6.3. hsd_baseline151Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter nameedge

broadline
clusteringalgorithm

deviationmask

parallelinfilesfield

antennaspw

description
Number of edge channels to be dropped from baseline subtraction. The
value must be a list with length of 2,
whose values specify left and right edge channels,
respectively.
example: [10,10]
Try to detect broad component of spectral line if True.
Selection of the algorithm used in the clustering analysis to check thevalidity of detected line features.
‘kmean’ algorithm and hierarchical clustering algorithm‘hierarchy’, and their combination (‘both’) are so far
implemented.
Apply deviation mask in addition to masks determined by the automaticline detection.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)
List of data files. These must be a name of MeasurementSets that areregistered to context via
hsd_importdata task.
example: vis=[‘X227.ms’, ‘X228.ms’]
Data selection by field. example: ‘1’ (select by FIELD_ID)‘M100*’ (select by field name)‘’ (all fields)
Data selection by antenna. example: ‘1’ (select by ANTENNA_ID)‘PM03’ (select by antenna name)‘’ (all antennas)
Data selection by spw. example: ‘3,4’ (generate caltable for spw 3 and 4)[‘0’,’2’] (spw 0 for first data, 2 for second)‘’ (all spws)
continues on next page

6.3. hsd_baseline152

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namepoldescription
Data selection by polarizations. example: ‘0’ (generate caltable for pol 0)[‘0~1’,’0’] (pol 0 and 1 for first data, only 0 for second)‘’ (all polarizations)6.3.3 Examples
1. Basic usage with automatic line detection and validation>>> hsd_baseline(antenna='PM03', spw='17,19')
2. Using pre-defined line windows without automatic line detection and edge channels>>> hsd_baseline(linewindow=[[100, 200], [1200, 1400]],
linewindowmode='replace', edge=[10, 10])
3. Using per spw pre-defined line windows with automatic line detection>>> hsd_baseline(linewindow={19: [[390, 550]], 23: [[100, 200], [1200, 1400]]},linewindowmode='merge')

6.4 hsd_blflag
6.4.1 Task description
Data are flagged based on several flagging rules. Available rules are: expected rms, calculated rms, and running meanof both pre-fit and post-fit spectra. Tsys flagging is also available.
In addition, the heuristics script creates many plots for each stage. Those plots are included in the weblog.
Output:results – The results object for the pipeline task is returned.

6.4.2 Parameter List
parameter nameiteration

description
Number of iterations to perform sigma clipping to calculate threshold valueof flagging.
continues on next page

6.4. hsd_blflag

153

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter nameedge

flag_tsys
tsys_thresh
flag_prfre
prfre_thresh
flag_pofre

pofre_thresh
flag_prfr
prfr_thresh
flag_pofr
pofr_thresh
flag_prfrm
prfrm_thresh
prfrm_nmean

description
Number of channels to be dropped from the edge. The value must be a listof integer with length of one or
two. If list length is one, same number will be appliedboth side of the band.
example: [10,20], [10]Activate (True) or deactivate (False) Tsys flag.
Threshold value for Tsys flag.
Activate (True) or deactivate (False) flag by expected rms of pre-fit spectra.
Threshold value for flag by expected rms of pre-fit spectra.
Activate (True) or deactivate (False) flag by expected rms of post-fit
spectra.
Threshold value for flag by expected rms of post-fit spectra.
Activate (True) or deactivate (False) flag by rms of pre-fit spectra.
Threshold value for flag by rms of pre-fit spectra.
Activate (True) or deactivate (False) flag by rms of post-fit spectra.
Threshold value for flag by rms of post-fit spectra.
Activate (True) or deactivate (False) flag by running mean of pre-fit spectra.
Threshold value for flag by running mean of pre-fit spectra.
Number of channels for running mean of pre-fit spectra.
continues on next page

6.4. hsd_blflag

154

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 4 – continued from previous page
parameter nameflag_pofrm

pofrm_thresh
pofrm_nmean
plotflagparallelinfilesantenna

field

spw

pol

6.4. hsd_blflag

description
Activate (True) or deactivate (False) flag by running mean of post-fit
spectra.
Threshold value for flag by running mean of post-fit spectra.
Number of channels for running mean of post-fit spectra.
True to plot result of data flagging.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)
ASDM or MS files to be processed. This parameter behaves as data
selection parameter. The name specified by
infiles must be registered to context before you run
hsd_blflag.
Data selection by antenna names or ids. example: ‘PM03,PM04’‘’ (all antennas)
Data selection by field names or ids. example: ‘*Sgr*,M100’‘’ (all fields)
Data selection by spw ids. example: ‘3,4’ (spw 3 and 4)‘’ (all spws)
Data selection by polarizations. example: ‘XX,YY’ (correlation XX andYY)‘’ (all polarizations)155

Pipeline Tasks Reference Manual, Release 2023.1.0

6.4.3 Examples
1. flagging with all rules>>> hsd_blflag()

6.5 hsd_exportdata
6.5.1 Task description
The hsd_exportdata task exports the data defined in the pipeline context and exports it to the data products directory,
converting and or packing it as necessary.
The current version of the task exports the following products• a FITS image for each selected science target source image• a tar file per ASDM containing the final flags version and blparam• a tar file containing the file web log
Output:results – The results object for the pipeline task is returned.

6.5.2 Parameter List
parameter namepprfiletargetimages

products_dir

6.5. hsd_exportdata

description
Name of the pipeline processing request to be exported. Defaults to a file
matching the template ‘PPR_*.xml’.
example: pprfile=[‘PPR_GRB021004.xml’]
List of science target images to be exported. Defaults to all science target
images recorded in the pipeline context.
example: targetimages=[‘r_aqr.CM02.spw5.line0.XXYY.sd.im’,‘r_aqr.CM02.spw5.XXYY.sd.cont.im’]Name of the data products subdirectory. Defaults to ‘./’ example:
products_dir=’../products’156

Pipeline Tasks Reference Manual, Release 2023.1.0

6.5.3 Examples
1. Export the pipeline results for a single session to the data products directory>>> !mkdir ../products>>> hsd_exportdata (products_dir='../products')

6.6 hsd_flagdata
6.6.1 Task description
The hsd_flagdata data performs basic flagging operations on a list of MeasurementSets including:• applying online flags• applying a flagging template• shadowed antenna data flagging• scan-based flagging by intent or scan number• edge channel flagging
Output:results – The results object for the pipeline task is returned.

6.6.2 Parameter List
parameter namevis

autocorrshadowscan
scannumber

description
The list of input MeasurementSets. Defaults to the list of MeasurementSets
defined in the pipeline context.
Flag autocorrelation data.
Flag shadowed antennas.
Flag a list of scans and intents specified by scannumber and intents.
A string containing a comma delimited list of scans to be flagged.
continues on next page

6.6. hsd_flagdata

157

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 6 – continued from previous page
parameter nameintentsedgespw
fracspw
fracspwfps
onlinefileonline

template
filetemplate

pointing
filepointing

incompleterasterdescription
A string containing a comma delimited list of intents against which thescans to be flagged are matched.
example: ‘*BANDPASS*’
Flag the edge spectral window channels.
Fraction of the baseline correlator TDM edge channels to be flagged.
Fraction of the ACS correlator TDM edge channels to be flagged.
Apply the online flags.
File containing the online flags. These are computed by the h_init or
hif_importdata data tasks. If the online flags files
are undefined a name of the form ‘msname.flagonline.txt’ is assumed.
Apply a flagging template.
The name of a text file that contains the flagging template for RFI, birdies,
telluric lines, etc. If the template flags files
is undefined a name of the form ‘msname.flagtemplate.txt’ is assumed.
Apply a flagging template for pointing flag.
The name of a text file that contains the flagging template for pointing flag.
If the template flags files is undefined a name of
the form ‘msname.flagpointing.txt’ is assumed.
Apply commands to flag incomplete raster sequence. If this is False,
relevant commands in filepointing are
simply commented out.
continues on next page

6.6. hsd_flagdata158

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 6 – continued from previous page
parameter namehm_tbuff

tbuff

qa0
qa2
parallelflagbackup

description
The heuristic for computing the default time interval padding parameter.
The options are ‘halfint’ and ‘manual’. In ‘halfint’ mode tbuff is set tohalf the maximum of the median integration time of the science andcalibrator target
observations.
The time in seconds used to pad flagging command time intervals if
hm_tbuff=’manual’.
QA0 flags
QA2 flags
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)
Back up any pre-existing flags before applying new ones.

6.6.3 Examples
1. Do basic flagging on a MeasurementSet>>> hsd_flagdata()
2. Do basic flagging on a MeasurementSet flagging additional scans selected by number as well.

>>> hsd_flagdata(scannumber='13,18')

6.7 hsd_imaging
6.7.1 Task description
The hsd_imaging task generates single dish images per antenna as well as combined image over whole antennas foreach field and spectral window. Image configuration (grid size, number of pixels, etc.) is automatically determinedbased on meta data such as antenna diameter, map extent, etc.
Note that generated images are always in LSRK frame.
Output: results – The results object for the pipeline task is returned.

6.7. hsd_imaging

159

Pipeline Tasks Reference Manual, Release 2023.1.0

6.7.2 Parameter List
parameter namemode

restfreq
infilesfield
spw

description
Imaging mode controls imaging parameters in the task. Accepts either“line” (spectral line imaging) or “ampcal”(image settings for amplitude calibrator)
Rest frequency
List of data files. These must be a name of MeasurementSets that areregistered to context via
hsd_importdata task.
example: vis=[‘uid___A002_X85c183_X36f.ms’,‘uid___A002_X85c183_X60b.ms’]
Data selection by field names or ids. example: “*Sgr*,M100”
Data selection by spw ids. example: “3,4” (generate images for spw 3 and4)

6.7.3 Examples
1. Generate images with default settings and context>>> hsd_imaging()
2. Generate images with amplitude calibrator and specific parameters>>> hsd_imaging(mode='ampcal', field='*Sgr*,M100', spw='17,19')

6.8 hsd_importdata
6.8.1 Task description
The hsd_importdata task loads the specified visibility data into the pipeline context unpacking and / or converting it asnecessary.
If the overwrite input parameter is set to False and the task is asked to convert an input ASDM input to an MS, thenwhen the output MS already exists in the output directory, the importasdm conversion step is skipped, and the existing
MS will be imported instead.
Output:results – The results object for the pipeline task is returned.
6.8. hsd_importdata160

Pipeline Tasks Reference Manual, Release 2023.1.0

6.8.2 Parameter List
parameter namevis

sessionhm_rasterscan

parallelasis

process_caldevice
overwrite

nocopybdfflags

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSes, or tar files of MSes, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
List of sessions to which the visibility files belong. Defaults to a single
session containing all the visibility files, otherwise
a session must be assigned to each vis file.
example: session=[‘Session_1’, ‘Sessions_2’]
Heuristics method for raster scan analysis. Two analysis modes,
time-domain analysis (‘time’) and direction analysis (‘direction’), areavailable.
Default is ‘time’.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)
Creates verbatim copies of the ASDM tables in the output MS. The value
given to this option must be a list of table names
separated by space characters.
example: ‘Receiver’, ‘’Ingest the ASDM caldevice table. example: True
Overwrite existing files on import. When converting ASDM to MS, if
overwrite=False and the MS alreadyexists in output directory, then this existing MS dataset will be usedinstead.
Disable copying of MS to working directory
Apply BDF flags on import.
continues on next page

6.8. hsd_importdata

161

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 8 – continued from previous page
parameter namedatacolumns

lazywith_pointing_correction

createmms

description
Dictionary defining the data types of existing columns. The format is:{‘data’: ‘data type 1’}or{‘data’: ‘data type 1’, ‘corrected’: ‘data type 2’}
For ASDMs the data type can only be RAW and one
can only specify it for the data column.
For MSes one can define two different data types
for the DATA and CORRECTED_DATA columns and they
can be any of the known data types (RAW,REGCAL_CONTLINE_ALL, REGCAL_CONTLINE_SCIENCE,SELFCAL_CONTLINE_SCIENCE, REGCAL_LINE_SCIENCE,SELFCAL_LINE_SCIENCE, BASELINED, ATMCORR). The
intent selection strings _ALL or _SCIENCE can beskipped. In that case the task determines thisautomatically by inspecting the existing intents
in the dataset.
Usually, a single datacolumns dictionary is used
for all datasets. If necessary, one can define alist of dictionaries, one for each EB, withdifferent setups per EB.
If no type is specified, {‘data’:’raw’} will
be assumed.
Use the lazy filter importadd (ASDM::Pointing::encoder - ASDM::Pointing::pointingDirection) to
the value to be written in MS::Pointing::direction
Create an MMS

6.8.3 Examples
1. Load an ASDM list in the ../rawdata subdirectory into the context.

>>> hsd_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e', '../rawdata/uid_A002_
˓→x30a93d_X44e'])2. Load an MS in the current directory into the context.

>>> hsd_importdata (vis=['uid___A002_X30a93d_X43e.ms'])
3. Load a tarred ASDM in ../rawdata into the context.

6.8. hsd_importdata

162

Pipeline Tasks Reference Manual, Release 2023.1.0>>> hsd_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e.tar.gz'])
4. Import a list of MeasurementSets.

>>> myvislist = ['uid___A002_X30a93d_X43e.ms', 'uid_A002_x30a93d_X44e.ms']>>> hsd_importdata(vis=myvislist)

6.9 hsd_k2jycal
6.9.1 Task description
Derive the Kelvin to Jy calibration for list of MeasurementSets.
results – The results object for the pipeline task is returned.

6.9.2 Parameter List
parameter namedbservice
endpoint

description
Whether or not accessing Jy/K DB to retrieve conversion factors.
Which endpoints to use for query options: ‘asdm’, ‘model-fit’,‘interpolation’continues on next page

6.9. hsd_k2jycal

163

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namereffiledescription
Path to a file containing Jy/K factors for science data, which must be
provided by associating calibrator reduction or the observatory
measurements. Jy/K factor must take into account all efficiencies, i.e.,
it must be a direct conversion factor from Ta* to Jy. The file must bein either MS-based or session-based format. The MS-based format must
be in an CSV format with five fields: MS name, antenna name, spectral
window id, polarization string, and Jy/K conversion factor. Example forthe file is as follows:
MS,Antenna,Spwid,Polarization,Factor
uid___A002_X316307_X6f.ms,CM03,5,XX,10.0
uid___A002_X316307_X6f.ms,CM03,5,YY,12.0
uid___A002_X316307_X6f.ms,PM04,5,XX,2.0
uid___A002_X316307_X6f.ms,PM04,5,YY,5.0
The first line in the above example is a header which may or may notexist. Example for the session-based format is as follows:#OUSID=XXXXXX#OBJECT=Uranus#FLUXJY=yy,zz,aa#FLUXFREQ=YY,ZZ,AA#sessionID,ObservationStartDate(UTC),ObservationEndDate(UTC),
Antenna,BandCenter(MHz),BandWidth(MHz),POL,Factor
1,2011-11-11 01:00:00,2011-11-11 01:30:00,CM02,86243.0,500.0,I,10.0
1,2011-11-11 01:00:00,2011-11-11 01:30:00,CM02,86243.0,1000.0,I,30.0
1,2011-11-11 01:00:00,2011-11-11 01:30:00,CM03,86243.0,500.0,I,50.0
1,2011-11-11 01:00:00,2011-11-11 01:30:00,CM03,86243.0,1000.0,I,70.0
1,2011-11-11 01:00:00,2011-11-11
01:30:00,ANONYMOUS,86243.0,500.0,I,30.0
1,2011-11-11 01:00:00,2011-11-11
01:30:00,ANONYMOUS,86243.0,1000.0,I,50.0
2,2011-11-13 01:45:00,2011-11-13 02:15:00,PM04,86243.0,500.0,I,90.0
2,2011-11-13 01:45:00,2011-11-13
02:15:00,PM04,86243.0,1000.0,I,110.0
2,2011-11-13 01:45:00,2011-11-13
02:15:00,ANONYMOUS,86243.0,500.0,I,90.0
2,2011-11-13 01:45:00,2011-11-13
02:15:00,ANONYMOUS,86243.0,1000.0,I,110.0
Lines starting with ‘#’ are meta data and header.
The header must exist. The factor to apply is identified by matching thecontinues on next page

6.9. hsd_k2jycal

164

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 9 – continued from previous page
parameter namedescription
session ID, antenna name, frequency and polarization of data in each line of
the file. Note the observation date is supplementary information and notused
for the matching so far. The lines whose antenna name is ‘ANONYMOUS’
are used
when there is no measurement for specific antenna in the session. In theabove
example, if science observation of session 1 contains the antenna PM04,
Jy/K
factor for ANONYMOUS antenna will be applied since there is no
measurement forPM04 in session 1.
If no file name is specified or specified file doesn’t exist, all Jy/K factors
are set to 1.0.
example: reffile=”, reffile=’working/jyperk.csv’

infiles

List of input MeasurementSets.
example: vis=’ngc5921.ms’

caltable

Name of output gain calibration tables.
example: caltable=’ngc5921.gcal’

6.9.3 Examples
1. Compute the Kevin to Jy calibration tables for a list of MeasurementSets:>>> hsd_k2jycal()

6.10 hsd_restoredata
6.10.1 Task description
The hsd_restoredata task restores flagged and calibrated data from archived ASDMs and pipeline flagging and calibration data products. Pending archive retrieval support hsd_restoredata assumes that the required products are available
in the rawdata_dir in the format produced by the hifa_exportdata task.
hsd_restoredata assumes that the following entities are available in the raw data directory:• the ASDMs to be restored• for each ASDM in the input list:– a compressed tar file of the final flagversions file, e.g. uid___A002_X30a93d_X43e.ms.flagversions.tar.gz
6.10. hsd_restoredata165

Pipeline Tasks Reference Manual, Release 2023.1.0– a text file containing the applycal instructions, e.g. uid___A002_X30a93d_X43e.ms.calapply.txt– a compressed tar file containing the
uid___A001_X74_X29.session_3.caltables.tar.gz

caltables

for

the

parentsession,e.g.

hsd_restoredata performs the following operations:• imports the ASDM(s)• removes the default MS.flagversions directory created by the filler• restores the final MS.flagversions directory stored by the pipeline• restores the final set of pipeline flags to the MS• restores the final calibration state of the MS• restores the final calibration tables for each MS• applies the calibration tables to each MS
When importing the ASDM and converting it to a Measurement Set (MS), if the output MS already exists in the output
directory, then the importasdm conversion step is skipped, and the existing MS will be imported instead.
Output:results – The results object for the pipeline task is returned.

6.10.2 Parameter List
parameter namevis

description
List of raw visibility data files to be restored. Assumed to be in the
directory specified by rawdata_dir.
example: vis=[‘uid___A002_X30a93d_X43e’]sessionproducts_dir

List of sessions one per visibility file.
example: session=[‘session_3’]
Name of the data products directory to copy calibration products from. The
parameter is effective only when copytoraw = True.
When copytoraw = False, calibration products in rawdata_dir will be used.
example: products_dir=’myproductspath’

copytoraw

Copy calibration and flagging tables from products_dir to rawdata_dir
directory.
example: copytoraw=False
continues on next page

6.10. hsd_restoredata

166

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 10 – continued from previous page
parameter namerawdata_dir

lazybdfflags

ocorr_mode

asis

description
Name of the raw data directory.
example: rawdata_dir=’myrawdatapath’
Use the lazy filler option
example: lazy=True
Set the BDF flags
example: bdfflags=False
Set ocorr_mode
example: ocorr_mode=’ca’Creates verbatim copies of the ASDM tables in the output MS. The value
given to this option must be a list of table names separated by spacecharacters.
example: asis=’Source Receiver’

hm_rasterscan

Heuristics method for raster scan analysis. Two analysis modes,
time-domain analysis (‘time’) and direction analysis (‘direction’), areavailable.
Default is ‘time’.

6.10.3 Examples
1. Restore the pipeline results for a single ASDM in a single session>>> hsd_restoredata (vis=['uid___A002_X30a93d_X43e'], session=['session_1'], ocorr_mode='ao')

6.11 hsd_skycal
6.11.1 Task description
The hsd_skycal generates a caltable for sky calibration that stores reference spectra, which is to be subtracted fromon-source spectra to filter out non-source contribution.
Output:results – The results object for the pipeline task is returned.

6.11. hsd_skycal

167

Pipeline Tasks Reference Manual, Release 2023.1.0

6.11.2 Parameter List
parameter namecalmodefraction

noff

width

elongatedparalleldescription
Calibration mode. Available options are ‘auto’ (default), ‘ps’, ‘otf’, and‘otfraster’. When ‘auto’ is set, the task will
use preset calibration mode that is determined by inspecting data.
‘ps’ mode is simple position switching using explicit referencescans. Other two modes, ‘otf’ and ‘otfraster’, will generatereference data from scans at the edge of the map. Those modes
are intended for OTF observation and the former is defined forgeneric scanning pattern such as Lissajous, while the latter is
specific use for raster scan.
options: ‘auto’, ‘ps’, ‘otf’, ‘otfraster’Sub-parameter for calmode. Edge marking parameter for ‘otf’ and‘otfraster’ mode. It specifies a number of OFF scans
as a fraction of total number of data points.
options: String style like ‘20%’, or float value less than 1.0.
For ‘otfraster’ mode, you can also specify ‘auto’.
Sub-parameter for calmode. Edge marking parameter for ‘otfraster’ mode.
It is used to specify a number of OFF scans near edge directly
instead to specify it by fractional number by ‘fraction’. If it isset, the value will come before setting by ‘fraction’.
options: any positive integer value
Sub-parameter for calmode. Edge marking parameter for ‘otf’ mode. It
specifies pixel width with respect to a median spatial
separation between neighboring two data in time. Default will
be fine in most cases.
options: any float value
Sub-parameter for calmode. Edge marking parameter for ‘otf’ mode.
Please set True only if observed area is elongatedin one direction.
Execute using CASA HPC functionality, if available. options: ‘automatic’,‘true’, ‘false’, True, False
default: None (equivalent to ‘automatic’)continues on next page

6.11. hsd_skycal

168

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 11 – continued from previous page
parameter nameinfiles

description
List of data files. These must be a name of MeasurementSets that areregistered to context via hsd_importdata task.
example: vis=[‘X227.ms’, ‘X228.ms’]

field

Data selection by field name.

spw

Data selection by spw. (defalut all spws) example: ‘3,4’ (generate caltable
for spw 3 and 4)[‘0’,’2’] (spw 0 for first data, 2 for second)

scan

Data selection by scan number. (default all scans) example: ‘22,23’ (usescan 22 and 23 for calibration)[‘22’,’24’] (scan 22 for first data, 24 for second)

6.11.3 Examples
1. Generate caltables for all data managed by context.

>>> default(hsd_skycal)>>> hsd_skycal()6.12 hsd_tsysflag
6.12.1 Task description
Flag deviant system temperature measurements for single dish measurements. This is done by running a sequence offlagging sub-tasks (tests), each looking for a different type of possible error.
If a file with manual Tsys flags is provided with the ‘filetemplate’ parameter, then these flags are applied prior to theevaluation of the flagging heuristics listed below.
The tests are:
1. Flag Tsys spectra with high median values
2. Flag Tsys spectra with high median derivatives. This is meant to spot
spectra that are ‘ringing’.
3. Flag the edge channels of the Tsys spectra in each SpW.
4. Flag Tsys spectra whose shape is different from that associated with
the BANDPASS intent.
5. Flag ‘birdies’.

6.12. hsd_tsysflag

169

Pipeline Tasks Reference Manual, Release 2023.1.0

6. Flag the Tsys spectra of all antennas in a timestamp and spw if
proportion of antennas already flagged in this timestamp and spw exceeds a threshold, and flag
Tsys spectra for all antennas and all timestamps in a spw, if proportion of antennas that are
already entirely flagged in all timestamps exceeds a threshold.
Outputresults – The results object for the pipeline task is returned.

6.12.2 Parameter List
parameter namevis
caltable

flag_nmedian
fnm_limit

fnm_byfield
flag_derivative
fd_max_limit

flag_edgechansfe_edge_limit

flag_fieldshape

ff_refintent

description
List of input MeasurementSets (Not used)
List of input Tsys calibration tables
default: [] - Use the table currently stored in the pipeline context.
example: caltable=[‘X132.ms.tsys.s2.tbl’]
True to flag Tsys spectra with high median value.
Flag spectra with median value higher than fnm_limit * median of thismeasure over all spectra.
Evaluate the nmedian metric separately for each field.
True to flag Tsys spectra with high median derivative.
Flag spectra with median derivative higher than fd_max_limit * median of
this measure over all spectra.
True to flag edges of Tsys spectra.
Flag channels whose channel to channel difference > fe_edge_limit *median across spectrum.
True to flag Tsys spectra with a radically different shape to those of the
ff_refintent.
Data intent that provides the reference shape for ‘flag_fieldshape’.
continues on next page

6.12. hsd_tsysflag

170

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 12 – continued from previous page
parameter nameff_max_limit
flag_birdies
fb_sharps_limit
flag_toomany

tmf1_limit

tmef1_limit

metric_ordernormalize_tsys

filetemplate

6.12. hsd_tsysflag

description
Flag Tsys spectra with ‘fieldshape’ metric values > ff_max_limit.
True to flag channels covering sharp spectral features.
Flag channels bracketing a channel to channel difference > fb_sharps_limit.
True to flag Tsys spectra for which a proportion of antennas for given
timestamp and/or proportion of antennas that are
entirely flagged in all timestamps exceeds their respective thresholds.
Flag Tsys spectra for all antennas in a timestamp and spw if proportion ofantennas already flagged in this timestamp and spw exceeds
tmf1_limit.
Flag Tsys spectra for all antennas and all timestamps in a spw, if proportionof antennas that are already entirely flagged
in all timestamps exceeds tmef1_limit.
Order in which to evaluate the flagging metrics that are enabled. Disabled
metrics are skipped.
True to create a normalized Tsys table that is used to evaluate the Tsys
flagging metrics. All newly found flags are also applied
to the original Tsys caltable that continues to be used for subsequentcalibration.
The name of a text file that contains the manual Tsys flagging template. If
the template flags file is undefined, a name of the form
‘msname.flagtsystemplate.txt’ is assumed.

171

Pipeline Tasks Reference Manual, Release 2023.1.0

6.12.3 Examples
1. Flag Tsys measurements using currently recommended tests:>>> hsd_tsysflag()
2. Flag Tsys measurements using all recommended tests apart from that using the ‘fieldshape’ metric:>>> hsd_tsysflag(flag_fieldshape=False)

6.12. hsd_tsysflag

172

CHAPTERSEVENNOBEYAMA

3 tasks available.
task namedescription

hsdn_exportdata
hsdn_importdata
hsdn_restoredata

Prepare single dish data for export
Imports Nobeyama data into the single dish pipeline
Restore flagged and calibration single dish data from a pipeline run

7.1 hsdn_exportdata
7.1.1 Task description
The hsdn_exportdata task exports the data defined in the pipeline context and exports it to the data products directory,
converting and or packing it as necessary.
The current version of the task exports the following products• a FITS image for each selected science target source image• a tar file per ASDM containing the final flags version and blparam• a tar file containing the file web log
Output:results – The results object for the pipeline task is returned.

7.1.2 Parameter List
parameter namepprfiledescription
Name of the pipeline processing request to be exported. Defaults to a file
matching the template ‘PPR_*.xml’.
example: pprfile=[‘PPR_GRB021004.xml’]continues on next page

173

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 1 – continued from previous page
parameter nametargetimages

products_dir

description
List of science target images to be exported. Defaults to all science target
images recorded in the pipeline context.
example: targetimages=[‘r_aqr.CM02.spw5.line0.XXYY.sd.im’,‘r_aqr.CM02.spw5.XXYY.sd.cont.im’]Name of the data products subdirectory. Defaults to ‘./’ example:
products_dir=’../products’

7.1.3 Examples
1. Export the pipeline results for a single session to the data products directory>>> !mkdir ../products>>> hsdn_exportdata (products_dir='../products')

7.2 hsdn_importdata
7.2.1 Task description
Imports Nobeyama data into the single dish pipeline. The hsdn_importdata task loads the specified visibility data into
the pipeline context unpacking and / or converting it as necessary.
If the overwrite input parameter is set to False and the task is asked to convert an input ASDM input to an MS, thenwhen the output MS already exists in the output directory, the importasdm conversion step is skipped, and the existing
MS will be imported instead.
Outputresults – The results object for the pipeline task is returned.

7.2.2 Parameter List
parameter namevis

description
List of visibility data files. These may be ASDMs, tar files of ASDMs,
MSes, or tar files of MSes, If ASDM files are specified, they will be
converted to MS format.
example: vis=[‘X227.ms’, ‘asdms.tar.gz’]
continues on next page

7.2. hsdn_importdata

174

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter namesessionhm_rasterscan

datacolumns

description
List of sessions to which the visibility files belong. Defaults to a single
session containing all the visibility files, otherwise
a session must be assigned to each vis file.
example: session=[‘Session_1’, ‘Sessions_2’]
Heuristics method for raster scan analysis. Two analysis modes,
time-domain analysis (‘time’) and direction analysis (‘direction’), areavailable.
Default is ‘time’.
Dictionary defining the data types of existing columns. The format is:{‘data’: ‘data type 1’}or{‘data’: ‘data type 1’, ‘corrected’: ‘data type 2’}
For ASDMs the data type can only be RAW and one
can only specify it for the data column.
For MSes one can define two different data types
for the DATA and CORRECTED_DATA columns and they
can be any of the known data types (RAW,REGCAL_CONTLINE_ALL, REGCAL_CONTLINE_SCIENCE,SELFCAL_CONTLINE_SCIENCE, REGCAL_LINE_SCIENCE,SELFCAL_LINE_SCIENCE, BASELINED, ATMCORR). The
intent selection strings _ALL or _SCIENCE can beskipped. In that case the task determines thisautomatically by inspecting the existing intents
in the dataset.
Usually, a single datacolumns dictionary is used
for all datasets. If necessary, one can define alist of dictionaries, one for each EB, withdifferent setups per EB.
If no type is specified, {‘data’:’raw’} will
be assumed.
continues on next page

7.2. hsdn_importdata

175

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 2 – continued from previous page
parameter nameoverwrite

nocopydescription
Overwrite existing files on import. When converting ASDM to MS, if
overwrite=False and the MS alreadyexists in output directory, then this existing MS dataset will be usedinstead.
Disable copying of MS to working directory.

createmms

Create an MMS

7.2.3 Examples
1. Load an ASDM list in the ../rawdata subdirectory into the context:>>> hsdn_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e', '../rawdata/uid_A002_
˓→x30a93d_X44e'])2. Load an MS in the current directory into the context:>>> hsdn_importdata (vis=[uid___A002_X30a93d_X43e.ms])
3. Load a tarred ASDM in ../rawdata into the context:>>> hsdn_importdata (vis=['../rawdata/uid___A002_X30a93d_X43e.tar.gz'])
4. Import a list of MeasurementSets:>>> myvislist = ['uid___A002_X30a93d_X43e.ms', 'uid_A002_x30a93d_X44e.ms']>>> hsdn_importdata(vis=myvislist)

7.3 hsdn_restoredata
7.3.1 Task description
The hsdn_restoredata task restores flagged and calibrated data from archived ASDMs and pipeline flagging and calibration data products. Pending archive retrieval support hsdn_restoredata assumes that the required products are available
in the rawdata_dir in the format produced by the hifa_exportdata task.
hsdn_restoredata assumes that the following entities are available in the raw data directory:• the ASDMs to be restored• for each ASDM in the input list:– a compressed tar file of the final flagversions file, e.g. uid___A002_X30a93d_X43e.ms.flagversions.tar.gz– a text file containing the applycal instructions, e.g. uid___A002_X30a93d_X43e.ms.calapply.txt

7.3. hsdn_restoredata176

Pipeline Tasks Reference Manual, Release 2023.1.0– a compressed tar file containing the
uid___A001_X74_X29.session_3.caltables.tar.gz

caltables

for

the

parentsession,e.g.

hsdn_restoredata performs the following operations:• imports the ASDM(s)• removes the default MS.flagversions directory created by the filler• restores the final MS.flagversions directory stored by the pipeline• restores the final set of pipeline flags to the MS• restores the final calibration state of the MS• restores the final calibration tables for each MS• applies the calibration tables to each MS
When importing the ASDM and converting it to a Measurement Set (MS), if the output MS already exists in the output
directory, then the importasdm conversion step is skipped, and the existing MS will be imported instead.
Output:results – The results object for the pipeline task is returned.

7.3.2 Parameter List
parameter namevis

description
List of raw visibility data files to be restored. Assumed to be in the
directory specified by rawdata_dir.
example: vis=[‘uid___A002_X30a93d_X43e’]caltable

Name of output gain calibration tables.
example: caltable=’ngc5921.gcal’continues on next page

7.3. hsdn_restoredata

177

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namereffiledescription
Path to a file containing scaling factors between beams. The format isequals to jyperk.csv with five fields:
MS name, beam name (instead of antenna name), spectral window id,
polarization string, and the scaling factor.
Example for the file is as follows:#MS,Beam,Spwid,Polarization,Factor
mg2-20181016165248-181017.ms,NRO-BEAM0,0,I,1.000000000
mg2-20181016165248-181017.ms,NRO-BEAM0,1,I,1.000000000
mg2-20181016165248-181017.ms,NRO-BEAM0,2,I,1.000000000
mg2-20181016165248-181017.ms,NRO-BEAM0,3,I,1.000000000
mg2-20181016165248-181017.ms,NRO-BEAM1,0,I,3.000000000
mg2-20181016165248-181017.ms,NRO-BEAM1,1,I,3.000000000
mg2-20181016165248-181017.ms,NRO-BEAM1,2,I,3.000000000
mg2-20181016165248-181017.ms,NRO-BEAM1,3,I,3.000000000
mg2-20181016165248-181017.ms,NRO-BEAM2,0,I,0.500000000
mg2-20181016165248-181017.ms,NRO-BEAM2,1,I,0.500000000
mg2-20181016165248-181017.ms,NRO-BEAM2,2,I,0.500000000
mg2-20181016165248-181017.ms,NRO-BEAM2,3,I,0.500000000
mg2-20181016165248-181017.ms,NRO-BEAM3,0,I,2.000000000
mg2-20181016165248-181017.ms,NRO-BEAM3,1,I,2.000000000
mg2-20181016165248-181017.ms,NRO-BEAM3,2,I,2.000000000
mg2-20181016165248-181017.ms,NRO-BEAM3,3,I,2.000000000
If no file name is specified or specified file doesn’t exist,
all the factors are set to 1.0.
example: reffile=”, reffile=’nroscalefactor.csv’products_dir

Name of the data products directory. Currently not used.
example: products_dir=’myproductspath’

copytoraw

rawdata_dir

Copy calibration and flagging tables to raw data directory.
example: copytoraw=False
Name of the raw data directory.
example: rawdata_dir=’myrawdatapath’continues on next page

7.3. hsdn_restoredata

178

Pipeline Tasks Reference Manual, Release 2023.1.0

Table 3 – continued from previous page
parameter namehm_rasterscan

description
Heuristics method for raster scan analysis. Two analysis modes,
time-domain analysis (‘time’) and direction analysis (‘direction’), areavailable.
Default is ‘time’.

7.3.3 Examples
1. Restore the pipeline results for a single ASDM in a single session>>> hsdn_restoredata (vis=['mg2-20181016165248-190320.ms'], reffile='nroscalefactor.csv')

7.3. hsdn_restoredata

179