Preliminary Progress Report on the High Frequency Single-DishSideband Separation
ALMA Technical Note Number: 12Status: FINAL
Prepared by:

Organization:

Date:

James Chibueze (NAOJ)

ALMA12 August 2014

Report on the High Frequency Single-Dish Sideband Separation
Prepared by James Chibueze (NAOJ) v2014/8/12
Contents
1. Reference JIRA tickets
2. Observation details
3. Known issues
4. Data Calibration
5. Sideband separation results
6. Recommendations
Reference JIRA tickets: CSV-2365, CSV-1526
Observation details:
BAND 9:Total number of executions: 6 (4 taken with 4 antennas, 2 taken with 3 antenna)
Software build ALMA-10_6_0-B-2014-05-28-01-00-00, 4 antennas (DA61, DA64, PM03, PM04),
PWV = 0.65 mm
Project "SBsep-test_v2" (Version 2.1, code 0000.0.00085.CSV)
SB "IRAS16293-offset_-204"
Start/End: 2014-06-03T05:19:26 - 05:48:43
ExecBlock: uid://A002/X833c94/X1cb
SB "IRAS16293-offset_-178"
Start/End: 06:58:31 - 07:28:09
ExecBlock: uid://A002/X833c94/X662
SB "IRAS16293-offset_-125"
Start/End: 07:33:15 - 08:02:25
ExecBlock: uid://A002/X833c94/X976
SB "IRAS16293-offset_-17"
Start/End: 08:03:45 - 08:32:51
ExecBlock: uid://A002/X833c94/Xccf##################################
1 hour
ALMA Build ALMA-10_6_0-B-2014-06-17-01-00-00
Antennas (3) DA61, PM03, PM04
PWV~0.7mm
SBsep-test_v2 (Version 2.1)

Project Code 0000.0.00085.CSV
PI skomugi* 1st execution:
SchedBlock IRAS16293-offset_0
ExecBlock uid://A002/X8505d8/X15c4* 2nd execution:
SchedBlock IRAS16293-offset_+204
ExecBlock uid://A002/X8505d8/X196f

BAND 10Total number of executions: 6 (all taken with 2 antennas; PM03 and PM04)
2 hours
SBsep-test_v2 (Version 2.1)
Project Code 0000.0.00085.CSV
PI skomugi
ALMA Build ALMA-10_6_0-B-2014-07-09-01-00-00
Antenna used: PM03 and PM04
PWV~0.6mm
Start/End 03:11:14 - 03:46:10
SchedBlock B10_IRAS16293-offset_-204
ExecBlock uid://A002/X881d57/X59
Start/End 03:46:38 - 04:20:57
SchedBlock B10_IRAS16293-offset_0
ExecBlock uid://A002/X881d57/X1e3
Start/End 04:21:24 - 04:52:23
SchedBlock B10_IRAS16293-offset_-17ExecBlock uid://A002/X881d57/X5ef
1 hour
ALMA Build ALMA-10_6_0-B-2014-07-30-01-00-00
Antenna used: PM03 and PM04
PWV=0.51mm
SBsep-test_v2 (Version 2.1)
Project Code 0000.0.00085.CSV

SchedBlock B10_IRAS16293-offset_-204
Start/End 03:30:49 - 04:05:20
ExecBlock uid://A002/X8867ff/X103
SchedBlock B10_IRAS16293-offset_-178
Start/End 04:05:56 - 04:40:15
ExecBlock uid://A002/X8867ff/X3f5
30 minutes
ALMA Build ALMA-10_6_0-B-2014-07-30-01-00-00
PWV=0.86mm
SBsep-test_v2 (Version 2.1)
Start/End 03:44:40 - 04:19:03
Project Code 0000.0.00085.CSV
SchedBlock B10_IRAS16293-offset_-125
ExecBlock uid://A002/X8886e3/X270# Note that the data was obtained under the not very good weather condition for B10. Please checkthe data quality, and let us know if we need additional data.

Known Issues:
The data were affected by ICT-3082: the first subscan in each scan was done on-source. Hence thefirst raster row in each scan should be flagged prior to imaging.
SysCal.xml needs to be regenerated using offline TelCal, excluding the scans which contain
subscanIntent=REFERENCE. Otherwise the script generator fails. The command is
tc_atmosphere(asdm='hoge', dataorigin='specauto', scans='3 6 9').
Negative Trx in Band 6 (PRTSPR-5770) was seen.
Some of the datasets especially the band 10 data were taken under bad weather conditions.
Data Calibration:* based on the generated script• no additional flagging was done
#SDcalibLine script# ALMA Data Reduction Script# Calibration
thesteps = []step_title = {0: 'listobs',
1: 'Split by antenna',
2: 'sdlist',
3: 'Filling the Tsys solutions in the dataset',
4: 'Do initial flagging',

5: 'Calibration of the data into Kelvins',
6: 'Subtracting the baseline',
7: 'Converting ASAP -> MS',
8: 'Split and concatenation'}
if 'applyonly' not in globals(): applyonly = False
try:print 'List of steps to be executed ...', mysteps
thesteps = mysteps
except:print 'global variable mysteps not set.'if (thesteps==[]):
thesteps = range(0,len(step_title))
print 'Executing all steps: ', thesteps# The Python variable 'mysteps' will control which steps# are executed when you start the script using# execfile('scriptForCalibration.py')# e.g. setting# mysteps = [2,3,4]# before starting the script will make the script execute# only steps 2, 3, and 4# Setting mysteps = [] will make it execute all steps.
import reimport filltsys
es = aU.stuffForScienceDataReduction()
if re.search('^4.2.1', casadef.casa_version) == None:sys.exit('ERROR: PLEASE USE THE SAME VERSION OF CASA THAT YOU USED FOR
GENERATING THE SCRIPT: 4.2.1')# CALIBRATE_AMPLI:# CALIBRATE_ATMOSPHERE: IRAS16293-2422,J1626-2951# CALIBRATE_BANDPASS:# CALIBRATE_FLUX:# CALIBRATE_FOCUS: J1626-2951# CALIBRATE_PHASE:# CALIBRATE_POINTING: J1626-2951# OBSERVE_TARGET: IRAS16293-2422# listobs
mystep = 0if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -rf uid___A002_X8886e3_X270.ms.listobs')

listobs(vis = 'uid___A002_X8886e3_X270.ms',
listfile = 'uid___A002_X8886e3_X270.ms.listobs')
aU.getTPSampling(vis = 'uid___A002_X8886e3_X270.ms', showplot = True, plotfile ='uid___A002_X8886e3_X270.ms.sampling.png')# Split by antenna
mystep = 1
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]for i in ['PM03', 'PM04']:os.system('rm -Rf uid___A002_X8886e3_X270.ms.'+i+'*')
sd.splitant(filename = 'uid___A002_X8886e3_X270.ms',
outprefix = 'uid___A002_X8886e3_X270.ms',
overwrite = True)# sdlist
mystep = 2
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM03.asap.sdlist')
sdlist(infile = 'uid___A002_X8886e3_X270.ms.PM03.asap',
outfile = 'uid___A002_X8886e3_X270.ms.PM03.asap.sdlist')os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM04.asap.sdlist')
sdlist(infile = 'uid___A002_X8886e3_X270.ms.PM04.asap',
outfile = 'uid___A002_X8886e3_X270.ms.PM04.asap.sdlist')# Filling the Tsys solutions in the dataset
mystep = 3
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.tsys')
gencal(vis = 'uid___A002_X8886e3_X270.ms',
caltable = 'uid___A002_X8886e3_X270.ms.tsys',
caltype = 'tsys')

from recipes.almahelpers import tsysspwmap
tsysmap = tsysspwmap(vis = 'uid___A002_X8886e3_X270.ms', tsystable ='uid___A002_X8886e3_X270.ms.tsys')
for i in [42]:filltsys.fillTsys('uid___A002_X8886e3_X270.ms.PM03.asap',
specif = i,tsysif = tsysmap[i],mode = 'linear',extrap = True)for i in [42]:filltsys.fillTsys('uid___A002_X8886e3_X270.ms.PM04.asap',
specif = i,tsysif = tsysmap[i],mode = 'linear',extrap = True)plotbandpass(caltable='uid___A002_X8886e3_X270.ms.tsys', overlay='time',
xaxis='freq', yaxis='amp', subplot=22, buildpdf=False, interactive=False,
showatm=True,pwv='auto',chanrange='5~123',showfdm=True,
field='',
figfile='uid___A002_X8886e3_X270.ms.tsys.plots.overlayTime/uid___A002_X8886e3_X270.ms.tsys')
es.checkCalTable('uid___A002_X8886e3_X270.ms.tsys',msName='uid___A002_X8886e3_X270.ms', interactive=False)# Do initial flagging
mystep = 4if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]
sdflag2(infile = 'uid___A002_X8886e3_X270.ms.PM03.asap',
specunit = 'channel',mode = 'manual',
ifs = [42],maskflag = [[0, 119], [3960, 4079]],
overwrite = True)sdflag2(infile = 'uid___A002_X8886e3_X270.ms.PM04.asap',
specunit = 'channel',mode = 'manual',
ifs = [42],maskflag = [[0, 119], [3960, 4079]],
overwrite = True)

# Calibration of the data into Kelvins
mystep = 5
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM03.asap.cal')sdcal(infile = 'uid___A002_X8886e3_X270.ms.PM03.asap',
calmode = 'ps',
iflist = [42],scanaverage = False,timeaverage = False,polaverage = False,
outfile = 'uid___A002_X8886e3_X270.ms.PM03.asap.cal',overwrite = True)
es.SDcheckSpectra('uid___A002_X8886e3_X270.ms.PM03.asap.cal', spwIds='42',
interactive=False)
os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM04.asap.cal')sdcal(infile = 'uid___A002_X8886e3_X270.ms.PM04.asap',
calmode = 'ps',
iflist = [42],scanaverage = False,timeaverage = False,polaverage = False,
outfile = 'uid___A002_X8886e3_X270.ms.PM04.asap.cal',overwrite = True)
es.SDcheckSpectra('uid___A002_X8886e3_X270.ms.PM04.asap.cal', spwIds='42',
interactive=False)# Subtracting the baseline
mystep = 6
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl')sdbaseline(infile = 'uid___A002_X8886e3_X270.ms.PM03.asap.cal',
iflist = [42],maskmode = 'auto',thresh = 5.0,
avg_limit = 4,
blfunc = 'poly',order = 1,
outfile = 'uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl',overwrite = True)
es.SDcheckSpectra('uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl', spwIds='42',
interactive=False)
os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl')sdbaseline(infile = 'uid___A002_X8886e3_X270.ms.PM04.asap.cal',
iflist = [42],maskmode = 'auto',
thresh = 5.0,
avg_limit = 4,
blfunc = 'poly',order = 1,
outfile = 'uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl',overwrite = True)
es.SDcheckSpectra('uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl', spwIds='42',interactive=False)# Converting ASAP -> MSmystep = 7
if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl.ms')
sdsave(infile = 'uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl',outfile = 'uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl.ms',
outform = 'MS2')
os.system('rm -Rf uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl.ms')
sdsave(infile = 'uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl',outfile = 'uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl.ms',
outform = 'MS2')# Split and concatenation
mystep = 8

if(mystep in thesteps):
casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
print 'Step ', mystep, step_title[mystep]os.system('rm -Rf uid___A002_X8886e3_X270.ms.cal')concat(vis = [ \'uid___A002_X8886e3_X270.ms.PM03.asap.cal.bl.ms', \'uid___A002_X8886e3_X270.ms.PM04.asap.cal.bl.ms' ], \concatvis = 'uid___A002_X8886e3_X270.ms.cal')Sideband Separation Results
Catepory A:
Performed with scantables from the same antenna but with different offsets.
Parameters:
infiles=['uid___A002_X833c94_X1cb.ms.DA61.asap.cal.bl','uid___A002_X833c94_X662.ms.DA61.asap.cal.bl','uid___A002_X833c94_X976.ms.DA61.asap.cal.bl','uid___A002_X833c94_Xccf.ms.DA61.asap.cal.bl','uid___A002_X8505d8_X15c4.ms.DA61.
asap.cal.bl','uid___A002_X8505d8_X196f.ms.DA61.asap.cal.bl']
sep = sd.sbseparator(infiles)
sep.set_frequency(42, 500, frame='TOPO')
sep.set_dirtol(dirtol=['2arcsec', '2arcsec'])
sep.set_limit(0.2)
sep.set_both(True)
sep.separate('DA_paramset_1.asap')# profile maps
sdplot('DA_paramset_1.asap.imageband', plottype='grid', subplot=66,
outfile='DA_paramset_1.asap.imageband.png')
sdplot('DA_paramset_1.asap.signalband', plottype='grid', subplot=66,
outfile='DA_paramset_1.asap.signalband.png')
A number of variations of the sdplot were also carried out and compared.

The ratio of signalband peak to imageband peak, S/I, is 5.3.
Increasing the rejection ratio by set_limit(0.4), S/I dropped to 4.1.
Recommended choice of set_limit is 0.2 or 0.1. Using 0.2 produces slightly better results compared
to results when 0.1 was adopted. The imageband spactra looks noisier when 0.1 was adopted, but
the signalband is similar to the case of 0.2.
The noise level decreased by a factor of 2, from ~0.5 K to ~0.2 K, after sideband separation.
A quick imaging of the imageband;Band 10 SD separator task failed, I am inquiring of a work around and will update this reportaccordingly.