Package: dask-ms
ConstructsxarrayDatasetsfrom CASA Tables viapython-casacore.
TheVariablescontained in theDatasetaredaskarrays backed by
deferred calls tocasacore.tables.table.getcol.Supports writingVariablesback to the respective column in the Table.The intention behind this package is to support the Measurement Set as
a data source and sink for the purposes of writing parallel, distributed
Radio Astronomy algorithms.InstallationTo install withxarraysupport:$pipinstalldask-ms[xarray]Withoutxarraysimilar, but reduced Dataset functionality is replicated
in dask-ms itself. Expert users may wish to use this option to reduce
python package dependencies.$pipinstalldask-msDocumentationhttps://dask-ms.readthedocs.ioGitter Pagehttps://gitter.im/dask-ms/communityExample Usageimportdask.arrayasdafromdaskmsimportxds_from_table,xds_to_table# Create xarray datasets from Measurement Set "WSRT.MS"ds=xds_from_table("WSRT.MS")# Set the flag Variable on first Dataset to it's inverseds[0]['flag']=(ds[0].flag.dims,da.logical_not(ds[0].flag))# Write the flag column back to the Measurement Setxds_to_table(ds,"WSRT.MS","FLAG").compute()printds[<xarray.Dataset>Dimensions:(chan:64,corr:4,row:6552,uvw:3)Coordinates:ROWID(row)int32dask.array<shape=(6552,),chunksize=(6552,)>Dimensionswithoutcoordinates:chan,corr,row,uvwDatavariables:IMAGING_WEIGHT(row,chan)float32dask.array<shape=(6552,64),chunksize=(6552,64)>ANTENNA1(row)int32dask.array<shape=(6552,),chunksize=(6552,)>STATE_ID(row)int32dask.array<shape=(6552,),chunksize=(6552,)>EXPOSURE(row)float64dask.array<shape=(6552,),chunksize=(6552,)>MODEL_DATA(row,chan,corr)complex64dask.array<shape=(6552,64,4),chunksize=(6552,64,4)>FLAG_ROW(row)booldask.array<shape=(6552,),chunksize=(6552,)>CORRECTED_DATA(row,chan,corr)complex64dask.array<shape=(6552,64,4),chunksize=(6552,64,4)>PROCESSOR_ID(row)int32dask.array<shape=(6552,),chunksize=(6552,)>WEIGHT(row,corr)float32dask.array<shape=(6552,4),chunksize=(6552,4)>FLAG(row,chan,corr)booldask.array<shape=(6552,64,4),chunksize=(6552,64,4)>TIME(row)float64dask.array<shape=(6552,),chunksize=(6552,)>SIGMA(row,corr)float32dask.array<shape=(6552,4),chunksize=(6552,4)>SCAN_NUMBER(row)int32dask.array<shape=(6552,),chunksize=(6552,)>INTERVAL(row)float64dask.array<shape=(6552,),chunksize=(6552,)>OBSERVATION_ID(row)int32dask.array<shape=(6552,),chunksize=(6552,)>TIME_CENTROID(row)float64dask.array<shape=(6552,),chunksize=(6552,)>ARRAY_ID(row)int32dask.array<shape=(6552,),chunksize=(6552,)>ANTENNA2(row)int32dask.array<shape=(6552,),chunksize=(6552,)>DATA(row,chan,corr)complex64dask.array<shape=(6552,64,4),chunksize=(6552,64,4)>FEED1(row)int32dask.array<shape=(6552,),chunksize=(6552,)>FEED2(row)int32dask.array<shape=(6552,),chunksize=(6552,)>UVW(row,uvw)float64dask.array<shape=(6552,3),chunksize=(6552,3)>Attributes:FIELD_ID:0DATA_DESC_ID:0]LimitationsMany Measurement Sets columns are defined as variably shaped,
but the actual data is fixed.dask-mswill infer the shape of the
data from the first row and must be consistent
with that of other rows.
For example, this may be issue where multiple Spectral Windows
are present in the Measurement Set with differing channels
per SPW.dask-msworks around this by partitioning the
Measurement Set into multiple datasets.
The first row’s shape is used to infer the shape of the partition.
Thus, in the case of multiple Spectral Window’s, we can partition
the Measurement Set by DATA_DESC_ID to create a dataset for
each Spectral Window.