Package: nds2utils
nds2utilsnds2utils is a user interface for the python nds2 client.nds2 error messages and data structures can be hard to parse.
nds2utils provides helper functions, defaults, warnings, and error messages for users trying to get ligo data with python.
Git repository is athttps://git.ligo.org/craig-cahillane/nds2utilsFeaturesQuick and easy data acquisition using nds2.acquire_data()Acquisition of data in real-timestitch_real_time_data()Automatic computation of power spectral densities, cross spectral densities, transfer functions, and coherence'get_psds(), get_csds()'Quick plotting of your dataplot_ASDs, plot_TFs_A, plot_TFs_AInteractive legend - click on line in legend to toggle itLogbinningCalibration using zpkExamplesDocstrings.  If in doubt about a function, look in the docstrings, using, e.g.nu.acquire_data?InstallationThis library requires numpy, scipy, nds2-client, python-nds2-client, and matplotlib.You can install thends2utilsAnaconda environment in this directory'senvironment.ymlby runningconda env create -f environment.ymlAlternatively, you can either conda install or pip installconda install numpy scipy nds2-client python-nds2-client matplotlibpip install numpy scipy nds2-client python-nds2-client matplotlibTested versions at the time of this publishing:python              3.7.3numpy               1.17.0scipy               1.3.1nds2-client         0.16.3python-nds2-client  0.16.4matplotlib          3.1.1You can install nds2utils usingpipathttps://pypi.org/project/nds2utils/pip install nds2utilsNew in version 0.0.25Added.hdf5saving and loading support usingdeepdish.If you want this functionality, please runpip install deepdishorcdto this directory on your computer and runconda env update -f environment.ymlwhich will update yournds2utilsenvironment to have deepdish.Quick StartYou will have tokinit albert.einstein@LIGO.ORGto start a kerberos authenticated session to allow you to access data.Get raw time series dataimportnds2utilsasnuchannels=['H1:CAL-DELTAL_EXTERNAL_DQ','H1:LSC-REFL_SERVO_ERR_OUT_DQ']gps_start=1256771562gps_stop=1256771712data_dict=nu.acquire_data(channels,gps_start,gps_stop)cal_deltal_external_data=data_dict['H1:CAL-DELTAL_EXTERNAL_DQ']['data']# this is where the data is stored# Plot raw time series datanu.plot_raw_data(data_dict,seconds=150,downsample=2**10)Get data and take PSDs, CSDs, and TFs of time series dataimportnds2utilsasnuchannels=['H1:CAL-DELTAL_EXTERNAL_DQ','H1:PSL-ISS_SECONDLOOP_RIN_OUTER_OUT_DQ']gps_start=1256771562gps_stop=1256771712binwidth=0.1# Hz, frequency vector spacingoverlap=0.5# FFT overlap ratiodata_dict=nu.get_csds(channels,gps_start,gps_stop,binwidth,overlap)nu.plot_asds(data_dict,title='Uncalibrated and Logbinned DARM and CARM spectra',logbin=True)Slow StartValid Host Sever:Port Number combos:host_server:port_number
h1nds1:8088                   # only good on LHO CDS computers
h1nds0:8088                   # only good on LHO CDS computers
nds.ligo-wa.caltech.edu:31200 # LHO frame cache
nds.ligo-la.caltech.edu:31200 # LLO frame cache
nds.ligo.caltech.edu:31200    # Caltech frame cacheImportant functions you will want to use:nu.find_channels('H1:CAL*DQ')# Accepts a channel glob, returns matching channelsnu.acquire_data()# Accepts array of channel names, gps_start, and gps_stop.  Returns dictionary containing the data.Open an ipython terminal, and import the data_utils library by runningimportnds2utilsasnuYou can't remember the exact name for the calibrated DARM channel you want, so you runnu.find_channels('H1:CAL*DQ')You see the channel you want in the output,H1:CAL-DELTAL_EXTERNAL_DQ.You want to look at the DARM and intensity noise witness channel amplitude spectral density and transfer function at the time there was excess intensity noise at LIGO Hanford.You happen to know the gps times you want to look at.You rerunfind_channels('H1:PSL-ISS_SECONDLOOP*DQ')and find the out of loop intensity noise witness channel,H1:PSL-ISS_SECONDLOOP_RIN_OUTER_OUT_DQ:channels=np.array(['H1:CAL-DELTAL_EXTERNAL_DQ','H1:PSL-ISS_SECONDLOOP_RIN_OUTER_OUT_DQ'])gps_start=1256805122duration=100# secondsgps_stop=gps_start+durationbinwidth=1# Hz, frequency bin spacing for the ASDs and TFs.overlap=0.5# fraction between 0 and 1.  overlap ratio for the data window.# Get time-series of data and sampling frequency,# and calculate PSDs, ASDs, CSDs, and TFs between all channels# stored in a dictionary with channel names as keys, and dictionaries for values.data_dict=nu.get_csds(channels,gps_start,gps_stop,binwidth,overlap)data_dictis a dictionary of dictionaries.
The top level dictionary uses channel names as keys, and returns values which are dictionaries containing the data relevant to the channel key.
You define a second-level dictionary,darmDict, to look only at the darm dictionary:print(data_dict.keys())darmD_dict=data_dict['H1:CAL-DELTAL_EXTERNAL_DQ']print(darm_dict.keys())data_dictcontains the following keys:Raw data keys:'data'= the raw data from the channel'fs'= sampling frequency of the data'gpsStart'= gps time representing the first data point acquired'duration'= time length of data acquiredPSD data keys:'averages'= int.   number of averages for the power spectral density calculation'binwidth'= float. Hz, frequency bin spacing for the ASDs and TFs'overlap'= float. fraction between 0 and 1.  overlap ratio for the data window.'fftLen'= float. length in seconds for the FFTs. Used in scipy.welch() PSD calculation.'nperseg'= int.   number of samples per FFT segment.'ff'= array. frequency vector for this channels PSDs'PSD'= array. power spectral density in units^2/Hz'df'= float. Hz. delta frequency vector, should be the same as binwidth.CSD and TF data keys: (names of all other channels, in this case,'ISS_SECONDLOOP_RIN_OUTER_OUT_DQ':'H1:PSL-ISS_SECONDLOOP_RIN_OUTER_OUT_DQ'= dictionary containing CSD, TF, and coherence info.We make another definition of the third level dictionary to get CSD and TF info:iss2darmDict=data_dict['H1:CAL-DELTAL_EXTERNAL_DQ']['H1:PSL-ISS_SECONDLOOP_RIN_OUTER_OUT_DQ']print(iss2darm_dict.keys())iss2darm_dictcontains the following keys:'ff'= array. frequency vector for these two channels CSDs, TFs, and coherences'CSD'= array. cross spectral density in units $\mathrm{unit_a} * \mathrm{unit_b}/\mathrm{Hz}$'TF'= array. transfer function in $\mathrm{unit_b} / \mathrm{unit_b}$'coh'= array. power coherence, unitless $\gamma^2 = |\mathrm{CSD}_{AB}|^2 / (\mathrm{PSD}_A * \mathrm{PSD}_B)$The convention for the cross spectral density: The first channel is the complex conjugated one.data_dict[chanB][chanA]['CSD']= $\mathrm{CSD}_{BA}$ = $\langle{b^*}\rvert{a}\rangle$The convention for the transfer function: chanB is the first channel in the data_dict, chanA is the second:data_dict[chanB][chanA]['TF']= H(f) = $\frac{\mathrm{CSD}_{AB}}{\mathrm{PSD}_{A}}$ = $\frac{\langle{a^*}\rvert{b}\rangle}{\langle{a^2}\rangle}$The convention for the coherence: coherence is the power coherence:data_dict[chanB][chanA]['coh']= $\gamma^2 = \frac{|\langle{a^*}\rvert{b}\rangle|^2}{\langle{a^2}\rangle \langle{b^2}\rangle}$If you add a calibration using calibrate_chan(), you will have new keys in your dictionary:'calPSD''calCSD''calTF'ChangelogMajor changes in versions0.0.25Addeddeepdishdependency0.1.0Removed individual storage of ASDs to conserve space, only PSDs are stored now.Changeddata_dictkey namefftLentofft_timeto be more descriptive.Addedcummulative_rms()function which theRMS()function now depends on.cummulateive_rms()usesscipy.integrate.cumtrapz()to estimate the root mean squared value from a PSD.Changed the names of many functions to be more consistent, more clear, and not haveCaMElCaSEbut insteadunderscore_basednames and argumentsdata_PSDs->calc_psdsdata_CSDs->calc_csdsdata_TFs->calc_tfslogbin_ASD->logbin_psdlinear_log_ASD->linear_log_psdget_PSDs->get_psdsget_CSDs->get_csdsget_TFs->get_tfsquick_ASD_grab->quick_asd_grabquick_PSD_grab->quick_psd_grabquick_CSD_grab->quick_csd_grabquick_TF_grab->quick_tf_grabplot_ASDs->plot_asdscompare_ASDs->compare_asdsplot_TF->plot_tfplot_TFs_A->plot_tfs_aplot_TFs_B->plot_tfs_b0.1.1Added and partially tested conditioned spectra and partial coherence functions todata_utils.py.Fixedmake_interactive_svg.pyto work for multiple legends on multiple subplots.