TIDE Level-Zero Processing Software Introduction

  TIDE Software Meeting - October 31, 1996
                Updated - July 28, 1997

  TIDE Level-Zero Input Datafiles

      MSFC-generated Parameter and List Files:
      MSFC-generated Mass/channel and Instrument Calibration Data
      MSFC-generated ion count mask files
      MSFC-generated sector sensitivity
      EFI spacecraft potential files
      GSFC-generated Common Data Format orbit and attitude files
      GSFC-generated TIDE Level-Zero Data files
          Data Product Description
          Collapse Option Description

  TIDE Level-Zero Processing software

      Location

      C Header Files
          tide_lz_def.h
          tide_lz_glob.h
          tide_lz_parm.h
          tide_lz_func.h 
          tide_lz_stru_def.h
          tide_lz_stru_glb.h 
          tide_lz_stru_ext.h

      C Source Code Summary
          lz_parm.c
          process_parm.c
          tide_lz_main.c 
          init_processing.c
          get_lz.c
          get_spins.c
          get_dp.c 
          ricedcmp.c
          process_mass.c
          mass_conv.c

      TIDE level-zero output file extensions

      TIDE Level-Zero plotting software:
          chromo_summary.pro
          graybar.pro
          load_color_cg.pro
          rgb_wheel.pro

          tide_lz_call.pro
          tide_lz_prompt.pro

          load_color.pro
          color_bar.pro

          esse_summary.pro
          htr_spect.pro
          spect_summary.pro

          moments_summary.pro

          psd_summary.pro
          plot_gsm_orbit.pro
          vel_dist_2_para.pro
          vel_dist_movie.pro
          vel_dist_perp_para.pro
          vel_dist_survey.pro

          Standalone Procedures - used to monitor TIDE/PSI health and safety
              de_spect.pro
              de_sum_spect.pro
              mass_avg_main.pro
              mass_avg_util.pro
              mirror_ratios.pro
              plot_mask.pro
              psi_hk_smry.pro
              tide_hk_smry.pro


TIDE Level-Zero Input Datafiles
-------------------------------

MSFC-generated Parameter and List Files:
  
  The parameter file is a binary structure that tells the level-zero processing
  software what data to retrieve and either plot or save in a file.

  If the list file is used from IDL, it contains the processing start and stop 
  times, the parameter filename, plot types (window,gif,ps), directory where 
  the plot files are to be placed, a process ID, and the names of the 
  level-zero, orbit, attitude, ion mask, spacecraft potential, peaks location,
  and/or mfe magnetic field files used in the processing.  If the list file is 
  used in the standalone tide_lz, it contains the processing start and stop 
  times and the names of the level-zero, orbit, attitude, ion mask, spacecraft 
  potential, peaks location, and/or mfe magnetic field files used in the 
  processing.

MSFC-generated Mass/Channel and Instrument Calibration Data:

  The mass description and mass/channel calibration data are in a text file 
  identifying each mass by amu, m/q, TOF bin boundaries, and mass label plus 
  it has the channel-depended mass peak corrections as a function of mass and
  time.  The file is call mass_calib.vn, where n is the current version.

  The instrument calibration data is a text file containing the mirror_ratio
  and channel-dependent geometric factor and E/Vrpa for ion data prior to 
  12/7/96 and Stops data after 12/7/96.  Data is based on flight calibration.
  The file is call tide_calib.vn, where n is the current version.

MSFC-generated ion count mask files:

  The sun pulse / noise level masks are binary files containing energy, spin, 
  channel-dependent mass correction values.  These files are created every
  other day from data collected near apogee background checks were made.  Mask 
  files for the entire mission are available on the web in 
  http://satyr.msfc.nasa.gov/tmpmo2/mask/

MSFC-generated sector sensitivity:

  The sector sensitivity file, sector_sens.vn, contains sensitivity corrections
  for specific dates.  If the dates are not in the file and the user wants a 
  correction applied, the values are calculated from a series of equations in 
  the C function, calc_sector_sens in init_processing.c

EFI spacecraft potential files
  The EFI spacecraft potential data are in large month-long text files with
  records very spin (6 seconds).  The files are available on the web in 
  http://satyr.msfc.nasa.gov/tmpmo2/scpot/

GSFC-generated Common Data Format orbit and attitude files:
  The orbit data are in day long CDF files, one record every minute.  The
  attitude data are in day long CDF files, one record every 10 minutes.

GSFC-generated TIDE Level-Zero Data files:

Each level-zero file contains one days worth of telemetry data, ~9392 records.
All TIDE records are 5800 bytes long in science and maneuver modes.  The first
record in each file is a header record.  It is followed by a level-zero record
every 9.2 seconds.  Each record has a 300 byte header followed by a major frame.
There are 250 minor frames per major frame.  The TIDE level-zero consists of 
22 bytes of data per minor frame.  The first 3 bytes of each minor frame contain
the spacecraft housekeeping.  All but the 49 bytes assigned to TIDE are zeroed.
The 4th byte in the first 142 minor frames is skipped, with the remaining 18 
containing TIDE science data.  All 19 bytes of the last 108 minor frames contain
TIDE science data.

The TIDE Science Data, 4608 bytes/major frame, is received in spin packets
reported every 6 seconds, ~14400 spins per day.  A spin packet includes a
170-byte spin header followed by a series of data products.  Each spin packet 
starts with a unique pair of sync-bytes.  Each spin header ends in a checksum.
Each data product starts with an 7-byte packet header including a sync byte and
ends with a checksum.  The number of data products per spin packet varies with
the instrument mode and the amount of Rice compression.  The are 190 different
data products:

    dp id   data product description
   -------  ----------------------------------------------------------------
     0-41   collapse option 0, one dp for each mass and sector, 31x32 cnts
    42-83   collapse option 1, one dp for each mass and sector, 16x32 cnts
    84-107  collapse option 2, dp size and content varies
   108-131  collapse option 3, dp size and content varies
   132-155  collapse option 4, dp size and content varies
   156-179  collapse option 5, dp size and content varies
     200    collapse option 6, ion count rate, calibration mode
     201    collapse option 6, starts,stops,start converts,timeouts,resets
   218-220  mass look-up table, instrument mode processor, and data 
            processor memory dumps
   221,222  instrument parameters (three different 222 data products)
     233    sector 0 direct-events
   -------  ----------------------------------------------------------------

TIDE Level-Zero Processing software:

Use the software downlosd page to get the latest version of the software.

C Header Files:

  tide_lz_def.h contains the variable definitions used in the TIDE level-zero 
  processing software.

  tide_lz_glob.h contains the global variables used in the TIDE level-zero 
  processing software including mass and image arrays, the 8-bit to 16-bit 
  decompression array, and the location of housekeeping bytes within a major 
  frame.  The file contains a description of each variable.

  tide_lz_parm.h makes all the global variables external (available to any 
  function in any source file)

  tide_lz_func.h makes all TIDE C functions external

  tide_lz_stru_def.h contains the data structure definitions used to process 
  TIDE level-zero data, tide_lz_stru_glb.h makes the structures global, and
  tide_lz_stru_ext.h makes the structures external.  The structure definitions
  are a good source of data descriptions.  Some of the are:

     structure/union      variable       description
     -------------------  -------------  ---------------------------------
     four_bytes           mdata          byte swapping union
     File_Label_Record    f_label        level-zero file label record 
     Hk                   hk,hk_de[2]    housekeeping data
                          hk_save[16]
     Spin_Packet_Header   sp, sp_ion,    spin packet header
                          sp_save[16]
     Data_Product_Header  dp             data product
     Eff_Num              eff_num        time_of_flight efficiency numbers
     ip                   ip_now,ip_now  instrument parameter data product
                          ip_save[16]
     Ion                  ions[100]      ion identification
     Input_Parameters     pint           processing input parameters
     Plot_Parameters      pprm           plot parameters
     DB_Spin              db1,db[16]     spin database table
     DB_HK                db_hk          housekeeping database table
     -------------------  -------------  ---------------------------------

C Source Code Summary:

  lz_parm.c is the driver program used to create a TIDE level-zero processing 
  parameter file, it calls init_pint and get_process_parm.

  process_parm.c contains the functions used to create the TIDE level-zero 
  processing parameter file including:

     chromo_option     determines parameters used to create TIDE chromograms,
                       it calls flux_units
     flux_units        determines the output units and the flux range
     get_process_parm  prompts the user for information necessary to deteremine 
                       the main TIDE level-zero processing options, it calls:
                       chromo_option, papco_option, and spect_option
     init_pint         initializes the processing option structure   
     papco_option      determines the parameters used to create the moments and
                       high time resolution spectrogram input for the Panel Plot
                       Composer software, it calls flux_units
     spect_option      determines parameters used to create TIDE spectrograms,
                       it calls flux_units

  tide_lz_main.c is the main program that controls how the TIDE level-zero data
  is processed.  All changes made to the software are documented as comments in
  this file.  It calls init_processing, read_mass_desc, read_calibration, 
  calc_sector_sens, get_file_label, get_lz_rec, process_data, and putline

  init_processing.c contains the functions used to set up TIDE level-zero     
  processing, initializes variables, and creates output files, routines 
  include:

     calc_sector_sens  reads or calculates sector sensitivity corrections
     init_ip           initializes instrument parameter structure
     init_masses       initializes ion count (masses) array  
     init_plt_prm      initializes the plot parameter data structure    
     init_processing   controls processing initialization, it calls init_spin, 
                       init_ip, init_plt_prm, set_output, and putline
     init_spin         initializes the spin header structure   
     process_data      starts and controls the processing of level-zero data, it
                       calls get_lz_rec, process_masses, get_spins, 
                       save_de_spect, and putline
     putline           adds comments to the message file (.err)
     read_mass_desc    reads the mass description and mass/channel calibration
                       file and puts it into a data structure     
     set_output        creates output files based on the content of parameter 
                       file, it calls calc_polar_az_trig_func and 
                       save_full_orbit

  get_lz.c contains the functions used to read TIDE level-zero records 
  including:

     get_file_label  reads and processes the first record, the file label 
                     record, in a level-zero data file.    
     get_lz_rec      reads a level-zero record, it calls get_file_label, get_mf,
                     and putline
     get_mf          gets housekeeping and major frame bytes from a level-zero
                     record, it calls hk_conv, hk_print, and putline
  

  get_spins.c contains the functions used to process TIDE spin data   

     calc_temp         converts TIDE housekeeping bytes to temperatures using
                       one of three 5th degree polynomials
     get_dp_packet     gets and processes the next data product, it calls 
                       get_lz_rec, putline, get_xpctd_sz, rice_decomp, 
                       save_masses, what_mass, get_instr_parms,
                       prnt_instr_parms, move_sing_bytes, and log_singles,
     get_spins         the main function used to get and process TIDE spin 
                       packets, it calls get_spin_header, init_masses, 
                       get_dp_packet, and save_moments
     get_spin_header   gets the next spin header, it calls get_lz_rec, putline,
                       get_spin_time, save_db_spin_stru, save_de_spect
     get_spin_time     calculates the time in millisecs at the end of a spin and
                       sets up the various time/date labels
     hk_conv           fills in the housekeeping (hk) structure and saves hk 
                       summary plot data, it calls calc_temp
     hk_print          creates the TIDE engineering data text display
     save_db_spin_stru saves spin information in the database parameter 
                       structure     
     save_de_spect     saves the direct-event color spectrograms, it calls
                       get_orbit_data
     save_moments      saves spin header centroid data and associated mass and 
                       instrument parameter data in a text file
     set_labels        determines the ion species and energy labels used by the
                       TIDE plot routines 

  get_dp.c contains the functions used to process TIDE data products

     get_instr_parms   fills in the instrument parameter data structure, the 
                       function is a modification of software provided by SwRI  
     log_singles       converts calibration mode singles event data from 8-bit
                       to 16-bit integers, logs the results, and saves them for
                       plotting    
     move_sing_bytes   puts the calibration mode singles event 8-bit words into
                       their correct array locations
     prnt_instr_parms  creates the instrument parameter text display
     save_ion_cnts     saves ion count rates for selected masses and times in a
                       large text file, one file per channel
     save_mass_avg     averages ion count rates over energy and spin angle and 
                       saves them as a function of time
     save_mass_sum     sums counts over energy and spin for a user-specfied 
                       time range   
     save_masses       uncollapses the ion count rate data products

  process_mass.c contains the functions used to process TIDE science mode mass
  data (collapse options 0 - 5), including:

     chromo_energy       calculates energy based TIDE chromograms, it calls 
                         change_units
     chromo_polar        calculates polar angle based TIDE chromograms, it calls
                         change_units
     chromo_spin         calculates spin angle based TIDE chromograms, it calls 
                         change_units
     create_image        creates and saves spin-averaged flux images, it calls 
                         calc_flux, get_oa_data, find_ram, set_labels,
                         chromo_polar, chromo_spin, chromo_energy, spect_data,
                         get_orbit_data, htr_rotated_ions, putline
     find_ram            uses spacecraft velocity (gse) and declination and 
                         right ascension to calculate ram velocity in the 
                         spacecraft fixed coordinate system
     flight_calibration  calculates ion count sums over spin angle for each 
                         energy and polar angle when the mirror steps down
     get_oa_data         reads orbit and attitude CDF files and extracts 
                         information necessary to calculate the velocity 
                         coordinate transformations
     get_orbit_data      reads orbit files and extracts plot label information 
     get_scpot_orbit     reads the spacecraft potential from the monthly EFI
                         files, it calls is_between, compare_time_to, and
                         time_diff
     htr_rotated_ions    fills in data missing as a result of rotated ions in 
                         the high time resolution spectrograms
     process_masses      controls the processing of mass data, it calls 
                         get_spins, get_scpot_data, save_mass_avg,
                         save_mass_sum, save_ion_cnts, psd_distribution,
                         flight_calibration, save_de_spect, create_image, and
                         putline
     psd_distribution    creates velocity distribution images parallel and/or
                         prependicular to the magnetic field, it calls
                         calc_flux, moments_calc, get_oa_data, and find_ram
     save_full_orbit     saves orbit data for a 17.5 hour time period for 
                         spectrogram and velocity distribution plots
     spect_data          calculates energy / spin angle mass spectrograms, it 
                         calls change_units 
     what_mass           determines what masses were reported based on the data
                         product id number
     is_between,         three functions used in get_scpot_data
     compare_time_to,
     time_diff

  mass_conv.c contains the functions used to change mass units, correct masses,
  calculate moments, and convert coordinate systems including:

     calc_flux               calculates mirror ratio calibration data, corrects
                             ion counts, and converts counts to user-specified 
                             flux units
     calc_polar_az_trig_func calculates the cosines and sines of the polar and
                             spin angle bins for use in calculating moments    
     change_units            converts ion counts to hertz   
     moments_calc            calculates plasma density, velocity, and 
                             temperature data, it calls get_oa_data and find_ram
     read_calibration        reads the instrument calibration data file

  ricedcmp.c contains the routines provided by SWRI to decompress the Rice
  compressed ion count rate data products including:
     get_bit     
     get_multi_bits    
     rice_decomp 
     get_xpctd_sz
     dcompnocode 
     dcompfs     
     dcompcfs    
     dcompcfs_bar
     dcomphigh   
     dcomplow    
     decompress  

TIDE level-zero output file extensions
--------------------------------------

The output created by tide_lz will vary depending on the processing done.  The
format for the filename is either 

      tyymmddh1m1_h2m2.ext
   or tyymmddh1m1_h2m2_mm.ext

where yymmdd is the start date, h1m1 is the start time, and h2m2 is the stop
time.  If the file contains data for a single mass, mm is the mass to charge
ratio of the ion or it is 00 for Singles or sp for Stops.  The extention is a
brief indication of the content of the file.  Possible extensions include:

   ext       file type
   --------  -------------------------------------------------------------------
   err       message/error file - contains information about the processing

   esp       chromogram - energy on horizontal axis, spin angle on vertical 
               axis, color based on polar angle, each panel is a mass
   sep       chromogram - spin angle on horizontal axis, energy on vertical 
               axis, color based on polar angle, each panel is a mass
   pes       chromogram - polar angle on horizontal axis, energy on vertical 
               axis, color based on spin angle, each panel is a mass
   pse       chromogram - polar angle on horizontal axis, spin angle on vertical
               axis, color based on energy, each panel is a mass

   espm      spectrogram - energy on horizontal axis, spin angle on vertical 
               axis, the panels are polar angle channels, one mass per page
   esmp      spectrogram - energy on horizontal axis, spin angle on vertical 
               axis, the panels are masses, one polar angle per page
   sepm      spectrogram - spin angle on horizontal axis, energy on vertical 
               axis, the panels are polar angle channels, one mass per page
   semp      spectrogram - spin angle on horizontal axis, energy on vertical 
               axis, the panels are masses, one polar angle per page
   esse      spectrogram - energy/spin and spin/energy panels for Stops plus 
               detailed orbit information

   ehtr      high time resolution spectrograms - counts summed over spin and 
               polar angle and display as a function of energy and time, each
               panel is a mass
   shtr      high time resolution spectrograms - counts summed over energy and 
               polar angle and display as a function of spin angle and time, 
               each panel is a mass
   phtr      high time resolution spectrograms - counts summed over energy and
               spin angle and display as a function of polar angle and time, 
               each panel is a mass
   esphtr    high time resolution spectrograms - one mass per page, panels are
               all three types of htr spectrograms
   eshtr     high time resolution spectrograms - one mass per page, panels are
               energy and spin angle htr spectrograms

   mom       moments text file
   psd       phase space density velocity distribution plots
   psd_txt   text file containing vx, vy, vz, and phase space density values for
             each energy / spin angle / polar angle count

   eng       TIDE engineering data text display
   ip        TIDE instrument parameters text display
   mir_rat   text file containing TIDE mirror ratios
   bfld      magnetic azimuth and elevation data

   hk        TIDE/PSI housekeeping bytes
   hk_tid    TIDE housekeeping data
   hk_psi    PSI housekeeping data

   sg_de     direct-event spectrogram
   de_sum    summed direct-event summary spectrogram
   de_chn    direct-event text file for channel n

   mss_avg   ion counts summed over energy and spin angle
   mss_sum   ion counts summed over energy and both spin and polar angle
   mss_chn   ion count rates for channel n

   db_spn    database spin table information
   db_hk     database housekeeping table information
   dmp       telemetry byte dumps

TIDE Level-Zero plotting software
---------------------------------

  tide_lz_call.pro        two main control procedures
  tide_lz_prompt.pro

  chromo_summary.pro      procedures used to create chromograms
  graybar.pro
  load_color_cg.pro
  rgb_wheel.pro

  moments_summary.pro     plots moments

  load_color.pro          procedures used by spectrogram and velocity 
  color_bar.pro           distribution routines

  esse_summary.pro        procedures used to plot mass spectrograms
  htr_spect.pro
  spect_summary.pro

  psd_summary.pro         procedures used to create velocity distribution plots
  plot_gsm_orbit.pro
  vel_dist_2_para.pro
  vel_dist_movie.pro
  vel_dist_perp_para.pro
  vel_dist_survey.pro

  Standalone Procedures - used to monitor TIDE/PSI health and safety

  de_spect.pro       creates daily direct-event spectrograms
  de_sum_spect.pro   creates monthly direct-event spectrograms
  mass_avg           a command file that runs mass_avg_main.pro and 
                     mass_avg_util.pro to create the count rate line plots 
                     averaged over energy and spin.
  mirror_ratios.pro  creates a plot of mirror ratio as a function of time and
                     energy step
  plot_mask.pro      creates 3-D surface plot of ion mask for a given mass and
                     channel
  psi_hk_smry.pro    creates PSI housekeeping daily summary plots
  tide_hk_smry.pro   creates TIDE housekeeping daily summary plots