Xlog user manual
 
                                Wolfgang Schwegmann
 
                                2006.12.15
 
 
                                Abstract
 
  This document describes program Xlog used for creation of cable and weather
calibration files for input into VLBI data bases by program DBCAL. It explains
the usage of Xlog and how the program works.
 
 
Questions and comments about this guide should be sent to:
 
Wolfgang Schwegmann ( wolfgang.schwegmann@bkg.bund.de )
 
 
 
                                Table of contents:
 
 
1 ................ Introduction
 
 
2 ................ Usage
 
 
3 ................ Format of Xlog configuration file
 
   3.1 ........... Description of directives of Xlog configuration file
   3.2 ........... Example configuration file
 
4 ................ Installing Xlog
 
 
5 ................ Restrictions
 
 
________________________________________________________________________________
 
        1 Introduction
        ==============
 
  Xlog is the automated version of the program PWXCB; it performs most tasks
done by Pwxcb automatically.
 
  Xlog creates cable and weather calibration files to be used as input for VLBI
databases by program Dbcal. Xlog extracts cable and weather data from Mark 3/4/5
log files, determines the cable sign, stores the data in weather and cable
calibration files and creates the control file for Dbcal. Additionally, the
program can be used to check and plot the data of existing cable and weather
calibration files or to extract the system temperature data from log files.
The data will be checked with respect to their plausibility, wrong points will
be deleted automatically and in case of suspicious data, the analyst's
attention will be directed to the data. Xlog can automatically download all
necessary files (e.g. station log files) from an IVS Data Center. The program
can be used to generate plots of the calibration data in GIF or POSTSCRIPT
format which can be sent directly to a local printer.
 
  Xlog supports two user modes: in the normal mode progress information will
be printed to the screen and the extracted calibration data can be edited via
an user interface. This mode is meant for interactive processing under control
of the analyst. In the silent mode everything is done automatically and only
error messages will be displayed on the screen. This mode is meant for
automatic processing under the control of Xlog. For the user interface the
"Dialogue Graphic Interface (DiaGI)" developed by Leonid Petrov, NASA Goddard
Space Flight Center (GSFC) has been applied.
 
To control the usage of Xlog several command line options are available as well
as a configuration file, where all parameters necessary for the usage of Xlog
can be specified. These options and parameters are described in the following
sections.
 
 
        2 Usage
        =======
 
Usage: xlog [-h] [-help] -s <ivs-session-code> [-c <config_file>]
            [-p <protocol_file>] [-v <verbosity_level>]
            [-r] [-b] [-x] [-f] [-u] [-d]
 
Options:
 
-help -- prints this information to screen.
 
-h    -- prints help on available options to screen.
 
-s    -- IVS session code of the experiment (this option is mandatory).
 
-c    -- specifies configuration file for Xlog, the default is
         "${MK5_ROOT}/local/{Center_label}.xlg". The format of the configuration
         file is described in section "Format of Xlog configuration file".
 
-p    -- specifies name of Xlog protocol file; a default name is being used if
         not specified. The default name is being build from the 6-character IVS
         session code followed by the suffix ".xlp"
         (e.g. for experiment IVS-R4135: "R4135.xlp").
         The default directory is specified by keyword "CONTROL_DIR" in the
         Xlog configuration file.
 
-v    -- sets verbosity level:
         0 - silent mode, only error messages will be printed to screen.
         1 - normal mode (default), progress information will be printed to
             screen.
 
-r    -- run DBCAL after processing log-files; DBCAL control file is being
         generated before as long as -b switch is not selected. The name of
         the control file is being build from the 6-character IVS
         session code followed by the suffix ".dbc"
         (e.g. for experiment IVS-R4135: "R4135.dbc").
         The default directory is specified by keyword "CONTROL_DIR" in the
         Xlog configuration file.
 
-b    -- don't build DBCAL control file (option "-r" will be suppressed).
 
-x    -- to process existing calibration files.
 
-f    -- fast mode: plausibility check of the data won't be done.
 
-u    -- update file "xlog_stations.txt" and set of master schedule files.
 
-d    -- discard processed data, i.e. don't save the data.
 
        3 Format of Xlog configuration file
        ===================================
 
Syntax of the configuration file:
 
1) lines starting with ## are recognized as comments and ignored.
2) directives for Xlog: lines starting with # and which are not a comment line.
   Directive consists of three or more words separated by one or more blanks.
 
   Word1 -- symbol # -- directive attribute
   Word2 -- keyword
   Word3,4... value(s) of the keyword
 
   Each keyword has ending symbol ":" which is considered as a part of the
   keyword. Keywords are case-insensitive, but values are case-sensitive.
   There are mandatory and optional keywords which may follow in arbitrary
   order. Default values will be used for not specified optional keywords.
   All supported keywords are listed in the next subsection.
 
                3.1 Description of directives of Xlog configuration file
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
Mandatory keywords:
 
LOG_DIR:          Directory where log-files to be processed by Xlog are stored.
 
CAL_DIR:          Directory where calibration files generated by Xlog will be
                  stored. When Xlog is called with -x switch calibrations will
                  be read from this directory.
 
CONTROL_DIR:      Directory where Xlog protocol files and DBCLA control files
                  generated by Xlog will be stored.
 
SKED_DIR:         Directory where schedule files of the VLBI experiments are
                  being stored. The schedule files are used by Xlog to read
                  read the system temperature at a VLBI station from the
                  stations log-file (cf. keyword EXTRACT_TSYS).
 
MASTER_DIR:       Directory where master files are stored. When Xlog is being
                  called with -u switch the current set of master files is
                  being downloaded from the URL specified by keyword
                  URL_IVS_CONTROL.
 
URL_IVS_AUX:      URL for the root directory in the IVS Data Center which
                  contains the station log-files. Value should end with letters
                  "aux/". Since there are several IVS Data Centers user can
                  select the Data Center to which connection is the best.
                  If value is NO then log-files won't be downloaded.
 
URL_IVS_CONTROL:  URL for the root directory in the IVS Data Center which
                  contains master files and file "ns_codes.txt". Value should
                  end with letters "ivscontrol/". Since there are several IVS
                  Data Centers user can select the Data Center to which
                  connection is the best.
                  If value is NO then switch -u is not possible.
 
TMP_DIR:          Directory where temporary files created by Xlog are being
                  stored.
 
WGET_EXE:         Filename with path of program wget. Program wget should be
                  installed before calling Xlog.
 
GZIP_EXE:         Filename with path of program gzip. Program gzip should be
                  installed before calling Xlog.
 
STATIONS_FILE:    Filename with path of file with station abbreviations
                  (IVS station codes, station names as they appear in the
                  log-files) used to build calibration filenames for program
                  DBCAL automatically. The file also contains information about
                  the default cable calibration sign of the station.
                  When Xlog is being called with -u switch this file is being
                  updated with respect to the file "ns_codes.txt" downloaded
                  from the ftp directory specified by keyword URL_IVS_CONTROL.
 
Optional keywords:
 
LOG_DIR_LONG:     Possible values: YES or NO (default). In the latter case
                  station log-files are stored in directory specified by
                  keyword LOG_DIR:. If YES log-files are stored in a session
                  specific sub-directory of the above log-file directory.
                  The name of the sub-directory corresponds to the IVS
                  session code.
                  Example:
                      LOG_DIR:      /data/logs/
                      LOG_DIR_LONG: YES
                      IVS session code is R4116
                  Directory for log-files is: /data/logs/r4116/
 
CAL_DIR_LONG:     Possible values: YES or NO (default). In the latter case
                  calibration files generated by Xlog are stored in directory
                  specified by keyword CAL_DIR:. If YES calibration files are
                  stored in a session  specific sub-directory of the above
                  calibration file directory. The name of the sub-directory
                  corresponds to the IVS session code.
                  Example:
                      CAL_DIR:      /data/cals/
                      CAL_DIR_LONG: YES
                      IVS session code is R4116
                  Directory for calibration files is: /data/cals/r4116/
 
COMPRESS_FILES:   Possible values: YES or NO (default). In the first case
                  log-files will be compressed after processing.
 
EXTRACT_TSYS:     Possible values: YES or NO (default). In the latter case
                  system temperature data won't be extracted.
 
EXTRACT_MET:      Possible values: YES (default) or NO. In the latter case
                  weather data won't be extracted.
                  If Dbcal control file will be generated, i.e. command line
                  switch "-b" is not selected, the value for this keyword will
                  be set to YES.
 
EXTRACT_CABLE:    Possible values: YES (default) or NO. In the latter case
                  cable calibration data won't be extracted.
                  If Dbcal control file will be generated, i.e. command line
                  switch "-b" is not selected, the value for this keyword will
                  be set to YES.
 
PLOT_DIR:         Directory where plots of the calibration data should be
                  stored. If value is NO (default) then plots won't be
                  generated. Otherwise plots are saved in directory:
                     "PLOT_DIR"/"YearOfSession"/"IVS session code"/xlog/
                  Additionally Xlog protocol file and DBCAL control file
                  are stored in directory:
                     "PLOT_DIR"/"YearOfSession"/"IVS session code"/control/
                  The sub-directories will be created in case they do not
                  exist.
 
PLOT_FMT:         Format of plots of calibration data.
                  Possible values: gif (default) or ps.
 
PRINT_FILES:      Possible values: YES or NO (default). If YES then plots of
                  the calibration files will be sent to printer using command
                  specified by keyword PRINT_COMMAND:.
 
PRINT_COMMAND:    Command to print plots of calibration files.
                  Default: "".
 
GAP:              Xlog prints a warning message if there are gaps in the data
                  which are greater than the value of this keyword (in hours).
                  Possible values: INTEGER greater than 0 and less equal 24.
                  Default value: 3
 
GAP_START:        Possible values: .TRUE. (default) or .FALSE.. If .TRUE. then
                  Xlog prints a warning message if the difference in time
                  (in hours) between start of experiment and first point with
                  data is greater than the value of keyword GAP:.
 
GAP_END:          Possible values: .TRUE. (default) or .FALSE.. If .TRUE. then
                  Xlog prints a warning message if the difference in time
                  (in hours) between last point with data and end of experiment
                  is greater than the value of keyword GAP:.
 
MAX_DEL:          Xlog sends a warning message if more than (MAX_DEL)% of all
                  points with data were deleted. Possible values: INTEGER
                  greater than 0 and less equal 100.
                  Default value: 20
 
MAX_EQUAL:        Xlog sends a warning message if more than (MAX_EQUAL)% of all
                  points with data have got the same values. Possible values:
                  INTEGER greater than 0 and less equal 100.
                  Default value: 60
 
                3.2 Example configuration file
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
#############################################################################
##                                                                         ##
##  Xlog configuration file for BKG Frankfurt                              ##
##                                                                         ##
##                                                             14-DEC-2006 ##
##                                                                         ##
#############################################################################
##
## Mandatory keywords:
##
# LOG_DIR:         /disk1/mk5_data/log/
# CAL_DIR:         /disk1/mk5_data/delog/
# CONTROL_DIR:     /disk1/mk5_data/control/
# SKED_DIR:        /disk1/mk5_data/log/
# MASTER_DIR:      /disk1/mk5_data/master/
# URL_IVS_AUX:     ftp://cddis.gsfc.nasa.gov/vlbi/ivsdata/aux/
# URL_IVSCONTROL:  ftp://cddis.gsfc.nasa.gov/vlbi/ivscontrol/
# TMP_DIR:         /disk1/mk5_data/tmp/
# WGET_EXE:        /usr/bin/wget
# GZIP_EXE:        /bin/gzip
# STATIONS_FILE:   /disk1/xlog/xlog/xlog_stations.txt
##
## Optional keywords:
##
# LOG_DIR_LONG:    YES
# CAL_DIR_LONG:    YES
# COMPRESS_FILES:  NO
# EXTRACT_TSYS:    NO
# EXTRACT_MET:     YES
# EXTRACT_CABLE:   YES
# PLOT_DIR:        NO
# PLOT_FMT:        gif
# PRINT_FILES:     NO
# PRINT_COMMAND:   lp -d laser3
# GAP:             5
# GAP_START:       .TRUE.
# GAP_END:         .TRUE.
# MAX_DEL:         20
# MAX_EQUAL:       60
#!
 
 
        4 Installing Xlog
        =================
 -------------------
 
  Firstly DiaGI should be installed. For details see:
http://gemini.gsfc.nasa.gov/solve_root/help/diagi_1.hlp
 
 
        5 Restrictions
        ==============
 ---------------
 
1. No more than 50 files can be processed in the same run (this is specified
   by parameter MNIC in file xlog.i).
 
2. No more than 3000 data points should be contained in the same file (this
   is specified by parameter MNDP in xlog.i).