Package: fesh2
Fesh2: Geodetic VLBI schedule file management and processingWhat is it?Fesh2is a Python package that provides automated schedule file preparation.
It requires an installation of theNASA Field System.Fesh2runs in a terminal window and regularly checks IVS schedule
repositories for new or updated versions of Master files and schedule files for
one or more specified stations. A check for the latest version of the Master
file(s) is done first, but skipped if the time since the last check is less than
a specified amount.
Similarly, checks on schedule files are only done if the time since the last
check exceeds a specified time. If new or updated schedules are found, they are
optionally processed withDrudgto producesnp,prcandlstfiles. By default, once the files have been
checked,fesh2will provide a summary and then go into a wait state before
carrying out another check.Fesh2can also be run once for a single
check or
status report and not go into a wait state.Fesh2can also be run
in amonitmode that gives a continually updating status report but
does not process any files. Multiple
instances can be run
simultaneously. If drudg output (snporprcfiles) have been modified by the user and a new schedule becomes available
,fesh2will download the file but not overwrite Drudg output, but it will
warn the user.Fesh2can be run as a foreground application or as a service in the
background.CompatibilityFesh2code is compatible with Python versions 3.8 and above.Note thatfesh2is not compatible with
Python version 2.Note that if you're
using an old version of Debian (Jessie or earlier) then you will need to
install python 3.8 or higher by hand. This could be done inside a virtual
environment by a user or on the system as root.InstallationFesh2is a stand-alone add-on to theField Systemso installation must be
carried out separately.PycURL dependencyFesh2depends on the python CURL
library (PycURL)
which should be installed automatically when you install fesh2. However, PycURL
depends on the Python development libraries, and it won't install if they're not
there. On a Debian machine,apt-get install python3-devshould do the trick. You may also need these:apt-get install libcurl4-openssl-dev libssl-devPlease make sure the above dependencies are satisfied before proceeding.Installation should be carried out under theprogaccount. You may wish to
install fesh2 within a python virtual environment, but regardless, the
installation commands are the same:Installing with pip:Run the following:pip install fesh2You will then need to edit thefesh2configuration file for your station(s). More information on configuration is provided below.A template configuration filefesh2.configis provided inthe fesh2 GitHub
repositoryand a copy of the contents is
in the Appendix at the end of this document.fesh2.configshould be
placed in/usr2/controland edited to suit your site.ConfigurationFesh2will take its configuration in the following priority order from
lowest to highest:Two config files, in this order:/usr2/control/skedf.ctl/usr2/control/fesh2.configEnvironment variablesCommand-line parametersNote that Fesh2 will only run Drudg if it DoDrudg is set to True and it is
configured appropriately. See the Drudg notes below.If you have the configuration files in a different location to/usr2/control,
use the command-line options--ConfigFileand--SkedConfigFileto set them
appropriately.Environment VariablesThe following environment variables are recognised byfesh2:NETRC_FILEandCOOKIES_FILE: the locations of the.netrcand.urs_cookiesfiles, used by CURL for the https protocol. Curl sets these to files in the
home directory by default. More information is given below on Curl
configuration.LIST_DIR: the directory wheredrudgputs.lstoutput files. If this
is not set, then theLstDirparameter infesh2.configis used if set.
Otherwisedrudgdefaults to the$schedulesparameter inskedf.ctlA number of environment variables used bydrudgare also recognised byfesh2. See thedrudgsection below.  The
following
environment variables areNOTrecognised byfesh2:STATIONis used in the Field System but not recognised byfesh2as it is
possible to configurefesh2to manage schedules for multiple stations .
TheStationsparametermustbe set infesh2.configThe fesh2 configuration fileOn startup,fesh2looks for a configuration file calledfesh2.config.This will need to be set up for your station before runningfesh2for the
first time.Use your favourite text editor to modify the file. The comments in
the file describe the parameters but here are a few points to note:Station settingsYou will want to edit options in the[Station]section to configure which
stations you want to process, what types of Master and schedule files you want
to obtain, how far into the future you want to search for schedule files and how
often you want to look for them.EmailIf enabled, notification can be sent via email when:a new schedule has been downloadeda schedule file has been updated on the server, downloaded and needs manual
processingDrudgAfter finding and downloading a new or updated schedule,Fesh2can
optionally runDrudgto produce new snp, prc and lst files. The[Drudg]section allows you to configure this behaviour. If you don't wantfesh2to
automatically process your schedules, this feature con be turned off by
settingDodDrudgtoFalse.Becausefesh2manages schedule files and usesdrudgto process them, and
to keep things consistent across the Field System, it will use settings
inskedf.ctl. However, there are some caveats regardingskedf.ctlfor the
case where there is not an overriding definition:Fesh2requires that$schedulesis defined inskedf.ctl. The default
of '.' (i.e. the directory the software is started in) is too arbitrary and
could causefesh2to lose track of files.If$snapor$procare not defined, then they will be set to the same as$schedulesDrudg may have been configured such that the user is prompted for a response, in
which case it cannot be automated. If this is the case, then Drudg will not be
run on SKD files.If you want Fesh2 to automatically process SKD files, then some configuration is required. Fesh2 will get it's configuration information as follows, in
this priority order (lowest to highest):Fesh2 will first look in/usr2/control/skedf.ctl. Unless parameters are
changed, Fesh2 will not run Drudg if the following parameters are set as
shown:tpicd YESvsi_align ASKcont_cal ASKcont_cal_polarity ASKFesh2 will then look for any configuration parameters in the[Drudg]secoion offesh2.config. These parameters can be different toskedf.ctlin case you want Drudg to behave differently when Fesh2 is running it for youThere are someFESHenvironment variables which can be set and these will
override skedf.ctl or this file if they are. These environment variables are
as follows with the lowest supported version of Drudge shown in brackets (text from FS documentation):FESH_GEO_TPICD(10.0.0-beta3, December 2020)Possible values are
non-negative integers. If set, the value is provided as the answer for the
drudg prompt for the tpicd interval for geodesy schedules.FESH_GEO_CONT_CAL(10.0.0-beta3, December 2020)Possible values
areonoroff. If set, the value is provided as the answer for the
drudg prompt for continuous cal use for geodesy schedules.FESH_GEO_CONT_CAL_POLARITY(10.0.0-beta3, December 2020)Possible
values are0,1,2,3,ornone. If set, the value is provided as
the answer for the drudg prompt for the continuous cal polarity for
geodesy schedules.FESH_GEO_VSI_ALIGN(10.0.0-beta3, December 2020)Possible values
are0,1, ornone. If set, the value is provided as the answer for
the drudg prompt for using vsi_align for geodesy schedules.FESH_GEO_USE_SETUP_PROC(10.1.0, 2021)Possible values areyesorno. If set, the value is provided as the answer for the drudg prompt
for the"use setup_proc"for geodesy schedules, an option to write
a.snpfile that skips setup on scans when the mode hasn’t changed.FESH_GEO_VDIF_SINGLE_THREAD_PER_FILE(10.1.0, 2021)Possible values
areyesorno. If set, the value is provided as the answer for the
drudg prompt for the"VDIF single thread per file"for geodesy
schedules.Lastly, if any Drudg-related command-line parameters are set, they will
override everything elseIf any Drudg settings recieved by Fesh2 require manual intervention, Drudg will
not be run on the SKD files.ServersThe[Servers]section allows you to list all the IVS schedule file servers you
want to search and the protocols to use for each. Specify the location of the
top directory (i.e. thevlbidirectory). Protocols known to work are https (
secure HTTP), ftp (anonymous FTP) and ftps (secure (SSL) anonymous FTP).CurlFesh2usescurlto access files
on the servers. If https is being used
(e.g. to access the CDDIS repository), thenfesh2needs to know the location
of your.netrcand (
optionally).urs_cookiesfiles. Curl puts these in the user's home directory
by default but they can be placed elsewhere if desired. This can be set on the
command line or via the NETRC_FILE and COOKIES_FILE environment variables (
see above
) or in thefesh2config file by setting the NetrcFile and CookiesFile
parameters.UsageFesh2can be started by just typingfesh2which starts it in its default mode.fesh2can also be run in monitoring
mode in a terminal window by typing:fesh2 --monitNote: If you runfesh2as multiple instances but with different
configuration (e.g. running it in the background with SnapDir set on the
command line, then running it as a monitor with the default SNAP file
directory) then you may see some unexpected behaviour. In the above
example, a schedule could be generated by drudg in the non-default SNAP
directory butfesh2 --monitwould think that the schedule hadn't
been processed.Command-line options exist to change many of the configuration parameters, allow
for a one-off checks
(no wait mode), forcing of file downloads etc. Some common usages are:Only consider the current or next experimentfesh2 -nJust run once then exit, don't go into a wait loopfesh2 --onceJust get a schedule for a specified session, then exit. e.g.:fesh2 --once -g r1456Force an update to the schedule file when there's a new one available to
replace the old one. The default behaviour is to give the new file the name<code>.skd.newand prompt the user to take action. The file will also be
drudged if the DoDrudg option is Truefesh2 --updateObtain schedule files, but don't process them:fesh2 --DoDrudg FalseGet a status report. Shows the status of schedule files and when schedule
servers were last queried.fesh2 --checkRun fesh2 with all terminal output suppressed. Useful when running fesh2 as a
service.fesh2 --quietAll command-line parameters are as follows:usage:       fesh2 [-h] [-c CONFIGFILE] [-k SKEDCONFIGFILE] [-g GET] [-m] [-u]
                   [-n] [-a] [-o] [--update] [--SchedDir SCHEDDIR]
                   [--ProcDir PROCDIR] [--SnapDir SNAPDIR] [--LstDir LSTDIR]
                   [--LogDir LOGDIR] [--Stations [STATIONS [STATIONS ...]]]
                   [--EmailNotifications [EMAILNOTIFICATIONS]]
                   [--GetMaster [GETMASTER]]
                   [--GetMasterIntensive [GETMASTERINTENSIVE]]
                   [--SchedTypes [SCHEDTYPES [SCHEDTYPES ...]]]
                   [-t MASTERCHECKTIME] [-s SCHEDULECHECKTIME]
                   [-l LOOKAHEADTIMEDAYS] [-d [DODRUDG]]
                   [--DrudgBinary DRUDGBINARY] [--TpiPeriod TPIPERIOD]
                   [--VsiAlign VSIALIGN] [--ContCalAction CONTCALACTION]
                   [--ContCalPolarity CONTCALPOLARITY] [--SetupProc SETUPPROC]
                   [--VdifSingleThreadPerFile VDIFSINGLETHREADPERFILE]
                   [--Servers [SERVERS [SERVERS ...]]] [--NetrcFile NETRCFILE]
                   [--CookiesFile COOKIESFILE]
                   [--CurlSecLevel1 [CURLSECLEVEL1]] [-y YEAR] [-e] [--monit]
                   [-q]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIGFILE, --ConfigFile CONFIGFILE
                        The fesh2 configuration file to use. e.g.
                        /usr2/control/fesh2.config
  -k SKEDCONFIGFILE, --SkedConfigFile SKEDCONFIGFILE
                        The location of the skedf.cfg configuration file
  -g GET, --get GET     Just get a schedule for this specified session. Give
                        the name of the session (e.g. r4951).
  -m, --master-update   Force a download of the Master Schedule (default =
                        False), but just on the first check cycle.
  -u, --sched-update    Force a download of the Schedules (default = False),
                        but just on the first check cycle.
  -n, --current, --now  Only process the current or next experiment
  -a, --all             Find the experiments with all "Stations" in it
  -o, --once            Just run once then exit, don't go into a wait loop
                        (default = False)
  --update              Force an update to the schedule file when there's a
                        new one available to replace the old one. The default
                        behaviour is to give the new file the name
                        <code>.skd.new and prompt the user to take action. The
                        file will also be drudged if the DoDrudg option is
                        True
  --SchedDir SCHEDDIR   Schedule file directory
  --ProcDir PROCDIR     Procedure (PRC) file directory
  --SnapDir SNAPDIR     SNAP file directory
  --LstDir LSTDIR       LST file directory [env var: LIST_DIR]
  --LogDir LOGDIR       Log file directory
  --Stations [STATIONS [STATIONS ...]]
                        Stations to consider (e.g. "hb ke yg ho", "mg")
  --EmailNotifications [EMAILNOTIFICATIONS]
                        Send notifications by email. The fesh2 config file
                        will be read for details on mail server, recipients
                        etc
  --GetMaster [GETMASTER]
                        Maintain a local copy of the main Multi-Agency
                        schedule, i.e. mostly 24h sessions (default = True)
  --GetMasterIntensive [GETMASTERINTENSIVE]
                        Maintain a local copy of the main Multi-Agency
                        Intensive schedule (default = True)
  --SchedTypes [SCHEDTYPES [SCHEDTYPES ...]]
                        Schedule file formats to be obtained? This is a
                        prioritised list with the highest priority first. Use
                        the file name suffix ("vex" and/or "skd") and comma-
                        separated.
  -t MASTERCHECKTIME, --MasterCheckTime MASTERCHECKTIME
                        Only check for a new master file if the last check was
                        more than this number of hours ago. The default is set
                        in the configuration file.
  -s SCHEDULECHECKTIME, --ScheduleCheckTime SCHEDULECHECKTIME
                        Only check for a new schedule file (SKD or VEX) if the
                        last check was more than this number of hours ago. The
                        default is set in the configuration file.
  -l LOOKAHEADTIMEDAYS, --LookAheadTimeDays LOOKAHEADTIMEDAYS
                        Only look for schedules less than this number of days
                        away (default is 7)
  -d [DODRUDG], --DoDrudg [DODRUDG]
                        Run Drudg on the downloaded/updated schedules (default
                        = True)
  --DrudgBinary DRUDGBINARY
                        Location of Drudg executable (default =
                        /usr2/fs/bin/drudg)
  --TpiPeriod TPIPERIOD
                        Drudg config: TPI period in centiseconds. 0 = don't
                        use the TPI daemon (default) [env var: FESH_GEO_TPICD]
  --VsiAlign VSIALIGN   Drudg config: Applicable only for PFB DBBCs, none =
                        never use dbbc=vsi_align=... (default) 0 = always use
                        dbbc=vsi_align=0 1 = always use dbbc=vsi_align=1 [env
                        var: FESH_GEO_VSI_ALIGN]
  --ContCalAction CONTCALACTION
                        Drudg config: Continuous cal option. Either 'on' or
                        'off'. Default is 'off' [env var: FESH_GEO_CONT_CAL]
  --ContCalPolarity CONTCALPOLARITY
                        Drudg config: If continuous cal is in use, what is the
                        polarity? Options are 0-3 or 'none'. [env var:
                        FESH_GEO_CONT_CAL_POLARITY]
  --SetupProc SETUPPROC
                        Drudg config: the answer for the drudg prompt for the
                        use setup_proc for geodesy schedules, an option to
                        write a `.snp` file that skips setup on scans when the
                        mode hasn’t changed. [env var:
                        FESH_GEO_USE_SETUP_PROC]
  --VdifSingleThreadPerFile VDIFSINGLETHREADPERFILE
                        VDIF single thread per file: only applies to Mark5C or
                        Flexbuff recorders. Can be 'yes' or 'no' [env var:
                        FESH_GEO_VDIF_SINGLE_THREAD_PER_FILE]
  --Servers [SERVERS [SERVERS ...]]
                        Schedule file server URLs. Each of these will be
                        checked for the most recent files. Use comma-separated
                        URLs and specify the top directory (i.e. the 'vlbi'
                        directory). Use protocols https (Curl), ftp (anonymous
                        FTP) or sftp (anonymous secure FTP)
  --NetrcFile NETRCFILE
                        The location of the .netrc file, needed by CURL for
                        the https protocol. (CURL puts this in ~/.netrc by
                        default) [env var: NETRC_FILE]
  --CookiesFile COOKIESFILE
                        The location of the .urs_cookies files used by CURL.
                        (CURL puts this in ~/.urs_cookies by default) [env
                        var: COOKIES_FILE]
  --CurlSecLevel1 [CURLSECLEVEL1]
                        Workaround for CDDIS https access in some Debian
                        distributions. See the documentation (default = False)
  -y YEAR, --year YEAR  The year of the Master Schedule (default is this year)
  -e, --check           Check the current fesh2 status. Shows the status of
                        schedule files and when schedule servers were last
                        queried.
  --monit               Similar to --check but text output format is intended
                        for a FS monit interface
  -q, --quiet           Runs fesh2 with all terminal output suppressed. Useful
                        when running fesh2 as a service.LoggingAs well as writing information to the screen on activity,fesh2also keeps a
log of activity in the Field System log directory (by default) at/usr2/log/fesh2.log. The log file location can be configured infesh2.configRunning fesh2 as a serviceFesh2 can be run in the background as asystemdservice. All output is
suppressed and status is available by examining the log file or using the--checkor-eflag. Here's how to set it up from the superuser account for
Debian Jessie or later:Type the following command to add asystemdservice:systemctl edit --force --full fesh2.serviceThis should open a text editor. Paste in the following:[Unit]
Description=Fesh2 Service
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/bin/sudo -H -u oper /usr/local/bin/fesh2 --quiet

[Install]
WantedBy=multi-user.targetSave and exit. This will configuresystemdto start fesh2, running as user
oper and suppress all output.Enable the service:sudo systemctl enable fesh2.serviceCheck the status of the service:sudo systemctl status fesh2.serviceYou can stop, start and query the service:sudo systemctl stop fesh2.service          # Stop running the service 
sudo systemctl start fesh2.service         # Start running the service 
sudo systemctl restart fesh2.service       # Restart the serviceTo see the current schedule file status (as user oper):fesh2 --checkIf you would prefer to set this up as a user service, some notes are here:https://www.unixsysadmin.com/systemd-user-services/For versions of Debian older thean Jessie (e.g. Wheezy), systemd can be enabled
and some notes are here:https://scottlinux.com/2014/10/20/how-to-switch-to-systemd-on-debian-wheezy/Appendix A: fesh2.config# Fesh2 configuration file.
#
# Note: all non-essential parameters (i.e. those with defaults) are commented out
#
# Field System directories
[FS]
  # Log directory. This is where the fesh2 log file will be written.
  # Default is /usr2/log
  #LogDir = /usr2/log

# Default settings for this station/facility
[Station]
  # station name(s) to process by default. Just a two-letter code for a single
  # station or comma separated two-letter codes inside square brackets for
  # multiple stations. e.g.:
  # Stations = [Hb, Ho, Ke, Yg]
  # Stations = Ke
  Stations =  Mg
  #
  ##################
  # Which master files should be processed? At least one of these must be selected
  # Get the main Multi-Agency schedule, i.e. mostly 24h sessions. Default is True
  #GetMaster = True
  # Get the Multi-Agency Intensives schedule. Default is True
  #GetMasterIntensive = True
  ##################
  # What schedule file formats should be obtained? This is a prioritised list with the highest one first
  # Use the file name suffix (vex and/or skd) and comma-separated. Default is skd
  #SchedTypes = skd
  #SchedTypes = [skd, vex]
  # What is the minimum time between checks for a new Master file on the server (hours)?
  MasterCheckTime = 12
  # What is the minimum time between checks for a new Schedule (SKD or VEX) file on the server (hours)?
  ScheduleCheckTime = 1
  # How far ahead in time should we look for schedule files (days)?
  LookAheadTimeDays = 14

[Email]
  # If Fesh2 finds a problem with a schedule or if a schedule needs manual processing,
  # a notification message can be sent via email. To turn this feature on, set email_notifications to True,
  # specify a recipient(s)  and configure with the address of the SMTP server.
  EmailNotifications = True
  # an array containing one or more email addresses to receive notifications
  EmailRecipients = [me@gmail.com]
  EmailSender = my_email_address@thing.com
  # The SMTP server
  EmailServer = myserver.thing.com.au
  # SMTP Port
  #EmailPort = 587
  # Server Password. Leave commented out if you don't need one
  #EmailPassword = "VerySecret"

[Drudg]
  # Drudg-related config
  # Run drudg on downloaded schedules? If False then files are downloaded but not processed
  DoDrudg = True
  # Location of the drudg executable
  DrudgBinary = /usr2/fs/bin/drudg
  # Directory for the output SNP file
  # Default is /usr2/sched
  SnapDir = /usr2/sched
  # Directory for the output LST (schedule summary) file
  # Default is /usr2/sched
  LstDir = /usr2/sched
  #-------------------------------------------------------------------------------------------------
  # If DoDrudg is True, then Fesh2 will run Drudg on the SKD files it downloads. It will drudg all new
  # SKD files and all updated SKD files provided they have not been changed locally in the meantime.
  #
  # However, Drudg may have been configured such that the user is prompted for a response, in which
  # case it cannot be automated. If this is the case, then Drudg will not be run on SKD files.
  #
  # If you want Fesh2 to automatically process SKD files, then some configuration of Drudg is required.
  # Fesh2 will get it's configuration information as follows, in this priority order (lowest to highest):
  # config file, env var, command line
  # 1. Fesh2 will first look in /usr2/control/skedf.ctl
  # 2. It will then look for any configuration parameters in this file (just below these comments)
  #    These parameters can be different to skedf.ctl in case you want Drudg to behave differently
  #    when Fesh2 is running Drudg for you
  # 3. There are some FESH environment variables which can be set and these will override skedf.ctl
  #    or this file if they are.
  # 4. Lastly, any Drudg-related command-line parameters are set, they will override everything else
  #
  # If any Drudg settings recieved by Fesh2 require manual intervention, Drudg will not be run on
  # the SKD files.
  #
  # If set, these parameters will override the corresponding settings in skedf.ctl but will be overridden
  # by environment variables and command-line parameters. Uncomment and set them if you wish, otherwise they are
  # ignored:
  #
  #  1. TPI period in centiseconds
  #     0 = don't use the TPI daemon
  #TpiPeriod = 0
  #
  #  2. Continuous cal option. Either "on" or "off". Default is 'off'
  #ContCalAction = off
  #
  #  3. If continuous cal is in use, what is the polarity? Options are 0-3 or "none". Default is none
  #ContCalPolarity = none
  #
  #  4. For PFB DBBCs only, vsi_align setup
  #     Applicable only for PFB DBBCs,
  #     none = never use dbbc=vsi_align=... (default)
  #     0 = always use dbbc=vsi_align=0
  #     1 = always use dbbc=vsi_align=1
  #VsiAlign = none
  #
  #  5. If set, the value is provided as the answer for the drudg prompt for the "use setup_proc" for geodesy
  #     schedules, an option to write a `.snp` file that skips setup on scans when the mode hasn’t changed.
  #     Possible values are `yes` or `no`.
  #SetupProc = yes
  #
  #  6. If set, the value is provided as the answer for the drudg prompt for the *"VDIF single thread per file"*
  #     for geodesy schedules.
  #     Possible values are `yes` or `no`.
  #VdifSingleThreadPerFile = yes
  #-------------------------------------------------------------------------------------------------

[Servers]
  # Schedule file server URLs. Each of these will be checked for the most recent files.
  # Use comma-separated URLs inside square brackets, and specify the top directory (i.e. the 'vlbi' directory).
  # Use protocols https (Curl), ftp (anonymous FTP) or sftp (anonymous secure FTP)
  Servers = [https://cddis.nasa.gov/archive/vlbi, ftp://ivs.bkg.bund.de/pub/vlbi, ftp://ivsopar.obspm.fr/pub/vlbi]
  # Alternative CDDIS (anonymous secure FTP):
  # Servers = ftps://cddis.nasa.gov/archive/vlbi

[Curl]
  # Curl configuration files.
  # A .netrc and a .urs_cookies file are needed by Curl for the https protocol. Curl puts these in the user's
  # home directory by default but they can be placed elsewhere if desired. This can be set
  # on the command line or via the NETRC_FILE and COOKIES_FILE environment variables. The command-line overrides
  # the environment variable which overrides the config file
  NetrcFile = /usr2/control/netrc_fesh2
  # The cookies file is optional and the default is /dev/null
  CookiesFile = /dev/null
  # In some recent Debian distributions, a connection to CDDIS will fail with error 35 (see
  # item 17 in the "CDDIS https Archive Access/File Download FAQ" https://cddis.nasa.gov/About/CDDIS_File_Download_FAQ
  # .html). You will see a WARNING message if this occurs and downloads from CDDIS will not be possible.
  # There is a workaround until the CDDIS server is changed to fix the problem: Set the following
  # parameter to True
  CurlSecLevel1 = False