Comments on the TIDE processing software:

Last updated (05/18/07).

-------------------------------------------------------------------------------
The latest version of the TIDE processing software is tide_lz_v6.1.3.
There are two types of the processing code, one runs without IDL and the other
runs from IDL.  I use the non-idl version for development, testing, and 
non-routine processing.  There is no reason why the IDL version can't be used
to do this.  The IDL version of the code is in the most recent directory of
the exlib account, tide_dir/tide_lz_vn.n.n.  The current development account
is /export/data/tide/exlib/new/tide_lz_v6.2.  Note:  not changes have been 
made to any of the code so it should be identical to 6.1.3.

The IDL version of the software is used for all the routine processing except
the creation of the high-time resolution (htr) cdf files.  It is also the 
software that users can download and that the web uses for data processing.
The only reason that the htr cdf files are created using the non-idl version is
to avoid tying up an IDL license to process a lot of data.  If necessary, the
crontab script used to create the htr cdf files can be modified to use the IDL
version.

Assosciated with each version of the software is lz_parm.  It is an interactive
program that determines the type of processing done.  It asks for responses to
a series of prompts that determine what type of processing is to be done.  It is
part of the input to the processing program.

The only difference in the software for the non-idl and the idl version of the
software are some changes in tide_lz_main.c At the top of the tide_lz_main.c,
just after the comments is a typedef structure that varies with the version of
IDL in use.  For the non-idl version, both need to be commented out.  For the
software run on goewin, the typedef for early versions of IDL needs to be
commented out.  Do a search on standalone.  This block of code needs to be
commented out for the IDL version.  Do a search on 'IDL Version'.  There are 4
blocks of code that start with this phraise in at comment at the beginning and
end for the block.  For the non-idl version of the code, these need to be
comment out.  To comment out a block of code have '/*' on the first line and 
'*/' as the last two characters of the last line of block.  Do not have these
two sequences any other place in the block.  To make sure that a block of code
gets used (not commented out), make sure that the first two characters of the
comment at the top of the block and the comment at the bottom of the block
(if there is one) are '/*' and the last two characters are '*/'.

Hopefully, the software is internally documented well enough so that you can 
find the area that needs to be edited.  There is a file called tide_lz.smry in 
this directory that is old but it has some information in it about the files 
that make up the processing code.

The command to compile either version of the software is simply 'make'.  The 
executable for the non-idl code is tide_lz.  For the idl version, make creates
a unix software library, tide_lz.so.  The command to compile lz_parm is 
'make -f Mparm'.

-------------------------------------------------------------------------------
Releasing a new version of the software: Before releasing a new version of the
of the software, make sure to update the version number in tide_lz_main.c and
to add comments to the top of that file that summarize any changes made.  Copy
the new tide_lz file to /export/data2/tide/db/exe so that it can be used to 
create the htr cdf files.  In exlib, copy the tar_the_files script from the 
previous version, edit it to change the name of the tar file to the new 
version and to add any new software/data files not listed.  The create the tar
file by simply typing 'tar_the_files'.  Copy tide_lz.so and lz_parm to 
/export/data2/tide/db/exe for use by the production software.  Also copy any
new input data files and plot routines.  ftp or secure copy the tar file to 
web on satyr and psloan on goewin.  On satyr in /home/web/htdocs/TIDE/software
and on goewin in /space/www1/tide/software, put a copy of the tar file for the
new version and then edit index.html and change the name of the current version
of the software.   Then on satyr in 
/home/web/httpd_1.5.2/cgi-bin/TIDE/lz_graphics-dir and on goewin in 
/space/cgi-bin/tide/lz_graphics-dir, create a directory for the new version, 
copy the tar file to it, and untar the file (tar xvf tide_lz_vn.n.n).  On
goewin edit tide_lz_main.c, search on typedef, and comment out the < idl 5.5
block of code and uncomment the >= idl 5.5 code.  Create both tide_lz.so and
lz_parm using 'make' and 'make -f Mparm'.  Copy everything in the new directory
to tide_lz_current.  If 'cp * ../tide_lz_current' does not work, the try issuing
the following commands:

cd ../tide_lz_current
/bin/rm *
cp ../tide_lz_vn.n.n/* .

These steps make the latest version of the software accessible from the web
interface software.

-------------------------------------------------------------------------------
Running the software:  To run either program, you must first create a parameter
file using lz_parm.  This file should have the .parm extension so its easy to
keep track of.  Once created, .parm files can be used again as long as the
desired processing doesn't change.  You also need to create an input file with
the start and end times of the data to be processed and the location of the 
input files.  The format of this file is slightly different for each version.

non-idl version input file format without a day crossing:
mm/dd/yy hh:mm:ss
mm/dd/yy hh:mm:ss
1 /archive/po_tid_lz_yymm/data/po/tid/lz/yymmddvv.dat
2 /archive/po_tid_lz_yymm/data/po/pre/or/yymmddvv.cdf
3 /archive/po_tid_lz_yymm/data/po/pre/at/yymmddvv.cdf
4 /archive/data/mask/tyymmdd_v2.mask
5 /archive/data/scpot/yyyymmxx.dat


idl version input file format without a day crossing:
mm/dd/yy hh:mm:ss
mm/dd/yy hh:mm:ss
parameter filename
wpg
1
directory
extension
1 /archive/po_tid_lz_yymm/data/po/tid/lz/yymmddvv.dat
2 /archive/po_tid_lz_yymm/data/po/pre/or/yymmddvv.cdf
3 /archive/po_tid_lz_yymm/data/po/pre/at/yymmddvv.cdf
4 /archive/data/mask/tyymmdd_v2.mask
5 /archive/data/scpot/yyyymmxx.dat

The first line is the start time, the second line is the stop time.  The last 
five lines are the names of the input files.  If the time crosses a day then,
include both files one after the other.  If the times cross a month, then two
spacecraft potential files are required. The first data file line (starts with 1) is the level-zero file, '2' is the orbit file, '3' is the attitude file, '4'
is the mask file (can be left blank if the mask is not used), '5' is the 
spacecraft protential file (can be left off if not used).  There are three
other files that can be used '6' is the peaks range file, '7' is the mfe b-field
file that is used instead of the b-field data in the tide telemetry, and '9'
which is the value of the mask correction file.  It overrides what is in the
parm file.  The third through 7th lines of the idl input file are the parameter
filename, what type of plot files to create, the color table number, the 
path location where the output goes (leave blank for current directory), and 
a file extension to change the standard filename (can be left blank).  See
run_tide.input.template for more details.

To run the non-idl version, type 'tide_lz'.  You will be asked to enter the 
parameter filename and the input filename.  Use the *multi idl command files
to plot results.  To run the idl version, get into idl and typy @run_tide.
You will be asked to enter the name of the input file.  All plotting is 
automatic.  

Use the 'Find files' link on the web pages to find out the name and location
of the input files.

The web interface takes care of creating both the input and parameter files.


-------------------------------------------------------------------------------
Extended studies:  Barry has set up a routine on the tide account in the 
production software directory, /export/data2/tide/db/exe.  A script called 
ksh_cr_batch_mas is set up to run multiple sets of data for a specific 
processing option.  The output files (plots and data files) are created in 
/export/data3/tide/tide_plots/batch_plots.  The command line input for the 
script is the name of the file that contains the list of start and stop times
and the name of the parameter file to be used in the processing.  The parameter
file is created with lz_parm.


-------------------------------------------------------------------------------
Areas that still need to be addressed:  The data has not been calibrated since
the Stops MCP voltages were change in October 2004.  Once the software has been
changed, the summary plots and htr cdf files for that time from October 2004 to
the point where the change was made probably should be redone.  They are
currently being done without calibration.