Package: livvkit
The land ice verification and validation toolkitLIVVkit is a python-based toolkit for verification and validation of ice sheet
models. It aims to provide the following capabilities:Model V&VNumerical verification -- "Are we solving the equations correctly?"Physical validation -- "Are we using the right physics?"Software V&VCode verification -- "did we build whatweintended?"Performance validation -- "did we build what theuserswanted?"Within LIVVkit, these capabilities are broken into four components:Model V&VNumericsValidationSoftware V&VVerificationPerformanceCurrently, LIVVkit is being used and developed in conjunction with E3SM
(Energy Exascale Earth System Model) and CISM
(Community Ice Sheet Model), but is designed
to be extensible to other models. For further documentation view thefull documentation.Users and contributors are welcome!We’ll help you out –open an issue on githubto contact us for any reason.InstallationThe latest LIVVkit release can be installed viapip:pipinstalllivvkitAdditionally, LIVVkit is released on github, and you can clone the source code:gitclonehttps://github.com/LIVVkit/LIVVkit.gitIf you are having any troubles with installation or dependencies, open an issue on theissue trackeror contact us!UsageLIVVkit is primarily controlled via options specified at the command line.
To see the full list of options, run:livv-hVerificationIn verification mode, LIVVkit analyzes and compares a regression testing
dataset to a reference dataset. For example, LIVVkit may analyze the dataset
produced from a proposed CISM 2.0.6 release (~400MB; downloadhere) and
compare it to the dataset produced from the CISM 2.0.0 release (~400MB;
downloadhere).
To run this example, first download the two aforementioned datasets to a
directory, open a terminal, and navigate to your download directory.
Then, un-tar the datasets:tar-zxvfcism-2.0.0-tests.20160728.tgz
tar-zxvfcism-2.0.6-tests.20160728.tgzFor ease, export the path to the two dataset directories:exportREF=$PWD/cism-2.0.0-tests/titan-gnu/CISM_glissadeexportTEST=$PWD/cism-2.0.6-tests/titan-gnu/CISM_glissadeTo run the suite, use:livv-v$TEST$REF-ocism206v200-sLIVVkit will run the verification suite, report a summary of the results
on the command line, produce an output website in the createdcism206v200directory specified by the-o/--out-diroption, and launch an http server
(the-s/--serve option) to easily view the output in your favorite web
browser. LIVVkit will tell you the address to view the website at on the
command line, which will typically look likehttp://0.0.0.0:8000/ver_test/index.html.Validation, ExtensionsLIVVkit is extensible to more in-depth or larger validation analyses.
However, because these validation analyses are particularly data intensive,
many of the observational and example model output files are much too
large to distribute in the LIVVkit package. Therefore, we've developed a
LIVVkit Extensions repository (LEX) which usesgit-lfs(Git Large File Support) in order to
distribute the required data.git-lfscan be installed either before or
after cloning this repository, but it will be neededbeforedownloading
the required data. You can determine if you havegit-lfsinstalled on
your system by running this command:command-vgit-lfsIfgit-lfsis not installed, you can install it by following the instructions here:https://git-lfs.github.comOncegit-lfsis installed, clone and enter this repository:gitlfsclonehttps://code.ornl.gov/LIVVkit/lex.gitcdlexEach extension will have an associated JSON configuration file which will describe
the extension's analysis code, data locations, and options. To see a list of
available extensions, you can run this command:find.-iname"*.json"To execute any of these extensions, pointlivvto any of these extensions config file via the-e/--extensionoption (or the-V/--validateoption). For example, to run the minimal example extension,
place the output website in theval_testdirectory, and serve the output website
you'd run this command:livv-eexample/example.json-ovv_test-sNote:All the extension configurations files assume you are working from the
top levellexdirectory. Youcanrun any of these extensions from any
directory, but you will need to edit the paths in the JSON configuration files so
thatlivvcan find the required files.Likewise, you can also apply these analyses to any new model run by adjusting
the paths to point to your model run.MoreFor more information about using LIVVkit see thedocumentation.ContactIf you would like to suggest features, request tests, discuss contributions,
report bugs, ask questions, or contact us for any reason, use theIssue Tracker.Want to send us a private message?Joseph H. Kennedygithub: @jhkennedyemail:kennedyjh [at] ornl.govKatherine J. Evansgithub: @kevans32email:evanskj [at] ornl.govIf you're emailing us, we recommend CC-ing all of us.