NISTIR 7739

Open Monte Carlo Engine 

User Manual

Ruediger Kessel

 
NISTIR 7739

Open Monte Carlo Engine 

User Manual 

Ruediger Kessel 

Information Technology Laboratory 

Applied and Computational Mathematics Divisiont 

November 2010

U.S. Department of Commerce 

Gary Locke, Secretary 

National Institute of Standards and Technology 

Patrick D. Gallagher, Director

 
Contents

1 Introduction 3

1.1 Theory of operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Limits and restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Distribution of the OMCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.4 Copyright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.5 Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Installing OMCE 4

3 Running OMCE 4

3.1 Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 OMCE file formats 10

4.1 Xml input file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.1.1 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.1.2 Quantities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.1.3 Correlations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1.4 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1.5 Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1.6 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1.8 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.1.9 Distribution elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.1.10 Expressions and equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.1.11 Function minimum() and maximum() . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1.12 Function pow() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1.13 Function iff() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1.14 Logical expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1.15 Symbol names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Binary data file formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2.1 OMCE binary format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2.2 Simple binary format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.3 Text data file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5 Histogram viewer 20

5.1 Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6 Adaptive mode 20

6.1 GUM Supplement 1 based algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6.2 Alternate algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

1
6.2.1 Sub-blocking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2.2 Establishing the statistical confidence level . . . . . . . . . . . . . . . . . . . . . . 22

6.2.3 Numerical tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2.4 Predicting the number of simulation blocks . . . . . . . . . . . . . . . . . . . . . 23

6.2.5 Algorithms based on the Stein-method . . . . . . . . . . . . . . . . . . . . . . . . 24

6.2.6 Algorithms based on a prediction of the needed runs . . . . . . . . . . . . . . . . 24

6.2.7 Statistical end test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

7 Running multiple simulations 25

8 Input correlation coefficients 26

9 Analysis of correlation 26

10 Constraints on the domain of the functions 26

10.1 Mean value and uncertainty calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

11 Testing and validation 28

12 Performance 28

12.1 Random number generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

References 28

A Program exit codes 30

2
1 Introduction

The Open Monte Carlo Engine (OMCE) is a full featured command line Monte Carlo engine that reads

a model description in xml-format and runs a Monte Carlo simulation to calculate a histogram, a best

estimate (mean) and the standard uncertainty. In Addition, probabilistically symmetric or shortest

coverage intervals are calculated. The OMCE can be used as an alternative method to the mainstream

GUM [1] to evaluate the measurement uncertainty. The OMCE implements the Monte Carlo integration

method as described in the Guide to the Expression of Uncertainty Supplement 1 [2]. The OMCE is

written in Python and is public domain (open source code).

1.1 Theory of operation

The OMCE reads the model description in xml-format and creates an internal list of quantities, equations

and results. It then evaluates the execution order for the equation system. A production and consumer

pipeline is set up with two tasks. One task calculates a block of results and analyzes it, and the other

task writes the block to a binary file and frees the memory. The pipeline stops when sufficient data are

produced and completely written to the binary file. The last step is to write the histogram to a text file.

1.2 Limits and restrictions

The OMCE has the following limits and restrictions:

• The number of quantities, equations and results is not limited by the software. In practice the

computer hardware, the memory and the computational power will limit the size of the model that

can be simulated.

• Only explicit model equations can be simulated. OMCE does not include an equation solver.

1.3 Distribution of the OMCE

The OMCE is distributed as source code accompanied by an Windows executable (exe) file for the

convenience of users who do not have Stackless Python [3] or Numpy [4] installed. The OMCE was

developed with Stackless Python 2.5 but should run with other Stackless Python versions as well.

1.4 Copyright

The OMCE is free software with no copyright associated with it. It can be freely copied, modified and

used, but it cannot be copyrighted. The idea is to have a Monte Carlo engine that can be freely studied

and modified to user needs.

1.5 Disclaimer

This software was developed at the National Institute of Standards and Technology by employees of

the Federal Government in the course of their official duties. Pursuant to title 17 Section 105 of the

United States Code this software is not subject to copyright protection and is in the public domain. This

software is experimental. NIST assumes no responsibility whatsoever for its use by other parties, and

makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic. We

would appreciate acknowledgement if the software is used.

3
2 Installing OMCE

OMCE is distributed as a zip-file (OMCE-1.1.14.zip) containing all necessary files to run it on a Windows

platform without Python. The zip-file should be unpacked in a separate directory OMCE\. The OMCE

can be executed directly from the command line if the OMCE\bin\ subdirectory is added to the search

path. The executable version (exe) contains all packages needed to run it. In addition, the source code

is included. Linux users should use the source code version. The OMCE source code makes use of

Stackless Python [3] and several non-standard open source Python packages imported in the beginning

of OMCE.py. The packages must be installed before the source code version can be run.

Contents of OMCE-1.1.14.zip

File Location Description

OMCE.exe OMCE\bin\ Exe version of the OMCE

OMCE.xsd OMCE\bin\ xml schema definition of the OMCE format

OMCE.ofd OMCE\bin\ OMCE statistical output file definition

OMCE.vfd OMCE\bin\ OMCE var file definition

ALT.ofd OMCE\bin\ alternative statistical output file definition

ALT.vfd OMCE\bin\ alternative var file definition

*.* OMCE\bin\ python runtime environment

OMCE.pdf OMCE\doc\ OMCE user manual

OMCE\Schema.html OMCE\doc\ OMCE xml schema description

OMCE.py OMCE\src\ OMCE Python source file

OMCE.xsd OMCE\src\ xml schema definition of the OMCE format

OMCE.ofd OMCE\src\ OMCE statistical output file definition

OMCE.vfd OMCE\src\ OMCE var file definition

ALT.ofd OMCE\src\ alternative statistical output file definition

ALT.vfd OMCE\src\ alternative var file definition

Viewer.py OMCE\src\ histogram viewer

Viewer.exe OMCE\Viewer\ EXE-version of the histogram viewer

*.* OMCE\Viewer\ python run-time environment for the viewer

*.omc OMCE\examples\ example xml files in OMCE format

*.xml OMCE\examples\ example xml files in alternativee format

!run_all_OMCE.bat OMCE\examples\ batch file to run all examples in OMCE format

!run_all_ALT.bat OMCE\examples\ batch file to run all examples in alternative format

*.ohd OMCE\examples\ref\ reference results for the examples in OMCE format

(-seed=1)

*.sta OMCE\examples\ref\ reference results for the examples in alternative format (-seed=1)

After the installation, the windows batch file !run_all_OMCE.bat or !run_all_ALT.bat in OMCE\examples\

should be run to verify the installation. This will run the example xml files and compare the results with

the reference results stored in OMCE\examples\ref\. The random number generator is seeded for these

runs so the results must exactly match the reference results.

3 Running OMCE

The OMCE is a command line tool which is designed to be used in batch scripts or to be called by other

programs. If the directory of OMCE (exe-version) is included in the path, it can be executed with the

following command:

OMCE filename.omc

This will start the OMCE, loading filename.omc and creating a file named filename.ohd. All options

have default values. If no file name is given, a list of options with their default values and the implemented

error codes is printed.

4
The general format for using the OMCE is:

OMCE <filename.omc>|<filename.xml> [<options>] [<parameters>]

The input file should be either in omc-format or in the alternative xml-format. See option -fi and

Section 4.1 on file formats for more details.

All option follow the following format:

-<option>=<value>

The option name <option> is string of one to four characters identifying the option and is always followed

by an equal sign. The format of the <value> depends on the option and can be either an integer, a

floating point number or a string. See Section 3.1 for a detailed discussion.

Parameters have the following format:

<name>=<value>

The parameter names are defined in the omc input file (see Section 4.1.1). The value must be a floating

point number.

Note: The name of the input file name must be specified before any parameters.

If there is an error, OMCE will terminate with a non-zero exit code. See Appendix A for a list of possible

error messages and exit codes.

3.1 Options

The options can be given in any order but they should be separated by a space and should be given after

the omc-file name.

-a This switch activates the generation of the statistical data file (.ohd) and is set to 1 by default.

This switch should be set to zero if the internal data analyzer is not used and the binary data

file (-wb=1 or -wb=2) is analyzed by some external tool.

-ac This option controls the correlation analysis of the simulated output data.

-ac= correlation analysis

0 no correlation analysis

1 between results (default)

2 results and their inputs

3 all

-ad This option sets the adaptive mode tolerance divisor to the value given (see section on adaptive

mode). The tolerance is calculated based on the standard uncertainty or the probability

interval and the number of significant digits divided by the tolerance divisor. The default

value is 1.0.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-af This option sets the probability level for the decisions during the adaptive mode. If option

-k=0 or if the alternate algorithms are used (-am > 0) a t-factor is calculated based on the

probability controlled by this option and the number of simulation runs so far.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-am This option controls the method how the number of runs are determined during the adaptive

mode. The default -am=0 uses the sequential checking scheme defined in GUM Supplement

1. Alternatively, -am > 0 activates alternative 2-stage schemes to predict the total number

of runs.

5
-am= adaptive algorithm used during simulation

0 GUM Supplement 1 based algorithm

1 modified Stein 2-Stage scheme (see [5]) with S1 accuracy definition

2 modified Stein 2-Stage scheme (see [5]) with fixed relative accuracy definition

3 prediction of the needed runs with S1 accuracy definition

4 prediction of the needed runs with fixed relative accuracy definition

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-ap This option sets the probability of the interval which is used for the tolerance check in the

adaptive mode. This interval is calculated for every block of data. If bit 2 of the option -at

is set then the standard deviation of the average of all calculated intervals is multiplied by

the k-factor (option -k) and compared with the tolerance based on option -sd and -ad.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-at This option controls in detail which parameter is used to for the check in the adaptive mode

(see Section 6). The value of this option is a bit combination which enables the different checks

The default is to the check the standard deviation of the mean, the standard uncertainty and

the limits.

-at= parameter used during the check

1 standard deviation of the means

2 standard deviation of the uncertainty

3 standard deviation of the means and the uncertainty

(see [2] Section 7.9.4 Note 5)

7 standard deviation of the means, the uncertainty and the limits

(default see [2] Section 7.9.4)

-bs The internal data processing in the OMCE operates on data blocks with a fixed block size.

The block size has influence on the performance and the statistical properties of the result.

Usually the default block size (10000 values) is a good compromise between performance and

statistical rigor. The block size can be changed for experimental purposes or if the value of

option -nl or -nc is reduced.

In the adaptive mode the block size is adjusted automatically and the value given with this

option will be overridden.

-cl This option controls the lower limit of the absolute value of the calculated correlation coefficient. The default value is 0.001.

-da By default (-da=0) an existing output file will be replaced by new simulation data. If this

option is set to -da=1, no header is printed to the file and the data is appended. If the data

should be distinguishable from prior existing data, set the option -sid to a different start

value. Warning: The format of the existing output file will not be checked.

-de This option controls how the name of the output files is formed from the given input filename.

If this option is set (default), the extension (.omc) is deleted from the filename before the

output file extension is added; otherwise the extension is added without deletion. Warning:

any existing output file (.dat, .bin, .ohd, .sta) will be overwritten without warning.

-e1 The OMCE will automatically try to correct for small negative eigenvalues in the correlation

matrix. This option controls the limit for the correction. The value must be negative and

the default is -0.1.

-e2 When the correlation matrix needs to be corrected, the Least Maximum Norm of the correction is calculated. This option controls the limit for this norm. The default value is

0.01.

6
-fi The OMCE supports two input formats which are both based on xml. The native OMCE

format allows a simple description of the evaluation model. The details of the format can be

found in OMCE schema.html. As an alternative, OMCE can read an alternative xml-format.

-fi= input format

0 OMCE format (default)

1 alternative xml-format

-fo As a default, OMCE forms the output file name based on the given input file name. With

this option an explicit output file name can be given.

-hc The resolution of the histogram is set by this option (number of bars). The value should be

between 1 and 100000. The default is 200 bars.

-hf The width of the histogram is extended by the factor given with this option. The value should

be between 0.1 and 100.0. The default is 1.2.

-hp The width of the histogram is controlled by a probability interval to capture the relevant

part of the distribution. The probability of the interval should be given as a value between

0.001 and 1. A value of 1 creates an interval that contains all values calculated during the

simulation. The default is 0.99 (99 %).

-i This options controls the interpretation of the coverage intervals. By default (-i=0), all

coverage intervals are probabilistically symmetric. If the option is non-zero then the shortest

coverage intervals are calculated.

-info If the info option is non zero, OMCE prints details about the xml input file or some internal

tables and quits.

-info= description

0 (no information)

1 details of the model given in the xml-file:

input quantities

equations

execution order

results

2 list of predefined symbols and functions

3 list of symbols usable in format definition files

4 list of reserved words not to be used as quantity names

5 list of error messages in LaTeX format

-io This option controls the sampling for imported quantities (see element <Import> in Section

4.1.9). If -io=0 (default) then the binary data is sampled at random from binary files. If

-io=1 then the data is sampled sequentially from the binary file in the same (reproducible)

order as it was written. If the number of runs (-mcs) is equal to the number of values in the

binary file then the exact same sequence for the quantities is reproduced.

-k The k-factor scales the tolerance check during the adaptive mode. The default value is 2.0.

A larger value will require more Monte Carlo runs. If this option is set to zero, the k-factor

will be evaluated from a t-table (see option -ap) look up based on the number of runs so far.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-l This option allows to repeat the simulation the given number of times is it is set not

equal to zero. Hint: This option is only effective if the omc-file does not contain a section

<Simulations> and will be ignored otherwise. The default value is -l=0.

-lc The correlation matrices will be printed if this option is set. The default is -lc=0.

7
-lc= description

0 no matrix printed (default)

1 print input correlation matrix

2 print result correlation matrix

3 print input and result correlation matrix

-lmn This option defines the matrix norm type.

-lmn= description

0 plain norm

1 weighted norm (default)

The weighted norm uses the larger correlation coefficient as a weight for the difference. The

minimum weight is 0.1.

-lt If this option is set (-lt=1), OMCE prints the tolerances used in the adaptive mode together

with the standard deviations evaluated (see section on adaptive mode for details).

-mcp This option selects the matrix correction procedure for the the correlation matrix in the event

that the matrix is found to be not positive semi-definite.

-mcp= description

0 spectral decomposition (default)

1 Near4 (N. J. Higham [6])

2 Random Walk

-mcs This option controls the number of simulation blocks. The total number of trails is the value

of this option multiplied by the block size (option -bs). This option will override the value

given in the xml-file. A value of zero or minus one activates the adaptive mode. The default

value is -1.

-mr To prevent the simulation from running for an excessive amount of time during the adaptive

mode, a maximum number of blocks (runs) can be set by this option. The value given here

is multiplied by the block size (see option -bs). The default value is 10000 blocks.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-nc This option controls the fraction of valid data that is needed to evaluate the correlation

coefficients. The default value is -nl=0.5. The simulation is aborted if the fraction of valid

data available to calculate the correlation coefficients in one data block is smaller than the

value controlled by this option. Lowering this value might only be advisable if the block size

is increased at the same time.

-nl This option controls the fraction of valid data that is needed to continue the simulation. The

default value is -nl=0.5. The simulation is aborted if the fraction of valid data in one data

block is smaller than the value controlled by this option. Lowering this value might only be

advisable if the block size is increased at the same time.

-ofd This option controls the file name for the output format definition file. The definition file

should be placed in the same directory as the OMCE executable or python source. The

default is ’OMCE’.

-p1 This option controls the exact probability of the 95 % interval. The default is -p1=0.95.

-p2 This option controls the exact probability of the 99 % interval. The default is -p2=0.99.

-p3 This option controls the exact probability of the 99.9 % interval. The default is -p3=0.999.

-pa This option controls the exponent used for the calculation of the average of the quantiles. If

a is the value set by this option then the average is calculated based on the equation

q ̄ = a

vuut

1

n

Xn

i=1

q

a

i

.

8
The default value is -pa=2.0

-pdf The data in the histogram can either be counts (-pdf=0) or samples of a PDF (probability

density function) (-pdf=1). The PDF samples are useful if the data is to be compared with

a given PDF. The default is -pdf=1.

-po If this option is given, OMCE will print a list of all effective options and quits. This option

is useful for documenting and debugging the command line options.

-pq This option controls the quantile estimation method. The details to the estimation methods

can be found in [7].

-lmn= description

0 no interpolation (see [2] Section 7.7.2) (default)

1 linear interpolation at nb × p + 1/2 (R-5)

2 linear interpolation at (nb + 1/3)p + 1/3 (R-8)

3 linear interpolation at (nb + 1/4)p + 3/8 (R-9)

4 linear interpolation at (nb + 2)p + 1/2

-pr The simulation is done in two steps. At the start a number of runs controlled by this option

will be executed. After the first runs are completed, the binning interval and the tolerance for

the adaptive mode are calculated and these values are used during the rest of the simulation

runs. The default is 10 first runs.

-sbs This option controls the sub block size for the alternate adaptive mode algorithms. The sub

block mechanism is not activated by default (-sbs=-1). The minimum sub block size is 100.

The block size must be devidable by the sub block size.

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-sd This option controls the number of significant digits for the adaptive mode. The default is 2

digits (-sd=2).

The adaptive mode is controlled by the following options: -ad, -af, -am, -ap, -at, -k, -mr,

-sd.

-seed The seed of the random number generator can be set to a fixed value given by this option.

A default value of zero will force a random seeding based on the system clock.

-sev This options controls the value of the smallest eigenvalue after correction. The default is

1E-7.

-sid This option controls the start value for the simulation ID. The default value is 0. The

simulation ID will be incremented after every simulation.

-t The OMCE will measure and print the time elapsed to run a complete simulation if -t=1.

More detailed information on the time elapsed for the omc-file reading, simulation and the

ohd-file creation will be printed if -t=2.

-v This option controls the verbose level between 0 for all messages and 10 for no messages. The

default value is 2.

-v= description

9 error messages

8 warnings

7 startup message

6 finish message

5 simulation progress

4 timing information

3 standard information

2 default value

1 additional information and calculation details

0 everything else

9
Note: information requested by other options (like -info, etc.) will be suppressed if the

verbose level is set larger as 3.

-vfd This option gives the file name for the var-file format definition file. The definition file should

be placed in the same directory as the OMCE executable or the Python source code. The

default is ’OMCE’.

-vm This option activates the validation mode if it is set. During validation mode all simulation

data are kept in memory and are analysed when the simulation runs are complete.

-wb The binary data calculated during the simulation can be written to a binary data file. This

option controls the generation and the format. For the sorted format (-wb=3) the binary data

will be sorted and made monotonic prior writing to the file. The option -wb=3 should not be

used if the data will be imported in other simulations via the element <Import>.

-wb= format description

0 no binary data file

1 OMCE format

2 simple format

3 sorted OMCE format

The default is -wb=0. See the section on binary data format for details.

-wv This option enables the creation of a .var output file. By Default no .var file is generated

(-wv=0).

-xsd The OMCE can validate the xml-input file against an xsd-file. With this option, the path to

the xsd-file can be given. By default, the option is ’-’ and OMCE.xsd is used for omc-files

and no validation is done for alternative xml-files. (see option -fi).

3.2 Example

OMCE S02.omc -pdf=1 -hc=400 -t=1

The OMCE will load the file S02.omc from the current directory and it will create the file S02.ohd

containing histogram data with a resolution of 400 bins converted to a PDF sampling. The whole

operation will be timed and the elapsed time will be shown.

4 OMCE file formats

The OMCE accepts two different xml-file formats discussed in the section about xml input formats. The

OMCE creates data files of different types. The raw data is written to binary files and the histogram

data and the other information is written to ASCII files in comma separated value format. The extension

is .ohd.

4.1 Xml input file format

The native OMCE input file format (extension omc) is a simple xml-format. It describes the model of

evaluation and the information about the input quantities. See OMCE\doc\OMCE_Schema.html for details

about the omc-file format structure. An omc-file has the following basic structure:

<?xml version="1.0" encoding="UTF-8"?>

<Model

Name="Model1"

Options=""

xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"

xs:noNamespaceSchemaLocation="OMCE.xsd">

10
<Parameters>

...

</Parameters>

<Quantities>

...

</Quantities>

<Correlations>

...

</Correlations>

<Functions>

...

</Functions>

<Equations>

...

</Equations>

<Constraints>

...

</Constraints>

<Results>

...

</Results>

<Simulations>

...

</Simulations>

</Model>

A <Model> element contains all information about the evaluation model. The Name and the Options

attributes are optional. The Options attribute may contain a list of space separated command line

options. The information about the schema is optional and is not used in OMCE. The validation schema

is controlled by the option -xsd.

4.1.1 Parameters

The optional section <Parameters> defines a list of symbolic model parameter values that can be used

in the parametric specifications of the quantities. In this section the default values of the parameters are

specified. A parameter specification has the following structure:

<Parameter Name="P" Value="1.0"/>

All attributes are mandatory. The attribute Name defines the parameter name (e.g. ”P”) by which the

parameter is referred to in the definition of the quantities. The attribute Value assigns the default value

to the parameter. The value of the parameter can be re-assigned by a command line parameter (see

Section 3) or inside the section <Simulations>.

4.1.2 Quantities

The section <Quantities> defines a list of quantities which can be used in the model equations. A

quantity has the following structure:

<Quantity Symbol="X" Unit="kg"> ... </Quantity>

The attribute Symbol is mandatory and defines the symbol name (e.g. ”X”) by which the quantity is

referred to in the equations. The attribute Unit is optional. A quantity element should contain one

distribution element.

11
4.1.3 Correlations

The section <Correlations> is optional and defines a list of correlation coefficients which are used during

the simulation runs. A correlation coefficient has the following structure:

<Coefficient Q1="X_1" Q2="X_2" Value="0.5" />

All attributes are mandatory. Attribute Q1 and Q2 define the quantities for the correlation coefficient

and Value defines its value. The value must be greater than or equal to -1 and smaller than or equal

to +1. All quantities where a correlation coefficient is given must be normal distributed. All correlation

coefficients together must form a matrix which is positive semi-definite.

4.1.4 Functions

The section <Functions> is optional and defines a list of user defined functions to be used in <Equation>

and <Result> elements. A function definition has the following structure:

<Function Symbol="f" Parameter="p1,p2" Global="X_1,X_2"> ... </Function>

The attribute Symbol is mandatory and defines the function name (e.g. ”f”) by which the function is

referred to in <Equation> and <Result> elements. The optional attribute Parameter defines a parameter

list for the function. The optional attribute Global defines a list of quantities or equation names that

can be used inside the function in addition to the given parameters. The element <Function> should

contain a valid expression.

4.1.5 Equations

The section <Equations> defines a list of equations which are evaluated during the simulation runs. A

equation has the following structure:

<Equ Symbol="Y" Unit="kg"> ... </Equ>

The attribute Symbol is mandatory and defines the symbol name (e.g. ”X”) by which the equation is

referred to in the result list or in other equations. Every equation must have a unique symbol name.

The attribute Unit is optional. An element <Equ> should contain an expression which can be evaluated.

4.1.6 Constraints

The section <Constraints> defines a list of constraints which need to be met by the data during the

simulation. A constraint has the following structure:

<Constraint Q="X"> ... </Constraint>

The attribute Q is mandatory and defines the name of the quantity (e.g. ”X”) referencing the data which

is validated by the constraint. An element <Constraint> should contain a logical expression which can

be evaluated to True or False. Logical expressions for constraints on input quantities can only refer to

input quantities while logical expressions for constraints on quantities defined by equations can refer to

any quantities. Quantity names referring to results cannot be used with constraints.

4.1.7 Results

The section <Results> defines a list of results which are observed during the simulation runs. A result

has the following structure:

<Result Symbol="Y">Y</Result>

12
The attribute Symbol is mandatory and defines the symbol name (e.g. ”Y”) by which the result is

referred to in the ohd- or var-file. The same symbol name may be used for both a result and an equation.

A result element should contain a valid expression. Usually this expression contains only the name of an

equation. Result symbols cannot be used in expressions for equations or results.

4.1.8 Simulations

The optional section <Simulations> provides a simple tool to define multiple simulations which are

sequentially executed. If the section is not present in an omc-file, only one simulation is executed. All

the result data for the simulations is written to the same file. The section <Simulations> contains a list

of the elements <Simulation> and <Loop> given in any order.

The element <Simulation> defines one simulation run and it has the following structure:

<Simulation Name="Comment">

<Parameter Name="P" Value="2.0"/>

...

</Simulation>

The attribute Name is optional and it specifies a comment that will be printed when the simulation is

executed. The element <Simulation> can contain a list of parameter re-assignments. All parameter

must be defined in the section <Parameters> before they can be used here.

The element <Loop> defines a for-loop for multiple simulations and it has the following structure:

<Loop Enum="i" From="1" To="5" Step="1.0" Name="Comment">

<Parameter Name="P" Value="2.0*i"/>

...

</Loop>

The attribute Enum is optional and specifies a name for the enumerator. If the attribute is present the

name given in Enum can be used to evaluate the values of the parameters.

The attributes From and To are mandatory and they define a range for the loop enumerator. The optional

attribute Step specifies the value of the increment that is added (or subtracted) to the enumerator after

every simulation. The default value for the Step is 1.0.

Note: The loop will always start with a value for the enumerator given in From independent if the value

given in To is smaller or larger than From. The given sign of Step is ignored and the sign is adjusted so

that the numerator is changed after every run in the direction of To.

The attribute Name is optional and specifies a comment that will be printed when the simulation is

executed.

4.1.9 Distribution elements

A distribution element defines the information which is available about the quantity. The following

distribution elements can be used inside a quantity element:

<Constant> <Value>1.0</Value> </Constant>

The element <Constant> defines a constant quantity without uncertainty.

<Normal> <Mean>0.0</Mean> <Sigma>1.0</Sigma> </Normal>

The element <Normal> defines a normally distributed quantity with the specified mean value

and the specified standard deviation (Sigma).

<Student> <Mean>0.0</Mean> <Scalefactor>1.0</Scalefactor> <Dof>5</Dof> </Student>

The element <Student> defines a student-t distributed quantity with the specified mean value

and degrees of freedom (Dof) scaled by the specified scaling factor (Scalefactor).

13
<Rectangle> <Mean>0.0</Mean> <Halfwidth>1.0</Halfwidth> </Rectangle>

The element <Rectangle> defines a rectangularly distributed quantity with the specified

mean value and the specified half-width (semi-width).

<Triangle> <Mean>0.0</Mean> <Halfwidth>1.0</Halfwidth> </Triangle>

The element <Triangle> defines a triangularly distributed quantity with the specified mean

value and the specified half-width (semi-width).

<Trapez> <Mean>0.0</Mean> <Halfwidth>1.0</Halfwidth> <Beta>0.5</Beta> </Trapez>

The element <Trapez> defines a trapezoidal distributed quantity with the specified mean

value and the specified halfwidth (semi-width). Beta is the ratio between the evenly distributed part and the width of the distribution.

<Ctrapez> <Mean>0.0</Mean><Halfwidth>1.0</Halfwidth><Beta>0.5</Beta> </Ctrapez>

The element <Ctrapez> defines a iso-curvilinear trapezoidal distributed quantity [8] with the

specified mean value and the specified half-width (semi-width). Beta is the ratio between the

evenly distributed part and the width of the distribution.

<Ushape> <Mean>0.0</Mean> <Halfwidth>1.0</Halfwidth> </Ushape>

The element <Ushape> defines a U-shaped (arcus-sinus) distributed quantity with the specified mean value and the specified half-width (semi-width).

<TypeA> <Value>1.0</Value> ... </TypeA>

The element <TypeA> defines an observed quantity. It contains at least 2 <Value> elements.

<Poisson> <Mean>1.0</Mean> </Poisson>

The element <Poisson> defines a Poisson distributed quantity with the specified mean value.

<Exponent> <Mean>1.0</Mean> </Exponent>

The element <Exponent> defines a exponentially distributed quantity with the specified mean

value.

<Gamma> <Shape>1.0</Shape> <Scale>1.0</Scale> </Gamma>

The element <Gamma> defines a Gamma distributed quantity with the specified shape and scale

value.

<Discrete> <Binfile Filename="filename.bin" Index="0" Q="Y"/> </Discrete>

The element <Discrete> defines a discrete distribution based on binary data (see [2] Section

7.5.1). The element <Discrete> contains the mandatory element <Binfile> defining the

binary data source. The binary data is read from a file given by the attribute Filename. The

file should exist and it should have a valid binary format (see option -wb). The path should

be either relative to the current XML-file or an absolute file path.

Binary files can contain binary data for multiple quantities. The optional attribute Index

defines the index of the quantity in the binary file. The first quantity is the default and has

the index 0. If the binary contains a symbol table the optional attribute Q can specify the

name of the quantity instead of its index. The attribute Index will be ignored if an attribute

Q is specified.

After the element <Binfile> two optional elements <Shift> and <Scale> can be used to

define a scale value all data is multiplied with and a shift value which is added to all (shifted)

values. Multiple quantities might share the same binary data with different shift and scale

values. Although the these quantities share the same data, they are uncorrelated.

<Import> <Binfile Filename="filename.bin" Index="0" Q="Y"/> </Import>

The element <Import> is similar to the element <Discrete>. It contains only the element <Binfile>. No other optional elements are allowed. See the paragraph above about

<Discrete> for details about the element <Binfile>. As long as only one quantity is imported per file there is virtually no difference between element <Discrete> and element

<Import>. The user might use the the element <Discrete> if the distribution needs to be

scaled or shifted.

14
If multiple quantities are imported from the same file with the element <Import> then the

dependency structure (correlation) present in the binary data will be maintained in the

current simulation by sampling jointly for all quantities from this file. Option -io controls if

a random sampling (-io=0) or a sequential sampling (-io=1) is used.

The same binary file can be used for a <Discrete> element and for a <Import> element by

using the same filenames and indices. The quantities share the distribution but they are

sampled independently (not correlated).

4.1.10 Expressions and equations

The elements <Equ>, <Function> and <Result> require an expression as their content. The format of

the expression is basically the format of a Python expression. The operator +, -, *, / and ** can be used

in expressions together with predefined functions, constants and the parentheses. The option -info=2

lists the predefined functions and constants.

Predefined functions

Function Description Remarks

abs() Absolute value, Modulus

acos() Arc cosine

asin() Arc sine

atan() Arc tangent

cos() Cosine

cosh() Hyperbolic cosine

exp() Natural exponential function (base e)

iff() Value selection see Section 4.1.13

log() Natural logarithm (base e)

log10() Common logarithm (base 10)

max() Maximum value see Section 4.1.11

maximum() Maximum value see Section 4.1.11

min() Minimum value see Section 4.1.11

minimum() Minimum value see Section 4.1.11

pow() Power see Section 4.1.12

sin() Sine

sinh() Hyperbolic sine

sqr() Square

sqrt() Square root

tan() Tangent

tanh() Hyperbolic tangent

Predefined constants

Constant Description Value

e Euler’s number 2.7182818284590451

pi number pi 3.1415926535897931

The predefined functions and constants are a direct mapping of the NumPy functions with the same

name. For details consult the NumPy documentation [4]. Most of the functions except minimum(),

min(), maximum(), max(), pow() and iff() have one argument. All function return one result.

OMCE allows to use distribution functions in <Equ>, <Function> and <Result> elements. These functions are useful to model statistical effects without assigning a quantity name to it. This is particular

useful in function definitions to create certain statistically independent properties every time the function

is used. All parameters of the distribution functions must evaluate to constant values by either being a

numerical constant or an arithmetic expression using only quantities which are defined as constants or

by using the quantity access functions val() or u() (see below).

15
Predefined distribution functions

Function Description Parameter

Rectangle() Rectangular distribution Mean value, half-width

Normal() Normal distribution Mean value, standard uncertainty

Student() Student-ıt distribution Mean value, scale factor, degrees of freedom

Triangle() Triangular distribution Mean value, half-width

Trapez() Cosine distribution Mean value, half-width, beta factor

Ushape() U-shaped (arcus-sinus) distribution Mean value, half-width

Poisson() Poisson distribution Mean value

Ctrapez() Iso-curvilinear trapezoidal distr. Mean value, half-width, beta factor

Exponent() Exponential distribution Mean value

Gamma() Gamma distribution Mean value

Laplace() Laplace distribution Mean value, scale factor

The OMCE defines two quantity access functions to access the mean value and the standard uncertainty

of named input quantities. The result of a quantity access function is a constant. These functions can be

used together with the distribution functions to model dependencies between the parameters of different

distributions.

Predefined quantity access functions

Function Description Parameter

val() Value function to access the mean value of an input quantity quantity name

u() Uncertainty function to access the standard uncertainty of quantity name

an input quantity

OMCE supports evaluation models with multiple equations. The result of one equation may be used

in the calculation of another equation. But no mathematical loops are allowed. It must be possible to

evaluate the equation system in a linear execution order. The model will be checked for any mathematical

loop before the simulation starts.

4.1.11 Function minimum() and maximum()

The functions minimum() (min()) and maximum() (max()) have one ore more parameter. The result

of the function is the minimum or maximum of all the given values. These functions might produce a

singular value in the PDF of the result if one of the parameter is a constant.

4.1.12 Function pow()

The function pow() can be alternatively used instead of the exponential operator **. The function has

two parameters. The first is the basis and the second is the exponent.

4.1.13 Function iff()

The function iff() allows to switch between two values based on a specified condition. The function

has three parameter. The first parameter is a logical expression. If the logical expression evaluates to

True then the second parameter will be returned as a result. If the condition evaluates to False then

the third parameter will be returned.

This function should be used with great care since it can produce serious discontinuities in the equation

system. The use of this function might violate the basis for the application of GUM Supplement 1 [2]

and the use is therefore not recommended. The function is provided for experimental purposes only.

4.1.14 Logical expressions

Logical expressions are used in constraints and as the first parameter of the iif()-function. They must

evaluate to a boolean value (True or False). Usually logical expressions used in simulations compare

16
data with limits or other data. The following comparison operators can be used:

Comparison operators

Comparison Operator Alternative Remarks

operator

equal to == .eq use with care in constraints

greater than > .gt

less than < .lt

greater than or equal to >= .ge

less than or equal to <= .le

not equal to != .ne

The characters < and > are used as special characters with XML-files and therefore cannot be entered

directly in logical expressions. Use either the escape sequences > and < or the alternative dot

operators instead.

Multiple comparisons can be combined with logical operators. The following operators are supported:

Logical operators

Operator Description

and logical and

or logical or

not logical complement

Some examples for valid logical expressions:

X .ge 0

X_1 < X_2

(X .ge 1) and (X .lt 2) and not(Y .lt 0)

The parenthesises are essential when combining comparisons.

4.1.15 Symbol names

Symbol names for quantities und equations are limited to the leters A. . . Z, a. . . z, the numbers 0. . . 9

and the underscore character (” ”). Additionally, the @-character can be used, but it will be translated

to a double underscore (” ”) internally (e.g. A@A is the same as A A). Therefore the @-character and

the double underscore should not be used in the same model. The following words are reserved words

and cannot be used as symbol names: and, as, assert, break, class, continue, def, del, elif, else, except,

exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while,

with, yield.

4.2 Binary data file formats

During the simulation a binary data file can be written (see option -wb). The OMCE supports two

different binary formats.

4.2.1 OMCE binary format

The native OMCE binary format is a versatile binary format optimised for reading and writing speed.

It has a block structure which corresponds to the processing block size of the data generation part of the

simulation engine. The OMCE binary file has the following structure:

OMCE binary file format

header symbol table binary data area

8 × 4 bytes list of symbol names runs × results × 8 bytes

17
The header is 32 bytes long and contains 6 × 32-bit integer values and 8 unused bytes. The header has

the following structure:

OMCE binary header

-1330463557 m results n runs block size b data offset flags not used

32-bit int. 32-bit int. 32-bit int. 32-bit int. 32-bit int. 32 bits 2 × 4 bytes

The first integer in the header is a format identifier and contains the value -1330463557. The following

fields describe the data structure in the binary data area and the number of bytes used for other purposes

(offset to the data). The offset to the data should be a multiple of 8 bytes.

If bit 0 in the flag word is one then a symbol table is included in the binary file. If a symbol table is

present, it will directly follow the header. The symbol table contains ASCII character strings with names

terminated by a zero character (00). The end of the list is an empty string or two zero characters. The

symbol table is padded with additional zero characters to fill a block with a size of a multiple of 8 bytes.

The binary data contains a number of data blocks. A data block contains double precision floating point

values (64-bit IEEE 754-1985) for one result only. The size is determined by the block size (see option

-bs). If more than one result is specified the first data block of the second result will follow the first

data block of the first result and so on until all first data blocks for all results are written. The same

scheme will be followed up to the last block of the last result. In case that the total number of runs is

not a multiple of the block size b, then the last block will be smaller than the standard block size.

Binary data structure

data block 0, result 0 . . . data block 0 result m . . . data block k result m

b × 8 bytes . . . b × 8 bytes . . . bk+1 × 8 bytes

A binary file reader needs to read the header first to find out about the data structure. The total number

of floating point values (64-bit) in the binary file is the product of results and runs m × n given in the

header. The reader can determine the number of full data blocks k by dividing the number of runs by

the block size given in the header n/b (integer division). In case the total number of values is not a

multiple of the block size, then the last block is smaller and has a block size bk+1 which can be calculated

from the number of runs modulo the block size bk+1 = n mod b (modulo). After reading the header the

reader should either read the symbol table if it is present or skip the number of bytes specified by the

data offset in the header. The binary file contains information other than result values in this area. The

value of the data offset might be zero. The unused integers in the header may be used later. The reader

should not rely on them to have any specific value although they are set to zero by the OMCE so long

they are not used.

A binary reader can distinguish between the OMCE binary format and the simple binary format (see

next section)) by the value of the first entry in the header. The OMCE header contains a special negative

number in this position while the number of runs n, the first integer in simple format, should always be

positive.

4.2.2 Simple binary format

The simple binary format is an interleaving format like the OMCE format but with a fixed block size of

1 value per block. The simple binary file has the following structure:

Simple binary file format

Header result 0, value 0 result 1, value 0 ... result 0, value 1 ... result n, value m

2 × 4 bytes 8 bytes 8 bytes ... 8 bytes ... 8 bytes

The header contains 2 × 32-bit integer values and is followed by a series of alternating values for all

results. The header has the following structure:

Simple binary header

n runs m results

32-bit integer 32-bit integer

18
A binary file reader for the simple format may read the header first to find out about the number of

runs and the number of results in the data file. While reading the data file the reader might separate

the values for the different results and form larger blocks of binary values per result to optimize the

later data handling. Reading and writing in simple format is less efficient than in OMCE format. The

conversion from internal arrays in OMCE to the simple interleaving format of one value per block and

back consumes significant computational effort.

4.3 Text data file format

OMCE supports the generation of two different output files in comma separated value format. The ohdfile contains the histogram data and the var-file contains the link between the binary data file and the

result quantity names given in the model description in the omc-file or xml-file and optinal correlation

information. An ohd-file is generated when -a=1 is activated (default) and and a var-file is generated

when -wv is set (deactivated by default). If a binary data file is written -wb <> 0 then option -wv=1

should be set as well.

The content and the format of the text files (.var and .ohd) can be customized (see option -ofd and -vfd).

The format of the different text output files is controlled by format definition files. The format definition

files are text files themselves and they have a line based format. Every line in a format definition file

is either a comment (starting with ’#’), an option (starting with ’-’) or a column definition (having an

equal sign). All lines which are not options and do not contain an equal sign are ignored.

This is the format definition for the OMCE var-file format:

#

# Native OMCE var-file format definition

#

-separator=","

-header=1

-extension=".var"

-quotechar="

SID=SID

ID=id

Symbol=name

Runs=runs

Binfile=filename

R[%s]=CORR

It defines a comma separated text file with the file name extension .var. The file has a header row and

5 or more columns named SID, ID, Symbol, Runs, Binfile and R[0] to R[n-1] for n results.

The following options should be used in the format definition files:

-extension This option defines the extension to the output file name. The extension should start

with a point on the line. The value should be put into quotes.

-header This option controls whether or not the file contains a header line with the column

headings. A value not equal 0 will generate a header line.

-quotechar This option controls the quoting character. No quotes should be used with this option.

-separator This option controls the character to separate the columns in the data file. Use the

combination \t for the tab-character. The value should be put into quotes.

All lines which are neither a comment nor contain an option define a column in the data file if they

contain an equal sign. The column name will be the string left of the equal sign and the content of the

column is evaluated from the string right of the equal sign. The right side should contain a predefined

symbol (see option -info=3 for a list of implemented symbols and the source code for details).

19
The symbol BINS, CORR and IRCORR are used to specify the histogram data and correlation data. The

string left of the equal sign must contain the sequence %s which will be replaced by an ident number or

a name of an input quantity.

In case the string on the right side is quoted it will be taken literally and a fixed string is written in this

column.

5 Histogram viewer

The OMCE includes a simple viewer to show the result in the text file containing the histogram. The

viewer can be executed with the following command:

Viewer <filename.ohd> [<options>]

The extension of the histogram data file must be given. The viewer reads the histogram data file and

shows histogram plots for the results as seperate plots. The plots can be saved in various graphic formats.

To run the source code version of the viewer, the python module matplotlib [9] should be installed.

5.1 Options

The viewer supports the following options:

-mark The option -mark specifies vertical markers in the diagram to mark for example the expectation value and the coverage interval. The value of this option must be a list of marker

specifications enclosed in square brackets [] A marker specification consists of a comma separated list of four values enclosed in parenthesis defining the column name, the color, the line

style and the line width of the marker. The default value is:

-mark=[(Mean,b,dashed,1),(Low95,b,dashed,1),(High95,b,dashed,1)]

The column names are referring to the names given in the ohd-file.

-max This option specifies the limit for the maximum number of plots that will be generated based

on the result data file (.ohd). The default is -max=20. If the number of results in the result

data file exceeds the limit specified with this option, only the first results are plotted.

-sid This option specifies a range selection of simulation ident numbers (SID). The format of the

option is -sid=<min sid>:<max sid> with <min sid> and <max sid> specify the lower and

upper limit of simulation id’s of the results which are plotted. The seperator is a colon (”:”).

If any of the values are obmitted then the minimum respectively the maximum SID in the

file will be used. This option is only useful if the result data file contains data from different

simulation runs.

6 Adaptive mode

The OMCE supports the automatic determination of the total number of Monte Carlo runs by the

method described in the GUM Supplement 1 section 7.9 [2] (-am=0, default) or by alternate algorithms

discuss in section 6.2. The adaptive mode is activated if the number of Monte Carlo blocks is set to zero

(see option -mcs=0).

6.1 GUM Supplement 1 based algorithm

The GUM S1 based algorithm performs the simulation in two steps. In the beginning a number of initial

runs npr, controlled by the option -pr, will be executed. After the initial runs the binning intervals are

calculated and OMCE continues with performing additional runs.

20
After every additional run the tolerances Ts, Th and Tl are clculated. Based on these tolerances OMCE

decides when to terminate the simulations. The tolerances are calculated based on the number of

significant digits ndig controlled by the option -sd and the tolerance divisor da controlled by the option

-ad.

For each data block i the average yi, the standard deviation si, and the interval limits hi and li are

calculated. Based on the nr data blocks simulated so far, the tolerances are calculated by

Ts =

5 × 10mag( ̄s)−ndig

da

Th =

5 × 10mag(h ̄−y ̄)−ndig

da

Tl =

5 × 10mag( ̄y− ̄l)−ndig

da

with

s ̄ =

vuut

1

nr

Xnr

i=1

s

2

i

, h ̄ =

1

nr

Xnr

i=1

hi, y ̄ =

1

nr

Xnr

i=1

yi,

 ̄l =

1

nr

Xnr

i=1

li.

The function mag(x) returns the order of magnitude of x.

During the additional simulation runs a multiple k of the standard deviation of the mean of the three

parameters si, hi and li are compared with their tolerances. The simulation terminates if all parameters

are within their limits. The multiplier k is controlled by the option -k. With the total number of

simulation runs nr the termination criteria are

bMEAN =



k × stddev(y1 . . . yn)

√

n

≤ Ts



bUNCERTAINTY =



k × stddev(s1 . . . sn)

√

n

≤ Ts



bLIMITS =



k × stddev(h1 . . . hn)

√

n

≤ Th



∧



k × stddev(l1 . . . ln)

√

n

≤ Tl



.

The function stddev(x1, . . . , xn) returns the experimental standard deviation of the valid data in the

series x1 . . . xn (see Section 10.1).

Which criterion is actually in use is controlled by the option -at. The value of this option is a bit

combination which enables the different criteria.

bTERMINATION = (¬bit(0) ∨ bMEAN) ∧ (¬bit(1) ∨ bUNCERTAINTY) ∧ (¬bit(2) ∨ bLIMITS).

The function bit() returns a logical value true if the bit in option -at with the number given as an

argument is set and the value false otherwise. By default all termination criteria are used (-at=7).

6.2 Alternate algorithms

The OMCE supports alternate adaptive algorithms for the determination of the total number of Monte

Carlo runs. The algorithms are chosen by the option -am. The standard GUM S1 algorithm is the

default (-am=0).

The numerical accuracy of all algorithms is controlled by the same options. The number of significant

digits ndig are controlled by the option -sd and the tolerance divisor da is controlled by the option -ad.

The probability level p for the test is controlled by the option -af and the block size nb is controlled by

the option -bs.

21
The algorithms selected with options -am=1 and -am=2 use a modified Stein-method [5]. After the initial

npr simulation blocks controlled by option -pr, this methods estimates the total number of runs bases

on a confidence interval.

The algorithms selected with options -am=3 and -am=4 are based on the fact that the numerical accuracy

is in principle defined relative to the variance, which allows the exact knowledge about the number of

runs needed for normally distributed results.

The accuracy definition in GUM S1 is defined as half of the least significant digit and therefore the

relative accuracy is dependent on the actual digits of the uncertainty (between 0.5% for the digits ”99”

and 5% for digits ”10”) the number of runs needed to meet this specification is dependent on the digits

of the uncertainty.

The algorithms selected with options -am=1 and -am=3 use the definition of the numerical accuracy as

defined in GUM S1 while the algorithm selected with option -am=2 and -am=4 use always the highest

accuracy (assuming the digits of the uncertainty are always ”99”).

The algorithm selected with option -am=3 uses a simple scheme to adapt the number of runs according

to the digits of the uncertainty.

At the end all algorithms perform a statistical test to ensure that the result meets the numerical accuracy

even in cases when the results are not normally distributed.

The alternate algorithms support also the option -at to limit the checking of the statistical parameter

to the mean (bit 0) or the variance (bit 1). Bit 2 has no function with these algorithms.

6.2.1 Sub-blocking

A sub-block mechanism is used to improve the statistical properties of the statistical evaluations. The

result data blocks of size nb are divided in sub-blocks of size nsb (controlled by option -sbs) resulting in

ns = nr × nb/nsb sub-blocks for nr simulated blocks. The standard deviation of the values in sub-block

number i is si(ysb) and the mean value of the same block is  ̄ysb(i). The degrees of freedom are improved

by the ratio nb/nsb. The sub-blocking is implemented efficiently without any data copying by creating

a second view on the data. The sub-blocking is advantageous compare to a reduction of the block size

because the performance of a Python based simulator and the accuracy of the evaluation of the coverage

intervals are dependent on a large block size (≥ 10000).

6.2.2 Establishing the statistical confidence level

As proposed in [5] OMCE uses a t-factor, based on the number of sub-blocks ns, to achieve the numerical

accuracy. As long as the sub-block mean values  ̄ysb(i) and the sub-block standard deviations si(ysb) can

be considered as random samples of a distribution with an existing variance, the variance of the overall

mean and the overall variance will be improved by a factor of ns. Therefore it is possible to use the

function Nsb(s, d) to calculate an estimate of the number of blocks needed to achieve the numerical

accuracy based on the standard deviation s and the tolerance d.

Nsb(s, d) = max int 

nsb

nb

×

s

2 × tp(ns − 1)2

d

2

+ 0.5



, 1



The function max [n1, . . . , nl] returns the maximum value of the values n1 . . . nl and the function tp(ns−1)

returns the t-factor based on the probability p and the degrees of freedom ns − 1. In case ns is large

(> 100), the t-factor could be replaced by quantile function of a normal distribution.

6.2.3 Numerical tolerance

The numerical tolerances can be either established by using the GUM S1 definition that the tolerance

is half of the least significant digit of the uncertainty or the tolerance can be defined as relative to the

uncertainty of the result.

22
The tolerance for the mean value dm based on the GUM S1 definition is

dm = 0.5 ×

10mag(u(y))+1 × 10−ndig

da

and the relative tolerance is dependent on the digits of the uncertainty (between 0.5% for the digits ”99”

and 5% for digits ”10”). The function mag(x) returns the order of magnitude of x. This tolerance is

used by the the algorithms selected with options -am=1 and -am=3.

For the algorithms selected with options -am=2 and -am=4, the relative tolerance is fixed (assuming the

digits of the uncertainty are always ”99”).

dm = 0.5 ×

u(y) × 10−ndig

da

.

The tolerance for the variance dv is for all algorithms

dv = 2 × u(y) × dm.

6.2.4 Predicting the number of simulation blocks

The algorithms selected with options -am=3 and -am=4 are based on the fact that the relative numerical

accuracy of the mean value is in principle predictable if the mean value is calculated based on random

samples from a distribution which has an existing variance.

Based on the number of digits ndig and the tolerance divisor da the relative tolerance factor m can be

calculated.

m = 10ndig × da

Assuming that the number of simulations is large ( 10000) we can assume that the mean value is

normally distributed and we can use the quantiles of a normal distribution to calculate a confidence

interval. The variance of the mean value is proportional to 1/n. Therefore we can predict the number

of simulation runs needed to establish the relative accuracy m. This number divided by the block size

nb and rounded to the nearest integer greater than or equal to one is the number of simulation blocks

n0 to achieve the relative accuracy 1/m.

n0 = max int 

m2 × norm(1 − (1 − p)/2)2 + nb − 1

nb



, 1



The function norm(p) calculates the quantile of the probability p of a normal distribution and the function

int (x) returns the integer part of x.

The algorithm selected with option -am=3 uses a multiple of n0 to adapt the number of runs to the digits

of the uncertainty.

The relative accuracy m is the correct numerical accuracy for the uncertainty digits ”50” based on the

GUM S1 definition. The correct numerical accuracy for the uncertainty digits ”99” which is needed

for the algorithm selected with option -am=4, results in a relative accuracy of 2 × m and therefore in a

number of simulation blocks n1.

n1 = max int 

4 × m2 × norm(1 − (1 − p)/2)2 + nb − 1

nb



, 1



The value of n0 is the number of simulation blocks needed to improve the variance of the mean value by

m and n1 is the number of simulation blocks needed to improve the variance of the mean value by 2 ×m.

23
6.2.5 Algorithms based on the Stein-method

The algorithms selected with options -am=1 and -am=2 use a modified Stein-methods [5]. After the initial

npr simulation blocks controlled by option -pr, this methods estimate the total number of runs bases on

confidence intervals for the mean and the variance using the function Nsb defined in Section 6.2.2.

nr = max Nsb(stddev( ̄ysb(1), . . . , y ̄sb(ns)), dm), Nsb(stddev(s

2

1

(ysb), . . . , s2

ns

(ysb)), dv)



.

The methods execute nr − npr additional runs and perform at the end the statistical test discussed in

Section 6.2.7.

6.2.6 Algorithms based on a prediction of the needed runs

The algorithms selected with options -am=3 and -am=4 are based on a prediction of the runs needed (see

Section 6.2.4).

If option -am=3 or -am=4 are selected then option -pr will be ignored and a number of n0×nb simulations

are run as initial runs.

After the initial n0 runs the algorithm selected with option -am=4 executes a fixed number of n1 − n0

runs as additional runs.

The algorithm selected with option -am=3 uses the significant digits du of the uncertainty of the result

u(y) determined by the simulations so far to evaluate the need for additional runs.

du =

u(y)

10mag(u(y))−1

The function mag(x) returns the order of magnitude of x.

The value of du is used to judge if additional runs are needed.

if du < 10 + 10/m or du > 50 − 10/m:

DoSimulation(n0)

if du < 10 + 10/m or du > 70 − 10/m:

DoSimulation(n0)

if du < 10 + 10/m or du > 85 − 10/m:

DoSimulation(n0)

The function DoSimulation(n) executes n simulation blocks and m is the relative tolerance factor

defined in Section 6.2.4. Hint: The value of du is re-evaluated after the execution of DoSimulation().

After the execution of the initial runs and the additional runs in total nr simulation blocks have been

executed. At the end both algorithms perform the statistical test (see next section).

6.2.7 Statistical end test

The improved adaptive algorithms selected by the option -am > 0 test the simulation results at the end

to verify that the standard deviation of the mean and the standard deviation of the variance meet the

tolerance specified by ndig, da and p. The sub-blocking mechanism (controlled by option -sbs) is used

to improve the statistical properties of this test (see Section 6.2.1).

The test verifies that the number of runs is greater than or equal to the square of the confidence interval

based on a t-factor tp(ns−1) and the standard deviation divided by the square of the calculated tolerance.

The test uses the function Nsb(s, d) defined in Section 6.2.2 with the tolerances dm and dv discussed in

Section 6.2.3.

The minimum number of runs nmin ensuring that the statistical properties for the mean value and the

variance are met is

nmin = max Nsb(stddev( ̄ysb(1), . . . , y ̄sb(ns)), dm), Nsb(stddev(s

2

1

(ysb), . . . , s2

ns

(ysb)), dv)



.

24
The function stddev(x1, . . . , xn) returns the experimental standard deviation of the valid data in the

series x1 . . . xn (see Section 10.1) and the function max [n1, . . . , nl] returns the maximum value of the

values n1 . . . nl.

WAn iterative test is used to verify the statistical properties since the number of sub-blocks ns is large

(> 100).

while nr < nmin:

DoSimulation(1)

nr = nr + 1

Hint: The value of nmin is re-evaluated after every execution of DoSimulation().

In case the sub-block size is large enough (nsb ≥ 200) and the distribution of the result y is not extreme

and the variance of the variance exists, one can assume a close to normal distribution for the mean values

y ̄sb(i) and the variances s

2

i

(ysb) of the sub-blocks (Central Limit Theorem). In this case the standard

deviations of the averages will be reduced by a factor of ns and the total number of runs can be estimated

or predicted by the algorithms very precisely. Then the statistical test at the end will find only in a

small number of cases that the number of runs nr is too small (< 5%, one additional block). The need

for a larger number of runs during the end test might be an indication that the Central Limit Theorem

is not applicable for the specific simulation problem.

7 Running multiple simulations

The OMCE is a command line program that is designed to be used in batch mode. In case it is necessary

to execute a Monte Carlo simulation several times based on the same underlying model, one can either

use the batch processing of the operating system and call OMCE several times or one can use the section

<Simulations> in the omc-file to define multiple simulations. If some of the parameters need to be

changed between the runs then they must be defined in the section <Parameters>. Using the batch

processing of the operating system has the advantage that several simulations can run in parallel and

that the same batch processing could be used for different models by changing only the omc-file name.

Using the section <Simulations> has the advantage to combine everything in one input file and to

generate one output file.

The following example demonstrates the use of the section Simulations to repeat the simulation three

times and re-assigning the value of P to 1.0, 4.0 and 9.0.

<Simulations>

<Simulation Name="First">

<Parameter Name="P" Value="1.0"/>

</Simulation>

<Simulation Name="Second">

<Parameter Name="P" Value="4.0"/>

</Simulation>

<Simulation Name="Third">

<Parameter Name="P" Value="9.0"/>

</Simulation>

</Simulations>

Alternatively, the same result can be achieved by using the element Loop.

<Simulations>

<Loop Name="Loop" Enum="i" From="1" To="3">

<Parameter Name="P" Value="i**2"/>

</Loop>

</Simulations>

25
The value of a parameter can be evaluated using mathematical operators and predefined mathematical

functions.

It should be noted that the parameter re-assignment in the section simulations override the parameter

values given in the command line. So care should be taken if a omc-file with a section simulations is

used together with batch processing.

8 Input correlation coefficients

The matrix based on the correlations coefficients given in the omc-file should be positive semi-definite.

This is checked by Cholesky decomposition and eigenvalue decomposition. In cases when the matrix

found to be not positive semi-definite the matrix is corrected by the chosen correction procedure (-mcp).

The default procedure is to shift all eigenvalues which are smaller a given limit (-sev) to the limit

and compose a new correlation matrix. A warning message together with the values of the modified

correlation matrix is printed if a modification is necessary. The modification of the matrix is controlled

by applying a norm [10] to the difference of the shifted and the non-shifted matrix. The norm must be

smaller than a given limit (see option -e2) or the engine terminates with an error message.

9 Analysis of correlation

The OMCE can analyse the correlation in the results data. The analysis is controlled by the option -ac.

The correlation is evaluated pair-wise by

r(qi, qj ) = 1

(n − 1) s(qi) s(qj )

Xn

l=1

(qi,l − qi) (qj,l − qj )

with qi,l and qj,l being the data of the two results with the average qi and qj and the standard deviation

s(qi) and s(qj ). The correlation is evaluated for every block and the result correlation matrix is the

average over all blocks. For m results (m2 − m)/2 correlation coefficients are calculated. The correlation

can be printed with the option -lc=2 or the data can be written to the output files.

10 Constraints on the domain of the functions

The evaluation models given in the section <Equations> might be constrained in their domain. The

use of the function sqrt() for example forms such constraint since the argument of the function cannot

be negative. Any values violating any domain of the functions involved will be marked as not valid

and the values will be ignore in further calculations. In addition, constraints specified in the section

<Constraints> might restrict the domain and invalidate some values as well. As a consequence the

number of valid values per block might be smaller than the number of trails. The number of runs in the

statistical data file is always the number of valid values used to evaluate the result.

Since the statistical treatment is relying on a sufficiently large number of values per block, the ratio

between valid values and trails is monitored. The simulation will be aborted as soon as the ratio drops

below the limit defined by -nl (default 50 %). It is possible to reduce this limit, but it is advisable to

increase the block-size (-bs) at the same time to guarantee that statistical treatment has still a good

performance.

The correlation analysis can only be performed based on valid common data. Any invalid values in one

quantity will automatically discard the values for al quantities from the data set. The ratio between the

total number of values and the number of valid values is monitored. The simulation will be aborted as

soon as the ratio drops below the limit defined by -nc (default 50 %).

The analysis of correlation can be disabled with the option -ac=0 if the correlation information is not

needed. It is also possible to reduce the limit with option -nc and increase the block-size sufficiently.

26
Non-valid data will reduce the performance of the simulation and therefore should be avoided if possible.

The result will only be calculated from the valid data and all data marked as non-valid will not contribute

to the result and therefore should not be calculated at all.

10.1 Mean value and uncertainty calculation

In case some of the data in a result data block i become not valid because of constraints, the effective

block size of block i is reduced (neff(i) ≤ nb).

The total number of simulation results n in nr simulation blocks is

n =

Xnr

i=1

neff(i)

and the average effective blocksize  ̄neff is

n ̄eff =

1

nr

Xnr

i=1

neff(i)

If y[i] is the set of valid results for result data block i then the effective mean value  ̄yeff(i) of this set is

y ̄eff(i) = 1

neff(i)

nXeff (i)

j=1

y[i]j

and the effective variance s

2

eff(i) of the data in this set is

s

2

eff(i) = 1

neff(i) − 1

nXeff (i)

j=1

(y[i]j − y ̄eff(i))2.

The overall mean value of the result  ̄y is

y ̄ =

1

n

Xnr

i=1

neff(i) × y ̄eff(i)

and the variance u

2

(y) is

u

2

(y) = 1

n − 1

Xnr

i=1

nXeff (i)

j=1

(y[i]j − y ̄)

2

which is equivalent of calculating

u

2

(y) = 1

n − 1

"Xnr

i=1

(neff(i) − 1) × s

2

eff(i) +Xnr

i=1

neff(i) × ( ̄yeff(i) − y ̄)

2

#

.

The average of the effective variance  ̄s

2

eff is

s ̄

2

eff =

1

n

Xnr

i=1

neff(i) × s

2

eff(i).

The variance s

2

( ̄yeff()) of the block mean values  ̄yeff(i) for i = 1 . . . nr is

s

2

( ̄yeff()) = 1

n ̄eff × (nr − 1)

Xnr

i=1

neff(i) × ( ̄yeff(i) − y ̄)

2

and the variance of the effective variances of the sets is

s

2



s

2

eff

=

1

n ̄eff × (nr − 1)

Xnr

i=1

neff(i) × (seff(i) − s ̄eff)

2

.

27
The nominal variance s

2

nom( ̄yeff()) of the block mean values  ̄yeff(i) for i = 1 . . . nr is

s

2

nom( ̄yeff()) = n ̄eff

nb

× s

2

( ̄yeff())

The nominal variance s

2

nom(y()) is an estimate of the variance one would have got in case all blocks would

have had the nominal size nb.

11 Testing and validation

OMCE makes use of the numerical library Numpy whenever possible. So the quality of the numerical

calculations are based on this library. The library is widely used and tested and is considered reliable in

the context of OMCE testing and validation.

The examples given in GUM Supplement 1 are included as examples and the results can be compared

with the published results.

Output (.ohd) files for all examples can be found in OMCE\examples\ref\. They were generated with

a seeded random generator (-seed=1). This is used as a basic regression test during the development

and it should be used to verify the installation. The windows batch file !run_all_OMCE.bat runs all

examples and compares the result with the reference results.

12 Performance

The OMCE is optimized for an efficient calculation of models with large equation systems. During the

simulation a major part of the computational power of the system will be used. The memory consumption

is dependent on the model and is about 1 MB per input quantity and result.

The binary data file can be rather large without being of much use after the simulation. By default the

data file will be not created. Although the binary data file is written in parallel with the calculation,

the time for writing the binary data file might contribute significantly to the overall execution time if

the data calculation part is significantly faster than the data writing part. At the end the OMCE has to

wait until all data blocks are written to the binary file before the simulation can be completed.

12.1 Random number generation

The OMCE uses the Mersenne Twister algorithm from the NumPy package. The Mersenne Twister

algorithm has a large period (219937 − 1), creates random floats to full 53-bit precision, and has been

widely tested.

References

[1] Guide to the Expression of Uncertainty in Measurement, International Organization for Standardization, 1995

[2] Expression of Uncertainty in Measurement Supplement 1: Numerical Methods for the Propagation

of Distributions, JCGM Working Group of the Expression of Uncertainty in Measurement, 2006

[3] Stackless Python - extended Python for thread-based programming, open source software package,

http://www.stackless.com

[4] NumPy - Numerical Python extension, open source software package, http://numpy.scipy.org

[5] G. W ̈ubbeler, P. M. Harris, M. G. Cox, C. Elster: A two-stage procedure for determining the number

of trials in the application of a Monte Carlo method for uncertainty evaluation, Metrologia 47(3):

317-324, 2010, doi:10.1088/0026-1394/47/3/023

28
[6] N. J. Higham: Computing the nearest correlation matrix - A problem from finance. IMA J. Numer.

Anal., 22(3):329-343, 2002

[7] R. J. Hyndman, Y. Fan: Sample Quantiles in Statistical Packages, The American Statistician Vol.

50, No. 4 (1996), pp. 361-365, http://www.jstor.org/stable/2684934

[8] R. N. Kacker, J. F. Lawrence: Rectangular distribution whose width is not exactly known: isocurvilinear trapezoidal distribution, Metrologia 46: 254260, 2009, doi:10.1088/0026-1394/46/3/012

[9] matplotlib - 2D plotting library, open source software package, http://matplotlib.sourceforge.net/

[10] S. K. Mishra: Optimal solution of the nearest correlation matrix problem by minimization of the

maximum norm, 2004, http://mpra.ub.uni-muenchen.de/1783/

29
A Program exit codes

Program exit codes

Exit Code Error message

001 Error: Option <opt> is not valid!

002 Error: File not found (<filename>)!

003 Error: It is not a file (<string>)!

004 Error: The histogram bins must increase monotonically.!

005 Error: Option <option> should be a number (<value>)!

006 Error: Option out of range (<option>=<from>...<to>)!

007 Error: The attribute mcsimulations is not set correctly (<value>)!

008 Error: The binary data format <number> is not supported!

009 Error: Format <format> is not supported!

010 Error: Mathematical loop in process list!

011 Error: Input quantity <name> is not defined!

012 Error: Duplicated quantity name (<name>)!

013 Error: Error in expression: <expression>!

014 Error: Quantity <name> in Result <name> is unknown!

015 Error: Model Parameter <parameter> not supported!

016 Error: Correlation matrix is not positive semi-definite!

017 Error: Quantity name unknown in correlation entry (<name>)!

018 Error: Correlations are not supported for non-Gaussian Quantities (<name>)!

019 Error: Correlation coefficient out of range r(<name>,<name>)=<value>!

020 Error: Matrix modification exceeds limit (-e2=<limit>)!

021 Error: Diagonal elements of the correlation matrix must be one

[r(<name>,<name>)=<value>]!

022 Error: Quantity name <name> is a reserved symbol!

023 Error: Quantity <name>, parameter <param>, syntax error in <value>!

024 Error: Parameter of function <name>() is not a constant!

025 Error: Function <name> is used without parameter!

026 Error: Quantity unknown for val(<name>)!

027 Error: The uncertainty for a t-distribution with dof<3 is not defined in u(<name>)!

028 Error: Quantity unknown for u(<name>)!

029 Error: Grammar type not supported (<type-id>)!

030 Error: Function <name>() is used recursively in its definition!

031 Error: Quantity <name> is defined as parameter and as global in function <name>()!

032 Error: Global quantity <name> in function <name>() is not defined!

033 Error: Quantity <name> in function <name>() is not defined as parameter or global!

034 Error: Unknown command line parameter name <name>!

035 Error: Quantity <name>, parameter <param>, value <value> is not an integer!

036 Error: Quantity <name>, parameter <param>, parameter unknown in <value>!

037 Error: Quantity <name>, parameter <param>, %-character is not supported

(<value>)!

038 Error: Quantity <name>, parameter <param>, syntax error in <value>!

039 Error: Quantity name <name> is a predefined function!

040 Error: Quantity name <name> is a predefined symbol!

041 Error: Quantity name <name> is a reserved name!

042 Error: Quantity name <name> is the name of a user defined function!

043 Error: Parameter name <name> is a predefined function!

044 Error: Parameter name <name> is a reserved name!

045 Error: Quantity <name> in <Discrete> or <Import> element is unknown in the

bin-file <filename>!

046 Error: Index unknown!

047 Error: Result name <name> is used twice in result section!

30
Program exit codes

Exit Code Error message

001 Error: Option <opt> is not valid!

002 Error: File not found (<filename>)!

003 Error: It is not a file (<string>)!

004 Error: The histogram bins must increase monotonically.!

005 Error: Option <option> should be a number (<value>)!

006 Error: Option out of range (<option>=<from>...<to>)!

007 Error: The attribute mcsimulations is not set correctly (<value>)!

008 Error: The binary data format <number> is not supported!

009 Error: Format <format> is not supported!

010 Error: Mathematical loop in process list!

011 Error: Input quantity <name> is not defined!

012 Error: Duplicated quantity name (<name>)!

013 Error: Error in expression: <expression>!

014 Error: Quantity <name> in Result <name> is unknown!

015 Error: Model Parameter <parameter> not supported!

016 Error: Correlation matrix is not positive semi-definite!

017 Error: Quantity name unknown in correlation entry (<name>)!

018 Error: Correlations are not supported for non-Gaussian Quantities (<name>)!

019 Error: Correlation coefficient out of range r(<name>,<name>)=<value>!

020 Error: Matrix modification exceeds limit (-e2=<limit>)!

021 Error: Diagonal elements of the correlation matrix must be one

[r(<name>,<name>)=<value>]!

022 Error: Quantity name <name> is a reserved symbol!

023 Error: Quantity <name>, parameter <param>, syntax error in <value>!

024 Error: Parameter of function <name>() is not a constant!

025 Error: Function <name> is used without parameter!

026 Error: Quantity unknown for val(<name>)!

027 Error: The uncertainty for a t-distribution with dof<3 is not defined in u(<name>)!

028 Error: Quantity unknown for u(<name>)!

029 Error: Grammar type not supported (<type-id>)!

030 Error: Function <name>() is used recursively in its definition!

031 Error: Quantity <name> is defined as parameter and as global in function <name>()!

032 Error: Global quantity <name> in function <name>() is not defined!

033 Error: Quantity <name> in function <name>() is not defined as parameter or global!

034 Error: Unknown command line parameter name <name>!

035 Error: Quantity <name>, parameter <param>, value <value> is not an integer!

036 Error: Quantity <name>, parameter <param>, parameter unknown in <value>!

037 Error: Quantity <name>, parameter <param>, %-character is not supported

(<value>)!

038 Error: Quantity <name>, parameter <param>, syntax error in <value>!

039 Error: Quantity name <name> is a predefined function!

040 Error: Quantity name <name> is a predefined symbol!

041 Error: Quantity name <name> is a reserved name!

042 Error: Quantity name <name> is the name of a user defined function!

043 Error: Parameter name <name> is a predefined function!

044 Error: Parameter name <name> is a reserved name!

045 Error: Quantity <name> in <Discrete> or <Import> element is unknown in the

bin-file <filename>!

046 Error: Index unknown!

047 Error: Result name <name> is used twice in result section!

31
Program exit codes

Exit Code Error message

048 Error: No result defined in source file!

049 Error: Unknown symbol <name> in parameter value <value>!

050 Error: Unknown symbol in parameter value <value>!

051 Error: Double assignment for r(<name>,<name>): <value> and <value>!

052 Error: A near correlation matrix could not be found in <runs> runs!

053 Error: No. of Runs in bin-file is zero!

054 Error: Bin-file format <format-id> not supported!

055 Error: Bin-file <filename> is corrupted!

056 Error: Less than <value> % of the simulated results are valid!

057 Error: The constrain for <name>: <expression> invalidates all data!

058 Error: Output File Definition not found (<filename>)!

059 Error: Constrain for <name>: <expression> cannot be evaluated!

060 Error: Unknown command line parameter argument <argument>!

061 Error: Parameter <param> in section <Simulation> is unknown!

062 Error: Loop increment (Step) must be non-zero!

063 Error: Parameter <param> in section <Loop> is unknown!

064 Error: Error in <element> name: <text>!

065 Error: quantile estimation method is unknown for -pq=<value>!

066 Error: The sub-block size -sbs=<value> must be a multiple of 10!

067 Error: The block size -bs=<value> must be a multiple of the sub-block size

-sbs=<value>!

068 Error: Result data entry ’<name>’ unknown in ofd-file!

069 Error: Result data entry ’<name>’ unknown in vfd-file!

254 Error: Internal, Matrix structure does not match!

255 Error: <error message>!

32