Package: maraboupy-NeuralNetworkVerification
MarabouDeep neural networks are revolutionizing the way complex systems are designed.
Instead of spending long hours hand-crafting complex software, many engineers
now opt to use deep neural networks (DNNs) - machine learning models, created by
training algorithms that generalize from a finite set of examples to previously
unseen inputs. Consequently, there is a pressing need for tools and techniques
for network analysis and certification. To help address that need, we present
Marabou, a framework for verifying deep neural networks.Marabou is an SMT-based tool that can answer queries about a network’s
properties by transforming these queries into constraint satisfaction problems.
It can accommodate networks with different activation functions and topologies,
and it performs high-level reasoning on the network that can curtail the search
space and improve performance. It also supports parallel execution to further
enhance scalability. Marabou accepts multiple input formats, including protocol
buffer files generated by the popular TensorFlow framework for neural networks.A DNN verification query consists of two parts: (i) a neural network, and (ii) a
property to be checked; and its result is either a formal guarantee that the
network satisfies the property, or a concrete input for which the property is
violated (a counter-example). There are several types of verification queries
that Marabou can answer:Reachability queries: if inputs is in a given range is the output
guaranteed to be in some, typically safe, range.Robustness queries: test whether there exist adversarial points around a given
input point that change the output of the network.Marabou supports fully connected feed-forward and convolutional NNs with
piece-wise linear activation functions, in the .nnet and TensorFlow formats.
Properties can be specified using inequalites over input and output variables or
via Python interface.For more details about the features of Marabou check out thetool
paperand theslides.For more information about the input formats please check thewiki.DownloadThe latest version of Marabou is available on [https://github.com/NeuralNetworkVerification/Marabou].Build and DependenciesMarabou depends on the Boost library,
which is automatically downloaded and built when you run make. Library CXXTEST
comes included in the repository.The marabou build process uses CMake version 3.12 (or later).
You can get CMakehere.Marabou can be built for Linux, MacOS, or Windows machines.Build Instructions for Linux or MacOSTo build marabou using CMake run:cd path/to/marabou/repo/folder
mkdir build 
cd build
cmake ..For configuring to build a static Marabou binary, use the following flagcmake .. -DBUILD_STATIC_MARABOU=ONTo build, run the following:cmake --build .To enable multiprocess build change the last command to:cmake --build . -j PROC_NUMTo compile in debug mode (default is release)cmake .. -DCMAKE_BUILD_TYPE=Debug
cmake --build .The compiled binary will be in thebuilddirectory, namedMarabouTo run tests we usectest.
The tests have labels according to level (unit/system/regress0/regress1...), and the code they are testing (engine/common etc...).For example to run all unit tests execute in the build directory:ctest -L unitOn every build we run the unit tests, and on every pull request we run unit,
system, regress0 and regress1.Another option to build and run all of the tests is:cd path/to/marabou/repo/folder
mkdir build 
cd build
cmake ..
make check -j PROC_NUMBuild Instructions for Windows using Visual StudioFirst, install Visual Studio 2017 or later and select the "Desktop development with C++" workload.
Ensure that CMake is installed and added to your PATH.Open a command prompt and run:cd path\to\marabou\repo\folder
mkdir build 
cd build
cmake .. -G"Visual Studio 15 2017 Win64"
cmake --build . --config ReleaseThis process builds Marabou using the generator "Visual Studio 15 2017 Win64".
For 32-bit machines, omit Win64. Other generators and older versions of Visual Studio can likely be used as well,
but only "Visual Studio 15 2017 Win64" has been tested.The Marabou executable file will be written to the build/Release folder. To build in
Debug mode, simply run "cmake --build . --config Debug", and the executables will be
written to build/Debug.Python APIIt may be useful to set up a Python virtual environment, seeherefor more information.The python interface was tested only on versions >3.5 and >2.7. The build process prefers python3 but will work if there is only python 2.7 available. (To control the default change the DEFAULT_PYTHON_VERSION variable).The Python interface requirespybind11(which is automatically downloaded).
By default Marabou builds also the python API, the BUILD_PYTHON variable
controls that.
This process will produce the binary file and the shared library for the Python
API. The shared library will be in the maraboupy folder for Linux and MacOS.
On Windows, the shared library is written to a Release subfolder in maraboupy,
so you will need to move the Release/*pyd file to the maraboupy folder:cd path\to\marabou\repo\folder\maraboupy
move Release\*pyd .Export maraboupy folder to Python and Jupyter paths:PYTHONPATH=PYTHONPATH:/path/to/marabou/folder
JUPYTER_PATH=JUPYTER_PATH:/path/to/marabou/folderand Marabou is ready to be used from a Python or a Jupyter script. On Windows,
edit your environmental variables so PYTHONPATH includes the marabou folder.TroubleshootingOn Windows - Make sure the detected python ("Found PythonInterp: ....") is a windows python and not cygwin or something like that (if it is cygwin, use -DPYTHON_EXECUTABLE flag to override the default python, or manuialy download the linux pybind and locate it in the tools directory)32bit Python - By default we install a 64bit Marabou and consequently a 64bit
python interface, the maraboupy/build_python_x86.sh file builds a 32bit
version.Getting StartedTo run Marabou from Command lineAfter building Marabou the binary is located atbuild/Marabou(orbuild\Release\Marabou.exefor Windows). The
repository contains sample networks and properties in theresourcesfolder.
For more information seeresources/README.md.To run Marabou, execute from the repo directory, for example:./build/Marabou resources/nnet/acasxu/ACASXU_experimental_v2a_2_7.nnet resources/properties/acas_property_3.txton Linux or MacOS, orbuild\Release\Marabou.exe resources\nnet\acasxu\ACASXU_experimental_v2a_2_7.nnet resources\properties\acas_property_3.txton Windows.Using Python interfacePlease see ourdocumentationfor the python interface, which contains examples, API documentation, and a developer's guide.Using the Split and Conquer (SNC) modeIn the SNC mode, activated by--sncMarabou decomposes the problem into2^n0sub-problems, specified by--initial-divides=n0. Each sub-problem will be
solved with initial timeout oft0, supplied by--initial-timeout=t0. Every
sub-problem that times out will be divided into2^nadditional sub-problems,--num-online-divides=n, and the timeout is multiplied by a factor off,--timeout-factor=f. Number of parallel threadstis specified by--num-workers=t.So to solve a problem in SNC mode with 4 initial splits and initial timeout of 5s, 4 splits on each timeout and a timeout factor of 1.5:build/Marabou resources/nnet/acasxu/ACASXU_experimental_v2a_2_7.nnet resources/properties/acas_property_3.txt --snc --initial-divides=4 --initial-timeout=5 --num-online-divides=4 --timeout-factor=1.5 --num-workers=4Use LP RelaxationMarabou has an option to use LP relaxation for bound tightening.
For now we use Gurobi as an LP solver. Gurobi requires a license (a free
academic license is available), after getting one the software can be downloadedhereandhereare
installation steps, there is acompatibility
issuethat should be addressed.
A quick installation reference:export INSTALL_DIR=/opt
sudo tar xvfz gurobi9.1.1_linux64.tar.gz -C $INSTALL_DIR
cd $INSTALL_DIR/gurobi911/linux64/src/build
sudo make
sudo cp libgurobi_c++.a ../../lib/Next it is recommended to add the following to the .bashrc (but not necessary)export GUROBI_HOME="/opt/gurobi911/linux64"
export PATH="${PATH}:${GUROBI_HOME}/bin"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${GUROBI_HOME}/lib"After installing Gurobi compile marabou as follows:cmake .. -DENABLE_GUROBI=ON
cmake --build .If you did not set the GUROBI_HOME environment variable, then use the following:cmake .. -DENABLE_GUROBI=ON -DGUROBI_DIR=<PATH_TO_GUROBI>TestsWe have three types of tests:unit tests - test specific small components, the tests are located alongside the code in atestsfolder (for example:src/engine/tests), to add a new set of tests, add a file namedTest_FILENAME(whereFILENAMEis what you want to test), and add it to the CMakeLists.txt file (for example src/engine/CMakeLists.txt)system tests - test an end to end use case but still have access to internal functionality. Those tests are located insrc/system_tests. To add new set of tests create a file namedTest_FILENAME, and add it also tosrc/system_tests/CMakeLists.txt.regression tests - test end to end functionality thorugh the API, each test is defined by:network_file - description of the "neural network" supporting nnet and mps formats (using the extension to decdie on the format)property_file - optional, constraint on the input and output variablesexpected_result - sat/unsatThe tests are divided into 5 levels to allow variability in running time, to add a new regression tests:add the description of the network and property to theresourcessub-folderadd the test to:regress/regressLEVEL/CMakeLists.txt(where LEVEL is within 0-5)
In each build we run unit_tests and system_tests, on pull request we run regression 0 & 1, in the future we will run other levels of regression weekly / monthly.AcknowledgmentsThe Marabou project is partially supported by grants from the Binational Science
Foundation (2017662), the Defense Advanced Research Projects Agency
(FA8750-18-C-0099), the Semiconductor Research Corporation (2019-AU-2898), the
Federal Aviation Administration, Ford Motor Company, Intel Corporation, the
Israel Science Foundation (683/18), the National Science Foundation (1814369,
DGE-1656518), Siemens Corporation, General Electric, and the Stanford CURIS program.Marabou is used in a number of flagship projects atStanford's AISafety
center.PeopleGuy KatzClark BarrettAleksandar ZeljicAhmed IrfanHaoze WuChristopher LazarusKyle JulianChelsea SidraneParth ShahShantanu ThakoorRachel LimDerek A. HuangDuligur IbelingElazar CohenBen GoldbergerOmri Cohen