Package: booltest
BoolTestBoolean PRNG tester - analysing statistical properties of PRNGs.Randomness tester based on our paper published atSecrypt
2017How does it work?BoolTest generates a set of boolean functions, computes the expected
result distribution when evaluated on truly random data and compares
this to the evaluation on the data being tested.Pip installationBoolTest is available viapip:pip3 install booltestLocal installationFrom the local dir:pip3 install --upgrade --find-links=. .The engineBoolTest does the heavy lifting with the native python extensionbitarray_ph4Bitarray operations are performed effectively using fast operations
implemented in C.ExperimentsFirst launchThe following commands generate two different files, random and
zero-filled. Both are tested, the difference between files should be
evident.dd if=/dev/urandom of=random-file.bin bs=1024 count=$((1024*10))
dd if=/dev/zero of=zero-file.bin bs=1024 count=$((1024*10))

booltest --degree 2 --block 256 --combine-deg 2 --top 128 --tv $((1024*1024*10)) --rounds 0 random-file.bin
booltest --degree 2 --block 256 --combine-deg 2 --top 128 --tv $((1024*1024*10)) --rounds 0 zero-file.binThe BoolTest with the given parameters constructs all polynomials of
degree 2 from monomials {x_0, …, x_{255}}Evaluates all polynomials on the input data (windowing), computes
zscore from the computed vs reference dataSelects 128 best polynomials (abs(zscore))Phase 2: Take the best 128 polynomials and combine them by XOR to the--combine-degnumber of terms.The resulting polynomials are evaluated again and results printed
out.Common testing parametersWe usually use BoolTest with the following testing parameters:--top 128 --no-comb-and --only-top-comb --only-top-deg --no-term-map --topterm-heap --topterm-heap-k 256The same can be done with the--default-paramsOutput and p-valuesBoolTest returns zscores of the best distinguishers.In order to obtain a p-value from the Z-score you need to compute a
reference experiments, i.e., compute N BoolTest experiments on a random
data and observe the z-score distribution. Z-score is data-size
invariant but it depends on the BoolTest parameters(n, deg, k).The most straightforward evaluation is to check whether z-score obtained
from the real experiment has been observed in the reference runs. If
not, we can conclude the BoolTest rejects the null hypothesis with
pvalue1/N.To obtain lower alpha you need to perform more reference experiments, to
obtain higher alpha integrate the z-score histogram from tails to mean
to obtain desired percentage of the area under z-score histogram.The filepval_db.jsoncontains reference z-score -> pvalue mapping for N=20 000 reference
runs.BoolTest now supports adding pvalue database as a parameter--ref-dbpath-to-db.jsonIf the database is not given, BoolTest
tries to locate the defaultpval_db.jsonin the BoolTest
installation directory and on the path.If the database is found, BoolTest shows also OK/reject result for the
best distinguisher, given the reference database contains the data for
given(n, deg, k)parameters.Example:- best poly zscore  -5.37867, expp: 0.0625, exp:   10240, obs:    9713, diff:  5.1464844 %, poly: [[64, 245, 207, 242]]
2019-12-13 20:25:17 PHX booltest.booltest_main[51363] INFO Ref samples: 40005, min-zscrore: 4.838657, max-zscore: 7.835336, best observed: 5.3786712268614005, rejected: False, alpha: 2.4996875390576178e-05Halving methodWe have implemented another evaluation method called halving, enabled
with commandline option--halving. It needs twice more data than the
default method, because of how it works:The input file is divided to two halvesBoolTest runs as before on the first half, picks the best
distinguisherBoolTest runs the best distinguisher on the second halfAs the best distinguisher selected to the second half “never seen”
the second half and there is only one polynomial the p-value can be
directly computed due to independence.The best distinguisher results are essentially following Binomial
distribution:Bi(number_of_blocks, probability_of_dist_eval_to_1).To compute the p-value we run the Binomial test:scipy.stats.binom_test(observed_ones, n=ntrials, p=dist_probab,alternative='two-sided')This method eliminates a need to have apval_db.jsondatabase
computed with the reference data for given parameters. The benefit is
the halving method gives directly a p-value, without a need to run
reference computations. The downside is the method needs twice more data
and can give weaker results than the original BoolTest evaluation.Example:- zscore[idx00]: -0.40825, observed: 00010200, expected: 00010240   idx:      0, poly: [[64, 245, 207, 242]]
2019-12-13 20:25:17 PHX booltest.booltest_main[51363] INFO Binomial dist, two-sided pval: 0.6868421673496484, pst: 0.0625, ntrials: 163840, succ: 10200Java randomAnalyze output of thejava.util.Random, use only polynomials in the
specified file. Analyze 100 MB of data:booltest --degree 2 --block 512 --combine-deg 2 --top 128 --tv $((1024*1024*100)) --rounds 0 \
  --poly-file data/polynomials/polynomials-randjava_seed0.txt \
  randjava_seed0.binInput dataBoolTest can test:Pregenerated data filesUse theCryptoStreamsconfiguration files to generate input data on the fly, usingCryptoStreams(library contains plenty round-reduced cryptographic primitives)Cluster computation (Metacentrum)Map / Reduce.Thebooltest/testjobs.pycreates job filesThebooltest/testjobsproc.pyprocesses result filesBoolTest job is configured via JSON file. Result of a computation is
JSON file.Thebooltest/testjobsbase.pyperforms job aggregation, i.e., more
BoolTest runs in one shell script as job planning overhead is
non-negligible. Useful for fast running jobs.Works with PBSPro, qsub queueing algorithmExample - generate jobs fromCryptoStreamsconfigurationspython../booltest/booltest/testjobs.py\--data-dir$RESDIR--job-dir$JOBDIR--result-dir=$RESDIR\--top128--matrix-size110100--matrix-block128256384512--matrix-deg123--matrix-comb-deg123\--no-comb-and--only-top-comb--only-top-deg--no-term-map--topterm-heap--topterm-heap-k256\--skip-finished--no-functions--ignore-existing\--generator-folder../bool-cfggens/--generator-path../bool-cfggens/crypto-streams_v2.3-13-gff877beFor allCryptoStreamsconfiguration files located under../bool-cfggens/it generates
BoolTest tests with parameters:input_size x block_size x deg x comb-deg
{1, 10, 100} x {128, 256, 384, 512} x {1, 2, 3} x {1, 2, 3}Command generates PBSPro shell scripts to$JOBDIR, results are
placed into$RESDIR.For one configuration file which is typically round reduced crypto
primitive it performs3*4*3*3 = 108 tests.When using CryptoStreams config files the config files have to
specify the longest tested input, in this case, 100 MB.Example - analyze input filespython../booltest/booltest/testjobs.py\--test-files../card_prng/*.bin\--data-dir$RESDIR--job-dir$JOBDIR--result-dir=$RESDIR\--top128--matrix-size110100--matrix-block128256384512--matrix-deg123--matrix-comb-deg123\--no-comb-and--only-top-comb--only-top-deg--no-term-map--topterm-heap--topterm-heap-k256\--skip-finished--no-functions--ignore-existingThis example generates job to analyze input files (e.g., smartcard
generated randomness)Example - reference statisticspython../booltest/booltest/testjobs.py\--data-dir$RESDIR--job-dir$JOBDIR--result-dir=$RESDIR\--generator-path--generator-path../bool-cfggens/crypto-streams_v2.3-13-gff877be\--top128--matrix-size10--matrix-block128256384512--matrix-deg123--matrix-comb-deg123\--no-comb-and--only-top-comb--only-top-deg--no-term-map--topterm-heap--topterm-heap-k256\--skip-finished--ref-only--test-rand-runs1000--skip-existing--counters-only--no-sac--no-rpcs--no-reinitComputes 1000 independent AES round 10 runs, each with different seed in
the counter mode. Tests BoolTest in various configurations.Reference dataReference data extracted from thepval_db.jsonfor standard
parameters are displayed in the table below. The z-score distribution
does not change with data length for random data.The table shows minimal and maximal observed z-score with given number
of samples for given BoolTest parameters run on a random data. If your
test gives z-score outside of the interval the null hypothesis is
rejected with the given alpha.blockdegcomb-degsamplesalphaminmaxmeanstddev128111000001.0e-051.6924516.1677062.8272120.392073128121000001.0e-053.2052856.2031243.9684380.296394128131000001.0e-054.0755436.7900434.7869120.252671128211000001.0e-052.9357936.4470603.8886120.336659128221000001.0e-054.3581327.3367125.2848950.303586128231000001.0e-054.8739837.7966485.7703700.295979128311000001.0e-053.8668046.8233214.7086140.290063128321000001.0e-055.6980858.3769276.5609980.265912128331000001.0e-056.7498719.5708487.6169650.282312256111000001.0e-051.9999395.3021643.0444720.368626256121000001.0e-053.1913166.3039233.9697170.297037256131000001.0e-054.0964777.0266204.7866300.252239256211000001.0e-053.3049466.6016294.2212270.310189256221000001.0e-054.7758417.6099385.6591430.293182256231000001.0e-055.1881208.3055006.0517430.289642256311000001.0e-054.3893937.0019345.1300260.263766256321000001.0e-056.4026189.0032597.1397770.245306256331000001.0e-057.4086629.8336988.2477230.263262384111000001.0e-052.1755565.3260583.1656710.359287384121000001.0e-053.2140396.1103983.9689370.296144384131000001.0e-054.1385957.0174274.7871640.252258384211000001.0e-053.5632576.4067804.4050900.297651384221000001.0e-054.9729457.6067785.8590610.288056384231000001.0e-055.3557418.0657696.2019060.284984384311000001.0e-054.6215117.3177375.3634790.251736384321000001.0e-056.7298018.9573347.4545640.235683384331000001.0e-057.66672210.2252078.5854600.254527512111000001.0e-052.2920195.5807883.2500470.351051512121000001.0e-053.1774575.9805003.9688290.295422512131000001.0e-054.1438496.8204004.7860890.251366512211000001.0e-053.7361986.5054834.5304100.289203512221000001.0e-055.1563397.8396745.9972710.286677512231000001.0e-055.4598048.3245286.3045740.283540512311000001.0e-054.8401987.1962325.5208810.242361512321000001.0e-056.9315619.3243137.6675480.228464512331000001.0e-057.88680210.3129778.8156870.249666Reference z-score distributionsInstallationScipy installation with pippip install pyopenssl
pip install pycrypto
pip install git+https://github.com/scipy/scipy.git
pip install --upgrade --find-links=. .Virtual environmentIt is usually recommended to create a new python virtual environment for
the project:virtualenv ~/pyenv
source ~/pyenv/bin/activate
pip install --upgrade pip
pip install --upgrade --find-links=. .DeploymentsFor various deployment information seeDeployments.md.Python 3.5+BoolTest does not work with lower Python version. Usepyenvto
install a new Python version. It internally downloads Python sources and
installs it to~/.pyenv.git clone https://github.com/pyenv/pyenv.git ~/.pyenv
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
exec $SHELL
pyenv install 3.7.1
pyenv local 3.7.1The recommended version is Python 3.5+SECRYPT 2017For SECRYPT 2017 related experiments check outSECRYPT2017.md