Package: tbp
Tensor Belief PropagationTensor Belief Propagation(TBP) is an experimental algorithm for
approximate inference in discrete graphical models [1]. It takes a factor graph in.uaior.fgformat and outputs approximate marginals for
each variable.[1]Wrigley, Andrew, Wee Sun Lee, and Nan Ye. "Tensor Belief Propagation." International Conference on Machine Learning. 2017.RequirementsLinux or OSXPython 3.6+InstallationInstall libDAI prerequisites:# Linux
$ sudo apt-get install g++ make doxygen graphviz libboost-dev libboost-graph-dev libboost-program-options-dev libboost-test-dev libgmp-dev cimg-dev

# OSX
$ brew install boost gmp doxygen graphvizInstall tbp with the Python package managerpip:$pipinstalltbp
...
Successfullyinstalledtbp-X.X.XThis will take a while as libDAI must be compiled.UsageTBP takes a factor graph in either.fgor.uaiformat as input, and
outputs the approximate marginal distribution of each variable in.MAR format.
This involves two steps — first, all potential functions in the graph must be decomposed
into sums of rank-1 tensors yielding a decomposed factor graph (.dfg). Then, the
message passing procedure must be run on the decomposed graph to give approximate marginals.Command lineAfter installation, the command line utilitytbpis available to do either or both of these steps. For usage
instructions, runtbp --help.ExamplesDecompose the factor graphising_8x8.fgand find marginals:$tbptests/ising_8x8.fg6420.5949610.4050392...0.6085730.391427Decompose input potentials into 3 rank-1 components and save the resulting decomposed graph (but don't find marginals):$tbptests/ising_8x8.fg-r3-otests/ising_8x8.dfg--verbosity2Readinggraphtests/ising_8x8.fg(libDAIformat)...
Decomposinginputgraph(r=3termsperfactor)...
Successfullysaveddecomposedgraphtotests/ising_8x8.dfg.Decompose the factor graphPromedus_11.uaiafter applying some evidence, find marginals using TBP with sample size of 1000, and save the output
toout.MAR:$tbptests/uai/MAR_prob/Promedus_11.uai-etests/uai/MAR_prob/Promedus_11.uai.evid-k1000-oout.MAR--verbosity2Readinggraphtests/uai/MAR_prob/Promedus_11.uai(UAIformat)...
Applyingevidencefiletests/uai/MAR_prob/Promedus_11.uai.evid...
Decomposinginputgraph(r=4termsperfactor)...
RunningTBPwithsamplesizeK=1000...
Successfullysavedmarginalstoout.MAR.Python libraryThetbppackage can also be used directly from Python, for example:importtbp# Load a factor graph in .uai formatg=tbp.load_uai_graph('tests/uai/MAR_prob/linkage_11.uai')# Apply evidence (fixed variable assignments)g.apply_evidence('tests/uai/MAR_prob/linkage_11.uai.evid')# Decompose each factor into a weighted sum of 4 rank-1 tensorsdg=g.decompose(r=4)# Run TBP to find marginals with sample size of 10000mar=dg.tbp_marg(K=10000)TroubleshootingInstalling into a virtual environmentIfpip installhas issues with dependencies or version conflicts, you can install the necessary
packages into a virtual environment (a project-specific folder rather than globally on your system):$sudopip3installvirtualenv# pip or pip3, depending on your system$virtualenv-ppython3venv# create venv folder to store packages$sourcevenv/bin/activate# activate virtual environment$pipinstalltbp# install tbp into venv folderNow when you invoketbp, the local versions will be used.Building from GitHub cloneTo use thetbpPython package from source without installation viapip install, libDAI must first be compiled:$gitclonegit@github.com:akxlr/tbp.git
$cdtbp/libdai
$cpMakefile.<platform>Makefile.conf# Choose <platform> according to your platform$make
...
libDAIbuiltsuccessfully!This produces a utilitylibdai/utils/dfgmargwhich is symlinked fromtbp/dfgmargand used during inference. SeelibDAI READMEfor full installation instructions.Using MATLAB for the decompositionThe decomposition of potential functions uses the non-negative CP decomposition algorithm in the Tensorly tensor
library. As an alternative to TensorLy, theMATLAB Tensor Toolboxcan be used
(this was what we used in [1]). To use this instead of Tensorly:Install MATLABInstall theMATLAB Python APIInstall theMATLAB Tensor ToolboxYou can now replacemethod='tensorly'withmethod='matlab'when calling decomposition functions incore.py.File formats.dfg (decomposed factor graph)We created the.dfgfile format based onlibDAI's .fg file formatto represent decomposed factor graphs. A decomposed factor graph is a
factor graph with all factors represented as sums of rank-1 tensors rather than multidimensional tables.The first line of a.dfgfile contains the number of factors in the graph, followed by a blank line. Then, factors
are described in turn by blocks separated by a single blank line. Each factor block is structured as follows:1. n_terms
 2. <weights>
 3. n_variables
 4. <variable indices>
 5. <variable cardinalities>
 6. n_nonzero_1
 7. 1 0.5
 8. 3 0.1
 9. 4 0.1
10. ...
11. n_nonzero_2
12. 1 0.5
13. 3 0.1
14. 4 0.3
15. ...In the header section of the factor block (lines 1-5),n_termsis the number of terms in the decomposition and<weights>,<variable indices>and<variable cardinalities>are self-explanatory space-separated lists of lengthn_terms,n_variablesandn_variablesrespectively.The remainder of the factor block (line 6 onwards) describes
a series ofn_variables2D matrices that together describe then_termsrank-1 tensors.
Each matrix corresponds to a single variable and has shape(cardinality, n_terms), wherecardinalityis
the cardinality of the variable andn_termsis the number of rank-1 terms in the decomposition (constant
for all variables). Each matrix begins with the
number of nonzero values in the matrix, followed by a series ofindex valuepairs describing the nonzero
entries of the matrix in column-major order. SeelibDAI's documentationfor examples of how to
reshape these lists back into matrices.Theith rank-1 tensor is constructed by taking the outer product of theith columns of
all matrices. The complete factor is then reconstructed by adding up these rank-1 tensors and weighting
according to<weights>.Other file formatsOther file formats used in this project are:.fg(libDAI factor graph):https://staff.fnwi.uva.nl/j.m.mooij/libDAI/doc/fileformats.html.uai(UAI factor graph):http://www.hlt.utdallas.edu/~vgogate/uai14-competition/modelformat.html.MAR(marginals):http://www.hlt.utdallas.edu/~vgogate/uai14-competition/resformat.html.evid(evidence):http://www.hlt.utdallas.edu/~vgogate/uai14-competition/evidformat.htmlTo doICML experiments - finish cleaning code used for experiments (seeicml17.pyfor partial code)Rewrite code that loads .uai files to handle all problems (currently breaks on some)Deal with Z <= 0 warning from C++ codeClean up C++ code and compiler warningsAdd more testsFeedbackBug reports, suggestions and comments are welcome. Please emailandrew@wrigley.ioor use the issue tracker.LicenseSeeLICENSE.txt(MIT).AcknowledgmentslibDAI(included inlibdaifolder with modifications; libDAI's junction tree implementation is used for the message passing step)Eigen(version 3.3.4 included inlibdai/vendor/includefolder)TensorLy(used to perform initial non-negative CP decomposition of potential functions)MATLAB Tensor Toolbox