Package: slgbuilder
Python package for building and solving Sparse Layered GraphsThis package allows building and solving image segmentation problems such asMarkov Random Fields(MRF) andSparse Layered Graphs(SLG) usings-tgraph cuts. The package itself is written purely in Python and contains logic for building graphs for both single- and multi-label problems. To solve the optimization problem it relies on a min-cut/max-flow algorithm (seeSolvers).InstallationInstall default package usingpip install slgbuilderor clone the repository. SeeDependenciesfor more.What is it for?The package is primarily targeted multi-label/multi-object image segmentation problems. Common uses include:Surface detectionConstrained multi-surface detectionObject segmentationInteracting multi-object segmentationCVPR 2020 teaser videoExamplesExample code and notebooksExperimental notebooks (advanced 2D and 3D segmentation) [CVPR2020]Experimental notebooks (advanced 3D segmentation with parallel QPBO) [ICCV 2021]Grid-graph vs. ordered multi-column graphThe package support both the common grid-graph structure, as used byDelong and Boykovand the ordered multi-column structure, popularized byLi et al.. Although representing image data in the grid structure may seem like the obvious choice, there are several advantages (e.g. smaller graph and "better" constraints) when using the ordered multi-column structure for segmentation if possible. However, doing so requires resample the data, which usually requires knowledge about the location of each object in the image.SolversThe package currently supports eight different solvers. In simple cases the default BK Maxflow and QPBO solvers should do fine. For submodular problems, all solver should give find a globally optimal solution. Only the QPBO solvers support nonsubmodular problems, and for such problems a complete solution cannot be guaranteed. For large or time-critical tasks it may be favorable to use a non-default solver, e.g., HPF or PQPBO. We are currently working on a comparative study of min-cut/max-flow algorithms, which should provide better insights into which algorithms perform the best on different computer vision tasks. If you're interested, check out thisrepository.BK MaxflowFor submodular problems (e.g. segmentation without exclusion constraints), the default solver, used by theMaxflowBuilderis anupdated versionthe Boykov-Kolmogorov Maxflow algorithm, accessed through thethinmaxflowpackage.MaxflowBuilderis synonymous withBKBuilder.QPBOFor nonsubmodular problems (e.g. segmentation with exclusion constraints) the solver used by theQPBOBuilderisthis versionof the QPBO algorithm, accessed through thethinqpbopackage.HPFTheHPFBuilderis an alternative to the standardMaxflowBuilder. It relies on the HPF algorithm, which is often superior in performance compared to BK Maxflow.HPFBuilderdepends on thethinhpfpackage. It supportsint32andfloat32capacities/energies, however,int32is recommended.OR ToolsAn alternative to the BK Maxflow solver is theGoogle Maxflowimplementation, which is a push-relabel algorithm. This can be done using theORBuilderclass. Apart from performance, the difference between the Google and BK Maxflow algorithms is that the Google implementation doesn't support floating type capacities. IfMaxflowBuilderis slow when solving, try using theORBuilderinstead.MBKA slightly optimized modern reimplementation of the BK Maxflow algorithm is available using theMBKBuilder. Depends on theshrdrpackage. Using this builder should reduce memory usage and increase performance slightly compared to theMaxflowBuilder/BKBuilder.PBKA parallel version of theMBKimplementation found in theshrdrpackage. Can significantly reduce solve for large problems on multicore systems. Jupyter notebooks with examples using the similarPQPBOBuildercan be found in thesupplementary materialof thisarticle.MQPBOA slightly optimized modern reimplementation of the QPBO algorithm is available using theQPBOBuilder. Depends on theshrdrpackage. Using this builder should reduce memory usage and increase performance slightly compared to theQPBOBuilder.PQPBOA parallel version of theMQPBOimplementation found in theshrdrpackage. Can significantly reduce solve for large problems on multicore systems. Jupyter notebooks with examples of use can be found in thesupplementary materialof thisarticle.DependenciesTo solve thes-tgraph-cut optimization problems the package relies on one of the following packages:thinmaxflow(installed by default) - Package for computing min-cut/max-flow of ans-tgraph using the Boykov-Kolmogorov (BK) Maxflow algorithm.thinqpbo(installed by default) - Package for solving submodular and nonsubmodular optimization problems using the Quadratic Pseudo-Boolean Optimization (QPBO) algorithm implementation by Kolmogorov. Solver based on the BK Maxflow.thinhpf- Package for computing min-cut/max-flow of ans-tgraph using the Hochbaum Pseudoflow (HPF) algorithm.ortools- Package for computing min-cut/max-flow of ans-tgraph using the Google OR Tools min-cut/max-flow implementation.shrdr- Package for computing min-cut/max-flow of ans-tgraph optimized serial or parallel implementations of BK Maxflow and QPBO algorithms by Jensen and Jeppesen.See links for further details and references.Install with extra dependenciesInstall with HPF solver.pip install slgbuilder[thinhpf]Install with Google OR Tools solver.pip install slgbuilder[ortools]Install with all extras.pip install slgbuilder[all]Theshrdrpackage is not yet available on PyPI. Get ithere!Related repositoriesshrdrPython package (ICCV 2021)thinqpboPython packagethinmaxflowPython packagethinhpfPython packageC++ implementationsof min-cut/max-flow algorithmsContributionsContributions are welcome, just create anissueor aPR.LicenseMIT License (see LICENSE file).ReferenceIf you use this any of this for academic work, please consider citing our work:N. Jeppesen, A. N. Christensen, V. A. Dahl and A. B. Dahl, "Sparse Layered Graphs for Multi-Object Segmentation," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 12774-12782, doi: 10.1109/CVPR42600.2020.01279.[paper]
[supp]
[CVF]
[IEEE]N. Jeppesen, P. M. Jensen, A. N. Christensen, A. B. Dahl and V. A. Dahl, "Faster Multi-Object Segmentation using Parallel Quadratic Pseudo-Boolean Optimization," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 6240-6249, doi: 10.1109/ICCV48922.2021.00620.[paper]
[supp]
[CVF]
[IEEE]BibTeX@INPROCEEDINGS{9156301,author={Jeppesen, Niels and Christensen, Anders N. and Dahl, Vedrana A. and Dahl, Anders B.},booktitle={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},title={Sparse Layered Graphs for Multi-Object Segmentation},year={2020},volume={},number={},pages={12774-12782},doi={10.1109/CVPR42600.2020.01279}}@INPROCEEDINGS{9710633,author={Jeppesen, Niels and Jensen, Patrick M. and Christensen, Anders N. and Dahl, Anders B. and Dahl, Vedrana A.},booktitle={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},title={Faster Multi-Object Segmentation using Parallel Quadratic Pseudo-Boolean Optimization},year={2021},volume={},number={},pages={6240-6249},doi={10.1109/ICCV48922.2021.00620}}