  Recently, inspired by quantum annealing, many solvers specialized for
unconstrained binary quadratic programming problems have been developed. For
further improvement and application of these solvers, it is important to
clarify the differences in their performance for various types of problems. In
this study, the performance of four quadratic unconstrained binary optimization
problem solvers, namely D-Wave Hybrid Solver Service (HSS), Toshiba Simulated
Bifurcation Machine (SBM), Fujitsu DigitalAnnealer (DA), and simulated
annealing on a personal computer, was benchmarked. The problems used for
benchmarking were instances of real problems in MQLib, instances of the
SAT-UNSAT phase transition point of random not-all-equal 3-SAT(NAE 3-SAT), and
the Ising spin glass Sherrington-Kirkpatrick (SK) model. Concerning MQLib
instances, the HSS performance ranked first; for NAE 3-SAT, DA performance
ranked first; and regarding the SK model, SBM performance ranked first. These
results may help understand the strengths and weaknesses of these solvers.
