  Empirical studies are fundamental in assessing the effectiveness of
implementations of branch-and-bound algorithms. The complexity of such
implementations makes empirical study difficult for a wide variety of reasons.
Various attempts have been made to develop and codify a set of standard
techniques for the assessment of optimization algorithms and their software
implementations; however, most previous work has been focused on classical
sequential algorithms. Since parallel computation has become increasingly
mainstream, it is necessary to re-examine and modernize these practices. In
this paper, we propose a framework for assessment based on the notion that
\emph{resource consumption} is at the heart of what we generally refer to as
the "effectiveness" of an implementation. The proposed framework carefully
distinguishes between an implementation's baseline efficiency, the efficacy
with which it utilizes a fixed allocation of resources, and its scalability, a
measure of how the efficiency changes as resources (typically additional
computing cores) are added or removed. Efficiency is typically applied to
sequential implementations, whereas scalability is applied to parallel
implementations. Efficiency and scalability are both important contributors in
determining the overall effectiveness of a given parallel implementation, but
the goal of improved efficiency is often at odds with the goal of improved
scalability. Within the proposed framework, we review the challenges to
effective evaluation and discuss the strengths and weaknesses of existing
methods of assessment.
