AND/OR Branch-and-Bound for Computational Protein Design Optimizing K *
Abstract
The importance of designing proteins, such as high afﬁnity antibodies, has become ever more apparent.
Computational Protein Design can cast such design problems as optimization tasks with the objective of maximizing K * , an approximation of binding afﬁnity.
Here we lay out a graphical model framework for K * optimization that enables use of compact AND/OR search algorithms.
We designed an AND/OR branch-and-bound algorithm, AOBBK * , for optimizing K * that is guided by a new K * heuristic and can incorporate specialized performance improvements with theoretical guarantees.
As AOBB-K * is inspired by algorithms from the well studied task of Marginal MAP, this work provides a foundation for harnessing advancements in state-of-the-art mixed inference schemes and adapting them to protein design.
INTRODUCTION
The powerful framework of graphical models allows for reasoning about conditional dependencies over many variables.
The Marginal MAP (MMAP) task ﬁnds an optimal conﬁguration of a subset of variables (MAP variables), that have the highest marginal probability.
We deﬁne a new related task,
K * MAP, which instead asks for the conﬁguration of MAP variables that maximizes a quotient of marginals of conditionally disjoint subsets of the remaining variables.
This ratio is known as K * [Lilien et al., 2004].
In computational protein design ( CPD ), K * estimates afﬁnity between interacting subunits.
Maximizing K * corresponds to maximizing the likelihood that the subunits will associate [Hallen and
Donald, 2019].
Like MMAP, K * MAP distinguishes between maximization (MAP) variables and summation (SUM) variables.
Moreover, the SUM variables are further partitioned into a subset whose marginal corresponds to the numerator of the K * ratio and a subset corresponding to the denominator.
Like MMAP, the K * MAP problem relies on max-sum inference and more difﬁcult than either pure max- or sum- inference tasks as the sum and max operations do not commute.
This forces constrained variable orderings that may have signiﬁcantly higher induced widths
[Dechter, 1999, 2019].
This also implies larger search spaces when using search algorithms or larger messages when using message-passing schemes.
Furthermore, bounding K * requires both upper and lower bounding of marginals, producing an additional challenge.
Over the last several years, there have been several advances in algorithms for solving MMAP [Lou et al., 2018a,b, Marinescu et al., 2019, 2018a,b], many of which can potentially be adapted for K * MAP.
Our objective in this work is to (1) create a framework for which future advancements to the
K * MAP task can be made leveraging these powerful algorithms, (2) to begin an exploration of new bounded heuristics for the task, and (3) to create a foundation for efﬁcient algorithms solving (and eventually approximating) the K * MAP.
As such, this work presents the following contributions:
Two formulations of K * MAP as a graphical model 2.
wMBE-K * , a weighted mini-bucket algorithm for K * MAP enhanced with a domain partitioning scheme 3.
AOBB-K * , a depth-ﬁrst branch-and-bound algorithm over AND/OR search spaces for solving K * MAP 4.
A thresholding scheme to exploit determinism accompanied with correctness guarantees 5.
Extensive empirical analysis comparing these schemes to state-of-the-art BBK * illustrating their potential
BACKGROUND
Here we provide a brief background about protein design and graphical models.
2.1 COMPUTATIONAL PROTEIN DESIGN
Proteins are complex macromolecules composed of chains of amino acids that fold into a three dimensional structure based on intra- and inter- molecular interactions.
Proteins are essential to life and act as the machinery for a myriad of biological functions including chemical metabolism, cellular regeneration, immune response, movement, and structural support.
Computational Protein Design ( CPD ) is a ﬁeld focused on designing new proteins (ie.
new amino acid sequences) to satisfy desired objectives [Gainza et al., 2016].
Two sub ﬁelds of CPD are de novo protein design: the generation of novel protein sequences unrelated to those found naturally [Huang et al., 2016], and protein redesign: the modiﬁcation of known proteins to alter their function or interactions [Gainza et al., 2016].
The task we focus on in this work is this latter sub ﬁeld of protein redesign, speciﬁcally for improving protein subunit interactions.
For simplicity, we will use CPD to refer to such redesign tasks.
A somewhat orthogonal task often confused with protein design is protein structure prediction, or PSP, which aims to predict the three-dimensional structure resulting from a given amino acid sequence.
PSP gained visibility recently with the advancements of AlphaFold [Jumper et al., 2021] and can be leveraged by certain protein design tasks.
For the protein redesign tasks that this work focusses on, certain amino acid positions (or residues ) of a protein-ofinterest are deemed as mutable - these are amino acid positions where different amino acid mutations will be considered - and through a computational process, a preferred sequence is determined [Donald, 2011].
Throughout the process, sets of mutations are explored, each comprising a speciﬁc amino acid sequence.
Given a sequence (or in some methods, even partial sequence) an estimate of the resulting protein’s goodness can be estimated.
This goodness is determined by considering the possible conformations of the protein’s backbone and amino acid side-chains.
The state space for these conformations is continuous (and even when discretized, is extremely large) leading to an intractable problem.
To remedy, simpliﬁcations can be made: (i) consider only a subset of residues as mutable, (ii) discretize side-chain conformations into rotamers, and (iii) assume a ﬁxed protein backbone conformation.
With these simplifying assumptions, many algorithms have been designed to ﬁnd mutations that can potentially result in improved protein functionality [Hallen and Donald, 2019, Zhou et al., 2016].
2.2 K * AND K * MAP
The afﬁnity between two protein subunits P and L relates to the rate at which they bind into a complexed form PL and dissociate reforming P and L (as indicated by the chemical equation: P + L (cid:10) PL ).
This said equilibrium is associated with a constant, K a , and can be determined in vitro by computing the ratio of persisting concentrations of each species by K a =
[ PL ] [ P ][ L ] [Rossotti and Rossotti, 1961].
However, in order to compare K a values of various designs in vitro, it is necessary to synthesize the protein subunits through molecular processes that are both timely and costly.
K a can also be approximated as K (cid:82) =
[Equation]
L , where
[Equation]
and Z
[Equation]
, Z
[Equation]
P , and Z (cid:82) L are approximations of partition functions of the bound and unbound form of the protein - namely, when the protein subunits are complexed (denoted PL ) and when they are dissociate (denoted P and L ), respectively - that capture the entropic contributions of their various conformations
C
[Hill, 1987,
McQuarrie, 2000].
E γ ( c ) represents the energy of a particular conformation c of the protein in form γ ∈ ϕ
where ϕ
= { P, L, PL } , R is the universal gas constant, and T is temperature (in Kelvin).
We can further use a model that discretizes the conformation space, denoted as D ( C ) .
This computed estimate is known as K * [Lilien et al., 2004]:
[Equation]
Due to independence of residue interactions between the different forms γ of the protein, we can generalize further as K ∗ =
Z B Z U , where B represents the bound (complexed) form(s) and U represents the unbound (dissociate) forms.
(For a two-subunit system, B = { PL } and U = { P } ∪ { L }).
This generalized representation can be used for K * computations involving more than two subunits.
A common goal in protein redesign is to maximize proteinligand interaction.
Previously, this was done by minimizing an objective called the GMEC (global minimum energy conformation) over only the complexed protein form PL [Rufﬁni et al., 2021, Hallen and Donald, 2019, Zhou et al., 2016].
The GMEC, a pure minimum over energies of the complex’s conformations, ignores the realization that protein structures are dynamic.
Also, by the GMEC focusing only on the complexed form, it ignores dynamicity of subunit associations.
However, since minimizing the GMEC results in a pure optimization task - much easier than that of mixed inference - many solvers use this objective.
Alternatively, the stronger K * objective (contrasted in Lilien et al.
[2004]) captures both the dynamicity of protein conformations and subunit interactions.
K * MAP is the formalization of computational protein design as a task to maximize K * ,
[Equation]
looking for amino acid assignments R = r that maximize
K * .
The goal of recent work, and that presented here, is to develop efﬁcient algorithms for computing K * MAP to predict a small set of promising sequences to experiment on in vitro and in vivo, saving great time and cost.
2.3 GRAPHICAL MODELS AND THEIR AND/OR SEARCH SPACES
Our work taps into algorithms developed for the MMAP task deﬁned on graphical models.
A graphical model , such as a Bayesian or Markov network [Pearl, 1988, Darwiche, 2009, Dechter, 2013], can be deﬁned by a 3-tuple M =
( X , D , F ) ,
where X =
X i : i ∈ V } is a set of variables indexed by a set V and D
=
{ D i : i ∈ D } is the set of ﬁnite domains of values for each X i .
Each function f α ∈ F is deﬁned over a subset of the variables called its scope,
X α , where α ⊆ V are the indices of variables in its scope and D
α denotes the Cartesian product of their domains, so that f α : D α
→ R ≥ 0 .
Primal graph G =
( V , E ) of
M associates each variable with a node ( V = X ), while arcs e ∈ E connect nodes whose variables appear in the scope of the same function.
M deﬁnes a global function, often a factorized probability distribution on X , P ( X ) =
Z (cid:81)
α f α ( X α ) , where Z , known as the partition function, is a normalization factor.
Three common paradigms for solving inference tasks over graphical models are inference, search, and sampling.
In our work we use inference for generation of a bounded heuristic to guide search for solving the K * MAP query.
We use
AND/OR search which utilizes an AND/OR space deﬁned relative to a pseudo tree of the problem’s primal graph generated with respect to a variable ordering.
A pseudo tree
T =
( V , E (cid:48) ) of a primal graph G =
( V , E ) is a directed rooted tree that spans G with every arc of G not in E (cid:48) as a back-arc in T connecting nodes to their ancestors (Figure 1(a),(b)).
The pseudo tree structure determines the structure of the corresponding AND/OR search space.
Its directed edges dictate how the search space should be explored with branchings in the pseudo tree corresponding to branchings emanating from the AND nodes in the search space allowing for conditional independencies to be exploited.
For mixed inference problems where a subset of variables are to be maximized (MAP variables) and the remaining variables (SUM variables) marginalized, the pseudo tree must be constrained so that MAP variables precede SUM variables in the variable ordering [Marinescu et al., 2014].
Given a pseudo tree T of a primal graph G , the AND/OR search tree T T guided by T has alternating levels of OR nodes corresponding to variables, and AND nodes corresponding to assignments from their domains with edge costs extracted from the original functions [Dechter and Mateescu, 2007].
Each arc into an AND node n has a cost c ( n ) deﬁned to be the product of all factors f α in M that are instantiated at n but not before (see Dechter and Mateescu [2007]).
The size of the search space is exponential in the depth of the pseudo tree (rather than the number of variables) and therefore mixed inference tasks such as MMAP can be solved more efﬁciently by traversing the AND/OR search space whenever its depth is bounded sufﬁciently.
GRAPHICAL MODEL FOR K * MAP
As the ﬁrst main contribution of this work, we describe two formulations of CPD problems as graphical models for computing K * MAP.
These build upon previous work from
MMAP (see Marinescu et al.
[2018b]) and formulations for optimizing the weaker GMEC objective [Zhou et al., 2016].
3.1 FORMULATION 1 (F1)
Variables and Domains: Let R =
{ R i | i ∈ { 1 , 2 , ..., N }} be the set of residue variables representing N different residues (i.e., positions) of the protein.
Each R i has a corresponding domain D R i
= { aa | aa is a possible amino acid assignment to residue i } .
For residues that are being considered for mutation ( mutable residues ), each R i considers one of ∼ 20 possible amino acid assignments in its domain of values.
These are the MAP variables maximized over in the K * MAP task.
C γ =
{ C γ ( i ) | i ∈ { 1 , 2 , ..., N }} are conformation variables , each indexing a discrete spacial conformation (rotamer) of the amino acid at residue R i when its subunit is in form γ .
Each C γ ( i ) has corresponding domain
D C γ ( i ) = { 1 , 2 , ..., M γ ( i ) } , where M γ ( i ) is the maximum number of rotamers for any amino acid assignment to R i in form γ .
The assignment to C γ ( i ) acts as an index to the possible side chain conformations of the amino acid assigned to
R i .
The C γ are the SUM variables to be marginalized over.
Functions: We consider two sets of functions: E sbγ =
{ E sbγ ( i ) ( R i , C γ ( i ) )
| i ∈ { 1 , 2 , ..., N }} that capture the energies of interaction of the amino acid at each residue i with itself and the surrounding backbone, and E pwγ =
{ E pwγ ( ij ) ( R i , C γ ( i ) , R j , C γ ( j ) ) | for i, j s.t.
R i and R j interact } that capture pair-wise energies of interaction between amino acids in close proximity.
For any assignment to C γ ( i ) (which corresponds to a rotamer-index) that is out of range of R
i ’s assigned amino acid’s possible rotamers, an inﬁnite energy value is assigned as an implicit constraint.
Objective: The K * objective can thus be expressed as
K ∗ ( R 1 ...R N ) =
Z B ( R 1 ...R N )
Z U ( R 1 ...R N ) , where we assume temperature T in Kelvin and Universal Gas Constant R and where:
Z
[Equation]
3.2 FORMULATION 2 (F2)
Formulation 2, inspired by the works of Viricel et al.
[2018] and Vucinic et al.
[2019], uses explicit constraints to restrict invalid amino acid - rotamer combinations.
Variables and Domains: As in F1, a set of residue variables R =
{ R i | i ∈ { 1 , 2 , ..., N }} and conformation variables C γ
= { C γ ( i ) | i ∈ { 1 , 2 , ..., N }} are considered.
Here, each C γ ( i ) represents a speciﬁc amino acid and conformation of the N different residues.
Namely, each C γ ( i ) has a domain D C γ ( i )
= { c | c is a rotamer for one of the possible amino acids of R i } .
The amino acid assignment to R i acts as a selector into the possible assignments to C γ ( i ) .
Functions: C =
{ C γ ( i ) ( R i , C γ ( i ) )
| i ∈ { 1 , 2 , ..., N } , γ ∈ ϕ } is a set of constraints ensuring that the assigned rotamer to C γ ( i ) belongs to the amino acid assigned to R i .
In addition, E sbγ =
{ E sbγ ( i ) ( C γ ( i ) )
| i ∈ { 1 , 2 , ..., N }} captures the energies of interaction of the amino acid at each residue i with itself and the surrounding backbone and E pwγ
= { E pwγ ( ij ) ( C γ ( i ) , C γ ( j ) )
| for i, j s.t.
R i and R j interact } captures the pair-wise energies of interaction between amino acids in close proximity.
Objective: Similar to before, the K * objective can be expressed as K ∗ ( R 1 ...R N ) =
Z B ( R 1 ...R N )
Z U ( R 1 ...R N ) , where
Z γ ( R 1 ...R N ) =
[Equation]
With a graphical model framework in place, next we describe a heuristic that can be used to bound the K * MAP value and that can be used to guide search.
WMBE-K *
Next we present a weighted mini-bucket based approximation scheme for K ∗ , which adapts weighted mini-bucket for
MMAP [Dechter and Rish, 2002, Marinescu et al., 2014] to the new K ∗ objective.
Since the constrained-order minibucket bounds are compatible with AND/OR search, we will leverage these bounds in the subsequent section to guide search as part of a branch-and-bound scheme.
1 begin
Partition the functions f ∈ F into buckets B n , ..., B 1 s.t.
each function is placed in the bucket corresponding to the highest-index variable in its scope.
foreach k n.. do
Generate a mini-bucket partitioning of the bucket functions MB k
= { MB 1 k , ..., MB Tk } s.t. vars ( MB tk ) ≤ i , for all MB tk ∈ MB
k if X k
MAP then foreach do
else
Algorithm wMBE-K * is described in Algorithm 1 and operates similarly to wMBE-MMAP
[Marinescu et al., 2014].
Two key similarities are that (1) it takes a variable ordering that constrains buckets of MAP variables to be processed last (line 3) for which maximization occurs, and (2) for any bucket that has a width larger than a provided i-bound, a bounded approximation is made by partitioning the bucket functions into T mini-buckets (line 4) and taking the product of their power-sums over the bucket variable (lines 9-12, 13-16), leveraging Holder’s Inequality [Hardy et al., 1988].
For K * MAP, two key innovations are required: (1) buckets corresponding to variables in C U , whose marginal belongs to the denominator of the K * expression, are lower-bounded (to lead to an upper bound on K * ) by using a modiﬁcation to Holder’s inequality that incorporates negative weights [Liu and Ihler, 2011] (lines 13-16), and (2) when messages are passed from buckets corresponding to variables in C U to that of R , the messages are inverted to accommodate being part of the denominator (line 18).
Although details are omitted here, wMBE-K * can also employ cost shifting to tighten its bounds (see Flerova et al.
[2011], Liu and Ihler [2011]).
In our empirical evaluation cost-shifting is implemented as well.
Finally, the complexity of the algorithm, which is parametrized by the i-bound i , is time and space exponential in i only.
As can be expected, bounding a ratio of functions (as in the case for K * ) is particularly challenging, relying on both upper and lower bounds.
Lower bounding of functions is particularly challenging.
For larger problems and low ibounds, this can often yield relatively weak bounds.
We provide an improvement to help remedy this next.
Domain-Partitioned MBE Until now, we used a minibucket scheme blind to explicit hard constraints and consistency issues.
This can be handicapping when lowerbounding since constraints are represented as zero’s in functions and can cause premature deﬂation of lower bounds.
In particular, in mini-bucket elimination [Dechter and Rish, 2003] where lower bounds are created via minimizing over function values, zeros in the functions being minimized will cause the resulting lower bound itself to drop to zero.
However, in the CPD domain where functions represent protein energetics, satisﬁable conﬁgurations corresponds to positive function values and thus can guarantee a positive lower bound by using the following simple remedy:
Let X , Y , and Z be three variables and let obj =
(cid:80)
X f ( x, y ) · g ( x, z ) , let X (cid:48)
= { x ∈ X | g ( x, z ) (cid:54) = 0 } be a set such that (cid:15) X (cid:48)
=
min x ∈ X (cid:48) g ( x, z ) .
Clearly, (cid:15) X (cid:48) >
and therefore we can derive:
obj =
(cid:80) x ∈ X (cid:48)
f ( x, y ) · g ( x, z ) + (cid:80) x ∈ X \ X (cid:48)
f ( x, y ) · g ( x, z ) =
(cid:80) x ∈ X (cid:48)
f ( x, y ) · g ( x, z ) ≥ (cid:15) X (cid:48) · (cid:80) x ∈ X (cid:48)
f ( x, y ) > 0 , assuming f ( x, z ) is not identically zero over X (cid:48) .
Mini-buckets can then be computed according to such domain-partitions to improve their bounds.
AOBB-K *
We present the key algorithmic contribution of this work:
AOBB-K * (Algorithm 2), a depth-ﬁrst AND/OR branchand-bound scheme for solving K * MAP.
With state-ofthe-art K * optimizers such as BBK * employing memoryintensive best-ﬁrst search [Ojewole et al., 2018, Hallen et al., 2018], depth-ﬁrst algorithms provide scheme linear in space allowing for solving problems unable to be solved by bestﬁrst methodologies due to memory [Zhou et al., 2016].
AOBB-K * traverses the underlying AND/OR search tree guided by the provided pseudo tree T , expanding nodes in a depth-ﬁrst manner (line 9), and pruning whenever any of three conditions are triggered: (1) the resulting variable assignments violate constraints encoded as inﬁnite values in M (line 10), (2) a subunit-stability constraint (SSC – a constraint which enforces the partition function of each protein subunit, Z γ , to be greater than a biologically-relevant threshold S γ [Ojewole et al., 2018]) is violated (line 12), do or (3) it can be asserted that the current amino acid conﬁguration cannot produce a K * better than any previously found (line 15).
Backtracking occurs when all of a node’s children have been explored and returned from (line 17), at which point the K * value of the sub problem the node roots is known exactly and bounds of its parents are tightened accordingly.
Inﬁnite energy tuples in M ’s functions model an inconsistent amino acid - rotamer pairs and correspond to hard constraints encoded into a CNF formula as in [Allen and Darwiche, 2003].
During search, unit-propagation (e.g.,
Eén and Sörensson [2004]) is propagates these constraints and detects infeasible variable conﬁgurations (line 10).
The algorithm progresses in this manner until it ﬁnally returns to, and updates, the root of the search tree with the maximal K * value corresponding to an amino acid conﬁguration that also satisﬁes the subunit-stability thresholds.
Throughout search, each node n maintains a progressive upper bound ub K ∗ ( n ) on the K * MAP of the sub problem it roots.
When a node is expanded, this value is initialized based on upper-bounding heuristic function h ubK ∗ ( . )
(line 4).
As search progresses, ub K ∗ ( n ) decreases converging towards the K * MAP of the sub problem rooted at n .
Furthermore, each node n also maintains a progressively improved upper bound on the partition function of each subunit γ consistent with the path to n , UB Z γ ( n ) (line 8).
At each step in the search, UB Z γ ( n ) is recomputed to ensure that it is greater than the given S γ , thus satisfying the SSC’s and enforcing consideration of only biologically relevant solutions [Ojewole et al., 2018].
Note that the SSC’s are not encoded into the problem, and thus add another layer of complexity not present in tasks such as MMAP.
Theorem 5.1 (correctness, completeness) .
AOBB-K * is sound and complete, returning the optimal K * value of all amino-acid conﬁgurations that do not violate the subunitstability constraints.
Theorem 5.2 (complexity) .
AOBB-K * is time O ( n · k d ) , where n is the number of variables, k is the maximum domain size, and d is the dept of the guiding pseudo tree, and is space O ( n ) (see Marinescu and Dechter [2009b]).
When modiﬁed to search the context minimal AND/OR search graph (as opposed to search tree) AOBB-K * is both time and space O ( n · k w ∗ ) , where w ∗ is the induced width of the pseudo tree (see Marinescu and Dechter [2009a]).
5.1 WEIGHTED SEARCH FOR K *
Weighted best-ﬁrst search (e.g., WA ∗ [Pohl, 1970], WAO ∗ [Desarkar et al., 1987]) is a well known principle for converting best-ﬁrst search into an anytime scheme by multiplying the heuristic function h ( n ) of a node n in the search space by a weight ω > 1 .
The solution is guaranteed to be ω -optimal (i.e., within a factor ω from the optimal one).
Therefore, AOBB-K * can easily be relaxed to an ωapproximation scheme (for ω ∈ [0 , 1) ) by multiplying h K ∗ ( n ) at each node n in the AND/OR search tree by a factor of ω .
It can be shown that the resulting solution will be at worst ω · K ∗ MAP [Pohl, 1970, Flerova et al., 2014].
We explore the performance of applying such approximations to a class of difﬁcult CPD problem instances in Section 6.
5.2 INFUSING DETERMINISM VIA THRESHOLDED UNDERFLOWS
Exploiting determinism via constraint propagation ( CP ) can be a powerful tool [Dechter, 2003, 2019] helping to prune invalid conﬁgurations corresponding to inconsistent amino acid - rotamer pairs or conﬁgurations not contributing to a subunit’s partition function.
CP can be accelerated by introducing additional determinism such that the solution is unchanged.
Namely, extremely small function values (corresponding to extremely unfavorable side-chain interactions that would not occur in feasible solutions) can be underﬂowed (ie.
replaced by zeros) and treated as hard constraints.
Deﬁnition 5.1 ( τ -underﬂow of f , f τ ) .
Let f be a nonnegative function and τ ∈ R + .
The τ -underﬂow of f is f τ ( x ) =
f ( x ) if f ( x ) ≥ τ and 0 , otherwise.
Deﬁnition 5.2 ( τ -underﬂow of M , M τ ) .
For M = (cid:104) X , D , F (cid:105) , the τ -underﬂow of M is M τ = (cid:104) X , D , F τ (cid:105) , where F τ = { f τ | f ∈ F } .
Notation.
We denote: f maxγ - the maximum value in all the functions associated with subunit γ , | f γ | - the number of functions included in the partition function computation for subunit γ , excluding explicit constraints, | C γ | ⇑ - the cardinality of the Cartesian product of greatest number of assignments to each variable in C γ that are individually consistent with any assignment R = r , Z minB - the smallest
Z B that could possibly lead to a valid K ∗ > K ∗ ( wt ) (here wt refers to the wild-type , or naturally occurring, amino acid sequence), prec ( x ) - the smallest decimal place for which the value of x is recorded.
We give two sufﬁcient conditions that, when upheld, allow us to assert K ∗ MAP ( M τ ) =
K ∗ MAP ( M ) : (1) τunderﬂows do not alter subunit partition functions that satisfy SSC’s, i.e., ∀ r ∈ R s.t.
Z M γ ( r ) ≥ S γ , Z M γ ( r ) =
Z M τ γ ( r ) and (2) τ -underﬂows do not alter any Z B ≥ Z minB , i.e., ∀ r ∈ R s.t.
Z M B ( r ) ≥ Z minB , Z M B
( r ) =
Z M τ B ( r ) .
Condition 1: A problem created with underﬂows using an underﬂow threshold τ such that ∀ γ ∈ ϕ : (i) if τ ≤ 1 , | C γ | ⇑ · ( f maxγ ) | f γ | · ( τ ) < 0 .
5 · prec ( S γ ) , or (ii) if τ > 1 , | C γ | ⇑ · ( f maxγ ) | f γ | · ( τ ) | f γ | < 0 .
5 · prec ( S γ ) will not violate any subunit stability constraints.
Condition 2: A problem created with underﬂows using an underﬂow threshold τ such that τ does not violate the SSC’s and such that: (i) if τ ≤ 1 , | C B | ⇑ · ( f maxB ) | f B | · ( τ ) < 0 .
5 · prec ( Z minB ) , or (ii) if τ > 1 , | C B | ⇑ · ( f maxB ) | f B | · ( τ ) |
f B | < 0 .
5 · prec ( Z minB ) , will also not alter the K * MAP solution.
Theorem 5.3.
If a chosen τ satisﬁes Conditions 1 and 2, then K ∗ MAP ( M τ ) = K ∗ MAP ( M ) .
Complexity.
Both Conditions 1 and 2 can be veriﬁed in linear time, thus the sufﬁcient condition presented in Theorem 5.3 can also be checked in linear time.
Although Theorem 5.3 provides a way to verify a τ used will still result in the correct optimal solution, the theorem’s underlying conditions are often overly conservative, and, in practice, even τ larger than those satisfying the theorem can yield exact solutions.
EMPIRICAL EVALUATION
We now present the empirical evaluation of our schemes on real protein benchmarks provided to us by the Bruce Donald
Lab at Duke University.
Benchmarks.
We experimented on 48 protein design benchmarks, 30 with two mutable residues (denoted "original"),
12 made harder expanding to three mutable residues ("expanded"), and six problems expanded to four mutable residues ("hard").
Problems were generated using OSPREY 3.0 [Hallen et al., 2018] to compute interaction energies and then formulated into F1 and F2.
AOBB-K * vs state-of-the-art BBK * .
We ran AOBB-K * and its derived schemes, [ ω, τ ]-AOBB-K * , built on top of
AOBB-MMAP [Marinescu et al., 2014] implemented in
C++.
For comparison, we also run BBK * [Ojewole et al., 2018], a state-of-the-art best-ﬁrst search algorithm available in OSPREY 3.0.
BBK * (implemented in Java, set to use rigid rotamers and with a bound-tightness of 1 × 10 − 200 1 ).
Contrasting major differences of the two schemes, (1)
AOBB-K * uses a depth-ﬁrst traversal allowing for memory efﬁciency and ease of facilitating constraint propagation while BBK * uses memory-intensive best-ﬁrst search, (2) our experiments with AOBB-K * use the statically compiled wMBE-K * generated before search while BBK * exclusively uses a dynamic heuristic that considers best and worst case scenarios from all possible ensuing conﬁgurations during search, (3) we use constraint propagation dynamically during search to prune provably invalid (or zero-weighted) regions of the search space, and (4) AOBB-K * is designed as an exact scheme that ﬁnds the optimal K * that satisﬁes
SSC’s (ie.
exact for the K * MAP task) whereas BBK * is inherently designed as a bounded approximation scheme that, even when using the very low tightness of 1 × 10 − 200 , seems not to always return the optimal K * MAP.
Experiments were allotted 1hr on a 2.66 GHz processor with 4 GB of memory and using subunit-stability thresholds
S γ
=
Z ( wt ) γ · e − 5 R T where Z ( wt ) γ is the partition function of the wild-type sequence.
As BBK * can take advantage of parallelism, it was run with access to 4 CPU cores.
AOBB-K * was run using wMBE-K * with moment matching [Flerova et al., 2011, Liu and Ihler, 2011] for guiding and bounding search.
For hard problems, a derivation of wMBEK * , wMBE + -K * , was tested that avoided consideration of zeros during lower-bounding approximations (motivated by Section 4).
For all experiments, wMBE-MMAP was used to upper-bound the partition function of each subunit.
According to Ojewole et al.
[2018], BBK * uses a dynamic greedy heuristic based on optimistic values for all variables not yet instantiated.
In all problems, each mutable residues considers 21 different amino acid assignments.
Conformation variables of non-mutable residues have a domain size of 2-14 rotamers (most having domain sizes 4-9).
Conformation variables of mutable residues have a domain size of 34-35 when formulated as F1 and 203-205 when formulated as F2.
[Table]
[Table] [Table]
[Table] [Table]
heuristic, τ : underﬂow-threshold, H : heuristic, iB : i-bound used, w* : induced width, d : pseudo tree depth, | R | : number of MAP variables, | X | : total number of variables, Dmax : maximum domain size, UB : heuristic upper bound at the root (empty cells representing no ﬁnite bound), pre-t : preprocessing time (ex. compiling heuristics), search : search time, time : total time, K * : the returned K * solution (in log 10 ), and Any-K * : best valid K * value found in 1hr.
The best performance points are highlighted.
[Table]
Formulation 1 vs. Formulation 2: We can see from the aggregated time statistics in Table 1 that F2 is generally superior to F1, outperforming BBK * more frequently.
F uses an indexing scheme between amino-acids and their rotamers, which implies that all functions need to include both residue and conformation variables thus leading to densely connected graphs compared with F2.
A strength of
F1 vs. F2 is its smaller domain sizes in the presence of MAP variables which facilitates use of higher i -bounds compared with F2.
AOBB-K * vs BBK * (solution quality): The summary in
Table 1 show that AOBB-K * and BBK * ﬁnd the same K * solution for the majority of the original problems.
As problems are expanded, AOBB-K * begins to ﬁnd better solutions more frequently.
Concrete examples are displayed in Tables 3 and 4 where greater K * solutions by AOBB-K * are highlighted in the K * MAP columns.
AOBB-K * vs BBK * (running time): The results in Table 3 show that AOBB-K * solved nearly all original benchmark problems faster than BBK * (see the time column).
As problems were expanded (Table 4), we see AOBB-K
* ’s time-performance begin to drop more rapidly than BBK * ’s;
AOBB-K * surpasses BBK * on four problems, but not on the other eight.
No hard problem was solved exactly without modiﬁcations (presented shortly).
However, it is important to note that: (1) AOBB-K * ﬁnds solutions greater than that of BBK * (which may account for increased time searching) and not explicitly shown here (2) AOBB-K * returns intermediate anytime solutions, some of which exceeded the K * value returned by BBK * .
[Table]
[Table]
Search: Each pair of rows in Table 5 compares AOBB-K * (denoted ω = 1 ) and ω -AOBB-K
* ( ω = 0 .
001 ) on all Expanded problems.
Moving to approximate search reduces the MAP search space and improves search time (sometimes by more than a factor of ten).
For 2rfe_prepped_00017 * , 2xgy_prepped_00020 * , and 4wwi_prepped_00019 * , we see the optimal i-bound is reduced from 5 to 4 when using weighted search highlighting the increased pruning of the search space enabled by the weighted heuristic even when using a weaker (but more quickly computed) heuristic.
(When using the same i-bound, the weighted search still showed signiﬁcant speed-ups solving these problems).
With other advances, weighted search was instrumental to solving the hard problems.
Infusing Determinism: We assessed the impact of infusing determinism as discussed in Section 5.2 by applying thresholded-underﬂow with τ = 1 × 10 − 5 and comparing to base problems ( τ = 0 ).
Table 6 shows the results obtained on expanded problems.
In all cases, we see that the τ -underﬂows improved search times (see "search" column), sometimes so much so that the best time corresponded to allowing for a more crudely computed heuristic to enter search more quickly.
Even when the same iB is used, the overall time is still reduced, however interestingly the speedup does not always correspond to a reduced search of the MAP space; we found that the heuristic can be adversely affected by underﬂows (as described in Section 4’s discussion on Domain-Partitioned MBE) resulting in less efﬁcient pruning of the MAP space.
However, this can potentially be remedied, which we explore next.
wMBE + -K * : Bounding a ratio of functions such as in K * is especially difﬁcult - particularly because it relies on lowerbounding, which is especially problematic in the presence of determinism.
To empirically test the intuition presented in Section 4 and in attempt to solve problems that could not be solved by vanilla AOBB-K * , we used wMBE + -K * (which avoids considering zeros when lower bounding) with ω, τAOBB-K * ( ω = 0 .
001 , τ = 1 × 10 − 5 ) on hard F1 problems, for which wMBE-K * was unbounded (Table 7).
In every case, the modiﬁed heuristic was able to provide a bounded estimate and in three of the six problems enabling AOBB-K * to ﬁnd a solution better than BBK * .
In summary , AOBB-K * shows strong performance on the original protein problems, especially with Formulation F2, outperforming state-of-the-art BBK* time-wise and ﬁnding better solutions.
AOBB-K * also performed admirably on expanded problems, ﬁnding solutions faster for several problems and ﬁnding better solutions.
The innovations of ω -approximate search, infusing determinism, and omitting zeros from lower bounding enabled the algorithm to ﬁnd solutions to harder problems, some of which were better than BBK * ’s.
However, these hard problems were a challenge for base AOBB-K * .
The increase in domain sizes as
MAP variables increase proved problematic memory-wise for compiling the heuristic, restricting to low iB’s resulting in less helpful bounds.
This challenge, and the realization
[Table]
[Table]
[Table]
that bounds can be improved by domain partitioning, motivates future work into new representations.
SUMMARY, CONCLUSION, AND FOUNDATION FOR FUTURE WORK
Summary.
This work creates a competitive framework for which future advancements to protein redesign optimizing
K * can be made leveraging advances in graphical model algorithms.
This work also features an exploration of new bounded heuristics for the K * MAP task and identiﬁes areas amenable to improvement.
As part of the presented framework, new efﬁcient algorithms solving the K * MAP task are introduced along with an empirical proof of concept by evaluation against a pre-existing state-of-the-art algorithm.
Conclusion.
We leave readers with 1.
two distinct graphical model formulations to address the K * MAP query, and analysis showing Formulation F2 is generally superior; 2.
a new guiding wMBE-K * heuristic and demonstration of the potential to improve its bounds by domain-partitioning; 3.
AOBB-K * , a depth-ﬁrst AND/OR branch-and-bound algorithm for optimizing K * (and an accompanying approximate ω -AOBB-K * ) that proved competitive against state-of-theart BBK * ; 4.
a scheme to exploit determinism into problems by introducing underﬂows, with theoretical guarantees; and 5.
extensive empirical analysis on over forty benchmarks providing the above-mentioned analysis.
As a part of our results, we note that AOBB-K * ﬁnds a K * greater than that of BBK * on several occasions, leaving an open question as to the cause and how it impacts time-performance.
As a foundation.
Research directions illuminated by our results are: (1) exploration of more compact representations able to exploit determinism [Mateescu and Dechter, 2008,
Larkin and Dechter, 2003]; (2) development of new K * heuristics (incorporating constraints, using alternative frameworks such as Deep Bucket Elimination [Razeghi et al., 2021]), employing dynamic heuristics); and (3) adaptation of state-of-the-art mixed-inference schemes to K * MAP [Lou et al., 2018a,b, Marinescu et al., 2019, 2018a,b]. This work can be tuned more to the protein domain by: (1) evaluating new CPD’s with additional independencies exploitable by AND/OR schemes, (2) integrating optimizations present in developed protein software such as BBK * ’s OSPREY 3.0; and (3) considering backbone ensembles.
Acknowledgements
Special thanks to Thomas Schiex, from INRAE, France, and Bruce Donald and his members of his lab at Duke University, including Graham Holt, Jonathan Jou, and Nathan Guerin.
We also thank our reviewers for their valuable comments.
This work was supported in part by NSF grant IIS-2008516.