Maximum n-times Coverage for Vaccine Design

We introduce the maximum n-times coverage problem that selects k overlays to maximize the summed coverage of weighted elements , where each element must be covered at least n times . We also define the min-cost n-times coverage problem where the objective is to select the minimum set of overlays such that the sum of the weights of elements that are covered at least n times is at least τ . Maximum n-times coverage is a generalization of the multi-set multi-cover problem , is NP-complete , and is not submodular . We introduce two new practical solutions for n-times coverage based on integer linear programming and sequential greedy optimization . We show that maximum n-times coverage is a natural way to frame peptide vaccine design , and find that it produces a pan-strain COVID-19 vaccine design that is superior to 29 other published designs in predicted population coverage and the expected number of peptides displayed by each individual ’ s HLA molecules . 1 INTRODUCTION . In the maximum n-times coverage problem , a set of overlays is selected to cover elements zero or more times , where each overlay is predetermined to cover one or more elements . Each element is assigned a weight that reflects its importance . The objective of the problem is to maximize the sum of the weights of elements that are covered at least n times by at most k overlays . When the coverage of elements by overlays is determined by a machine learning method , the result summarizes machine learning results with a compact solution that is shaped by the element weights employed . Our introduction of weighted elements and required n-times coverage creates a new class of problem without a known solution or complexity bound for approximate solutions . The closest problem to the maximum n-times coverage problem is the multi-set multi-cover problem which does not assign weights to the elements and thus can be formulated as the Covering Integer Program ( CIP ) problem ( Srinivasan , 1999 ) . Thus like CIP , maximum n-times coverage is NP-complete . A log ( n ) time approximation algorithm for CIP can violate coverage constraints ( Dobson , 1982 ; Kolliopoulos , 2003 ; Kolliopoulos & Young , 2001 ) . Deletion-robust submodular maximization protects against adversarial deletion ( Bogunovic et al. , 2018 ; Mirzasoleiman et al. , 2017 ) and robust submodular optimization protects against objective function uncertainty ( Iyer , 2019 ) , but neither guarantees that each element is covered n times . In this work , we investigate properties of the maximum n-times coverage problem , provide a practical solution to the problem , and use the solution for machine learning based vaccine design . We show that maximum n-times coverage is not submodular , and introduce NTIMES-ILP and MARGINALGREEDY , efficient algorithms for solving the n-times coverage problem on both synthetic data and real vaccine design . In our framing of vaccine design , an element is a specific collection of HLA alleles ( a genotype ) , weights are the frequency of genotypes in the population , n is the desired number of peptides displayed by each individual , and an overlay is a peptide that is predicted to be displayed by each genotype a specified number of times . The solution of the maximum n-times coverage problem allows us to find a set of overlays that maximizes the sum of element weights ( population coverage ) . We show that framing vaccine design as maximum n-times coverage produces a solution that produces superior predicted population coverage when compared to 29 previous published vaccines for COVID-19 with less than 150 peptides . 2 THE MAXIMUM n-TIMES COVERAGE PROBLEM 2.1 MULTI-SET MULTI-COVER PROBLEM . In the standard SET COVER and MAXIMUM COVERAGE problems , we are given a set U of |U| elements ( also known as the universe ) and a collection S = { S1 , S2 , . . . , Sm } of m subsets of U such that ⋃ i Si = U . The goal in the SET COVER problem is to select a minimal-cardinality set of subsets from S such that their union covers U . The MULTI-SET MULTI-COVER ( MSMC ) problem is a generalization of the SET COVER problem , where multi-sets are sets in which an element can appear more than once . The objective of the MSMC problem is to determine the minimum number of multisets ( a multi-set can be chosen multiple times ) such that each element i is covered at least bi times . It can be formulated into Covering Integer Program ( CIP ) problem ( Srinivasan , 1999 ) : Definition 1 . ( Covering Integer Program , CIP ) Given A ∈ Rn×m+ , b ∈ Rn+ , w ∈ Rm+ , d ∈ Rm+ , a CIP P = ( A , b , w , d ) seeks to minimize wTx , subject to Ax ≥ b , x ∈ Zm+ , and x ≤ d. Here Aij represents the number of times i-th element appears in the j-th multi-set . The w is set to be all 1 in MSMC . The constraints x ≤ b are called multiplicity constraints which limit the number of times a single multi-set can be reused , and they generally make covering problems much harder as natural linear programming ( LP ) relaxation can have an unbounded integrality gap ( Chuzhoy & Naor , 2006 ) . Dobson ( 1982 ) provides a combinatorial greedy H ( maxj ∑ i Aij ) -approximation algorithm ( H ( t ) stands for t-th harmonic number ) but multiplicity constraints can be dealt with effectively only in the ( 0,1 ) case , and thus this algorithm can be as bad as polynomial . Kolliopoulos & Young ( 2001 ) ; Kolliopoulos ( 2003 ) gave a tighter-bound solution that can obtain O ( log n ) -approximation . 2.2 MAXIMUM n-TIMES COVERAGE We introduce the maximum n-times coverage problem , a variant of the MSMC problem that accounts for multiple coverage of each element while also assigning weights to different elements . We are given a set X with l elements { X1 , X2 , . . . , Xl } each associated with a non-negative weight w ( Xi ) , and a set of m overlays A = { A1 , A2 , . . . , Am } . Each overlay Aj covers each element i in X an element specific number of times cj ( Xi ) , which is similar to a multi-set . When cj ( Xi ) = 0 the element Xi is not covered by overlay Aj . We use a very strict multiplicity constraint such that each overlay can be used only once . Given a subset of overlays O ⊆ A , the total number of times an element Xi is covered by O is the sum of cj ( Xi ) for each overlay j in O : C ( Xi|O ) = ∑ j∈O cj ( Xi ) ( 1 ) We define the n-times coverage function fn ( O ) as the sum of weights of elements in X that are covered at least n times by O . Figure 1 shows an example computation of the n-times coverage function . fn ( O ) = l∑ i=1 w ( Xi ) 1 { C ( Xi|O ) ≥n } = l∑ i=1 w ( Xi ) 1 { ∑ j∈O cj ( Xi ) ≥n } ( 2 ) The objective of the MAXIMUM n-TIMES COVERAGE problem is to select a set of k overlays O ⊆ A such that fn ( O ) is maximized . This can be formulated as the maximization of the monotone set function fn ( O ) under cardinality constraint k : O∗ = argmax O⊆A , |O|≤k fn ( O ) ( 3 ) We define MIN-COST n-TIMES COVERAGE as the minimum set of overlays such that the sum of the weights of elements covered at least n times is ≥ τ . We assume A provides sufficient n-times coverage for fn ( A ) ≥ τ . We define the n-TIMES SET COVER problem as the special case of MIN-COST n-TIMES COVERAGE where τ = ∑ i w ( Xi ) . MIN-COST n-TIMES COVERAGE O∗ = argmin O |O| s.t . fn ( O ) ≥ τ ( 4 ) n-TIMES SET COVER O∗ = argmin O |O| s.t . fn ( O ) = fn ( A ) ( 5 ) Theorem 1 . The n-times coverage function fn ( O ) is not submodular . The proof can be found in Appendix A . Since the n-times coverage problem is not submodular , we can not take advantage of the proven near-optimal performance of the greedy algorithm for the SUBMODULAR MAXIMUM COVERAGE problem . Thus we seek new solutions . 3 THE NTIMES-ILP SOLUTION FOR MAXIMUM n-TIMES COVERAGE We first formulate the maximum-n times coverage problem as an integer linear program ( ILP ) . We associate a binary variable ai with each overlay Ai such that ai = 1 if Ai ∈ O∗ and otherwise ai = 0 . Thus the cardinality constraint can be written as |O| = ∑m i=1 ai ≤ k , and the number of times an element Xj is covered by O is C ( Xj |O ) = ∑ i∈O ci ( Xj ) = ∑m i=1 aici ( Xj ) . The main challenge is to encode the objective function fn ( O ) = ∑l j=1 w ( Xj ) 1 { ∑m i=1 aici ( Xj ) ≥n } in a linear fashion . We replace the step function 1 { ∑mi=1 aici ( Xj ) ≥n } with a variable tj such that for each element Xj : tj = 1 ⇐⇒ m∑ i=1 aici ( Xj ) ≥ n and tj = 0 ⇐⇒ m∑ i=1 aici ( Xj ) < n ( 6 ) We enforce the conditions in ( 6 ) with the Big M method ( Sherali et al. , 2013 ) , where we include the following inequalities for each element Xj ∈ X : −M ( 1− tj ) ≤ m∑ i=1 aici ( Xj ) − n+ ϵ ≤Mtj , ( 7 ) We set ϵ ∈ ( 0 , 1 ) and choose M to be a large number such as 10,000 which is larger than the term∑m i=1 aici ( Xj ) − n+ ϵ . Here we prove that tj = 1 ⇐⇒ ∑m i=1 aici ( Xj ) ≥ n. Proof . When tj = 1 , inequality ( 7 ) becomes 0 ≤ ∑m i=1 aici ( Xj ) − n + ϵ ≤ M . If M is a large number , we can ignore the right inequality and rearrange to get ∑m i=1 aici ( Xj ) ≥ n − ϵ. Since∑m i=1 aici ( Xj ) and n are both integers and ϵ ∈ ( 0 , 1 ) , we find ∑m i=1 aici ( Xj ) ≥ n. All these steps can be taken in the reverse order to prove that ∑m i=1 aici ( Xj ) ≥ n =⇒ tj = 1 , given that tj is a binary variable . We also prove that tj = 0 ⇐⇒ ∑m i=1 aici ( Xj ) < n. Proof . We start by showing the forward direction . If tj = 0 , then inequality ( 7 ) becomes −M ≤∑m i=1 aici ( Xj ) − n+ ϵ ≤ 0 . As M is large , −M is a large negative number , so we can ignore the left inequality . Hence , ∑m i=1 aici ( Xj ) ≤ n− ϵ < n. In the reverse direction , if ∑m i=1 aici ( Xj ) < n , because both the quantities ∑m i=1 aici ( Xj ) and n are integers , for any ϵ ∈ ( 0 , 1 ) , ∑m i=1 aici ( Xj ) ≤ n− ϵ . Rearranging this we get ∑m i=1 aici ( Xj ) − n+ ϵ ≤ 0 , which forces tj as a binary variable to be equal to 0 from inequality ( 7 ) . Therefore , the objective function to maximize becomes fn ( O ) = ∑l j=1 w ( Xj ) 1 { C ( Xj |O ) ≥n } =∑l j=1 tjw ( Xj ) , which is linear as a sum over l terms of a binary variable multiplied by a constant . The complete NTIMES-ILP formulation of n-times coverage is : • Variables – a1 , ... , am representing presence of each overlay A1 , ... , Am in final solution – t1 , ... , tl representing if each element X1 , ... , Xl has been covered at least n times • Constraints – ∑m i=1 ai ≤ k ( the maximum total number of overlays allowed in the final subset ) – −M ( 1− tj ) ≤ ∑m i=1 aici ( Xj ) − n+ ϵ ≤Mtj for each j ∈ { 1 , ... , l } • Objective to maximize – fn ( O ) = ∑l j=1 w ( Xj ) 1 { C ( Xj |O ) ≥n } = ∑l j=1 tjw ( Xj ) If we want to enforce non-redundancy constraints such that pairs of overlays that violate certain distance criteria are not both chosen , we can include additional constraints at + ar ≤ 1 for every pair of overlays ( At , Ar ) that we don ’ t want both to be included in the final subset . 4 THE MARGINALGREEDY ALGORITHM FOR MAXIMUM n-TIMES COVERAGE Although NTIMES-ILP can produce near-optimal solutions on problems with reasonable size , it may become intractable for problems where hundreds of thousands of elements and/or overlays are involved , such as certain variants of the vaccine design problem . Thus , we seek a polynomial time algorithm that provides good solutions to the maximum n-times coverage problem . A naive greedy solution is problematic when the n-times objective is directly approached . This is a consequence of potential early bad overlay choices by a greedy approach that can cause it to fail later to find overlays with sufficient n-times coverage . In addition , during greedy optimization available overlay choices may not provide differential marginal gain to avoid ties and random overlay selection . The MARGINALGREEDY algorithm is specifically designed to avoid early bad choices that will lead to failure by marginally approaching the n-times coverage objective . MARGINALGREEDY preserves marginal gains by employing look-ahead tie-breaking that assists in selecting overlays that benefit longer term objectives . Algorithm 1 MARGINALGREEDY algorithm ( for MIN-COST n-TIMES COVERAGE ) Input : Weights of the elements in X : w ( X1 ) , w ( X2 ) , . . . , w ( Xl ) , ground set of overlaysA where each overlay j in A covers Xi for cj ( Xi ) times , target minimum # times being covered ntarget , coverage cutoff for different n : τ1 , τ2 , . . . , τntarget , beam size b Initialize beam B0 ← { ∅ } , t = 0 for n = 1 , . . . , ntarget do Using set function fn ( S ) = ∑l i=1 w ( Xi ) 1 { ∑ j∈S cj ( Xi ) ≥n } as objective function . repeat Kt ← ∅ ▷ Kt is the set of candidate solutions for O ∈ Bt do ▷ Beam search for a ∈ A \O do Kt ← Kt ∪ { O ∪ { a } } ▷ Add the candidate solution { O ∪ { a } } to Kt Bt+1 ← { Top b candidate solutions in Kt as scored by fn ( S ) } t← t+ 1 mt ← median score of candidate solutions in the beam Bt until mt ≥ τn ( in ntarget-th cycle the termination condition is maxS∈Bt fn ( S ) ≥ τtarget ) ( for MAXIMUM n-TIMES COVERAGE with cardinality constraint k , there is additional termination condition t > k ) O∗ = argmaxS∈Bt fntarget ( S ) Output : The final selected subset of overlays O∗ MARGINALGREEDY optimizes n-times coverage with a sequence of greedy optimization cycles where the n-th cycle optimizes the coverage function fn ( S ) . We establish coverage starting at n = 1 and incrementally increase the n-times criteria to ntarget . Thus early overlay selections are guided by less stringent cycle-specific coverage objectives . A set of coverage cutoffs { τ1 , τ2 , . . . , τtarget } is used as the termination condition for each greedy optimization cycle , and when not specified , we assume τ1 = τ2 = · · · = τtarget by default . We use beam search to keep track of top b candidate solutions at each iteration . In general the larger the beam size , the closer the result is to the true optimal . However , there is a tradeoff between beam size and running time . In our results we choose the largest beam size that maintains a practical running time as we describe below . The full algorithm is given in Algorithm 1 . A similar algorithm can be used to solve the n-TIMES SET COVER problem in which τtarget = fntarget ( A ) . For the MAXIMUM n-TIMES COVERAGE problem with cardinality constraint k , the optimization terminates when t > k. When beam search is not used to reduce computation time ( b = 1 ) , we have extended the algorithm to break ties during the n-times coverage iteration by looking ahead to ( n+ 1 ) -times coverage . We call this extension look ahead tie-breaking . Another advantage of MARGINALGREEDY is that it is capable of guaranteeing high coverage for nt < ntarget by controlling the cycle-specific coverage cutoffs τt . This is desired in vaccine design where wider population coverage is also important and we want to make sure that almost 100 % of the population will be covered at least once .

Summary: This paper suggest the n-times coverage problem (a natural generalization of set-cover where items need to be covered n>1 times by the sets we choose) and motivates it with an application for the design of COVID vaccines. This application involves some ML pipeline, and this new combinatorial optimization problem helps with one of the steps.  The problem is shown to be not-submodular, and therefore the most standard methods don't quite work. Nonetheless, the authors present a greedy algorithm to get a reasonable approximation.  The paper evaluates the quality of their algorithm on artificial data and shows that it outperforms the more standard greedy method. Then they show the impact of this improvement for vaccine design.