Faster Matchings via Learned Duals

1 Introduction . Classical algorithm analysis considers worst case performance of algorithms , capturing running times , approximation and competitive ratios , space complexities , and other notions of performance . Recently there has been a renewed interest in finding formal ways to go beyond worst case analysis [ 41 ] , to better understand performance of algorithms observed in practice , and develop new methods tailored to typical inputs observed . An emerging line of research dovetails this with progress in machine learning , and asks how algorithms can be augmented with machine-learned predictors to circumvent worst case lower bounds when the predictions are good , and approximately match them otherwise ( see Mitzenmacher and Vassilvitskii [ 36 ] for a survey ) . Naturally , a rich area of applications of this paradigm has been in online algorithms , where the additional information revealed by the predictions reduces the uncertainty about the future and can lead to better choices , and thus better competitive ratios . For instance , see the work by Lykouris and Vassilvitskii [ 33 ] , Rohatgi [ 40 ] , Jiang et al . [ 29 ] on caching ; Antoniadis et al . [ 3 ] , Dütting et al . [ 19 ] on the classic secretary problem ; Purohit et al . [ 39 ] , Lattanzi et al . [ 32 ] on scheduling ; Purohit et al . [ 39 ] , Anand et al . [ 2 ] on ski rental ; and Bamas et al . [ 8 ] on set cover . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . However , the power of predictions is not limited to improving online algorithms . Indeed , the aim of the empirical paper that jump-started this area by Kraska et al . [ 31 ] was to improve running times for basic indexing problems . The main goal and contribution of this work is to show that at least in one important setting ( weighted bipartite matching ) , we can give formal justification for using machine learned predictions to improve running times : there are predictions which can provably be learned , and if these predictions are “ good ” then we have running times that outperform standard methods both in theory and empirically . How can predictions help with running time ? One intuitive approach , which has been used extensively in practice , is through the use of “ warm-start ” heuristics [ 46 , 23 , 22 , 37 ] , where instead of starting with a blank slate , the algorithm begins with some starting state ( which we call a warm-start “ solution ” or “ seed ” ) which hopefully allows for faster completion . While it is a common technique , there is a dearth of analysis understanding what constitutes a good warm-start , when such initializations are helpful , and how they can best be leveraged . Thus we have a natural goal : put warm-start heuristics on firm theoretical footing by interpreting the warm-start solution as learned predictions . In this set up we are given a number of instances of the problem ( the training set ) , and we can use them to compute a warm-start solution that will ( hopefully ) allow us to more quickly compute the optimal solution on future , test-time , instances . There are three challenges that we must address : ( i ) Feasibility . The learned prediction ( warm-start solution ) might not even be feasible for the specific instance we care about ! For example , the learned solution may be matching an edge that does not exist in the graph at testing time . ( ii ) Optimization . If the warm-start solution is feasible and near-optimal then we want the algorithm to take advantage of it . In other words , we would like our running time to be a function of the quality of the learned solution . ( iii ) Learnability . It is easy to design predictions that are enormously helpful but which can not actually be learned ( e.g. , the “ prediction ” is the optimal solution ) . We need to ensure that a typical solution learned from a few instances of the problem generalizes well to new examples , and thus offers potential speedups . If we can overcome these three challenges , we will have an end-to-end framework for speeding up algorithms via learned predictions : use the solution to challenge ( iii ) to learn the predictions from historical data , use the solution to challenge ( i ) to quickly turn the prediction into something feasible for the particular problem instance while preserving near-optimality , and then use this as a warm-start seed in the solution to challenge ( ii ) . 1.1 Our Contributions . We focus on one of the fundamental primitives of combinatorial optimization : computing bipartite matchings . For the bipartite minimum-weight perfect matching ( MWPM ) problem , as well as its extension to b-matching , we show that the above three challenges can be solved . A key conceptual question is finding a specification of the seed , and an algorithm to use it that satisfies the desiderata above . We have discussed warm-start “ solutions ” , so it is tempting to think that a good seed is a partial solution : a set of matched edges that can then be expanded to a optimal matching . After all , this is the structure we maintain in most classical matching algorithms . Moreover , any such solution is feasible ( one can simply set non-existing edges to have very high weight ) , eschewing the need for the feasibility step . At the same time , as has been observed previously in the context of online matchings [ 13 , 45 ] , this primal solution is brittle , and a minor modification in the instance ( e.g . an addition of a single edge ) can completely change the set of optimal edges . Instead , following the work of [ 13 , 45 ] , we look at the dual problem ; that is , the dual to the natural linear program . We quantify the “ quality ” of a prediction ŷ by its ` 1-distance from the true optimal dual y∗ , i.e. , by ‖ŷ − y∗‖1 . The smaller quantities correspond to better predictions . Since the dual is a packing problem we must contend with feasibility : we give a simple linear time algorithm that converts the prediction ŷ into a feasible dual while increasing the ` 1 distance by a factor of at most 3 . Next , we run the Hungarian method starting with the resulting feasible dual . Here , we show that the running time is in proportional to the ` 1 distance of the feasible dual to the optimal dual ( Theorem 5 ) . Finally , we show via a pseudo-dimension argument that not many samples are needed before the empirically optimal seed is a good approximation of the true optimum ( Theorem 6 ) , and that this empirical optimum can be computed efficiently . For the learning argument , we assume that matching instances are drawn from a fixed but unknown distribution D. Putting it all together gives us our main result . Theorem 1 ( Informal ) . There are three algorithms ( feasibility , optimization , learning ) with the following guarantees . • Given a ( possibly infeasible ) dual ŷ from the learning algorithm , there exists an O ( m+ n ) time algorithm that takes a problem instance c , and outputs a feasible dual ŷ′ ( c ) such that ‖ŷ′ ( c ) − y∗ ( c ) ‖1 ≤ 3‖ŷ − y∗ ( c ) ‖1 . • The optimization algorithm takes as input feasible dual ŷ′ ( c ) and outputs a minimum weight perfect matching , and runs in time Õ ( m √ n ·min { ‖ŷ′ ( c ) − y∗ ( c ) ‖1 , √ n } ) . • After Õ ( C2n3 ) samples from an unknown distribution D over problem instances , the learning algorithm produces duals ŷ so that Ec∼D [ ‖ŷ−y∗ ( c ) ‖1 ] is approximately minimum among all possible choices of ŷ , where C is the maximum edge cost and y∗ ( c ) is an optimal dual for instance c. Combining these gives a single algorithm that , with access to Õ ( C2n3 ) problem instance samples from D , has expected running time on future instances from D of only Õ ( m √ nmin { α , √ n } ) , where α = miny Ec∼D [ ‖y − y∗ ( c ) ‖1 ] . We emphasize that the Hungarian method with Õ ( mn ) running time is the standard algorithm in practice . Although there are other theoretically faster exact algorithms for bipartite minimumweight perfect matching [ 38 , 21 , 20 , 18 ] that run in O ( m √ n log ( nC ) ) , they are relatively complex ( using various scaling techniques ) . Very recent breakthroughs give algorithms of run time Õ ( ( m+ n1.5 ) log2 ( C ) ) for the minimum-weight perfect matching problem and several interesting extensions [ 43 , 44 ] . However , the algorithms are highly complicated and their practical performance is yet to be demonstrated . In fact , we could not find any implementation of the above algorithms , except for the Hungarian method , of which multiple implementations are readily available . Note that our result shows that we can speed up the Hungarian method as long as the ` 1-norm error of the learned dual , i.e. , ‖ŷ − y∗ ( c ) ‖1 is o ( √ n ) . Further , as the projection step that converts the learned dual into a feasible dual takes only linear time , the overhead of our method is essentially negligible . Therefore , even if the prediction is of poor quality , our method has worst-case running time that is never worse than that of the Hungarian algorithm . Even our learning algorithm is simple , consisting of a straightforward empirical risk minimization algorithm ( the analysis is more complex and involves bounding the “ pseudo-dimension ” of the loss functions ) . We validate our theoretical results via experiments . For each dataset we first feed a small number of samples ( fewer than our theoretical bounds ) to our learning algorithm . We then compare the running time of our algorithm to that of the classical Hungarian algorithm on new instances . Details of these experiments can be found in Section 4 . At a high level they show that our algorithm is significantly faster in practice . Further , our experiment shows only very few samples are needed to achieve a notable speed-up . This confirms the power of our approach , giving a theoretically rigorous yet also practical method for warm-start primal-dual algorithms . 1.2 Related Work . Matchings and b-Matchings : Bipartite matchings are one of the most well studied problems in combinatorial optimization , with a long history of algorithmic improvements . We refer the interested reader to Duan and Pettie [ 17 ] for an overview . We highlight some particular results here . If edges have no weights and thus the goal is to find the maximum cardinality matching ( see Section 2 for a formal definition ) , the fastest running time had long been O ( m √ n ) [ 14 , 30 , 27 ] until the recent breakthrough with Õ ( m+n1.5 ) running time was discovered [ 43 ] . We are interested in the weighted versions of these problems and when all edge weights are integral . Let C be the maximum edge weight , n be the number of vertices , and m the number of edges . For finding exact solutions to the minimum weight perfect matching problem , the scaling technique leads to a running time of O ( m √ n log ( C ) ) [ 38 , 21 , 20 , 18 ] . Large scale bipartite matchings have been studied extensively in the online setting , as they represent the basic problem in ad allocations [ 34 ] . While the ad allocation is inherently online , most of the methods precompute a dual based solution based on a sample of the input [ 13 , 45 ] , and then argue that this solution is approximately optimal on the full instance . In contrast , we strive to compute the exactly optimal solution , but use previous instances to improve the running time of the approach . Algorithms with Predictions : Kraska et al . [ 31 ] showed how to use machine learned predictions to improve heavily optimized indexing algorithms . The original paper was purely empirical and came with no rigorous guarantees ; recently there has been a flurry of work putting such approaches on a strong theoretical foundation , evaluating the benefit of augmenting classical algorithms with machine learned predictions , see [ 36 ] for a survey . Online and streaming algorithms in particular have seen significant successes , as predictions reveal information about the future and can help guide the algorithms ’ choices . This has led to the design of new methods for caching [ 33 , 40 , 29 ] , scheduling [ 39 , 32 ] , frequency counting [ 12 , 28 , 1 ] , and membership testing [ 35 , 42 ] that can break through worst-case lower bounds when the predictions are of sufficiently high quality . Most of the above work abstracts the predictions as access to an error-prone oracle and asks how to best use predictions : getting performance gains when the predictions are good , but limiting the losses when they are not . A related emergent area is that of data driven algorithm design [ 25 , 7 , 5 , 4 , 6 , 11 ] . Here , the objective is to “ learn ” a good algorithm for a particular family of inputs . The goal is not typically to tie the performance of the algorithm to the quality of the prediction , but rather to show that the prediction makes sense ; that is only a small number of problem samples are needed in order to ensure the learned algorithm generalizes to new data points .

Summary: The paper proposed a learning-augmented algorithm that receives a learned dual solution and uses it as a warm-start to improve upon the running time of the Hungarian method for weighted bipartite matching. The paper provides a neat 3-stage approach for obtaining “faster” algorithms for matching problem: 1) Learn a helpful dual solution for a given instance, 2) Convert a given predicted dual solution to a feasible dual solution, and 3) Lastly, get a near-optimal solution starting from the feasible “warm-start” solution. The paper provides an end-to-end algorithm for this problem using the described approach.