Beam search is a go-to strategy for decoding neural sequence models . The algorithm can naturally be viewed as a subset optimization problem , albeit one where the corresponding set function does not reflect interactions between candidates . Empirically , this leads to sets often exhibiting high overlap , e.g. , strings may differ by only a single word . Yet in use-cases that call for multiple solutions , a diverse or representative set is often desired . To address this issue , we propose a reformulation of beam search , which we call determinantal beam search . Determinantal beam search has a natural relationship to determinantal point processes ( DPPs ) , models over sets that inherently encode intra-set interactions . By posing iterations in beam search as a series of subdeterminant maximization problems , we can turn the algorithm into a diverse subset selection process . In a case study , we use the string subsequence kernel to explicitly encourage n-gram coverage in text generated from a sequence model . We observe that our algorithm offers competitive performance against other diverse set generation strategies in the context of language generation , while providing a more general approach to optimizing for diversity . The decoding of neural sequence models is a fundamental component of many tasks in NLP . Yet , many proposed decoding methods aim to produce only a single solution ; further , decoding strategies that provide a set , such as beam search , admit high overlap between solutions . Such approaches fail to reflect that for many NLP tasks,1 there can be multiple correct solutions-or that we may desire a diverse set of solutions . As it stands , standard beam search chooses items based purely on individual scores , with no means for encoding interaction between candidates ; this is the limitation which we attempt to address in this work . We derive determinantal beam search , a novel generalization of beam search that casts subset selection as the subdeterminant optimization problem . Specifically , we formulate each iteration of beam search as a subdeterminant maximization problem parameterized by a positive semi-definite matrix that encodes interactions between the possible candidates ; standard beam search is recovered by a specific diagonal matrix . This framing creates a natural paradigm for taking the relationships between candidates during the decoding process , and can thus assign higher scores to diversified sets ; we show how this approach relates to k-determinantal point processes ( DPPs ) . Given the wealth of research on efficient kernel computation ( Rousu and Shawe-Taylor , 2005 ; Farhan et al . , 2017 ) and DPP inference strategies ( Li et al . , 2016 ; Han et al . , 2017 ; Chen et al . , 2018 ) , we find the impact on runtime to be quite reasonable in comparison to standard decoding techniques . In a case study on neural machine translation ( NMT ) , we demonstrate how to make use of the string subsequence kernel ( Lodhi et al . , 2002 ) to encode the notion of n-gram diversity in the language generation process , allowing us to derive an elegant diverse beam search . Under this scheme , we observe that determinantal beam search generates more diverse sets than standard beam search with minimal trade-off in terms of BLEU . We see improved performance over stochastic beam search ( SBS ; Kool et al . , 2019 ) , which is reported to encourage diversity , and a slight improvement over Vijayakumar et al . ( 2018 ) 's diverse beam search ( DBS ) while providing a more general approach to optimizing for intra-set diversity . We propose determinantal beam search ( DetBS ): a new way of framing beam search that allows us to optimize set generation for diversity and coverage rather than simply individual scores . Formally , we redefine beam search as an iterative subdeterminant maximization problems where we select the approximately maximizing set according to the PSD matrix parameterizing our score function . This gives us the ability to encode the notion of intra-set diversity into the beam search optimization problem . We discuss and experiment with efficient methods for inference and kernel computation that make DetBS an efficient decoding strategy in practice . We use DetBS in the context of language generation , where we explicitly encourage n-gram coverage through the string subsequence kernel . In our NMT experiments , we find DetBS generates much more diverse sets of strings than standard beam search and stochastic beam search with a small tradeoff in median BLEU . We observe competitive performance compared with diverse beam search .

Challenge: Beam search does not reflect the interactions between candidates leading to high overlaps while there are use-cases that call for multiple solutions.
Approach: They propose determinantal beam search which models over sets while encoding intra-set interactions to generate subsets with high diversity and coverage.
Outcome: The proposed algorithm performs competitively in BLEU to baseline methods including the stochastic beam search on the machine translation task while optimizing for diversity.