text: 413,100,1438215447783030786,346719335,Keyon Vafa,"New paper: <LINK> Consider a sequence generated by a language model. Which words were most important for generating each word? We propose greedy rationalization: greedily finding the smallest subset of words that would make the same prediction as the full text. <LINK> Consider a sequence generated by GPT-2: ""The court struck down the law because it was unconstitutional"" Which words were most important for predicting ""unconstitutional""? The greedy algorithm starts with an empty set and adds words until ""unconstitutional"" is the top prediction <LINK> How do we evaluate sequential rationales?  There are some datasets with annotated rationales for classification, but these don't extend to sequence models. So we collected our own sequential rationale dataset based on Lambada. <LINK> Paper: <LINK> Github: <LINK> Demo: <LINK> With: @yuntiandeng, @blei_lab, @srush_nlp",https://arxiv.org/abs/2109.06387,"Sequence models are a critical component of modern NLP systems, but their predictions are difficult to explain. We consider model explanations though rationales, subsets of context that can explain individual model predictions. We find sequential rationales by solving a combinatorial optimization: the best rationale is the smallest subset of input tokens that would predict the same output as the full sequence. Enumerating all subsets is intractable, so we propose an efficient greedy algorithm to approximate this objective. The algorithm, which is called greedy rationalization, applies to any model. For this approach to be effective, the model should form compatible conditional distributions when making predictions on incomplete subsets of the context. This condition can be enforced with a short fine-tuning step. We study greedy rationalization on language modeling and machine translation. Compared to existing baselines, greedy rationalization is best at optimizing the combinatorial objective and provides the most faithful rationales. On a new dataset of annotated sequential rationales, greedy rationales are most similar to human rationales. ",Rationales for Sequential Predictions,4,"['New paper: <LINK>\n\nConsider a sequence generated by a language model. Which words were most important for generating each word?\n\nWe propose greedy rationalization: greedily finding the smallest subset of words that would make the same prediction as the full text. <LINK>', 'Consider a sequence generated by GPT-2: ""The court struck down the law because it was unconstitutional""\n\nWhich words were most important for predicting ""unconstitutional""?\n\nThe greedy algorithm starts with an empty set and adds words until ""unconstitutional"" is the top prediction https://t.co/a9NXHFTjuR', ""How do we evaluate sequential rationales? \n\nThere are some datasets with annotated rationales for classification, but these don't extend to sequence models.\n\nSo we collected our own sequential rationale dataset based on Lambada. https://t.co/liPVCfNoLM"", 'Paper: https://t.co/8CjhtqB1j8\nGithub: https://t.co/4NzpZ255bg\nDemo: https://t.co/C9dmp255VV\n\nWith: @yuntiandeng, @blei_lab, @srush_nlp']",21,09,872