text:   Distillation for Efficient Question Answering",5,"['🚨#EMNLP2021 Paper\n\nNew💡for Efficient QA➡️\nFilter questions that will not be answered by QA system\n\nInteresting 🔎: Transformer-based QA scores can be approximated only using question text via ""partial-input distillation""\n\n@AmazonScience @amoschitti1\n\n📰: <LINK> <LINK>', '@AmazonScience @amoschitti1 Practical QA systems operate at high Prec. (for customer req.) and end up not answering a large % of ques by failing system threshold on answer confidence score\n\nWe train filters to preemptively remove ques that are not answered by the system, saving (retrieval+answering) compute', '@AmazonScience @amoschitti1 We propose 2 loss objectives for learning 2 types of filters: regression &amp; classification head by distilling knowledge of QA system scores\n\nTraining does not require any human labels, only system generated scores\n\nDifferent from KD since teacher &amp; student use different inputs https://t.co/dqjrB8Hncz', '@AmazonScience @amoschitti1 Experimental results show: \n\n(i) Question filters can approximate Pr/Re of QA system very well\n\n(ii) Filters can provide large efficiency gains, with only a small drop in Recall (user-tunable tradeoff) https://t.co/WxV4o5ecr6', '@AmazonScience @amoschitti1 Code to be released soon at https://t.co/ucyWnAqajH\n\nPaper #AmazonScience: https://t.co/PnHOgoaL68\n\nFeel free to reach out to us in case of any questions 😀']",21,09,1263