Selective Perception: Learning Concise State Descriptions for Language Model Actors (2024)

Abstract: The latest large language models (LMs) support increasingly longer contexts. While this trend permits using substantial amounts of text with SOTA LMs, requiring these large LMs to process potentially redundant or irrelevant data needlessly increases inference time and cost. To remedy this problem, we propose BLINDER, a method that leverages a small finetuned LM to sample the minimal set of input features that maximizes the performance of a downstream LM. BLINDER trains an LM with a value head to estimate the likelihood of optimal outputs from a downstream LM given an input. We evaluate BLINDER on embodied decision making tasks with notoriously verbose state descriptions: NetHack and robot planning. BLINDER reduces the length of LM actor input by 87% and 99% while improving task success rates by 158% and 54% on NetHack and robot planning respectively which represents substantial inference cost savings while actually increasing performance.

## introduction large language models (lms) continue to scale in both number of parameters and supported context length. this trend has given rise to lms that are able to solve complex problems across many large documents. however, this comes at a price (often literally) as context length contributes directly to the time and compute cost of lm inference. while previous work has optimized fewshot examples @xcite or prompt instructions @xcite , the remaining input is left unmodified. also, previous work has focused on improving performance rather than conciseness. the goal of this work is to develop a method for cheaply reducing input length while maintaining or improving downstream task performance. we choose to explore learning concise inputs in the context of embodied decision making which exacerbates the problem of input length with verbose scene information @xcite . embodied decision making tasks evaluate lms as actors by generating actions given a task and state description. while an exhaustive list of state features is often used as a description for an lm actor @xcite , such inputs can contain many features that are irrelevant to or distract from the current task. lms are adept summarizers @xcite , and an lm summarizer could be used to summarize the input to a much larger lm for a difficult downstream task. however, this comes with an obvious trade off as better summaries will come from larger lms that do not satisfy our primary objective: reducing overall compute requirements. lm actor blinder task demonstrations full state description selected state description target action (a*) p(a*|x,τ) reward task description ## blinder we explore learning to select a concise state description from a set of state features given a task. ## conclusion in this work, we explore using small finetuned lms to select concise state descriptions that reduce compute and improve performance. rather than restrict prompt tuning to task examples or instructions as in previous work, we propose modifying the entire input space. our results indicate that small finetuned lms are well equipped to process entire inputs for larger more powerful lms to use on downstream tasks. zeroshot summarization from large lms is another possible approach to decreasing the length of input for a downstream lm. zeroshot summaries have the benefit of not requiring a user to finetune a model, but they do not perform as consistently and generally require larger models than their finetuned counterparts such as blinder. we find that blinder makes a significant difference in both downstream performance and overall inference costs. we hope to encourage continued research into learning more efficient lm contexts through methods such as pruning, summarization, and retrieval.