Large Language Models Can Learn Representation in Natural Language (2024)

Abstract: One major challenge for Large Language Models (LLMs) is completing complex tasks involving multiple entities, such as tool APIs. To tackle this, one approach is to retrieve relevant entities to enhance LLMs in task completion. A crucial issue here is obtaining accurate natural language representations for each entity to aid in retriever precision. In this paper, we propose the Natural Language Representation Optimization Problem, which aims to refine entity descriptions for improved retrieval and LLM utilization. We introduce the Learning to Represent with Natural Language method, which utilizes LLMs to optimize entity representations consisting of text patterns based on environmental feedback. We iteratively prompt LLMs to enhance or adjust patterns based on entity samples and evaluate their effectiveness through environmental feedback. Our method successfully learns human-understandable representations for classification tasks (e.g., instructions and documents) and API call tasks (e.g., APIbench and Virtual Home), significantly improving GPT-4’s task performance.

## introduction large language models (llms), such as gpt-4 (openai, 2023), llama-2 @xcite , and gemini @xcite , successfully achieved strong zero-shot/few-shot performance on various natural language tasks @xcite @xcite @xcite . to complete more challenging tasks that requires operating multiple tools @xcite @xcite , llms need to understand and use these new entities (e.g., classes and tools). to address these tasks, a straightforward way is to fine-tune llms on the training corpus @xcite , which needs many gpus. @xcite retrieves relevant entities from all en-task description: given a user instruction, the llm calls api from the huggingface api set to finish the instruction. retriever: parametric retriever and non-parametric retriever (e.g., bm25). entities (apis) and their plain natural language representation: facebookai /roberta-large (natural language processing feature extraction); facebook/dino-vitb16 (computer vision image classification); microsoft/codebert-base (multimodal feature extraction); ... user instruction: write an api implementation that takes customer reviews as input and extracts features to analyze customer sentiment. label: apis of the natural language processing feature extraction domain (e.g., facebookai /roberta-large). the natural language representation of apis of the natural language processing feature extraction domain learned by our method: pattern 1: language-specific processing -adapting feature extraction techniques to handle text in a specific language or multiple languages (e.g., 'extract some features from a korean text'). pattern 2: text analysis techniques this pattern includes the methods or techniques used for feature extraction and text analysis (e.g., 'create contextual embeddings for text'). pattern 3: this pattern is about extracting features from text to determine sentiment or emotion, often used in analyzing customer reviews (e.g., 'provide the overall sentiment'). pattern 4: feature extraction for classification and categorization this pattern involves the extraction of features from text to be used in classification models, such as support vector machines (svms) or categorization based on similarity (e.g., 'build a classifier to identify the most suitable applicants'). to address the poor entity representation, we propose the natural language representation optimization problem, which seeks the best representations of entities for complex task completion. cur-rently, users must experiment with various representations to find the most suitable, lacking knowledge of compatibility with specific retrievers and llms. to streamline this process, we introduce the learning to represent with natural language method, leveraging llms to automatically optimize representations. our method aims to learn entity representations comprising multiple patterns, each describing a unique property. it operates in two main stages: (1) multi-step iterations to refine entity representations. (2) within each iteration, llms are prompted to add new patterns and edit existing ones based on entity samples, evaluating effectiveness through feedback. we demonstrate our method's learned representation alongside typical elements of the api call task in ## related work large language models and prompt optimization (e.g., gpt3 @xcite , gpt-4 (ope-nai, 2023), gemini @xcite , and llama @xcite ) can solve various basic natural language tasks @xcite , write mathematical proof draft @xcite , and pass human standard examinations @xcite . furthermore, with the help of tools (e.g., apis @xcite and search engine @xcite ), llms can complete ppt tasks @xcite ) and control operating system @xcite . prompting optimization methods further improve the llm's performance by optimizing the input prompt based on feedback. ape @xcite and instruction induction @xcite update task instruction by selecting the best instruction candidate based on the validation performance. learning to plan @xcite optimizes the task plan consisting of step-by-step solutions based on the error made by llms. evoprompt @xcite iteratively optimizes the prompts based on the evolutionary algorithms. representation learning in pretrained language models the recent mainstream paradigm for natural language representation learning is to first pre-train the language model on a large-scale corpus with self-supervised losses like auto-regressive loss and masked language modeling loss @xcite . analysis experiments show that the pretrained models (e.g., gpt @xcite , roberta @xcite , and t5 @xcite ) can learn rich linguistic knowledge @xcite @xcite and world knowledge @xcite @xcite during the pre-training process. however, learning natural language representations of entities of the task has not been investigated. ## natural language representation optimization using llms we start with a task defined by a dataset d @xmath0, a), where q represents the input query and a the corresponding answer containing entities e (e.g., a class or api sequence). typically, to tackle this task, we employ a retriever r to select an entity e from a set s that relates to the query q. this selection is based on ranking the similarity between the query and the natural language descriptions p e of entities in s. this process can be formalized as ê = r(q, p ), where p comprises the natural language representations of all entities, and ê is the retrieved entity. subsequently, this approach enriches the language model (lm) m with the retrieved ê to generate an answer m (ê) for completing q. the objective of optimizing natural language representations is to find a set p such that r can accurately retrieve relevant entities based on p , and m (ê) = a. formally, we formulate this task as an optimization problem aiming to maximize the expectation of a per-sample score f (m (ê), q, a) = f (m (r(q, p )), q, a) over possible (q, a). in this section, we present our method, "learning to represent with natural language," for automatically optimizing the natural language representa-tion p . our method features two key aspects: (1) each entity's representation is managed as a list of patterns describing its properties, which are updated via add and edit operations. (2) inspired by ape @xcite , our approach employs an iteration-based optimization strategy to discover the optimal representation. each iteration involves the following three steps. ## conclusion this paper focuses on challenging tasks requiring processing multiple entities for completion. we propose the natural language representation optimization problem to find the optimal entity representation for improved retrieval and llm utilization. we introduce the learning to represent with natural language method, which automatically optimizes representation based on feedback. experiments demonstrate our method boosts the llm's task performance significantly, along with highquality entity natural language representation.