Are LLM s Aware that Some Questions are not Open-ended? (2024)

Abstract: Large Language Models (LLMs) have shown the impressive capability of answering questions in a wide range of scenarios. However, when LLMs face different types of questions, it is worth exploring whether LLMs are aware that some questions have limited answers and need to respond more deterministically but some do not. We refer to this as question awareness of LLMs. The lack of question awareness in LLMs leads to two phenomena that LLMs are: (1) too casual to answer non-open-ended questions or (2) too boring to answer open-ended questions. In this paper, we first evaluate the question awareness in LLMs. The experimental results show that LLMs have the issues of lacking awareness of questions in certain domains, e.g. factual knowledge, resulting in hallucinations during the generation. To mitigate these, we propose a method called Question Awareness Temperature Sampling (QuATS). This method enhances the question awareness of LLMs by adaptively adjusting the output distributions based on question features. The automatic adjustment in QuATS eliminates the need for manual temperature tuning in text generation and consistently improves model performance in various benchmarks.

## introduction large language models (llms) @xcite @xcite @xcite have emerged as groundbreaking innovations in achieving a remarkable level of fluency and comprehension in question-answering using the human language @xcite @xcite . though llms can answer enormous questions with their knowledge bases, it is hard to tell if llms are aware of the difference between the questions they are answering. in other words, do llms understand that, open-ended questions encourage more casual and creative answers, but non-open-ended questions, e.g. problems about calculations and factual knowledge, need more deterministic answers? we refer to this as question awareness of llms that one knows which type of questions requires deterministic answers and which does not. it is significant to explore the question awareness of llms because it has a deep relationship to the model hallucinations that llms are prone to generate inaccurate content when they are not sure. in this paper, we explore whether llms have question awareness on different types of questions. because llms sample next tokens from output distributions, as shown in figure 1 , we examine the degree of the determinacy of llms from the steepness of the output distributions. a steeper output distribution means the model has confidence in selecting which token in the vocabulary to be the next token and a flat one means the model does not have a clear preference for the next token. therefore, the steepness of the output distributions reflects the question awareness by indicating determinacy about the generated answers. we utilize the kurtosis to measure the steepness of the distribution and investigate the question awareness by checking kurtosises of output distributions when llms are asked different types of questions. we evaluate llama 2 @xcite and falcon @xcite on different types of non-open-ended/open-ended questions for question awareness evaluation. experimental results show that llms have a certain degree of question awareness but lack the awareness in some scenarios, e.g., factual knowledge, thus easily giving more casual and hallucinated answers. as the steepness of output distributions reflects the question awareness, we utilize the temperature of the softmax function @xcite to adjust that city is located in the california united china singapore llm can we go somewhere warm else by special llm 16% 5.2% 1.7% 1.1% 7.6% 6.8% 5.4% 3.9% (non-open-ended) (open-ended) high kurtosis low kurtosis selection narrow wide figure 1: llms should choose to be deterministic to answer the question on the left but can have more choices to answer the one on the right. the steepness to externally change the question awareness. we evaluate the model with different temperatures to explore the influence of question awareness on model performance. the results show a relatively lower temperature (steeper distribution) makes the model more deterministic and have better performance on non-open-ended questions. inspired by the adjustment of temperature on the question awareness, we propose question awareness temperature sampling (quats), a method that enhances question awareness of llms by adjusting the output distributions through the temperature. when facing different questions, llms choose to be more deterministic or not using an adaptive temperature strategy of quats, avoiding the tedious process of temperature tuning in the text generation. to sum up, our contributions are stated as follows: • we evaluate the question awareness in llms and observe that llms have the fundamental ability to identify open-ended and non-openended questions but lack effective awareness in some domains, e.g., factual knowledge. ## question awareness evaluation in this section, we evaluate the question awareness of llms and how it influences the model performance on downstream tasks. ## question awareness temperature sampling based on the findings above, we propose the question awareness temperature sampling (quats) to enhance the question awareness of llms by an adaptive temperature strategy, which greatly improves the model performance. ## experiment in this section, we conduct experiments to showcase that quats can enhance question awareness using the adaptive temperature strategy and consistently improve the model performance. ## related work controlling text generation in llms has seen significant advancements in recent years. sampling methods play a crucial role in controlling the output quality and diversity of generated text. we introduce temperature sampling and corresponding advanced techniques in text generation. temperature sampling greedy sampling selects the token with the highest predicted proba-1 https://github.com/lm-sys/fastchat/tree/main/fastchat bility, resulting in deterministic and often repetitive text. random sampling selects tokens based on the probabilities, introducing randomness to alleviate the repetition. we can further adjust the temperature in the softmax function @xcite to control the token probabilities. temperature sampling can be seen as the trade-off between creativity and determinacy in the generated text. our quats adaptively controls the steepness of output distributions by adjusting the temperature. post-processing techniques because the tokens with higher probabilities are probably appropriate choices, we can choose only to select these tokens, avoiding sampling nonsensical tokens. top-k sampling @xcite narrows down the token selection to the top-k most probable tokens, increasing the likelihood of coherent text and balancing diversity and quality. similar to the motivation of top-k sampling, nucleus sampling @xcite , also known as top-p sampling, dynamically selects the top-p fraction of tokens with the highest probabilities. locally typical sampling @xcite posits the abstraction of natural language generation as a discrete stochastic process and samples tokens according to conditional entropy. entmax sampling (martins et al., 2020) leverages entmax transformation to train and sample from a natively sparse language model. keyword-based sampling (au2 and akhtar, 2023) uses knowledge distillation techniques to extract keywords and samples using these extracted keywords. it is noted that these postprocessing techniques are compatible with quats because quats only adjusts the output distribution, which can be further post-processed. ## conclusion in this paper, we highlight the question awareness of llms, which receives little attention from previous studies. while llms exhibit a fundamental awareness of open-ended and non-open-ended questions, they do falter in certain domains, often leading to casual or inaccurate responses. to bridge the gap, we introduce question awareness temperature sampling (quats), enabling llms to autonomously adapt their response determinacy based on question type. our experiments showcased the efficacy of quats, significantly enhancing llm performance across various benchmarks.