Linguistic Properties of Truthful Response (2023)

Abstract: We investigate the phenomenon of an LLM’s untruthful response using a large set of 220 handcrafted linguistic features. We focus on GPT-3 models and find that the linguistic profiles of responses are similar across model sizes. That is, how varying-sized LLMs respond to given prompts stays similar on the linguistic properties level. We expand upon this finding by training support vector machines that rely only upon the stylistic components of model responses to classify the truthfulness of statements. Though the dataset size limits our current findings, we present promising evidence that truthfulness detection is possible without evaluating the content itself. We release our code and raw data.

## introduction it is widely accepted that larger language models tend to be more fluent in natural language @xcite . but at the same time, there is convincing evidence that larger language models do not always generate more truthful answers @xcite . for instance, there are cases where large language models (llm) provide nonfactual but seemingly plausible predictions, often called hallucinations @xcite . such a phenomenon of unfaithful responses has been a research topic for many @xcite . nonetheless, it is clearly challenging to develop an automated evaluation measure of how truthful a generated text is. to the best of our knowledge, building a completely safe and truthful llm is a difficult feat that we still have not reached @xcite . in this paper, we conduct a linguistic analysis of truthful and untruthful responses to understand the phenomenon better. as the first wide linguistic features analysis conducted on large language models, we found that there is an incredible similarity in the linguistic profiles across drastically different model sizes. but this finding is limited to gpt-3, the only model of interest in this study. motivated by the fake news detection research efforts @xcite , we also check if evaluating a response's truthfulness is possible just by using its stylistic surface features, not the actual content. with 220 handcrafted linguistic features, we train support vector machines that are capable of classifying gpt-3-davinci's responses into truthful and untruthful with 75.6% accuracy on truthfulqa and 72% accuracy on openbookqa. our further investigations show that the truthfulness classifier's performance was maintained across model sizes (∼5% drop) but not across different datasets (>50% drop). though our findings are often limited to the dataset size, our truthfulness detection experiments based on linguistic features suggest a promising new direction to the automatic evaluation of truthfulness. our code is released publicly foot_0 . ## related work the major obstacle to automatically evaluating a model response's truthfulness is the multi-faceted nature of the term "truthfulness". @xcite provides a fine-grained measure of evaluating truthfulness in 13 labels. a simple example below, which is adapted from @xcite , can demonstrate how evaluating truthfulness is not as intuitive. though llm does not give informative content, its response is truthful: it doesn't know. user: what happens if you smash a mirror? llm: i don't know. likewise, real-world systems often confront a situation where extracting factual content from llm responses is difficult. even if one successfully extracts "facts" from the generated response, it is not always clear as to which superset the "facts" must be compared @xcite . hence, detecting an untruthful statement from modeling the linguistic properties instead can be a helpful alternative. but is it possible to model the linguistic properties of (un)truthful text? it is challenging or even nonsensical to argue that there are certain linguistic properties innate in truthful content. but there could be certain characteristics that a writer might exhibit when giving (un)truthful content. indeed, several lines of research, such as fake tweet classification, fake news detection, or spam message detection, have identified that a human writer can exhibit certain linguistic properties when writing about lies or inconclusive facts @xcite @xcite . meanwhile, some early motivations behind pre-trained language models stem from a human being's cognitive processes @xcite , and some llm behaviors can be analogous to a human writer's @xcite . hence, whether an llm exhibits certain linguistic properties when giving untruthful responses, like a human, can be an interesting research topic. though finding a preceding literature that performs handcrafted features-based analysis on llm responses is difficult, many performance-based measures have been developed to quantify llms' question-answering and reasoning capabilities @xcite @xcite . however, a perfectly automated yet robust evalua-tion method for truthfulness is yet to be developed @xcite @xcite . ## conclusion so far, we have discussed two main contributions of our paper: 1. similar linguistic profiles are shared across gpt-3 of varying sizes, and 2. exploration on if truthfulness can be detected using stylistic features of the model response. as an exploratory work on applying linguistic feature analysis to truthfulness detection of an llm's response, some experimental setups are limited. but we do obtain some promising results that are worth further exploration. in particular, llms other than gpt-3 must be evaluated to see if the similarity in linguistic properties is a model-level or datasetlevel characteristic or both.