Generating Faithful and Salient Text from Multimodal Data (2024)

Abstract: While large multimodal models (LMMs) have obtained strong performance on many multimodal tasks, they may still hallucinate while generating text. Their performance on detecting salient features from visual data is also unclear. In this paper, we develop a framework to generate faithful and salient text from mixed-modal data, which includes images and structured data ( represented in knowledge graphs or tables). Specifically, we train a vision critic model to identify hallucinated and non-salient features from the image modality. The critic model also generates a list of salient image features. This information is used in the post editing step to improve the generation quality. Experiments on two datasets show that our framework improves LMMs’ generation quality on both faithfulness and saliency, outperforming recent techniques aimed at reducing hallucination. The dataset and code are available at https://github.com/TahsinaHashem/FaithD2T.

## introduction in many real-world scenarios, data is presented in mixed modalities, in which complementary information is contained. examples include product brochures, scientific/technical publications, and news articles. structured data-to-text generation is the task of generating natural language sentences from the data in a structured format, such as tables, knowledge graphs, or databases. researchers have proposed several models to make this structured information more accessible to humans, aiming to generate fluent, informative, and faithful text descriptions or summaries from the structured data. this task has a wide range of applications across different industries and domains i.e. house advertising, financial reporting, automated journalism, medical reporting, e-commerce product descriptions, generating biographies, etc. significant progress has been made in data-totext generation tasks. several well-known models have utilized pre-trained language models (plms) such as bart @xcite , t5 @xcite or gpt @xcite with appropriate structure-aware frameworks @xcite @xcite to generate text descriptions from the structured data. however, the importance of multimodal input with structured data was not extensively addressed. the problem was explored on a small scale by @xcite . their proposed model aimed to generate a one-line summary sentence from a given table and an associated image. they showed that integrating vision data with structured data would lead to more informative and relevant text. however, the research did not consider their generation task's saliency and faithfulness. recently, several open-sourced large multimodal models (lmms) @xcite @xcite show promising performance in a variety of multimodal tasks @xcite @xcite @xcite i.e. image captioning, visual question answering, multimodal conversation, crossmodal retrieval, etc. in this research, we exploit these powerful lmms to generate text from structured data (knowledge graph and table) with images. we have examined the performance of two prominent lmms, llava-1.5 @xcite and minigpt4 @xcite on two advertising multimodal (structured data with images) datasets i.e. the real-estate house dataset @xcite and the e-commerce product dataset @xcite . the models generate good-quality advertising text but have two types of limitations: (1) generate some hallucinated information that is not aligned with the vision input; (2) unable to detect salient image features. these limitations hamper the faithfulness and saliency of the generated text. ## method to generate high-quality text, finetuning lmm may not be feasible for proprietary models, and it may not be practical due to the prohibitively high resource and data requirements. thus, we propose a cost-effective post-hoc detection and correction approach that trains a small vlm to act as a vision critic model that identifies errors in the lmmgenerated text. with the feedback provided by the critic model, a capable llm (such as gpt-3.5) is then employed to update the text using this feedback. figure 2 depicts the overall architecture of our proposed method. given the text generated from the mixed-modal data by the lmm, we first prompt a capable llm (such as gpt-3.5) to extract the list of image features from the text by filtering out features from the structured data. with this list and the images as input, we employ our trained vision critic model to identify hallucinated image features and