How to iterate through all the items in this list and printing them as key value pairs? Give Python code
[{'entry_id': 'http://arxiv.org/abs/2305.11747v3',
  'updated': '2023-10-23 01:49:32+00:00',
  'published': '2023-05-19 15:36:27+00:00',
  'title': 'HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models',
  'authors': ['Junyi Li',
   'Xiaoxue Cheng',
   'Wayne Xin Zhao',
   'Jian-Yun Nie',
   'Ji-Rong Wen'],
  'summary': 'Large language models (LLMs), such as ChatGPT, are prone to generate\nhallucinations, i.e., content that conflicts with the source or cannot be\nverified by the factual knowledge. To understand what types of content and to\nwhich extent LLMs are apt to hallucinate, we introduce the Hallucination\nEvaluation benchmark for Large Language Models (HaluEval), a large collection\nof generated and human-annotated hallucinated samples for evaluating the\nperformance of LLMs in recognizing hallucination. To generate these samples, we\npropose a ChatGPT-based two-step framework, i.e., sampling-then-filtering.\nBesides, we also hire some human labelers to annotate the hallucinations in\nChatGPT responses. The empirical results suggest that ChatGPT is likely to\ngenerate hallucinated content in specific topics by fabricating unverifiable\ninformation (i.e., about $19.5\\%$ responses). Moreover, existing LLMs face\ngreat challenges in recognizing the hallucinations in texts. However, our\nexperiments also prove that providing external knowledge or adding reasoning\nsteps can help LLMs recognize hallucinations. Our benchmark can be accessed at\nhttps://github.com/RUCAIBox/HaluEval.',
  'comment': 'Accepted to EMNLP 2023 Main Conference (Long Paper)',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL'],
  'links': ['http://arxiv.org/abs/2305.11747v3',
   'http://arxiv.org/pdf/2305.11747v3'],
  'pdf_url': 'http://arxiv.org/pdf/2305.11747v3'},
 {'entry_id': 'http://arxiv.org/abs/2309.05922v1',
  'updated': '2023-09-12 02:34:06+00:00',
  'published': '2023-09-12 02:34:06+00:00',
  'title': 'A Survey of Hallucination in Large Foundation Models',
  'authors': ['Vipula Rawte', 'Amit Sheth', 'Amitava Das'],
  'summary': "Hallucination in a foundation model (FM) refers to the generation of content\nthat strays from factual reality or includes fabricated information. This\nsurvey paper provides an extensive overview of recent efforts that aim to\nidentify, elucidate, and tackle the problem of hallucination, with a particular\nfocus on ``Large'' Foundation Models (LFMs). The paper classifies various types\nof hallucination phenomena that are specific to LFMs and establishes evaluation\ncriteria for assessing the extent of hallucination. It also examines existing\nstrategies for mitigating hallucination in LFMs and discusses potential\ndirections for future research in this area. Essentially, the paper offers a\ncomprehensive examination of the challenges and solutions related to\nhallucination in LFMs.",
  'comment': None,
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.AI',
  'categories': ['cs.AI', 'cs.CL', 'cs.IR'],
  'links': ['http://arxiv.org/abs/2309.05922v1',
   'http://arxiv.org/pdf/2309.05922v1'],
  'pdf_url': 'http://arxiv.org/pdf/2309.05922v1'},
 {'entry_id': 'http://arxiv.org/abs/2308.15126v3',
  'updated': '2023-10-10 11:57:26+00:00',
  'published': '2023-08-29 08:51:24+00:00',
  'title': 'Evaluation and Analysis of Hallucination in Large Vision-Language Models',
  'authors': ['Junyang Wang',
   'Yiyang Zhou',
   'Guohai Xu',
   'Pengcheng Shi',
   'Chenlin Zhao',
   'Haiyang Xu',
   'Qinghao Ye',
   'Ming Yan',
   'Ji Zhang',
   'Jihua Zhu',
   'Jitao Sang',
   'Haoyu Tang'],
  'summary': "Large Vision-Language Models (LVLMs) have recently achieved remarkable\nsuccess. However, LVLMs are still plagued by the hallucination problem, which\nlimits the practicality in many scenarios. Hallucination refers to the\ninformation of LVLMs' responses that does not exist in the visual input, which\nposes potential risks of substantial consequences. There has been limited work\nstudying hallucination evaluation in LVLMs. In this paper, we propose\nHallucination Evaluation based on Large Language Models (HaELM), an LLM-based\nhallucination evaluation framework. HaELM achieves an approximate 95%\nperformance comparable to ChatGPT and has additional advantages including low\ncost, reproducibility, privacy preservation and local deployment. Leveraging\nthe HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we\nanalyze the factors contributing to hallucination in LVLMs and offer helpful\nsuggestions to mitigate the hallucination problem. Our training data and human\nannotation hallucination data will be made public soon.",
  'comment': '11 pages, 5 figures',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.LG',
  'categories': ['cs.LG', 'cs.AI', 'cs.CL', 'cs.CV'],
  'links': ['http://arxiv.org/abs/2308.15126v3',
   'http://arxiv.org/pdf/2308.15126v3'],
  'pdf_url': 'http://arxiv.org/pdf/2308.15126v3'},
 {'entry_id': 'http://arxiv.org/abs/2311.05232v1',
  'updated': '2023-11-09 09:25:37+00:00',
  'published': '2023-11-09 09:25:37+00:00',
  'title': 'A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions',
  'authors': ['Lei Huang',
   'Weijiang Yu',
   'Weitao Ma',
   'Weihong Zhong',
   'Zhangyin Feng',
   'Haotian Wang',
   'Qianglong Chen',
   'Weihua Peng',
   'Xiaocheng Feng',
   'Bing Qin',
   'Ting Liu'],
  'summary': 'The emergence of large language models (LLMs) has marked a significant\nbreakthrough in natural language processing (NLP), leading to remarkable\nadvancements in text understanding and generation. Nevertheless, alongside\nthese strides, LLMs exhibit a critical tendency to produce hallucinations,\nresulting in content that is inconsistent with real-world facts or user inputs.\nThis phenomenon poses substantial challenges to their practical deployment and\nraises concerns over the reliability of LLMs in real-world scenarios, which\nattracts increasing attention to detect and mitigate these hallucinations. In\nthis survey, we aim to provide a thorough and in-depth overview of recent\nadvances in the field of LLM hallucinations. We begin with an innovative\ntaxonomy of LLM hallucinations, then delve into the factors contributing to\nhallucinations. Subsequently, we present a comprehensive overview of\nhallucination detection methods and benchmarks. Additionally, representative\napproaches designed to mitigate hallucinations are introduced accordingly.\nFinally, we analyze the challenges that highlight the current limitations and\nformulate open questions, aiming to delineate pathways for future research on\nhallucinations in LLMs.',
  'comment': 'Work in progress; 49 pages',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL'],
  'links': ['http://arxiv.org/abs/2311.05232v1',
   'http://arxiv.org/pdf/2311.05232v1'],
  'pdf_url': 'http://arxiv.org/pdf/2311.05232v1'},
 {'entry_id': 'http://arxiv.org/abs/2310.03368v4',
  'updated': '2023-10-25 07:49:37+00:00',
  'published': '2023-10-05 07:57:09+00:00',
  'title': 'Evaluating Hallucinations in Chinese Large Language Models',
  'authors': ['Qinyuan Cheng',
   'Tianxiang Sun',
   'Wenwei Zhang',
   'Siyin Wang',
   'Xiangyang Liu',
   'Mozhi Zhang',
   'Junliang He',
   'Mianqiu Huang',
   'Zhangyue Yin',
   'Kai Chen',
   'Xipeng Qiu'],
  'summary': 'In this paper, we establish a benchmark named HalluQA (Chinese Hallucination\nQuestion-Answering) to measure the hallucination phenomenon in Chinese large\nlanguage models. HalluQA contains 450 meticulously designed adversarial\nquestions, spanning multiple domains, and takes into account Chinese historical\nculture, customs, and social phenomena. During the construction of HalluQA, we\nconsider two types of hallucinations: imitative falsehoods and factual errors,\nand we construct adversarial samples based on GLM-130B and ChatGPT. For\nevaluation, we design an automated evaluation method using GPT-4 to judge\nwhether a model output is hallucinated. We conduct extensive experiments on 24\nlarge language models, including ERNIE-Bot, Baichuan2, ChatGLM, Qwen, SparkDesk\nand etc. Out of the 24 models, 18 achieved non-hallucination rates lower than\n50%. This indicates that HalluQA is highly challenging. We analyze the primary\ntypes of hallucinations in different types of models and their causes.\nAdditionally, we discuss which types of hallucinations should be prioritized\nfor different types of models.',
  'comment': 'Work in progress',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL'],
  'links': ['http://arxiv.org/abs/2310.03368v4',
   'http://arxiv.org/pdf/2310.03368v4'],
  'pdf_url': 'http://arxiv.org/pdf/2310.03368v4'},
 {'entry_id': 'http://arxiv.org/abs/2210.07688v2',
  'updated': '2023-02-10 04:11:26+00:00',
  'published': '2022-10-14 10:27:22+00:00',
  'title': 'Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training',
  'authors': ['Wenliang Dai',
   'Zihan Liu',
   'Ziwei Ji',
   'Dan Su',
   'Pascale Fung'],
  'summary': 'Large-scale vision-language pre-trained (VLP) models are prone to hallucinate\nnon-existent visual objects when generating text based on visual information.\nIn this paper, we systematically study the object hallucination problem from\nthree aspects. First, we examine recent state-of-the-art VLP models, showing\nthat they still hallucinate frequently, and models achieving better scores on\nstandard metrics (e.g., CIDEr) could be more unfaithful. Second, we investigate\nhow different types of image encoding in VLP influence hallucination, including\nregion-based, grid-based, and patch-based. Surprisingly, we find that\npatch-based features perform the best and smaller patch resolution yields a\nnon-trivial reduction in object hallucination. Third, we decouple various VLP\nobjectives and demonstrate that token-level image-text alignment and controlled\ngeneration are crucial to reducing hallucination. Based on that, we propose a\nsimple yet effective VLP loss named ObjMLM to further mitigate object\nhallucination. Results show that it reduces object hallucination by up to 17.4%\nwhen tested on two benchmarks (COCO Caption for in-domain and NoCaps for\nout-of-domain evaluation).',
  'comment': 'Accepted at EACL 2023',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL', 'cs.CV'],
  'links': ['http://arxiv.org/abs/2210.07688v2',
   'http://arxiv.org/pdf/2210.07688v2'],
  'pdf_url': 'http://arxiv.org/pdf/2210.07688v2'},
 {'entry_id': 'http://arxiv.org/abs/2309.05217v1',
  'updated': '2023-09-11 03:35:00+00:00',
  'published': '2023-09-11 03:35:00+00:00',
  'title': 'Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis',
  'authors': ['Li Du',
   'Yequan Wang',
   'Xingrun Xing',
   'Yiqun Ya',
   'Xiang Li',
   'Xin Jiang',
   'Xuezhi Fang'],
  'summary': 'Although demonstrating superb performance on various NLP tasks, large\nlanguage models (LLMs) still suffer from the hallucination problem, which\nthreatens the reliability of LLMs. To measure the level of hallucination of\nLLMs, previous works first categorize the hallucination according to the\nphenomenon similarity, then quantify the proportion that model outputs contain\nhallucinatory contents. However, such hallucination rates could easily be\ndistorted by confounders. Moreover, such hallucination rates could not reflect\nthe reasons for the hallucination, as similar hallucinatory phenomena may\noriginate from different sources. To address these issues, we propose to\ncombine the hallucination level quantification and hallucination reason\ninvestigation through an association analysis, which builds the relationship\nbetween the hallucination rate of LLMs with a set of risk factors. In this way,\nwe are able to observe the hallucination level under each value of each risk\nfactor, examining the contribution and statistical significance of each risk\nfactor, meanwhile excluding the confounding effect of other factors.\nAdditionally, by recognizing the risk factors according to a taxonomy of model\ncapability, we reveal a set of potential deficiencies in commonsense\nmemorization, relational reasoning, and instruction following, which may\nfurther provide guidance for the pretraining and supervised fine-tuning process\nof LLMs to mitigate the hallucination.',
  'comment': None,
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.AI',
  'categories': ['cs.AI', 'cs.CL'],
  'links': ['http://arxiv.org/abs/2309.05217v1',
   'http://arxiv.org/pdf/2309.05217v1'],
  'pdf_url': 'http://arxiv.org/pdf/2309.05217v1'},
 {'entry_id': 'http://arxiv.org/abs/2311.16839v1',
  'updated': '2023-11-28 14:54:37+00:00',
  'published': '2023-11-28 14:54:37+00:00',
  'title': 'Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization',
  'authors': ['Zhiyuan Zhao',
   'Bin Wang',
   'Linke Ouyang',
   'Xiaoyi Dong',
   'Jiaqi Wang',
   'Conghui He'],
  'summary': 'Multimodal large language models have made significant advancements in recent\nyears, yet they still suffer from a common issue known as the "hallucination\nproblem" where the models generate textual descriptions that contain inaccurate\nor non-existent content from the image. To address this issue, this paper\nintroduces a novel strategy: Hallucination-Aware Direct Preference Optimization\n(HA-DPO). Our approach treats the hallucination problem as a unique preference\nselection issue, where the model is trained to favor the non-hallucinating\nresponse when presented with two responses of the same image (one accurate and\none hallucinating). This paper also presents an efficient process for\nconstructing hallucination sample pairs to ensure high-quality,\nstyle-consistent pairs for stable HA-DPO training. We applied this strategy to\ntwo mainstream multimodal models, and the results showed a significant\nreduction in the hallucination problem and an enhancement in the models\'\ngeneralization capabilities. With HA-DPO, the MiniGPT-4 model demonstrates\nsignificant advancements: POPE accuracy increases from 51.13% to 85.66% (34.5%\nabsolute improvement), and the MME score escalates from 968.58 to 1365.76 (41%\nrelative improvement). The code, models, and datasets will be made publicly\navailable.',
  'comment': 'Preprint',
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CV',
  'categories': ['cs.CV', 'cs.CL'],
  'links': ['http://arxiv.org/abs/2311.16839v1',
   'http://arxiv.org/pdf/2311.16839v1'],
  'pdf_url': 'http://arxiv.org/pdf/2311.16839v1'},
 {'entry_id': 'http://arxiv.org/abs/2311.09335v1',
  'updated': '2023-11-15 19:49:24+00:00',
  'published': '2023-11-15 19:49:24+00:00',
  'title': 'Lighter, yet More Faithful: Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization',
  'authors': ['George Chrysostomou',
   'Zhixue Zhao',
   'Miles Williams',
   'Nikolaos Aletras'],
  'summary': 'Despite their remarkable performance on abstractive summarization, large\nlanguage models (LLMs) face two significant challenges: their considerable size\nand tendency to hallucinate. Hallucinations are concerning because they erode\nthe reliability of LLMs and raise safety issues. Pruning is a technique that\nreduces model size by removing redundant weights to create sparse models that\nenable more efficient inference. Pruned models yield comparable performance to\ntheir counterpart full-sized models, making them ideal alternatives when\noperating on a limited budget. However, the effect that pruning has upon\nhallucinations in abstractive summarization with LLMs has yet to be explored.\nIn this paper, we provide an extensive empirical study on the hallucinations\nproduced by pruned models across three standard summarization tasks, two\npruning approaches, three instruction-tuned LLMs, and three hallucination\nevaluation metrics. Surprisingly, we find that pruned LLMs hallucinate less\ncompared to their full-sized counterparts. Our follow-up analysis suggests that\npruned models tend to depend more on the source input and less on their\nparametric knowledge from pre-training for generation. This greater dependency\non the source input leads to a higher lexical overlap between generated content\nand the source input, which can be a reason for the reduction in\nhallucinations.',
  'comment': None,
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL', 'cs.AI'],
  'links': ['http://arxiv.org/abs/2311.09335v1',
   'http://arxiv.org/pdf/2311.09335v1'],
  'pdf_url': 'http://arxiv.org/pdf/2311.09335v1'},
 {'entry_id': 'http://arxiv.org/abs/2305.18248v2',
  'updated': '2023-09-13 13:58:36+00:00',
  'published': '2023-05-29 17:12:03+00:00',
  'title': "Do Language Models Know When They're Hallucinating References?",
  'authors': ['Ayush Agrawal',
   'Mirac Suzgun',
   'Lester Mackey',
   'Adam Tauman Kalai'],
  'summary': 'State-of-the-art language models (LMs) are famous for "hallucinating"\nreferences. These fabricated article and book titles lead to harms, obstacles\nto their use, and public backlash. While other types of LM hallucinations are\nalso important, we propose hallucinated references as the "drosophila" of\nresearch on hallucination in large language models (LLMs), as they are\nparticularly easy to study. We show that simple search engine queries reliably\nidentify such hallucinations, which facilitates evaluation. To begin to dissect\nthe nature of hallucinated LM references, we attempt to classify them using\nblack-box queries to the same LM, without consulting any external resources.\nConsistency checks done with "direct" queries about whether the generated\nreference title is real (inspired by Kadavath et al. 2022, Lin et al. 2022,\nManakul et al. 2023) are compared to consistency checks with "indirect" queries\nwhich ask for ancillary details such as the authors of the work. These\nconsistency checks are found to be partially reliable indicators of whether or\nnot the reference is a hallucination. In particular, we find that LMs often\nhallucinate differing authors of hallucinated references when queried in\nindependent sessions, while consistently identify authors of real references.\nThis suggests that the hallucination may be more a generation issue than\ninherent to current training techniques or representation.',
  'comment': None,
  'journal_ref': None,
  'doi': None,
  'primary_category': 'cs.CL',
  'categories': ['cs.CL', 'cs.AI'],
  'links': ['http://arxiv.org/abs/2305.18248v2',
   'http://arxiv.org/pdf/2305.18248v2'],
  'pdf_url': 'http://arxiv.org/pdf/2305.18248v2'}]