Challenging LLMs with Factual Knowledge: A Fine-grained Unanswerable Question Dataset
Authors: Chuanyuan Tan, Wenbiao Shao, Xiong Hao, Tong Zhu, Zhenhua Liu, Kai Shi, Wenliang Chen


Reviewer paper summaries: This paper suggests a novel dataset consisting of answerable/unanswerable questions that do/do not have factual answers.
The FUAQ (Fine-grained UnAnswerable Question) dataset aims to evaluate

1) the ability to identify and refuse unanswerable questions
2) the inner knowledge of factual information
3) the effect of CoT prompts for unanswerable questions

Compared to previous datasets, FUAQ is constructed from the knowledge base, provides more quesitons than previous datasets, and contains fine-grained annotations for each question.
This paper explained how authors developed the FUAQ dataset and the result of evaluations using the FUAQ dataset and six open-sourced LLMs.

To improve how large language models (LLMs) handle unanswerable questions and prevent misleading answers, researchers created a new dataset called FUAQ (Fine-grained Unanswerable Question dataset) built with factual knowledge. This dataset goes beyond existing ones by including reasons why questions are unanswerable, allowing for more in-depth evaluation. Experiments revealed that even with internal knowledge or external clues, LLMs often struggle with FUAQ, highlighting the need for further development in this area.
Summary of strengths: - providing unanswerable questions based on knowledge base; therefore, we can get factual knowledge triples for each question
- multiple tasks involving unanswerable questions, knowledge triples, and CoT effects
- careful evaluations with six open-sourced LLMs using the FUQA dataset

- The authors published the FUAQ dataset, the first dataset with detailed annotations to assess the non-response of LLMs. The questions of the data set were created from the Knowledge graph.
- The authors also define two new problems to evaluate LLM: determining whether the LLM has enough knowledge not to answer an unanswerable question and estimating the ability of the LLM to combine internal information and various external knowledge clues.
- Based on FUAQ, the authors evaluated several open-source LLMs to summarize some of the behavior of LLMs on FUAQ.
Summary of weaknesses: - consider changing target LLMs in the evaluation
  - only open-sourced 7b models seem too weak
  - if you can, consider to include commercial LLMs such as GPT and Claude series
  - the current evaluation does not explain whether current LLMs can handle unanswerable question problems or not, so I'm not convinced how this dataset/task is challenging and important for current LLMs
- consider giving concrete examples for each evaluation for a better understanding
  - ex. For 4.3, consider providing a sample of FK_R and FK_A (at least in the appendix)
  - ex2. For 4.4, consider showing how the answer of LLM changes for the same question in Base -> CoT -> CoT-Q -> CoT-QA
  - if you can, also consider conducting/showing a case study/error analysis for each task

- Why are those 3 question types in the FUAQ dataset? It's still quite simple and I think it will be easy to learn because the question template is also created by GPT3.5, not humans, so I think it doesn't reflect the true use-case of LLMs encountered today.
- I have seen some attack methods widely applied in Q&A systems, have you tried surveying and using them?
Comments, suggestions and typos: See `Weaknesses`.

Please address the weakness
Overall assessment scores: 3.5

3