Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries (2025)

Abstract: As LLMs grow more powerful, their most profound achievement may be recognising when to say “I don’t know”. Existing studies on LLM self-knowledge have been largely constrained by human-defined notions of feasibility, often neglecting the reasons behind unanswerability by LLMs and failing to study deficient types of self-knowledge. This study aims to obtain intrinsic insights into different types of LLM self-knowledge with a novel methodology: allowing them the flexibility to set their own feasibility boundaries and then analysing the consistency of these limits. We find that even frontier models like GPT-4o and Mistral Large are not sure of their own capabilities more than 80% of the time, highlighting a significant lack of trustworthiness in responses. Our analysis of confidence balance in LLMs indicates that models swing between overconfidence and conservatism in feasibility boundaries depending on task categories and that the most significant self-knowledge weaknesses lie in temporal awareness and contextual understanding. These difficulties in contextual comprehension additionally lead models to question their operational boundaries, resulting in considerable confusion within the self-knowledge of LLMs. We make our code and results available publicly.

## introduction the hallmark of a truly intelligent system lies not in the breadth of its knowledge, but in the clarity with which it demarcates the boundaries of known and unknown. while we continue to broaden llms' access to data and find new application areas @xcite @xcite , it is crucial to study how this affects their perception of self-knowledge. to achieve a state of true reliability and trustworthiness, an llm must show its 1 https://github.com/knowledge-verse-ai/ llm-self_knowledge_eval figure 1 : overview of our methodology depicting key steps ability to confidently, consistently and accurately recognise the boundary beyond which it does not know. there has been considerable research in recent times analysing the current status of llms' awareness about their feasibility boundaries, referred to as self-knowledge @xcite . self-knowledge for llms, especially when utilised in critical fields such as healthcare, finance, and scientific research is of paramount importance, where overestimating competence can cause significant repercussions and losses. most existing work focuses on assessing selfknowledge by analysing responses to unanswerable questions @xcite , or quantifying uncertainty in outputs through logits output by the model @xcite @xcite . while such methods are successful in identifying specific knowledge gaps, they lack generalisation since they are restricted to analysis of the fixed, predetermined dataset used. moreover, almost all approaches rely solely on classification-based metrics by measuring self-knowledge through answerable or unanswerable labels, failing to take into ac-count llms' perception of self-knowledge boundaries when prompted to generate tasks that lie beyond these limits. consequently, to gain more universal and essential insights into llms' self-knowledge, we shift our focus to a more intrinsic evaluation of feasibility boundaries. thus, we seek to answer two important research questions, rq1: can llms delineate self-knowledge boundaries and accurately generate tasks that test these limits? and further, rq2: do llms adhere to the same self-knowledge boundaries when prompted to attempt such selfgenerated tasks? our approach uses generation-classification consistency in llms' self-perception of knowledge boundaries as the basis for evaluation, similar to @xcite . we provide a novel view of llm selfknowledge by encouraging llms to both set and cross their own boundaries to generate infeasible tasks and verify if such views of knowledge limits remain consistent while attempting these tasks. as seen in figure 1 , our methodology is universally applicable across open-source and black-box models. by giving llms the flexibility to set their own feasibility boundaries, we do not restrict the llm to human-annotated limits and provide a more authentic and reliable perspective on self-knowledge. our research holds the potential to improve several aspects of ai trustworthiness and reliability: it elucidates llms' perceptions of their own boundaries, identifies and classifies strong and weak types of self-knowledge and common confusions, and provides alternate explanations and reasons for other undesirable tendencies of llms, including overrefusal @xcite , adversarial helpfulness @xcite and overconfidence @xcite . the main contributions from our research can be summarised as follows: ## related work existing studies on self-knowledge in llms primarily focus on analysing responses and quantifying uncertainty in question-answering tasks with binary labels (answerable and unanswerable) @xcite . however, such approaches are not only restricted by humangenerated views of feasibility and infeasibility, they do not try to explore why llms deem certain questions unanswerable and fail to identify the types of self-knowledge most lacking in llms. also, uncertainty detection methods often lack feasible alternatives for black-box models @xcite . prompt-based solutions (yin et al., 2024) and training llms to identify uncertainty by parameterefficient tuning (chen et al., 2023) can address limitations imposed by datasets, but cannot reduce the over-reliance on question-answering tasks. while semi-open-ended question-answering proposed by wen et al. (2024) partially addresses the rigidity of human perceptions of feasibility, almost all existing methods lack intrinsic exploration of selfknowledge boundaries. prior evaluations have shown llms have a poor perception of their knowledge boundaries, often displaying low abstention with a tendency to be overconfident @xcite , even while explaining incorrect answers @xcite . however, a comprehensive study identifying knowledge areas where such behaviour is most persistent remains lacking. examining these tendencies through a self-knowledge lens can uncover new opportunities for enhancing ai trustworthiness. ## result discussion our findings are presented as follows: ## conclusion improving llm self-knowledge is fundamental for developing more trustworthy models and diversifying applications. in this study, we quantify different types of llm self-knowledge by giving them the flexibility to set their own feasibility boundaries and then exploring consistency in these limits. we find that even the best-performing models cannot accurately judge their capabilities more than 80% of the time, highlighting a significant lack of trustworthiness in complex tasks. we also observe that models are much more likely to be overconfident about their functional and ethical boundaries if not prompted to answer selfgenerated tasks. we also investigate common confusions in llms' perceptions of self-knowledge types and find that struggles in understanding context make models question their own operational boundaries. also, even powerful llms greatly struggle to extract logical tasks accompanied by incoherent context, completely dismissing them as illogical. by identifying and elaborating on gaps in selfknowledge in our work in depth, we hope that further research built upon our findings improves the trustworthiness, and subsequently, the reliable usability of ai in real-world scenarios.