Title: Knowledge Fitness Criterion: Measure-Theoretic Knowledge Assessment via Manifolds for Multi-Agent LLM Systems
Primary area: foundation or frontier models, including LLMs
Keywords: Large language models, Knowledge Adaptability Score, measure theory, statistical manifolds, multi-agent systems, autonomous driving, safety-critical applications

Abstract: Evaluating the compatibility between activated knowledge and task objectives is a critical challenge in LLM-based multi-agent systems. Existing methods rely on task-specific indirect metrics and lack a unified framework for directly mapping knowledge to objectives. We introduce the Knowledge Fitness Criterion (KFC), a general knowledge evaluation paradigm grounded in measure theory. KFC models knowledge states as measure spaces and establishes a chain of measurable mappings—from knowledge to features, features to indicators, and indicators to normalized scores—enabling direct, quantitative assessment of knowledge-task alignment. We present the Knowledge Goal Quantified-Quality (KGQQ) Theorem, linking scoring stability to feature distribution density. We validate KFC across three diverse domains: autonomous driving (nuScenes/CARLA), social role simulation (CAMEL on AI Society Dataset), and collaborative software development (ChatDev on SRDD Dataset), encompassing 2,450 knowledge states. Results show KFC outperforms supervised baselines, achieving MSE of 0.12 (22.5% lower), PCC of 0.85 (15.3% higher), and MAE of 0.08 (18.7% lower) in autonomous driving; MSE of 0.18 (20.0% lower), PCC of 0.76 (12.5% higher), and MAE of 0.11 (16.7% lower) in social simulation; and MSE of 0.15 (21.1% lower), PCC of 0.80 (14.3% higher), and MAE of 0.09 (18.2% lower) in software development (all p<0.05). KFC demonstrates robust performance across task complexities (cross-domain correlation r=0.82, p<0.001), with ablation studies confirming the critical roles of contrastive learning (28.5% performance gain) and semi-supervised strategies. Our framework transcends task-specific limitations, offering a universal, quantifiable solution for knowledge assessment in multi-agent systems.