A Tale of LLMs and Induced Small Proxies: Scalable Agents for Knowledge
  Mining
Authors: name: Sipeng Zhang, name: Longfei Yun, name: Zilong Wang, name: Jingbo Shang, name: Letian Peng
Organization: University of California San Diego

At the core of Deep Research is knowledge mining, the task of extracting
structured information from massive unstructured text in response to user
instructions. Large language models (LLMs) excel at interpreting such
instructions but are prohibitively expensive to deploy at scale, while
traditional pipelines of classifiers and extractors remain efficient yet
brittle and unable to generalize to new tasks. We introduce Falconer, a
collaborative framework that combines the agentic reasoning of LLMs with
lightweight proxy models for scalable knowledge mining. In Falconer, LLMs act
as planners, decomposing user instructions into executable pipelines, and as
annotators, generating supervision to train small proxies. The framework
unifies classification and extraction into two atomic operations, get label and
get span, enabling a single instruction-following model to replace multiple
task-specific components. To evaluate the consistency between proxy models
incubated by Falconer and annotations provided by humans and large models, we
construct new benchmarks covering both planning and end-to-end execution.
Experiments show that Falconer closely matches state-of-the-art LLMs in
instruction-following accuracy while reducing inference cost by up to 90% and
accelerating large-scale knowledge mining by more than 20x, offering an
efficient and scalable foundation for Deep Research.