Title: Real-Time Evaluation for Novel Class Discovery at Test Time
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Test-Time Discovery, Novel Category Discovery, Online Learning, Test-Time Training

Abstract: We introduce Test-Time Discovery (TTD), a real-time evaluation protocol for novel class discovery under sequential test-time conditions. Unlike post-hoc NCD evaluation, which assesses clustering only after the full test set is processed, TTD requires models to classify known categories and discover novel ones in real time as samples arrive. To address this setting, we propose a training-free Hash Memory (HM) method. HM encodes feature norm and direction into semantic-aware hash codes, enabling Locality-Sensitive Hashing (LSH) for efficient retrieval and consistent reuse of discovered classes. A global-to-local strategy combines prototypes for stable known-class predictions with memory-based reasoning for flexible novel discovery. A lightweight self-correction mechanism further improves reliability by removing mislabeled entries from early discoveries. Experiments on diverse benchmarks show that HM achieves more accurate and stable real-time discovery than NCD and TTT methods, while maintaining performance on known classes. Our code will be released.

TL;DR: We introduce Test-Time Discovery (TTD), a real-time evaluation protocol for novel class discovery under sequential test-time conditions.