High-performance file-based data collection and distribution system