use stream based processing for data storage