function to handle data directly rather than load from disk