some updates

draft towards processing the data once, then loading