updates and compression of data i collected