refactor and get ready for offline compressing