add in compression options and do some refactoring