major refactoring of mining and parsing