write to pickle for intermediary steps to save time, and added a 'to json' command to generate the final json