Added get_all_tokens() and moved around the corpus as per the old TODO