added progressbars, fixed download, generating substrings for reference