Pass the $no_dedup_sentences and $no_vector parameters forward to the process_download_for_extractor() (although they're not actually being used in the ExtractAndVector Gearman worker)
When skipping the "split story into sentences, words and determine language" part of the extraction, report that fact to STDERR
In the extractor code path, don't return or die without logging the reason; in Languages.pm, return undef (instead of an empty array) if it wasn't possible to split given text into sentences
Fix "$ignore_date_range or _story_within_media_source_story_words_date_range( $db, $story )" condition
After encoding / hashing the sentence, check if the sentence got encoded / hashed; normalize heredoc SQL queries by adding double quotes around them
Remove irrelevant comment
Fetch only the "downloads_id"
fixed bug triggered by empty response
Merge branch 'master' of github.com:berkmancenter/mediacloud
typo
update stories_public to return processed_stories_id so that last_processed_stories_id parameter can work for paging
Merge branch 'master' of https://github.com/berkmancenter/mediacloud
Temporarily install Mallet::CrfWrapper from Linas's repository because it contains experimental fixes that should fix the Vagrant test run
Revert "Temporarily install Mallet::CrfWrapper from Linas's repository because it contains experimental fixes that should fix the Vagrant test run"

This reverts commit 4504454930879e9750e77aa0512147f9ca4cd719.
When installing Mallet::CrfWrapper, don't run unit tests because the web service test ends up as a Perl zombie process during the Vagrant test run
Change "solr_select_url" to "solr_url" in the rest of the code (default configuration, configuration example file, helper Python scripts); initial change was introduced in f12f55462917fcc32bf3707e5df6da2904aef2c9
Add simple, YML-configurable Bit.ly API helper package
Add basic subroutine to canonicalize URLs
Add some more garbage URL parameters to remove
Add script that runs a sample list of story URLs against Bit.ly API