Branch for language identification functionality
Source downloader for language identification, refs #2782
Moving langid scripts to script_archive/ as they're one-off utilities
First attempt at a script that converts HTML samples to plain text
candidate methods to evaluate language
convert HTML files to plain text
hardcoded path to CLD module
there exists Lingua-Identify-CLD Perl module already
Lingua::Identify
CLD
Lingua::Identify vs CLD
removing language identification assessment tools as we have already decided on CLD
Merge trunk -> language_identification
renamed language modules to ISO 639-1 names because now we can have a standard for naming languages, and the language detector wouldn't be able to separate en_US from en_GB anyway
add Lingua::Identify::CLD dependency
add Lingua::Identify::CLD dependency #2
don't set "exit on error" flag in set_perl_brew_environment.sh because if we 'source' this script in ~/.bash_profile (as recommended by the perlbrew installer), the shell exits on each and every error; check for errors manually instead
Upstart (RC) scripts for Ubuntu
correct number of arguments to query_paged_hashes(), doesn't fail on the fresh Media Cloud install
Merge trunk -> branch