Initial changes to WikiParser to allow case insensitivity in tags, and also to select areas of wikitext that are "active" when searching for templates etc.
Change RegExp that caused strip_syntx to blow up on strings like "[http://ab][http://cd][http://ef]"
Use namespace IDs rather than text grepping to check for page type: most of these will be used in later code modifications.
Allow us to check if a page is a redirect, and if so, where it is pointing
Make sure when we look for templates, image urls, etc we are looking at the active wikitext, rather than stuff in <!-- comments --> or <nowiki> tags.
Change name of function from "images()" to "media_on_page()", since we want to get audio, video, etc. Also normalize media filenames so they match correctly against page titles.
Some wikimedia files have extension .jpeg
Be polite to Commons, by allowing encoding "gzip,deflate" to be sent to wiki API. Also allow for specific agent, as detailed in
http://meta.wikimedia.org/wiki/User-Agent_policy

This is not set yet. We should decide on a suitable EoL string, so that MediaWiki know who all these requests are coming from
Something like: 'EoLHarvestingCode/1.0 (https://github.com/EOL; XXX@eol.org) pretendingtobeMozilla/5.0'
Change taxonomy parsing, so that taxonomic levels can be passed in in any order, in preparation for
parsing the {{Taxonavigation}} template, rather than the page HTML.
Also allow for a "species" field only containing the epithet (e.g. 'sapiens', not 'Homo sapiens')
as long as certain criteria are met (e.g. must have a genus).
Don't allow unicode or wikitext {{}} in names, as per code in the taxonomy() function.
remove check for unicode & wikitext from taxonomy(): it is now done on a per-level basis in taxon_parameters()
A few extra unicode + multiple-line modifiers for the RegExps used to parse wikitext
Implement new template parser for wikitext. This is unused for the moment, but will be used for {{Information}}, {{Location}} (gps data) and most importantly, {{Taxonavigation}} and {{TaxonavigationIncluded}} templates. Code is submitted here for review before being used.
cosmetic: improve indentation
cosmetic - improve commenting
Separate licence-detection code into a single routine, with license detection happening in
order of preference (in case there are multiple licenses, they are picked in order
PD < CC0 < CopyrightFree < CC-BY < CC-BY-SA and variants < CC-BY-NC and variants

The most preferred license, in case of multiple licenses, is chosen by passing in to this
routine a string containing all licenses, one per line.

Separating the detecting into a separate function also allow the same routine to be used
if we wish to detect licenses from categories.
Use php autoincrement to match part_aa, part_ab, files, etc, and obviate the need to hard code or pass global variables into process_file().
Also means that filenames (wikipedia/part_xx) are only listed in a single place.
Also avoid messy checking via ls() system call, and stop when no more files to read. As an additional check, look for correct ending of the last file, to make sure all the XML dump has been downloaded and split correctly. Warn otherwise.
minor correction to last commit - correct function name
Merge branch 'UncontroversialWikimediaImprovements' of https://github.com/hyanwong/eol_php_code into hyanwong-UncontroversialWikimediaImprovements
Merge branch 'TemplateParser' of https://github.com/hyanwong/eol_php_code into hyanwong-TemplateParser

Conflicts:
	vendor/wikipedia/WikiParser.php
Refactored how parameters are passed to several functions