remove files on Windows so --revert and conversion can proceed (without Error 183, file already exists)

Windows won't allow a rename to overwrite an existing file.
tweak "dying words" error message to try to improve clarity

The existing error message is oblique enough that any developer who knew enough to understand what wmllint was driving at would know better than to use "die" for last words in the first place. A more direct message would be more likely to set newbie UMC authors on the right path.
miscellaneous spelling corrections
increase the number of orcish warlord portrait replacements to improve variety

With three new orc alternate portraits added during 1.9, it is no longer necessary to double-up any of the substitutions for the five old-style orcish warlord portraits.

In fact, with six portraits available, one has to draw the short straw, and I chose grunt-4. I prefer grunt-5 and grunt-6, and also find them more leaderly.
revert removal of usage class check (r48544, ai0867): caught real errors

The rationale for removing the usage class check was, "Don't make wmllint check if the usage type matches a standard one. Non-standard ones work just fine".

However, custom usage classes are relatively rare, while the check caught real errors: typos, bogus classes, using unit types instead of usage types. Other wmllint checks have higher rates of false positives.

The next commit will handle non-standard usage classes, by creating a magic comment for them.
make usage_types appendable and create magic comment to recognize non-standard usage types

An explanation of the magic comment is also added to the introduction.
code to change Windows-style backslashes in file paths to frontslashes

The task is to replace Windows backslashes in paths, without indiscriminately replacing backslashes in legitimate use as escapes (or bridge terrain).

Breaking this down:

* "no-syntax-rewrite": I don't think this is really necessary, but I will follow the practice of the hack_syntax section.
* if '\\': Technically, this code worked going straight to the while statement, but running every line through the complicated regex made it sluggish. Faster to make sure the line meets a simple filter first.
* while: It is possible, though very uncommon, that a line could contain more than one file path. Looking at the regex test: a) Let's preclude any matches in comments. They don't affect the game, and it's theoretically possible that a comment-match could be an explanation for Windows users.  b) After matching the backslash itself, be on the lookout for a character used to set apart a file path from other text. By excluding these, we ensure that there is an unbroken chain from the backslash to the file extension.  c) Then we have a list of file extensions, bracketed on the left by a period and the right by \b, to make sure they do not coincidentally match a string. These are the file types that might be referenced by a value (for instance, translation files are not referred to directly, so their extensions are not included). As a practical matter, EVERY instance in the wild I know of involves png, not counting one commented-out path in an ancient campaign.  d) File extensions can include capitals, particularly on Windows, where the effects of DOS unicase linger. So we make the search case-insensitive.
* regex object: This splits the line into groups. The differences from the regex used in the while statement: a) We start off matching any character, but there is no danger of crossing over into comments. Because the match is non-greedy, it will replace the leftmost instances first, and when it hits a comment, the while test will end.  b) We also look for a non-pathbreaking string to the left of the backslash as well as the right. Together with the fact that the preceding match is non-greedy, it means that group(2) will match the entire file path except the extension.
* lines[i]: Here, we simply reconstruct the line from its group components, except all backslashes in group(2) are replaced by frontslashes.
* print: Besides reporting the substitution to stdout, I include a plea for cross-platform compatibility.
regex sub to remove 'userdata/' from paths

The misguided authors who put userdata/ in their paths cause problems not just for non-Windows users, but fellow Windows users who chose not to put userdata in the install directory. This error can be removed by an approach similar to that just used to purge backslashes:

* if 'userdata': A basic filter to cut down the number of lines being run through complicated regex speeds up performance.
* while: It is possible, though rare, for a line to contain more than one path with userdata. Points about the regex: a) It precludes matches in comments, though most (all?) of these are commented-out old paths that it would be well to correct also, lest they mislead any more UMC writers.  b) In case you're wondering why I made one 'data/' string optional, there's a set of add-ons in 1.4 that use "userdata/campaigns" instead of "userdata/data/campaigns".  c) The '[ac]' at the end is something of an artifact of the time before I excluded comments, but it provides another safety measure insuring that the string is actually a value.
* regex object: This splits the line into groups. Notes: a) We start with a non-greedy match, so the leftmost instances will be replaced, and the while loop will stop when it hits a comment.  b) Originally, the big group(2) did not exist, and the groups inside it were regular groups. However, when I referred to them in the print statement, it would output 'None' instead of nothing when there was no match. The only way I found to get print to work properly was to fold them inside a larger group, and after I did that, I decided it would be clearer to make them non-capturing groups.  c) For reporting to stdout, group(3) is extended to the next '/', though this part of the match is optional, to insure that there's no way to get trapped in a while loop.
* lines[i]: Here, we simply reconstruct the line from its group components, except we simply drop group(2).
* print: In case designers don't get the point from seeing the elimination logged in stdout, I include an all-caps admonition against "userdata/". This is a really irritating bug.
update 1.4's {LOYAL_UNIT} macro to {NAMED_LOYAL_UNIT}, using a regex to distinguish it from the current {LOYAL_UNIT}

When 1.4's {LOYAL_UNIT} changed to {NAMED_LOYAL_UNIT}, the old name went to a new macro. However, the current {LOYAL_UNIT} ends after the x,y coordinates, so if a regular expression detects two more blocks of text afterwards, it must be the old macro.

Step by step:

* This is going to be the first of several operations involving lines with macros, so we will start with an if filter looking for lines with an opening brace, and indent the rest of the code inside it.
* Then we look for a regex match. The first field of the macro is the side number, which should be a number. Then we have the unit type field, for which we have three rules, depending on whether the author chose to use parentheses, quotes, or a bare entry. Then we have the x,y fields, which will normally be numbers, but which could also be a variable. Different variable names could be used, but we will assume that there will be a $ sign, and an x (or y) somewhere. Then we get to the key. After the coordinates, there is another string that doesn't include a space or a closing brace, followed by at least one more string without a space or closing brace. For current purposes, we only need to confirm that there are at least two more blocks of text before the right brace, and we needn't concern ourselves if they fit a format.
* If we have a match, the change is a simple replacement.
* Log the change to stdout. I'm a little worried about a newbie seeing the change in the log and getting the idea that ALL instances of {LOYAL_UNIT} should be replaced, so I added a little message that this is the 1.4 version of the macro.
auto-recognize the people in the {NAMED_*UNIT} macros

The heart of this code is a regex object, named, which will be used to split out the group containing the macro's character id field. Going over the regular expression:

* We don't want to process any commented-out macros.
* We handle all the NAMED_*UNIT macros by allowing any string of caps and underscores to match. This could also match UMC macros, like "Attack of the Western Cavalry"'s NAMED_SHIP_UNIT. (In theory, there could be false positives from UMC macros that followed the mainline naming convention but not the format. However, such macros would most likely fail the regex, and even if they didn't, false entries in "present" are generally not a major problem.)
* The first macro field, for the side, should be a number.
* For the unit type field, three rules, depending on whether the author uses parentheses, quotes, or a bare entry. At least one campaign uses these macros to recall units in subsequent scenarios with a blank unit type field, so I do not require there to be something inside the parentheses or quotes.
* Then we have the x,y fields, which will normally be numbers, but which could also be a variable. Different variable names could be used, but we will assume that there will be a $ sign, and an x (or y) somewhere.
* Now we come to the id field, the one we're interested in. As with the unit type field, there is a threefold rule. The only difference is that while there are authors who have left this field blank, there's no point in auto-recognizing those cases, so I do require parentheses and quotes to contain something.
* I also check that there is one more field in the macro. Originally, I also checked for a closing brace, but I became aware that some instances of NAMED_UNIT have WML that continues onto another line.

If there is a match, we strip group(4) of parentheses, quotes, and any waste spaces this leaves, and append it to the list of the present.

I also tweak the wording in the intro about the "wmllint: recognize" magic comment to reflect that wmllint is now recognizing ids in some macros.
add conversion of "~/campaigns" to "~add-ons"

grep turns up several old add-ons that use this for map_data.
convert old UMC paths from data/campaigns/ to data/add-ons/ in a mainline-safe way

My first step is to draw up a list of mainline campaigns to compare path directories against.

As for the conversion itself, I want wmllint to be able to accurately handle a possible case where a line contains both a UMC and a mainline path - even though I don't know of any actual example in the wild. So:

cleanup_main: This will be used to convert mainline paths back to normal, as we shall see.
while: As long as there is any instance of "data/campaigns/<dir>", we will keep replacing
  dc : This regex object breaks down the line into the following groups:  (1) - The line, up to 'data/' in the first path. (Since the while statement operates on the whole line, a greedy match would also work fine, but I'd gotten used to non-greedy.) (x) 'campaigns/' is going to be replaced, so it doesn't need to go in a group. (2) The name of the campaign/add-on directory. (It would get the job done with just '\w+', since all mainline campaigns use only letters and underscores, but I've added characters that are used in some UMC names so that the full directory name gets printed to stdout.)
  if: group(2) is checked against the list of mainline campaigns. If it's not there, the line is rebuilt substituting 'add-ons/' where 'campaigns/' was, and a report printed to stdout.
  else: The alternative means that it is a mainline path. However, we need the 'while' test to go forward, so in this case, a meaningless string is substituted for 'campaigns/'. The line is flagged by setting cleanup_main to True.
if cleanup_main: Now we can change mainline paths back to "data/campaigns", a simple replace operation. This means that mainline campaign files will not be touched, since the net effect is to leave the line unchanged.