Merge pull request #3 from nltk/master

Merge updates from NLTK master
updated copyright year
Fixed docstring concerning integrated sentence tokenization and sentence-internal word-tokenization. Resolves issue #586
Merge pull request #608 from stevenbird/tokenize

Fix for issue #586
Fix broken contingency cube calculation in TrigramAssocMeasures (#610)
Merge pull request #611 from jnothman/fix_tri_assoc

Fix broken contingency cube calculation in TrigramAssocMeasures (#610)
removed ngram model package, pending resolution of issues #133, #157, #167, #367, #380, #388, #396, #602, #603; package saved in a new branch
renamed batch_parse/tag/tokenize to parse/tag/tokenize_sents, cf issue #561
renamed batch_classify to classify_many, cf issue #561
s/bigram/trigram
Merge pull request #616 from eltermann/collocations-typofix

s/bigram/trigram
fixed method names in relextract doctests
added 'untagged_filler' to the relextract doctest
made '_tree2semi_rel' public
Add approximate tests for association measures
Merge pull request #618 from jnothman/assoc_test

Add approximate tests for association measures
In ISRI stemmer's norm(), fix erroneous raw-string substitution
ISRI stemmer: Corrections to logic (and many style fixes)

Let me stress up front: I do not speak any Arabic at all. There are a small
number of places where I have fixed some very obviously broken logic, but by
and large I have left the actual work being done by this file alone.

Places where I did change the logic:

* suf32() and pre32(). These both start with "if word is at least 6 letters
  long", and go through the length-three affixes to see if each can be
  removed. The only sensible way for the "if the word is at least 5 letters
  long, go through the length-two affixes" block to work is for it to be at
  the same indentation level, but instead (in both methods) it was indented to
  be inside the length-three affix removal block. I've reindented these.

* pro_w64(). The first check in this method was

     if (self.stm[0] and self.stm[4])=='\u0627':

  This is clearly wrong; the intent is to check whether both the first and
  fifth letters are \u0627, but instead it always fails, because a boolean is
  not equal to a string. I've changed this to be two separate conditions.

The majority of corrections in this commit are:

* pep8 changes - putting spaces around operators, standardizing the spacing of
  inline comments, etc. (I did *not* change the line lengths, though many of
  the lines are quite long, because I wasn't sure what the standard for NLTK
  is.)

* Pythonic changes - having all methods take the stem as an argument and
  return the stem, rather than modifying "self.stm" and also (sometimes, but
  not always) returning self.stm. Changing "if <condition> & <condition>" to
  "if <condition> and <condition>". That sort of thing.

* Assorted other stylistic changes - moving the hard-to-read inline comments
  on the lists in __init__ to appear above each list instead. Correcting the
  spelling of "intial" to "initial" in a variable name. Etc.
Merge pull request #613 from stevenbird/batch

renamed batch_x to x_sents
Remove 'SVN Revision' from the list of fields that can be displayed in the downloader GUI, since it's not applicable anymore.