Added autoadjust parameter to build_vocab to allow automatic
vocabulary pruning to keep the model within memory constraints.
Merge branch 'develop' into add_min_count_autoadjust

Merging changes from upstream
Fixed syntax error in string formatting
This records a failed attempt to fill up memory for testing
min_count_autoadjust. I am recording it in case I want to try
something like it again.
The next iteration of the test code for min_count_autoadjust.

If I make the word vectors large enough that memory is exhausted
quickly, the kernel out-of-memory manager kills it before it has a
chance to adjust the vocabulary. If I make the word vectors smaller,
it has to make the sentence lists too large. I ran it for 12 hours
without it running the final min_count_autoadjust iteration. I just
realized that my sentence iterator is just returning large single-word
sentences. Oops. I'll try fixing that and see if things improve.
Last attempt at test code. With the sentence iterator corrected the process gets killed almost immediately. 5 sentences doesn't get a MemoryError and 6 sentences is killed by the Kernel. I'll remove the test code in the next commit.
Removed all the test code. The low memory functionality is too difficult to test. I will leave it untested.
Split out some code into the create_unique_index method to prepare for merging with a codebase that has summitborar's changes. (Plus, it was a good idea he/she had.)
Renamed create_unique_index to create_indexed_vocab because this better captures the method's purpose.

Also updated method docstrings to mention that create_unique_index is called from build_vocab and is a prerequisite to create_binary_tree
Removed min_count_autoadjust - using MemoryError was too prone to problems. Instead, I moved the vocabulary reading code into vocab_counts so I can cache that value separately. I also added code for adjusting min_count to give a certain maximum vocabulary size (which can be chosen to avoid running out of memory independent of the corpus size.)

The separate invocation of vocab_counts allows me to read the vocabulary once (time-consuming on large files) and then experiment with different memory sizes and then finally put this into the final training.
Added equality function to Vocab object so I can use it in assertEqual calls. Also modified vocab_counts docstring to reflect that it returns Vocab objects.
Added (passing) test case for vocab_counts. Other tests don't yet pass.
Fixed bug - I'd forgotten that vocab_counts was a member function and wasn't calling it correctly
Fixed calls to build_vocab in test code to reflect new parameters
Re-enabled parallel test
Added Vocab.__repr__ for better test message printing and a comment that create_indexed_vocab doesn't enforce the max_words constraint
Added code to test create_indexed_vocab
Fixed Vocab.__eq__ to deal with numpy arrays in attributes and made __hash__ consistent with __eq__
Added code to test whether max_words works as expected within build_vocab
Fixed typo in logger, wrong way of calculating # word types and bug with incrementing a member variable