JohnLangford/vowpal_wabbit — pull request #78 by nicknussbaum (2012-12-28 23:01:49+00:00)
Title: allow multi-pass from library api  now using do_reset_source

This patch allows --passes to work from the library VW initialization
with a cache file specification

It also adds a small test to the cs_test/Program.cs
The point of this code is to allow --passes to work from the library.
It requires enabling the cache.

The prediction results now seem to work. I've added a .bat file to test to invoke the windows vw with test 1 so the two predict files can be compared.

Per Discussion with John, I've changed the code to use a new vw global do_reset_source rather than fiddling with the pass_length and stdin _off.

Changes;

changed parsed_atomic_example to have an optional do_read that defaults to
true. If false it will not read the example (which the library loaded) but
will cache it and also do ngram processing which was missing.

parsed_atomic_example is now called from VW:read_example, and various
VW:import_example functions before the setup_example.

on VWdll.cpp VW_Finish if the pass count is > 1 do_reset_source is set, some values are adjusted
and then the start_parser, driver end parser loop is called invoke the main_parse_loop to read the
remaining passes from the cache.

the main_parse_loop was changed
If do_reset source was called, the first thing it does is reset_source which in the library case
will go from waiting for examples to be loading to reading the cache file.

By calling the parser loop after the reads, the multi pass from cache is
done, but possible other pre-read sequencing is missed for other
functionality. A more general fix would be to make a mock reader which is
really a co-routine for the library functions that are invoked between
init and finish. This would allow stuff like merge caches before
prediction and other stuff to work.

Also this hasn't been tested with unix. I'm a little uncertain if the
fiddling to get it to do the passes has all the multi-thread
initialization done right.