JohnLangford/vowpal_wabbit — pull request #253 by fulluey (2014-03-20 02:37:59+00:00)
Title: fix duplicate cubic crossings when loading model

When specify "--cubic pnf" crossing at command line and load an initial model "-i vw.bin" with the same cubic cross, it will be applied twice and causing large mismatch. Feature count jumped from 1434 to 1616 for the first example. Looked at the source code. "-q" was properly safe guarded with de-dup logic, but triples were left out. Hence proposing the simple fix for it. It runs fine locally in the same test.

out/vw.darwin --loss_function logistic -l 0.1 --initial_t 1e6 -b 27 --holdout_off -d /Users/huig/projects/ps_opt/data/vw_data.txt --keep c --keep d --keep e --keep f --keep j --keep k --keep l --keep m --keep n --keep o --keep p --keep r --keep s --keep t --keep u --keep v --keep w -q ev -q ew -q fj -q fk -q fl -q fm -q fn -q fo -q fp -q st -q r: --cubic pnf -f vw.bin
creating quadratic features for pairs: ev ew fj fk fl fm fn fo fp st r:
creating cubic features for triples: pnf
using namespaces beginning with: c d e f j k l m n o p r s t u v w
Num weight bits = 27
learning rate = 0.1
initial_t = 1e+06
power_t = 0.5
using no cache
Reading datafile = /Users/huig/projects/ps_opt/data/vw_data.txt
num sources = 1
average since example example current current current
loss last counter weight label predict features
2.373233 2.373233 1 1.0 1.0000 -2.2754 1434
2.846391 3.319548 2 2.0 1.0000 -3.2827 731
1.903481 0.960571 5 4.0 1.0000 -0.8174 897
0.420754 0.138329 8 25.0 -1.0000 -3.1213 1196
0.221305 0.102586 12 67.0 -1.0000 -2.8766 897
0.267882 0.314458 22 134.0 1.0000 -3.2036 1210

=============

out/vw.darwin --loss_function logistic -l 0.1 --initial_t 1e6 -b 27 --holdout_off -d /Users/huig/projects/ps_opt/data/vw_data.txt --keep c --keep d --keep e --keep f --keep j --keep k --keep l --keep m --keep n --keep o --keep p --keep r --keep s --keep t --keep u --keep v --keep w -q ev -q ew -q fj -q fk -q fl -q fm -q fn -q fo -q fp -q st -q r: --cubic pnf -i vw.bin -f vw2.bin
creating quadratic features for pairs: ev ew fj fk fl fm fn fo fp st r:
creating cubic features for triples: pnf
final_regressor = vw2.bin
using namespaces beginning with: c d e f j k l m n o p r s t u v w
Num weight bits = 27
learning rate = 0.1
initial_t = 1e+06
power_t = 0.5
using no cache
Reading datafile = /Users/huig/projects/ps_opt/data/vw_data.txt
num sources = 1
average since example example current current current
loss last counter weight label predict features
3.338374 3.338374 1 1.0 1.0000 -3.3022 1616
3.231990 3.125606 2 2.0 1.0000 -3.0807 909
2.151710 1.071431 5 4.0 1.0000 -0.9686 1031
0.452659 0.129031 8 25.0 -1.0000 -3.8653 1372
0.232247 0.101049 12 67.0 -1.0000 -3.2103 1075
0.259889 0.287531 22 134.0 1.0000 -3.2256 1372