dev of experiment for lattice2seg(l2s), staring from gen_ins(tance)_to_train_score(ing)_model
specification of instance generation function for training scoring model, which requires update of viterbi_search as well
viterbi_search.py ready for inference/decoding purpose
  - fix the bug in initial_best score
  - use score_it (score_model, bigram, incoming_char) interface to gen word hypothesis/state score
  - add score_model and sent as parameter of viterbi_search
  - add simple function get_incoming_char(sent, sent_index)
small issue fix
done with training instance gen for scoring model.
- read segmented sentences train.seg and build lattice train.lattice (calling lattice_gen.py)
- traverse train.lattice to visit each bigram state <bigram, incoming_char> and generate its features using feature_gen.py
- collect instane, label = "T" if the state is in the gold-standard segmentation, "F" otherwise
done with training instance gen for scoring model.
- read segmented sentences train.seg and build lattice train.lattice (calling lattice_gen.py)
- traverse train.lattice to visit each bigram state <bigram, incoming_char> and generate its features using feature_gen.py
- collect instane, label = "T" if the state is in the gold-standard segmentation, "F" otherwise
*trivial* clean ipynb
Done with exp_l2s_gen_*.py, which generates instances for training scoring function of lattice-based segmentation.
- multiprocessing code
- auxiliary code such as reading corpus and writing the instances