joshua-decoder/joshua — pull request #133 by gwenniger (2013-09-12 22:43:52+00:00)
Title: Gideon : Added a new Feature function  src/joshua/decoder/ff/LabelCombinationFeatureFunction.java  plus a test for it

Gideon : Added a new Feature function src/joshua/decoder/ff/LabelCombinationFeatureFunction.java plus a test for it. Refactored the "get" method in FeatureFunction / FeatureVector, to get a certain feature.
This method was returning 0 if the element did not exist, and this was used in other methods
(add) however, this is a risky kind of programming and indeed can lead to bugs and did.
What was happening is that the feature table was queried with an uppercase version of
certain features, so that it looked like those feature values were 0, while in fact
they were not : they were being queried for by a wrong name.
This is now fixed: the get method throws a runtime exception when an element does not
exist, as it should. The containsKey method should be used before querying, this
more defensive style of programming (failing fast) prevents such nasty bugs.
Finally made a first step towards improving JoshuaConfiguration by implementing a
reset method. This makes it possible to run the decoder twice within the same
java program - provided the reset method is called after the first run.
This is nescessary because the globally shared state of JoshuaConfiguration leads
to errors when already initialized and set a second time (which was what was
happening before). The global state of StatefullFF faces the similar nasty
problem with shared state and also needed a reset method for the same reason.

This all points to one inevitable conclusion though: JoshuaConfiguration must
be refactored, and made into a normal (non-static) object that is passed
to the classes that need it. This makes the whole enterprise more clear and
clean and more structured at the price of some work. This is the next possible
step however and not part of this commit.

modified: src/joshua/decoder/Decoder.java
modified: src/joshua/decoder/JoshuaConfiguration.java
modified: src/joshua/decoder/ff/FeatureFunction.java
modified: src/joshua/decoder/ff/FeatureVector.java
new file: src/joshua/decoder/ff/LabelCombinationFeatureFunction.java
modified: src/joshua/decoder/ff/StatefulFF.java
modified: src/joshua/decoder/ff/tm/GrammarReader.java
modified: src/joshua/util/FileUtility.java
new file: src/joshua/util/NBestListUtility.java
new file: test/joshua/decoder/FeatureFunctionsTest.java