From here: https://github.com/rjurney/datafu/tree/master/data


On Tue, Jan 7, 2014 at 9:49 PM, Matt Hayes <notifications@github.com> wrote:

> In src/java/datafu/pig/text/opennlp/TokenizeME.java:
>
> > +public class TokenizeME extends EvalFunc<DataBag>
> > +{
> > +    private boolean isFirst = true;
> > +    InputStream is = null;
> > +    TokenizerModel model = null;
> > +    private TokenizerME tokenizer = null;
> > +    public static final String MODEL_FILE = "tokens";
> > +    TupleFactory tf = TupleFactory.getInstance();
> > +    BagFactory bf = BagFactory.getInstance();
> > +    String modelPath;
> > +
> > +    public TokenizeME(String modelPath) {
> > +        this.modelPath = modelPath;
> > +    }
> > +
> > +    public TokenizeME() {
>
> But where does this file come from? The default constructor uses a path
> that won't exist.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/linkedin/datafu/pull/84/files#r8716175>
> .
>



-- 
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com