I think the model should be required and specified in the constructor.  This can then be loaded into the distributed cache.

The alternative is to pick default models and have these be packaged either in this JAR or in a separate supplemental JAR.  However the issue I have with this is that 1) sometimes there are multiple models, and it's not clear which should be the default, and 2) the model files can be quite large.  If the model files were stored in the datafu JAR then it would bloat the size. 

Here is how you can use the distributed cache:

```java

public static final String MODEL_FILE = "MODEL_FILE";
private TokenizerME tokenizer;

public Tokenize(String modelPath) {
  this.modelPath = modelPath;
}

@Override
public List<String> getCacheFiles() {
  List<String> list = new ArrayList<String>(1);
  list.add(this.modelPath + "#" + MODEL_FILE);
  return list;
}

public DataBag exec(Tuple input) throws IOException
{
  if (this.tokenizer == null) {
    initTokenizer();
  }

  // etc.
}

private void initTokenizer() {
  String loadFile = getFilename();
  InputStream file = new FileInputStream(loadFile);
  InputStream buffer = new BufferedInputStream(file);
  TokenizerModel model = new TokenizerModel(buffer);
  this.tokenizer = new TokenizerME(model);
}

private String getFilename() throws IOException {
  // if the symlink exists, use it, if not, use the raw name if it exists
  // note: this is to help with testing, as it seems distributed cache doesn't work with PigUnit
  String loadFile = MODEL_FILE;
  if (!new File(loadFile).exists()) {
    if (new File(this.filename).exists()) {
      loadFile = this.modelPath;
    } else {
      throw new IOException(String.format("could not load model, neither symlink %s nor file %s exist", MODEL_FILE, this.modelPath));
    }
  }
  return loadFile;
}
```