linkedin/datafu — pull request #78 by king821221 (2013-12-21 15:53:51+00:00)
Title: create initial version of entroy UDFs

This commit creates a bunch of UDFs to calculate entropy.

It supports entropy calculation given sample frequencies, please refer to Entropy.java. We only support empirical entropy estimator.

or given a stream of raw data samples, please refer to StreamingEntropy.java. For streaming scenario, we have supported empirical and ChaoShen entropy estimators.

Unit tests are done.

Will add counters and is doing test on real clusters