Entropy partitioning to measure diversity