The motivation is that the Shannon entropy of a probability distribution measures the amount of information contained in the distribution.
The motivation is that the Shannon entropy of a probability distribution measures the amount of information contained in the distribution.