Entropy (information theory) | Wikipedia audio article
Channel: wikipedia tts
Date: 28/12/2018

This is an audio version of the Wikipedia Article:
https://en.wikipedia.org/wiki/Entropy_(information_theory)


00:03:58 1 Introduction
00:10:25 2 Definition
00:13:01 3 Example
00:13:13 4 Rationale
00:14:50 5 Aspects
00:15:09 5.1 Relationship to thermodynamic entropy
00:17:59 5.2 Entropy as information content
00:18:46 5.3 Entropy as a measure of diversity
00:24:17 5.4 Data compression
00:24:26 5.5 World's technological capacity to store and communicate information
00:29:38 5.6 Limitations of entropy as information content
00:31:32 5.7 Limitations of entropy in cryptography
00:31:55 5.8 Data as a Markov process
00:32:28 5.9 spanib/i
00:33:23 6 Efficiency
00:36:35 7 Characterization
00:36:53 7.1 Continuity
00:37:02 7.2 Symmetry
00:38:46 7.3 Maximum
00:41:28 7.4 Additivity
00:41:46 8 Further properties
00:42:01 9 Extending discrete entropy to the continuous case
00:43:10 9.1 Differential entropy
00:43:52 9.2 Limiting density of discrete points
00:45:57 9.3 Relative entropy
00:46:47 10 Use in combinatorics
00:46:59 10.1 Loomis–Whitney inequality
00:47:03 10.2 Approximation to binomial coefficient
00:47:12 11 See also
00:47:24 12 References
00:48:15 13 Further reading
00:49:45 13.1 Textbooks on information theory
00:52:00 14 External links
00:52:10 bn
00:52:24 0. This implies that the efficiency of a source alphabet with n symbols can be defined simply as being equal to its n-ary entropy. See also Redundancy (information theory).
00:52:39 Further properties
01:00:04 Extending discrete entropy to the continuous case
01:00:16 Differential entropy
01:08:09 Limiting density of discrete points
01:10:14 Relative entropy
01:11:54 m as measures. It is defined for any measure space, hence coordinate independent and invariant under co-ordinate reparameterizations if one properly takes into account the transformation of the measure m. The relative entropy, and implicitly entropy and differential entropy, do depend on the "reference" measure m.
01:12:16 Use in combinatorics
01:12:27 Entropy has become a useful quantity in combinatorics.
01:12:34 Loomis–Whitney inequality
01:15:46 log|A|, where |A| denotes the cardinality of A. Let Si
01:16:55 Approximation to binomial coefficient



Listening is a more natural way of learning, when compared to reading. Written language only began at around 3200 BC, but spoken language has existed long ago.

Learning by listening is a great way to:
- increases imagination and understanding
- improves your listening skills
- improves your own spoken accent
- learn while on the move
- reduce eye strain

Now learn the vast amount of general knowledge available on Wikipedia through audio (audio article). You could even learn subconsciously by playing the audio while you are sleeping! If you are planning to listen a lot, you could try using a bone conduction headphone, or a standard speaker instead of an earphone.

Listen on Google Assistant through Extra Audio:
https://assistant.google.com/services/invoke/uid/0000001a130b3f91
Other Wikipedia audio articles at:
https://www.youtube.com/results?search_query=wikipedia+tts
Upload your own Wikipedia articles through:
https://github.com/nodef/wikipedia-tts
Speaking Rate: 0.9984535686745089
Voice name: en-GB-Wavenet-D


"I cannot teach anybody anything, I can only make them think."
- Socrates


SUMMARY
=======
Information entropy is the average rate at which information is produced by a stochastic source of data.
The measure of information entropy associated with each possible data value is the negative logarithm of the probability mass function for the value: 

  
    
      S
      =
      −
      
        ∑
        
          i
        
      
      
        P
        
          i
        
      
      ln
      ⁡
      
        
          P
          
            i
          
        
      
    
    S=-\sum _{i}P_{i}\ln {P_{i}}
  . When the data source has a lower-probability value (i.e., when a low-probability event occurs), the event carries more "information" ("surprisal") than when the source data has a higher-probability value. The amount of information conveyed by each event defined in this way becomes a random variable whose expected value is the information entropy. Generally, entropy refers to disorder or uncertainty, and the definition of entropy used in information theory is directly analogous to the definition used in statistical thermodynamics. The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication".The basic model of a data communication system is composed of three elements, a source of data, a communication channel, and a receiver, and – as expressed by Shannon – the "fundamental problem of communication" is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel. The entropy provides an absolute limit on the shortest possible average leng ...