In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential states or possible outcomes.
Dalam teori informasi, entropi variabel acak adalah tingkat rata-rata "informasi", "kejutan", atau "ketidakpastian" yang melekat pada kemungkinan hasil variabel.
The choice of base for log , the logarithm, varies for different applications.
Pilihan dasar untuk log , logaritma, bervariasi untuk aplikasi yang berbeda.
Base 2 gives the unit of bits (or "shannons"), while base e gives "natural units" nat, and base 10 gives units of "dits", "bans", or "hartleys".
Basis 2 memberikan satuan bit (atau "shannon"), sedangkan basis e memberikan "satuan alami" nat, dan basis 10 memberikan satuan "dit", "ban", atau "hartley".
An equivalent definition of entropy is the expected value of the self-information of a variable.
Definisi entropi yang setara adalah nilai yang diharapkan dari informasi diri suatu variabel.
The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication", and is also referred to as Shannon entropy.
Claude Shannon dalam makalahnya tahun 1948 " Sebuah Teori Matematika Komunikasi ", dan juga disebut sebagai entropi Shannon.
Shannon's theory defines a data communication system composed of three elements: a source of data, a communication channel, and a receiver.
Teori Shannon mendefinisikan sistem komunikasi data yang terdiri dari tiga elemen: sumber data, saluran komunikasi, dan penerima.
The "fundamental problem of communication" – as expressed by Shannon – is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel.
"Masalah mendasar komunikasi" - seperti yang diungkapkan oleh Shannon - adalah agar penerima dapat mengidentifikasi data apa yang dihasilkan oleh sumber, berdasarkan sinyal yang diterimanya melalui saluran.
Shannon considered various ways to encode, compress, and transmit messages from a data source, and proved in his source coding theorem that the entropy represents an absolute mathematical limit on how well data from the source can be losslessly compressed onto a perfectly noiseless channel.
Shannon mempertimbangkan berbagai cara untuk mengkodekan, memampatkan, dan mengirimkan pesan dari sumber data, dan membuktikan dalam teorema pengkodean sumbernya yang terkenal bahwa entropi mewakili batas matematis absolut tentang seberapa baik data dari sumber dapat dimampatkan tanpa mengalami kehilangan ke saluran tanpa derau sama sekali.
Entropy in information theory is directly analogous to the entropy in statistical thermodynamics.
Entropi dalam teori informasi secara langsung dianalogikan dengan entropi dalam termodinamika statistik.
The analogy results when the values of the random variable designate energies of microstates, so Gibbs's formula for the entropy is formally identical to Shannon's formula.
Hasil analogi ketika nilai-nilai variabel acak menunjukkan energi keadaan mikro, sehingga rumus Gibbs untuk entropi secara formal identik dengan rumus Shannon.
Entropy has relevance to other areas of mathematics such as combinatorics and machine learning.
Entropi memiliki relevansi dengan bidang matematika lainnya seperti kombinatorika dan pembelajaran mesin.
The definition can be derived from a set of axioms establishing that entropy should be a measure of how informative the average outcome of a variable is.
Definisi tersebut dapat diturunkan dari serangkaian aksioma yang menetapkan bahwa entropi harus menjadi ukuran seberapa "mengejutkan" hasil rata-rata suatu variabel.
For a continuous random variable, differential entropy is analogous to entropy.
Untuk variabel acak kontinu, entropi diferensial analog dengan entropi.
The Shannon entropy is restricted to random variables taking discrete values.
Untuk variabel acak kontinu, entropi diferensial analog dengan entropi.
This is the differential entropy (or continuous entropy).
Untuk variabel acak kontinu, entropi diferensial analog dengan entropi.
Another useful measure of entropy that works equally well in the discrete and the continuous case is the relative entropy of a distribution.
Untuk variabel acak kontinu, entropi diferensial analog dengan entropi.
Cover, T.M., Thomas, J.A. (2006), Elements of Information Theory – 2nd Ed., Wiley-Interscience, ISBN 978-0-471-24195-9 MacKay, D.J.C. (2003), Information Theory, Inference and Learning Algorithms, Cambridge University Press, ISBN 978-0-521-64298-9 Arndt, C. (2004), Information Measures: Information and its Description in Science and Engineering, Springer, ISBN 978-3-540-40855-0 Gray, R. M. (2011), Entropy and Information Theory, Springer.
Cover, T.M., Thomas, J.A. (2006), Elements of Information Theory - 2nd Ed., Wiley-Interscience, ISBN 978-0-471-24195-9 MacKay, D.J.C. (2003), Information Theory, Inference and Learning Algorithms , Cambridge University Press, ISBN 978-0-521-64298-9 Arndt, C. (2004), Information Measures: Information and its Description in Science and Engineering, Springer, ISBN 978-3-540-40855-0 Gray, R. M. (2011), Entropy and Information Theory, Springer.
ISBN 0-252-72548-4 Stone, J. V. (2014), Chapter 1 of Information Theory: A Tutorial Introduction Archived 3 June 2016 at the Wayback Machine, University of Sheffield, England.
ISBN 0-252-72548-4ISBN 0-252-72548-4 Stone, J. V. (2014), Chapter 1 of Information Theory: A Tutorial Introduction, University of Sheffield, England.
"Entropy", Encyclopedia of Mathematics, EMS Press, 2001 [1994] "Entropy" Archived 4 June 2016 at the Wayback Machine at Rosetta Code—repository of implementations of Shannon entropy in different programming languages.
[1994], "Entropy", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4 "Entropy" at Rosetta Code—repository of implementations of Shannon entropy in different programming languages.