Entropy
/ˈɛntrəpi/en·tro·pynoun
Definition
1.[in information theory] a measure of the uncertainty or randomness inherent in a random variable or source of data.
The high entropy of the source indicated that the data was highly unpredictable.
Formal statement
H(X) = -sum(p(x_i) log_2 p(x_i))The base-2 logarithm yields the unit of bits.
Etymology
From Greek entropein, meaning 'to transform' or 'to change', adopted into thermodynamics by Rudolf Clausius.
Synonyms
- disordersense 1 · Near
References
- Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory, 2nd ed., ch. 2.Wiley.