Random Information Theory Fact of the Day
The amount of information associated with the outcome of a random variable can be expressed as bits (aka “digits”), nats (aka “dits”), or hartleys (aka “bans”), depending on the base of the logarithm used in the calculation of the self-information of the random variable’s outcome. (Base 2, e, or 10 respectively.)
The self-information of outcome x is calculated as: I(x) = -log P(x)
For example. each side of a 10 sided die contains 3.32 bits, or 2.3 nats, or 1 ban, of information.
Related Links:
Self-Information on Wikipedia
