p for though your symbol is p i but the code length we assign is based on p hat a lie so then this is our expected code length simply don't need to round up to get no natural numbers for the encoding so simple calculation then we add and subtract the same quantity now this quantity over here in the front we recognize as KL divergence the thing in the back we recognize as entropy so we see the expected coloured length when we use a distribution estimate P hat is going to be the entropy plus we know it's always going to be more any