mp3enc: fix a triggerable assert

We have to check against the number of bytes actually needed, not the
theoretical maximum size.

(cherry picked from commit 12700b0219521a5f20c8ba47b3ad7857ea9e0554)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 871d99ef77336069e5a8ece947c8160d9bc4d5ea)
Signed-off-by: Anton Khirnov <anton@khirnov.net>