mp3enc: fix a triggerable assert

We have to check against the number of bytes actually needed, not the
theoretical maximum size.

(cherry picked from commit 12700b0219521a5f20c8ba47b3ad7857ea9e0554)
Signed-off-by: Anton Khirnov <anton@khirnov.net>