Ваш формат не совместим с описанным в задании.
> The <b>first two bytes</b> are read, starting from the current file pointer, as if by <b>readUnsignedShort</b>. This value gives the number of following bytes that are in the encoded string, not the length of the resulting string. The following bytes are then interpreted as bytes encoding characters in the modified UTF-8 format and are converted into characters.

Сравните:
https://github.com/dkomanov/fizteh-java-2013/blob/master/tasks/03-FileMap.md