If this is causing to problems feel free to omit the base32 test. As long as we have round-tripped some binary data for any encoding I'm happy that the system is working. Configuring Jackson for base32 (if that is your chosen encoding) is left as an exercise for the user.

It's possible to override the byte[] serializer/deserializer in Jackson without using @JsonSerialize by registering a module with the ObjectMapper. This is probably how I'd suggest people should do it as they then don't need a  custom annotator, unless they need to support many binary encodings in a single system, which I think is highly unlikely.