Only base64-encode binary result when formatting as JSON, as the other formats are binary safe