Repository: cfug/dio (dart)

Issue:
Opportunity to improve the json decoding performance
### Request Statement

Hi there!

I experimented a bit with the hidden `_JsonUtf8Decoder` which is created when [fusing the `utf8` decoder with the `json` decoder](https://github.com/dart-lang/sdk/blob/5b2ea0c7a227d91c691d2ff8cbbeb5f7f86afdb9/sdk/lib/_internal/vm/lib/convert_patch.dart#L40).

This significantly improves the decoding performance.

I created a benchmark [here](https://github.com/knaeckeKami/dio_benchmark).

It shows an improvement of around ~10x in AOT mode on sufficiently large json responses. (around 64 KiB).
 
Here's the improved Transformer:

```dart
import 'dart:convert';
import 'dart:typed_data';

import 'package:dio/dio.dart';

class UTF8JsonTransformer extends SyncTransformer {
  /// on VM, this creates a special fast decoder.
  /// see https://github.com/dart-lang/sdk/blob/5b2ea0c7a227d91c691d2ff8cbbeb5f7f86afdb9/sdk/lib/_internal/vm/lib/convert_patch.dart#L54
  final decoder = const Utf8Decoder().fuse(const JsonDecoder());

  UTF8JsonTransformer();

  @override
  Future<dynamic> transformResponse(
    RequestOptions options,
    ResponseBody responseBody,
  ) async {
    final responseType = options.responseType;
    // Do not handled the body for streams.
    if (responseType == ResponseType.stream) {
      return responseBody;
    }

    // Return the finalized bytes if the response type is bytes.
    if (responseType == ResponseType.bytes) {
      final chunks = await responseBody.stream.toList();
      final responseBytes =
          Uint8List.fromList(chunks.expand((c) => c).toList());
      return responseBytes;
    }

    final isJsonContent = Transformer.isJsonMimeType(
      responseBody.headers[Headers.contentTypeHeader]?.first,
    );

    if (isJsonContent) {
      final stream = responseBody.stream;
      final decodedStream = decoder.bind(stream);
      final decoded = await decodedStream.toList();
      assert(decoded.length == 1);
      return decoded.first;
    }

    // If the response is not JSON, we should return the response as a string

    final chunks = await responseBody.stream.toList();
    final responseBytes = Uint8List.fromList(chunks.expand((c) => c).toList());

    return utf8.decode(responseBytes, allowMalformed: true);
  }
}
```

Note: you need a recent version of Dart to see these performance improvements.

This would be a nice improvement for Dio. Who doesn't like free improved performance?

A caveat is, that this is not compatible with the default `BackgroundTransformer`,  as the callback in this transformer takes a String, not a Uint8List. 
To get the benefits there, a change in the API would be required (or we could add a new `BackgroundTransformer` class). 

Would you be open to a PR that integrates these changes in the existing SyncTransformer? 





### Solution Brainstorm

_No response_

Pull request:
<!-- Write down your pull request descriptions. -->

### New Pull Request Checklist

- [x] I have searched for a similar pull request in the [project](https://github.com/cfug/dio/pulls) and found none
- [x] I have read the [Documentation](https://pub.dev/documentation/dio/latest/)
- [x] I have updated this branch with the latest `main` branch to avoid conflicts (via merge from master or rebase)
- [x] I have added the required tests to prove the fix/feature I'm adding
- [x] I have updated the documentation (if necessary)
- [x] I have run the tests without failures
- [x] I have updated the `CHANGELOG.md` in the corresponding package

### Additional context and info (if any)

fixes #2238 

<!-- Provide more context and info about the PR. -->

This PR adds the `Utf8JsonTransformer`.

I made sure that it is as close to `BackgroundTransformer` as possible.
The only behavior change is that is checks to Content-Length header for deciding whether to switch to a background isolate for decoding or not.
If the Content-Lenght header is not set or not valid, it will fall back to counting the bytes in the response.

This is because it's slightly faster to feed to the response stream directly into the decoder without consolidating all the response bytes into a single Uint8List first. In case the server does not send a valid Content-Length header, the Transformer will check the actual bytes from the response, as the `BackgroundTransformer` does.


The remaining question is if we should set the Transformer as default. 
IMO it would make sense to change the default transformer to

```dart
  /// The default [Transformer] that transfers requests and responses
  /// into corresponding content to send.
  @override
  Transformer transformer = Utf8JsonTransformer(contentLengthIsolateThreshold: 50 * 1024);
```

so all users get the improved performance by default.

Patch:
diff --git a/dio/CHANGELOG.md b/dio/CHANGELOG.md
index abeaeb470..262d05877 100644
--- a/dio/CHANGELOG.md
+++ b/dio/CHANGELOG.md
@@ -9,6 +9,7 @@ See the [Migration Guide][] for the complete breaking changes list.**
 - Add constructor for `DioExceptionType.badCertificate`.
 - Create type alias `DioMediaType` for `http_parser`'s `MediaType`.
 - Fix the type conversion regression when using `MultipartFile.fromBytes`.
+- Add FusedTransformer for improved performance when decoding JSON.
 - Improves `InterceptorState.toString()`.
 
 ## 5.4.3+1
diff --git a/dio/lib/dio.dart b/dio/lib/dio.dart
index 60c9044c8..a7ae5da00 100644
--- a/dio/lib/dio.dart
+++ b/dio/lib/dio.dart
@@ -20,4 +20,5 @@ export 'src/redirect_record.dart';
 export 'src/response.dart';
 export 'src/transformer.dart';
 export 'src/transformers/background_transformer.dart';
+export 'src/transformers/fused_transformer.dart';
 export 'src/transformers/sync_transformer.dart';
diff --git a/dio/lib/src/transformer.dart b/dio/lib/src/transformer.dart
index a70ff80fb..d7dfc087d 100644
--- a/dio/lib/src/transformer.dart
+++ b/dio/lib/src/transformer.dart
@@ -5,6 +5,12 @@ import 'adapter.dart';
 import 'options.dart';
 import 'utils.dart';
 
+/// The callback definition for decoding a JSON string.
+typedef JsonDecodeCallback = FutureOr<dynamic> Function(String);
+
+/// The callback definition for encoding a JSON object.
+typedef JsonEncodeCallback = FutureOr<String> Function(Object);
+
 /// [Transformer] allows changes to the request/response data before
 /// it is sent/received to/from the server.
 ///
@@ -73,4 +79,28 @@ abstract class Transformer {
         mediaType.mimeType == 'text/json' ||
         mediaType.subtype.endsWith('+json');
   }
+
+  static FutureOr<String> defaultTransformRequest(
+    RequestOptions options,
+    JsonEncodeCallback jsonEncodeCallback,
+  ) {
+    final Object data = options.data ?? '';
+    if (data is! String && Transformer.isJsonMimeType(options.contentType)) {
+      return jsonEncodeCallback(data);
+    } else if (data is Map) {
+      if (data is Map<String, dynamic>) {
+        return Transformer.urlEncodeMap(data, options.listFormat);
+      }
+      debugLog(
+        'The data is a type of `Map` (${data.runtimeType}), '
+        'but the transformer can only encode `Map<String, dynamic>`.\n'
+        'If you are writing maps using `{}`, '
+        'consider writing `<String, dynamic>{}`.',
+        StackTrace.current,
+      );
+      return data.toString();
+    } else {
+      return data.toString();
+    }
+  }
 }
diff --git a/dio/lib/src/transformers/fused_transformer.dart b/dio/lib/src/transformers/fused_transformer.dart
new file mode 100644
index 000000000..b3ea6d7f0
--- /dev/null
+++ b/dio/lib/src/transformers/fused_transformer.dart
@@ -0,0 +1,185 @@
+import 'dart:convert';
+import 'dart:typed_data';
+
+import '../adapter.dart';
+import '../compute/compute.dart';
+import '../headers.dart';
+import '../options.dart';
+import '../transformer.dart';
+
+/// A [Transformer] that has a fast path for decoding UTF8-encoded JSON.
+/// If the response is utf8-encoded JSON and no custom decoder is specified in the [RequestOptions], this transformer
+/// is significantly faster than the default [SyncTransformer] and the [BackgroundTransformer].
+/// This improvement is achieved by using a fused [Utf8Decoder] and [JsonDecoder] to decode the response,
+/// which is faster than decoding the utf8-encoded JSON in two separate steps, since
+/// Dart uses a special fast decoder for this case.
+/// See https://github.com/dart-lang/sdk/blob/5b2ea0c7a227d91c691d2ff8cbbeb5f7f86afdb9/sdk/lib/_internal/vm/lib/convert_patch.dart#L40
+///
+/// By default, this transformer will transform responses in the main isolate,
+/// but a custom threshold can be set to switch to an isolate for large responses by passing
+/// [contentLengthIsolateThreshold].
+class FusedTransformer extends Transformer {
+  FusedTransformer({this.contentLengthIsolateThreshold = -1});
+
+  /// Always decode the response in the same isolate
+  factory FusedTransformer.sync() =>
+      FusedTransformer(contentLengthIsolateThreshold: -1);
+
+  // whether to switch decoding to an isolate for large responses
+  // set to -1 to disable, 0 to always use isolate
+  final int contentLengthIsolateThreshold;
+
+  static final _utf8JsonDecoder = const Utf8Decoder().fuse(const JsonDecoder());
+
+  @override
+  Future<String> transformRequest(RequestOptions options) async {
+    return Transformer.defaultTransformRequest(options, jsonEncode);
+  }
+
+  @override
+  Future<dynamic> transformResponse(
+    RequestOptions options,
+    ResponseBody responseBody,
+  ) async {
+    final responseType = options.responseType;
+    // Do not handle the body for streams.
+    if (responseType == ResponseType.stream) {
+      return responseBody;
+    }
+
+    // Return the finalized bytes if the response type is bytes.
+    if (responseType == ResponseType.bytes) {
+      return _consolidateStream(responseBody.stream);
+    }
+
+    final isJsonContent = Transformer.isJsonMimeType(
+      responseBody.headers[Headers.contentTypeHeader]?.first,
+    );
+
+    final customResponseDecoder = options.responseDecoder;
+
+    // No custom decoder was specified for the response,
+    // and the response is json -> use the fast path decoder
+    if (isJsonContent && customResponseDecoder == null) {
+      return _fastUtf8JsonDecode(responseBody);
+    }
+    final responseBytes = await _consolidateStream(responseBody.stream);
+
+    // A custom response decoder overrides the default behavior
+    final String? decodedResponse;
+
+    if (customResponseDecoder != null) {
+      final decodeResponse = customResponseDecoder(
+        responseBytes,
+        options,
+        responseBody..stream = const Stream.empty(),
+      );
+
+      if (decodeResponse is Future) {
+        decodedResponse = await decodeResponse;
+      } else {
+        decodedResponse = decodeResponse;
+      }
+    } else {
+      decodedResponse = null;
+    }
+
+    if (isJsonContent && decodedResponse != null) {
+      // slow path decoder, since there was a custom decoder specified
+      return jsonDecode(decodedResponse);
+    } else if (decodedResponse != null) {
+      return decodedResponse;
+    } else {
+      // If the response is not JSON and no custom decoder was specified,
+      // assume it is an utf8 string
+      return utf8.decode(
+        responseBytes,
+        allowMalformed: true,
+      );
+    }
+  }
+
+  Future<Object?> _fastUtf8JsonDecode(ResponseBody responseBody) async {
+    final contentLengthHeader =
+        responseBody.headers[Headers.contentLengthHeader];
+
+    final hasContentLengthHeader =
+        contentLengthHeader != null && contentLengthHeader.isNotEmpty;
+
+    // The content length of the response, either from the content-length header
+    // of the response or the length of the eagerly decoded response bytes
+    final int contentLength;
+
+    // The eagerly decoded response bytes
+    // which is set if the content length is not specified and
+    // null otherwise (we'll feed the stream directly to the decoder in that case)
+    Uint8List? responseBytes;
+
+    // If the content length is not specified, we need to consolidate the stream
+    // and count the bytes to determine if we should use an isolate
+    // otherwise we use the content length header
+    if (!hasContentLengthHeader) {
+      responseBytes = await _consolidateStream(responseBody.stream);
+      contentLength = responseBytes.length;
+    } else {
+      contentLength = int.parse(contentLengthHeader.first);
+    }
+
+    // The decoding in done on an isolate if
+    // - contentLengthIsolateThreshold is not -1
+    // - the content length, calculated from either
+    //   the content-length header if present or the eagerly decoded response bytes,
+    //   is greater than or equal to contentLengthIsolateThreshold
+    final shouldUseIsolate = !(contentLengthIsolateThreshold < 0) &&
+        contentLength >= contentLengthIsolateThreshold;
+    if (shouldUseIsolate) {
+      // we can't send the stream to the isolate, so we need to decode the response bytes first
+      return compute(
+        _decodeUtf8ToJson,
+        responseBytes ?? await _consolidateStream(responseBody.stream),
+      );
+    } else {
+      if (!hasContentLengthHeader || contentLength == 0) {
+        // This path is for backwards compatibility.
+        // If content-type indicates a json response,
+        // but the body is empty, null is returned.
+        // _utf8JsonDecoder.bind(responseBody.stream) would throw if the body is empty.
+        // So we need to check if the body is empty and return null in that case
+        responseBytes ??= await _consolidateStream(responseBody.stream);
+        if (responseBytes.isEmpty) {
+          return null;
+        }
+        return _utf8JsonDecoder.convert(responseBytes);
+      } else {
+        assert(responseBytes == null);
+        // The content length is specified and we can feed the stream directly to the decoder,
+        // without eagerly decoding the response bytes first.
+        final decodedStream = _utf8JsonDecoder.bind(responseBody.stream);
+        final decoded = await decodedStream.toList();
+        if (decoded.isEmpty) {
+          return null;
+        }
+        assert(decoded.length == 1);
+        return decoded.first;
+      }
+    }
+  }
+
+  static Future<Object?> _decodeUtf8ToJson(Uint8List data) async {
+    if (data.isEmpty) {
+      return null;
+    }
+    return _utf8JsonDecoder.convert(data);
+  }
+}
+
+/// Consolidates a stream of [Uint8List] into a single [Uint8List]
+Future<Uint8List> _consolidateStream(Stream<Uint8List> stream) async {
+  final builder = BytesBuilder(copy: false);
+
+  await for (final chunk in stream) {
+    builder.add(chunk);
+  }
+
+  return builder.takeBytes();
+}
diff --git a/dio/lib/src/transformers/sync_transformer.dart b/dio/lib/src/transformers/sync_transformer.dart
index 5129a516b..12ce3585e 100644
--- a/dio/lib/src/transformers/sync_transformer.dart
+++ b/dio/lib/src/transformers/sync_transformer.dart
@@ -6,17 +6,10 @@ import '../adapter.dart';
 import '../headers.dart';
 import '../options.dart';
 import '../transformer.dart';
-import '../utils.dart';
 
 @Deprecated('Use BackgroundTransformer instead')
 typedef DefaultTransformer = SyncTransformer;
 
-/// The callback definition for decoding a JSON string.
-typedef JsonDecodeCallback = FutureOr<dynamic> Function(String);
-
-/// The callback definition for encoding a JSON object.
-typedef JsonEncodeCallback = FutureOr<String> Function(Object);
-
 /// If you want to custom the transformation of request/response data,
 /// you can provide a [Transformer] by your self, and replace
 /// the transformer by setting the [Dio.transformer].
@@ -31,24 +24,7 @@ class SyncTransformer extends Transformer {
 
   @override
   Future<String> transformRequest(RequestOptions options) async {
-    final Object data = options.data ?? '';
-    if (data is! String && Transformer.isJsonMimeType(options.contentType)) {
-      return jsonEncodeCallback(data);
-    } else if (data is Map) {
-      if (data is Map<String, dynamic>) {
-        return Transformer.urlEncodeMap(data, options.listFormat);
-      }
-      debugLog(
-        'The data is a type of `Map` (${data.runtimeType}), '
-        'but the transformer can only encode `Map<String, dynamic>`.\n'
-        'If you are writing maps using `{}`, '
-        'consider writing `<String, dynamic>{}`.',
-        StackTrace.current,
-      );
-      return data.toString();
-    } else {
-      return data.toString();
-    }
+    return Transformer.defaultTransformRequest(options, jsonEncodeCallback);
   }
 
   @override


Test patch:
diff --git a/dio/test/transformer_test.dart b/dio/test/transformer_test.dart
index 2bd563ee7..5c8aa748c 100644
--- a/dio/test/transformer_test.dart
+++ b/dio/test/transformer_test.dart
@@ -1,3 +1,6 @@
+import 'dart:convert';
+import 'dart:typed_data';
+
 import 'package:dio/dio.dart';
 import 'package:test/test.dart';
 
@@ -54,4 +57,201 @@ void main() {
     );
     expect(jsonResponse, null);
   });
+
+  group(FusedTransformer(), () {
+    test(
+        'transformResponse transforms json without content-length set in response',
+        () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.json),
+        ResponseBody.fromString(
+          '{"foo": "bar"}',
+          200,
+          headers: {
+            Headers.contentTypeHeader: ['application/json'],
+          },
+        ),
+      );
+      expect(response, {'foo': 'bar'});
+    });
+
+    test('transformResponse transforms json with content-length', () async {
+      final transformer = FusedTransformer();
+      const jsonString = '{"foo": "bar"}';
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.json),
+        ResponseBody.fromString(
+          jsonString,
+          200,
+          headers: {
+            Headers.contentTypeHeader: ['application/json'],
+            Headers.contentLengthHeader: [
+              utf8.encode(jsonString).length.toString(),
+            ],
+          },
+        ),
+      );
+      expect(response, {'foo': 'bar'});
+    });
+
+    test('transforms json in background isolate', () async {
+      final transformer = FusedTransformer(contentLengthIsolateThreshold: 0);
+      final jsonString = '{"foo": "bar"}';
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.json),
+        ResponseBody.fromString(
+          jsonString,
+          200,
+          headers: {
+            Headers.contentTypeHeader: ['application/json'],
+            Headers.contentLengthHeader: [
+              utf8.encode(jsonString).length.toString(),
+            ],
+          },
+        ),
+      );
+      expect(response, {'foo': 'bar'});
+    });
+
+    test('transformResponse transforms that arrives in many chunks', () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.json),
+        ResponseBody(
+          Stream.fromIterable(
+              /* utf-8 encoding of {"foo": "bar"} */
+              [
+                Uint8List.fromList([123]),
+                Uint8List.fromList([34]),
+                Uint8List.fromList([102, 111, 111]),
+                Uint8List.fromList([34]),
+                Uint8List.fromList([58]),
+                Uint8List.fromList([34]),
+                Uint8List.fromList([98, 97, 114]),
+                Uint8List.fromList([34]),
+                Uint8List.fromList([125]),
+              ]),
+          200,
+          headers: {
+            Headers.contentTypeHeader: ['application/json'],
+          },
+        ),
+      );
+      expect(response, {'foo': 'bar'});
+    });
+
+    test('transformResponse handles bytes', () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.bytes),
+        ResponseBody.fromBytes(
+          [1, 2, 3],
+          200,
+        ),
+      );
+      expect(response, [1, 2, 3]);
+    });
+
+    test('transformResponse handles when response stream has multiple chunks',
+        () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.bytes),
+        ResponseBody(
+          Stream.fromIterable([
+            Uint8List.fromList([1, 2, 3]),
+            Uint8List.fromList([4, 5, 6]),
+            Uint8List.fromList([7, 8, 9]),
+          ]),
+          200,
+        ),
+      );
+      expect(response, [1, 2, 3, 4, 5, 6, 7, 8, 9]);
+    });
+
+    test('transformResponse handles plain text', () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.plain),
+        ResponseBody.fromString(
+          'plain text',
+          200,
+          headers: {
+            Headers.contentTypeHeader: ['text/plain'],
+          },
+        ),
+      );
+      expect(response, 'plain text');
+    });
+
+    test('transformResponse handles streams', () async {
+      final transformer = FusedTransformer();
+      final response = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.stream),
+        ResponseBody.fromBytes(
+          [1, 2, 3],
+          200,
+        ),
+      );
+      expect(response, isA<ResponseBody>());
+    });
+
+    test('null response body only when the response is JSON', () async {
+      final transformer = FusedTransformer();
+      for (final responseType in ResponseType.values) {
+        final response = await transformer.transformResponse(
+          RequestOptions(responseType: responseType),
+          ResponseBody.fromBytes([], 200),
+        );
+        switch (responseType) {
+          case ResponseType.json:
+          case ResponseType.plain:
+            expect(response, '');
+            break;
+          case ResponseType.stream:
+            expect(response, isA<ResponseBody>());
+            break;
+          case ResponseType.bytes:
+            expect(response, []);
+            break;
+          default:
+            throw AssertionError('Unknown response type: $responseType');
+        }
+      }
+      final jsonResponse = await transformer.transformResponse(
+        RequestOptions(responseType: ResponseType.json),
+        ResponseBody.fromBytes(
+          [],
+          200,
+          headers: {
+            Headers.contentTypeHeader: [Headers.jsonContentType],
+          },
+        ),
+      );
+      expect(jsonResponse, null);
+    });
+
+    test('transform the request using urlencode', () async {
+      final transformer = FusedTransformer();
+
+      final request = await transformer.transformRequest(
+        RequestOptions(responseType: ResponseType.json, data: {'foo': 'bar'}),
+      );
+      expect(request, 'foo=bar');
+    });
+
+    test('transform the request using json', () async {
+      final transformer = FusedTransformer();
+
+      final request = await transformer.transformRequest(
+        RequestOptions(
+          responseType: ResponseType.json,
+          data: {'foo': 'bar'},
+          headers: {'Content-Type': 'application/json'},
+        ),
+      );
+      expect(request, '{"foo":"bar"}');
+    });
+  });
 }