Repository: flutter
Issue: Provide a way to schedule a callback after widgets of the current frame are visible
Labels: c: new feature,framework,engine,c: proposal,P3,team-engine,triaged-engine

Internal bug: b/203035704

Internally, there are core 1P host-side APIs that support reporting when the app is *interactive* for instrumentation of startup latency. These APIs are usually called when an interactive frame is viaible, and it will take care of recording the instant of the call. The collected metric is standardized across all mobile apps. It looks something like this:

```
void markAppInteractive();
```

In the context of Flutter 1P apps, a mechanism is needed to call this method when a particular set of widgets are *visible* to the user. Consider this toy example, where the app needs to mark that it is interactive after the "Interactive!" text has been displayed after an async suspension.

```dart
import 'package:flutter/material.dart';

void main() => runApp(
      MaterialApp(
        home: ExampleWithLoadingFrame(),
      ),
    );

/// Simulated.
Future<String> asyncLoad() async {
  await Future.delayed(const Duration(seconds: 1));
  return 'foo';
}

// 1st frame: "Loading..."
// 2nd frame: "Interactive!"
class ExampleWithLoadingFrame extends StatelessWidget {
  final _future = asyncLoad();

  @override
  Widget build(BuildContext context) {
    return FutureBuilder<String>(
      future: _future,
      builder: (_, AsyncSnapshot<String> snapshot) {
        if (snapshot.hasData) {
          // Need some mechanism to report that the app is interactive after this
          // widget is visible on screen.
          return Text('Interactive! ${snapshot.data}');
        } else {
          return const Text('Loading...');
        }
      },
    );
  }
}
```

## Existing Related APIs

There are a few related APIs that are inadequate:

### Host-side Embedding APIs

On Android, there is [`onFlutterUiDisplayed`](https://github.com/flutter/engine/blob/2962099077b36704071802ea5595f4a016a1c214/shell/platform/android/io/flutter/embedding/engine/renderer/FlutterUiDisplayListener.java#L18) and on iOS there is [`setFlutterViewDidRenderCallback`](https://github.com/flutter/engine/blob/260d9797263be10dd8320a28535c15f5f57032cf/shell/platform/darwin/ios/framework/Headers/FlutterViewController.h#L117). These APIs are insufficient because the interactive frame is not necessarily the *first* frame, as seen in the example above.

### `addPostFrameCallback`

[`addPostFrameCallback`](https://api.flutter.dev/flutter/scheduler/SchedulerBinding/addPostFrameCallback.html) schedules a callback that is executed after the build phase on the UI thread. This gives an approximation of when the widgets are visible, but notably does not include the raster duration for those widgets.

### `FrameTiming`

Using the `FrameTiming` API from [`addTimingsCallback`](https://api.flutter.dev/flutter/scheduler/SchedulerBinding/addTimingsCallback.html), it is possible to collect the raster duration of the upcoming frame. Rough implementation:

```dart
/// Marks the upcoming frame as when the app is interactive.
///
/// The metric will be recorded after roughly two seconds.
void markAppInteractiveAfterThisFrame() async {
  final binding = WidgetsBinding.instance;

  final allFrameTimings = <FrameTiming>[];

  late final TimingsCallback timingsCallback;
  timingsCallback = (List<FrameTiming> currentFrameTimings) {
    allFrameTimings.addAll(currentFrameTimings);
  };
  binding.addTimingsCallback(timingsCallback);

  final nextFrameNumber =
      binding.platformDispatcher.frameData.frameNumber + 1;

  // FrameTimings are batched and reported approximately once per second in
  // release mode, so add a delay to avoid missing those events. See
  // [SchedulerBinding.addTimingsCallback].
  // TODO(flutter/flutter#64808): Remove this once timings can be flushed.
  await Future.delayed(const Duration(seconds: 2));
  binding.removeTimingsCallback(timingsCallback);

  final interactiveFrameTiming = allFrameTimings.firstWhere(
    (frameTiming) => frameTiming.frameNumber == nextFrameNumber,
  );

  final interactiveFrameRasterMs =
      interactiveFrameTiming.rasterDuration.inMilliseconds;

  recordDuration(
    'InteractiveFrameRasterDurationMillis',
    interactiveFrameRasterMs,
  );
}
```

It is also inadequate as these durations are provided *after* the raster happens, but the 1P APIs take care of recording the instant and must be called as close to when the widgets are visible as possible.

It is also unclear how much will build / raster jank affect this implementation.

## Proposal

Similar to `addPostFrameCallback`, Flutter can provide a `addPostFrameRasterCallback`. This callback will be scheduled from the raster thread to the UI thread after rasterization of the current frame.

A potential issue is that the UI thread could be blocked when this callback is scheduled. This inaccuracy could be acceptable if the delta is just a few frames. Internally, an additional metric can also be collected using a combination of `addPostFrameCallback` + raster duration of the last frame (from `FrameTiming`) to measure how much this inaccuracy is.

cc @xster @dnfield