Repository: flutter
Issue: [Proposal] keep ui and raster thread on fast CPUs, keep work threads on slow CPUs
Labels: platform-android,engine,c: proposal,P2,team-android,triaged-android

On many mobile devices, the L1 caches reside on specific CPUs, and L2 caches reside on the set of CPUs that share a clock.
Keeping ui and raster thread on fast CPUs will help to maximize L2 cache hits and operate at faster speed.
There is a trade off to this as we will incur more power usage when running on fast CPUs.

See also:
https://developer.android.com/games/optimize?hl=zh-tw#keep-mem-heavy-threads-one-cpu
https://developer.android.com/games/optimize?hl=zh-tw#defer-work-to-slow-cpus

I create this [PR](https://github.com/flutter/engine/pull/29909) as a proof of concept implementation base on [swappy](https://android.googlesource.com/platform/frameworks/opt/gamesdk/+/master/src/swappy/common).

Here's the flutter_gallery__transitions_perf benchmark report on a HUAWEI ELS AN00 phone

Before
```json
 "average_frame_build_time_millis": 6.579668269230771,
  "90th_percentile_frame_build_time_millis": 10.818,
  "99th_percentile_frame_build_time_millis": 23.7,
  "worst_frame_build_time_millis": 140.948,
  "missed_frame_build_budget_count": 9,
  "average_frame_rasterizer_time_millis": 16.873414457831323,
  "90th_percentile_frame_rasterizer_time_millis": 20.421,
  "99th_percentile_frame_rasterizer_time_millis": 53.492,
  "worst_frame_rasterizer_time_millis": 104.351,
  "missed_frame_rasterizer_budget_count": 181,
  "frame_count": 416,
  "frame_rasterizer_count": 415,

  "average_frame_build_time_millis": 6.523223277909746,
  "90th_percentile_frame_build_time_millis": 10.393,
  "99th_percentile_frame_build_time_millis": 33.673,
  "worst_frame_build_time_millis": 124.373,
  "missed_frame_build_budget_count": 10,
  "average_frame_rasterizer_time_millis": 16.824433333333342,
  "90th_percentile_frame_rasterizer_time_millis": 19.946,
  "99th_percentile_frame_rasterizer_time_millis": 52.629,
  "worst_frame_rasterizer_time_millis": 101.326,
  "missed_frame_rasterizer_budget_count": 177,
  "frame_count": 421,
  "frame_rasterizer_count": 420,
```

After
```json
 "average_frame_build_time_millis": 4.494899521531101,
  "90th_percentile_frame_build_time_millis": 8.089,
  "99th_percentile_frame_build_time_millis": 14.804,
  "worst_frame_build_time_millis": 107.017,
  "missed_frame_build_budget_count": 4,
  "average_frame_rasterizer_time_millis": 17.21504086538461,
  "90th_percentile_frame_rasterizer_time_millis": 21.238,
  "99th_percentile_frame_rasterizer_time_millis": 56.181,
  "worst_frame_rasterizer_time_millis": 87.671,
  "missed_frame_rasterizer_budget_count": 186,
  "frame_count": 418,
  "frame_rasterizer_count": 416,

  "average_frame_build_time_millis": 4.61509478672986,
  "90th_percentile_frame_build_time_millis": 8.691,
  "99th_percentile_frame_build_time_millis": 15.92,
  "worst_frame_build_time_millis": 123.315,
  "missed_frame_build_budget_count": 4,
  "average_frame_rasterizer_time_millis": 16.785314285714286,
  "90th_percentile_frame_rasterizer_time_millis": 20.222,
  "99th_percentile_frame_rasterizer_time_millis": 52.744,
  "worst_frame_rasterizer_time_millis": 101.359,
  "missed_frame_rasterizer_budget_count": 181,
  "frame_count": 422,
  "frame_rasterizer_count": 420,
```
We can see the obvious optimization of frame_build_time, but no  effect of frame_rasterizer_time.
I use perfetto to see which cpu the thread runs on, before the ui thread runs on slow cpus, and after it runs on  fast cpus.