Repository: flutter
Issue: Integration tests to produce useful failure messages
Labels: c: new feature,tool,t: flutter driver,c: proposal,P3,team-tool,triaged-tool

## Use case

I followed the cookbook article ["An introduction to integration testing"](https://flutter.dev/docs/cookbook/testing/integration/introduction) and wrote a test.  The test failed.  The log says "get_text message is taking a long time to complete", "Test timed out after 30 seconds", and "Failed to fulfill GetText due to remote error".  It does not say what widget it was waiting for.  It does not say the line number of the error.  How is an engineer supposed to debug a Flutter integration test that will have tens of such calls?

CC: Flutter PM @timsneath

Steps to reproduce:
1. `flutter create example3`
2. Write these files:

<details><summary> pubspec.yaml </summary>

```yaml
# pubspec.yaml
name: example3
description: A new Flutter project.
publish_to: 'none'
version: 1.0.0+1
environment:
  sdk: ">=2.12.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter

dev_dependencies:
  flutter_driver:
    sdk: flutter
  test: any

flutter:
```

</details>

<details><summary>lib/example_main.dart</summary>

```dart
// lib/example_main.dart
import 'package:flutter/cupertino.dart'
    show CupertinoApp, CupertinoPage, CupertinoPageScaffold, runApp;
import 'package:flutter/widgets.dart'
    show
        BuildContext,
        Center,
        Navigator,
        StatelessWidget,
        Text,
        ValueKey,
        Widget;

class _MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext ctx) => CupertinoApp(
        home: Navigator(
          onPopPage: (_route, _result) => true,
          pages: [
            CupertinoPage(
              title: "Home",
              child: CupertinoPageScaffold(
                child: Center(
                  child: Text("content1",
                      key: const ValueKey('content-key-value')),
                ),
              ),
            )
          ],
        ),
      );
}

void main() {
  runApp(_MyApp());
}
```

</details>

<details><summary>test_driver/app.dart</summary>

```dart
// test_driver/app.dart
import 'package:example3/example_main.dart' as example_main;
import 'package:flutter_driver/driver_extension.dart'
    show enableFlutterDriverExtension;

void main() {
  enableFlutterDriverExtension();
  example_main.main();
}
```

</details>

<details><summary>test_driver/app_test.dart</summary>

```dart
// test_driver/app_test.dart
import 'package:flutter_driver/flutter_driver.dart' show find, FlutterDriver;
import 'package:test/test.dart' show expect, group, setUpAll, tearDownAll, test;

void main() {
  group('test-group', () {
    late FlutterDriver driver;
    setUpAll(() async {
      driver = await FlutterDriver.connect();
      await driver.waitUntilFirstFrameRasterized();
    });
    tearDownAll(() async {
      driver.close();
    });
    test('test', () async {
      expect(await driver.getText(find.byValueKey('nonexistent')), "content1");
    });
  });
}
```

</details>

3. Run `flutter drive`
<details><summary>flutter drive output</summary>

```
% flutter drive --device-id test-sim --target=test_driver/app.dart
Running "flutter pub get" in example3...                           952ms
Running Xcode build...                                                  
 └─Compiling, linking and signing...                      2,322ms
Xcode build done.                                            6.6s
00:00 +0: test-group (setUpAll)

VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:58511/TOXO8tPOU-k=/
VMServiceFlutterDriver: Isolate found with number: 277489066499947
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
VMServiceFlutterDriver: Connected to Flutter application.
00:01 +0: test-group test

VMServiceFlutterDriver: get_text message is taking a long time to complete...
00:31 +0 -1: test-group test [E]

  TimeoutException after 0:00:30.000000: Test timed out after 30 seconds. See https://pub.dev/packages/test#timeouts

  package:test_api/src/backend/invoker.dart 318:28  Invoker._handleError.<fn>
  dart:async/zone.dart 1346:47                      _rootRun
  dart:async/zone.dart 1258:19                      _CustomZone.run
  package:test_api/src/backend/invoker.dart 316:10  Invoker._handleError
  package:test_api/src/backend/invoker.dart 272:9   Invoker.heartbeat.<fn>.<fn>
  dart:async/zone.dart 1354:13                      _rootRun
  dart:async/zone.dart 1258:19                      _CustomZone.run
  package:test_api/src/backend/invoker.dart 271:38  Invoker.heartbeat.<fn>
  dart:async-patch/timer_patch.dart 18:15           Timer._createTimer.<fn>
  dart:isolate-patch/timer_impl.dart 395:19         _Timer._runTimers
  dart:isolate-patch/timer_impl.dart 426:5          _Timer._handleMessage
  dart:isolate-patch/isolate_patch.dart 184:12      _RawReceivePortImpl._handleMessage


00:31 +0 -1: test-group (tearDownAll)

00:31 +0 -1: test-group test [E]

  DriverError: Failed to fulfill GetText due to remote error
  Original error: ext.flutter.driver: (-32000) Service connection disposed
  Original stack trace:
  dart:async/future_impl.dart 25:44                               _Completer.completeError
  package:vm_service/src/vm_service.dart 2060:26                  VmService.dispose.<fn>
  dart:collection-patch/compact_hash.dart 397:8                   _LinkedHashMapMixin.forEach
  package:vm_service/src/vm_service.dart 2059:26                  VmService.dispose
  ===== asynchronous gap ===========================
  dart:async/zone.dart 1286:19                                    _CustomZone.registerUnaryCallback
  dart:async-patch/async_patch.dart 45:22                         _asyncThenWrapperHelper
  package:flutter_driver/src/driver/vmservice_driver.dart 519:26  VMServiceFlutterDriver.close
  test_driver/app_test.dart 13:14                                 main.<fn>.<fn>
  test_driver/app_test.dart 12:17                                 main.<fn>.<fn>
  dart:async/future.dart 218:31                                   new Future.sync
  package:test_api/src/util/test.dart 21:12                       errorsDontStopTest.<fn>
  package:test_api/src/backend/invoker.dart 231:15                Invoker.waitForOutstandingCallbacks.<fn>
  package:test_api/src/backend/invoker.dart 228:14                Invoker.waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:test_api/src/backend/invoker.dart 228:5                 Invoker.waitForOutstandingCallbacks
  package:test_api/src/util/test.dart 20:20                       errorsDontStopTest
  package:test_api/src/backend/declarer.dart 358:19               Declarer._tearDownAll.<fn>.<fn>.<fn>
  package:test_api/src/backend/declarer.dart 356:44               Declarer._tearDownAll.<fn>.<fn>.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:test_api/src/backend/invoker.dart 248:12                Invoker.unclosable
  package:test_api/src/backend/declarer.dart 356:33               Declarer._tearDownAll.<fn>.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:test_api/src/backend/declarer.dart 355:14               Declarer._tearDownAll.<fn>
  package:test_api/src/backend/invoker.dart 231:15                Invoker.waitForOutstandingCallbacks.<fn>
  package:test_api/src/backend/invoker.dart 228:14                Invoker.waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:test_api/src/backend/invoker.dart 228:5                 Invoker.waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 383:17                Invoker._onRun.<fn>.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async/zone.dart 1286:19                                    _CustomZone.registerUnaryCallback
  dart:async-patch/async_patch.dart 45:22                         _asyncThenWrapperHelper
  package:test_api/src/backend/invoker.dart                       Invoker._onRun.<fn>.<fn>.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:test_api/src/backend/invoker.dart 370:9                 Invoker._onRun.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 415:15                Invoker._guardIfGuarded
  package:test_api/src/backend/invoker.dart 369:7                 Invoker._onRun.<fn>
  package:stack_trace/src/chain.dart 94:24                        Chain.capture.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1789:10                                    _runZoned
  dart:async/zone.dart 1711:10                                    runZoned
  package:stack_trace/src/chain.dart 92:12                        Chain.capture
  package:test_api/src/backend/invoker.dart 368:11                Invoker._onRun
  package:test_api/src/backend/live_test_controller.dart 153:11   LiveTestController.run
  dart:async/future.dart 198:37                                   new Future.microtask.<fn>
  dart:async/zone.dart 1346:47                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1162:7                                     _CustomZone.runGuarded
  dart:async/zone.dart 1202:23                                    _CustomZone.bindCallbackGuarded.<fn>
  dart:async/zone.dart 1354:13                                    _rootRun
  dart:async/zone.dart 1258:19                                    _CustomZone.run
  dart:async/zone.dart 1162:7                                     _CustomZone.runGuarded
  dart:async/zone.dart 1202:23                                    _CustomZone.bindCallbackGuarded.<fn>
  dart:async/schedule_microtask.dart 40:21                        _microtaskLoop
  dart:async/schedule_microtask.dart 49:5                         _startMicrotaskLoop
  dart:isolate-patch/isolate_patch.dart 120:13                    _runPendingImmediateCallback
  dart:isolate-patch/timer_impl.dart 402:11                       _Timer._runTimers
  dart:isolate-patch/timer_impl.dart 426:5                        _Timer._handleMessage
  dart:isolate-patch/isolate_patch.dart 184:12                    _RawReceivePortImpl._handleMessage



  package:flutter_driver/src/driver/vmservice_driver.dart 313:7  VMServiceFlutterDriver.sendCommand
  ===== asynchronous gap ===========================
  dart:async/zone.dart 1294:19                                   _CustomZone.registerBinaryCallback
  dart:async-patch/async_patch.dart 61:8                         _asyncErrorWrapperHelper
  package:test_api/src/backend/invoker.dart                      Invoker.waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1354:13                                   _rootRun
  dart:async/zone.dart 1258:19                                   _CustomZone.run
  dart:async/zone.dart 1789:10                                   _runZoned
  dart:async/zone.dart 1711:10                                   runZoned
  package:test_api/src/backend/invoker.dart 228:5                Invoker.waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 383:17               Invoker._onRun.<fn>.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async/zone.dart 1286:19                                   _CustomZone.registerUnaryCallback
  dart:async-patch/async_patch.dart 45:22                        _asyncThenWrapperHelper
  package:test_api/src/backend/invoker.dart                      Invoker._onRun.<fn>.<fn>.<fn>
  dart:async/zone.dart 1354:13                                   _rootRun
  dart:async/zone.dart 1258:19                                   _CustomZone.run
  dart:async/zone.dart 1789:10                                   _runZoned
  dart:async/zone.dart 1711:10                                   runZoned
  package:test_api/src/backend/invoker.dart 370:9                Invoker._onRun.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 415:15               Invoker._guardIfGuarded
  package:test_api/src/backend/invoker.dart 369:7                Invoker._onRun.<fn>
  package:stack_trace/src/chain.dart 94:24                       Chain.capture.<fn>
  dart:async/zone.dart 1354:13                                   _rootRun
  dart:async/zone.dart 1258:19                                   _CustomZone.run
  dart:async/zone.dart 1789:10                                   _runZoned
  dart:async/zone.dart 1711:10                                   runZoned
  package:stack_trace/src/chain.dart 92:12                       Chain.capture
  package:test_api/src/backend/invoker.dart 368:11               Invoker._onRun
  package:test_api/src/backend/live_test_controller.dart 153:11  LiveTestController.run
  dart:async/future.dart 198:37                                  new Future.microtask.<fn>
  dart:async/zone.dart 1346:47                                   _rootRun
  dart:async/zone.dart 1258:19                                   _CustomZone.run
  dart:async/zone.dart 1162:7                                    _CustomZone.runGuarded
  dart:async/zone.dart 1202:23                                   _CustomZone.bindCallbackGuarded.<fn>
  dart:async/zone.dart 1354:13                                   _rootRun
  dart:async/zone.dart 1258:19                                   _CustomZone.run
  dart:async/zone.dart 1162:7                                    _CustomZone.runGuarded
  dart:async/zone.dart 1202:23                                   _CustomZone.bindCallbackGuarded.<fn>
  dart:async/schedule_microtask.dart 40:21                       _microtaskLoop
  dart:async/schedule_microtask.dart 49:5                        _startMicrotaskLoop
  dart:isolate-patch/isolate_patch.dart 120:13                   _runPendingImmediateCallback
  dart:isolate-patch/timer_impl.dart 402:11                      _Timer._runTimers
  dart:isolate-patch/timer_impl.dart 426:5                       _Timer._handleMessage
  dart:isolate-patch/isolate_patch.dart 184:12                   _RawReceivePortImpl._handleMessage


00:31 +0 -1: Some tests failed.


Unhandled exception:
Dummy exception to set exit code.
Failed to stop app
```

</details>

<details><summary>flutter doctor output</summary>

```
% flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.5.1 20G80 darwin-x64, locale en-US)
    • Flutter version 2.2.3 at /Users/user/.flutter
    • Framework revision f4abaa0735 (7 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/user/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to find bundled Java version.
    • Try updating or re-installing Android Studio.

[✓] Connected device (3 available)
    • test-sim (mobile) • E1A7325E-D299-42D4-A89C-B5DD7CF7DB06 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • iPhone 8 (mobile) • 87606CD6-83B2-433B-9458-FD6944930D2F • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • Chrome (web)      • chrome                               • web-javascript • Google Chrome 92.0.4515.159
    ! Error: User's iPhone is not connected. Xcode will continue when User's iPhone is connected. (code -13)

! Doctor found issues in 1 category.
```

</details>

## Proposal

The fix likely has 3 parts:
1. Add reasonable default `timeout` arg values to every [`FlutterDriver`](https://api.flutter.dev/flutter/flutter_driver/FlutterDriver-class.html) function.  Alternatively, update the integration test example to add a `timeout:Duration(seconds:5)` kwarg to each `driver.tap()` and `driver.getText()` call.  This will let the users see which test operation failed, instead of a generic "Test timed out after 30 seconds" error.  This also eliminates the confusing and useless teardown error and its enormous 124-line stack trace.
2. Emit the stack trace of the test.  This will let users see the line number of the test failure.  When a developer investigates a failing test, the line number of the failure is the first piece of information they look for.
3. Update every FlutterDriver function that takes a finder to return a description of the finder when the finder fails.  Include that description in the exception message.  This will let users see which widget failed to appear/disappear.  This information is crucial for debugging table-driven tests and tests that use computed values.