Repository: flutter
Issue: Error when remove ScrollView, which has a ScrollController, from widget tree
Labels: framework,f: scrolling,d: stackoverflow,P2,team-framework,triaged-framework

<!-- Thank you for using Flutter!

     If you are looking for support, please check out our documentation
     or consider asking a question on Stack Overflow:
      * https://flutter.dev/
      * https://api.flutter.dev/
      * https://stackoverflow.com/questions/tagged/flutter?sort=frequent

     If you have found a bug or if our documentation doesn't have an answer
     to what you're looking for, then fill out the template below. Please read
     our guide to filing a bug first: https://flutter.dev/docs/resources/bug-reports
-->

## Steps to Reproduce

The example below use `LayoutBuilder` to show `Scrollbar` when screen width is smaller than 960dp and remove the `Scrollbar` when larger than 960dp.

If we want `isAlwaysShown: true`, we must provide the same `ScrollController` to both `Scrollbar` and `ScrollView`.
When remove the `ScrollView` from the current widget tree, an exception is thrown.

```
import 'package:flutter/material.dart';

void main() => runApp(const MyApp());

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(home: MyStatelessWidget());
  }
}

class MyStatelessWidget extends StatefulWidget {
  const MyStatelessWidget({Key? key}) : super(key: key);

  @override
  _MyStatelessWidgetState createState() => _MyStatelessWidgetState();
}

class _MyStatelessWidgetState extends State<MyStatelessWidget> {
  ScrollController _scrollController = ScrollController();

  @override
  Widget build(BuildContext context) {
    Widget _scaffold = Scaffold(
      appBar: AppBar(
        title: const Text('LayoutBuilder Example'),
      ),
      body: Container(
        child: Text(
          'Example Content',
          style: TextStyle(fontSize: 200),
        ),
      ),
    );
    return LayoutBuilder(builder: (
      BuildContext context,
      BoxConstraints constraints,
    ) {
      Widget _result = _scaffold;
      if (constraints.minWidth < 960) {
        _result = wrapWithWidthScroll(_result);
      }
      return _result;
    });
  }

  Widget wrapWithWidthScroll(Widget _scaffold) {
    return Scrollbar(
      isAlwaysShown: true,
      controller: _scrollController,
      child: SingleChildScrollView(
        controller: _scrollController,
        child: Container(width: 960, child: _scaffold),
        scrollDirection: Axis.horizontal,
      ),
    );
  }
}
```

**Expected results:** <!-- what did you want to see? -->

No error when we expand screen width (web or desktop) from ex. 900dp to ex. 1000dp.

**Actual results:** <!-- what did you see? -->
Error when we expand screen width (web or desktop) from ex. 900dp to ex. 1000dp.

```
======== Exception caught by scheduler library =====================================================
The following assertion was thrown during a scheduler callback:
The Scrollbar's ScrollController has no ScrollPosition attached.

A Scrollbar cannot be painted without a ScrollPosition. 

The Scrollbar attempted to use the PrimaryScrollController. This ScrollController should be associated with the ScrollView that the Scrollbar is being applied to. When providing your own ScrollController, ensure both the Scrollbar and the Scrollable widget use the same one.

When the exception was thrown, this was the stack: 
#0      RawScrollbarState._maybeTriggerScrollbar.<anonymous closure>.<anonymous closure> (package:flutter/src/widgets/scrollbar.dart:960:13)
#1      RawScrollbarState._maybeTriggerScrollbar.<anonymous closure> (package:flutter/src/widgets/scrollbar.dart:985:10)
#2      SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1144:15)
#3      SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1090:9)
#4      SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:998:5)
...
====================================================================================================
```

**flutter doctor -v**
```
[√] Flutter (Channel beta, 2.2.0-10.3.pre, on Microsoft Windows [Version 10.0.19042.964], locale en-DE)
    • Flutter version 2.2.0-10.3.pre at C:\...\flutter
    • Framework revision 06e2fd6357 (5 days ago), 2021-05-08 11:28:22 -0700
    • Engine revision a123e75c60
    • Dart version 2.13.0 (build 2.13.0-211.14.beta)
```