Repository: flutter
Issue: The widget size jumps on pop when using Hero 
Labels: framework,f: material design,f: routes,has reproducible steps,found in release: 2.2,found in release: 2.5,team-design,triaged-design

Hi!
I found a problem with font size and padding scaling when using Hero and Transform.scale
Take a look at this gif :
<img src="https://user-images.githubusercontent.com/61683634/120150975-670cb880-c1f4-11eb-80ce-4b07a9b73d1c.gif" alt="issue" width="500"/>


## Steps to Reproduce

Put this inside main.dart and run it

<details>
<summary>Code</summary>
<p>

```dart
import 'package:flutter/material.dart';
import 'package:flutter/widgets.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
        title: 'Flutter Demo',
        theme: ThemeData(
          primarySwatch: Colors.blue,
        ),
        home: FirstScreen());
  }
}

class FirstScreen extends StatelessWidget {
  const FirstScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('First screen'),
      ),
      body: Column(
        children: [
          Transform.scale(
            scale: 0.5,
            child: Hero(tag: 'hero', child: ColorBox()),
          ),
          Align(
            alignment: Alignment.bottomCenter,
            child: ElevatedButton(
              onPressed: () => Navigator.of(context)
                  .push(MaterialPageRoute(builder: (_) => SecondScreen())),
              child: Material(
                color: Colors.transparent,
                child: Text('Toggle screens'),
              ),
            ),
          ),
        ],
      ),
    );
  }
}

class SecondScreen extends StatelessWidget {
  const SecondScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Hero animation'),
      ),
      body: Column(
        mainAxisAlignment: MainAxisAlignment.center,
        children: [
          Center(
            child: Hero(
              tag: 'hero',
              child: ColorBox(),
            ),
          )
        ],
      ),
    );
  }
}

class ColorBox extends StatelessWidget {
  const ColorBox({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Container(
      height: 200,
      width: 200,
      //color: Colors.pink,
      child: Column(
        children: [
          Flexible(
            child: Padding(
              padding: EdgeInsets.symmetric(horizontal: 25),
              child: Container(
                color: Colors.blue,
                child: Center(
                  child: Material(
                    color: Colors.transparent,
                    child: Text(
                      'Text',
                      style: TextStyle(fontSize: 15),
                    ),
                  ),
                ),
              ),
            ),
          )
        ],
      ),
    );
  }
}

```

</p>
</details>  

<details>
  <summary>flutter doctor</summary>

```
[✓] Flutter (Channel stable, 2.2.1, on Linux, locale en_US.UTF-8)
    • Flutter version 2.2.1 at /home/alexey/dev/flutter
    • Framework revision 02c026b03c (3 days ago), 2021-05-27 12:24:44 -0700
    • Engine revision 0fdb562ac8
    • Dart version 2.13.1

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/alexey/Android/Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /opt/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /opt/android-studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] Connected device (2 available)
    • Mi Note 3 (mobile) • 6668c500 • android-arm64  • Android 9 (API 28)
    • Chrome (web)       • chrome   • web-javascript • Google Chrome 90.0.4430.212

• No issues found!
```

</details>