Repository: flutter
Issue: Broken Hero animation with images.
Labels: framework,a: animation,f: routes,a: images,has reproducible steps,P2,found in release: 3.10,found in release: 3.11,team-framework,triaged-framework

### Is there an existing issue for this?

- [X] I have read the [guide to filing a bug](https://flutter.dev/docs/resources/bug-reports)
- [X] I have searched the [existing issues](https://github.com/flutter/flutter/issues)

### Steps to reproduce

1. Open [dartPad](https://dartpad.dev/?id=4ab3102284c2a9534fa99c5e6a5f7f0d)
2. Click any where on the screen to navigate back and forward.
3. un-comment `fit` property and try again or try to resize the web page window.

### Expected results

The animation should run smoothly.

### Actual results

On Mobile platform there is stuttering and sometimes wrong scaling during transition.
On Web platform the animation is broken.

### Code sample

<details><summary>Code sample</summary>

```dart
import 'package:flutter/material.dart';

void main() {
  runApp(
    const MaterialApp(
      home: MyHomePage(),
    ),
  );
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({super.key});

  @override
  Widget build(BuildContext context) {
    return GestureDetector(
      onTap: () {
        Navigator.of(context).push(
          PageRouteBuilder(
            pageBuilder: (context, animation, secondaryAnimation) {
              return FadeTransition(
                opacity: animation,
                child: const SecondPage(),
              );
            },
          ),
        );
      },
      child: Scaffold(
        body: Stack(
          children: [
            Transform.scale(
              scale: 1.0,
              child: Hero(
                tag: 'flutter',
                child: Image.network(
                  'https://images.unsplash.com/photo-1682687980115-a37b56ea7271?ixlib=rb-4.0.3&ixid=M3wxMjA3fDF8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=600&q=80',
                  fit: BoxFit.cover,
                ),
              ),
            )
          ],
        ),
      ),
    );
  }
}

class SecondPage extends StatelessWidget {
  const SecondPage({super.key});

  @override
  Widget build(BuildContext context) {
    return GestureDetector(
      onTap: () {
        Navigator.pop(context);
      },
      child: Scaffold(
        body: Stack(
          children: [
            Transform.scale(
              scale: 2.5,
              child: Hero(
                tag: 'flutter',
                child: Image.network(
                  'https://images.unsplash.com/photo-1682687980115-a37b56ea7271?ixlib=rb-4.0.3&ixid=M3wxMjA3fDF8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=600&q=80',
                  fit: BoxFit.cover,
                ),
              ),
            ),
          ],
        ),
      ),
    );
  }
}

```

</details>


### Screenshots or Video

https://github.com/flutter/flutter/assets/95232508/cc361f89-fe1f-445f-8a49-81bd1febf957


### Logs

_No response_

### Flutter Doctor output

<details><summary>Doctor output</summary>

```console
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.10.1, on macOS 13.2 22D49 darwin-arm64, locale
    en-EG)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
[✓] Chrome - develop for the web
[✓] Android Studio
[✓] VS Code (version 1.78.2)
[✓] Connected device (2 available)
[✓] Network resources

• No issues found!
```

</details>