Repository: storybook
Issue: [Bug]: Storybook Background is flashing when switching stories
Labels: bug,needs triage

### Describe the bug

Hi @shilman I'm using **Storybook V8.0.10 along with storybook-dark-mode v4.0.0.** 

Here's the present scenario: upon initially navigating to a story after clearing the cache, the screen flickers between dark and light modes. However, if I've previously visited that particular story, this flashing behavior doesn't occur.

[Untitled_ May 9, 2024 10_19 PM.webm](https://github.com/storybookjs/storybook/assets/54674189/00bc7e28-c404-402c-a529-9e29d02ee6f3)


### main.ts

```js
import type { StorybookConfig } from '@storybook/react-vite';
import { mergeConfig } from 'vite';

const glob = require('glob');
const path = require('path');
const appDirectory = path.resolve(__dirname, '../');
const getStories = () =>
  glob.sync(
    `${appDirectory}/packages/**/*.@(stories.js|stories.jsx|stories.ts|stories.tsx)`,
    {
      ignore: `${appDirectory}/packages/**/dist/**/*.@(stories.js|stories.jsx|stories.ts|stories.tsx)`,
    }
  );

const config: StorybookConfig = {
  core: {},
  framework: {
    name: '@storybook/react-vite',
    options: {},
  },
  stories: getStories(),
  addons: [
    '@storybook/addon-links',
    '@storybook/addon-essentials',
    '@storybook/addon-a11y',
    '@storybook/addon-interactions',
    '@storybook/addon-coverage',
    'storybook-dark-mode',
  ],
  async viteFinal(storybookConfig) {
    return mergeConfig(storybookConfig, {
      resolve: {
        preserveSymlinks: true,
      },
      server: {
        fs: {
          allow: ['..'],
        },
        watch: {
          ignored: ['!**/node_modules/@banking-platform/**'],
        },
      },
      assetsInclude: ['**/*.md'],
    });
  },
  docs: {
    autodocs: true,
  },
};

export default config;
```


### preview.jsx

```js
import { create } from '@storybook/theming';

import Docs from '../../.storybook/docs';

import { createDarkTheme, createLightTheme } from '../../.storybook/theme';

import '../../.storybook/assets/stories.scss';
import '../../.storybook/assets/style.scss';

import ThemeDecorator from './theme-decorator';

const lightTheme = create({
  ...createLightTheme,
});

const darkTheme = create({
  ...createDarkTheme,
});

const customViewports = {
  mobile: {
    name: 'Mobile',
    styles: {
      width: '600px',
      height: '963px',
    },
  },
  tablet: {
    name: 'Tablet',
    styles: {
      width: '840px',
      height: '1194px',
    },
  },
};

const parameters = {
  backgrounds: {
    default: 'Surface',
    values: [
      {
        name: 'Background',
        value: 'var(--color-background-default)',
      },
      {
        name: 'Surface',
        value: 'var(--color-surface-default)',
      },
    ],
  },
  controls: {
    matchers: {
      color: /(background|color)$/i,
      date: /Date$/,
    },
  },
  options: {
    storySort: {
      order: ['Foundation', 'Actions', 'Inputs'],
    },
  },
  darkMode: {
    current: 'light',
    dark: darkTheme,
    light: lightTheme,
  },
  docs: {
    page: () => <Docs framework="react" />,
  },
  viewport: { viewports: customViewports },
};

const decorators = [
  (Story) => (
    <ThemeDecorator>
      <Story />
    </ThemeDecorator>
  ),
];

const preview = {
  parameters,
  decorators,
};

export default preview;
```

### Reproduction link

N/A

### Reproduction steps

_No response_

### System

```bash
Storybook Environment Info:

  System:
    OS: Windows 11 10.0.22631
    CPU: (16) x64 AMD Ryzen 7 PRO 4750U with Radeon Graphics
  Binaries:
    Node: 20.9.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.22 - C:\Program Files (x86)\Yarn\bin\yarn.CMD <----- active
    npm: 10.1.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (127.0.2651.74)
  npmPackages:
    @storybook/addon-a11y: ^8.2.6 => 8.2.6
    @storybook/addon-actions: ^8.2.6 => 8.2.6
    @storybook/addon-coverage: ^1.0.4 => 1.0.4
    @storybook/addon-essentials: ^8.2.6 => 8.2.6
    @storybook/addon-interactions: ^8.2.6 => 8.2.6
    @storybook/addon-links: ^8.2.6 => 8.2.6
    @storybook/manager-api: ^8.2.6 => 8.2.6
    @storybook/react: ^8.2.6 => 8.2.6
    @storybook/react-vite: ^8.2.6 => 8.2.6
    @storybook/test: ^8.2.6 => 8.2.6
    @storybook/test-runner: ^0.19.1 => 0.19.1
    eslint-plugin-storybook: ^0.8.0 => 0.8.0
    storybook: ^8.2.6 => 8.2.6
    storybook-dark-mode: ^4.0.2 => 4.0.2
```


### Additional context

_No response_