Repository: create-react-app
Issue: Mock promises silently fail when created outside of the test scope
Labels: needs triage,issue: bug report

<!--
    Please note that your issue will be fixed much faster if you spend about
    half an hour preparing it, including the exact reproduction steps and a demo.

    If you're in a hurry or don't feel confident, it's fine to report bugs with
    less details, but this makes it less likely they'll get fixed soon.

    In either case, please use this template and fill in as many fields below as you can.

    Note that we don't provide help for webpack questions after ejecting.
    You can find webpack docs at https://webpack.js.org/.
-->

### Describe the bug

As per [this StackOverflow post](https://stackoverflow.com/a/66999030/6335363), it appears that [CRA has `resetMocks` enabled by default](https://github.com/facebook/create-react-app/blob/7e4949a20fc828577fb7626a3262832422f3ae3b/packages/react-scripts/scripts/utils/createJestConfig.js#L69) for testing.

While I understand this design decision, as it improves test quality, it also means that anyone who doesn't read the source code for CRA will be unaware of this change (given that it is not documented).

As such, you can run into extremely confusing behaviour where tests work when run directly with `jest` but fail due to the silent resetting of the mocks between tests when run with `react-scripts test`.

This behaviour is extremely confusing, especially given the lack of documentation, and so steps should be taken to make users aware of it. I think the most reasonable solution is to give a warning to users when they create a mock function in the global scope, and link them to some documentation on the changes that CRA has made to Jest's default behaviour (which will need to be created).

### Did you try recovering your dependencies?

<!--
  Your module tree might be corrupted, and that might be causing the issues.
  Let's try to recover it. First, delete these files and folders in your project:

    * node_modules
    * package-lock.json
    * yarn.lock

  Then you need to decide which package manager you prefer to use.
  We support both npm (https://npmjs.com) and yarn (https://yarnpkg.com/).
  However, **they can't be used together in one project** so you need to pick one.

  If you decided to use npm, run this in your project directory:

    npm install -g npm@latest
    npm install

  This should fix your project.

  If you decided to use yarn, update it first (https://yarnpkg.com/en/docs/install).
  Then run in your project directory:

    yarn

  This should fix your project.

  Importantly, **if you decided to use yarn, you should never run `npm install` in the project**.
  For example, yarn users should run `yarn add <library>` instead of `npm install <library>`.
  Otherwise your project will break again.

  Have you done all these steps and still see the issue?
  Please paste the output of `npm --version` and/or `yarn --version` to confirm.
-->

N/A

### Which terms did you search for in User Guide?

<!--
  There are a few common documented problems, such as watcher not detecting changes, or build failing.
  They are described in the Troubleshooting section of the User Guide:

  https://facebook.github.io/create-react-app/docs/troubleshooting

  Please scan these few sections for common problems.
  Additionally, you can search the User Guide itself for something you're having issues with:

  https://facebook.github.io/create-react-app/

  If you didn't find the solution, please share which words you searched for.
  This helps us improve documentation for future readers who might encounter the same problem.
-->

* Mock function
* `resetMocks`
* Testing
* Promise testing

### Environment

<!--
  To help identify if a problem is specific to a platform, browser, or module version, information about your environment is required.
  This enables the maintainers quickly reproduce the issue and give feedback.

  Run the following command in your React app's folder in terminal.
  Note: The result is copied to your clipboard directly.

  `npx create-react-app --info`

  Paste the output of the command in the section below.
-->

```
⋙   npx create-react-app --info

Environment Info:

  current version of create-react-app: 5.0.1
  running from /home/migue/.npm/_npx/c67e74de0542c87c/node_modules/create-react-app

  System:
    OS: Linux 6.2 Fedora Linux 37 (Workstation Edition)
    CPU: (12) x64 Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
  Binaries:
    Node: 19.1.0 - ~/.nvm/versions/node/v19.1.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v19.1.0/bin/yarn
    npm: 9.6.5 - ~/.nvm/versions/node/v19.1.0/bin/npm
  Browsers:
    Chrome: Not Found
    Firefox: 112.0.1
  npmPackages:
    react: ^17.0.2 => 17.0.2 
    react-dom:  17.0.2 
    react-scripts: 5.0.1 => 5.0.1 
  npmGlobalPackages:
    create-react-app: Not Found
```

### Steps to reproduce

<!--
  How would you describe your issue to someone who doesn’t know you or your project?
  Try to write a sequence of steps that anybody can repeat to see the issue.
-->

Add a test as follows:

```ts
const mockPromise = jest.fn().mockResolvedValue('example');

test('mock promise resolves', async () => {
  const result = await mockPromise();
  expect(result).toStrictEqual('example');
});
```

### Expected behavior

<!--
  How did you expect the tool to behave?
  It’s fine if you’re not sure your understanding is correct.
  Just write down what you thought would happen.
-->

The test would run normally (see output when running Jest directly), or I would be given a clear error message informing me of what my mistake was.

```
⋙ npx jest src/components/example.test.ts
 PASS  src/components/example.test.ts
  ✓ mock promise resolves (2 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.305 s
Ran all test suites matching /src\/components\/example.test.ts/i.
```

### Actual behavior

<!--
  Did something go wrong?
  Is something broken, or not behaving as you expected?
  Please attach screenshots if possible! They are extremely helpful for diagnosing issues.
-->

When run with `react-scripts test`, it appears that the mock function completely failed.

```
⋙ npm t -- src/components/example.test.ts

> frontend@0.1.0 test
> react-scripts test --watchAll=false src/components/example.test.ts

 FAIL  src/components/example.test.ts
  ✕ mock promise resolves (3 ms)

  ● mock promise resolves

    expect(received).toEqual(expected) // deep equality

    Expected: "example"
    Received: undefined

      4 | test('mock promise resolves', async () => {
      5 |   const result = await mockPromise();
    > 6 |   expect(result).toEqual('example');
        |                  ^
      7 | });
      8 |

      at Object.<anonymous> (src/components/example.test.ts:6:18)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        0.93 s, estimated 1 s
Ran all test suites matching /src\/components\/example.test.ts/i.
```

### Reproducible demo

<!--
  If you can, please share a project that reproduces the issue.
  This is the single most effective way to get an issue fixed soon.

  There are two ways to do it:

    * Create a new app and try to reproduce the issue in it.
      This is useful if you roughly know where the problem is, or can’t share the real code.

    * Or, copy your app and remove things until you’re left with the minimal reproducible demo.
      This is useful for finding the root cause. You may then optionally create a new project.

  This is a good guide to creating bug demos: https://stackoverflow.com/help/mcve
  Once you’re done, push the project to GitHub and paste the link to it below:
-->

See above for test case.

<!--
  What happens if you skip this step?

  We will try to help you, but in many cases it is impossible because crucial
  information is missing. In that case we'll tag an issue as having a low priority,
  and eventually close it if there is no clear direction.

  We still appreciate the report though, as eventually somebody else might
  create a reproducible example for it.

  Thanks for helping us help you!
-->