Repository: rust
Issue: Test suite compilation spends a large amount of time in `evaluate_obligation`
Labels: I-compiletime,T-compiler

I originally posted this on the [Rust forums](https://users.rust-lang.org/t/extremely-slow-compile-times-in-test-suite/61842) but was directed here since the cause could be related to a compiler bug.

I'm looking for some help debugging why the compile times in the [`mongodb`](https://github.com/mongodb/mongo-rust-driver) crate's test suite are much slower than the crate itself. For reference, a single incremental change can take a minute or longer, even when running just `cargo check --tests`. The crate itself compiles quickly--the issue is only with the tests.

The output of compiling with `-Zself-profile` indicates that nearly the entire time (~1m) is spent in `evaluate_obligation`, but I can't figure what this implies or how to possibly reduce this.

A few possible culprits that I've investigated so far:
* the tests make heavy use of `#[cfg_attr(...)]` to conditionally select either `#[tokio::test]` or `#[async_std::test]`. Removing all usages of this and defaulting to just `#[tokio::test]` has an insignificant impact on compile times
* the tests also make heavy use of a function that accepts a closure that returns a future. According to [this thread](https://users.rust-lang.org/t/exponential-type-size-of-async-function/36852/3), such functions can cause type length / compilation time issues, but converting it to a macro that calls a function directly didn't seem to change much. Perhaps that suffers from a similar issue?
* We generate a lot of builders via the `typed-builder` crate. Perhaps using them in the tests leads to long compilation times due to the generated builders having lots of generic parameters.
* [this compiler issue](https://github.com/rust-lang/rust/issues/75992) related to `async` and compilation times, though someone posted their profile output and `evaluate_obligation` is not a factor in it.
* most of our tests are in `src/test` rather than `tests`, not sure if this has any impact

Anyways, I'm not sure where to go from here, and I was wondering if anyone else has encountered similar issues in the past or knows how I could further debug this to identify the root cause. Thanks!

Here are the top few results from the self-profile output:
```
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| Item                                             | Self time | % of total time | Time     | Item count | Incremental load time |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| evaluate_obligation                              | 58.34s    | 85.300          | 58.39s   | 110480     | 0.00ns                |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| expand_crate                                     | 3.54s     | 5.179           | 3.59s    | 1          | 0.00ns                |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| typeck                                           | 2.74s     | 4.011           | 61.39s   | 1159       | 71.43ms               |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| self_profile_alloc_query_strings                 | 979.78ms  | 1.433           | 980.49ms | 1          | 0.00ns                |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
| hir_lowering                                     | 204.80ms  | 0.299           | 204.80ms | 1          | 0.00ns                |
+--------------------------------------------------+-----------+-----------------+----------+------------+-----------------------+
```