By directive I mean the top of the test file which looks like:

```
// compile-flags: --test
// aux-build:foo.rs
```

(there are many other directives). You've made it such that this code is run, but it is no longer run in the benchmark harness so we're no longer getting any numbers relevant for these benchmarks, it would be a little sad to just remove benchmarks.