Build only tests and benchmarks when needed