Integration tests for benchmark API.

Adds randomized integration tests for the benchmark API.
Adds negative tests for cases where the cluster cannot run benchmarks.
Small fixes for NPE's exposed during testing.

Closes #6003