Add a compilationOnly option which skips running the evaluation if true.