Repository: google/swift-benchmark
Pull request #47 (2020-06-15 18:02:10)

Allow 0 --iterations
It's useful to run print benchmark results without actually running benchmarks to enumerate all available benchmarks while interacting with benchmarks programmatically. 

saeta: > without actually running benchmarks to enumerate all available benchmarks while interacting with benchmarks programmatically

Note: would it make sense to explicitly support this feature (e.g. by having a `--list` which just prints things out). This would make the capability more discoverable & would allow us to customize the printing.
saeta: Thinking a bit further, it might be convenient to allow `--list` to take an extra optional parameter which would specify the format of the printing (e.g. json, or one-per-line, or some fancy tree form. This would seem to push us towards having subcommands which could be unnecessary complexity, however.
shabalind: @saeta I considered subcommands but it seems to complicate the common case of "just run this benchmarks" mode that most users are going to deal with in their day-to-day development. 

Patch:
diff --git a/Sources/Benchmark/BenchmarkArguments.swift b/Sources/Benchmark/BenchmarkArguments.swift
index 7f97828..d78e164 100644
--- a/Sources/Benchmark/BenchmarkArguments.swift
+++ b/Sources/Benchmark/BenchmarkArguments.swift
@@ -78,8 +78,9 @@ public struct BenchmarkArguments: ParsableArguments {
         if isDebug && !allowDebugBuild {
             throw ValidationError(debugBuildErrorMessage)
         }
-        if iterations != nil && iterations! <= 0 {
-            throw ValidationError(positiveNumberError(flag: "--iterations", of: "integer"))
+        if iterations != nil && iterations! < 0 {
+            throw ValidationError(
+                nonNegativeNumberError(flag: "--iterations", of: "integer"))
         }
         if warmupIterations != nil && warmupIterations! < 0 {
             throw ValidationError(


Test patch:
diff --git a/Tests/BenchmarkTests/BenchmarkCommandTests.swift b/Tests/BenchmarkTests/BenchmarkCommandTests.swift
index 121ee93..494c8be 100644
--- a/Tests/BenchmarkTests/BenchmarkCommandTests.swift
+++ b/Tests/BenchmarkTests/BenchmarkCommandTests.swift
@@ -61,10 +61,10 @@ final class BenchmarkCommandTests: XCTestCase {
         }
     }
 
-    func testParseZeroIterationsError() throws {
-        AssertFailsToParse(
-            ["--iterations", "0", "--allow-debug-build"],
-            with: "Value provided via --iterations must be a positive integer.")
+    func testParseZeroIterations() throws {
+        AssertParse(["--iterations", "0", "--allow-debug-build"]) { settings in
+            XCTAssertEqual(settings.iterations, 0)
+        }
     }
 
     func testParseWarmupIterations() throws {
@@ -114,7 +114,7 @@ final class BenchmarkCommandTests: XCTestCase {
         ("testDebugBuildError", testDebugBuildError),
         ("testParseFilter", testParseFilter),
         ("testParseIterations", testParseIterations),
-        ("testParseZeroIterationsError", testParseZeroIterationsError),
+        ("testParseZeroIterations", testParseZeroIterations),
         ("testParseWarmupIterations", testParseWarmupIterations),
         ("testParseZeroWarmupIterations", testParseZeroWarmupIterations),
         ("testParseNoWarmupIterations", testParseNoWarmupIterations),