Repository: swiftlang/swift-testing

Issue #1037: Include the location of a parameterized test argument which recorded an issue (if possible) in console output for better traceability
Created: 2025-03-25T14:24:52Z
Motivation
When using @Test(arguments:) to test multiple input patterns in a single test function, it can be difficult to identify which argument caused a failure, especially when several similar test cases are used.
For example, consider the following test setup:
Supporting code:
struct User: Equatable {
    let name: String
    let age: Int
    let email: String
    let username: String
    let password: String
    let isActive: Bool
    let signupDate: Date
    let lastLoginDate: Date
    let phoneNumber: String
    let address: String
    let bio: String
}

extension User {
    static func mock(
        name: String = "John Doe",
        age: Int = 30,
        email: String = "john@example.com",
        username: String = "johndoe",
        password: String = "password123",
        isActive: Bool = true,
        signupDate: Date = Date(),
        lastLoginDate: Date = Date(),
        phoneNumber: String = "000-0000-0000",
        address: String = "123 Apple Street",
        bio: String = "Just a test user."
    ) -> Self {
        .init(
            name: name,
            age: age,
            email: email,
            username: username,
            password: password,
            isActive: isActive,
            signupDate: signupDate,
            lastLoginDate: lastLoginDate,
            phoneNumber: phoneNumber,
            address: address,
            bio: bio
        )
    }
}
Test code:
struct UserTests {
    @Test(arguments: [
        User.mock(),
        .mock(name: "hoge", isActive: false),
        .mock(lastLoginDate: .distantPast, bio: "Hello World"),

        // Multiple mock cases follow

        .mock(email: "fuga@example.com"),
        .mock(phoneNumber: "999-9999-9999")
    ])
    func someTest(_ user: User) async throws {
        let expected: User = .mock() // In practice, this would call some logic
        #expect(user == expected)
    }
}
When a test fails, the output looks like this:
✘ Test someTest(_:) recorded an issue with 1 argument user → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 0001-01-01 00:00:00 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Hello World") at TestTests.swift:72:9: Expectation failed: (user → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 0001-01-01 00:00:00 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Hello World")) == (expected → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 2025-03-25 09:59:04 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Just a test user."))
However, this output alone doesn’t clearly indicate which argument (i.e., which entry in the arguments array) caused the failure. This makes it difficult to trace which case needs to be fixed, especially in large or complex test sets.
Proposed solution
A helpful enhancement would be to include the argument’s index in the failure output. For example, if the second item in the arguments array fails, the output might look like this:
✘ Test someTest(_:) [arguments index: 2] recorded an issue with 1 argument user → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 0001-01-01 00:00:00 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Hello World") at TestTests.swift:72:9: Expectation failed: (user → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 0001-01-01 00:00:00 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Hello World")) == (expected → User(name: "John Doe", age: 30, email: "john@example.com", username: "johndoe", password: "password123", isActive: true, signupDate: 2025-03-25 09:59:04 +0000, lastLoginDate: 2025-03-25 09:59:04 +0000, phoneNumber: "000-0000-0000", address: "123 Apple Street", bio: "Just a test user."))
This small addition would make it significantly easier to identify the failing case and improve the debugging experience, especially when dealing with multiple test arguments.
Currently, developers can work around this by wrapping the argument in a custom struct with a label, like so:
struct UserTestCase {
    let label: String
    let user: User
}

@Test(arguments: [
    UserTestCase(label: "default", user: .mock()),
    UserTestCase(label: "inactive", user: .mock(name: "hoge", isActive: false)),
])
func someTest(_ testCase: UserTestCase) async throws {
    #expect(testCase.user == .mock(), "Failure in case: \(testCase.label)")
}
While this pattern works, it adds boilerplate and isn’t ideal for simpler test cases.
Including the argument index in the default output would be a lightweight and widely useful improvement that benefits many test scenarios.
Alternatives considered
No response
Additional information
No response

Duplicate issue #519: Capture source location of elements in collection literal test arguments
Created: 2024-07-02T11:05:11Z
Description
For parameterized tests, I'd like the error report to point to the source line of the failing test case (perhaps as well as the #expect that failed). This makes more sense for sequences of test cases than combinatorial tests.
In XCTest, I used a struct wrapping each test case that captured #file and #line, so I could pass those through to the assertions. (Sub-issue: there's no corresponding #sourceLocation macro so I can't capture the exact location.)
Wrapping each line was annoying. It would really be nice if there were an implicit way to get this information - e.g., automatically capture the source location for each test case, and add an argument or implicit value that you could pass to #expect.
Expected behavior
No response
Actual behavior
No response
Steps to reproduce
No response
swift-testing version/commit hash
No response
Swift & OS version (output of swift --version && uname -a)
No response

Closing comment (2025-03-25T15:41:54Z): This topic came up again recently in dupe #1037.