Repository: NVIDIA/cuda-python
Pull request #559: NEW: Make event timing error messages more specific and actionable
Author: login: carterbox
Labels: name: enhancement
description: Any code-related improvements
name: P1
description: Medium priority - Should do
name: cuda.core
description: Everything related to the cuda.core module
Created: 2025-04-14T23:26:10Z | Merged: 2025-04-27T04:26:14Z

## Description

<!-- Every PR should have a corresponding issue that describes and motivates the work done in the PR -->
closes #556

The CUDA driver provides different error messages for various errors when trying to compute elapsed time, and the documentation explains each of these scenarious. Surface each of these to Python uses with actionable error messages.

<!-- Provide a standalone description of changes in this PR. -->

<!-- Note: The pull request title will be included in the CHANGELOG. -->

## Checklist
<!-- TODO: - [ ] I am familiar with the [Contributing Guidelines](). -->
- [x] New or existing tests cover these changes.
- [ ] The documentation is up to date with these changes.

\* Not sure what documentation should be added. Sometimes python functions document what errors are raised, but __sub__ doesn't have a docs section.



Linked issue #556: [BUG]: Event timing error misleading
### Is this a duplicate?

- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cuda-python/issues) for this bug and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Type of Bug

Something else

### Component

cuda.core

### Describe the bug

`cuEventElapsedTime()` has multiple possibilities for not returning success.

> If [cuEventRecord()](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EVENT.html#group__CUDA__EVENT_1g95424d3be52c4eb95d83861b70fb89d1) has not been called on either event then [CUDA_ERROR_INVALID_HANDLE](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1ggc6c391505e117393cc2558fff6bfc2e90fd2e63260c6317ba943af0f7e4b8d21) is returned. If [cuEventRecord()](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EVENT.html#group__CUDA__EVENT_1g95424d3be52c4eb95d83861b70fb89d1) has been called on both events but one or both of them has not yet been completed (that is, [cuEventQuery()](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EVENT.html#group__CUDA__EVENT_1g6f0704d755066b0ee705749ae911deef) would return [CUDA_ERROR_NOT_READY](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1ggc6c391505e117393cc2558fff6bfc2e9edd9cef666ce620352e619a36b6c3f34) on at least one of the events), [CUDA_ERROR_NOT_READY](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1ggc6c391505e117393cc2558fff6bfc2e9edd9cef666ce620352e619a36b6c3f34) is returned. If either event was created with the [CU_EVENT_DISABLE_TIMING](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1gg5ae04079c671c8e659a3a27c7b23f629daa5463f64794c10b78c603d23c0bff2) flag, then this function will return [CUDA_ERROR_INVALID_HANDLE](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1ggc6c391505e117393cc2558fff6bfc2e90fd2e63260c6317ba943af0f7e4b8d21).

However only the last of these explanations is mentioned in the Python error message which replaces these error codes: "Timing capability must be enabled in order to subtract two Events; timing is disabled by default." 

The underlying error code should be unsupressed and the user should be directed to the CUDA documentation or we should make the error message more inclusive.

Upstream docs: https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EVENT.html#group__CUDA__EVENT_1gdfb1178807353bbcaa9e245da497cf97

### How to Reproduce

n/a

### Expected behavior

n/a

### Operating System

_No response_

### nvidia-smi output

_No response_
Linked issue comments: user: login: carterbox
created_at: 2025-04-10T22:32:30Z
body: Assign me if desired.
author_association: CONTRIBUTOR
user: login: rwgk
created_at: 2025-04-10T22:35:14Z
body: > Assign me if desired.

Yes please :-)

> The underlying error code should be unsupressed

Purely from memory, I would have thought we talked about it and did something like raise from (conceptually). If that's not the case, I agree completely.
author_association: CONTRIBUTOR

Diff:
diff --git a/.github/workflows/test-wheel-windows.yml b/.github/workflows/test-wheel-windows.yml
index 4e48590a36f..9cb5f41eba1 100644
--- a/.github/workflows/test-wheel-windows.yml
+++ b/.github/workflows/test-wheel-windows.yml
@@ -55,9 +55,9 @@ jobs:
 
           if ('${{ inputs.local-ctk }}' -eq '1') {
             if ($TEST_CUDA_MAJOR -eq '12') {
-              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "nvjitlink"]'
+              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "nvjitlink", "thrust"]'
             } else {
-              $MINI_CTK_DEPS = '["nvcc", "nvrtc"]'
+              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "thrust"]'
             }
           }
 
diff --git a/cuda_core/cuda/core/experimental/_event.py b/cuda_core/cuda/core/experimental/_event.py
index 9710af2c00e..60654ec6c17 100644
--- a/cuda_core/cuda/core/experimental/_event.py
+++ b/cuda_core/cuda/core/experimental/_event.py
@@ -8,7 +8,15 @@
 from dataclasses import dataclass
 from typing import TYPE_CHECKING, Optional
 
-from cuda.core.experimental._utils.cuda_utils import CUDAError, check_or_create_options, driver, handle_return
+from cuda.core.experimental._utils.cuda_utils import (
+    CUDAError,
+    check_or_create_options,
+    driver,
+    handle_return,
+)
+from cuda.core.experimental._utils.cuda_utils import (
+    _check_driver_error as raise_if_driver_error,
+)
 
 if TYPE_CHECKING:
     import cuda.bindings
@@ -117,13 +125,31 @@ def __rsub__(self, other):
 
     def __sub__(self, other):
         # return self - other (in milliseconds)
+        err, timing = driver.cuEventElapsedTime(other.handle, self.handle)
         try:
-            timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
+            raise_if_driver_error(err)
+            return timing
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
-        return timing
+            if err == driver.CUresult.CUDA_ERROR_INVALID_HANDLE:
+                if self.is_timing_disabled or other.is_timing_disabled:
+                    explanation = (
+                        "Both Events must be created with timing enabled in order to subtract them; "
+                        "use EventOptions(enable_timing=True) when creating both events."
+                    )
+                else:
+                    explanation = (
+                        "Both Events must be recorded before they can be subtracted; "
+                        "use Stream.record() to record both events to a stream."
+                    )
+            elif err == driver.CUresult.CUDA_ERROR_NOT_READY:
+                explanation = (
+                    "One or both events have not completed; "
+                    "use Event.sync(), Stream.sync(), or Device.sync() to wait for the events to complete "
+                    "before subtracting them."
+                )
+            else:
+                raise e
+            raise RuntimeError(explanation) from e
 
     @property
     def is_timing_disabled(self) -> bool:
diff --git a/cuda_core/tests/test_event.py b/cuda_core/tests/test_event.py
index 108b8b14018..4895c0a6737 100644
--- a/cuda_core/tests/test_event.py
+++ b/cuda_core/tests/test_event.py
@@ -7,12 +7,15 @@
 # is strictly prohibited.
 
 import os
+import pathlib
 import time
 
+import numpy as np
 import pytest
 
 import cuda.core.experimental
-from cuda.core.experimental import Device, EventOptions
+from cuda.core.experimental import Device, EventOptions, LaunchConfig, Program, ProgramOptions, launch
+from cuda.core.experimental._memory import _DefaultPinnedMemorySource
 
 
 def test_event_init_disabled():
@@ -20,37 +23,27 @@ def test_event_init_disabled():
         cuda.core.experimental._event.Event()  # Ensure back door is locked.
 
 
-@pytest.mark.parametrize("enable_timing", [True, False, None])
-def test_timing(init_cuda, enable_timing):
-    options = EventOptions(enable_timing=enable_timing)
+def test_timing_success(init_cuda):
+    options = EventOptions(enable_timing=True)
     stream = Device().create_stream()
     delay_seconds = 0.5
     e1 = stream.record(options=options)
     time.sleep(delay_seconds)
     e2 = stream.record(options=options)
     e2.sync()
-    for e in (e1, e2):
-        assert e.is_timing_disabled == (True if enable_timing is None else not enable_timing)
-    if enable_timing:
-        elapsed_time_ms = e2 - e1
-        assert isinstance(elapsed_time_ms, float)
-        # Using a generous tolerance, to avoid flaky tests:
-        # We only want to exercise the __sub__ method, this test is not meant
-        # to stress-test the CUDA driver or time.sleep().
-        delay_ms = delay_seconds * 1000
-        if os.name == "nt":  # noqa: SIM108
-            # For Python <=3.10, the Windows timer resolution is typically limited to 15.6 ms by default.
-            generous_tolerance = 100
-        else:
-            # Most modern Linux kernels have a default timer resolution of 1 ms.
-            generous_tolerance = 20
-        assert delay_ms - generous_tolerance <= elapsed_time_ms < delay_ms + generous_tolerance
+    elapsed_time_ms = e2 - e1
+    assert isinstance(elapsed_time_ms, float)
+    # Using a generous tolerance, to avoid flaky tests:
+    # We only want to exercise the __sub__ method, this test is not meant
+    # to stress-test the CUDA driver or time.sleep().
+    delay_ms = delay_seconds * 1000
+    if os.name == "nt":  # noqa: SIM108
+        # For Python <=3.10, the Windows timer resolution is typically limited to 15.6 ms by default.
+        generous_tolerance = 100
     else:
-        with pytest.raises(RuntimeError) as e:
-            elapsed_time_ms = e2 - e1
-            msg = str(e)
-            assert "disabled by default" in msg
-            assert "CUDA_ERROR_INVALID_HANDLE" in msg
+        # Most modern Linux kernels have a default timer resolution of 1 ms.
+        generous_tolerance = 20
+    assert delay_ms - generous_tolerance <= elapsed_time_ms < delay_ms + generous_tolerance
 
 
 def test_is_sync_busy_waited(init_cuda):
@@ -80,3 +73,100 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+# TODO: improve this once path finder can find headers
+@pytest.mark.skipif(os.environ.get("CUDA_PATH") is None, reason="need libcu++ header")
+@pytest.mark.skipif(tuple(int(i) for i in np.__version__.split(".")[:2]) < (2, 1), reason="need numpy 2.1.0+")
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+
+    # This kernel is designed to busy loop until a signal is received
+    code = """
+#include <cuda/atomic>
+
+extern "C"
+__global__ void wait(int* val) {
+    cuda::atomic_ref<int, cuda::thread_scope_system> signal{*val};
+    while (true) {
+        if (signal.load(cuda::memory_order_relaxed)) {
+            break;
+        }
+    }
+}
+"""
+
+    arch = "".join(f"{i}" for i in device.compute_capability)
+    program_options = ProgramOptions(
+        std="c++17",
+        arch=f"sm_{arch}",
+        include_path=str(pathlib.Path(os.environ["CUDA_PATH"]) / pathlib.Path("include")),
+    )
+    prog = Program(code, code_type="c++", options=program_options)
+    mod = prog.compile(target_type="cubin")
+    ker = mod.get_kernel("wait")
+
+    mr = _DefaultPinnedMemorySource()
+    b = mr.allocate(4)
+    arr = np.from_dlpack(b).view(np.int32)
+    arr[0] = 0
+
+    config = LaunchConfig(grid=1, block=1)
+    ker_args = (arr.ctypes.data,)
+
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    launch(stream, config, ker, *ker_args)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on wait() to complete
+    with pytest.raises(RuntimeError, match="^One or both events have not completed."):
+        event3 - event1
+
+    arr[0] = 1
+    event3.sync()
+    event3 - event1  # this should work


Reviews:
user: login: kkraus14
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T01:52:43Z
user: login: rwgk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T03:42:14Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:05:18Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:08:08Z
user: login: rwgk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:31:27Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:37:38Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:41:24Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T21:44:46Z
user: login: rwgk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T22:01:23Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-17T19:59:27Z
user: login: kkraus14
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-18T01:35:21Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-18T02:00:26Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-18T02:01:01Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-18T02:02:40Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-18T02:06:36Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-18T02:13:41Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-21T21:35:09Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-21T21:53:18Z
user: login: carterbox
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-21T21:54:30Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-27T02:36:22Z
user: login: leofang
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-04-27T02:38:46Z
user: login: leofang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-27T04:09:13Z
user: login: leofang
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-27T04:25:37Z
user: login: rwgk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T18:43:32Z

Review comments:
diff_hunk: @@ -120,9 +120,27 @@ def __sub__(self, other):
         try:
             timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
+            error_message = str(e)
+            if "CUDA_ERROR_INVALID_HANDLE" in error_message:
path: cuda_core/cuda/core/experimental/_event.py
user: login: kkraus14
body: I think instead of using `handle_return` above and working on the error message here we're be better off using the actual return code from the `driver.cuEventElapsedTime(other.handle, self.handle)` and constructing the message to raise with based on that instead?
created_at: 2025-04-15T01:52:02Z
author_association: COLLABORATOR
diff_hunk: @@ -120,9 +120,27 @@ def __sub__(self, other):
         try:
             timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
+            error_message = str(e)
+            if "CUDA_ERROR_INVALID_HANDLE" in error_message:
path: cuda_core/cuda/core/experimental/_event.py
user: login: rwgk
body: Maybe:

In _utils/cuda_utils.py, rename `_check_driver_error` (private name) → `raise_if_driver_error` (new and public name), then

```python
err, timing = driver.cuEventElapsedTime(other.handle, self.handle)
try:
    raise_if_driver_error(err) 
    return timing
except CUDAError as e:
    if err == driver.CUresult.CUDA_ERROR_INVALID_HANDLE:
        ....
    elif err == driver.CUresult. CUDA_ERROR_NOT_READY:
        .... 
    else:
        raise e
    raise RuntimeError(explanation) from e
```

created_at: 2025-04-15T03:05:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=enabled)
+    stream.sync()
+    event2 - event1
path: cuda_core/tests/test_event.py
user: login: rwgk
body: I'd delete this last block, because that's already covered more comprehensively under the existing `test_timing` (`test_timing_success`).
created_at: 2025-04-15T03:28:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
path: cuda_core/tests/test_event.py
user: login: rwgk
body: I'd insert the new tests right after the existing `test_timing`.

Looking at the existing test with a fresh eye, I believe this needs to be unindented (i.e. the existing test is incorrect):

* https://github.com/NVIDIA/cuda-python/blob/37b401a0b6d8d92dbd4f8b662e02243f9e3e7097/cuda_core/tests/test_event.py#L51-L53

However, that part is made completely redundant by the new tests anyway: better remove?

I think ideally it's best to rename the existing test to `test_timing_success` and to remove the `enable_timing` argument entirely.

Then simply add a couple lines in the new test:

```
    event1 = stream.record(options=enabled)
    event2 = stream.record(options=disabled)
    assert event1.enable_timing is True
    assert event2.enable_timing is False
```
created_at: 2025-04-15T03:29:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -120,9 +120,27 @@ def __sub__(self, other):
         try:
             timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
+            error_message = str(e)
+            if "CUDA_ERROR_INVALID_HANDLE" in error_message:
path: cuda_core/cuda/core/experimental/_event.py
user: login: carterbox
body: e7f0f85
created_at: 2025-04-15T21:05:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=enabled)
+    stream.sync()
+    event2 - event1
path: cuda_core/tests/test_event.py
user: login: carterbox
body: d66580e64ad13c92b
created_at: 2025-04-15T21:08:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=enabled)
+    stream.sync()
+    event2 - event1
path: cuda_core/tests/test_event.py
user: login: rwgk
body: Did you maybe forget to push the commits?
created_at: 2025-04-15T21:31:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=enabled)
+    stream.sync()
+    event2 - event1
path: cuda_core/tests/test_event.py
user: login: carterbox
body: No. I'm still working on addressing feedback and would prefer to push the commits all at once instead of triggering the CI multiple times.
created_at: 2025-04-15T21:37:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
path: cuda_core/tests/test_event.py
user: login: carterbox
body: 99218bb9a7d53bbe3ff2d0f0d1cc97e0547bcfd8
created_at: 2025-04-15T21:41:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
+    with pytest.raises(RuntimeError, match="^One or both events have not completed."):
+        event3 - event1
path: cuda_core/tests/test_event.py
user: login: carterbox
body: Locally, this test will randomly fail (failure meaning it completes without error on line 129). I don't understand why. Any ideas? I believe that I have stated on line 127 is true.
created_at: 2025-04-15T21:44:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
+    with pytest.raises(RuntimeError, match="^One or both events have not completed."):
+        event3 - event1
path: cuda_core/tests/test_event.py
user: login: rwgk
body: I can only glance through right now. Looks good to me. I will look carefully later today.

> Any ideas?

No ... but my mental model of the event handling is still very much under development.

What do you think about the ChatGPT analysis?

https://chatgpt.com/share/67fed709-6618-8008-99e2-b3db7ca79a21
created_at: 2025-04-15T22:01:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
+    with pytest.raises(RuntimeError, match="^One or both events have not completed."):
+        event3 - event1
path: cuda_core/tests/test_event.py
user: login: carterbox
body: ChatGPT suggest synchronizing the stream to ensure that event3 is recorded, but if we synchronize the stream, wouldn't the host wait indefinitely? My understanding of stream.synchronize() is to make the host wait for the stream to complete all its current tasks, but by design, we are trying to make a stream that never completes so that event3 is not ready when we try to time it.

So, I don't trust ChatGPT's response.

created_at: 2025-04-17T19:59:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: kkraus14
body: Wouldn't there be no work recorded in `event2` here so the `stream.wait(event2)` wouldn't actually wait on anything? Then `event3` recording on the stream there again wouldn't be any actual work recorded so it very well could be finished?
created_at: 2025-04-18T01:35:21Z
author_association: COLLABORATOR
diff_hunk: @@ -120,9 +120,27 @@ def __sub__(self, other):
         try:
             timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
+            error_message = str(e)
+            if "CUDA_ERROR_INVALID_HANDLE" in error_message:
path: cuda_core/cuda/core/experimental/_event.py
user: login: leofang
body: Do we want to re-raise `RuntimeError` or just the same `CUDAError`?
created_at: 2025-04-18T02:00:26Z
author_association: MEMBER
diff_hunk: @@ -80,3 +80,63 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=enabled)
+    stream.sync()
+    event2 - event1
path: cuda_core/tests/test_event.py
user: login: leofang
body: Our CI is manual trigger only, so don't worry about excessive pushes 🙂 
created_at: 2025-04-18T02:01:01Z
author_association: MEMBER
diff_hunk: @@ -120,9 +120,27 @@ def __sub__(self, other):
         try:
             timing = handle_return(driver.cuEventElapsedTime(other.handle, self.handle))
         except CUDAError as e:
-            raise RuntimeError(
-                "Timing capability must be enabled in order to subtract two Events; timing is disabled by default."
-            ) from e
+            error_message = str(e)
+            if "CUDA_ERROR_INVALID_HANDLE" in error_message:
path: cuda_core/cuda/core/experimental/_event.py
user: login: leofang
body: nvm, we already raise `RuntimeError` before this PR
created_at: 2025-04-18T02:02:40Z
author_association: MEMBER
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: leofang
body: This is interesting, I would think that because `event2` has no work recorded, `stream.wait(event2)` should raise (maybe `cudaErrorInvalidResourceHandle`?) instead of no-op.
created_at: 2025-04-18T02:06:36Z
author_association: MEMBER
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: leofang
body: Took a quick look at the actual implementation, and Keith was right here. It's a no-op and `cudaSuccess` is returned if `event2` is not recorded.
created_at: 2025-04-18T02:13:41Z
author_association: MEMBER
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: carterbox
body: So what's the solution for testing this case? Launching a kernel that takes a long time to complete or never returns? I believe cuda.core now has all the necessary parts to compile and launch such a kernel from this test?
created_at: 2025-04-21T21:35:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: carterbox
body: [4314886](https://github.com/NVIDIA/cuda-python/pull/559/commits/4314886115551cbf7245cc8157298845cb464b1e)
created_at: 2025-04-21T21:53:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
+    with pytest.raises(RuntimeError, match="^One or both events have not completed."):
+        event3 - event1
path: cuda_core/tests/test_event.py
user: login: carterbox
body: Added a custom kernel to the test that never completes. [4314886](https://github.com/NVIDIA/cuda-python/pull/559/commits/4314886115551cbf7245cc8157298845cb464b1e)
created_at: 2025-04-21T21:54:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -80,3 +70,60 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+def test_error_timing_incomplete():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    stream.wait(event2)
+    event3 = stream.record(options=enabled)
+
+    # event3 will never complete because the stream is waiting on event2 which is never recorded
path: cuda_core/tests/test_event.py
user: login: leofang
body: @carterbox It is nerve wrecking to leave the kernel spinning and not clean it up after the event test is completed. I pushed commit add0ba1 to allow a signal passed from host to the busy kernel.
created_at: 2025-04-27T02:36:21Z
author_association: MEMBER
diff_hunk: @@ -55,9 +55,9 @@ jobs:
 
           if ('${{ inputs.local-ctk }}' -eq '1') {
             if ($TEST_CUDA_MAJOR -eq '12') {
-              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "nvjitlink"]'
+              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "nvjitlink", "thrust"]'
             } else {
-              $MINI_CTK_DEPS = '["nvcc", "nvrtc"]'
+              $MINI_CTK_DEPS = '["nvcc", "nvrtc", "thrust"]'
path: .github/workflows/test-wheel-windows.yml
user: login: leofang
body: Filed an internal ticket KITMAKER-2018. We need to use `thrust` here to get CCCL headers...
created_at: 2025-04-27T04:09:13Z
author_association: MEMBER
diff_hunk: @@ -80,3 +73,100 @@ def test_is_done(init_cuda):
     # Without a sync, the captured work might not have yet completed
     # Therefore this check should never raise an exception
     assert event.is_done in (True, False)
+
+
+def test_error_timing_disabled():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    disabled = EventOptions(enable_timing=False)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = stream.record(options=disabled)
+    assert not event1.is_timing_disabled
+    assert event2.is_timing_disabled
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+    event1 = stream.record(options=disabled)
+    event2 = stream.record(options=disabled)
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be created with timing enabled"):
+        event2 - event1
+
+
+def test_error_timing_recorded():
+    device = Device()
+    device.set_current()
+    enabled = EventOptions(enable_timing=True)
+    stream = device.create_stream()
+
+    event1 = stream.record(options=enabled)
+    event2 = device.create_event(options=enabled)
+    event3 = device.create_event(options=enabled)
+
+    stream.sync()
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event2 - event1
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event1 - event2
+    with pytest.raises(RuntimeError, match="^Both Events must be recorded"):
+        event3 - event2
+
+
+# TODO: improve this once path finder can find headers
+@pytest.mark.skipif(os.environ.get("CUDA_PATH") is None, reason="need libcu++ header")
+@pytest.mark.skipif(tuple(int(i) for i in np.__version__.split(".")[:2]) < (2, 1), reason="need numpy 2.1.0+")
+def test_error_timing_incomplete():
path: cuda_core/tests/test_event.py
user: login: rwgk
body: Wow, this is an amazingly sophisticated unit test setup.
created_at: 2025-04-28T18:43:31Z
author_association: CONTRIBUTOR

PR comments:
user: login: copy-pr-bot[bot]
created_at: 2025-04-14T23:26:13Z
body: This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities [here](https://docs.gha-runners.nvidia.com/cpr/vetters).

Contributors can view more details about this message [here](https://docs.gha-runners.nvidia.com/cpr/contributors).

author_association: CONTRIBUTOR
user: login: carterbox
created_at: 2025-04-14T23:31:50Z
body: /ok to test 8782dfadb0f1a04f135ca21cab9ae3602ebd59bb
author_association: CONTRIBUTOR
user: login: kkraus14
created_at: 2025-04-15T01:18:04Z
body: /ok to test 8782dfadb0f1a04f135ca21cab9ae3602ebd59bb
author_association: COLLABORATOR
user: login: github-actions[bot]
created_at: 2025-04-15T01:36:43Z
body: Doc Preview CI
:---:
| <p></p> :rocket: View preview at <br> https://nvidia.github.io/cuda-python/pr-preview/pr-559/ <br>
| <br> https://nvidia.github.io/cuda-python/pr-preview/pr-559/cuda-core/ <br>
| <br> https://nvidia.github.io/cuda-python/pr-preview/pr-559/cuda-bindings/ <br><br>
| <h6><br> Preview will be ready when the GitHub Pages deployment is complete. <br><br></h6>
<!-- Sticky Pull Request Commentpr-preview -->
author_association: NONE
user: login: rwgk
created_at: 2025-04-15T21:42:57Z
body: I think we are set up so that the CI will never trigger automatically. Only of you leave the ok to test comment. Certainly in draft mode. 
author_association: CONTRIBUTOR
user: login: carterbox
created_at: 2025-04-15T21:46:04Z
body: /ok to test 99218bb9a7d53bbe3ff2d0f0d1cc97e0547bcfd8
author_association: CONTRIBUTOR
user: login: leofang
created_at: 2025-04-27T02:35:06Z
body: /ok to test add0ba1
author_association: MEMBER
user: login: leofang
created_at: 2025-04-27T03:11:17Z
body: /ok to test 638990d
author_association: MEMBER
user: login: leofang
created_at: 2025-04-27T03:57:01Z
body: /ok to test 9904f3d
author_association: MEMBER
user: login: leofang
created_at: 2025-04-27T04:27:10Z
body: Thanks for nice improvements, Daniel!
author_association: MEMBER
user: login: github-actions[bot]
created_at: 2025-04-27T04:45:13Z
body: Doc Preview CI
:---:
Preview removed because the pull request was closed or merged.
<!-- Sticky Pull Request Commentpr-preview -->
author_association: NONE