Repository: hpcc-systems/HPCC-Platform
Pull request #19635: HPCC-33646 Protect against overlapping start and stop recording
Author: login: ghalliday
Created: 2025-03-18T16:39:27Z | Merged: 2025-03-19T17:17:59Z

<!-- Thank you for submitting a pull request to the HPCC project

 PLEASE READ the following before proceeding.

 This project only accepts pull requests related to open JIRA issues.
 If suggesting a new feature or change, please discuss it in a JIRA issue first.
 If fixing a bug, there should be an issue describing it with steps to reproduce.
 The title line of the pull request (and of each commit within it) should refer to the
 associated issue using the format:

 HPCC-nnnnn Short description of issue

 This will allow the Jira ticket to be automatically updated to refer to this pull request,
 and will ensure that the automatically-generated changelog is properly formatted.
 Where a pull request contains a single commit the pull request title will be set automatically,
 assuming that the commit has followed the proper guidelines.

 Please go over all the following points, and put an `x` in all the boxes that apply. You may find
 it easier to press the 'Create' button first then click on the checkboxes to edit the comment.
-->

## Type of change:
- [ ] This change is a bug fix (non-breaking change which fixes an issue).
- [ ] This change is a new feature (non-breaking change which adds functionality).
- [x] This change improves the code (refactor or other change that does not change the functionality)
- [ ] This change fixes warnings (the fix does not alter the functionality or the generated code)
- [ ] This change is a breaking change (fix or feature that will cause existing behavior to change).
- [ ] This change alters the query API (existing queries will have to be recompiled)

## Checklist:
- [x] My code follows the code style of this project.
  - [x] My code does not create any new warnings from compiler, build system, or lint.
- [x] The commit message is properly formatted and free of typos.
  - [x] The commit message title makes sense in a changelog, by itself.
  - [x] The commit is signed.
- [ ] My change requires a change to the documentation.
  - [ ] I have updated the documentation accordingly, or...
  - [ ] I have created a JIRA ticket to update the documentation.
  - [ ] Any new interfaces or exported functions are appropriately commented.
- [x] I have read the CONTRIBUTORS document.
- [x] The change has been fully tested:
  - [x] I have added tests to cover my changes.
  - [x] All new and existing tests passed.
  - [ ] I have checked that this change does not introduce memory leaks.
  - [ ] I have used Valgrind or similar tools to check for potential issues.
- [ ] I have given due consideration to all of the following potential concerns:
  - [ ] Scalability
  - [ ] Performance
  - [ ] Security
  - [ ] Thread-safety
  - [ ] Cloud-compatibility
  - [ ] Premature optimization
  - [ ] Existing deployed queries will not be broken
  - [ ] This change fixes the problem, not just the symptom
  - [ ] The target branch of this pull request is appropriate for such a change.
- [ ] There are no similar instances of the same problem that should be addressed
  - [ ] I have addressed them here
  - [ ] I have raised JIRA issues to address them separately
- [ ] This is a user interface / front-end modification
  - [ ] I have tested my changes in multiple modern browsers
  - [ ] The component(s) render as expected

## Smoketest:
- [ ] Send notifications about my Pull Request position in Smoketest queue.
- [ ] Test my draft Pull Request.

## Testing:
<!-- Please describe how this change has been tested.-->

<!-- Thank you for taking the time to submit this pull request and to answer all of the above-->



Diff:
diff --git a/system/jlib/jevent.cpp b/system/jlib/jevent.cpp
index 91a5f6e5e20..85039a57054 100644
--- a/system/jlib/jevent.cpp
+++ b/system/jlib/jevent.cpp
@@ -224,12 +224,15 @@ void EventRecorder::checkAttrValue(EventAttr attr, size_t size)
     assertex(expectedSize == 0 || expectedSize == size);
 }
 
-void EventRecorder::startRecording(const char * optionsText, const char * filename)
+bool EventRecorder::startRecording(const char * optionsText, const char * filename)
 {
     assertex(filename);
     CriticalBlock block(cs);
-    if (isActive())
-        return;
+    if (!isStopped)
+        return false;
+    assertex(!isStarted);
+    isStarted = true;
+    isStopped = false;
 
     auto processOption = [this](const char * option, const char * valueText)
     {
@@ -268,12 +271,18 @@ void EventRecorder::startRecording(const char * optionsText, const char * filena
         counts[i] = 0;
 
     recordingEvents.store(true, std::memory_order_release);
+    return true;
 }
 
-void EventRecorder::stopRecording()
+bool EventRecorder::stopRecording()
 {
-    if (!isActive())
-        return;
+    {
+        CriticalBlock block(cs);
+        if (!isStarted)
+            return false;
+        isStarted = false;
+        assertex(!isStopped);
+    }
 
     //MORE: Protect against startRecording() being called concurrently, by introducing another boolean to
     //indicate if it is active, which is only cleared once this function completes.
@@ -301,7 +310,10 @@ void EventRecorder::stopRecording()
         //Flush the data, after waiting for a little while (or until committed == offset)?
         output->close();
         output.clear();
+        isStopped = true;
     }
+
+    return true;
 }
 
 //See notes above about reseving and committing events
@@ -371,7 +383,7 @@ static_assert(getSizeOfAttrs(true, 32768U, 1ULL, "boris", "blob") == 5 * sizeof(
 
 void EventRecorder::recordIndexLookup(unsigned fileid, offset_t offset, byte nodeKind, bool hit)
 {
-    if (!isActive())
+    if (!isRecording())
         return;
 
     size32_t requiredSize = sizeMessageHeaderFooter + getSizeOfAttrs(fileid, offset, nodeKind, hit);
@@ -387,7 +399,7 @@ void EventRecorder::recordIndexLookup(unsigned fileid, offset_t offset, byte nod
 
 void EventRecorder::recordIndexLoad(unsigned fileid, offset_t offset, byte nodeKind, size32_t size, __uint64 elapsedTime, __uint64 readTime)
 {
-    if (!isActive())
+    if (!isRecording())
         return;
 
     size32_t requiredSize = sizeMessageHeaderFooter + getSizeOfAttrs(fileid, offset, nodeKind, size, elapsedTime, readTime);
@@ -405,7 +417,7 @@ void EventRecorder::recordIndexLoad(unsigned fileid, offset_t offset, byte nodeK
 
 void EventRecorder::recordIndexEviction(unsigned fileid, offset_t offset, byte nodeKind, size32_t size)
 {
-    if (!isActive())
+    if (!isRecording())
         return;
 
     size32_t requiredSize = sizeMessageHeaderFooter + getSizeOfAttrs(fileid, offset, nodeKind, size);
@@ -421,7 +433,7 @@ void EventRecorder::recordIndexEviction(unsigned fileid, offset_t offset, byte n
 
 void EventRecorder::recordDaliConnect(const char * path, __uint64 id)
 {
-    if (!isActive())
+    if (!isRecording())
         return;
 
     size32_t requiredSize = sizeMessageHeaderFooter + getSizeOfAttrs(path, id);
@@ -435,7 +447,7 @@ void EventRecorder::recordDaliConnect(const char * path, __uint64 id)
 
 void EventRecorder::recordDaliDisconnect(__uint64 id)
 {
-    if (!isActive())
+    if (!isRecording())
         return;
 
     size32_t requiredSize = sizeMessageHeaderFooter + getSizeOfAttrs(id);
diff --git a/system/jlib/jevent.hpp b/system/jlib/jevent.hpp
index 45784f7ee69..7892d637840 100644
--- a/system/jlib/jevent.hpp
+++ b/system/jlib/jevent.hpp
@@ -80,10 +80,10 @@ class jlib_decl EventRecorder
 public:
     EventRecorder();
 
-    bool isActive() const { return recordingEvents.load(std::memory_order_acquire); }
+    bool isRecording() const { return recordingEvents.load(std::memory_order_acquire); }    // Are events being recorded? false if recording is paused
 
-    void startRecording(const char * optionsText, const char * filename);
-    void stopRecording();
+    bool startRecording(const char * optionsText, const char * filename);
+    bool stopRecording();
 
 //Functions for each of the events that can be recorded..
     void recordIndexLookup(unsigned fileid, offset_t offset, byte nodeKind, bool hit);
@@ -142,7 +142,9 @@ class jlib_decl EventRecorder
     static constexpr offset_t bufferMask = OutputBufferSize-1;
     static constexpr offset_t blockMask = OutputBlockSize-1;
 
-    std::atomic<bool> recordingEvents{false};
+    std::atomic<bool> recordingEvents{false};       // Are events being recorded? false if recording is paused.
+    bool isStarted{false};                      // Use 2 flags for whether started and stopped to ensure clean
+    bool isStopped{true};                       // termination in stopRecording()
     offset_type nextOffset{0};
     offset_type nextWriteOffset{0};
     unsigned counts[numBlocks] = {0};
@@ -164,6 +166,6 @@ extern jlib_decl EventRecorder eventRecorder;
 }
 
 inline EventRecorder & queryRecorder() { return EventRecorderInternal::eventRecorder; }
-inline bool recordingEvents() { return EventRecorderInternal::eventRecorder.isActive(); }
+inline bool recordingEvents() { return EventRecorderInternal::eventRecorder.isRecording(); }
 
 #endif
diff --git a/testing/unittests/jlibtests2.cpp b/testing/unittests/jlibtests2.cpp
index e6f5bbd26d9..22639122d15 100644
--- a/testing/unittests/jlibtests2.cpp
+++ b/testing/unittests/jlibtests2.cpp
@@ -66,11 +66,14 @@ class JlibEventTest : public CppUnit::TestFixture
             EventRecorder &recorder = queryRecorder();
 
             // Test that recording is initially inactive
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
 
             // Start recording
-            recorder.startRecording("traceid", "eventtrace.evt");
-            CPPUNIT_ASSERT(recorder.isActive());
+            CPPUNIT_ASSERT(recorder.startRecording("traceid", "eventtrace.evt"));
+            CPPUNIT_ASSERT(recorder.isRecording());
+
+            //Check that overlapping starts fail
+            CPPUNIT_ASSERT(!recorder.startRecording("traceid", "eventtrace.evtxxx"));
 
             // Record some events
             recorder.recordIndexLookup(1, branchOffset, NodeBranch, true);
@@ -79,12 +82,15 @@ class JlibEventTest : public CppUnit::TestFixture
             recorder.recordIndexEviction(1, branchOffset, NodeBranch, nodeSize);
 
             // Stop recording
-            recorder.stopRecording();
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(recorder.stopRecording());
+            CPPUNIT_ASSERT(!recorder.isRecording());
+
+            // Check that stopping again fails
+            CPPUNIT_ASSERT(!recorder.stopRecording());
 
             // Restart recording with a different filename
-            recorder.startRecording("threadid", "testfile.bin");
-            CPPUNIT_ASSERT(recorder.isActive());
+            CPPUNIT_ASSERT(recorder.startRecording("threadid", "testfile.bin"));
+            CPPUNIT_ASSERT(recorder.isRecording());
 
             // Record more events
             recorder.recordIndexLookup(2, 400, NodeLeaf, false);
@@ -100,8 +106,8 @@ class JlibEventTest : public CppUnit::TestFixture
             recorder.recordDaliDisconnect(987);
 
             // Stop recording again
-            recorder.stopRecording();
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(recorder.stopRecording());
+            CPPUNIT_ASSERT(!recorder.isRecording());
         }
         catch (IException * e)
         {
@@ -120,11 +126,11 @@ class JlibEventTest : public CppUnit::TestFixture
             EventRecorder &recorder = queryRecorder();
 
             // Test that recording is initially inactive
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
 
             // Start recording
-            recorder.startRecording("threadid", "eventtrace.evt");
-            CPPUNIT_ASSERT(recorder.isActive());
+            CPPUNIT_ASSERT(recorder.startRecording("threadid", "eventtrace.evt"));
+            CPPUNIT_ASSERT(recorder.isRecording());
 
             // Record some events
             for (unsigned i=0; i < 100'000; i++)
@@ -134,8 +140,8 @@ class JlibEventTest : public CppUnit::TestFixture
             }
 
             // Stop recording
-            recorder.stopRecording();
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(recorder.stopRecording());
+            CPPUNIT_ASSERT(!recorder.isRecording());
         }
         catch (IException * e)
         {
@@ -175,11 +181,11 @@ class JlibEventTest : public CppUnit::TestFixture
             EventRecorder &recorder = queryRecorder();
 
             // Test that recording is initially inactive
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
 
             // Start recording
-            recorder.startRecording("threadid", "eventtrace.evt");
-            CPPUNIT_ASSERT(recorder.isActive());
+            CPPUNIT_ASSERT(recorder.startRecording("threadid", "eventtrace.evt"));
+            CPPUNIT_ASSERT(recorder.isRecording());
 
             CIArrayOf<Thread> threads;
             for (unsigned t=0; t < numThreads; t++)
@@ -191,16 +197,16 @@ class JlibEventTest : public CppUnit::TestFixture
             if (delay)
             {
                 MilliSleep(delay);
-                recorder.stopRecording();
+                CPPUNIT_ASSERT(recorder.stopRecording());
             }
 
             ForEachItemIn(t2, threads)
                 threads.item(t2).join();
 
             if (!delay)
-                recorder.stopRecording();
+                CPPUNIT_ASSERT(recorder.stopRecording());
 
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
         }
         catch (IException * e)
         {


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: Copilot wasn't able to review any files in this pull request.
<details>
<summary>Files not reviewed (3)</summary>

* **system/jlib/jevent.cpp**: Language not supported
* **system/jlib/jevent.hpp**: Language not supported
* **testing/unittests/jlibtests2.cpp**: Language not supported
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T16:39:38Z
user: login: timothyklemm
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-18T17:23:42Z
user: login: timothyklemm
state: APPROVED
author_association: NONE
submitted_at: 2025-03-19T15:20:35Z

Review comments:
diff_hunk: @@ -120,11 +126,11 @@ class JlibEventTest : public CppUnit::TestFixture
             EventRecorder &recorder = queryRecorder();
 
             // Test that recording is initially inactive
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
 
             // Start recording
             recorder.startRecording("threadid", "eventtrace.evt");
path: testing/unittests/jlibtests2.cpp
user: login: timothyklemm
body: Should the return value be checked?
created_at: 2025-03-18T17:18:06Z
author_association: NONE
diff_hunk: @@ -135,7 +141,7 @@ class JlibEventTest : public CppUnit::TestFixture
 
             // Stop recording
             recorder.stopRecording();
path: testing/unittests/jlibtests2.cpp
user: login: timothyklemm
body: Ditto...
created_at: 2025-03-18T17:18:29Z
author_association: NONE
diff_hunk: @@ -175,11 +181,11 @@ class JlibEventTest : public CppUnit::TestFixture
             EventRecorder &recorder = queryRecorder();
 
             // Test that recording is initially inactive
-            CPPUNIT_ASSERT(!recorder.isActive());
+            CPPUNIT_ASSERT(!recorder.isRecording());
 
             // Start recording
             recorder.startRecording("threadid", "eventtrace.evt");
path: testing/unittests/jlibtests2.cpp
user: login: timothyklemm
body: Ditto...
created_at: 2025-03-18T17:18:59Z
author_association: NONE
diff_hunk: @@ -200,7 +206,7 @@ class JlibEventTest : public CppUnit::TestFixture
             if (!delay)
                 recorder.stopRecording();
path: testing/unittests/jlibtests2.cpp
user: login: timothyklemm
body: Ditto...
created_at: 2025-03-18T17:19:27Z
author_association: NONE

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-18T16:46:10Z
body: Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-33646

Jirabot Action Result:
Assigning user: gavin.halliday@lexisnexisrisk.com
Workflow Transition To: Merge Pending
Updated PR

author_association: NONE
user: login: ghalliday
created_at: 2025-03-19T14:23:44Z
body: I guess there is no harm in checking - better more checks than fewer - I will add.
author_association: MEMBER
user: login: ghalliday
created_at: 2025-03-19T14:30:15Z
body: Added CPPUNIT_ASSERT() around each call.  Squashed because it was a trival change - you could double check the changes by clicking on Compare above.

author_association: MEMBER
user: login: github-actions[bot]
created_at: 2025-03-19T17:47:18Z
body: Jirabot Action Result:
Added fix version: 9.12.0
Workflow Transition: 'Resolve issue'

author_association: NONE