Repository: CDCgov/prime-reportstream
Pull request #17756: [17510] - moving convert queue message call to report function
Author: login: oluwie
Labels: name: platform
description: Platform Team
Created: 2025-04-07T21:05:10Z | Merged: 2025-04-14T22:43:21Z

This PR removes the manual call to add the convert step to the queue in favor of using the pre-exising `actionHistory.track` paradigm.

**If you are suggesting a fix for a currently exploitable issue, please disclose the issue to the prime-reportstream team directly outside of GitHub instead of filing a PR, so we may immediately patch the affected systems before a disclosure. See [SECURITY.md/Reporting a Vulnerability](https://github.com/CDCgov/prime-reportstream/blob/main/SECURITY.md#reporting-a-vulnerability) for more information.**

Test Steps:
1. *Include steps to test these changes*
    - Run ./gradlew quickRun & submit a report to the reports endpoint and make sure the report runs through the pipeline as usual. 

## Changes
 - Replaces manual call to send convert event to the queue in favor of `actionHistory.trackFhirMessage` which keeps track all queued fhir messages and routes them to the appropriate queue. This was done to continue using existing paradigms by having the `ActionHististory` class route messages/events to their respective queues using it's internal logic. The existing `actionHistory.track` call couldn't be used because it's only used to process messages of type `Event`. Typically, a `CONVERT` Event call would have been used to to transition a message from the receive stop to the convert step. However, the convert event type lacks necessary data that the convert azure function requires. The azure convert function requires a message of base type `QueueMessage` and it would have been a heavier and more complicated lift to modify the `Event` type with the extra necessary data.  A new function `actionHistory.trackFhirMessage` was created that expects a message of type `QueueMessage` which is what the convert function expects.

Note: While this fixes the issue of queueing the convert message as the last step of the submission workflow, a more complete solution would be to also ensure all writes in a the submission flow occur in a single transaction. This would prevent all data integrity issues that can occur within the submission flow. A new ticket (#17785 ) has been created to handle this scenario

## Checklist

### Testing
- [x] Tested locally?
- [x] Ran `./prime test` or `./gradlew testSmoke` against local Docker ReportStream container?
- [ ] (For Changes to /frontend-react/...) Ran `npm run lint:write`?
- [x] Added tests?

### Process
- [x] Are there licensing issues with any new dependencies introduced?
- [x] Includes a summary of what a code reviewer should test/verify?
- [] Updated the release notes?
- [ ] Database changes are submitted as a separate PR?
- [ ] DevOps team has been notified if PR requires ops support?

## Linked Issues
- Fixes  #17510
- #17785 

## Specific Security-related subjects a reviewer should pay specific attention to
- Does this PR introduce new endpoints?
    - No
- Does this PR include changes in authentication and/or authorization of existing endpoints?
   - No
- Does this change introduce new dependencies that need vetting?
   - No
- Does this change require changes to our infrastructure?
  - No
- Does logging contain sensitive data?
   - No
- Does this PR include or remove any sensitive information itself?
  - No



Linked issue #17510: Receive Step: Only Send Queue Message Once Fully Complete
### User Story
<!--
What is a User Story?
A user story is a requirement for any functionality or feature, written down in one or two lines, about one and only one functionality or feature.

Use the following template when creating new User Stories/Tickets. Remove/replace any  _italicized_ text:

"As a _[User Persona - who are we building this for?]_
I want to _[Intent - what is this user trying to achieve?]_, 
so that _[Outcome - what is the value add to the user]_."
-->
As a maintainer of the Universal Pipeline,
I want the receive step to enqueue a Queue Message for the next step only when it has successfully completed ALL operations,
so that the downstream steps do not encounter data integrity issues.

### Description/Use Case
<!--
Use this section to describe the 'Why', and/or provide an example scenario in which this feature/functionality would be valueable.
-->
A Convert step Queue Message can be sent from the Receive step before the database has been properly updated. If for some reason the database does not get updated (due to a DB exception), then the convert step will try to update report_lineage with a parentID that does not exist and encounter an exception. 

This happened in prod: Search for Operation ID `c10285c27d4876f6fa9855ca34bb350b` in prod app insights for more info (March 4th). Here is a snapshot of the logging that shows a database write encountering an exception AFTER the convert step has already been queued:

<img width="822" src="https://api.zenhub.com/attachedFiles/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBOU1xQ1E9PSIsImV4cCI6bnVsbCwicHVyIjoiYmxvYl9pZCJ9fQ==--39ec63413cb44dc2b6183b5dfef6afd6ae0dae0a/Screenshot%202025-03-07%20at%202.56.12%E2%80%AFPM.png" alt="Screenshot 2025-03-07 at 2.56.12 PM.png" />


### Risks/Impacts/Considerations
<!--
Use this section to briefly list out any risks/impacts that may come about, as a result of the proposed solution.
- _[System performance may be slowed]_
- _[Only a single search parameter can be used]_
-->

### Dev Notes
<!--
Use this section to describe any useful technical information to duplicate an issue or explain requirements related to this ticket without providing implementation details.
- _[Provided is the data that was used to replicate the issue...]_
- _[To test, use SimpleReport upload CSV page to ...]_
-->
This scenario is handled for the UP-specific steps. See `FHIRFunctions.kt` for inspo:

```
    private fun runFhirEngine(
        message: String,
        dequeueCount: Int,
        fhirEngine: FHIREngine,
        actionHistory: ActionHistory,
    ): List<QueueMessage> {
        val messageContent = readMessage(fhirEngine.engineType, message, dequeueCount)

        try {
            val newMessages = databaseAccess.transactReturning { txn ->
                val results = fhirEngine.run(messageContent, actionLogger, actionHistory, txn)
                recordResults(message, actionHistory, txn)
                results
            }
            reportStreamEventService.sendQueuedEvents()
            return newMessages
        }
```

### Acceptance Criteria
<!--
What is Acceptance Criteria?
A set of conditions or business rules, as defined by the Product Owner, which the functionality or feature should satisfy, in order to be accepted by the Product Owner.

Write the acceptance criteria as a list that can be checked off as work progresses. For example:

- [ ] _[Time must be displayed as HH:MM:SS]_
- [ ] _[Delivery rate must be shown as a percentage]_
-->
- [x] DB writes in Receive step are guaranteed to complete before a QueueMessage for the next step is sent out.
Linked issue comments: user: login: arnejduranovic
created_at: 2025-03-10T16:17:20Z
body: Hey team! Please [add your planning poker estimate](https://app.zenhub.com/workspaces/Platform-6182b02547c1130010f459db/issues/gh/CDCgov/prime-reportstream/17510?planning-poker) with Zenhub @adegolier @JFisk42 @wcutshall @kant777
author_association: CONTRIBUTOR

Diff:
diff --git a/prime-router/src/main/kotlin/SubmissionReceiver.kt b/prime-router/src/main/kotlin/SubmissionReceiver.kt
index fe819504005..2ba790050d7 100644
--- a/prime-router/src/main/kotlin/SubmissionReceiver.kt
+++ b/prime-router/src/main/kotlin/SubmissionReceiver.kt
@@ -1,7 +1,6 @@
 package gov.cdc.prime.router
 
 import gov.cdc.prime.reportstream.shared.BlobUtils
-import gov.cdc.prime.reportstream.shared.QueueMessage
 import gov.cdc.prime.router.azure.ActionHistory
 import gov.cdc.prime.router.azure.Event
 import gov.cdc.prime.router.azure.ProcessEvent
@@ -344,19 +343,17 @@ class UniversalPipelineReceiver : SubmissionReceiver {
         val processEvent = ProcessEvent(eventAction, report.id, options, defaults, routeTo)
         workflowEngine.insertProcessTask(report, report.bodyFormat.toString(), blobInfo.blobUrl, processEvent)
 
-        // Only add to queue if the sender/ is enabled
+        // Only track and add to queue if the sender/ is enabled
         if (sender.customerStatus != CustomerStatus.INACTIVE) {
-            // move to processing (send to <elrProcessQueueName> queue)
-            workflowEngine.queue.sendMessage(
-                QueueMessage.elrConvertQueueName,
+            actionHistory.trackFhirMessage(
                 FhirConvertQueueMessage(
-                    report.id,
-                    blobInfo.blobUrl,
-                    BlobUtils.digestToString(blobInfo.digest),
-                    sender.fullName,
-                    sender.topic,
-                    sender.schemaName
-                ).serialize()
+                report.id,
+                blobInfo.blobUrl,
+                BlobUtils.digestToString(blobInfo.digest),
+                sender.fullName,
+                sender.topic,
+                sender.schemaName
+            )
             )
         }
 
diff --git a/prime-router/src/main/kotlin/azure/ActionHistory.kt b/prime-router/src/main/kotlin/azure/ActionHistory.kt
index 3f620279a36..1d2ed4c6212 100644
--- a/prime-router/src/main/kotlin/azure/ActionHistory.kt
+++ b/prime-router/src/main/kotlin/azure/ActionHistory.kt
@@ -8,6 +8,7 @@ import com.microsoft.azure.functions.HttpStatusType
 import com.networknt.org.apache.commons.validator.routines.InetAddressValidator
 import fhirengine.engine.CustomFhirPathFunctions
 import gov.cdc.prime.reportstream.shared.BlobUtils
+import gov.cdc.prime.reportstream.shared.QueueMessage
 import gov.cdc.prime.router.ActionLog
 import gov.cdc.prime.router.ActionLogLevel
 import gov.cdc.prime.router.ClientSource
@@ -113,6 +114,11 @@ class ActionHistory(
      */
     val messages = mutableListOf<Event>()
 
+    /**
+     * FHIR Queue Messages to be queued in an azure queue as part of the result of this action.
+     */
+    val fhirQueueMessages = mutableListOf<QueueMessage>()
+
     /**
      *
      * Collection of all the parent-child report relationships created by this action.
@@ -140,6 +146,11 @@ class ActionHistory(
         messages.add(event)
     }
 
+    /** Adds a FHIR queue message to the fhirQueueMessages list to be added to the queue later */
+    fun trackFhirMessage(message: QueueMessage) {
+        fhirQueueMessages.add(message)
+    }
+
     /**
      * Track a list of logs that happened during this action
      *
@@ -770,6 +781,16 @@ class ActionHistory(
         }
     }
 
+    /**
+     *  Sends any FHIR queue messages to the appropriate queue
+     */
+    fun queueFhirMessages(workflowEngine: WorkflowEngine) {
+        fhirQueueMessages.forEach { message ->
+            workflowEngine.queue.sendMessage(message.messageQueueName, message.serialize())
+            logger.debug("FHIR Message queued: $message")
+        }
+    }
+
     /**
      * Use the detailed item lineage to exactly/correctly generate the report parent/child relationships.
      *
diff --git a/prime-router/src/main/kotlin/azure/ReportFunction.kt b/prime-router/src/main/kotlin/azure/ReportFunction.kt
index 9cfdabecb60..9b0d8bffd9e 100644
--- a/prime-router/src/main/kotlin/azure/ReportFunction.kt
+++ b/prime-router/src/main/kotlin/azure/ReportFunction.kt
@@ -596,9 +596,12 @@ class ReportFunction(
             submission
         )
 
-        // queue messages here after all task / action records are in
+        // queue events here after all task / action records are in
         actionHistory.queueMessages(workflowEngine)
 
+        // queue fhir messages after all tasks / action records are in
+        actionHistory.queueFhirMessages(workflowEngine)
+
         return response
     }
 
diff --git a/prime-router/src/test/kotlin/SubmissionReceiverTests.kt b/prime-router/src/test/kotlin/SubmissionReceiverTests.kt
index bcfd859ebc2..550b0ce1305 100644
--- a/prime-router/src/test/kotlin/SubmissionReceiverTests.kt
+++ b/prime-router/src/test/kotlin/SubmissionReceiverTests.kt
@@ -3,7 +3,6 @@ package gov.cdc.prime.router
 import assertk.assertFailure
 import assertk.assertThat
 import assertk.assertions.isInstanceOf
-import gov.cdc.prime.reportstream.shared.QueueMessage
 import gov.cdc.prime.router.azure.ActionHistory
 import gov.cdc.prime.router.azure.BlobAccess
 import gov.cdc.prime.router.azure.DatabaseAccess
@@ -785,7 +784,6 @@ class SubmissionReceiverTests {
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { SubmissionReceiver.doDuplicateDetection(any(), any(), any()) } returns Unit
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act
         receiver.validateAndMoveToProcessing(
@@ -808,7 +806,7 @@ class SubmissionReceiverTests {
             // SubmissionReceiver.doDuplicateDetection(any(), any(), any())
             actionHistory.trackLogs(emptyList())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -868,7 +866,6 @@ class SubmissionReceiverTests {
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { SubmissionReceiver.doDuplicateDetection(any(), any(), any()) } returns Unit
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act
         receiver.validateAndMoveToProcessing(
@@ -893,7 +890,7 @@ class SubmissionReceiverTests {
             engine.insertProcessTask(any(), format.toString(), any(), any())
         }
         verify(exactly = 0) {
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -939,7 +936,6 @@ class SubmissionReceiverTests {
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { SubmissionReceiver.doDuplicateDetection(any(), any(), any()) } returns Unit
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act
         receiver.validateAndMoveToProcessing(
@@ -962,7 +958,7 @@ class SubmissionReceiverTests {
             engine.insertProcessTask(any(), any(), any(), any())
         }
         verify(exactly = 0) {
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
             SubmissionReceiver.doDuplicateDetection(any(), any(), any())
         }
     }
@@ -1009,7 +1005,6 @@ class SubmissionReceiverTests {
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { SubmissionReceiver.doDuplicateDetection(any(), any(), any()) } returns Unit
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act
         receiver.validateAndMoveToProcessing(
@@ -1032,7 +1027,7 @@ class SubmissionReceiverTests {
             engine.insertProcessTask(any(), any(), any(), any())
         }
         verify(exactly = 0) {
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
             SubmissionReceiver.doDuplicateDetection(any(), any(), any())
         }
     }
@@ -1078,7 +1073,6 @@ class SubmissionReceiverTests {
         every { engine.recordReceivedReport(any(), any(), any(), any(), any()) } returns blobInfo
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act
         var exceptionThrown = false
@@ -1106,7 +1100,7 @@ class SubmissionReceiverTests {
             engine.recordReceivedReport(any(), any(), any(), any(), any())
             actionHistory.trackLogs(emptyList())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -1151,7 +1145,6 @@ class SubmissionReceiverTests {
         every { engine.recordReceivedReport(any(), any(), any(), any(), any()) } returns blobInfo
         every { engine.routeReport(any(), any(), any(), any(), any()) } returns routeResult
         every { engine.insertProcessTask(any(), any(), any(), any()) } returns Unit
-        every { queueMock.sendMessage(QueueMessage.elrConvertQueueName, any()) } returns ""
 
         // act / assert
         assertFailure {
@@ -1173,7 +1166,7 @@ class SubmissionReceiverTests {
             engine.recordReceivedReport(any(), any(), any(), any(), any())
             actionHistory.trackLogs(emptyList())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -1232,7 +1225,7 @@ class SubmissionReceiverTests {
             engine.recordReceivedReport(any(), any(), any(), any(), any())
             actionHistory.trackLogs(emptyList())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -1290,7 +1283,7 @@ class SubmissionReceiverTests {
         verify(exactly = 0) {
             engine.recordReceivedReport(any(), any(), any(), any(), any())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
@@ -1346,7 +1339,7 @@ class SubmissionReceiverTests {
         verify(exactly = 0) {
             engine.recordReceivedReport(any(), any(), any(), any(), any())
             engine.insertProcessTask(any(), any(), any(), any())
-            queueMock.sendMessage(QueueMessage.elrConvertQueueName, any())
+            actionHistory.trackFhirMessage(any())
         }
     }
 
diff --git a/prime-router/src/test/kotlin/azure/ActionHistoryTests.kt b/prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
index b6f26ae29b0..2efe4ee74eb 100644
--- a/prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
+++ b/prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
@@ -10,6 +10,7 @@ import assertk.assertions.isNull
 import assertk.assertions.isTrue
 import com.microsoft.azure.functions.HttpMethod
 import gov.cdc.prime.reportstream.shared.BlobUtils
+import gov.cdc.prime.reportstream.shared.QueueMessage
 import gov.cdc.prime.router.ActionLog
 import gov.cdc.prime.router.ActionLogLevel
 import gov.cdc.prime.router.ClientSource
@@ -31,6 +32,7 @@ import gov.cdc.prime.router.azure.observability.event.AzureEventService
 import gov.cdc.prime.router.azure.observability.event.ReportEventData
 import gov.cdc.prime.router.azure.observability.event.ReportStreamEventService
 import gov.cdc.prime.router.common.JacksonMapperUtilities
+import gov.cdc.prime.router.fhirengine.engine.FhirConvertQueueMessage
 import gov.cdc.prime.router.fhirengine.utils.FhirTranscoder
 import gov.cdc.prime.router.report.ReportService
 import gov.cdc.prime.router.unittest.UnitTestUtils
@@ -39,6 +41,7 @@ import io.mockk.mockk
 import io.mockk.mockkClass
 import io.mockk.mockkConstructor
 import io.mockk.mockkObject
+import io.mockk.slot
 import io.mockk.spyk
 import io.mockk.unmockkAll
 import io.mockk.verify
@@ -48,6 +51,7 @@ import java.time.OffsetDateTime
 import java.util.UUID
 import kotlin.test.Test
 import kotlin.test.assertContains
+import kotlin.test.assertEquals
 import kotlin.test.assertNotEquals
 import kotlin.test.assertNotNull
 
@@ -873,7 +877,7 @@ class ActionHistoryTests {
     }
 
     @Test
-    fun `trackActionParams the azure client ip`() {
+    fun `test trackActionParams the azure client ip`() {
         val actionHistory = ActionHistory(TaskAction.receive)
 
         val mockHttpRequestMessage = MockHttpRequestMessage()
@@ -884,7 +888,7 @@ class ActionHistoryTests {
     }
 
     @Test
-    fun `trackActionParams correctly filters headers and query params`() {
+    fun `test trackActionParams correctly filters headers and query params`() {
         val actionHistory = ActionHistory(TaskAction.receive)
 
         val mockHttpRequestMessage = MockHttpRequestMessage(method = HttpMethod.POST)
@@ -908,4 +912,56 @@ class ActionHistoryTests {
             )
         )
     }
+
+    @Test
+    fun `test queueFhirMessages properly add a fhir message to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        assertEquals(1, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly tracks and queues a fhir message`() {
+        val metadata = UnitTestUtils.simpleMetadata
+        val actionHistory = spyk(ActionHistory(TaskAction.receive))
+        val workflowEngine = mockkClass(WorkflowEngine::class)
+        val messageQueue = mockk<QueueAccess>()
+
+        every { workflowEngine.metadata } returns metadata
+        every { workflowEngine.queue } returns messageQueue
+
+        val queueNameSlot = slot<String>()
+        val messageSlot = slot<String>()
+
+        every { messageQueue.sendMessage(capture(queueNameSlot), capture(messageSlot), any()) } returns "some-id"
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        actionHistory.queueFhirMessages(workflowEngine)
+
+        verify(exactly = 1) {
+            messageQueue.sendMessage(any(), any(), any())
+        }
+
+        assertEquals(queueNameSlot.captured, QueueMessage.elrConvertQueueName)
+        assertEquals(messageSlot.captured, queueMessage.serialize())
+    }
 }
\ No newline at end of file


Reviews:
user: login: arnejduranovic
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-07T22:03:10Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T16:00:51Z
user: login: arnejduranovic
body: I think this is a good attempt, just needs a bit of clean up. The changes to ActionHistory are simple and fit in with what that class is doing, so its not making it less cohesive. Although, ActionHistory itself probably needs to be overhauled at some point, that's a task for later.

Please make sure to include good detail in the Pull Request description on why this change was necessary (why we couldn't just use trackEvent). 
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T15:05:58Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T15:22:14Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T15:24:17Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T15:46:10Z
user: login: JFisk42
body: Everything is looking good and was able to verify locally that there are no hiccups with this change 🚀 . Only thing that would be really nice is if we could add a unit test for the ActionHistory changes.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T00:57:06Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-11T19:40:33Z
user: login: JFisk42
body: Great looking tests!
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-11T21:19:34Z
user: login: arnejduranovic
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T15:48:01Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T16:08:38Z
user: login: oluwie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T16:08:47Z
user: login: arnejduranovic
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T21:23:59Z

Review comments:
diff_hunk: @@ -541,7 +549,7 @@ class ReportFunction(
 
                     reportEventService.sendReportEvent(
path: prime-router/src/main/kotlin/azure/ReportFunction.kt
user: login: arnejduranovic
body: This also probably should get moved to the bottom once all DB writes have completed?
created_at: 2025-04-07T21:49:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -596,6 +603,22 @@ class ReportFunction(
             submission
         )
 
+        // Only add to queue if the sender/ is enabled
+        if (sender.customerStatus != CustomerStatus.INACTIVE && reportWithBlob != null) {
path: prime-router/src/main/kotlin/azure/ReportFunction.kt
user: login: arnejduranovic
body: This conditional logic makes sense in the context of this ticket, but it won't really make logical sense to anyone down the line. For example, it makes sense that we want to send a queue message if the customer is NOT inactive, but what does whether a "reportWithBlob" exists or not have anything to do with it? Logic that is closer to the actual underlying _business requirement_ is something like:

`if (sender.customerStatus != CustomerStatus.INACTIVE && sender.topic.isUniversalPipeline) {`
created_at: 2025-04-07T21:57:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -596,6 +603,22 @@ class ReportFunction(
             submission
         )
 
+        // Only add to queue if the sender/ is enabled
+        if (sender.customerStatus != CustomerStatus.INACTIVE && reportWithBlob != null) {
+            val (report, blobInfo) = reportWithBlob
+            workflowEngine.queue.sendMessage(
+                QueueMessage.elrConvertQueueName,
+                FhirConvertQueueMessage(
+                    report.id,
+                    blobInfo!!.blobUrl,
+                    BlobUtils.digestToString(blobInfo.digest),
+                    sender.fullName,
+                    sender.topic,
+                    sender.schemaName
+                ).serialize()
+            )
+        }
+
         // queue messages here after all task / action records are in
         actionHistory.queueMessages(workflowEngine)
path: prime-router/src/main/kotlin/azure/ReportFunction.kt
user: login: arnejduranovic
body: This line actually sends the queue message but only if `TopicReceiver.validateAndMoveToProcessing` was called. It does not do anything if `UniversalPipelineReceiver.validateAndMoveToProcessing` was called instead. Instead of copy/pasting the special UP queue message code above, have we investigated what it would take to get this line to work for the `UniversalPipelineReceiver.validateAndMoveToProcessing` path?
created_at: 2025-04-07T21:59:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -53,7 +53,7 @@ abstract class Event(val eventAction: EventAction, val at: OffsetDateTime?) {
                 RECEIVER_ENRICHMENT -> TaskAction.receiver_enrichment
                 RECEIVER_FILTER -> TaskAction.receiver_filter
                 RECEIVE -> TaskAction.receive
-                CONVERT -> TaskAction.convert
path: prime-router/src/main/kotlin/azure/Event.kt
user: login: oluwie
body: Instead of updating this event, another option would be to update `toQueueName()` to properly return the correct queue. But made this change to ensure all event names and queue match up even though that tech isn't required
created_at: 2025-04-09T16:00:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -113,6 +114,8 @@ class ActionHistory(
      */
     val messages = mutableListOf<Event>()
 
+    val fhirMessages = mutableListOf<FhirConvertQueueMessage>()
path: prime-router/src/main/kotlin/azure/ActionHistory.kt
user: login: arnejduranovic
body: Can this be the base class instead of being hardcoded to `FhirConvertQueueMessage` specifically? 

nit: Once that is updated, then we could update the variable name to reflect what base type it is supporting. For example, on line 115, perhaps `eventMessages` or something like that would have been a better name.
created_at: 2025-04-10T14:46:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -113,6 +114,8 @@ class ActionHistory(
      */
     val messages = mutableListOf<Event>()
 
+    val fhirMessages = mutableListOf<FhirConvertQueueMessage>()
path: prime-router/src/main/kotlin/azure/ActionHistory.kt
user: login: arnejduranovic
body: Also, make sure to follow the example in this document and add comment blocks to new vars/methods.
created_at: 2025-04-10T14:56:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -76,6 +77,7 @@ abstract class Event(val eventAction: EventAction, val at: OffsetDateTime?) {
                 SEND,
                 WIPE,
                 -> this.toString().lowercase()
+                CONVERT -> QueueMessage.elrConvertQueueName
path: prime-router/src/main/kotlin/azure/Event.kt
user: login: arnejduranovic
body: Good job on simplifying and getting rid of the unneeded changes in this file. However, this change is no longer needed as well, correct?
created_at: 2025-04-10T14:59:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -76,6 +77,7 @@ abstract class Event(val eventAction: EventAction, val at: OffsetDateTime?) {
                 SEND,
                 WIPE,
                 -> this.toString().lowercase()
+                CONVERT -> QueueMessage.elrConvertQueueName
path: prime-router/src/main/kotlin/azure/Event.kt
user: login: oluwie
body: Yes, it's been deleted.
created_at: 2025-04-10T15:22:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -53,7 +53,7 @@ abstract class Event(val eventAction: EventAction, val at: OffsetDateTime?) {
                 RECEIVER_ENRICHMENT -> TaskAction.receiver_enrichment
                 RECEIVER_FILTER -> TaskAction.receiver_filter
                 RECEIVE -> TaskAction.receive
-                CONVERT -> TaskAction.convert
path: prime-router/src/main/kotlin/azure/Event.kt
user: login: oluwie
body: no longer needed
created_at: 2025-04-10T15:24:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -113,6 +114,8 @@ class ActionHistory(
      */
     val messages = mutableListOf<Event>()
 
+    val fhirMessages = mutableListOf<FhirConvertQueueMessage>()
path: prime-router/src/main/kotlin/azure/ActionHistory.kt
user: login: oluwie
body: Updated
created_at: 2025-04-10T15:46:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -770,6 +781,16 @@ class ActionHistory(
         }
     }
 
+    /**
+     *  Sends any FHIR queue messages to the appropriate queue
+     */
+    fun queueFhirMessages(workflowEngine: WorkflowEngine) {
path: prime-router/src/main/kotlin/azure/ActionHistory.kt
user: login: JFisk42
body: Can we add a happy path test for this? Some parts of this file are a beast but I think it should be pretty easy to add in some messages with `trackFhirMessage` and then verify they get processed here. 
created_at: 2025-04-11T00:53:52Z
author_association: COLLABORATOR
diff_hunk: @@ -770,6 +781,16 @@ class ActionHistory(
         }
     }
 
+    /**
+     *  Sends any FHIR queue messages to the appropriate queue
+     */
+    fun queueFhirMessages(workflowEngine: WorkflowEngine) {
path: prime-router/src/main/kotlin/azure/ActionHistory.kt
user: login: oluwie
body: Added
created_at: 2025-04-11T19:40:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -908,4 +914,135 @@ class ActionHistoryTests {
             )
         )
     }
+
+    @Test
+    fun `test queueFhirMessages properly add a fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        assertEquals(1, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly adds multiple fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        actionHistory.trackFhirMessage(
+            FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        actionHistory.trackFhirMessage(
+            FhirReceiverEnrichmentQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        assertEquals(2, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly tracks and queues a fhir messages`() {
path: prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
user: login: arnejduranovic
body: ```suggestion
    fun `test queueFhirMessages properly tracks and queues a fhir message`() {
```
created_at: 2025-04-14T15:39:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -908,4 +914,135 @@ class ActionHistoryTests {
             )
         )
     }
+
+    @Test
+    fun `test queueFhirMessages properly add a fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        assertEquals(1, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly adds multiple fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        actionHistory.trackFhirMessage(
+            FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        actionHistory.trackFhirMessage(
+            FhirReceiverEnrichmentQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        assertEquals(2, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly tracks and queues a fhir messages`() {
path: prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
user: login: arnejduranovic
body: Good job on these tests! `nit:` you could probably remove the two tests that test adding only a single message, I think they are redundant (if you can add two, why wouldn't you be able to add one?)
created_at: 2025-04-14T15:47:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -908,4 +914,135 @@ class ActionHistoryTests {
             )
         )
     }
+
+    @Test
+    fun `test queueFhirMessages properly add a fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        assertEquals(1, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly adds multiple fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        actionHistory.trackFhirMessage(
+            FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        actionHistory.trackFhirMessage(
+            FhirReceiverEnrichmentQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        assertEquals(2, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly tracks and queues a fhir messages`() {
path: prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
user: login: oluwie
body: Updated
created_at: 2025-04-14T16:08:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -908,4 +914,135 @@ class ActionHistoryTests {
             )
         )
     }
+
+    @Test
+    fun `test queueFhirMessages properly add a fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        val queueMessage = FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+
+        actionHistory.trackFhirMessage(queueMessage)
+        assertEquals(1, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly adds multiple fhir messages to be tracked`() {
+        val actionHistory = ActionHistory(TaskAction.receive)
+
+        actionHistory.trackFhirMessage(
+            FhirConvertQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        actionHistory.trackFhirMessage(
+            FhirReceiverEnrichmentQueueMessage(
+            UUID.randomUUID(),
+            "",
+            "",
+            "",
+            Topic.FULL_ELR,
+            ""
+        )
+        )
+
+        assertEquals(2, actionHistory.fhirQueueMessages.size)
+    }
+
+    @Test
+    fun `test queueFhirMessages properly tracks and queues a fhir messages`() {
path: prime-router/src/test/kotlin/azure/ActionHistoryTests.kt
user: login: oluwie
body: Updated
created_at: 2025-04-14T16:08:47Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-04-07T21:05:40Z
body: <h1>Dependency Review</h1>
✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.<h2>OpenSSF Scorecard</h2>
<table><tr><th>Package</th><th>Version</th><th>Score</th><th>Details</th></tr>
</table><h2>Scanned Manifest Files</h2>


<!-- dependency-review-pr-comment-marker -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-07T21:22:22Z
body: ## Test Results
1 306 tests  +2   1 302 :white_check_mark: +2   7m 45s :stopwatch: -9s
  170 suites ±0       4 :zzz: ±0 
  170 files   ±0       0 :x: ±0 

Results for commit a03dcb51. ± Comparison against base commit cd379a90.

<details>
  <summary>This pull request <b>removes</b> 2 and <b>adds</b> 4 tests. <i>Note that renamed tests count towards both.</i></summary>

```
gov.cdc.prime.router.azure.ActionHistoryTests ‑ trackActionParams correctly filters headers and query params()
gov.cdc.prime.router.azure.ActionHistoryTests ‑ trackActionParams the azure client ip()
```

```
gov.cdc.prime.router.azure.ActionHistoryTests ‑ test queueFhirMessages properly add a fhir message to be tracked()
gov.cdc.prime.router.azure.ActionHistoryTests ‑ test queueFhirMessages properly tracks and queues a fhir message()
gov.cdc.prime.router.azure.ActionHistoryTests ‑ test trackActionParams correctly filters headers and query params()
gov.cdc.prime.router.azure.ActionHistoryTests ‑ test trackActionParams the azure client ip()
```
</details>

[test-results]:data:application/gzip;base64,H4sIABCG/WcC/13MTQ6DIBCG4asY1l0AI5D0Mg0OkJCqNPysGu9exFZJl+8zme9NnJ9tIveBKXobSCo+d2lK1NmHtcIoRYV6zO0MVP7ykQriYfyyp3/tXyc47ecK9AQbY4hfiWXtVvf6Gz3o2mzdTbbuFzEsi881iKZgcBKMARdqchonUMAMapAWAJUxmjM+KrJ9ADvB9KANAQAA

:recycle: This comment has been updated with latest results.
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-07T21:22:34Z
body: ## Integration Test Results
 60 files  ±0   60 suites  ±0   42m 29s :stopwatch: -20s
428 tests ±0  418 :white_check_mark: ±0  10 :zzz: ±0  0 :x: ±0 
431 runs  ±0  421 :white_check_mark: ±0  10 :zzz: ±0  0 :x: ±0 

Results for commit a03dcb51. ± Comparison against base commit cd379a90.

[test-results]:data:application/gzip;base64,H4sIABmN/WcC/03MQQ6DIBCF4asY1l0wjErbyzQ4QEKq0iCsTO9eJFZY/t9L3s6sm83Gnt3Ibx3bkou1dAoqOr/mFkP/yJLHeMy9uP/rtSWig6Cht/tkAn6JVW7OUsGE4MMpIa3lFOGM61NUaS4LNI+l20Pyy+JiDqY4apoGABSDnKyiCSWCJoWjQSSptRIgesm+P6UChgkKAQAA

:recycle: This comment has been updated with latest results.
author_association: CONTRIBUTOR
user: login: sonarqubecloud[bot]
created_at: 2025-04-14T22:13:28Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=CDCgov_prime-data-hub&pullRequest=17756) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [4 New issues](https://sonarcloud.io/project/issues?id=CDCgov_prime-data-hub&pullRequest=17756&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=CDCgov_prime-data-hub&pullRequest=17756&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=CDCgov_prime-data-hub&pullRequest=17756&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=CDCgov_prime-data-hub&pullRequest=17756&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=CDCgov_prime-data-hub&pullRequest=17756&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=CDCgov_prime-data-hub&pullRequest=17756)


author_association: NONE