Repository: awell-health/awell-extensions
Pull request #606: Rev-54: Improve Summarize Form AI Action to Maintain Question Order
Author: login: belmai
Labels: name: ai-action
name: Review effort 3/5
Created: 2025-03-11T15:31:56Z | Merged: 2025-03-11T16:05:52Z

[REV-54: Improve Summarize Form AI Action to support order of form questions](https://linear.app/awell/issue/REV-54/improve-summarize-form-ai-action-to-support-order-of-form-questions)


This PR includes the following updates to the Summarize Form AI Action:

- Form responses are now prepared in the correct question order.
- Care flow title and definition ID are added at the beginning (popular request from Wellpath).
- Added an optional additional instructions field for more control over summaries.

⚠️ ⚠️⚠️ **NOTE:** Since this AI Action is used in production, please double-check that new additional instruction field is set up correctly.



Diff:
diff --git a/extensions/shelly/README.md b/extensions/shelly/README.md
index 5110c17b4..81c7eb5d3 100644
--- a/extensions/shelly/README.md
+++ b/extensions/shelly/README.md
@@ -5,13 +5,103 @@ description: Library of AI-powered actions
 
 # Shelly
 
-Shelly is a library of AI-powered actions for enhancing workflow automation and data processing. It includes the following actions:
-
-1. **summarizeForm**: Summarizes form data (last form in a step)
-2. **summarizeFormsInStep**: Summarizes all forms within a step.
-3. **summarizeCareFlow**: Provides a summary of a care flow.
-4. **categorizeMessage**: Categorizes messages or content.
-5. **generateMessage**: Generates personalized messages.
-6. **medicationFromImage**: Extracts medication information from images.
-7. **reviewMedicationExtraction**: Reviews and validates extracted medication information.
-8. **summarizeTrackOutcome**: Provides an AI-generated summary of track activities and overall track outcomes.
\ No newline at end of file
+Shelly is a library of AI-powered actions for enhancing workflow automation and data processing. It leverages advanced language models and specialized APIs to provide intelligent automation capabilities.
+
+## Actions
+
+### Categorize Message
+Uses AI to classify messages into predefined categories.
+
+- Analyzes message content using LLM
+- Matches against provided categories
+- Provides explanation for the categorization
+
+Features:
+- Supports custom category lists
+- Handles multiple languages
+- Returns "None" if no category matches
+- Includes detailed explanation of the classification
+
+### Generate Message
+Creates personalized messages using AI.
+
+- Takes communication objective and context
+- Generates appropriate message content
+- Formats according to stakeholder needs
+
+Features:
+- Customizable tone and style
+- Multi-language support
+- Personalization based on recipient data
+- Context-aware message generation
+
+### Medication From Image
+Extracts structured medication information from images.
+
+- Processes uploaded medication images
+- Uses specialized API for text extraction
+- Returns structured medication data
+
+Features:
+- Extracts medication names, dosages, and instructions
+- Supports various image formats
+- Returns both structured data and text summary
+- Includes error handling for unclear images
+
+### Review Medication Extraction
+Allows stakeholders to review and validate extracted medication information.
+
+- Displays extracted medication data
+- Provides interface for validation
+- Captures reviewer feedback
+
+Features:
+- Side-by-side image and data comparison
+- Validation workflow
+- Support for corrections and annotations
+
+### Summarize Care Flow
+Provides an AI-generated summary of the entire care flow progress.
+
+- Collects all relevant care flow data
+- Analyzes activities and outcomes
+- Generates a comprehensive summary
+
+Features:
+- Customizable for different stakeholders (e.g., clinicians, patients)
+- Optional focus areas through additional instructions
+- Includes relevant context and disclaimers
+
+### Summarize Form
+Uses AI to generate concise summaries of form responses.
+
+- Retrieves the latest form data from the current step
+- Processes form responses with an LLM
+- Generates a structured summary in the specified format
+
+Features:
+- Supports bullet-point or paragraph format
+- Customizable language selection
+- Optional additional instructions for focused summaries
+- Includes context-aware disclaimers
+
+### Summarize Forms in Step
+Similar to summarizeForm but processes all forms within a step.
+
+Features:
+- Combines data from multiple forms
+- Maintains context between form responses
+- Same formatting options as single form summaries
+
+### Summarize Track Outcome
+Generates comprehensive summaries of track activities and outcomes.
+
+- Analyzes track data and activities
+- Identifies key outcomes and decisions
+- Creates structured summary
+
+Features:
+- Customizable focus areas
+- Support for specific outcome analysis
+- Flexible formatting options
+- Context-aware summaries
\ No newline at end of file
diff --git a/extensions/shelly/actions/summarizeForm/config/fields.ts b/extensions/shelly/actions/summarizeForm/config/fields.ts
index a76faff14..7bd520c5b 100644
--- a/extensions/shelly/actions/summarizeForm/config/fields.ts
+++ b/extensions/shelly/actions/summarizeForm/config/fields.ts
@@ -1,13 +1,11 @@
 import { type Field, FieldType } from '@awell-health/extensions-core'
 import z, { type ZodTypeAny } from 'zod'
 
-// Step 1: Define the enum
 export enum SummaryFormatEnum {
   BULLET_POINTS = 'Bullet-points',
   TEXT_PARAGRAPH = 'Text paragraph',
 }
 
-// Step 2: Define the fields object
 export const fields = {
   summaryFormat: {
     id: 'summaryFormat',
@@ -31,6 +29,14 @@ export const fields = {
     type: FieldType.STRING,
     required: false,
   },
+  additionalInstructions: {
+    id: 'additionalInstructions',
+    label: 'Additional Instructions',
+    description:
+      '[Optional] Specify additional instructions for the AI to generate the form summary. You can include details about: specific questions to focus on, desired level of detail, information to emphasize or exclude, etc.',
+    type: FieldType.TEXT,
+    required: false,
+  },
 } satisfies Record<string, Field>
 
 // Step 3: Define the validation schema using zod
@@ -45,4 +51,5 @@ export const FieldsValidationSchema = z.object({
         : SummaryFormatEnum.BULLET_POINTS
     ),
   language: z.string().optional().default('Default'),
+  additionalInstructions: z.string().optional().default('No additional instructions'),
 } satisfies Record<keyof typeof fields, ZodTypeAny>)
diff --git a/extensions/shelly/actions/summarizeForm/summarizeForm.test.ts b/extensions/shelly/actions/summarizeForm/summarizeForm.test.ts
index 275ef5829..b025ed12a 100644
--- a/extensions/shelly/actions/summarizeForm/summarizeForm.test.ts
+++ b/extensions/shelly/actions/summarizeForm/summarizeForm.test.ts
@@ -1,12 +1,33 @@
 /* eslint-disable @typescript-eslint/no-var-requires */
 
 import { TestHelpers } from '@awell-health/extensions-core'
-import { generateTestPayload } from '@/tests'
 import { summarizeForm } from '.'
 import { mockFormDefinitionResponse } from './__mocks__/formDefinitionResponse'
 import { mockFormResponseResponse } from './__mocks__/formResponseResponse'
 import { markdownToHtml } from '../../../../src/utils'
 
+// Simple payload generator function to replace the external import
+const generateTestPayload = (overrides = {}) => {
+  return {
+    pathway: {
+      id: 'test-pathway-id',
+      definition_id: 'test-definition-id',
+      tenant_id: 'test-tenant-id',
+      org_slug: 'test-org-slug',
+      org_id: 'test-org-id'
+    },
+    activity: {
+      id: 'test-activity-id'
+    },
+    fields: {},
+    settings: {},
+    patient: {
+      id: 'test-patient-id'
+    },
+    ...overrides
+  }
+}
+
 // Mock the OpenAI modules
 jest.mock('../../../../src/lib/llm/openai/createOpenAIModel', () => ({
   createOpenAIModel: jest.fn().mockResolvedValue({
@@ -23,14 +44,36 @@ jest.mock('../../../../src/lib/llm/openai/createOpenAIModel', () => ({
   }),
 }))
 
+jest.mock('../../lib/summarizeFormWithLLM', () => ({
+  summarizeFormWithLLM: jest.fn().mockImplementation(({ disclaimerMessage, additionalInstructions }) => {
+    return Promise.resolve('The patient reported good overall health. They experienced fatigue and headache in the last 7 days. Additionally, they mentioned occasional dizziness when standing up too quickly.')
+  }),
+}))
+
 describe('summarizeForm - Mocked LLM calls', () => {
   const { onComplete, onError, helpers, extensionAction, clearMocks } =
     TestHelpers.fromAction(summarizeForm)
 
+  // Define mock pathway details for the disclaimer
+  const mockPathwayDetails = {
+    pathway: {
+      success: true,
+      code: 200,
+      pathway: {
+        id: 'ai4rZaYEocjB',
+        title: 'Test Care Flow',
+        pathway_definition_id: 'whatever',
+      }
+    }
+  }
+
   beforeEach(() => {
     clearMocks()
     jest.clearAllMocks()
     const mockQuery = jest.fn()
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
       .mockResolvedValueOnce({
         activity: {
           success: true,
@@ -43,6 +86,7 @@ describe('summarizeForm - Mocked LLM calls', () => {
           }
         }
       })
+      // Third query: get activities in current step
       .mockResolvedValueOnce({
         pathwayStepActivities: {
           success: true,
@@ -61,9 +105,11 @@ describe('summarizeForm - Mocked LLM calls', () => {
           }]
         }
       })
+      // Fourth query: get form definition
       .mockResolvedValueOnce({
         form: mockFormDefinitionResponse,
       })
+      // Fifth query: get form response
       .mockResolvedValueOnce({
         formResponse: mockFormResponseResponse,
       })
@@ -105,4 +151,36 @@ describe('summarizeForm - Mocked LLM calls', () => {
 
     expect(onError).not.toHaveBeenCalled()
   })
+
+  it('Should summarize form with LLM and additional instructions', async () => {
+    const payload = generateTestPayload({
+      pathway: {
+        id: 'ai4rZaYEocjB',
+        definition_id: 'whatever',
+      },
+      activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+      fields: {
+        summaryFormat: 'Bullet-points',
+        language: 'Default',
+        additionalInstructions: 'Focus on medication details and side effects.',
+      },
+      settings: {},
+    })
+
+    await extensionAction.onEvent({
+      payload,
+      onComplete,
+      onError,
+      helpers,
+    })
+
+    const expected = await markdownToHtml('The patient reported good overall health. They experienced fatigue and headache in the last 7 days. Additionally, they mentioned occasional dizziness when standing up too quickly.')
+    expect(onComplete).toHaveBeenCalledWith({
+      data_points: {
+        summary: expected,
+      },
+    })
+
+    expect(onError).not.toHaveBeenCalled()
+  })
 })
diff --git a/extensions/shelly/actions/summarizeForm/summarizeForm.ts b/extensions/shelly/actions/summarizeForm/summarizeForm.ts
index fab17eed1..73d3284f1 100644
--- a/extensions/shelly/actions/summarizeForm/summarizeForm.ts
+++ b/extensions/shelly/actions/summarizeForm/summarizeForm.ts
@@ -6,7 +6,6 @@ import { fields, dataPoints, FieldsValidationSchema } from './config'
 import { getFormResponseText } from '../../lib/getFormResponseText'
 import { getLatestFormInCurrentStep } from '../../../../src/lib/awell'
 import { markdownToHtml } from '../../../../src/utils'
-import { DISCLAIMER_MSG_FORM } from '../../lib/constants'
 
 /**
  * Awell Action: Form Summarization
@@ -32,7 +31,7 @@ export const summarizeForm: Action<
 
   onEvent: async ({ payload, onComplete, onError, helpers }): Promise<void> => {
     // 1. Validate input fields
-    const { summaryFormat, language } = FieldsValidationSchema.parse(
+    const { summaryFormat, language, additionalInstructions } = FieldsValidationSchema.parse(
       payload.fields,
     )
 
@@ -45,9 +44,27 @@ export const summarizeForm: Action<
       hideDataForTracing: true, // Hide input and output data when tracing
     })
 
+    const awellSdk = await helpers.awellSdk()
+
+    // Get pathway details for the disclaimer
+    const pathwayDetails = await awellSdk.orchestration.query({
+      pathway: {
+        __args: {
+          id: payload.pathway.id,
+        },
+        code: true,
+        success: true,
+        pathway: {
+          id: true,
+          title: true,
+          pathway_definition_id: true,
+        },
+      },
+    })
+
     // 3. Get form data
     const { formDefinition, formResponse } = await getLatestFormInCurrentStep({
-      awellSdk: await helpers.awellSdk(),
+      awellSdk,
       pathwayId: payload.pathway.id,
       activityId: payload.activity.id,
     })
@@ -63,7 +80,8 @@ export const summarizeForm: Action<
       formData,
       summaryFormat,
       language,
-      disclaimerMessage: DISCLAIMER_MSG_FORM,
+      disclaimerMessage: `Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "${pathwayDetails.pathway?.pathway?.title ?? 'Unknown'}" (ID: ${pathwayDetails.pathway?.pathway?.pathway_definition_id ?? 'Unknown'}).`,
+      additionalInstructions,
       metadata,
       callbacks,
     })
diff --git a/extensions/shelly/actions/summarizeForm/summarizeFormRealOpenAI.test.ts b/extensions/shelly/actions/summarizeForm/summarizeFormRealOpenAI.test.ts
index 9ce2dfc2e..0faa4f6f9 100644
--- a/extensions/shelly/actions/summarizeForm/summarizeFormRealOpenAI.test.ts
+++ b/extensions/shelly/actions/summarizeForm/summarizeFormRealOpenAI.test.ts
@@ -1,18 +1,60 @@
 import 'dotenv/config'
 import { TestHelpers } from '@awell-health/extensions-core'
-import { generateTestPayload } from '@/tests'
 import { summarizeForm } from '.'
 import { mockFormDefinitionResponse } from './__mocks__/formDefinitionResponse'
 import { mockFormResponseResponse } from './__mocks__/formResponseResponse'
 import { mockPathwayActivitiesResponse } from './__mocks__/pathwayActivitiesResponse'
-import { DISCLAIMER_MSG_FORM } from '../../lib/constants'
 
 jest.setTimeout(30000) // Increase timeout if needed for real LLM calls
 
+const generateTestPayload = (overrides = {}) => {
+  return {
+    pathway: {
+      id: 'test-pathway-id',
+      definition_id: 'test-definition-id',
+      tenant_id: 'test-tenant-id',
+      org_slug: 'test-org-slug',
+      org_id: 'test-org-id'
+    },
+    activity: {
+      id: 'test-activity-id'
+    },
+    fields: {},
+    settings: {},
+    patient: {
+      id: 'test-patient-id'
+    },
+    ...overrides
+  }
+}
+
 describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
   const { onComplete, onError, helpers, extensionAction, clearMocks } =
     TestHelpers.fromAction(summarizeForm)
 
+  // Define base payload to be reused across tests
+  const basePayload = {
+    pathway: {
+      id: 'ai4rZaYEocjB',
+      definition_id: 'whatever',
+    },
+    activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+    settings: {}
+  }
+
+  // Define mock pathway details for the disclaimer
+  const mockPathwayDetails = {
+    pathway: {
+      success: true,
+      code: 200,
+      pathway: {
+        id: 'ai4rZaYEocjB',
+        title: 'Test Care Flow',
+        pathway_definition_id: 'whatever',
+      }
+    }
+  }
+
   beforeEach(() => {
     clearMocks()
     jest.clearAllMocks()
@@ -26,31 +68,109 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
 
   it('Should call the real model with Bullet-points format', async () => {
     const payload = generateTestPayload({
-      pathway: {
-        id: 'ai4rZaYEocjB',
-        definition_id: 'whatever',
+      ...basePayload,
+      fields: {
+        summaryFormat: 'Bullet-points',
+        language: 'Default',
       },
-      activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+    })
+
+    const mockQuery = jest.fn()
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
+      .mockResolvedValueOnce({
+        activity: {
+          success: true,
+          activity: mockPathwayActivitiesResponse.activities[0]
+        }
+      })
+      // Third query: get activities in current step
+      .mockResolvedValueOnce({
+        pathwayStepActivities: {
+          success: true,
+          activities: mockPathwayActivitiesResponse.activities
+        }
+      })
+      // Fourth query: get form definition
+      .mockResolvedValueOnce({
+        form: {
+          form: {
+            id: 'OGhjJKF5LRmo',
+            questions: [{
+              id: 'q1',
+              title: 'Test Question',
+              type: 'TEXT',
+              options: []
+            }]
+          }
+        }
+      })
+      // Fifth query: get form response
+      .mockResolvedValueOnce({
+        formResponse: {
+          response: {
+            answers: [{
+              question_id: 'q1',
+              value: 'Test Answer'
+            }]
+          }
+        }
+      })
+
+    helpers.awellSdk = jest.fn().mockReturnValue({
+      orchestration: {
+        query: mockQuery
+      }
+    })
+
+    await extensionAction.onEvent({
+      payload,
+      onComplete,
+      onError,
+      helpers,
+    })
+
+    expect(helpers.awellSdk).toHaveBeenCalled()
+    expect(mockQuery).toHaveBeenCalledTimes(5)
+    expect(onComplete).toHaveBeenCalledWith({
+      data_points: {
+        summary: expect.stringMatching(
+          new RegExp(`Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "Test Care Flow".*Test Question.*Test Answer.*`, 's')
+        ),
+      },
+    })
+    expect(onError).not.toHaveBeenCalled()
+  }, 30000)
+
+  it('Should call the real model with additional instructions', async () => {
+    const payload = generateTestPayload({
+      ...basePayload,
       fields: {
         summaryFormat: 'Bullet-points',
         language: 'Default',
+        additionalInstructions: 'Focus on highlighting any abnormal values or concerning responses.',
       },
-      settings: {}
     })
 
     const mockQuery = jest.fn()
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
       .mockResolvedValueOnce({
         activity: {
           success: true,
           activity: mockPathwayActivitiesResponse.activities[0]
         }
       })
+      // Third query: get activities in current step
       .mockResolvedValueOnce({
         pathwayStepActivities: {
           success: true,
           activities: mockPathwayActivitiesResponse.activities
         }
       })
+      // Fourth query: get form definition
       .mockResolvedValueOnce({
         form: {
           form: {
@@ -64,6 +184,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
+      // Fifth query: get form response
       .mockResolvedValueOnce({
         formResponse: {
           response: {
@@ -89,11 +210,11 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
     })
 
     expect(helpers.awellSdk).toHaveBeenCalled()
-    expect(mockQuery).toHaveBeenCalledTimes(4)
+    expect(mockQuery).toHaveBeenCalledTimes(5)
     expect(onComplete).toHaveBeenCalledWith({
       data_points: {
         summary: expect.stringMatching(
-          new RegExp(`${DISCLAIMER_MSG_FORM}.*Test Question.*Test Answer.*`, 's')
+          new RegExp(`Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "Test Care Flow".*Test Question.*Test Answer.*`, 's')
         ),
       },
     })
@@ -102,20 +223,17 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
 
   it('Should call the real model with Text paragraph format', async () => {
     const payload = generateTestPayload({
-      pathway: {
-        id: 'ai4rZaYEocjB',
-        definition_id: 'whatever',
-      },
-      activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+      ...basePayload,
       fields: {
         summaryFormat: 'Text paragraph',
         language: 'Default',
       },
-      settings: {}
     })
 
     const mockQuery = jest.fn()
-      // First query: get current activity
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
       .mockResolvedValueOnce({
         activity: {
           success: true,
@@ -127,7 +245,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
-      // Second query: get activities in current step
+      // Third query: get activities in current step
       .mockResolvedValueOnce({
         pathwayStepActivities: {
           success: true,
@@ -146,7 +264,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }]
         }
       })
-      // Third query: get form definition
+      // Fourth query: get form definition
       .mockResolvedValueOnce({
         form: {
           form: {
@@ -160,7 +278,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
-      // Fourth query: get form response
+      // Fifth query: get form response
       .mockResolvedValueOnce({
         formResponse: {
           response: {
@@ -186,11 +304,11 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
     })
 
     expect(helpers.awellSdk).toHaveBeenCalled()
-    expect(mockQuery).toHaveBeenCalledTimes(4)
+    expect(mockQuery).toHaveBeenCalledTimes(5)
     expect(onComplete).toHaveBeenCalledWith({
       data_points: {
         summary: expect.stringMatching(
-          new RegExp(`${DISCLAIMER_MSG_FORM}.*Test Question.*Test Answer.*`, 's')
+          new RegExp(`Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "Test Care Flow".*Test Question.*Test Answer.*`, 's')
         ),
       },
     })
@@ -199,16 +317,17 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
 
   it('Should call the real model and use mocked form data with Text paragraph format', async () => {
     const payload = generateTestPayload({
-      pathway: { id: 'ai4rZaYEocjB', definition_id: 'whatever' },
-      activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+      ...basePayload,
       fields: {
         summaryFormat: 'Text paragraph',
         language: 'Default',
       },
-      settings: {}
     })
 
     const mockQuery = jest.fn()
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
       .mockResolvedValueOnce({
         activity: {
           success: true,
@@ -220,6 +339,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
+      // Third query: get activities in current step
       .mockResolvedValueOnce({
         pathwayStepActivities: {
           success: true,
@@ -238,6 +358,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }]
         }
       })
+      // Fourth query: get form definition
       .mockResolvedValueOnce({
         form: {
           form: {
@@ -251,6 +372,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
+      // Fifth query: get form response
       .mockResolvedValueOnce({
         formResponse: {
           response: {
@@ -276,11 +398,11 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
     })
 
     expect(helpers.awellSdk).toHaveBeenCalled()
-    expect(mockQuery).toHaveBeenCalledTimes(4)
+    expect(mockQuery).toHaveBeenCalledTimes(5)
     expect(onComplete).toHaveBeenCalledWith({
       data_points: {
         summary: expect.stringMatching(
-          new RegExp(`${DISCLAIMER_MSG_FORM}.*Test Question.*Test Answer.*`, 's')
+          new RegExp(`Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "Test Care Flow".*Test Question.*Test Answer.*`, 's')
         ),
       },
     })
@@ -289,8 +411,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
 
   it('Should call the real model and use mocked form data', async () => {
     const payload = generateTestPayload({
-      pathway: { id: 'ai4rZaYEocjB', definition_id: 'whatever' },
-      activity: { id: 'X74HeDQ4N0gtdaSEuzF8s' },
+      ...basePayload,
       fields: {
         summaryFormat: 'Bullet-points',
         language: 'Default',
@@ -301,18 +422,23 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
     })
 
     const mockQuery = jest.fn()
+      // First query: get pathway details for disclaimer
+      .mockResolvedValueOnce(mockPathwayDetails)
+      // Second query: get current activity
       .mockResolvedValueOnce({
         activity: {
           success: true,
           activity: mockPathwayActivitiesResponse.activities[0]
         }
       })
+      // Third query: get activities in current step
       .mockResolvedValueOnce({
         pathwayStepActivities: {
           success: true,
           activities: mockPathwayActivitiesResponse.activities
         }
       })
+      // Fourth query: get form definition
       .mockResolvedValueOnce({
         form: {
           form: {
@@ -352,6 +478,7 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
           }
         }
       })
+      // Fifth query: get form response
       .mockResolvedValueOnce({
         formResponse: {
           response: {
@@ -387,11 +514,11 @@ describe.skip('summarizeForm - Real LLM calls with mocked Awell SDK', () => {
     })
 
     expect(helpers.awellSdk).toHaveBeenCalled()
-    expect(mockQuery).toHaveBeenCalledTimes(4)
+    expect(mockQuery).toHaveBeenCalledTimes(5)
     expect(onComplete).toHaveBeenCalledWith({
       data_points: {
         summary: expect.stringMatching(
-          new RegExp(`${DISCLAIMER_MSG_FORM}.*Patient Health Questionnaire.*Overall health.*Symptoms.*health concerns.*`, 's')
+          new RegExp(`Important Notice: The content provided is an AI-generated summary of form responses in Care Flow "Test Care Flow".*Patient Health Questionnaire.*Overall health.*Symptoms.*health concerns.*`, 's')
         ),
       },
     })
diff --git a/extensions/shelly/lib/getFormResponseText/getFormResponseText.ts b/extensions/shelly/lib/getFormResponseText/getFormResponseText.ts
index a3dd0e91c..e01540522 100644
--- a/extensions/shelly/lib/getFormResponseText/getFormResponseText.ts
+++ b/extensions/shelly/lib/getFormResponseText/getFormResponseText.ts
@@ -124,30 +124,28 @@ export const getFormResponseText = (opts: {
   const formAnswers: string[] = []
   const omittedFormAnswers: OmittedFormAnswer[] = []
  
-  opts.formResponse.answers.forEach((questionResponse) => {
-    const questionDefinition = opts.formDefinition.questions.find(
-      (q) => q.id === questionResponse.question_id
+  opts.formDefinition.questions.forEach((questionDefinition) => {
+    const questionResponse = opts.formResponse.answers.find(
+      (a) => a.question_id === questionDefinition.id
     )
 
-    if (questionDefinition === undefined) {
+    if (questionResponse === undefined) {
       omittedFormAnswers.push({
-        questionId: questionResponse.question_id,
-        reason:
-          'No corresponding question definition found in the form definition',
+        questionId: questionDefinition.id,
+        reason: 'No corresponding answer found in the form response',
       })
       return
     }
 
     try {
       const answer = getQuestionAndAnswer(questionDefinition, questionResponse)
-
       formAnswers.push(answer)
     } catch (error) {
       const errorMessage =
         error instanceof Error ? error.message : 'Unknown error'
 
       omittedFormAnswers.push({
-        questionId: questionResponse.question_id,
+        questionId: questionDefinition.id,
         reason: `Error processing answer: ${errorMessage}`,
       })
     }
diff --git a/extensions/shelly/lib/summarizeFormWithLLM/prompt.ts b/extensions/shelly/lib/summarizeFormWithLLM/prompt.ts
index 8ea4928b1..a66ea464c 100644
--- a/extensions/shelly/lib/summarizeFormWithLLM/prompt.ts
+++ b/extensions/shelly/lib/summarizeFormWithLLM/prompt.ts
@@ -59,6 +59,9 @@ export const systemPromptBulletPoints = ChatPromptTemplate.fromTemplate(`
 
   {language}
 
+  **Additional Instructions:**
+  {additionalInstructions}
+
   ---
 
   **Content to Summarize:**
@@ -113,6 +116,9 @@ export const systemPromptTextParagraph = ChatPromptTemplate.fromTemplate(`
 
   {language}
 
+  **Additional Instructions:**
+  {additionalInstructions}
+
   ---
 
   **Content to Summarize:**
diff --git a/extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts b/extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts
index 7005e94a5..94914bbf9 100644
--- a/extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts
+++ b/extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts
@@ -17,6 +17,7 @@ import type { BaseCallbackHandler } from "@langchain/core/callbacks/base"
  *   summaryFormat: "Bullet-points",
  *   language: "English",
  *   disclaimerMessage: "AI generated...",
+ *   additionalInstructions: "Focus on medication details",
  *   metadata: { ... }
  * })
  */
@@ -26,6 +27,7 @@ export const summarizeFormWithLLM = async ({
   summaryFormat,
   language,
   disclaimerMessage,
+  additionalInstructions,
   metadata,
   callbacks,
 }: {
@@ -34,6 +36,7 @@ export const summarizeFormWithLLM = async ({
   summaryFormat: string
   language: string
   disclaimerMessage: string
+  additionalInstructions?: string
   metadata: AIActionMetadata
   callbacks?: BaseCallbackHandler[]
 }): Promise<string> => {
@@ -45,6 +48,7 @@ export const summarizeFormWithLLM = async ({
     language,
     input: formData,
     disclaimerMessage,
+    additionalInstructions: additionalInstructions ?? 'No additional instructions',
   })
 
   try {


Reviews:
user: login: sharlotta93
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T16:00:58Z
user: login: sharlotta93
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T16:02:41Z
user: login: belmai
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T16:05:36Z

Review comments:
diff_hunk: @@ -31,6 +29,14 @@ export const fields = {
     type: FieldType.STRING,
     required: false,
   },
+  additionalInstructions: {
+    id: 'additionalInstructions',
+    label: 'Additional Instructions',
+    description:
+      '[Optional] Specify additional instructions for the AI to generate the form summary. You can include details about: specific questions to focus on, desired level of detail, information to emphasize or exclude, etc.',
path: extensions/shelly/actions/summarizeForm/config/fields.ts
user: login: sharlotta93
body: I don't think we need the [optional] here - when the field is not required it's optional by default
created_at: 2025-03-11T16:00:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -31,6 +29,14 @@ export const fields = {
     type: FieldType.STRING,
     required: false,
   },
+  additionalInstructions: {
+    id: 'additionalInstructions',
+    label: 'Additional Instructions',
+    description:
+      '[Optional] Specify additional instructions for the AI to generate the form summary. You can include details about: specific questions to focus on, desired level of detail, information to emphasize or exclude, etc.',
path: extensions/shelly/actions/summarizeForm/config/fields.ts
user: login: belmai
body: Thanks, yes but I will keep it. 
We introduce it to make it obvious for designers it is optional and they do not need to input anything if they do not want (fields for AI Action tend to be scary for some reason :D)

We have it for language as well:
<img width="345" alt="image" src="https://github.com/user-attachments/assets/f59b64eb-c16c-4774-9326-a2f966120aaa" />

created_at: 2025-03-11T16:05:36Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-11T15:33:03Z
body: ## PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

<table>
<tr><td>⏱️&nbsp;<strong>Estimated effort to review</strong>: 3 🔵🔵🔵⚪⚪</td></tr>
<tr><td>🧪&nbsp;<strong>PR contains tests</strong></td></tr>
<tr><td>🔒&nbsp;<strong>No security concerns identified</strong></td></tr>
<tr><td>⚡&nbsp;<strong>Recommended focus areas for review</strong><br><br>

<details><summary><a href='https://github.com/awell-health/awell-extensions/pull/606/files#diff-7beb14de1e304f38e082414956acf681dded2ff2601b540cbadabd22ee5661aaR9-R29'><strong>Test Duplication</strong></a>

The generateTestPayload helper function is defined in multiple test files. Consider centralizing this helper to reduce duplicate code and simplify future maintenance.
</summary>

```typescript
// Simple payload generator function to replace the external import
const generateTestPayload = (overrides = {}) => {
  return {
    pathway: {
      id: 'test-pathway-id',
      definition_id: 'test-definition-id',
      tenant_id: 'test-tenant-id',
      org_slug: 'test-org-slug',
      org_id: 'test-org-id'
    },
    activity: {
      id: 'test-activity-id'
    },
    fields: {},
    settings: {},
    patient: {
      id: 'test-patient-id'
    },
    ...overrides
  }
}
```

</details>

<details><summary><a href='https://github.com/awell-health/awell-extensions/pull/606/files#diff-3b0137c4fdb9c0eacdc807d16a3418a9c54588768d2187077b9c7253d1888e2eR32-R39'><strong>Default Value</strong></a>

The additionalInstructions field is assigned a default string "No additional instructions". Please verify that this default value correctly represents the absence of user input and aligns with the intended summarization behavior.
</summary>

```typescript
additionalInstructions: {
  id: 'additionalInstructions',
  label: 'Additional Instructions',
  description:
    '[Optional] Specify additional instructions for the AI to generate the form summary. You can include details about: specific questions to focus on, desired level of detail, information to emphasize or exclude, etc.',
  type: FieldType.TEXT,
  required: false,
},
```

</details>

</td></tr>
</table>

author_association: NONE
user: login: github-actions[bot]
created_at: 2025-03-11T15:33:47Z
body: ## PR Code Suggestions ✨

No code suggestions found for the PR.
author_association: NONE