Research task (Technical Documentation):
create a simple agentic ai for personal mental-wellness support. id like to have
- two agents: one would be a 'Mood Agent' that reads your check-ins (text or voice) to see what your emotions are. then theres an 'Intervention Agent' that suggests quick exercises, like breathing or journaling prompts
- the workflow should be: the mood bot flags a really low or high mood, and then pings the intervention bot to help out
- maybe I can use an NLU tool like dialogflow? and we MUST encrypt all the data and let people delete their history
- if it gets confused it should ask follow up questions and if it sees someone is repeatedly low (eg 3 times in a row) it needs to escalate to a human counselor

how do I test it? we could measure engagement and self-reported mood improvement. maybe just pilot it with 20 users for 2 weeks?

Evaluation rubric:
criterion: The response contains a description of the Mood Agent and its functionalities, including analyzing user text and/or voice check-ins for emotional cues.
weight: 4
axis: Explicit Criteria

criterion: The response contains a description of the Intervention Agent and its functionalities, including offering quick wellness exercises (e.g., breathing, journaling, short walks, guided meditation).
weight: 4
axis: Explicit Criteria

criterion: The response contains a description of the workflow where the Mood Agent flags low or high moods using defined indicators or metrics (e.g., sentiment scores, keyword detection, tone analysis, frequency of negative expressions) and subsequently pings the Intervention Agent.
weight: 4
axis: Explicit Criteria

criterion: The response specifies at least one concrete Natural Language Understanding (NLU) tool or platform for intent and slot extraction and describes how the tool can be integrated into the agentic system (e.g., Dialogflow for detecting mood intents, Rasa for routing between agents, Wit.ai for extracting emotional entities, IBM Watson Assistant for dialogue management).
weight: 5
axis: Explicit Criteria

criterion: The response explicitly describes how all stored and transmitted user data will be encrypted and explains how the technique upholds recognized privacy standards (e.g., AES-256 encryption for storage meeting HIPAA compliance, TLS 1.3 for transmission adhering to GDPR guidelines, key rotation policies aligned with NIST standards, secure credential management following ISO/IEC 27001).
weight: 4
axis: Explicit Criteria

criterion: The response describes how users are given the ability to delete their interaction history on demand and specifies the mechanism by which this is enabled (e.g., an in-app “Delete History” button, time-based auto-deletion settings, a user-initiated API call, secure erasure confirmation).
weight: 5
axis: Explicit Criteria

criterion: The response describes a clarification fallback where the system asks follow-up questions when uncertain about user input and specifies how this mechanism is implemented (e.g., prompting the user to rephrase unclear text, requesting confirmation of detected mood, offering multiple-choice options for intent clarification, escalating to a simplified dialogue mode).
weight: 5
axis: Explicit Criteria

criterion: The response describes a fallback mechanism where repeated low-mood detections (e.g., three consecutive low-mood check-ins) by the Mood Agent lead to an escalation to a human counselor and specifies how this is carried out (e.g., direct connection to a crisis hotline, scheduling a live counselor session, sending a secure alert to a designated mental-health professional).
weight: 5
axis: Explicit Criteria

criterion: The response lists “engagement time” or similar engagement metrics and explains how they are collected and by which agent in the system (e.g., Mood Agent tracking duration of check-ins, Intervention Agent logging time spent on exercises, system-level analytics recording overall session length, cross-agent aggregation of user activity logs).
weight: 5
axis: Explicit Criteria

criterion: The response lists “mood improvement” as an evaluation metric and explains how it is measured (e.g., Mood Agent comparing sentiment scores across sessions, user self-reports on mood scales, aggregated pre/post-pilot survey results).
weight: 5
axis: Explicit Criteria

criterion: The response specifies a pilot plan involving approximately 20 users for testing and describes how the pilot will be conducted in two weeks (e.g., daily mood check-ins, scheduled intervention exercises, pre- and post-pilot evaluation surveys, aggregated engagement/mood improvement reports).
weight: 4
axis: Explicit Criteria

criterion: The response is organized into clearly labelled sections and discusses factors relevant to the labels (i.e., Agents and Roles, Workflow, Tech and Privacy, Fallbacks, Metrics and Pilot).
weight: 5
axis: Instruction Following

criterion: The response specifies a pilot plan with a cohort size different from 20 users.
weight: -4
axis: Instruction Following

criterion: The response specifies a duration different from two weeks for the pilot plan.
weight: -3
axis: Instruction Following

criterion: The response explains the multi-modal fusion of text and audio emotion cues and describes the processes used to combine them (e.g., extracting sentiment features from text and pitch/tempo features from audio, aligning modalities through time-stamped embeddings, applying feature-level fusion in a neural network, using decision-level fusion with weighted voting).
weight: 3
axis: Implicit Criteria

criterion: The response mentions a model that can assist the agents in detecting and classifying emotions and specifies how it is applied within the system (e.g., BERT for sentiment classification of text check-ins, RoBERTa for nuanced emotion detection, Wav2Vec for audio-based emotion recognition, multimodal transformers for joint analysis of text and speech).
weight: 3
axis: Implicit Criteria

criterion: The response outlines a dialogue-management component that maintains context across agent interactions and explains how it functions within the system (e.g., tracking conversation state across Mood and Intervention Agents, storing dialogue history in a context manager, applying slot-filling techniques, using reinforcement learning to optimize turn-taking).
weight: 4
axis: Implicit Criteria

criterion: The response describes ongoing ethical oversight by mental-health professionals beyond basic escalation and specifies how this oversight is incorporated into the system (e.g., regular audits of agent recommendations by licensed counselors, advisory board reviews of intervention strategies, continuous feedback loops with clinical psychologists, ethical risk assessments conducted before pilot expansions).
weight: 4
axis: Implicit Criteria

criterion: The response specifies how the deletion process happens, describes the technical procedure that ensures data is permanently removed, and explains how the method aligns with recognized privacy standards (e.g., user-initiated deletion request triggering cryptographic erasure in compliance with GDPR’s right to be forgotten, multi-pass overwrite of database entries following NIST guidelines, secure erase commands for SSD storage meeting HIPAA requirements, cloud provider API calls with verification logs aligned to ISO/IEC 27001).
weight: 4
axis: Explicit Criteria

criterion: The response states the system’s scope and limitations for mental-wellness support and provides a non-diagnostic disclaimer (e.g., clarifies it is not a substitute for therapy, advises contacting emergency services in crisis, provides crisis resources, distinguishes wellness tips from clinical treatment).
weight: 4
axis: Implicit Criteria

criterion: The response explains threshold calibration and validation of Mood Agent signals prior to deployment (e.g., hold-out validation set, inter-rater reliability with human annotators, ROC/PR analysis, periodic recalibration schedule).
weight: 3
axis: Implicit Criteria

criterion: The response explains the format or protocol used for data exchange between agents (e.g., JSON messages, gRPC calls, structured events with schemas, shared database entries).
weight: 3
axis: Implicit Criteria

criterion: The response specifies how agents exchange information and describes the communication mechanism (e.g., direct API calls between agents, message queues, shared state storage, event-driven notifications).
weight: 3
axis: Implicit Criteria

criterion: The response explains whether agent interaction is orchestrated through a central controller or peer-to-peer and how the coordination logic is implemented (e.g., workflow engine, custom orchestrator, rule-based routing, policy-based delegation).
weight: 3
axis: Implicit Criteria