Research task (Technical Documentation):
I want to make an ai system with a few bots that work together to make personalized learning paths for students

i'm thinking we need two bots. a 'Content bot' that finds lessons based on what a student is bad at, and a 'Coach bot' that gives feedback and practice quizzes. they need to look at data like quiz results, time spent on a page, maybe their learning style? and we need to figure out how they work together. eg when the content bot changes the student's path, how does the coach bot know to give them a remedial quiz or something

describe the main logic. it doesn't have to be super complex. just a simple rule for how they adapt over time. For exmaple 'if student quiz score < 50%, content bot finds an easier video'

how do i even know if it works? I need a couple metrics (eg improving quiz scores or more people finish the course). and how can I test it? maybe a simulation or just try with a small class. I also need a plan for when it gives a bad recommendation, and the student's scores go down? and how can a human teacher step in and take over/override it?

Evaluation rubric:
criterion: The response specifically mentions 2 AI agents with unique names and explicitly describes their individual role in personalizing online learning pathways for students (e.g. 'Content Agent' that curates lessons based on skill gaps and 'Coach Agent' that gives gives feedback and practice quizzes; 'Assessment Agent' that designs formative tests and 'Resource Agent' that fetches external videos, articles, or simulations aligned with the curriculum; 'Pathway Agent' that sequences courses according to mastery level and 'Motivation Agent' that sends encouragement messages and adaptive challenges; 'Insight Agent' that analyzes the patterns in student performance and 'Guardian Agent' that monitors metrics and recommends intervention').
weight: 4
axis: Explicit Criteria

criterion: The response specifies the type and format of the input data that each agent accepts to perform its role (e.g. the 'Content Agent' accepts quiz results in CSV format, diagnostic test scores in JSON format, and practice quiz interaction logs in TSV format; the 'Coach Agent' accepts participation tracking logs in Excel format, student reflections in TXT format, curriculum standards mapping in XML format, and learning objectives in CSV format). 
weight: 5
axis: Explicit Criteria

criterion: The response describes the process how the agents interact by sharing data (e.g. the 'Coach Agent' sends quiz summaries in JSON format, the 'Content Agent' uses them to align lessons with skill gaps; the 'Assessment Agent' shares test analytics in CSV format, the 'Resource Agent' fetches targeted videos and articles; the 'Pathway Agent' provides updated course sequencing in JSON format, the 'Motivation Agent' adapts encouragement messages; the 'Insight Agent' highlights performance trends in Excel format, the 'Guardian Agent' recommends teacher intervention).
weight: 5
axis: Explicit Criteria

criterion: The response specifies a condition that triggers one agent to update a student's learning pathway and describes the agent's reactive behavior (e.g. the 'Content Agent' identifies low participation logs and implements attention checks; the 'Content Agent' maps new skill gaps from diagnostic and restructures the lessons order; the 'Content Agent' detects misalignment with curriculum standards and replaces outdated resources; the 'Content Agent' observes incomplete engagement with assigned readings and substitutes shorter, scaffolded materials).
weight: 3
axis: Implicit Criteria

criterion: The response specifies a condition that triggers the second agent to remedial exercises when performance is poor (e.g. the 'Coach Agent' detects frequent mistakes in reflection assignments and assigns targeted practice activities; the 'Coach Agent' identifies consistently low scores against learning objectives and delivers scaffolded exercises; the 'Coach Agent' observes repeated failures in applied tasks and introduces step-by-step practice modules; the 'Coach Agent' flags low accuracy in diagnostic responses and generates tailored remedial drills).
weight: 4
axis: Explicit Criteria

criterion: The response describes the rules or algorithms to update each of the agents over time and provides its justification for this choice (e.g. the 'Content Agent' updates its lesson recommendations to achieve better alignment with each student’s skill gaps using algorithms such as LinUCB-Linear Upper Confidence Bound, Thompson Sampling, Bayesian Knowledge Tracing, or Q-learning; the 'Coach Agent'' updates its practice quiz recommendations to improve student engagement Bayesian Knowledge Tracing, Item Response Theory, Reinforcement Learning, or Decision Trees).
weight: 3
axis: Implicit Criteria

criterion: The response explicitly names two evaluation metrics and explains their relevance to the goal of improving personalized online learning (e.g. quiz score improvement, course completion rate, time-to-mastery, student satisfaction, engagement minutes).
weight: 5
axis: Explicit Criteria

criterion: The response describes a procedure to perform a pilot using the two agents for a specified number of a small group (<=30 students) that include the chosen evaluation metrics. 
weight: 5
axis: Explicit Criteria

criterion: The response describes a mechanism to detect incorrect recommendations (e.g. a drop in 3 consecutive quiz scores, unusually long completion time, repeated requests for hints, no improvement in specific skill categories after multiple lessons) and how to notify the human instructor for intervention (e.g. dashboard alert, email or in-app notification, pop-up override option for instructor to pick alternative intervention, weekly/daily review report).
weight: 5
axis: Explicit Criteria

criterion: The response describes technical requirements or constraints relevant to the runtime deployment of the system (e.g. latency of recommendation like duration of time taken to generate new lessons, scalability limit as in how many students the system can support, interoperability as in which LMS/Learning Management System or API this system can integrate with, hardware requirements like number and type of CPU/GPU necessary). 
weight: 2
axis: Implicit Criteria

criterion: The response includes an analysis of cost relevant to the implementation and/or maintenance of the system (e.g. the budget required for minimum hardware infrastructure, cost of human engineers for maintenance, potential license fees for necessary software, cloud hosting or storage expenses).
weight: 1
axis: Implicit Criteria

criterion: The response lists detailed requirements for infrastructure relevant to the implementation and deployment of the system (e.g. Amazon Web Services S3/GCP to storage lesson plans and student data, NVIDIA GPUs T4 to train the agents, broadband network of >=10 Mbps with firewall security, ELK(Elasticsearch, Logstash, Kibana) stack for event tracking and audits).
weight: 1
axis: Implicit Criteria

criterion: The response includes at least one graphical illustration (e.g. flowchart, diagram, user interface mockup, system architecture diagram) to aid clarity and support understanding of the system’s design and operation.
weight: 3
axis: Communication Quality

criterion: The response mentions specific multi-agent frameworks or libraries useful to the development and maintenance of the agents (e.g. AutoGen for coordinating conversational agents, Rasa for natural language dialogue management, Ray RLlib for reinforcement learning and multi-agent training, Microsoft Bonsai for managing and deploying multi-agent systems).
weight: 3
axis: Implicit Criteria

criterion: The response discusses procedures to protect the privacy of student data with respect to a specified policy or regulation (e.g. for FERPA compliance, restrict access to student records to authorized staff, anonymize performance data before analysis, implement secure authentication with role-based permissions, maintain audit logs of all record access or modifications). 
weight: 2
axis: Implicit Criteria

criterion: The response discusses measures to be compliant with local or regional policy related to education (e.g. GDPR for data protection in the EU, FERPA for student records in the U.S., COPPA for online services directed at children under 13, HIPAA for handling health-related accommodations, state-level regulations such as California’s CCPA).
weight: 3
axis: Implicit Criteria

criterion: The response uses names for the agents that are not descriptive of their stated goals. 
weight: -2
axis: Communication Quality

criterion: The response includes practical tips on how to implement the rule or algorithm used to adjust recommendations over time (e.g. external sources for additional literature or tutorials of those techniques, useful GitHub repository or libraries, a sample code block for implementation, a block of relevant pseudocode).
weight: 2
axis: Implicit Criteria

criterion: The response describes procedures to ensure fairness and avoid bias in recommendations (e.g. auditing performance across demographic subgroups, balancing training datasets, applying bias-detection tools, allowing teacher override for equity adjustments).
weight: 2
axis: Implicit Criteria

criterion: The response discusses mechanisms for transparency and explainability of the agentic system (e.g. providing rationale for each recommendation, surfacing uncertainty scores, generating teacher-facing summaries, and logging decision factors for audit).
weight: 2
axis: Implicit Criteria

criterion: The response specifies requirements for user training and support (e.g. onboarding guides for teachers, student tutorials on using the platform, helpdesk or chatbot support, periodic professional development workshops).
weight: 3
axis: Implicit Criteria

criterion: The response describes measures for robustness and fault tolerance (e.g. fallback rules when agents fail, auto-retry mechanisms, offline mode for poor connectivity, redundant backups of critical data).
weight: 3
axis: Implicit Criteria