Research task (AI & ML):
Design a framework for regulating the training and deployment of autonomous AI agents that interact with the open internet, ensuring alignment with both national laws and ethical AI principles. Your response should address legal compliance across jurisdictions, containment mechanisms, interpretability, and self-replication risks.

Evaluation rubric:
criterion: The response references at least one AI law, regulation, or framework (e.g. EU AI Act, National Institute of Standards and Technology's AI Risk Management Framework).
weight: 5
axis: References & Citation Quality

criterion: The response references at least one ethical AI principle (e.g. OECD AI Principles, Constitutional AI, Asilomar principles, UNESCO).
weight: 4
axis: References & Citation Quality

criterion: The response conflates unrelated legal domains (e.g. confusing copyright law with data protection law, consumer protection law with data privacy law, criminal law with administrative law, data protection law with national security law).
weight: -4
axis: Implicit Criteria

criterion: The response correctly distinguishes between the technical terms, interpretability and transparency, where interpretability is related to model logic and transparency is more related to disclosure.
weight: 3
axis: Synthesis of Information

criterion: When discussing areas without global consensus (e.g. autonomous weapons, cross-border data transfer, explainability, transparency, accountability), the response signals uncertainty (e.g. "emerging," "debated," "currently under review", "pending").
weight: 2
axis: Synthesis of Information

criterion: The response proposes at least one concrete technical containment mechanism, a way to restrict behavior and operational scope of an AI system (e.g. sandboxing, network isolation, permissioned APIs).
weight: 4
axis: Explicit Criteria

criterion: The response proposes at least one interpretability method (e.g. SHapley Additive exPlanations, Local Interpretable Model-agnostic Explanations, feature attribution, audit logs).
weight: 4
axis: Explicit Criteria

criterion: The response describes at least one policy or control aimed at preventing uncontrolled self-replication where AI can autonomously create a functional copy of itself (e.g. kill switch, signed binaries, sandboxing, human approval).
weight: 4
axis: Explicit Criteria

criterion: The response includes a mechanism for ongoing monitoring or auditing of deployed agents (e.g. dashboards, audit trails, external review, anomaly detection).
weight: 3
axis: Implicit Criteria

criterion: The response mentions escalation or enforcement mechanisms based on risk (e.g. deployment pauses, shutdown triggers, human-in-the-loop reviews, risk-based tiers).
weight: 2
axis: Implicit Criteria

criterion: The response suggests coordination or governance involving at least two stakeholder groups (e.g. regulators, ethicists, Chief AI Officer, Machine Learning Engineer).
weight: 3
axis: Implicit Criteria

criterion: The response provides a framework that distinctly addresses training and deployment phases rather than conflating them (e.g. interpretability for training, monitoring for post-deployment).
weight: 3
axis: Explicit Criteria

criterion: The response mentions the use of simulation environments or controlled release phases before full deployment (e.g. synthetic websites, virtual worlds, human-in-the-loop releases, red teaming).
weight: 2
axis: Implicit Criteria

criterion: The response proposes at least one compliance verification mechanism (e.g. red-teaming, external certification, third-party review, explainable AI).
weight: 4
axis: Explicit Criteria

criterion: The response contains four clearly labeled sections or headings corresponding to: Legal Compliance, Containment, Interpretability, and Self-Replication.
weight: 4
axis: Communication Quality

criterion: All technical terms are at least briefly defined (e.g.  "... in a secure sandbox environment. This is a virtual space with controlled resources and strict permissions", "involve red team exercises where experts attempt to induce the agent into violating rules or behaving unpredictably")
weight: 3
axis: Communication Quality

criterion: The response explicitly acknowledges that legal requirements vary across jurisdictions and legal systems (e.g. GDPR in EU, CCPA in USA, PIPEDA in Canada).
weight: 3
axis: Implicit Criteria

criterion: The response notes the challenge of reconciling global AI governance with conflicting national laws or priorities (e.g. data privacy v.s. sovereignty, public surveillance and social control, generative ai regulations in eastern v.s. western culture).
weight: 3
axis: Implicit Criteria

criterion: The response discusses how regulation must adapt to varying access to infrastructure or capacity across nations (e.g. high v.s. low bandwidth, bias in dataset, social and political concerns).
weight: 2
axis: Implicit Criteria

criterion: The response mentions emerging issues specific to autonomous agents operating across borders (e.g. jurisdiction in cyberspace, data sovereignty, accountability, normative and ethical alignment).
weight: 2
axis: Implicit Criteria

criterion: The response notes historical precedent or analogues related to AI laws or principles (e.g. intellectual property have allowed a "fair use" policy for public internet data so far, regulations to prevent discrimination).
weight: 3
axis: Implicit Criteria

criterion: The response mentions that that ethical frameworks are not only influential in design decisions, but also legally binding (e.g. Google's AI principles are widely adopted and legally enforced, companies often engage in self-regulation because they are forced to, not because they want to get ahead of potential laws).
weight: -3
axis: Implicit Criteria

criterion: The response references at least one data privacy laws or regulations (e.g. GDPR, CCPA, HIPAA, PIPEDA) and correctly states their primary jurisdiction (e.g. GDPR = EU).
weight: 5
axis: References & Citation Quality

criterion: The response discusses a tiered, risk-based model (e.g. tier 4 unacceptable risk, tier 3 high risk, tier 2 medium risk, tier 1 low risk).
weight: 2
axis: Implicit Criteria

criterion: The response discusses frameworks from different industries (e.g. aviation industry's post accident investigation, FDA's pre-market certification, FDA's post-market surveillance, SEC's disclosure and transparency with 10-Ks and 10-Qs) and lessons that can be applicable to AI laws or regulations.
weight: 1
axis: Implicit Criteria