2_lab2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to the Second Lab - Week 1, Day 3\n",
    "\n",
    "Today we will work with lots of models! This is a way to get comfortable with APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Important point - please read</h2>\n",
    "            <span style=\"color:#ff7800;\">The way I collaborate with you may be different to other courses you've taken. I prefer not to type code while you watch. Rather, I execute Jupyter Labs, like this, and give you an intuition for what's going on. My suggestion is that you carefully execute this yourself, <b>after</b> watching the lecture. Add print statements to understand what's going on, and then come up with your own variations.<br/><br/>If you have time, I'd love it if you submit a PR for changes in the community_contributions folder - instructions in the resources. Also, if you have a Github account, use this to showcase your variations. Not only is this essential practice, but it demonstrates your skills to others, including perhaps future clients or employers...\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start with imports - ask ChatGPT to explain any package that you don't know\n",
    "\n",
    "import os\n",
    "import json\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from anthropic import Anthropic\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Always remember to do this!\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API Key exists and begins sk-proj-\n",
      "Anthropic API Key exists and begins xxxx\n",
      "Google API Key exists and begins xx\n",
      "DeepSeek API Key exists and begins xxx\n",
      "Groq API Key not set (and this is optional)\n"
     ]
    }
   ],
   "source": [
    "# Print the key prefixes to help with any debugging\n",
    "\n",
    "openai_api_key = os.getenv('OPENAI_API_KEY')\n",
    "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
    "google_api_key = os.getenv('GOOGLE_API_KEY')\n",
    "deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')\n",
    "groq_api_key = os.getenv('GROQ_API_KEY')\n",
    "\n",
    "if openai_api_key:\n",
    "    print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"OpenAI API Key not set\")\n",
    "    \n",
    "if anthropic_api_key:\n",
    "    print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
    "else:\n",
    "    print(\"Anthropic API Key not set (and this is optional)\")\n",
    "\n",
    "if google_api_key:\n",
    "    print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
    "else:\n",
    "    print(\"Google API Key not set (and this is optional)\")\n",
    "\n",
    "if deepseek_api_key:\n",
    "    print(f\"DeepSeek API Key exists and begins {deepseek_api_key[:3]}\")\n",
    "else:\n",
    "    print(\"DeepSeek API Key not set (and this is optional)\")\n",
    "\n",
    "if groq_api_key:\n",
    "    print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n",
    "else:\n",
    "    print(\"Groq API Key not set (and this is optional)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "request = \"Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. \"\n",
    "request += \"Answer only with the question, no explanation.\"\n",
    "messages = [{\"role\": \"user\", \"content\": request}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'role': 'user',\n",
       "  'content': 'Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. Answer only with the question, no explanation.'}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "If you could design an ideal society with a unique governance structure that incorporates elements of democracy, meritocracy, and technocracy, what foundational principles would guide this society, and how would you address potential ethical dilemmas arising from each element?\n"
     ]
    }
   ],
   "source": [
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"gpt-4o-mini\",\n",
    "    messages=messages,\n",
    ")\n",
    "question = response.choices[0].message.content\n",
    "print(question)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "competitors = []\n",
    "answers = []\n",
    "messages = [{\"role\": \"user\", \"content\": question}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Designing an ideal society that melds elements of democracy, meritocracy, and technocracy involves establishing a governance structure that balances representation, expertise, and equitable opportunity. Below are foundational principles and strategies for addressing potential ethical dilemmas associated with each governance element.\n",
       "\n",
       "### Foundational Principles\n",
       "\n",
       "1. **Participatory Governance**: \n",
       "   - **Democracy**: Every citizen should have an opportunity to participate in decision-making through direct or indirect means, ensuring diverse representation. Regular referenda and citizen assemblies could be used to enhance direct participation in key issues.\n",
       "   - **Ethical Dilemma**: The challenge of uninformed decision-making by the general populace can lead to poor choices. To mitigate this, a well-designed education system emphasizing critical thinking and civic knowledge is essential.\n",
       "\n",
       "2. **Competence-based Leadership**:\n",
       "   - **Meritocracy**: Leaders and decision-makers should be selected based on demonstrated skills, experience, and achievements relevant to their roles. This would ensure that the most capable individuals are in charge of significant decisions, fostering efficiency.\n",
       "   - **Ethical Dilemma**: A strict meritocratic system may lead to elitism and may undervalue contribution from diverse backgrounds. To address this, merit evaluations can be made holistic, considering a range of intelligence forms, such as emotional and social intelligence, along with traditional academic and professional accomplishments.\n",
       "\n",
       "3. **Science and Technology Integration**:\n",
       "   - **Technocracy**: Technological and scientific expertise should guide policy-making, especially in areas of critical importance like health, environment, and infrastructure. Utilizing data-driven approaches and expert recommendations can enhance decision quality.\n",
       "   - **Ethical Dilemma**: A technocratic approach risks sidelining human values, customary practices, and ethical considerations in favor of efficiency or data. To handle this, a framework for ethical review must be established, where technology and science are applied not just for efficacy but are also subject to moral scrutiny.\n",
       "\n",
       "4. **Transparency and Accountability**:\n",
       "   - All actions and decisions within the governance structure must be transparent, with mechanisms for accountability. This can help combat corruption and enhance public trust in both leaders and the systematic approach to governance.\n",
       "   - **Ethical Dilemma**: Over-transparency may threaten privacy or decision-making freedom. A balance can be achieved by ensuring that while public interest is prioritized, individual rights and necessary confidentiality in certain matters are also respected.\n",
       "\n",
       "5. **Equity and Inclusion**:\n",
       "   - Ensure equitable access to opportunities, resources, and political representation for all citizens, no matter their background. This principle should permeate all aspects of society, ensuring that meritocratic systems are not dominated by systemic inequalities.\n",
       "   - **Ethical Dilemma**: Striving for equity can lead to tensions between opportunity and the pursuit of excellence. To reconcile this, mechanisms such as targeted mentorship programs and affirmative actions can provide support without undermining standards.\n",
       "\n",
       "6. **Adaptability and Evolution**:\n",
       "   - The governance structure should include mechanisms for continual self-assessment and adaptation based on complexity and changing circumstances. Debate, feedback loops, and pilot programs can facilitate this adaptability.\n",
       "   - **Ethical Dilemma**: Frequent changes can destabilize governance structures. To mitigate this, substantial changes should be subject to rigorous analysis and general consensus, ensuring thorough public discourse and understanding.\n",
       "\n",
       "### Ethical Oversight\n",
       "\n",
       "To navigate potential ethical dilemmas, an independent ethics committee can be established. This body would be tasked with reviewing decisions led by democratic, meritocratic, and technocratic elements and ensuring they align with the overarching values of the society. Public representation on this committee can also enhance transparency and accountability.\n",
       "\n",
       "In summary, the ideal society would be built on participatory governance, competence-based leadership, scientific integration, transparency, equity, and adaptability. These foundational principles would not only guide the society's operation but would serve as a bedrock for resolving the inherent ethical dilemmas of each governance element, fostering a balanced and just society."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# The API we know well\n",
    "\n",
    "model_name = \"gpt-4o-mini\"\n",
    "\n",
    "response = openai.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Anthropic has a slightly different API, and Max Tokens is required\n",
    "\n",
    "model_name = \"claude-3-7-sonnet-latest\"\n",
    "\n",
    "claude = Anthropic()\n",
    "response = claude.messages.create(model=model_name, messages=messages, max_tokens=1000)\n",
    "answer = response.content[0].text\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gemini = OpenAI(api_key=google_api_key, base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\")\n",
    "model_name = \"gemini-2.0-flash\"\n",
    "\n",
    "response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deepseek = OpenAI(api_key=deepseek_api_key, base_url=\"https://api.deepseek.com/v1\")\n",
    "model_name = \"deepseek-chat\"\n",
    "\n",
    "response = deepseek.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groq = OpenAI(api_key=groq_api_key, base_url=\"https://api.groq.com/openai/v1\")\n",
    "model_name = \"llama-3.3-70b-versatile\"\n",
    "\n",
    "response = groq.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For the next cell, we will use Ollama\n",
    "\n",
    "Ollama runs a local web service that gives an OpenAI compatible endpoint,  \n",
    "and runs models locally using high performance C++ code.\n",
    "\n",
    "If you don't have Ollama, install it here by visiting https://ollama.com then pressing Download and following the instructions.\n",
    "\n",
    "After it's installed, you should be able to visit here: http://localhost:11434 and see the message \"Ollama is running\"\n",
    "\n",
    "You might need to restart Cursor (and maybe reboot). Then open a Terminal (control+\\`) and run `ollama serve`\n",
    "\n",
    "Useful Ollama commands (run these in the terminal, or with an exclamation mark in this notebook):\n",
    "\n",
    "`ollama pull <model_name>` downloads a model locally  \n",
    "`ollama ls` lists all the models you've downloaded  \n",
    "`ollama rm <model_name>` deletes the specified model from your downloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Super important - ignore me at your peril!</h2>\n",
    "            <span style=\"color:#ff7800;\">The model called <b>llama3.3</b> is FAR too large for home computers - it's not intended for personal computing and will consume all your resources! Stick with the nicely sized <b>llama3.2</b> or <b>llama3.2:1b</b> and if you want larger, try llama3.1 or smaller variants of Qwen, Gemma, Phi or DeepSeek. See the <A href=\"https://ollama.com/models\">the Ollama models page</a> for a full list of models and sizes.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ollama pull llama3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Designing an ideal society with a governance structure that combines the principles of democracy, meritocracy, and technocracy presents several challenges. However, by understanding the core values and implications of each governance type, we can create a framework for an innovative society that strives to balance individual freedoms, collective well-being, and technological advancements.\n",
       "\n",
       "Foundational Principles:\n",
       "\n",
       "1. **Participatory Democracy**: Inclusive participation is essential for creating a citizen-led government where the voices of citizens are actively engaged in decision-making processes.\n",
       "2. **Meritocratic Hiring Process**: A fair, unbiased selection method would be implemented to recruit highly skilled technocrats and policymakers who prioritize the common good over personal interests.\n",
       "3. **Technocratic Expertise**: Technicians with expertise in evidence-based policy analysis and development would work alongside elected officials, ensuring informed decision-making is prioritized.\n",
       "4. **Personal Responsibility and Accountability**: Citizens would have a vested interest in taking personal responsibility for their actions, and policymakers would be held accountable for the outcomes of their decisions.\n",
       "\n",
       "Governance Structure:\n",
       "\n",
       "1.  **Elected Legislative Assembly**: A fair, competitive election system would select representatives who embody democratic values. The number of seats allocated to each seat type and how votes are counted may vary depending on population.\n",
       "2.  **Technocratic Expert Advisory Panel**: Established by the Executive or Senate Board, this commission comprises impartial experts in public policy, technology, and economics.\n",
       "3.  **Governor-Representative Role**: Each constituency governor will have executive authority to act as a mediator and support their constituents directly.\n",
       "\n",
       "Addressing Ethical Dilemmas:\n",
       "\n",
       "1.  Ensuring representation from underrepresented groups might be difficult.\n",
       "2.  Balancing individual freedoms with the collective good.\n",
       "3.  Making informed decisions about policies that impact society's sustainability.\n",
       "4.  Addressing privacy concerns related to surveillance technologies.\n",
       "\n",
       "Ethics Frameworks would be necessary.\n",
       "\n",
       "For instance we may use:\n",
       "\n",
       "1.  **Utilitarianism**: This perspective maximizes overall happiness or well-being.\n",
       "2.  **Deontology**: Prioritize moral rules and duties, such as protecting human rights and life expectancy.\n",
       "3.  **Virtue Ethics**: This approach emphasizes the development of desirable character traits like empathy and justice in leaders.\n",
       "\n",
       "Implementation Challenges\n",
       "\n",
       "The following factors may influence the successful implementation of this vision:\n",
       "\n",
       "*   Overcoming historical narratives tied to existing governance models: \n",
       "*   Rebuilding trust among citizens after being perceived as oppressive or ineffective.\n",
       "    Incentivizing participation and encouraging civic engagement\n",
       "*   Balancing individual freedoms with group identity needs\n",
       "*   Preventing corruption in the technocratic process\n",
       "\n",
       "In conclusion, designing an ideal society with a governance structure based on democracy, meritocracy, and technocracy will require ongoing evaluation and refinement. This would necessitate open communication among all stakeholders, collaborative decision-making processes, and innovative strategies to address existing challenges. The ethical frameworks used may play a substantial role in facilitating societal harmony."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ollama = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
    "model_name = \"llama3.2\"\n",
    "\n",
    "response = ollama.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['gpt-4o-mini', 'llama3.2']\n",
      "[\"Designing an ideal society that melds elements of democracy, meritocracy, and technocracy involves establishing a governance structure that balances representation, expertise, and equitable opportunity. Below are foundational principles and strategies for addressing potential ethical dilemmas associated with each governance element.\\n\\n### Foundational Principles\\n\\n1. **Participatory Governance**: \\n   - **Democracy**: Every citizen should have an opportunity to participate in decision-making through direct or indirect means, ensuring diverse representation. Regular referenda and citizen assemblies could be used to enhance direct participation in key issues.\\n   - **Ethical Dilemma**: The challenge of uninformed decision-making by the general populace can lead to poor choices. To mitigate this, a well-designed education system emphasizing critical thinking and civic knowledge is essential.\\n\\n2. **Competence-based Leadership**:\\n   - **Meritocracy**: Leaders and decision-makers should be selected based on demonstrated skills, experience, and achievements relevant to their roles. This would ensure that the most capable individuals are in charge of significant decisions, fostering efficiency.\\n   - **Ethical Dilemma**: A strict meritocratic system may lead to elitism and may undervalue contribution from diverse backgrounds. To address this, merit evaluations can be made holistic, considering a range of intelligence forms, such as emotional and social intelligence, along with traditional academic and professional accomplishments.\\n\\n3. **Science and Technology Integration**:\\n   - **Technocracy**: Technological and scientific expertise should guide policy-making, especially in areas of critical importance like health, environment, and infrastructure. Utilizing data-driven approaches and expert recommendations can enhance decision quality.\\n   - **Ethical Dilemma**: A technocratic approach risks sidelining human values, customary practices, and ethical considerations in favor of efficiency or data. To handle this, a framework for ethical review must be established, where technology and science are applied not just for efficacy but are also subject to moral scrutiny.\\n\\n4. **Transparency and Accountability**:\\n   - All actions and decisions within the governance structure must be transparent, with mechanisms for accountability. This can help combat corruption and enhance public trust in both leaders and the systematic approach to governance.\\n   - **Ethical Dilemma**: Over-transparency may threaten privacy or decision-making freedom. A balance can be achieved by ensuring that while public interest is prioritized, individual rights and necessary confidentiality in certain matters are also respected.\\n\\n5. **Equity and Inclusion**:\\n   - Ensure equitable access to opportunities, resources, and political representation for all citizens, no matter their background. This principle should permeate all aspects of society, ensuring that meritocratic systems are not dominated by systemic inequalities.\\n   - **Ethical Dilemma**: Striving for equity can lead to tensions between opportunity and the pursuit of excellence. To reconcile this, mechanisms such as targeted mentorship programs and affirmative actions can provide support without undermining standards.\\n\\n6. **Adaptability and Evolution**:\\n   - The governance structure should include mechanisms for continual self-assessment and adaptation based on complexity and changing circumstances. Debate, feedback loops, and pilot programs can facilitate this adaptability.\\n   - **Ethical Dilemma**: Frequent changes can destabilize governance structures. To mitigate this, substantial changes should be subject to rigorous analysis and general consensus, ensuring thorough public discourse and understanding.\\n\\n### Ethical Oversight\\n\\nTo navigate potential ethical dilemmas, an independent ethics committee can be established. This body would be tasked with reviewing decisions led by democratic, meritocratic, and technocratic elements and ensuring they align with the overarching values of the society. Public representation on this committee can also enhance transparency and accountability.\\n\\nIn summary, the ideal society would be built on participatory governance, competence-based leadership, scientific integration, transparency, equity, and adaptability. These foundational principles would not only guide the society's operation but would serve as a bedrock for resolving the inherent ethical dilemmas of each governance element, fostering a balanced and just society.\", \"Designing an ideal society with a governance structure that combines the principles of democracy, meritocracy, and technocracy presents several challenges. However, by understanding the core values and implications of each governance type, we can create a framework for an innovative society that strives to balance individual freedoms, collective well-being, and technological advancements.\\n\\nFoundational Principles:\\n\\n1. **Participatory Democracy**: Inclusive participation is essential for creating a citizen-led government where the voices of citizens are actively engaged in decision-making processes.\\n2. **Meritocratic Hiring Process**: A fair, unbiased selection method would be implemented to recruit highly skilled technocrats and policymakers who prioritize the common good over personal interests.\\n3. **Technocratic Expertise**: Technicians with expertise in evidence-based policy analysis and development would work alongside elected officials, ensuring informed decision-making is prioritized.\\n4. **Personal Responsibility and Accountability**: Citizens would have a vested interest in taking personal responsibility for their actions, and policymakers would be held accountable for the outcomes of their decisions.\\n\\nGovernance Structure:\\n\\n1.  **Elected Legislative Assembly**: A fair, competitive election system would select representatives who embody democratic values. The number of seats allocated to each seat type and how votes are counted may vary depending on population.\\n2.  **Technocratic Expert Advisory Panel**: Established by the Executive or Senate Board, this commission comprises impartial experts in public policy, technology, and economics.\\n3.  **Governor-Representative Role**: Each constituency governor will have executive authority to act as a mediator and support their constituents directly.\\n\\nAddressing Ethical Dilemmas:\\n\\n1.  Ensuring representation from underrepresented groups might be difficult.\\n2.  Balancing individual freedoms with the collective good.\\n3.  Making informed decisions about policies that impact society's sustainability.\\n4.  Addressing privacy concerns related to surveillance technologies.\\n\\nEthics Frameworks would be necessary.\\n\\nFor instance we may use:\\n\\n1.  **Utilitarianism**: This perspective maximizes overall happiness or well-being.\\n2.  **Deontology**: Prioritize moral rules and duties, such as protecting human rights and life expectancy.\\n3.  **Virtue Ethics**: This approach emphasizes the development of desirable character traits like empathy and justice in leaders.\\n\\nImplementation Challenges\\n\\nThe following factors may influence the successful implementation of this vision:\\n\\n*   Overcoming historical narratives tied to existing governance models: \\n*   Rebuilding trust among citizens after being perceived as oppressive or ineffective.\\n    Incentivizing participation and encouraging civic engagement\\n*   Balancing individual freedoms with group identity needs\\n*   Preventing corruption in the technocratic process\\n\\nIn conclusion, designing an ideal society with a governance structure based on democracy, meritocracy, and technocracy will require ongoing evaluation and refinement. This would necessitate open communication among all stakeholders, collaborative decision-making processes, and innovative strategies to address existing challenges. The ethical frameworks used may play a substantial role in facilitating societal harmony.\"]\n"
     ]
    }
   ],
   "source": [
    "# So where are we?\n",
    "\n",
    "print(competitors)\n",
    "print(answers)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Competitor: gpt-4o-mini\n",
      "\n",
      "Designing an ideal society that melds elements of democracy, meritocracy, and technocracy involves establishing a governance structure that balances representation, expertise, and equitable opportunity. Below are foundational principles and strategies for addressing potential ethical dilemmas associated with each governance element.\n",
      "\n",
      "### Foundational Principles\n",
      "\n",
      "1. **Participatory Governance**: \n",
      "   - **Democracy**: Every citizen should have an opportunity to participate in decision-making through direct or indirect means, ensuring diverse representation. Regular referenda and citizen assemblies could be used to enhance direct participation in key issues.\n",
      "   - **Ethical Dilemma**: The challenge of uninformed decision-making by the general populace can lead to poor choices. To mitigate this, a well-designed education system emphasizing critical thinking and civic knowledge is essential.\n",
      "\n",
      "2. **Competence-based Leadership**:\n",
      "   - **Meritocracy**: Leaders and decision-makers should be selected based on demonstrated skills, experience, and achievements relevant to their roles. This would ensure that the most capable individuals are in charge of significant decisions, fostering efficiency.\n",
      "   - **Ethical Dilemma**: A strict meritocratic system may lead to elitism and may undervalue contribution from diverse backgrounds. To address this, merit evaluations can be made holistic, considering a range of intelligence forms, such as emotional and social intelligence, along with traditional academic and professional accomplishments.\n",
      "\n",
      "3. **Science and Technology Integration**:\n",
      "   - **Technocracy**: Technological and scientific expertise should guide policy-making, especially in areas of critical importance like health, environment, and infrastructure. Utilizing data-driven approaches and expert recommendations can enhance decision quality.\n",
      "   - **Ethical Dilemma**: A technocratic approach risks sidelining human values, customary practices, and ethical considerations in favor of efficiency or data. To handle this, a framework for ethical review must be established, where technology and science are applied not just for efficacy but are also subject to moral scrutiny.\n",
      "\n",
      "4. **Transparency and Accountability**:\n",
      "   - All actions and decisions within the governance structure must be transparent, with mechanisms for accountability. This can help combat corruption and enhance public trust in both leaders and the systematic approach to governance.\n",
      "   - **Ethical Dilemma**: Over-transparency may threaten privacy or decision-making freedom. A balance can be achieved by ensuring that while public interest is prioritized, individual rights and necessary confidentiality in certain matters are also respected.\n",
      "\n",
      "5. **Equity and Inclusion**:\n",
      "   - Ensure equitable access to opportunities, resources, and political representation for all citizens, no matter their background. This principle should permeate all aspects of society, ensuring that meritocratic systems are not dominated by systemic inequalities.\n",
      "   - **Ethical Dilemma**: Striving for equity can lead to tensions between opportunity and the pursuit of excellence. To reconcile this, mechanisms such as targeted mentorship programs and affirmative actions can provide support without undermining standards.\n",
      "\n",
      "6. **Adaptability and Evolution**:\n",
      "   - The governance structure should include mechanisms for continual self-assessment and adaptation based on complexity and changing circumstances. Debate, feedback loops, and pilot programs can facilitate this adaptability.\n",
      "   - **Ethical Dilemma**: Frequent changes can destabilize governance structures. To mitigate this, substantial changes should be subject to rigorous analysis and general consensus, ensuring thorough public discourse and understanding.\n",
      "\n",
      "### Ethical Oversight\n",
      "\n",
      "To navigate potential ethical dilemmas, an independent ethics committee can be established. This body would be tasked with reviewing decisions led by democratic, meritocratic, and technocratic elements and ensuring they align with the overarching values of the society. Public representation on this committee can also enhance transparency and accountability.\n",
      "\n",
      "In summary, the ideal society would be built on participatory governance, competence-based leadership, scientific integration, transparency, equity, and adaptability. These foundational principles would not only guide the society's operation but would serve as a bedrock for resolving the inherent ethical dilemmas of each governance element, fostering a balanced and just society.\n",
      "Competitor: llama3.2\n",
      "\n",
      "Designing an ideal society with a governance structure that combines the principles of democracy, meritocracy, and technocracy presents several challenges. However, by understanding the core values and implications of each governance type, we can create a framework for an innovative society that strives to balance individual freedoms, collective well-being, and technological advancements.\n",
      "\n",
      "Foundational Principles:\n",
      "\n",
      "1. **Participatory Democracy**: Inclusive participation is essential for creating a citizen-led government where the voices of citizens are actively engaged in decision-making processes.\n",
      "2. **Meritocratic Hiring Process**: A fair, unbiased selection method would be implemented to recruit highly skilled technocrats and policymakers who prioritize the common good over personal interests.\n",
      "3. **Technocratic Expertise**: Technicians with expertise in evidence-based policy analysis and development would work alongside elected officials, ensuring informed decision-making is prioritized.\n",
      "4. **Personal Responsibility and Accountability**: Citizens would have a vested interest in taking personal responsibility for their actions, and policymakers would be held accountable for the outcomes of their decisions.\n",
      "\n",
      "Governance Structure:\n",
      "\n",
      "1.  **Elected Legislative Assembly**: A fair, competitive election system would select representatives who embody democratic values. The number of seats allocated to each seat type and how votes are counted may vary depending on population.\n",
      "2.  **Technocratic Expert Advisory Panel**: Established by the Executive or Senate Board, this commission comprises impartial experts in public policy, technology, and economics.\n",
      "3.  **Governor-Representative Role**: Each constituency governor will have executive authority to act as a mediator and support their constituents directly.\n",
      "\n",
      "Addressing Ethical Dilemmas:\n",
      "\n",
      "1.  Ensuring representation from underrepresented groups might be difficult.\n",
      "2.  Balancing individual freedoms with the collective good.\n",
      "3.  Making informed decisions about policies that impact society's sustainability.\n",
      "4.  Addressing privacy concerns related to surveillance technologies.\n",
      "\n",
      "Ethics Frameworks would be necessary.\n",
      "\n",
      "For instance we may use:\n",
      "\n",
      "1.  **Utilitarianism**: This perspective maximizes overall happiness or well-being.\n",
      "2.  **Deontology**: Prioritize moral rules and duties, such as protecting human rights and life expectancy.\n",
      "3.  **Virtue Ethics**: This approach emphasizes the development of desirable character traits like empathy and justice in leaders.\n",
      "\n",
      "Implementation Challenges\n",
      "\n",
      "The following factors may influence the successful implementation of this vision:\n",
      "\n",
      "*   Overcoming historical narratives tied to existing governance models: \n",
      "*   Rebuilding trust among citizens after being perceived as oppressive or ineffective.\n",
      "    Incentivizing participation and encouraging civic engagement\n",
      "*   Balancing individual freedoms with group identity needs\n",
      "*   Preventing corruption in the technocratic process\n",
      "\n",
      "In conclusion, designing an ideal society with a governance structure based on democracy, meritocracy, and technocracy will require ongoing evaluation and refinement. This would necessitate open communication among all stakeholders, collaborative decision-making processes, and innovative strategies to address existing challenges. The ethical frameworks used may play a substantial role in facilitating societal harmony.\n"
     ]
    }
   ],
   "source": [
    "# It's nice to know how to use \"zip\"\n",
    "for competitor, answer in zip(competitors, answers):\n",
    "    print(f\"Competitor: {competitor}\\n\\n{answer}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's bring this together - note the use of \"enumerate\"\n",
    "\n",
    "together = \"\"\n",
    "for index, answer in enumerate(answers):\n",
    "    together += f\"# Response from competitor {index+1}\\n\\n\"\n",
    "    together += answer + \"\\n\\n\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Response from competitor 1\n",
      "\n",
      "Designing an ideal society that melds elements of democracy, meritocracy, and technocracy involves establishing a governance structure that balances representation, expertise, and equitable opportunity. Below are foundational principles and strategies for addressing potential ethical dilemmas associated with each governance element.\n",
      "\n",
      "### Foundational Principles\n",
      "\n",
      "1. **Participatory Governance**: \n",
      "   - **Democracy**: Every citizen should have an opportunity to participate in decision-making through direct or indirect means, ensuring diverse representation. Regular referenda and citizen assemblies could be used to enhance direct participation in key issues.\n",
      "   - **Ethical Dilemma**: The challenge of uninformed decision-making by the general populace can lead to poor choices. To mitigate this, a well-designed education system emphasizing critical thinking and civic knowledge is essential.\n",
      "\n",
      "2. **Competence-based Leadership**:\n",
      "   - **Meritocracy**: Leaders and decision-makers should be selected based on demonstrated skills, experience, and achievements relevant to their roles. This would ensure that the most capable individuals are in charge of significant decisions, fostering efficiency.\n",
      "   - **Ethical Dilemma**: A strict meritocratic system may lead to elitism and may undervalue contribution from diverse backgrounds. To address this, merit evaluations can be made holistic, considering a range of intelligence forms, such as emotional and social intelligence, along with traditional academic and professional accomplishments.\n",
      "\n",
      "3. **Science and Technology Integration**:\n",
      "   - **Technocracy**: Technological and scientific expertise should guide policy-making, especially in areas of critical importance like health, environment, and infrastructure. Utilizing data-driven approaches and expert recommendations can enhance decision quality.\n",
      "   - **Ethical Dilemma**: A technocratic approach risks sidelining human values, customary practices, and ethical considerations in favor of efficiency or data. To handle this, a framework for ethical review must be established, where technology and science are applied not just for efficacy but are also subject to moral scrutiny.\n",
      "\n",
      "4. **Transparency and Accountability**:\n",
      "   - All actions and decisions within the governance structure must be transparent, with mechanisms for accountability. This can help combat corruption and enhance public trust in both leaders and the systematic approach to governance.\n",
      "   - **Ethical Dilemma**: Over-transparency may threaten privacy or decision-making freedom. A balance can be achieved by ensuring that while public interest is prioritized, individual rights and necessary confidentiality in certain matters are also respected.\n",
      "\n",
      "5. **Equity and Inclusion**:\n",
      "   - Ensure equitable access to opportunities, resources, and political representation for all citizens, no matter their background. This principle should permeate all aspects of society, ensuring that meritocratic systems are not dominated by systemic inequalities.\n",
      "   - **Ethical Dilemma**: Striving for equity can lead to tensions between opportunity and the pursuit of excellence. To reconcile this, mechanisms such as targeted mentorship programs and affirmative actions can provide support without undermining standards.\n",
      "\n",
      "6. **Adaptability and Evolution**:\n",
      "   - The governance structure should include mechanisms for continual self-assessment and adaptation based on complexity and changing circumstances. Debate, feedback loops, and pilot programs can facilitate this adaptability.\n",
      "   - **Ethical Dilemma**: Frequent changes can destabilize governance structures. To mitigate this, substantial changes should be subject to rigorous analysis and general consensus, ensuring thorough public discourse and understanding.\n",
      "\n",
      "### Ethical Oversight\n",
      "\n",
      "To navigate potential ethical dilemmas, an independent ethics committee can be established. This body would be tasked with reviewing decisions led by democratic, meritocratic, and technocratic elements and ensuring they align with the overarching values of the society. Public representation on this committee can also enhance transparency and accountability.\n",
      "\n",
      "In summary, the ideal society would be built on participatory governance, competence-based leadership, scientific integration, transparency, equity, and adaptability. These foundational principles would not only guide the society's operation but would serve as a bedrock for resolving the inherent ethical dilemmas of each governance element, fostering a balanced and just society.\n",
      "\n",
      "# Response from competitor 2\n",
      "\n",
      "Designing an ideal society with a governance structure that combines the principles of democracy, meritocracy, and technocracy presents several challenges. However, by understanding the core values and implications of each governance type, we can create a framework for an innovative society that strives to balance individual freedoms, collective well-being, and technological advancements.\n",
      "\n",
      "Foundational Principles:\n",
      "\n",
      "1. **Participatory Democracy**: Inclusive participation is essential for creating a citizen-led government where the voices of citizens are actively engaged in decision-making processes.\n",
      "2. **Meritocratic Hiring Process**: A fair, unbiased selection method would be implemented to recruit highly skilled technocrats and policymakers who prioritize the common good over personal interests.\n",
      "3. **Technocratic Expertise**: Technicians with expertise in evidence-based policy analysis and development would work alongside elected officials, ensuring informed decision-making is prioritized.\n",
      "4. **Personal Responsibility and Accountability**: Citizens would have a vested interest in taking personal responsibility for their actions, and policymakers would be held accountable for the outcomes of their decisions.\n",
      "\n",
      "Governance Structure:\n",
      "\n",
      "1.  **Elected Legislative Assembly**: A fair, competitive election system would select representatives who embody democratic values. The number of seats allocated to each seat type and how votes are counted may vary depending on population.\n",
      "2.  **Technocratic Expert Advisory Panel**: Established by the Executive or Senate Board, this commission comprises impartial experts in public policy, technology, and economics.\n",
      "3.  **Governor-Representative Role**: Each constituency governor will have executive authority to act as a mediator and support their constituents directly.\n",
      "\n",
      "Addressing Ethical Dilemmas:\n",
      "\n",
      "1.  Ensuring representation from underrepresented groups might be difficult.\n",
      "2.  Balancing individual freedoms with the collective good.\n",
      "3.  Making informed decisions about policies that impact society's sustainability.\n",
      "4.  Addressing privacy concerns related to surveillance technologies.\n",
      "\n",
      "Ethics Frameworks would be necessary.\n",
      "\n",
      "For instance we may use:\n",
      "\n",
      "1.  **Utilitarianism**: This perspective maximizes overall happiness or well-being.\n",
      "2.  **Deontology**: Prioritize moral rules and duties, such as protecting human rights and life expectancy.\n",
      "3.  **Virtue Ethics**: This approach emphasizes the development of desirable character traits like empathy and justice in leaders.\n",
      "\n",
      "Implementation Challenges\n",
      "\n",
      "The following factors may influence the successful implementation of this vision:\n",
      "\n",
      "*   Overcoming historical narratives tied to existing governance models: \n",
      "*   Rebuilding trust among citizens after being perceived as oppressive or ineffective.\n",
      "    Incentivizing participation and encouraging civic engagement\n",
      "*   Balancing individual freedoms with group identity needs\n",
      "*   Preventing corruption in the technocratic process\n",
      "\n",
      "In conclusion, designing an ideal society with a governance structure based on democracy, meritocracy, and technocracy will require ongoing evaluation and refinement. This would necessitate open communication among all stakeholders, collaborative decision-making processes, and innovative strategies to address existing challenges. The ethical frameworks used may play a substantial role in facilitating societal harmony.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(together)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge = f\"\"\"You are judging a competition between {len(competitors)} competitors.\n",
    "Each model has been given this question:\n",
    "\n",
    "{question}\n",
    "\n",
    "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
    "Respond with JSON, and only JSON, with the following format:\n",
    "{{\"results\": [\"best competitor number\", \"second best competitor number\", ...]}}\n",
    "\n",
    "Here are the responses from each competitor:\n",
    "\n",
    "{together}\n",
    "\n",
    "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are judging a competition between 2 competitors.\n",
      "Each model has been given this question:\n",
      "\n",
      "If you could design an ideal society with a unique governance structure that incorporates elements of democracy, meritocracy, and technocracy, what foundational principles would guide this society, and how would you address potential ethical dilemmas arising from each element?\n",
      "\n",
      "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
      "Respond with JSON, and only JSON, with the following format:\n",
      "{\"results\": [\"best competitor number\", \"second best competitor number\", ...]}\n",
      "\n",
      "Here are the responses from each competitor:\n",
      "\n",
      "# Response from competitor 1\n",
      "\n",
      "Designing an ideal society that melds elements of democracy, meritocracy, and technocracy involves establishing a governance structure that balances representation, expertise, and equitable opportunity. Below are foundational principles and strategies for addressing potential ethical dilemmas associated with each governance element.\n",
      "\n",
      "### Foundational Principles\n",
      "\n",
      "1. **Participatory Governance**: \n",
      "   - **Democracy**: Every citizen should have an opportunity to participate in decision-making through direct or indirect means, ensuring diverse representation. Regular referenda and citizen assemblies could be used to enhance direct participation in key issues.\n",
      "   - **Ethical Dilemma**: The challenge of uninformed decision-making by the general populace can lead to poor choices. To mitigate this, a well-designed education system emphasizing critical thinking and civic knowledge is essential.\n",
      "\n",
      "2. **Competence-based Leadership**:\n",
      "   - **Meritocracy**: Leaders and decision-makers should be selected based on demonstrated skills, experience, and achievements relevant to their roles. This would ensure that the most capable individuals are in charge of significant decisions, fostering efficiency.\n",
      "   - **Ethical Dilemma**: A strict meritocratic system may lead to elitism and may undervalue contribution from diverse backgrounds. To address this, merit evaluations can be made holistic, considering a range of intelligence forms, such as emotional and social intelligence, along with traditional academic and professional accomplishments.\n",
      "\n",
      "3. **Science and Technology Integration**:\n",
      "   - **Technocracy**: Technological and scientific expertise should guide policy-making, especially in areas of critical importance like health, environment, and infrastructure. Utilizing data-driven approaches and expert recommendations can enhance decision quality.\n",
      "   - **Ethical Dilemma**: A technocratic approach risks sidelining human values, customary practices, and ethical considerations in favor of efficiency or data. To handle this, a framework for ethical review must be established, where technology and science are applied not just for efficacy but are also subject to moral scrutiny.\n",
      "\n",
      "4. **Transparency and Accountability**:\n",
      "   - All actions and decisions within the governance structure must be transparent, with mechanisms for accountability. This can help combat corruption and enhance public trust in both leaders and the systematic approach to governance.\n",
      "   - **Ethical Dilemma**: Over-transparency may threaten privacy or decision-making freedom. A balance can be achieved by ensuring that while public interest is prioritized, individual rights and necessary confidentiality in certain matters are also respected.\n",
      "\n",
      "5. **Equity and Inclusion**:\n",
      "   - Ensure equitable access to opportunities, resources, and political representation for all citizens, no matter their background. This principle should permeate all aspects of society, ensuring that meritocratic systems are not dominated by systemic inequalities.\n",
      "   - **Ethical Dilemma**: Striving for equity can lead to tensions between opportunity and the pursuit of excellence. To reconcile this, mechanisms such as targeted mentorship programs and affirmative actions can provide support without undermining standards.\n",
      "\n",
      "6. **Adaptability and Evolution**:\n",
      "   - The governance structure should include mechanisms for continual self-assessment and adaptation based on complexity and changing circumstances. Debate, feedback loops, and pilot programs can facilitate this adaptability.\n",
      "   - **Ethical Dilemma**: Frequent changes can destabilize governance structures. To mitigate this, substantial changes should be subject to rigorous analysis and general consensus, ensuring thorough public discourse and understanding.\n",
      "\n",
      "### Ethical Oversight\n",
      "\n",
      "To navigate potential ethical dilemmas, an independent ethics committee can be established. This body would be tasked with reviewing decisions led by democratic, meritocratic, and technocratic elements and ensuring they align with the overarching values of the society. Public representation on this committee can also enhance transparency and accountability.\n",
      "\n",
      "In summary, the ideal society would be built on participatory governance, competence-based leadership, scientific integration, transparency, equity, and adaptability. These foundational principles would not only guide the society's operation but would serve as a bedrock for resolving the inherent ethical dilemmas of each governance element, fostering a balanced and just society.\n",
      "\n",
      "# Response from competitor 2\n",
      "\n",
      "Designing an ideal society with a governance structure that combines the principles of democracy, meritocracy, and technocracy presents several challenges. However, by understanding the core values and implications of each governance type, we can create a framework for an innovative society that strives to balance individual freedoms, collective well-being, and technological advancements.\n",
      "\n",
      "Foundational Principles:\n",
      "\n",
      "1. **Participatory Democracy**: Inclusive participation is essential for creating a citizen-led government where the voices of citizens are actively engaged in decision-making processes.\n",
      "2. **Meritocratic Hiring Process**: A fair, unbiased selection method would be implemented to recruit highly skilled technocrats and policymakers who prioritize the common good over personal interests.\n",
      "3. **Technocratic Expertise**: Technicians with expertise in evidence-based policy analysis and development would work alongside elected officials, ensuring informed decision-making is prioritized.\n",
      "4. **Personal Responsibility and Accountability**: Citizens would have a vested interest in taking personal responsibility for their actions, and policymakers would be held accountable for the outcomes of their decisions.\n",
      "\n",
      "Governance Structure:\n",
      "\n",
      "1.  **Elected Legislative Assembly**: A fair, competitive election system would select representatives who embody democratic values. The number of seats allocated to each seat type and how votes are counted may vary depending on population.\n",
      "2.  **Technocratic Expert Advisory Panel**: Established by the Executive or Senate Board, this commission comprises impartial experts in public policy, technology, and economics.\n",
      "3.  **Governor-Representative Role**: Each constituency governor will have executive authority to act as a mediator and support their constituents directly.\n",
      "\n",
      "Addressing Ethical Dilemmas:\n",
      "\n",
      "1.  Ensuring representation from underrepresented groups might be difficult.\n",
      "2.  Balancing individual freedoms with the collective good.\n",
      "3.  Making informed decisions about policies that impact society's sustainability.\n",
      "4.  Addressing privacy concerns related to surveillance technologies.\n",
      "\n",
      "Ethics Frameworks would be necessary.\n",
      "\n",
      "For instance we may use:\n",
      "\n",
      "1.  **Utilitarianism**: This perspective maximizes overall happiness or well-being.\n",
      "2.  **Deontology**: Prioritize moral rules and duties, such as protecting human rights and life expectancy.\n",
      "3.  **Virtue Ethics**: This approach emphasizes the development of desirable character traits like empathy and justice in leaders.\n",
      "\n",
      "Implementation Challenges\n",
      "\n",
      "The following factors may influence the successful implementation of this vision:\n",
      "\n",
      "*   Overcoming historical narratives tied to existing governance models: \n",
      "*   Rebuilding trust among citizens after being perceived as oppressive or ineffective.\n",
      "    Incentivizing participation and encouraging civic engagement\n",
      "*   Balancing individual freedoms with group identity needs\n",
      "*   Preventing corruption in the technocratic process\n",
      "\n",
      "In conclusion, designing an ideal society with a governance structure based on democracy, meritocracy, and technocracy will require ongoing evaluation and refinement. This would necessitate open communication among all stakeholders, collaborative decision-making processes, and innovative strategies to address existing challenges. The ethical frameworks used may play a substantial role in facilitating societal harmony.\n",
      "\n",
      "\n",
      "\n",
      "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\n"
     ]
    }
   ],
   "source": [
    "print(judge)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge_messages = [{\"role\": \"user\", \"content\": judge}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"results\": [\"1\", \"2\"]}\n"
     ]
    }
   ],
   "source": [
    "# Judgement time!\n",
    "\n",
    "ollama = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
    "model_name = \"llama3.2\"\n",
    "\n",
    "response = ollama.chat.completions.create(model=model_name, messages=judge_messages)\n",
    "results = response.choices[0].message.content\n",
    "print(results)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rank 1: gpt-4o-mini\n",
      "Rank 2: llama3.2\n"
     ]
    }
   ],
   "source": [
    "# OK let's turn this into results!\n",
    "\n",
    "results_dict = json.loads(results)\n",
    "ranks = results_dict[\"results\"]\n",
    "for index, result in enumerate(ranks):\n",
    "    competitor = competitors[int(result)-1]\n",
    "    print(f\"Rank {index+1}: {competitor}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/exercise.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Exercise</h2>\n",
    "            <span style=\"color:#ff7800;\">Which pattern(s) did this use? Try updating this to add another Agentic design pattern.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/business.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Commercial implications</h2>\n",
    "            <span style=\"color:#00bfff;\">These kinds of patterns - to send a task to multiple models, and evaluate results,\n",
    "            are common where you need to improve the quality of your LLM response. This approach can be universally applied\n",
    "            to business projects where accuracy is critical.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}