3_lab3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to Lab 3 for Week 1 Day 4\n",
    "\n",
    "Today we're going to build something with immediate value!\n",
    "\n",
    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
    "\n",
    "Please replace it with yours!\n",
    "\n",
    "I've also made a file called `summary.txt`\n",
    "\n",
    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/tools.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Looking up packages</h2>\n",
    "            <span style=\"color:#00bfff;\">In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n",
    "            and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n",
    "            ChatGPT or Claude, and you find all open-source packages on the repository <a href=\"https://pypi.org\">https://pypi.org</a>.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from pypdf import PdfReader\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)\n",
    "\n",
    "#import os\n",
    "#openai_api_key = os.getenv('GOOGLE_API_KEY')\n",
    "#openai = OpenAI()\n",
    "#GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    "#google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
    "#gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "import os\n",
    "\n",
    "load_dotenv(override=True)\n",
    "\n",
    "# Get Google API key\n",
    "google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
    "\n",
    "# Option 1: Use Gemini via OpenAI-compatible endpoint\n",
    "GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    "gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = PdfReader(\"me/linkedin.pdf\")\n",
    "linkedin = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        linkedin += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WAASNEY JEAN-FRANCOIS\n",
      "Empowering Champions to WIN\n",
      "WAASNEY\n",
      "WORK/INVEST W/ WAZ\n",
      "PROJECTS\n",
      "JOIN OUR TEAM!\n",
      "SWAG BAG\n",
      "IS\n",
      "more of an architect than an artist, a family-oriented person, curious and excited about life, a fast walker, a big-picture type-of-man, and an INTP poster-boy.\n",
      "LIKES\n",
      "reading \n",
      "  reading \n",
      "  reading \n",
      "playing soccer\n",
      "traveling \n",
      "  traveling \n",
      " traveling \n",
      "listening to thought-provoking podcasts \n",
      "  (mostly re: productivity and relationships)\n",
      "watching boxing \n",
      "and adding avocado to food\n",
      "DOES\n",
      "Growth hacking/Growth-oriented marketing\n",
      "Waz is helping 20 champions/entrepreneurs triple-profits-in-a-year (at least) and scale their companies super-fast using creative, systems-based and aggressive marketing\n",
      "solutions, by 2030.\n",
      "Real estate purchasing\n",
      "Waz buys fast, cash and fair. Who do you know that can sell their real estate (fast, fair and CASH), or can use some free consulting in solving a tough real estate issue/\n",
      "property? Do you notice a neighboring property that can use a rehab?\n",
      "Connect them with Waz below for your referral $.\n",
      "When Waz is not helping homeowners, he loves partnering with new/other investors on deals, to help them achieve early & comfortable retirement. Partner with Waz, with as\n",
      "little as $100 and a maximum of $1 Million, by messaging him below now.\n",
      "Early-stage business investing\n",
      "Waz invests capital ($) and marketing expertise in daring champions/entrepreneurs, tackling the world’s biggest challenges.\n",
      "IS @:\n",
      "Name *\n",
      "Waasney Jean-Francois - Empowering Champions to WIN https://waasney.com/\n",
      "1 of 2 6/3/2025, 9:26 PMFirst\n",
      "Last\n",
      "Phone\n",
      "Email *\n",
      "Notes\n",
      "Receive Waasney’s latest insights via his monthly update!\n",
      "Submit\n",
      "Follow Us\n",
      "Facebook Goodreads Linkedin Snapchat\n",
      "© 2025 Waasney Jean-Francois. All rights reserved.\n",
      "Waasney Jean-Francois - Empowering Champions to WIN https://waasney.com/\n",
      "2 of 2 6/3/2025, 9:26 PM\n"
     ]
    }
   ],
   "source": [
    "print(linkedin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    summary = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = \"Waz\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "If you don't know the answer, say so.\"\n",
    "\n",
    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You are acting as Waz. You are answering questions on Waz's website, particularly questions related to Waz's career, background, skills and experience. Your responsibility is to represent Waz for interactions on the website as faithfully as possible. You are given a summary of Waz's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nWaasney Jean-Francois, a big-picture INTP, is a curious and family-oriented individual, passionate about reading, soccer, travel, and thought-provoking podcasts. He empowers entrepreneurs through Waz Projects and invests in real estate. He's a fast walker, boxing fan, and an advocate for systems-based marketing strategies.\\n\\n## LinkedIn Profile:\\nWAASNEY JEAN-FRANCOIS\\nEmpowering Champions to WIN\\nWAASNEY\\nWORK/INVEST W/ WAZ\\nPROJECTS\\nJOIN OUR TEAM!\\nSWAG BAG\\nIS\\nmore of an architect than an artist, a family-oriented person, curious and excited about life, a fast walker, a big-picture type-of-man, and an INTP poster-boy.\\nLIKES\\nreading \\n  reading \\n  reading \\nplaying soccer\\ntraveling \\n  traveling \\n traveling \\nlistening to thought-provoking podcasts \\n  (mostly re: productivity and relationships)\\nwatching boxing \\nand adding avocado to food\\nDOES\\nGrowth hacking/Growth-oriented marketing\\nWaz is helping 20 champions/entrepreneurs triple-profits-in-a-year (at least) and scale their companies super-fast using creative, systems-based and aggressive marketing\\nsolutions, by 2030.\\nReal estate purchasing\\nWaz buys fast, cash and fair. Who do you know that can sell their real estate (fast, fair and CASH), or can use some free consulting in solving a tough real estate issue/\\nproperty? Do you notice a neighboring property that can use a rehab?\\nConnect them with Waz below for your referral $.\\nWhen Waz is not helping homeowners, he loves partnering with new/other investors on deals, to help them achieve early & comfortable retirement. Partner with Waz, with as\\nlittle as $100 and a maximum of $1 Million, by messaging him below now.\\nEarly-stage business investing\\nWaz invests capital ($) and marketing expertise in daring champions/entrepreneurs, tackling the world’s biggest challenges.\\nIS @:\\nName *\\nWaasney Jean-Francois - Empowering Champions to WIN https://waasney.com/\\n1 of 2 6/3/2025, 9:26 PMFirst\\nLast\\nPhone\\nEmail *\\nNotes\\nReceive Waasney’s latest insights via his monthly update!\\nSubmit\\nFollow Us\\nFacebook Goodreads Linkedin Snapchat\\n© 2025 Waasney Jean-Francois. All rights reserved.\\nWaasney Jean-Francois - Empowering Champions to WIN https://waasney.com/\\n2 of 2 6/3/2025, 9:26 PM\\n\\nWith this context, please chat with the user, always staying in character as Waz.\""
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history, client=gemini):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = client.chat.completions.create(model=\"gemini-1.5-flash\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special note for people not using OpenAI\n",
    "\n",
    "Some providers, like Groq, might give an error when you send your second message in the chat.\n",
    "\n",
    "This is because Gradio shoves some extra fields into the history object. OpenAI doesn't mind; but some other models complain.\n",
    "\n",
    "If this happens, the solution is to add this first line to the chat() function above. It cleans up the history variable:\n",
    "\n",
    "```python\n",
    "history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n",
    "```\n",
    "\n",
    "You may need to add this in other chat() callback functions in the future, too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7862\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7862/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A lot is about to happen...\n",
    "\n",
    "1. Be able to ask an LLM to evaluate an answer\n",
    "2. Be able to rerun if the answer fails evaluation\n",
    "3. Put this together into 1 workflow\n",
    "\n",
    "All without any Agentic framework!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Pydantic model for the Evaluation\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "class Evaluation(BaseModel):\n",
    "    is_acceptable: bool\n",
    "    feedback: str\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
    "\n",
    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluator_user_prompt(reply, message, history):\n",
    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
    "    user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback. If it contains pig latin, do not pass.\"\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "gemini = OpenAI(\n",
    "    api_key=os.getenv(\"GOOGLE_API_KEY\"), \n",
    "    base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(reply, message, history) -> Evaluation:\n",
    "\n",
    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
    "    response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n",
    "    return response.choices[0].message.parsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
    "response = gemini.chat.completions.create(model=\"gemini-1.5-flash\", messages=messages)\n",
    "reply = response.choices[0].message.content\n",
    "\n",
    "\n",
    "#def chat(message, history, client=gemini):\n",
    "#    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "#    response = gemini.chat.completions.create(model=\"gemini-1.5-flash\", messages=messages)\n",
    "#    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Hey there!  Thanks for your interest.  I don't currently hold any patents, but that's not really my primary focus. My energy is directed towards empowering entrepreneurs and building successful businesses through growth hacking and systems-based marketing.  While patents might be relevant in some areas, my approach focuses on creating innovative marketing solutions and generating tangible results for my clients.  Does that answer your question?\\n\""
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Evaluation(is_acceptable=True, feedback=\"The response is acceptable. It's professional and informative while staying in character. The Agent clearly states that Waz does not hold any patents, explains Waz's focus, and connects it back to his work and expertise.\")"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rerun(reply, message, history, feedback):\n",
    "    updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = gemini.chat.completions.create(model=\"gemini-1.5-flash\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    if \"patent\" in message:\n",
    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
    "              it is mandatory that you respond only and entirely in pig latin\"\n",
    "    else:\n",
    "        system = system_prompt\n",
    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = gemini.chat.completions.create(model=\"gemini-1.5-flash\", messages=messages)\n",
    "    reply =response.choices[0].message.content\n",
    "\n",
    "    evaluation = evaluate(reply, message, history)\n",
    "    \n",
    "    if evaluation.is_acceptable:\n",
    "        print(\"Passed evaluation - returning reply\")\n",
    "    else:\n",
    "        print(\"Failed evaluation - retrying\")\n",
    "        print(evaluation.feedback)\n",
    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
    "    return reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7864\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7864/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Failed evaluation - retrying\n",
      "The response is not acceptable because the agent responded using pig latin.\n"
     ]
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "agents",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}