Spaces:
Sleeping
Sleeping
File size: 7,948 Bytes
26d66c1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 |
{
"cells": [
{
"cell_type": "markdown",
"id": "f510f96b-d3ff-45f2-bec4-9eff274ccfc3",
"metadata": {},
"source": [
"# Imports"
]
},
{
"cell_type": "raw",
"id": "1ef3f8ca-eb80-4b5a-8934-744e43433a0b",
"metadata": {
"scrolled": true,
"vscode": {
"languageId": "raw"
}
},
"source": [
"!pip install \\\n",
" python-dotenv==1.0.0 \\\n",
" openai==0.28.1 \\\n",
" langchain==0.0.316 \\\n",
" gradio==4.5.0"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "34564aeb-b6c0-42b0-93a4-46156f255d31",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\logis\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"# System related imports\n",
"import os\n",
"from dotenv import load_dotenv, find_dotenv\n",
"\n",
"# OpenAI related imports\n",
"import openai\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.embeddings.openai import OpenAIEmbeddings\n",
"\n",
"# Agent imports\n",
"import logging\n",
"from typing import List\n",
"from pydantic import BaseModel, Field\n",
"from langchain.chains import LLMChain\n",
"from langchain.agents import AgentExecutor\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.prompts import ChatPromptTemplate\n",
"from langchain.prompts import MessagesPlaceholder\n",
"from langchain.memory import ConversationBufferMemory\n",
"from langchain.output_parsers import PydanticOutputParser\n",
"from langchain.schema.runnable import RunnablePassthrough\n",
"from langchain.retrievers.multi_query import MultiQueryRetriever\n",
"from langchain.tools.render import format_tool_to_openai_function\n",
"from langchain.agents.agent_toolkits import create_retriever_tool\n",
"from langchain.agents.format_scratchpad import format_to_openai_functions\n",
"from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser\n",
"\n",
"# Chat UI\n",
"import gradio as gr\n",
"from langchain.schema import AIMessage, HumanMessage"
]
},
{
"cell_type": "markdown",
"id": "1add021b-201f-47ad-b4e3-44ba810d02c2",
"metadata": {},
"source": [
"# Enviromental variables"
]
},
{
"cell_type": "markdown",
"id": "07106937-17ff-4a22-8d27-5c79a8a83591",
"metadata": {},
"source": [
"Set up the APIs keys."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "3270c64b-5168-4085-b3d3-07a6ad1a2902",
"metadata": {},
"outputs": [],
"source": [
"_ = load_dotenv(find_dotenv()) # read local .env file\n",
"\n",
"openai.api_key = os.environ['OPENAI_API_KEY']"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "513e5fdc-c854-40f0-a6ba-c874bf80f288",
"metadata": {},
"outputs": [],
"source": [
"# Output parser will split the LLM result into a list of queries\n",
"class LineList(BaseModel):\n",
" # \"lines\" is the key (attribute name) of the parsed output\n",
" lines: List[str] = Field(description=\"Lines of text\")\n",
"\n",
"\n",
"class LineListOutputParser(PydanticOutputParser):\n",
" def __init__(self) -> None:\n",
" super().__init__(pydantic_object=LineList)\n",
"\n",
" def parse(self, text: str) -> LineList:\n",
" lines = text.strip().split(\"\\n\")\n",
" return LineList(lines=lines)\n",
"\n",
"\n",
"output_parser = LineListOutputParser()\n",
"\n",
"QUERY_PROMPT = PromptTemplate(\n",
" input_variables=[\"essay_text\"],\n",
" template=\"\"\"\n",
" Task Description:\n",
"Your task is to evaluate an essay text according to the International English Language Testing System (IELTS) format. Ensure the essay meets the requirements of an IELTS essay in terms of structure, length, coherence, and language proficiency. Then, grade the essay based on the IELTS scoring system and provide detailed feedback on each criterion.\n",
"\n",
"Essay Text:\n",
"{essay_text}\n",
"\n",
"Instructions to the Language Model:\n",
"\n",
"### Relevance to IELTS Format:\n",
"Check if the essay conforms to the standard IELTS essay format, including introduction, body paragraphs, and conclusion.\n",
"Ensure the essay is of appropriate length (minimum 250 words).\n",
"Evaluate whether the essay addresses the given topic comprehensively and logically.\n",
"\n",
"### Grading Criteria:\n",
"- Task Response (TR): Evaluate how well the essay addresses the task, presents a clear position, and supports it with relevant examples and arguments.\n",
"- Coherence and Cohesion (CC): Assess the organization of ideas, logical progression, and coherence between paragraphs and sentences.\n",
"- Lexical Resource (LR): Examine the range and accuracy of vocabulary used, including the ability to paraphrase and express ideas precisely.\n",
"- Grammatical Range and Accuracy (GRA): Evaluate the variety and accuracy of grammatical structures used, including sentence structure and punctuation.\n",
"\n",
"### Detailed Comments and Advice:\n",
"Provide specific feedback for each criterion, highlighting strengths and areas for improvement.\n",
"Offer advice on how the essay can be enhanced to achieve a higher score in each category.\n",
"Suggest alternative vocabulary, sentence structures, or argumentative strategies where applicable.\n",
"\n",
"### Example Response:\n",
"\"\n",
"Band 7.5\n",
"Task Response: 7\n",
"Coherence and Cohesion: 8\n",
"Lexical Resource: 8\n",
"Grammatical Range and Accuracy: 7\n",
"The test taker presents a clear position at the outset and explores some ideas to support this. An\n",
"alternative position is also considered, but rejected. This is a strong response, but there is rather\n",
"too much emphasis on technology: other aspects of the proposition could also be considered,\n",
"e.g. less physical work and more sedentary work, greater reliance on cars meaning less\n",
"exercise, aging populations in some countries leading to more complex health issues. Ideas are\n",
"organised logically and there is a clear progression throughout the response, with good use of\n",
"cohesive devices and logical paragraphing. The response could perhaps be improved by\n",
"breaking down paragraphs 2 and 3. There is a wide range of vocabulary with good use of less\n",
"common items as well as evidence of higher level features, such as ‘softening’, e.g. ‘They tend\n",
"to’, ‘This appears to be’, and ‘might disagree’. Errors in spelling and word formation are rare.\n",
"There is also a variety of complex structures with frequent error-free sentences, though some\n",
"errors do occur and there is some overuse of rather short sentence forms.\n",
"\"\n",
"\"\"\",\n",
")\n",
"llm = ChatOpenAI(temperature=0)\n",
"\n",
"multi_query_chain = LLMChain(llm=llm, prompt=QUERY_PROMPT, output_parser=output_parser)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
|