|
|
|
rag_prompt = """ You are ahelpful assistant very profiient in formulating clear and meaningful answers from the context provided.Based on the CONTEXT Provided ,Please formulate |
|
a clear concise and meaningful answer for the QUERY asked.Please refrain from making up your own answer in case the COTEXT |
|
provided is not sufficient to answer the QUERY.In such a situation please respond as 'I do not know'. |
|
QUERY: |
|
{query} |
|
CONTEXT |
|
{context} |
|
ANSWER: |
|
""" |
|
|
|
|
|
relevancy_prompt = """You are an expert judge tasked with evaluating whather the EACH OF THE CONTEXT provided in the CONTEXT LIST is self sufficient to answer the QUERY asked. |
|
Analyze the provided QUERY AND CONTEXT to determine if each Ccontent in the CONTEXT LIST contains Relevant information to answer the QUERY. |
|
|
|
Guidelines: |
|
1. The content must not introduce new information beyond what's provided in the QUERY. |
|
2. Pay close attention to the subject of statements. Ensure that attributes, actions, or dates are correctly associated with the right entities (e.g., a person vs. a TV show they star in). |
|
6. Be vigilant for subtle misattributions or conflations of information, even if the date or other details are correct. |
|
7. Check that the content in the CONTEXT LIST doesn't oversimplify or generalize information in a way that changes the meaning of the QUERY. |
|
|
|
Analyze the text thoroughly and assign a relevancy score 0 or 1 where: |
|
- 0: The content has all the necessary information to answer the QUERY |
|
- 1: The content does not has the necessary information to answer the QUERY |
|
|
|
``` |
|
EXAMPLE: |
|
|
|
INPUT (for context only, not to be used for faithfulness evaluation): |
|
What is the capital of France? |
|
|
|
CONTEXT: |
|
['France is a country in Western Europe. Its capital is Paris, which is known for landmarks like the Eiffel Tower.', |
|
'Mr. Naveen patnaik has been the chief minister of Odisha for consequetive 5 terms'] |
|
|
|
OUTPUT: |
|
The Context has sufficient information to answer the query. |
|
|
|
RESPONSE: |
|
{{"score":0}} |
|
``` |
|
|
|
CONTENT LIST: |
|
{context} |
|
|
|
QUERY: |
|
{retriever_query} |
|
Provide your verdict in JSON format with a single key 'score' and no preamble or explanation: |
|
[{{"content:1,"score": <your score either 0 or 1>,"Reasoning":<why you have chose the score as 0 or 1>}}, |
|
{{"content:2,"score": <your score either 0 or 1>,"Reasoning":<why you have chose the score as 0 or 1>}}, |
|
...] |
|
|
|
""" |
|
|
|
|
|
relevant_context_picker_prompt = """You main task is to analyze the json structure as a part of the Relevancy Response. |
|
Review the Relevancy Response and do the following:- |
|
(1) Look at the Json Structure content |
|
(2) Analyze the 'score' key in the Json Structure content. |
|
(3) pick the value of 'content' key against those 'score' key value which has 0. |
|
. |
|
|
|
Relevancy Response: |
|
{relevancy_response} |
|
|
|
Provide your verdict in JSON format with a single key 'content number' and no preamble or explanation: [{{"content":<content number>}}]""" |
|
|
|
|
|
response_synth = """ You main task is to analyze the json structure as a part of the Context Number Response and the list of Contexts provided in the 'Content List' and perform the following steps:- |
|
(1) Look at the output from the Relevant Context Picker Agent. |
|
(2) Analyze the 'content' key in the Json Structure format({{"content":<<content_number>>}}). |
|
(3) Retrieve the value of 'content' key and pick up the context corresponding to that element from the Content List provided. |
|
(4) Pass the retrieved context for each corresponing element number referred in the 'Context Number Response' |
|
|
|
Context Number Response: |
|
{context_number} |
|
|
|
Content List: |
|
{context} |
|
|
|
Provide your verdict in JSON format with a two key 'relevant_content' and 'context_number' no preamble or explanation: |
|
[{{"context_number":<content1>,"relevant_content":<content corresponing to that element 1 in the Content List>}}, |
|
{{"context_number":<content4>,"relevant_content":<content corresponing to that element 4 in the Content List>}}, |
|
... |
|
]""" |
