②watsonx.ai/.governaceによるRAGの検証

# PROJECTの作成＆PROJECT_IDの取得

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/f4c010d4-0157-41a6-afc4-d5088f52e52c.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/ad04e599-5262-4d5a-af5d-916da6da9246.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/225597dc-d6ae-41bd-87c5-049fe11dc79d.png)

# PROMPT TEMPLATEをPROJECTに登録

```python:detached_prompt_template.py
# %%
# %%
from dotenv import load_dotenv

load_dotenv()

# %%
import os
from ibm_aigov_facts_client import AIGovFactsClient
from ibm_aigov_facts_client import CloudPakforDataConfig

service_url = os.environ.get("CPD_URL")
username = os.environ.get("CPD_USERNAME")
api_key = os.environ.get("CPD_API_KEY")
creds=CloudPakforDataConfig(
    service_url=service_url,
    username=username,
    api_key=api_key
)

project_id = os.environ.get("PROJECT_ID")
facts_client = AIGovFactsClient(
    cloud_pak_for_data_configs=creds,
    container_id=project_id,
    container_type="project",
    disable_tracing=True
)

# %%
from ibm_aigov_facts_client import DetachedPromptTemplate, PromptTemplate

tmp = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME")
detached_information = DetachedPromptTemplate(
    prompt_id="detached-aoai-prompt",
    model_id=f"azure/{tmp}",
    model_provider="Azure OpenAI",
    model_name="GPT-3.5-turbo",
    model_url=os.environ.get("AZURE_OPENAI_ENDPOINT"),
    prompt_url="prompt_url",
    prompt_additional_info={"model_owner": "Microsoft", "model_version": "gpt-3.5-turbo-1106"}
)

# %%
template = """# System:
Answer the question based on the given context.

# Context:
{context}

# Question:
{question}

# Answer:
"""

# %%
prompt_variables = {"context": "", "question": ""}
input = template
input_prefix = ""
output_prefix = ""

prompt_template = PromptTemplate(
    input=input,
    prompt_variables=prompt_variables,
    input_prefix=input_prefix,
    output_prefix=output_prefix,
)

# %%
name = "RAG based on Azure OpenAI GPT-3.5-turbo"
description = name
tmp = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME")
model_id = f"azure/{tmp}"
task_id = "retrieval_augmented_generation"

response = facts_client.assets.create_detached_prompt(
    model_id=model_id,
    task_id=task_id,
    name=name,
    description=description,
    prompt_details=prompt_template,
    detached_information=detached_information)

# %%
response

# %%
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/fb3d7422-8e98-49ec-a573-a5d5ca815bdf.png)

# 検証データの作成
context、question、reference_output（正解）と生成された回答（generated_output）の4つが必要。

```python:generated_output.py
# %%
from dotenv import load_dotenv
import os

load_dotenv()

# %%
import pandas as pd

filepath = "data.csv"
df = pd.read_csv(filepath_or_buffer=filepath)
df

# %%
from rag import llm, prompt_template

# %%
generated_output = []
for index, row in df.iterrows():
    print(index)
    context = row.context
    question = row.question
    prompt = prompt_template.format(context=context, question=question)

    try:
        response = llm.invoke(input=prompt)
        generated_output.append(response.content)
    except Exception as e:
        generated_output.append("")

# %%
generated_output

# %%
df['generated_output'] = generated_output
df

# %%
import csv

df.iloc[:10].to_csv("test.csv", index=False, quoting=csv.QUOTE_ALL)

# %%
```

# 検証

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/f2b19714-592c-470b-9317-a9f548f836b1.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/904062ec-e660-498a-b311-7fa4e4859218.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/01d1aa4b-54e3-40c0-8857-3ac2a2ab95ea.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/c1c9cd6e-9237-4fed-babe-d8f09355141b.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/e194dfa1-55b5-42fa-a154-d57dc67ca95b.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/426b3aef-f7e2-47d4-bcaa-bfd2da6c811a.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/c98f68fe-b857-4e6f-b6f5-a67ced0aa3be.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/ac142ef7-97d6-4008-9a17-0787ee740de6.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/790b505f-5939-491b-9f8c-721b3954cbab.png)

# 結果
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/6f0bfcd6-29bf-4a67-a6de-1f6c87b0ef70.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/75531ada-9962-4181-b70c-5e2ab6617797.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/7e4ab4f5-855c-44e4-8ae9-012a8efd8a35.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/cd98929c-c472-40fc-9c6a-0b8ab4d29d4d.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/326d6707-998b-4d47-946f-0ab22f2c229a.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/2dcf69c7-afcc-48b4-9ac1-4808ef21b1ac.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3756172/6b5c9239-f541-4bc3-a736-a6dd7e1fffc4.png)