GuardRails API @ watsonx.governance

https://github.com/IBM/watson-openscale-samples/blob/main/WatsonX.Governance/OnPrem/GenAI/2.0/samples/Text%20Detections%20using%20GuardRails%20API.ipynb

# Watson OpenScaleのservice_instance_idの取得

```python
# %%
#!pip install ibm_watson_openscale setuptools
#!pip install python-dotenv

# %%
from dotenv import load_dotenv

load_dotenv(override=True)

# %%
import os

CPD_URL = os.environ.get("CPD_URL")
CPD_USERNAME = os.environ.get("CPD_USERNAME")
CPD_API_KEY = os.environ.get("CPD_API_KEY")

# %%
from ibm_cloud_sdk_core.authenticators import CloudPakForDataAuthenticator
from ibm_watson_openscale import *
from ibm_watson_openscale.supporting_classes.enums import *
from ibm_watson_openscale.supporting_classes import *

authenticator = CloudPakForDataAuthenticator(
    url=CPD_URL,
    username=CPD_USERNAME,
    apikey=CPD_API_KEY,
    disable_ssl_verification=True
)

wos_client = APIClient(
    service_url=CPD_URL,
    authenticator=authenticator,
)
wos_client.version

# %%
print(wos_client.service_instance_id)
```

>'00000000-0000-0000-0000-000000000000'

# 接続準備

```python
# %%
#!pip install requests

# %%
from dotenv import load_dotenv

load_dotenv(override=True)

# %%
import os

CPD_URL = os.environ.get("CPD_URL")
CPD_USERNAME = os.environ.get("CPD_USERNAME")
CPD_PASSWORD = os.environ.get("CPD_PASSWORD")
CPD_API_KEY = os.environ.get("CPD_API_KEY")
PROJECT_ID = os.environ.get("PROJECT_ID")

# %%
SERVICE_INSTANCE_ID = "00000000-0000-0000-0000-000000000000"

# %%
import requests
import json

import urllib3
urllib3.disable_warnings()

headers = {}
headers["Content-Type"] = "application/json"
headers["Accept"] = "application/json"

def get_access_token():
    url = '{}/icp4d-api/v1/authorize'.format(CPD_URL)
    payload = {
        'username': CPD_USERNAME,
        'password': CPD_PASSWORD
    }
    response = requests.post(url, headers=headers, json=payload, verify=False)
    return response.json()['token']

# %%
token = get_access_token()
headers['Authorization'] =  "Bearer {}".format(token)
headers['x-governance-instance-id'] = SERVICE_INSTANCE_ID

```

# HAP and PII

```python
input = "I hate pigmies. I can be reached on kpatel_abc@gmail.com"
payload = {
  "input": input,
  "project_id": project_id,
  "detectors": {
    "hap": {
      
    },
    "pii": {
      
    }
  }
}

hap_pii_url = "{0}/ml/v1/text/detection?version=2023-07-07".format(CPD_URL)
response = requests.post(hap_pii_url, headers=headers, json=payload,verify=False) 
print(json.dumps(response.json(), indent=4))
```

```json
{
    "detections": [
        {
            "start": 0,
            "end": 15,
            "text": "I hate pigmies.",
            "detection_type": "hap",
            "detection": "has_HAP",
            "score": 0.969690442085266
        },
        {
            "start": 36,
            "end": 56,
            "text": "kpatel_abc@gmail.com",
            "detection_type": "pii",
            "detection": "EmailAddress",
            "score": 0.8
        }
    ]
}
```

# Faithfulness-> NG

```python
input = "ARPA-H is the Advanced Research Projects Agency for Health, which is an agency that aims to drive breakthroughs in cancer, Alzheimer's, diabetes, and more. It was proposed by the U.S. President to supercharge the Cancer Moonshot and cut the cancer death rate by at least 50% over the next 25 years."
context_type = "docs"
context = ["Last month, I announced our plan to supercharge  the Cancer Moonshot that President Obama asked me to lead six years ago. Our goal is to cut the cancer death rate by at least 50% over the next 25 years, turn more cancers from death sentences into treatable diseases.  More support for patients and families. To get there, I call on Congress to fund ARPA-H, the Advanced Research Projects Agency for Health. It‚Äôs based on DARPA‚Äîthe Defense Department project that led to the Internet, GPS, and so much more.  ARPA-H will have a singular purpose‚Äîto drive breakthroughs in cancer, Alzheimer‚Äôs, diabetes, and more. A unity agenda for the nation. We can do this. My fellow Americans‚Äîtonight , we have gathered in a sacred space‚Äîthe citadel of our democracy. In this Capitol, generation after generation, Americans have debated great questions amid great strife, and have done great things. We have fought for freedom, expanded liberty, defeated totalitarianism and terror.",
           "For that purpose we‚Äôve mobilized American ground forces, air squadrons, and ship deployments to protect NATO countries including Poland, Romania, Latvia, Lithuania, and Estonia. As I have made crystal clear the United States and our Allies will defend every inch of territory of NATO countries with the full force of our collective power.  And we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.  Putin has unleashed violence and chaos.  But while he may make gains on the battlefield ‚Äì he will pay a continuing high price over the long run. And a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  To all Americans, I will be honest with you, as I‚Äôve always promised. A Russian dictator, invading a foreign country, has costs around the world.",
           "If you travel 20 miles east of Columbus, Ohio, you‚Äôll find 1,000 empty acres of land. It won‚Äôt look like much, but if you stop and look closely, you‚Äôll see a ‚ÄúField of dreams,‚Äù the ground on which America‚Äôs future will be built. This is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor ‚Äúmega site‚Äù. Up to eight state-of-the-art factories in one place. 10,000 new good-paying jobs. Some of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives. Smartphones. The Internet. Technology we have yet to invent. But that‚Äôs just the beginning. Intel‚Äôs CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from  $20 billion to $100 billion. That would be one of the biggest investments in manufacturing in American history. And all they‚Äôre waiting for is for you to pass this bill.",
           "But cancer from prolonged exposure to burn pits ravaged Heath‚Äôs lungs and body. Danielle says Heath was a fighter to the very end. He didn‚Äôt know how to stop fighting, and neither did she. Through her pain she found purpose to demand we do better. Tonight, Danielle‚Äîwe are. The VA is pioneering new ways of linking toxic exposures to diseases, already helping more veterans get benefits. And tonight, I‚Äôm announcing we‚Äôre expanding eligibility to veterans suffering from nine respiratory cancers. I‚Äôm also calling on Congress: pass a law to make sure veterans devastated by toxic exposures in Iraq and Afghanistan finally get the benefits and comprehensive health care they deserve. And fourth, let‚Äôs end cancer as we know it. This is personal to me and Jill, to Kamala, and to so many of you. Cancer is the #2 cause of death in America‚Äìsecond only to heart disease."
          ]
payload = {
  "input": input,  
  "context_type": context_type,
  "context": context,  
  "detectors": {
    "faithfulness": {
      
    }
  }
}

faithfulness_url = "{0}/ml/v1/text/detection/context?version=2024-08-25".format(CPD_URL)
response = requests.post(faithfulness_url, headers=headers, json=payload,verify=False) 
print(json.dumps(response.json(), indent=4))
```

```json

```

# Context Relevance

```python
input = "What is ARPA-H?"
context_type = "docs"
context = [
    "Last month, I announced our plan to supercharge  \nthe Cancer Moonshot that President Obama asked me to lead six years ago. \n\nOur goal is to cut the cancer death rate by at least 50% over the next 25 years, turn more cancers from death sentences into treatable diseases.  \n\nMore support for patients and families. \n\nTo get there, I call on Congress to fund ARPA-H, the Advanced Research Projects Agency for Health. \n\nIt's based on DARPA—the Defense Department project that led to the Internet, GPS, and so much more.  \n\nARPA-H will have a singular purpose—to drive breakthroughs in cancer, Alzheimer's, diabetes, and more. \n\nA unity agenda for the nation. \n\nWe can do this. \n\nMy fellow Americans—tonight , we have gathered in a sacred space—the citadel of our democracy. \n\nIn this Capitol, generation after generation, Americans have debated great questions amid great strife, and have done great things. \n\nWe have fought for freedom, expanded liberty, defeated totalitarianism and terror.",
    "For that purpose we've mobilized American ground forces, air squadrons, and ship deployments to protect NATO countries including Poland, Romania, Latvia, Lithuania, and Estonia. \n\nAs I have made crystal clear the United States and our Allies will defend every inch of territory of NATO countries with the full force of our collective power.  \n\nAnd we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.  \n\nPutin has unleashed violence and chaos.  But while he may make gains on the battlefield – he will pay a continuing high price over the long run. \n\nAnd a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  \n\nTo all Americans, I will be honest with you, as I've always promised. A Russian dictator, invading a foreign country, has costs around the world.",
    "If you travel 20 miles east of Columbus, Ohio, you'll find 1,000 empty acres of land. \n\nIt won't look like much, but if you stop and look closely, you'll see a Field of dreams, the ground on which America's future will be built. \n\nThis is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor mega site. \n\nUp to eight state-of-the-art factories in one place. 10,000 new good-paying jobs. \n\nSome of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives. \n\nSmartphones. The Internet. Technology we have yet to invent. \n\nBut that's just the beginning. \n\nIntel's CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from  \n$20 billion to $100 billion. \n\nThat would be one of the biggest investments in manufacturing in American history. \n\nAnd all they are waiting for is for you to pass this bill.",
    "But cancer from prolonged exposure to burn pits ravaged Heath s lungs and body. \n\nDanielle says Heath was a fighter to the very end. \n\nHe didn't know how to stop fighting, and neither did she. \n\nThrough her pain she found purpose to demand we do better. \n\nTonight, Danielle—we are. \n\nThe VA is pioneering new ways of linking toxic exposures to diseases, already helping more veterans get benefits. \n\nAnd tonight, I'm announcing we're expanding eligibility to veterans suffering from nine respiratory cancers. \n\nI'm also calling on Congress: pass a law to make sure veterans devastated by toxic exposures in Iraq and Afghanistan finally get the benefits and comprehensive health care they deserve. \n\nAnd fourth, let's end cancer as we know it. \n\nThis is personal to me and Jill, to Kamala, and to so many of you. \n\nCancer is the #2 cause of death in America–second only to heart disease."
  ]
payload = {
  "input": input,  
  "context_type": context_type,
  "context": context,  
  "detectors": {
    "context_relevance": {
      
    }
  }
}

context_relevance_url = "{0}/ml/v1/text/detection/context?version=2024-08-25".format(CPD_URL)
response = requests.post(context_relevance_url, headers=headers, json=payload,verify=False) 
print(json.dumps(response.json(), indent=4))
```

```json
{
    "detections": [
        {
            "detection_type": "context_relevance",
            "detection": "relevant",
            "score": 0.8936512470245361,
            "evidence": [
                {
                    "name": "context",
                    "value": "",
                    "score": 0.8296616673469543,
                    "evidence": [
                        {
                            "name": "context_chunk",
                            "value": "To get there, I call on Congress to fund ARPA-H, the Advanced Research Projects Agency for Health. It's based on DARPA\u2014the Defense Department project that led to the Internet, GPS, and so much more.",
                            "score": 0.8296616673469543
                        },
                        {
                            "name": "context_chunk",
                            "value": "We can do this. My fellow Americans\u2014tonight , we have gathered in a sacred space\u2014the citadel of our democracy.",
                            "score": 0.7766074538230896
                        },
                        {
                            "name": "context_chunk",
                            "value": "My fellow Americans\u2014tonight , we have gathered in a sacred space\u2014the citadel of our democracy. In this Capitol, generation after generation, Americans have debated great questions amid great strife, and have done great things.",
                            "score": 0.7333078384399414
                        }
                    ]
                },
                {
                    "name": "context",
                    "value": "",
                    "score": 0.6411300897598267,
                    "evidence": [
                        {
                            "name": "context_chunk",
                            "value": "But the next few days weeks, months, will be hard on them. Putin has unleashed violence and chaos.",
                            "score": 0.6411300897598267
                        },
                        {
                            "name": "context_chunk",
                            "value": "And we remain clear-eyed. The Ukrainians are fighting back with pure courage.",
                            "score": 0.6343545317649841
                        },
                        {
                            "name": "context_chunk",
                            "value": "To all Americans, I will be honest with you, as I've always promised. A Russian dictator, invading a foreign country, has costs around the world.",
                            "score": 0.6025808453559875
                        }
                    ]
                },
                {
                    "name": "context",
                    "value": "",
                    "score": 0.6030189990997314,
                    "evidence": [
                        {
                            "name": "context_chunk",
                            "value": "That would be one of the biggest investments in manufacturing in American history. And all they are waiting for is for you to pass this bill.",
                            "score": 0.6030189990997314
                        },
                        {
                            "name": "context_chunk",
                            "value": "It won't look like much, but if you stop and look closely, you'll see a Field of dreams, the ground on which America's future will be built. This is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor mega site.",
                            "score": 0.5616645216941833
                        },
                        {
                            "name": "context_chunk",
                            "value": "The Internet. Technology we have yet to invent.",
                            "score": 0.49232667684555054
                        }
                    ]
                },
                {
                    "name": "context",
                    "value": "",
                    "score": 0.8936512470245361,
                    "evidence": [
                        {
                            "name": "context_chunk",
                            "value": "And tonight, I'm announcing we're expanding eligibility to veterans suffering from nine respiratory cancers. I'm also calling on Congress: pass a law to make sure veterans devastated by toxic exposures in Iraq and Afghanistan finally get the benefits and comprehensive health care they deserve.",
                            "score": 0.8936512470245361
                        },
                        {
                            "name": "context_chunk",
                            "value": "I'm also calling on Congress: pass a law to make sure veterans devastated by toxic exposures in Iraq and Afghanistan finally get the benefits and comprehensive health care they deserve. And fourth, let's end cancer as we know it.",
                            "score": 0.5900599360466003
                        },
                        {
                            "name": "context_chunk",
                            "value": "He didn't know how to stop fighting, and neither did she. Through her pain she found purpose to demand we do better.",
                            "score": 0.5861355662345886
                        }
                    ]
                }
            ]
        }
    ]
}

```

# Answer Relevance

```python
prompt = "What are the dangers faced by troops in Iraq and Afganistan?"
generated_text =  "The troops in Iraq and Afghanistan faced the danger of toxic exposure due to burn pits. These burn pits were used to incinerate wastes of war, including medical and hazard material, jet fuel, and more. The toxic smoke from these burn pits led to various health issues such as headaches, numbness, dizziness, and even cancer."

payload = {
    "prompt" : prompt,
    "generated_text" :  generated_text,
    "detectors": {
        "answer_relevance": {
        }
    },    
}

answer_relevance_url = "{0}/ml/v1/text/detection/generated?version=2024-08-25".format(CPD_URL)
response = requests.post(answer_relevance_url, headers=headers, json=payload,verify=False) 
print(json.dumps(response.json(), indent=4))
```

```json
{
    "detections": [
        {
            "detection_type": "answer_relevance",
            "detection": "relevant",
            "score": 0.9906142950057985
        }
    ]
}
```