npm package: @pot-sdk2/langchain
Description: LangChain integration for ThoughtProof Protocol — verify AI outputs in any chain or agent
Author: ThoughtProof
Latest version: 0.1.0

# @pot-sdk2/langchain

LangChain integration for the [ThoughtProof Protocol](https://thoughtproof.ai). Verify AI outputs — in any agent, chain, or retriever — using multi-model consensus verification.

## Overview

Three drop-in components:

| Component | What it does |
|---|---|
| `ThoughtProofVerifyTool` | A `StructuredTool` agents can call to fact-check any claim |
| `ThoughtProofCallbackHandler` | Automatically verifies every LLM output in a chain |
| `ThoughtProofRetrieverWrapper` | Filters retrieved documents by verification confidence |

## Install

```bash
npm install @pot-sdk2/langchain
```

Peer dependencies (install separately):

```bash
npm install pot-sdk @langchain/core
```

## Quick Start

### 1. ThoughtProofVerifyTool — Agent fact-checker

Give any LangChain agent the ability to verify claims on demand.

```typescript
import { ThoughtProofVerifyTool } from '@pot-sdk2/langchain';
import { createToolCallingAgent, AgentExecutor } from 'langchain/agents';
import { ChatAnthropic } from '@langchain/anthropic';
import { pull } from 'langchain/hub';

// Set API keys in env:
//   ANTHROPIC_API_KEY, XAI_API_KEY, DEEPSEEK_API_KEY

const verifyTool = new ThoughtProofVerifyTool({
  models: [
    'anthropic/claude-sonnet-4-5',
    'xai/grok-4-1-fast',
    'deepseek/deepseek-chat',
  ],
  mode: 'adversarial',   // adversarial | resistant | balanced | calibrative
  threshold: 0.7,        // minimum confidence to pass
  domain: 'general',     // optional: medical | legal | financial | code | creative
});

const llm = new ChatAnthropic({ model: 'claude-sonnet-4-5' });
const prompt = await pull('hwchase17/openai-tools-agent');

const agent = createToolCallingAgent({ llm, tools: [verifyTool], prompt });
const executor = new AgentExecutor({ agent, tools: [verifyTool] });

const result = await executor.invoke({
  input: 'Verify this claim: The Great Wall of China is visible from space.',
});

console.log(result.output);
// Agent calls verify_claim → UNCERTAIN (60.3%) — not supported by evidence
```

**Tool schema** — the agent calls it with:
```json
{ "claim": "The claim text to verify" }
```

**Tool output** — a human-readable string with:
```
Verdict: VERIFIED
Confidence: 87.4%
Threshold: 70.0% — PASSED
Synthesis: Multiple sources confirm that water boils at 100°C at sea level...
```

---

### 2. ThoughtProofCallbackHandler — Automatic chain verification

Attach to any chain to verify every LLM output automatically. Verification happens in the background — the chain output is not blocked or modified.

```typescript
import { ThoughtProofCallbackHandler } from '@pot-sdk2/langchain';
import { ChatAnthropic } from '@langchain/anthropic';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { StringOutputParser } from '@langchain/core/output_parsers';

const handler = new ThoughtProofCallbackHandler({
  models: ['anthropic/claude-sonnet-4-5', 'xai/grok-4-1-fast'],
  mode: 'calibrative',
  threshold: 0.6,
  onVerified: (result, text) => {
    console.log(`✓ Output verified (${(result.confidence * 100).toFixed(0)}%):`, text.slice(0, 60));
  },
  onFailed: (result, text) => {
    console.warn(`✗ Verification failed (${(result.confidence * 100).toFixed(0)}%, ${result.verdict}):`, text.slice(0, 60));
  },
});

const llm = new ChatAnthropic({ model: 'claude-sonnet-4-5' });
const prompt = ChatPromptTemplate.fromMessages([
  ['human', '{question}'],
]);
const chain = prompt.pipe(llm).pipe(new StringOutputParser());

const answer = await chain.invoke(
  { question: 'What is the boiling point of water?' },
  { callbacks: [handler] },
);

console.log('Answer:', answer);

// Access all accumulated results:
console.log('All results:', handler.results);
```

**How it works:**
1. `handleLLMStart` captures the prompt as claim context
2. `handleLLMEnd` verifies each generated text against the captured prompt
3. `onVerified` / `onFailed` fire asynchronously with the result
4. Verification errors are logged and do not break the chain

---

### 3. ThoughtProofRetrieverWrapper — Verified RAG

Wrap any retriever to filter out low-confidence documents before they reach the LLM context. Passing documents are annotated with `tp_verdict`, `tp_confidence`, and `tp_flags` in their metadata.

```typescript
import { ThoughtProofRetrieverWrapper } from '@pot-sdk2/langchain';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAIEmbeddings } from '@langchain/openai';

const vectorstore = await MemoryVectorStore.fromTexts(
  ['Paris is the capital of France.', 'The Moon is made of cheese.'],
  [{}, {}],
  new OpenAIEmbeddings(),
);

const baseRetriever = vectorstore.asRetriever({ k: 5 });

const verifiedRetriever = new ThoughtProofRetrieverWrapper({
  retriever: baseRetriever,
  models: ['anthropic/claude-sonnet-4-5', 'deepseek/deepseek-chat'],
  minConfidence: 0.6,    // documents below this are filtered out
  mode: 'resistant',
});

// Only verified documents reach the LLM
const docs = await verifiedRetriever.getRelevantDocuments('What is the capital of France?');

console.log(docs.length); // 1 — the Moon/cheese document was filtered
console.log(docs[0].metadata);
// {
//   tp_verdict: 'VERIFIED',
//   tp_confidence: 0.91,
//   tp_flags: [],
// }
```

---

## Configuration Reference

### Model Specs

Models can be specified as `"provider/model"` strings or full `ProviderConfig` objects:

```typescript
// String format — API key from environment variable
models: ['anthropic/claude-sonnet-4-5']
// → reads process.env.ANTHROPIC_API_KEY

// Full config — explicit API key
models: [{ name: 'anthropic', model: 'claude-opus-4-6', apiKey: 'sk-...' }]
```

**Supported providers and their env vars:**

| Provider string | Environment variable |
|---|---|
| `anthropic` | `ANTHROPIC_API_KEY` |
| `openai` | `OPENAI_API_KEY` |
| `xai` | `XAI_API_KEY` |
| `deepseek` | `DEEPSEEK_API_KEY` |
| `google` | `GOOGLE_API_KEY` |
| `moonshot` | `MOONSHOT_API_KEY` |
| `mistral` | `MISTRAL_API_KEY` |
| `cohere` | `COHERE_API_KEY` |
| `custom` | `CUSTOM_API_KEY` (auto-derived) |

### Critic Modes (`mode`)

| Mode | Behavior |
|---|---|
| `adversarial` | Red-team mode — find every flaw. Highest dissent. |
| `resistant` | Require evidence for each objection. Fewer false positives. |
| `balanced` | Adversarial on facts, resistant on logic. Default in pot-sdk. |
| `calibrative` | Re-score confidence without generating new objections. |

### ThoughtProofVerifyToolOptions

```typescript
{
  models: ModelSpec[];          // required
  mode?: CriticMode;            // default: pot-sdk default
  threshold?: number;           // default: 0.7
  domain?: DomainProfile;       // 'medical' | 'legal' | 'financial' | 'code' | 'creative' | 'general'
  verificationMode?: 'basic' | 'standard'; // default: 'basic'
}
```

### ThoughtProofCallbackHandlerOptions

```typescript
{
  models: ModelSpec[];          // required
  mode?: CriticMode;
  threshold?: number;           // default: 0.6
  domain?: DomainProfile;
  onVerified?: (result: VerificationResult, text: string) => void;
  onFailed?: (result: VerificationResult, text: string) => void;
}
```

### ThoughtProofRetrieverWrapperOptions

```typescript
{
  retriever: BaseRetriever;     // required — the underlying retriever
  models: ModelSpec[];          // required
  mode?: CriticMode;
  minConfidence?: number;       // default: 0.6
  domain?: DomainProfile;
}
```

---

## TypeScript

All types are exported:

```typescript
import type {
  ModelSpec,
  ThoughtProofVerifyToolOptions,
  ThoughtProofCallbackHandlerOptions,
  ThoughtProofRetrieverWrapperOptions,
  VerificationResult,
  CriticMode,
  DomainProfile,
  ProviderConfig,
} from '@pot-sdk2/langchain';
```

## ESM + CJS

The package ships both ESM and CJS builds:

```js
// ESM
import { ThoughtProofVerifyTool } from '@pot-sdk2/langchain';

// CJS
const { ThoughtProofVerifyTool } = require('@pot-sdk2/langchain');
```

## License

MIT