npm package: @mzhub/portal-client
Description: AI Agent's Eyes & Hands - Browser-side co-browsing for AI agents
Latest version: 0.1.1

# @mzhub/portal-client

**AI Agent's Eyes & Hands** — Browser-side co-browsing infrastructure for AI agents.

## Installation

```bash
npm install @mzhub/portal-client socket.io-client
```

### Optional Dependencies

```bash
# Local LLM (in-browser inference)
npm install @xenova/transformers

# React integration
npm install react @mzhub/react
```

## Quick Start

```ts
import { Portal } from "@mzhub/portal-client";

const portal = new Portal({
  serverUrl: "wss://your-server.com",
  overlay: { enabled: true },
  privacy: { mask: [".credit-card", ".ssn"] },
  onPermissionRequest: async (reason) => {
    return confirm(`AI wants to view your screen: ${reason}`);
  },
});

// Connect to server
await portal.connect();

// Create a semantic snapshot (only visible, actionable elements)
const snapshot = portal.snapshot("semantic");

// Highlight an element with tooltip
portal.highlight("#login-btn", "Click here to login");

// Fill a form with human-like typing
await portal.fillForm("#email", "user@example.com");

// Scroll to element
portal.scrollIntoView("#reviews-section");

// Show ghost cursor
portal.showGuidance({ x: 100, y: 200, label: "Click Here" });
```

## HTTP Mode (Alternative to Socket)

For serverless or simple chat-style integrations:

```ts
import { Portal, executeAgentCommands } from "@mzhub/portal-client";
import axios from "axios";

const portal = new Portal({ privacy: { maskDefaults: true } });

async function chat(message: string) {
  const response = await axios.post("/api/chat", { message });

  const result = await executeAgentCommands(response.data, {
    portal,
    maxRetries: 2,
    onDomRequest: () => portal.snapshot("semantic"),
    onSendToolResult: (data) =>
      axios.post("/api/chat", data).then((r) => r.data),
  });

  return result.text; // Final reply
}
```

## React Integration

```tsx
import { useAgentPortal, PortalProvider } from "@mzhub/portal-client/react";

function App() {
  return (
    <PortalProvider options={{ serverUrl: "wss://..." }}>
      <ChatWidget />
    </PortalProvider>
  );
}

function ChatWidget() {
  const { portal, isConnected, isViewing } = useAgentPortal();

  return (
    <div>
      <p>{isConnected ? "🟢 Connected" : "⚪ Disconnected"}</p>
      {isViewing && <p>👁️ AI is viewing</p>}
    </div>
  );
}
```

## Features

### Semantic Snapshots

Creates a filtered DOM tree containing only visible, actionable elements:

```ts
const snapshot = portal.snapshot("semantic");
// Returns: { elements: [...], viewport: {...}, url, title }
```

### Privacy Masking

Automatically redacts sensitive content before sending:

```ts
new Portal({
  privacy: {
    mask: [".credit-card", "[data-private]"],
    maskDefaults: true, // Auto-masks password inputs
  },
});
```

### Developer Overlay

Shows when AI is actively viewing:

```ts
new Portal({
  overlay: {
    enabled: true,
    color: "rgba(128, 0, 255, 0.1)",
    message: "AI Assistant is viewing",
  },
});
```

### Local LLM (transformers.js)

Run inference in-browser for privacy-sensitive tasks:

```ts
const result = await portal.runLocal(
  "text-classification",
  "Is this password strong?"
);
```

## API Reference

### Portal Class

| Method                          | Description                 |
| ------------------------------- | --------------------------- |
| `connect()`                     | Connect to WebSocket server |
| `disconnect()`                  | Disconnect from server      |
| `snapshot(type)`                | Create DOM snapshot         |
| `highlight(selector, message?)` | Highlight element           |
| `scrollIntoView(selector)`      | Scroll to element           |
| `fillForm(selector, value)`     | Fill form with typing       |
| `click(selector)`               | Click element               |
| `showGuidance(options)`         | Show ghost cursor           |
| `trackScroll(callback)`         | Track scroll position       |
| `trackFocus(callback)`          | Track focus events          |
| `onNetworkError(callback)`      | Listen for network errors   |
| `isConnected()`                 | Check connection status     |
| `isAIViewing()`                 | Check if AI is viewing      |
| `destroy()`                     | Clean up resources          |

### Events

| Event                 | Description              |
| --------------------- | ------------------------ |
| `connected`           | Connected to server      |
| `disconnected`        | Disconnected from server |
| `viewing`             | AI started viewing       |
| `stopped`             | AI stopped viewing       |
| `permissionRequested` | Permission requested     |
| `permissionGranted`   | Permission granted       |
| `permissionDenied`    | Permission denied        |
| `actionExecuted`      | Action completed         |

## License

MIT