npm package: @maderatools/visual-ai
Description: AI-assisted visual testing framework for Playwright with MCP-style tools, automatic bug reporting, and Gemini integration
Author: Maderatools
Latest version: 1.0.0

# 🎭 @maderatools/visual-ai

**AI-assisted visual testing framework for Playwright with automatic bug reporting and Gemini integration.**

[![npm version](https://badge.fury.io/js/@maderatools%2Fvisual-ai.svg)](https://www.npmjs.com/package/@maderatools/visual-ai)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Node.js Version](https://img.shields.io/node/v/@maderatools/visual-ai)](https://nodejs.org)

---

## 🌟 Features

- **🛠️ MCP-Style Tools**: Composable, chainable helpers for robust testing
- **📸 Smart Screenshots**: Automatic capture with AI-ready context metadata
- **🔍 Real Visibility Checks**: Verify elements are ACTUALLY visible, not just in DOM
- **🎯 Console & Network Monitoring**: Capture errors, warnings, and API calls automatically
- **🤖 AI Integration**: Built for Gemini Pro - generates tests on-demand
- **📋 Auto Bug Reports**: Generates structured JSON for Claude CLI to fix bugs
- **⚡ Zero Config**: Works out of the box with sensible defaults
- **🎨 Visual-First**: Optimized for debugging complex UI interactions

---

## 📦 Installation

```bash
npm install --save-dev @maderatools/visual-ai @playwright/test playwright
```

Or with your preferred package manager:

```bash
yarn add -D @maderatools/visual-ai @playwright/test playwright
pnpm add -D @maderatools/visual-ai @playwright/test playwright
```

---

## 🚀 Quick Start

### 1. Create a Test

```javascript
// tests/example.spec.js
import { test, expect } from '@playwright/test';
import {
  VisualHelpers,
  ScreenshotManager,
  ConsoleCapture,
  NetworkMonitor
} from '@maderatools/visual-ai';

test.describe('My Feature', () => {
  let consoleCapture, networkMonitor;

  test.beforeEach(async ({ page }) => {
    // Setup monitoring
    consoleCapture = new ConsoleCapture();
    await consoleCapture.setup(page);

    networkMonitor = new NetworkMonitor();
    await networkMonitor.startMonitoring(page);

    await page.goto('https://your-app.com');
  });

  test('Button opens modal', async ({ page }) => {
    // Wait for page fully loaded
    await VisualHelpers.waitForFullLoad(page);

    // Capture initial state
    await ScreenshotManager.captureStep(page, 'page_loaded');

    // Click button with real visibility check
    await VisualHelpers.clickVisible(page, '.open-modal-btn');

    // Verify modal is REALLY visible
    const modalVisible = await VisualHelpers.verifyModalVisible(page, '.modal');
    expect(modalVisible).toBeTruthy();

    // Capture final state
    await ScreenshotManager.captureStep(page, 'modal_opened');
  });
});
```

### 2. Run Tests

```bash
npx playwright test
```

### 3. Get AI-Ready Bug Reports

If tests fail, a JSON report is automatically generated in `tests/reports/` with:
- Screenshots with context
- Console errors & warnings
- Network failures
- Affected files with line numbers
- Root cause hypothesis
- Suggested fixes

Copy the JSON → Paste to Claude CLI → Bugs get fixed! 🎯

---

## 🛠️ API Reference

### VisualHelpers

Interaction helpers with REAL visibility verification.

```javascript
import { VisualHelpers } from '@maderatools/visual-ai';

// Wait for page fully loaded with custom checks
await VisualHelpers.waitForFullLoad(page, [
  () => typeof window.myApp !== 'undefined',
  () => document.querySelectorAll('.item').length > 0
]);

// Click with visibility verification
await VisualHelpers.clickVisible(page, '.button');

// Scroll element into view
await VisualHelpers.scrollToVisible(page, '.element');

// Verify modal is really visible
const visible = await VisualHelpers.verifyModalVisible(page, '.modal');

// Wait for animations to complete
await VisualHelpers.waitForAnimations(page);

// Check if element is interactable (not disabled/covered)
const canClick = await VisualHelpers.isInteractable(page, '.button');

// Fill input with verification
await VisualHelpers.fillAndVerify(page, '#email', 'test@example.com');

// Wait for element to disappear
await VisualHelpers.waitForDisappear(page, '.loading');

// Wait for text to appear
await VisualHelpers.waitForText(page, '.message', 'Success!');
```

### ScreenshotManager

Capture screenshots with AI-ready context.

```javascript
import { ScreenshotManager } from '@maderatools/visual-ai';

// Capture a step
await ScreenshotManager.captureStep(page, 'step_name', {
  note: 'Description for AI analysis'
});

// Capture before/after comparison
await ScreenshotManager.captureComparison(
  page,
  'button_click',
  null,
  async () => await page.click('.button')
);

// Capture specific element
await ScreenshotManager.captureElement(page, '.modal', 'modal_content');

// Capture full page
await ScreenshotManager.captureFullPage(page, 'full_page');

// Get all screenshots for a test
const screenshots = ScreenshotManager.getTestScreenshots('test_id');
```

### ConsoleCapture

Monitor browser console automatically.

```javascript
import { ConsoleCapture } from '@maderatools/visual-ai';

const consoleCapture = new ConsoleCapture();
await consoleCapture.setup(page);

// Get errors
const errors = consoleCapture.getErrors();

// Get structured report
const report = consoleCapture.getStructuredReport();

// Detect error patterns
const patterns = consoleCapture.detectErrorPatterns();

// Print summary
consoleCapture.printSummary();
```

### NetworkMonitor

Track API calls and network activity.

```javascript
import { NetworkMonitor } from '@maderatools/visual-ai';

const networkMonitor = new NetworkMonitor();
await networkMonitor.startMonitoring(page, {
  apiPatterns: ['/api/', '/v1/']
});

// Get API calls
const apiCalls = networkMonitor.getApiCalls();

// Get failed requests
const failures = networkMonitor.getFailedRequests();

// Export HAR file
await networkMonitor.exportHAR('network-activity.har');
```

### ReportBuilder

Generate JSON reports for AI bug fixing.

```javascript
import { ReportBuilder } from '@maderatools/visual-ai';

const reportBuilder = new ReportBuilder();

const report = await reportBuilder.buildReport(
  testResult,
  {
    screenshots,
    consoleLogs: consoleCapture.getStructuredReport(),
    networkLogs: networkMonitor.getStructuredReport()
  }
);

await reportBuilder.exportForClaude(report, 'bug-report.json');
```

---

## 🤖 AI Integration (Gemini)

This framework is designed to work with **Gemini Pro** for on-demand test generation.

### Workflow:

1. **User**: "Test the login modal"
2. **Gemini**: Reads examples, generates custom test using the tools
3. **Gemini**: Runs test via Playwright
4. **If fail**: Gemini analyzes screenshots, generates JSON for Claude
5. **User**: Copy JSON → Paste to Claude CLI
6. **Claude**: Reads context, fixes bugs, commits changes

See [`docs/GEMINI-GUIDE.md`](docs/GEMINI-GUIDE.md) for complete instructions for AI.

---

## 📚 Examples

Check [`examples/`](examples/) for complete test templates:

- **`extension-test.spec.js`** - Chrome extension testing
- **`dragdrop-test.spec.js`** - Drag-and-drop interactions
- **`api-test.spec.js`** - Backend API workflows

---

## ⚙️ Configuration

### Using Default Config

```javascript
// playwright.config.js
import { defineConfig } from '@playwright/test';
import { defaultPlaywrightConfig } from '@maderatools/visual-ai';

export default defineConfig({
  ...defaultPlaywrightConfig,
  // Override what you need
  testDir: './my-tests',
  use: {
    ...defaultPlaywrightConfig.use,
    baseURL: 'https://my-app.com'
  }
});
```

### Custom Config

```javascript
// playwright.config.js
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  timeout: 60000,
  use: {
    viewport: { width: 1920, height: 1080 },
    slowMo: 500, // Slow down for visual debugging
    screenshot: 'on',
    video: 'retain-on-failure',
    trace: 'retain-on-failure'
  }
});
```

---

## 🎯 Best Practices

1. **Always use `VisualHelpers`** instead of plain Playwright methods
2. **Capture screenshots** at every important step
3. **Setup monitoring** in `beforeEach` (ConsoleCapture + NetworkMonitor)
4. **Verify REAL visibility** not just DOM presence
5. **Wait for animations** before assertions
6. **Use custom checks** in `waitForFullLoad()` for your app's specific state

---

## 🤝 Contributing

Contributions welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md) first.

### Development Setup

```bash
git clone https://github.com/mussonking/madera_tools.git
cd @maderatools/visual-ai
npm install
```

---

## 📄 License

MIT © Maderatools

---

## 🙏 Acknowledgments

- Built on top of [Playwright](https://playwright.dev/)
- Inspired by visual regression testing best practices
- Designed for AI-assisted development workflows

---

## 📞 Support

- 🐛 [Report a bug](https://github.com/mussonking/madera_tools/issues)
- 💡 [Request a feature](https://github.com/mussonking/madera_tools/issues)
- 📖 [Read the docs](https://github.com/mussonking/madera_tools#readme)

---

**Made with ❤️ for the Playwright community**