npm package: @majkapp/bot-plugin-server
Description: Playwright-based automation and testing framework for MAJK plugins
Author: Jules White
Latest version: 0.4.0

# @majkapp/bot-plugin-server

Playwright-based automation and testing framework for MAJK plugins. Test your plugin UIs using semantic `data-majk-*` attributes for stable, maintainable tests.

## ✨ What's New

- 🎯 **Real Plugin Functions** - Automatically loads and invokes your actual plugin functions (no mocks needed!)
- 📋 **Console Log Capture** - Captures all client-side `console.log`, `console.error`, etc. with timestamps and source locations
- 🐛 **Error Tracking** - Auto-captures page errors with stack traces and auto-screenshots on errors
- 🎮 **Interactive Mode** - Open your plugin in a browser for manual testing while streaming logs in real-time
- 📁 **Organized Output** - Logs saved alongside screenshots in per-test directories

## Quick Start with npx

The fastest way to test your MAJK plugin is with `npx`:

### Example 1: Basic Test with Auto-Managed Server

```bash
npx @majkapp/bot-plugin-server \
  --script ./my-test.js \
  --plugin ./my-plugin \
  --headed
```

**What this does:**
1. 🚀 Automatically starts a plugin server for your plugin
2. 🌐 Launches Chrome browser (visible because of `--headed`)
3. 📝 Runs your test script
4. 📸 Saves screenshots to `./screenshots/`
5. 🛑 Cleans up (stops server and browser)

**Where things go:**
- Screenshots: `./screenshots/` (configurable with `--screenshots`)
- Logs: `./logs/` (configurable with `--logs`)
- Test results: Printed to console

---

### Example 2: Complete Test with Mock Handlers

First, create your test fixtures:

**`fixtures/handlers.js`** - Mock function handlers:
```javascript
module.exports = {
  async getUserProfile({ userId }) {
    return {
      success: true,
      data: {
        id: userId,
        name: 'Test User',
        email: 'test@example.com',
        role: 'admin'
      }
    };
  },

  async saveUserProfile({ userId, profile }) {
    console.log('Saving profile:', profile);
    return {
      success: true,
      data: { saved: true, timestamp: Date.now() }
    };
  },

  async deleteUser({ userId }) {
    return {
      success: true,
      data: { deleted: true }
    };
  }
};
```

**`tests/user-profile-test.js`** - Your test script:
```javascript
module.exports = async function(bot) {
  console.log('🧪 Testing user profile page...');

  // Navigate to plugin
  await bot.navigate();
  await bot.screenshot('01-home');

  // Navigate to profile page (if using hash routing)
  const page = bot.getPage();
  await page.goto(`${page.url()}#/profile`);
  await bot.waitForState('loaded', { timeout: 5000 });
  await bot.screenshot('02-profile-loaded');

  // Edit profile
  await bot.click('editButton');
  await bot.type('nameField', 'Updated Name');
  await bot.type('emailField', 'updated@example.com');
  await bot.screenshot('03-profile-edited');

  // Save changes
  await bot.click('saveButton');
  await bot.waitForState('success', { timeout: 3000 });
  await bot.screenshot('04-profile-saved');

  console.log('✅ Profile test completed!');
};
```

**Run the complete test:**
```bash
npx @majkapp/bot-plugin-server \
  --script ./tests/user-profile-test.js \
  --plugin ./my-plugin \
  --handlers ./fixtures/handlers.js \
  --screenshots ./test-results/screenshots \
  --slow-mo 300 \
  --headed
```

**What this does:**
1. 🚀 Starts plugin server with your plugin
2. 🔧 Loads mock handlers from `./fixtures/handlers.js`
3. 🌐 Opens Chrome browser (visible, slowed down 300ms per action)
4. 📝 Runs your test: navigate → edit → save → verify
5. 📸 Saves 4 screenshots to `./test-results/screenshots/`:
   - `01-home.png` - Initial state
   - `02-profile-loaded.png` - Profile page loaded
   - `03-profile-edited.png` - After editing fields
   - `04-profile-saved.png` - Success state
6. ✅ Prints success/failure to console
7. 🛑 Cleans up everything

---

### Example 3: Form Validation Test

**`tests/form-validation.js`:**
```javascript
module.exports = async function(bot) {
  console.log('🧪 Testing form validation...');

  await bot.navigate();
  const page = bot.getPage();
  await page.goto(`${page.url()}#/contact`);
  await bot.screenshot('01-empty-form');

  // Test 1: Submit empty form (should show errors)
  console.log('  → Testing empty form submission...');
  await bot.click('submitButton');
  await page.waitForTimeout(500);
  await bot.screenshot('02-validation-errors');

  // Verify errors appeared
  const nameError = await page.locator('[data-majk-error="name"]').isVisible();
  const emailError = await page.locator('[data-majk-error="email"]').isVisible();
  console.log(`  → Name error visible: ${nameError}`);
  console.log(`  → Email error visible: ${emailError}`);

  // Test 2: Fill form with valid data
  console.log('  → Filling form with valid data...');
  await bot.type('name', 'John Doe');
  await bot.type('email', 'john@example.com');
  await bot.type('message', 'This is a test message!');
  await bot.screenshot('03-form-filled');

  // Test 3: Submit valid form
  console.log('  → Submitting valid form...');
  await bot.click('submitButton');
  await bot.waitForState('success', { timeout: 5000 });
  await bot.screenshot('04-success');

  console.log('✅ Validation test completed!');
};
```

**Run it:**
```bash
npx @majkapp/bot-plugin-server \
  --script ./tests/form-validation.js \
  --plugin ./my-plugin \
  --headless
```

**Output:**
```
🚀 Starting plugin server for _/my-plugin
🚀 MAJK Plugin Server running on http://localhost:54123
   Serving 1 plugin(s)
✅ Server started on http://localhost:54123
🌐 Starting browser...
✅ Browser started
📍 Navigating to http://localhost:54123/plugins/_/my-plugin/ui
✅ Page loaded
📸 Taking screenshot: screenshots/01-empty-form.png
✅ Screenshot saved
🧪 Testing form validation...
  → Testing empty form submission...
🖱️  Clicking element: submitButton
✅ Clicked: submitButton
📸 Taking screenshot: screenshots/02-validation-errors.png
✅ Screenshot saved
  → Name error visible: true
  → Email error visible: true
  → Filling form with valid data...
⌨️  Typing into field: name
✅ Typed into: name
⌨️  Typing into field: email
✅ Typed into: email
⌨️  Typing into field: message
✅ Typed into: message
📸 Taking screenshot: screenshots/03-form-filled.png
✅ Screenshot saved
  → Submitting valid form...
🖱️  Clicking element: submitButton
✅ Clicked: submitButton
⏳ Waiting for state: success
✅ State reached: success
📸 Taking screenshot: screenshots/04-success.png
✅ Screenshot saved
✅ Validation test completed!
🛑 Stopping...
✅ Browser closed
✅ Server stopped

✅ Test completed successfully!
```

---

### Example 4: Multi-Step Wizard Test

**`tests/wizard-flow.js`:**
```javascript
module.exports = async function(bot) {
  console.log('🧪 Testing wizard flow...');

  await bot.navigate();
  const page = bot.getPage();
  await page.goto(`${page.url()}#/wizard`);
  await bot.waitForState('step-1');
  await bot.screenshot('01-step-1');

  // Step 1
  console.log('  → Completing step 1...');
  await bot.type('firstName', 'John');
  await bot.type('lastName', 'Doe');
  await bot.click('nextButton');
  await bot.waitForState('step-2');
  await bot.screenshot('02-step-2');

  // Step 2
  console.log('  → Completing step 2...');
  await bot.type('email', 'john@example.com');
  await bot.type('phone', '555-1234');
  await bot.click('nextButton');
  await bot.waitForState('step-3');
  await bot.screenshot('03-step-3');

  // Step 3
  console.log('  → Completing step 3...');
  await bot.click('agreeCheckbox');
  await bot.click('completeButton');
  await bot.waitForState('completed', { timeout: 5000 });
  await bot.screenshot('04-completed');

  console.log('✅ Wizard test completed!');
};
```

**Run it:**
```bash
npx @majkapp/bot-plugin-server \
  --script ./tests/wizard-flow.js \
  --plugin ./my-plugin \
  --handlers ./fixtures/handlers.js \
  --middleware ./fixtures/middleware.js \
  --screenshots ./test-results/wizard \
  --slow-mo 500 \
  --headed \
  --verbose
```

---

### Example 5: CI/CD Integration

**`.github/workflows/test.yml`:**
```yaml
name: Plugin Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: |
          cd my-plugin
          npm install
          npm run build

      - name: Install Playwright
        run: npx playwright install chromium

      - name: Run bot tests
        run: |
          npx @majkapp/bot-plugin-server \
            --script ./tests/all-tests.js \
            --plugin ./my-plugin \
            --handlers ./fixtures/handlers.js \
            --headless \
            --screenshots ./test-results/screenshots

      - name: Upload screenshots
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: test-screenshots
          path: test-results/screenshots/
```

---

## CLI Options Reference

### Required Options

| Option | Description | Example |
|--------|-------------|---------|
| `--script <path>` | Path to test script file | `--script ./tests/my-test.js` |

### Plugin Source (choose one)

| Option | Description | Example |
|--------|-------------|---------|
| `--plugin <path>` | Auto-start server for this plugin | `--plugin ./my-plugin` |
| `--url <url>` | Connect to existing server | `--url http://localhost:3000` |

### Optional Configuration

| Option | Description | Default |
|--------|-------------|---------|
| `--handlers <path>` | Function handlers file | None |
| `--middleware <path>` | Middleware file | None |
| `--screenshots <path>` | Screenshots directory | `./screenshots` |
| `--logs <path>` | Logs directory | `./logs` |
| `--asset-path-prefix <path>` | Asset path prefix | `/plugin-screens` |
| `--headless` | Run browser in headless mode | `true` |
| `--headed` | Show browser window | N/A |
| `--slow-mo <ms>` | Slow down actions (milliseconds) | `0` |
| `--verbose` | Show detailed output | `false` |
| `-h, --help` | Show help message | N/A |

---

## Writing Test Scripts

Your test script should export a function that receives a `bot` instance:

```javascript
// Simple test
module.exports = async function(bot) {
  await bot.navigate();
  await bot.click('myButton');
  await bot.screenshot('after-click');
};
```

Or using ES modules:
```javascript
export default async function(bot) {
  await bot.navigate();
  await bot.click('myButton');
  await bot.screenshot('after-click');
};
```

### Bot API

| Method | Description | Example |
|--------|-------------|---------|
| `navigate(url?)` | Navigate to plugin (or custom URL) | `await bot.navigate()` |
| `click(id)` | Click element by `data-majk-id` | `await bot.click('submitButton')` |
| `type(field, value)` | Type into `data-majk-field` | `await bot.type('email', 'test@example.com')` |
| `waitForState(state, opts?)` | Wait for `data-majk-state` | `await bot.waitForState('success', { timeout: 5000 })` |
| `screenshot(name)` | Capture screenshot | `await bot.screenshot('my-screenshot')` |
| `getPage()` | Get Playwright Page for advanced operations | `const page = bot.getPage()` |

### Advanced Usage

For more control, access the Playwright page directly:

```javascript
module.exports = async function(bot) {
  const page = bot.getPage();

  // Custom navigation
  await page.goto('https://example.com');

  // Custom selectors
  await page.click('.my-custom-class');

  // Custom waits
  await page.waitForSelector('.loaded', { timeout: 10000 });

  // Extract data
  const text = await page.textContent('[data-majk-id="message"]');
  console.log('Message:', text);

  // Network interception
  await page.route('**/api/**', route => {
    route.fulfill({
      status: 200,
      body: JSON.stringify({ mocked: true })
    });
  });

  // Standard bot API still works
  await bot.click('submitButton');
  await bot.screenshot('result');
};
```

---

## UI Development Guide

To make your plugin testable, add `data-majk-*` attributes to your components:

```jsx
<Button
  data-majk-action="submit"
  data-majk-id="submitButton"
  onPress={handleSubmit}
>
  Submit
</Button>

<Input
  label="Email"
  data-majk-field="email"
  data-majk-id="emailInput"
  value={email}
  onChange={setEmail}
/>

{submitted && (
  <div data-majk-state="success" data-majk-id="successMessage">
    Success!
  </div>
)}

{error && (
  <div data-majk-error="validation">
    {error}
  </div>
)}
```

**See [DEVELOPER_GUIDE.md](./DEVELOPER_GUIDE.md) for complete UI development guide.**

---

## Installation (for project usage)

```bash
npm install --save-dev @majkapp/bot-plugin-server
```

Then use programmatically:

```javascript
const { runBot } = require('@majkapp/bot-plugin-server');

await runBot({
  plugin: {
    dir: './my-plugin',
    assetPathPrefix: '/plugin-screens'
  },
  output: {
    screenshots: './screenshots'
  },
  browser: {
    headless: true
  }
}, async (bot) => {
  await bot.navigate();
  await bot.click('submitButton');
  await bot.screenshot('result');
});
```

---

## Common Patterns

### Pattern 1: Test Suite with Multiple Tests

**`tests/all-tests.js`:**
```javascript
const testLogin = require('./login-test');
const testProfile = require('./profile-test');
const testSettings = require('./settings-test');

module.exports = async function(bot) {
  console.log('🧪 Running test suite...\n');

  try {
    console.log('1️⃣  Running login test...');
    await testLogin(bot);
    console.log('✅ Login test passed\n');

    console.log('2️⃣  Running profile test...');
    await testProfile(bot);
    console.log('✅ Profile test passed\n');

    console.log('3️⃣  Running settings test...');
    await testSettings(bot);
    console.log('✅ Settings test passed\n');

    console.log('🎉 All tests passed!');
  } catch (error) {
    console.error('❌ Test failed:', error.message);
    throw error;
  }
};
```

### Pattern 2: Reusable Test Helpers

**`tests/helpers.js`:**
```javascript
async function login(bot, username, password) {
  const page = bot.getPage();
  await page.goto(`${page.url()}#/login`);
  await bot.type('username', username);
  await bot.type('password', password);
  await bot.click('loginButton');
  await bot.waitForState('logged-in', { timeout: 5000 });
}

async function logout(bot) {
  await bot.click('userMenu');
  await bot.click('logoutButton');
  await bot.waitForState('logged-out', { timeout: 5000 });
}

module.exports = { login, logout };
```

**Usage:**
```javascript
const { login, logout } = require('./helpers');

module.exports = async function(bot) {
  await bot.navigate();

  await login(bot, 'testuser', 'password');
  await bot.screenshot('logged-in');

  // Do something as logged-in user
  await bot.click('profileButton');

  await logout(bot);
  await bot.screenshot('logged-out');
};
```

---

## Debugging Tips

### 1. Use `--headed` and `--slow-mo`

```bash
npx @majkapp/bot-plugin-server \
  --script ./tests/my-test.js \
  --plugin ./my-plugin \
  --headed \
  --slow-mo 1000
```

This lets you watch the browser in slow motion!

### 2. Add Verbose Logging

```bash
npx @majkapp/bot-plugin-server \
  --script ./tests/my-test.js \
  --plugin ./my-plugin \
  --verbose
```

### 3. Capture Screenshots at Every Step

```javascript
module.exports = async function(bot) {
  await bot.navigate();
  await bot.screenshot('step-1-navigate');

  await bot.type('email', 'test@example.com');
  await bot.screenshot('step-2-email-filled');

  await bot.click('submitButton');
  await bot.screenshot('step-3-submitted');

  await bot.waitForState('success');
  await bot.screenshot('step-4-success');
};
```

### 4. Use Browser Console

```javascript
module.exports = async function(bot) {
  const page = bot.getPage();

  // Listen to console
  page.on('console', msg => console.log('🖥️  Browser:', msg.text()));
  page.on('pageerror', err => console.error('❌ Page error:', err.message));

  // Rest of your test
  await bot.navigate();
  await bot.click('submitButton');
};
```

---

## Examples

See the [`examples/`](./examples/) directory for complete working examples:

- `test-button-clicks.js` - Button interactions and counters
- `test-form-filling.js` - Form validation and submission
- `test-plugin-home.js` - Basic navigation
- `wizard-screenshot.js` - Screenshot capture

---

## Dependencies

- **@majkapp/majk-plugin-server** ^1.1.0 - Plugin server
- **playwright** ^1.40.0 - Browser automation
- **@playwright/test** ^1.40.0 - Testing utilities

---

## License

MIT

---

## Automated Testing Framework

For automated plugin testing with test runners, use **`@majkapp/plugin-test`** which provides a higher-level testing API built on top of this bot framework.

### Quick Example with `@majkapp/plugin-test`

Install:
```bash
npm install --save-dev @majkapp/plugin-test
```

Write UI tests with `uiTest`:

```javascript
const { uiTest } = require('@majkapp/plugin-test');

// Configure once
uiTest.configure({
  plugin: { dir: process.cwd() },
  browser: { headless: true, slowMo: 200 }
});

// Write tests
uiTest('switch to Wizards tab', async (bot) => {
  await bot.navigate();
  await bot.screenshot('01-home-buttons-tab');

  const page = bot.getPage();
  await page.click('button[data-key="wizards"]');
  await page.waitForTimeout(500);
  await bot.screenshot('02-wizards-tab-active');

  const wizardsContent = await page.locator('text=Wizard Examples').isVisible();
  console.log(`✅ Wizards tab content visible: ${wizardsContent}`);
});

uiTest('navigate to Conversations Sample', async (bot) => {
  await bot.navigate();
  await bot.screenshot('01-home-page');

  const page = bot.getPage();
  await page.click('a:has-text("Conversations Sample")');
  await page.waitForTimeout(1000);
  await bot.screenshot('02-conversations-sample-loaded');

  const url = page.url();
  console.log(`✅ Current URL: ${url}`);
});
```

### UI Testing with Mock Data

The bot framework supports mock handlers for plugin functions:

```javascript
const { runBot } = require('@majkapp/bot-plugin-server');

await runBot({
  plugin: { dir: './my-plugin' },
  handlers: {
    async getConversations({ limit = 100, offset = 0 }) {
      return [
        {
          id: 'conv-1',
          title: 'Plugin Architecture Discussion',
          createdAt: new Date('2025-11-10').toISOString(),
          updatedAt: new Date('2025-11-12').toISOString(),
          messageCount: 15
        },
        {
          id: 'conv-2',
          title: 'Testing Framework Design',
          createdAt: new Date('2025-11-11').toISOString(),
          updatedAt: new Date('2025-11-13').toISOString(),
          messageCount: 8
        }
      ];
    },

    async getConversationMessages({ conversationId }) {
      return [
        {
          id: 'msg-1',
          role: 'user',
          content: 'Can you help me understand how plugins work?',
          timestamp: new Date().toISOString()
        },
        {
          id: 'msg-2',
          role: 'assistant',
          content: 'Of course! MAJK plugins are modular extensions.',
          timestamp: new Date().toISOString()
        }
      ];
    }
  },
  output: { screenshots: './screenshots' },
  browser: { headless: true }
}, async (bot) => {
  await bot.navigate();

  const page = bot.getPage();
  await page.click('a:has-text("Conversations Sample")');
  await page.waitForTimeout(1000);
  await bot.screenshot('conversations-loaded');

  // Verify mock data appeared
  const conversationItems = await page.locator('[data-majk-id="conversationItem"]').count();
  console.log(`✅ Found ${conversationItems} conversation items`);

  // Click first conversation
  await page.click('[data-majk-id="conversationItem"]:first-child');
  await page.waitForTimeout(500);
  await bot.screenshot('conversation-selected');
});
```

**How Mock Handlers Work:**
1. Configure handlers in `BotConfig.handlers`
2. When UI makes API calls like `/plugins/{org}/{id}/api/getConversations`
3. The bot server's `functionInvoker` intercepts and calls your handler
4. Mock data is returned to the UI
5. You can verify the UI displays the mock data correctly

### Screenshot Management

`uiTest` automatically manages screenshots with organized naming:

```javascript
uiTest('test with multiple screenshots', async (bot) => {
  await bot.navigate();
  // Creates: screenshots/test-with-multiple-screenshots/00-home.png
  await bot.screenshot('home');

  await bot.click('submitButton');
  // Creates: screenshots/test-with-multiple-screenshots/01-after-submit.png
  await bot.screenshot('after-submit');

  // Error screenshots are automatic:
  try {
    await bot.click('nonExistentButton');
  } catch (error) {
    // Creates: screenshots/test-with-multiple-screenshots/02-error-click-nonExistentButton.png
  }
});
```

### Running Tests

With `@majkapp/plugin-test` CLI:

```bash
# Run all tests
npx majk-test

# Run only UI tests
npx majk-test --type ui

# Run specific test file
npx majk-test tests/plugin/ui/e2e/my-test.test.js

# Generate HTML report
npx majk-test --reporter html --output test-results.html
```

**Test output includes:**
- ✅ Pass/fail status for each test
- 📸 Screenshot paths
- ⏱️ Test duration
- 📊 Summary statistics

### Complete Testing Workflow

**1. Setup (`tests/plugin/ui/e2e/my-flow.test.js`):**

```javascript
const { uiTest } = require('@majkapp/plugin-test');

uiTest.configure({
  plugin: { dir: process.cwd() },
  browser: { headless: true, slowMo: 200 }
});
```

**2. Write tests with mock data:**

```javascript
const mockHandlers = {
  async myFunction(params) {
    return { success: true, data: { result: 'mocked' } };
  }
};

uiTest('test with mocks', async (bot) => {
  // Configure handlers
  bot.configure({ handlers: mockHandlers });

  await bot.navigate();
  await bot.click('triggerButton');
  await bot.screenshot('after-trigger');

  // Verify UI shows mock data
  const page = bot.getPage();
  const text = await page.textContent('[data-majk-id="result"]');
  console.log(`✅ Result: ${text}`);
});
```

**3. Run tests:**

```bash
cd my-plugin
npx majk-test --type ui --env e2e
```

**4. Review results:**
- Console output shows pass/fail
- Screenshots in `screenshots/{test-name}/` directory
- HTML report (if using `--reporter html`)

### Plugin Inspection

The bot framework includes plugin inspection tools for debugging:

```bash
# Inspect plugin structure
npx bot-plugin-server inspect ./my-plugin

# Test function execution
npx bot-plugin-server inspect ./my-plugin --function myFunction --params '{"key":"value"}'

# With authentication
npx bot-plugin-server inspect ./my-plugin \
  --function myFunction \
  --auth-endpoint http://localhost:3000/auth
```

**Inspection shows:**
- 📦 Plugin metadata (name, version, org, ID)
- 🔧 Available functions with input/output schemas
- 🎨 UI screens and routes
- 🔌 API routes
- 🛠️ Tools registered by the plugin

See [AUTH_CONFIGURATION.md](./AUTH_CONFIGURATION.md) for authentication setup.

---

## Related Documentation

- [DEVELOPER_GUIDE.md](./DEVELOPER_GUIDE.md) - Complete guide for building bot-testable UIs
- [AUTH_CONFIGURATION.md](./AUTH_CONFIGURATION.md) - Authentication setup for testing
- [@majkapp/plugin-test README](../majk-plugin-test/README.md) - Automated testing framework
- [examples/](./examples/) - Working test examples
- [UI_TESTING_DESIGN.md](../UI_TESTING_DESIGN.md) - UI testing architecture
- [PLUGIN_INSPECTION_DESIGN.md](../PLUGIN_INSPECTION_DESIGN.md) - Plugin inspection framework

---

**Happy Testing! 🤖**

---

## 🎯 Using Real Plugin Functions  

By default, if you don't provide `--handlers`, the bot will automatically load your **real plugin functions**:

```bash
npx @majkapp/bot-plugin-server \
  --script ./my-test.js \
  --plugin ./my-plugin
```

Your plugin's actual functions will be called when the UI invokes them!

---

## 📋 Console Log & Error Capture

All client-side JavaScript logs are automatically captured with timestamps and source locations.

**Output files** in `./logs/`:
- `console.txt` - Human-readable timeline
- `console.json` - Structured data
- `errors.txt` - Page errors with stacks
- `summary.json` - Quick overview

---

## 🎮 Interactive Mode

Test manually in a browser while capturing logs:

```bash
npx @majkapp/bot-plugin-server \
  --interactive \
  --plugin ./my-plugin
```

Press Ctrl+C to stop and save all logs.

---