npm package: @singhey/spa-ssr-renderer
Description: A Node.js server that intelligently serves SPAs with server-side rendering for bots and crawlers
Author: singhey
Latest version: 1.1.0

# @singhey/spa-ssr-renderer

A Node.js server application that intelligently serves Single Page Applications (SPAs) by providing server-side rendered content to bots and crawlers while serving the original SPA files to regular users.

## Features

- **Bot Detection**: Identifies web crawlers and search engine bots using ua-parser-js
- **Server-Side Rendering**: Uses Playwright for headless browser rendering
- **Intelligent Caching**: TTL-based cache with `@isaacs/ttlcache`
- **Static File Serving**: Efficient serving of static assets with SPA fallback
- **Sitemap Support**: Parse sitemaps to discover paths for pre-rendering
- **Exclusion Patterns**: Flexible path exclusion with wildcard support
- **Graceful Fallbacks**: Continues operation even when components fail
- **Library & CLI**: Can be used as an importable library or standalone CLI tool

## Installation

```bash
npm install @singhey/spa-ssr-renderer
# or
pnpm add @singhey/spa-ssr-renderer
# or
yarn add @singhey/spa-ssr-renderer
```

## Usage

### As a Library

Import and use the server in your Node.js application:

```typescript
import { SPASSRServer, ServerConfig } from '@singhey/spa-ssr-renderer';

// Define your configuration
const config: ServerConfig = {
  port: 3000,
  staticDir: 'public',
  spaEntryPoint: 'index.html',
  prerender: {
    // Explicit paths to pre-render
    paths: ['/', '/about', '/products'],
    
    // Sitemaps to parse (URLs or local file paths)
    sitemaps: ['https://example.com/sitemap.xml', './public/sitemap.xml'],
    
    // Paths or patterns to exclude (supports wildcards)
    exclude: ['/admin/*', '/api/*', '/private']
  },
  cache: {
    type: 'memory',
    ttl: 300000, // 5 minutes
    maxSize: 100,
  },
  renderer: {
    timeout: 30000,
    viewport: { width: 1280, height: 720 },
    waitForNetworkIdle: false,
  },
  botDetection: {
    customPatterns: [],
    enableVerification: false,
  },
};

// Create and start server
const server = new SPASSRServer({ config });

await server.start();
console.log('Server running!');

// Access underlying Fastify instance for custom routes
const fastifyInstance = server.getServer();
fastifyInstance.get('/api/custom', async () => {
  return { message: 'Custom endpoint' };
});

// Graceful shutdown
process.on('SIGTERM', async () => {
  await server.stop();
});
```

### As a CLI Tool

Run directly from the command line:

```bash
# Using npx
npx @singhey/spa-ssr-renderer

# Or install globally
npm install -g @singhey/spa-ssr-renderer
spa-ssr-renderer
```

### Environment Variables

Configure the server using environment variables:

- `PORT` - Server port (default: 3000)
- `STATIC_DIR` - Static files directory (default: public)
- `SPA_ENTRY_POINT` - SPA entry file (default: index.html)
- **Pre-rendering Configuration:**
  - `PRERENDER_PATHS` - Comma-separated list of paths to pre-render (e.g., "/,/about,/products")
  - `PRERENDER_SITEMAPS` - Comma-separated list of sitemap URLs or file paths (e.g., "https://example.com/sitemap.xml,./public/sitemap.xml")
  - `PRERENDER_EXCLUDE` - Comma-separated list of paths or patterns to exclude (supports wildcards: `*` and `?`)
  - `PRERENDER_CONCURRENCY` - Number of pages to render in parallel (default: 5, recommended: 5-20)
- `CACHE_TYPE` - Cache type: memory|redis (default: memory)
- `CACHE_TTL` - Cache TTL in ms (default: 300000)
- `RENDER_TIMEOUT` - Render timeout in ms (default: 30000)
- `VIEWPORT_WIDTH` - Render viewport width (default: 1280)
- `VIEWPORT_HEIGHT` - Render viewport height (default: 720)

## How It Works

1. **Static Files**: If a file exists in the static directory, it's served directly
2. **Parallel Pre-rendering**: On startup, specified paths are rendered in parallel using Playwright
   - **Explicit Paths**: Define specific paths to pre-render
   - **Sitemap Support**: Parse sitemaps (URLs or local files) to discover paths
   - **Exclusions**: Use patterns to exclude paths from pre-rendering (e.g., `/admin/*`, `/api/*`)
   - **Concurrency Control**: Configure how many pages render simultaneously (default: 5)
   - **Network Idle**: Waits for all network requests to complete before capturing HTML
3. **Bot Detection**: Incoming requests are analyzed for bot User-Agents using ua-parser-js
4. **Smart Serving**:
   - **Bots**: Served pre-rendered, cached HTML for instant SEO-friendly content
   - **Regular Users**: Served the SPA entry point for full client-side interactivity
5. **SPA Fallback**: For routes without file extensions, serves the SPA entry point (index.html)
6. **Caching**: Pre-rendered content is cached using `@isaacs/ttlcache` with configurable TTL

### Pre-rendering Configuration Examples

**Explicit paths only:**
```typescript
prerender: {
  paths: ['/', '/about', '/products', '/contact']
}
```

**Using sitemaps:**
```typescript
prerender: {
  sitemaps: [
    'https://example.com/sitemap.xml',  // Remote sitemap
    './public/sitemap.xml'               // Local sitemap
  ]
}
```

**With exclusions and concurrency:**
```typescript
prerender: {
  paths: ['/', '/about', '/products'],
  sitemaps: ['./public/sitemap.xml'],
  exclude: [
    '/admin/*',      // Exclude all admin paths
    '/api/*',        // Exclude all API paths
    '/private',      // Exclude specific path
    '*/draft'        // Exclude all draft pages
  ],
  concurrency: 10    // Render 10 pages in parallel (default: 5)
}
```

**Combined configuration:**
```typescript
prerender: {
  paths: ['/', '/about'],              // Always pre-render these
  sitemaps: ['./public/sitemap.xml'],  // Plus paths from sitemap
  exclude: ['/admin/*', '/api/*'],     // But exclude these patterns
  concurrency: 8                       // Render 8 pages at a time
}
```

### Performance Tips

- **Concurrency**: Higher values (10-20) speed up pre-rendering but use more memory
- **Network Idle**: Pages wait for all requests to complete, ensuring dynamic content is captured
- **Batch Processing**: Pages are rendered in batches to prevent overwhelming the system
- **Timeout**: Configure `renderer.timeout` for slow-loading pages (default: 30s)

## Project Structure

```
src/
├── components/          # Core application components
│   ├── BotDetector.ts   # Bot detection logic
│   ├── FileServer.ts    # Static file serving
│   ├── RequestRouter.ts # Request routing and classification
│   ├── CacheManager.ts  # Caching system
│   ├── SSRRenderer.ts   # Server-side rendering
│   └── index.ts         # Component exports
├── config/              # Configuration management
│   └── index.ts         # Default config and environment loading
├── types/               # TypeScript interfaces and types
│   └── index.ts         # All type definitions
├── utils/               # Shared utilities
│   ├── logger.ts        # Logging utilities
│   └── index.ts         # Utility exports
├── __tests__/           # Test files
│   └── setup.test.ts    # Foundation tests
└── index.ts             # Main application entry point
```

## Development

### Prerequisites

- Node.js 18+
- pnpm

### Installation

```bash
pnpm install
npx playwright install
```

### Scripts

- `pnpm dev` - Start development server with hot reload
- `pnpm build` - Build for production
- `pnpm start` - Start production server
- `pnpm test` - Run tests
- `pnpm test:watch` - Run tests in watch mode
- `pnpm test:coverage` - Run tests with coverage

### Environment Variables

- `PORT` - Server port (default: 3000)
- `STATIC_DIR` - Static files directory (default: public)
- `SPA_ENTRY_POINT` - SPA entry file (default: index.html)
- `CACHE_TYPE` - Cache type: memory|redis (default: memory)
- `CACHE_TTL` - Cache TTL in ms (default: 300000)
- `RENDER_TIMEOUT` - Render timeout in ms (default: 30000)
- `VIEWPORT_WIDTH` - Render viewport width (default: 1280)
- `VIEWPORT_HEIGHT` - Render viewport height (default: 720)

## Implementation Status

This is the foundation setup. Core functionality will be implemented in subsequent tasks:

- [x] Task 1: Project foundation and interfaces
- [ ] Task 2: Bot Detection System
- [ ] Task 3: File Server Component
- [ ] Task 4: Request Router
- [ ] Task 5: Cache Manager
- [ ] Task 6: SSR Renderer
- [ ] Task 7: Main Server Integration
- [ ] Task 8: Configuration System
- [ ] Task 9: Final Integration

## Testing

The project uses Jest for unit testing and fast-check for property-based testing. Each component will have comprehensive test coverage including:

- Unit tests for specific scenarios
- Property-based tests for universal behaviors
- Integration tests for component interactions