npm package: @juhomat/hexagonal-ai-framework
Description: AI-powered hexagonal framework with OpenAI integration, database adapters, web scraping, and Next.js demo application
Author: Juho Mattila
Latest version: 0.2.2

# Node.js Hexagonal Architecture Framework

A comprehensive hexagonal architecture framework for Node.js applications with integrations for OpenAI, web scraping, screenshot capture, and more.

## 🏗️ Architecture

This project implements a clean hexagonal architecture (ports and adapters pattern) with:

- **Core Domain**: Business logic and interfaces
- **Adapters**: External service integrations
- **Ports**: Interface definitions for external dependencies
- **Configuration**: Centralized service configuration

## 🚀 Features

### Core Services
- **Text Generation**: OpenAI GPT models integration with conversation support
- **Image Generation**: DALL-E image generation
- **Chat**: Conversational AI with multiple models (including o3)
- **Web Scraping**: Web content extraction using Puppeteer
- **Crawling Discovery**: Preview which pages would be discovered and crawled from a website
- **Screenshot Capture**: Full webpage screenshots using Puppeteer
- **Web Search**: Tavily search integration
- **Payment Processing**: Stripe integration (framework ready)
- **Authentication**: Firebase integration (framework ready)

### Applications
- **Next.js Web Application**: Full-featured dashboard with all services
- **CLI Examples**: Command-line interface examples for each service

## 📁 Project Structure

```
├── framework/                 # Core hexagonal architecture
│   ├── adapters/             # External service adapters
│   ├── config/               # Configuration management
│   ├── core/                 # Core domain logic
│   └── utils/                # Utility functions
├── application/              # Next.js web application
│   ├── src/
│   │   ├── components/       # React components
│   │   ├── pages/           # Next.js pages
│   │   ├── services/        # Client-side services
│   │   └── types/           # TypeScript types
├── examples/                 # CLI examples
└── src/                     # Additional source files
```

## 🛠️ Installation

### Prerequisites
- Node.js 18+ 
- npm or yarn

### Quick Setup

1. **Clone the repository**
   ```bash
   git clone https://github.com/juhomat/node-js-hexacon.git
   cd node-js-hexacon
   ```

2. **Environment Configuration**
   ```bash
   # Copy and configure environment variables
   cp .env.example .env
   # Edit .env with your API keys and settings
   ```

3. **Install dependencies and test**
   ```bash
   npm install
   npm run test:connection
   ```

### Minimum Configuration

Required environment variables in `.env`:
```env
# OpenAI API Key (required for AI features)
OPENAI_API_KEY=sk-your-openai-api-key-here

# Optional: Tavily for web search
TAVILY_API_KEY=your_tavily_api_key
```

## 🚀 Usage

### Web Application

Start the Next.js development server:

```bash
cd application
npm run dev
```

Visit `http://localhost:3000` to access the dashboard.

### CLI Examples

Run individual service examples:

```bash
# Text generation
npm run example:text-generation

# Image generation  
npm run example:image-generation

# Chat with o3 model
npm run example:o3-chat

# Screenshot capture
npm run example:screenshot

# Web search
npm run example:web-search
```

### Web-based Demos

Access interactive demos through the web dashboard:

- **Text Generation**: `/demo/text-generation`
- **Image Generation**: `/demo/image-generation`
- **Chat with o3**: `/demo/o3-chat`
- **Web Scraping**: `/demo/web-scraping`
- **Crawling Discovery**: `/demo/crawling-discovery` - Preview pages that would be crawled
- **Screenshot Capture**: `/demo/screenshot`
- **Web Search**: `/demo/web-search`

## 🔧 Configuration

### Service Configuration

The framework uses a centralized configuration system:

```typescript
import { FrameworkConfig } from './framework/config';

const config = new FrameworkConfig({
  openai: {
    apiKey: process.env.OPENAI_API_KEY,
    baseURL: 'https://api.openai.com/v1'
  },
  tavily: {
    apiKey: process.env.TAVILY_API_KEY
  }
});
```

### Available Services

- **OpenAI**: Text generation, image generation, chat
- **Tavily**: Web search and content extraction
- **Puppeteer**: Screenshot capture and web scraping
- **Stripe**: Payment processing (framework ready)
- **Firebase**: Authentication (framework ready)

## 🏛️ Architecture Principles

### Hexagonal Architecture

1. **Ports**: Define interfaces for external dependencies
2. **Adapters**: Implement external service integrations
3. **Core Domain**: Contains business logic independent of external services
4. **Configuration**: Centralized service configuration

### Benefits

- **Testability**: Easy to mock external dependencies
- **Flexibility**: Swap implementations without changing core logic
- **Maintainability**: Clear separation of concerns
- **Scalability**: Add new services without affecting existing code

## 📚 API Reference

### Application Framework

```typescript
import { ApplicationFramework } from './framework';

const framework = new ApplicationFramework();
await framework.initialize();

// Text generation
const text = await framework.generateText('Explain quantum computing', {
  model: 'gpt-4',
  conversationHistory: [
    { role: 'user', content: 'Hello' },
    { role: 'assistant', content: 'Hi! How can I help?' }
  ]
});

// Text generation with streaming
await framework.generateTextStream('Write a story', {
  model: 'gpt-4',
  onChunk: (chunk) => console.log(chunk),
  onComplete: () => console.log('Done!'),
  onError: (error) => console.error(error)
});

// Image generation
const images = await framework.generateImage('A futuristic cityscape', {
  size: '1024x1024',
  quality: 'hd',
  style: 'vivid'
});

// Screenshot capture
const screenshot = await framework.captureScreenshot('https://example.com', {
  fullPage: true,
  width: 1920,
  height: 1080,
  format: 'png'
});

// Web scraping
const scrapedData = await framework.scrapeWebPage('https://example.com', {
  selector: '.content',
  includeImages: true,
  followLinks: false
});

// Crawling discovery
const crawlPreview = await framework.discoverCrawlablePages('https://example.com', {
  maxPages: 10,
  maxDepth: 2,
  includePatterns: ['docs/*'],
  excludePatterns: ['admin/*']
});

// Web search
const searchResults = await framework.searchWeb('latest AI developments', {
  maxResults: 5,
  searchDepth: 'advanced'
});

// Health checks
const health = await framework.getHealthStatus();
const isHealthy = await framework.isHealthy();

// Cleanup
await framework.dispose();
```

### Client Framework Service

For client-side usage (without Puppeteer):

```typescript
import { ClientFrameworkService } from './application/src/services/ClientFrameworkService';

const clientFramework = new ClientFrameworkService();

// Safe for client-side use
const health = await clientFramework.getHealthStatus();
```

## 🧪 Testing

```bash
# Test framework connectivity
npm run test:connection

# Run all tests
npm test
```

### Test Scripts

- **`test:connection`** - Verify framework initialization and service health

## 📦 Deployment

### Production Build

```bash
cd application
npm run build
npm start
```

### Environment Variables

Ensure all required environment variables are set in production:

- `OPENAI_API_KEY`
- `TAVILY_API_KEY` (optional)
- `NEXT_PUBLIC_API_BASE_URL`

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

## 📄 License

This project is licensed under the MIT License.

## 🆘 Support

For issues and questions:

1. Check the [Issues](https://github.com/juhomat/node-js-hexacon/issues) page
2. Create a new issue with detailed information
3. Include error messages and reproduction steps

## 🔄 Changelog

### v1.2.0 (Current)
- **BREAKING**: Removed RAG functionality to focus on core building blocks
- **ENHANCED**: Streamlined framework for better separation of concerns
- **MAINTAINED**: All existing functionality (OpenAI, web scraping, screenshots, search)
- **IMPROVED**: Hexagonal architecture purity - framework provides building blocks, applications build business logic

### v1.1.0
- **NEW**: Crawling Discovery feature - Preview which pages would be discovered and crawled from any website
- Added `/demo/crawling-discovery` interactive demo
- Enhanced Web Scraping Services section in dashboard
- Updated framework services to support crawling discovery
- Improved breadth-first crawling algorithm with domain filtering and depth control

### v1.0.0
- Initial release
- Hexagonal architecture implementation
- OpenAI integration
- Screenshot capture with Puppeteer
- Next.js web application
- CLI examples
- Tavily web search integration