npm package: @omed0/infra-kit
Description: A comprehensive, production-ready TypeScript SDK for self-hosted infrastructure services including queue management, caching, object storage, pub/sub events, rate limiting, distributed locking, and session management
Author: Omed0
Latest version: 1.0.1

# 🚀 Infra-Kit

A comprehensive, production-ready TypeScript/JavaScript SDK for self-hosted infrastructure services. Build scalable applications with queue management, caching, object storage, pub/sub events, rate limiting, distributed locking, and session management.

[![npm version](https://img.shields.io/npm/v/@omed0/infra-kit.svg)](https://www.npmjs.com/package/@omed0/infra-kit)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![TypeScript](https://img.shields.io/badge/TypeScript-Ready-blue.svg)](https://www.typescriptlang.org/)

## 🌟 Why Infra-Kit?

- **🔒 Security First**: Input validation, protection against injection attacks, and secure token generation
- **⚡ High Performance**: Atomic operations, connection pooling, and optimized batch processing
- **🛡️ Type Safe**: Full TypeScript support with comprehensive type definitions
- **📦 Modular**: Import only what you need with tree-shaking support
- **🔧 Self-Hosted**: Full control over your infrastructure with Redis and MinIO
- **📖 Well Documented**: Extensive examples and guides for every feature
- **🚀 Production Ready**: Battle-tested with error handling and retry strategies

## Features

- 📦 **Queue Management** - Background jobs with BullMQ
- ⚡ **Cache** - High-performance caching with Redis
- 📁 **Object Storage** - S3-compatible storage with MinIO
- 🔔 **Event Bus** - Pub/Sub messaging with Redis
- 🛡️ **Rate Limiting** - Sliding window rate limiting
- 🔒 **Distributed Locking** - Redis-based distributed locks
- 👤 **Session Management** - Secure session storage

## 📦 Installation

```bash
npm install @omed0/infra-kit bullmq ioredis minio
# or
pnpm add @omed0/infra-kit bullmq ioredis minio
# or
yarn add @omed0/infra-kit bullmq ioredis minio
# or
bun add @omed0/infra-kit bullmq ioredis minio
```

### Peer Dependencies

- `bullmq`: >=5.0.0 - Queue management
- `ioredis`: >=5.0.0 - Redis client
- `minio`: >=8.0.0 - S3-compatible storage

## Quick Start

### 1. Initialize Configuration

```typescript
import { initConfig, initFromEnv } from '@omed0/infra-kit';

// Option 1: Programmatic configuration
initConfig({
  redis: {
    host: 'localhost',
    port: 6379,
    password: 'your-password', // optional
    db: 0, // optional, default: 0
    keyPrefix: 'myapp:', // optional
  },
  storage: {
    endPoint: 'localhost',
    port: 9000,
    accessKey: 'minioadmin',
    secretKey: 'minioadmin',
    useSSL: false, // optional, default: false
    region: 'us-east-1', // optional
  }
});

// Option 2: From environment variables
initFromEnv();
```

**Queue Management**

```typescript
import { addJob, processJobs, scheduleJob } from '@omed0/infra-kit/queue';

// Add a job
await addJob('emails', 'send-welcome', {
  to: 'user@example.com',
  subject: 'Welcome!'
}, {
  delay: 5000, // delay in ms
  attempts: 3, // retry attempts
  backoff: {
    type: 'exponential',
    delay: 1000
  }
});

// Process jobs with error handling
processJobs('emails', async (job) => {
  console.log('Processing:', job.id, job.data);
  
  // Your business logic here
  await sendEmail(job.data);
  
  return { sent: true, timestamp: Date.now() };
}, { 
  concurrency: 5, // process 5 jobs concurrently
  limiter: {
    max: 100, // max 100 jobs
    duration: 1000 // per second
  }
});

// Schedule recurring job (cron)
await scheduleJob('backups', 'database-backup', {}, {
  pattern: '0 0 * * *' // every day at midnight
});
```
**Queue Management**

```typescript
import { addJob, processJobs } from '@infra-kit/core/queue';
**Cache**

```typescript
import { setCache, getCache, getMany, increment } from '@omed0/infra-kit/cache';

// Set with TTL (in seconds)
await setCache('user:123', { 
  name: 'John', 
  email: 'john@example.com',
  roles: ['admin']
}, 3600);

// Get single value
**Object Storage**

```typescript
import { 
  uploadFile, 
  downloadFile, 
  getDownloadUrl, 
  getUploadUrl,
  listObjects,
  deleteObject 
} from '@omed0/infra-kit/storage';

**Event Bus (Pub/Sub)**

```typescript
import { publish, subscribe, subscribeMany } from '@omed0/infra-kit/events';

// Subscribe to specific event
const unsubscribe = await subscribe('user.created', (data) => {
  console.log('New user:', data.userId, data.email);
});

// Subscribe with pattern matching
await subscribe('user.*', (data) => {
  console.log('Any user event:', data);
});

// Subscribe to multiple events at once
await subscribeMany(
  ['user.created', 'user.updated', 'user.deleted'],
  (data) => {
    console.log('User lifecycle event:', data);
  }
);

// Publish event
const subscriberCount = await publish('user.created', {
  userId: '123',
  email: 'user@example.com',
  timestamp: Date.now()
});
console.log(`Published to ${subscriberCount} subscribers`);

// Later: unsubscribe when done
unsubscribe();

// Get subscriber count
**Rate Limiting (Sliding Window)**

```typescript
import { checkRateLimit, createRateLimiter } from '@omed0/infra-kit/rate-limit';

// Check rate limit
const result = await checkRateLimit('user:123', {
  limit: 100, // max requests
  window: 60, // time window in seconds
  key: 'api-requests' // namespace for this limiter
});

if (!result.allowed) {
  console.error(`Rate limit exceeded. Retry in ${result.retryAfter}ms`);
  throw new Error('Too many requests');
}
**Distributed Locking**

```typescript
import { acquireLock, releaseLock, withLock } from '@omed0/infra-kit/lock';

// Manual lock management
const token = await acquireLock('process:data', { 
  ttl: 30000, // 30 seconds
  retries: 3, // retry 3 times
  retryDelay: 100 // wait 100ms between retries
});

if (token) {
  try {
    // Critical section - only one process can execute this
    await processData();
  } finally {
    await releaseLock('process:data', token);
  }
} else {
  console.log('Could not acquire lock');
}

// Automatic lock management (recommended)
const result = await withLock('critical-section', async () => {
  // Your critical code here
  await updateDatabase();
  return { success: true };
}, { ttl: 30000 });

// Extend lock if needed
await extendLock('process:data', token, 15000); // add 15 more seconds
**Session Management**

```typescript
import { 
  setSession, 
  getSession, 
  updateSession, 
  deleteSession,
  refreshSession 
} from '@omed0/infra-kit/session';

// Create session
await setSession('sess_abc123', {
  userId: '456',
  email: 'user@example.com',
  roles: ['user'],
  preferences: {
    theme: 'dark',
    language: 'en'
  }
}, { ttl: 86400 }); // 24 hours

// Get session
const session = await getSession('sess_abc123');
if (session) {
  console.log('User ID:', session.userId);
  console.log('Email:', session.email);
}

// Update session (merge with existing data)
await updateSession('sess_abc123', {
  lastActivity: Date.now(),
  'preferences.theme': 'light'
});

// Refresh session TTL (keep alive)
await refreshSession('sess_abc123', 86400);

// Check if session exists
const exists = await hasSession('sess_abc123');

// Get remaining TTL
const ttl = await getSessionTTL('sess_abc123');
console.log(`Session expires in ${ttl} seconds`);

// Delete session (logout)
await deleteSession('sess_abc123');

// Delete all sessions for a user
await deleteSessionsByPattern('user:456:*');
```
// Get remaining TTL
const ttl = await getTTL('user:123');
```cessJobs('emails', async (job) => {
  console.log('Sending email:', job.data);
  // Send email logic
  return { sent: true };
}, { concurrency: 5 });
```

**Cache**

```typescript
import { setCache, getCache } from '@infra-kit/core/cache';

// Set with TTL
await setCache('user:123', { name: 'John', email: 'john@example.com' }, 3600);

// Get
const user = await getCache('user:123');
console.log(user?.name);
```

**Object Storage**

```typescript
import { uploadFile, getDownloadUrl } from '@infra-kit/core/storage';

// Upload file
await uploadFile('uploads', 'images/photo.jpg', '/tmp/photo.jpg');

// Get presigned download URL
const url = await getDownloadUrl('uploads', 'images/photo.jpg', 3600);
```

**Event Bus**

```typescript
import { publish, subscribe } from '@infra-kit/core/events';

// Subscribe
await subscribe('user.created', (data) => {
  console.log('New user:', data);
});

// Publish
await publish('user.created', {
  userId: '123',
  email: 'user@example.com'
});
```

**Rate Limiting**

```typescript
import { checkRateLimit } from '@infra-kit/core/rate-limit';

const result = await checkRateLimit('user:123', {
  limit: 100,
  window: 60,
  key: 'api-requests'
});

if (!result.allowed) {
  throw new Error(`Rate limit exceeded. Retry in ${result.retryAfter}ms`);
}
```

**Distributed Locking**

```typescript
import { withLock } from '@infra-kit/core/lock';

await withLock('critical-section', async () => {
  // Critical work here
  console.log('Locked section');
}, { ttl: 30000 });
```

**Session Management**

```typescript
import { setSession, getSession } from '@infra-kit/core/session';

// Create session
await setSession('sess_123', {
  userId: '456',
  email: 'user@example.com'
}, { ttl: 3600 });

// Get session
const session = await getSession('sess_123');
```

## 📚 Documentation

- **[TypeScript Backend Integration](./docs/TYPESCRIPT_BACKEND.md)** - Integration patterns for backend projects
- **[Deployment Guide](./docs/DEPLOYMENT.md)** - Self-hosted and cloud provider deployment

## 🏗️ Module Architecture

Import specific modules for better tree-shaking:

```typescript
// Import only what you need
import { addJob, processJobs } from '@omed0/infra-kit/queue';
import { setCache, getCache } from '@omed0/infra-kit/cache';
import { uploadFile } from '@omed0/infra-kit/storage';
import { publish, subscribe } from '@omed0/infra-kit/events';
import { checkRateLimit } from '@omed0/infra-kit/rate-limit';
import { acquireLock } from '@omed0/infra-kit/lock';
import { setSession } from '@omed0/infra-kit/session';

// Or import entire namespaces
import { initConfig, queue, cache, storage } from '@omed0/infra-kit';
```

## API Documentation

### Queue

- `addJob(queueName, jobName, data, options)` - Add a job
- `addBulkJobs(queueName, jobs)` - Add multiple jobs
- `scheduleJob(queueName, jobName, data, repeatOptions)` - Schedule recurring job
- `processJobs(queueName, processor, options)` - Process jobs
- `getQueueMetrics(queueName)` - Get queue stats
- `cleanQueue(queueName, grace, limit, type)` - Clean old jobs
- `pauseQueue(queueName)` / `resumeQueue(queueName)` - Control queue

### Cache

- `setCache(key, value, ttl)` - Set cache entry
- `getCache(key)` - Get cache entry
- `deleteCache(key)` - Delete entry
- `hasCache(key)` - Check if exists
- `increment(key, amount)` / `decrement(key, amount)` - Atomic operations
- `getMany(keys)` / `setMany(entries)` - Bulk operations
- `clearCache()` - Clear all cache

### Storage

- `uploadFile(bucket, objectName, filePath, metadata)` - Upload file
- `downloadFile(bucket, objectName, filePath)` - Download file
- `deleteObject(bucket, objectName)` - Delete object
- `listObjects(bucket, prefix, recursive)` - List objects
- `getUploadUrl(bucket, objectName, expiry)` - Presigned upload URL
- `getDownloadUrl(bucket, objectName, expiry)` - Presigned download URL

### Events

- `publish(channel, data)` - Publish event
- `subscribe(channel, handler)` - Subscribe to events
- `subscribeMany(channels, handler)` - Subscribe to multiple channels
- `unsubscribe(channel)` - Unsubscribe from channel

### Rate Limit

- `checkRateLimit(identifier, options)` - Check and consume rate limit
- `peekRateLimit(identifier, options)` - Check without consuming
- `resetRateLimit(identifier, key)` - Reset limits
- `createRateLimiter(options)` - Create reusable limiter

### Lock

- `acquireLock(key, options)` - Acquire lock
- `releaseLock(key, token)` - Release lock
- `extendLock(key, token, ttl)` - Extend lock TTL
- `withLock(key, fn, options)` - Execute with lock

### Session

- `setSession(sessionId, data, options)` - Create/update session
- `getSession(sessionId)` - Get session data
- `updateSession(sessionId, data, options)` - Merge update
- `deleteSession(sessionId)` - Delete session
- `refreshSession(sessionId, ttl)` - Refresh TTL

## 🐳 Docker Setup

Use this `docker-compose.yml` for local development:

```yaml
version: '3.8'

services:
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: redis-server --requirepass your-password
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

  minio:
    image: bitnami/minio:latest
    ports:
      - "9000:9000"  # API
      - "9001:9001"  # Console
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
## 🔒 Security Features

- **Input Validation**: All inputs are validated to prevent injection attacks
- **Safe Pattern Matching**: Protection against accidental mass deletion
- **Path Traversal Prevention**: Object storage validates paths
- **Atomic Operations**: Rate limiting uses Lua scripts for atomicity
- **Secure Tokens**: Distributed locks use cryptographically secure tokens
- **Connection Retry**: Automatic reconnection with exponential backoff
- **Error Handling**: Comprehensive error handling throughout

## 🚀 Performance Optimizations

- **Connection Pooling**: Reuses Redis and MinIO connections
- **Batch Operations**: Supports bulk cache/storage operations
- **Tree-Shaking**: Modular exports for minimal bundle size
- **Lua Scripts**: Atomic operations in Redis reduce round trips
- **Lazy Connection**: Optional lazy Redis connection initialization
- **Chunked Deletion**: Large deletions are batched to prevent blocking

## 🛠️ TypeScript Support

Full TypeScript support with comprehensive type definitions:
## 📖 License

MIT © [Omed0](https://github.com/omed0)

## 🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## 📮 Support

- 📖 [Documentation](https://github.com/omed0/infra-kit/tree/main/docs)
- 🐛 [Report Issues](https://github.com/omed0/infra-kit/issues)
- 💬 [Discussions](https://github.com/omed0/infra-kit/discussions)
- 📧 Email: support@example.com

## 🌟 Show Your Support

If you find this project helpful, please give it a ⭐️ on [GitHub](https://github.com/omed0/infra-kit)!

## 📝 Changelog

See [CHANGELOG.md](./CHANGELOG.md) for a list of changes.

---

**Built with ❤️ for the TypeScript community**
} from '@omed0/infra-kit';

// Type-safe configuration
const config: Config = {
  redis: {
    host: 'localhost',
    port: 6379
  }
};

// Type-safe job data
interface EmailJob {
  to: string;
  subject: string;
  body: string;
}

await addJob<EmailJob>('emails', 'send', {
  to: 'user@example.com',
  subject: 'Hello',
  body: 'Welcome!'
});
```
Access MinIO Console: http://localhost:9001

## Module Exports

Import specific modules for better tree-shaking:

```typescript
import { addJob } from '@infra-kit/core/queue';
import { setCache } from '@infra-kit/core/cache';
import { uploadFile } from '@infra-kit/core/storage';
import { publish } from '@infra-kit/core/events';
import { checkRateLimit } from '@infra-kit/core/rate-limit';
import { acquireLock } from '@infra-kit/core/lock';
import { setSession } from '@infra-kit/core/session';
```

## TypeScript Support

Full TypeScript support with type definitions included.

```typescript
import type { Config, RateLimitResult } from '@infra-kit/core';
```

## License

MIT © Omed0

## Contributing

Contributions welcome! Please open an issue or PR.

## Support

- 📖 [Documentation](https://github.com/omed0/infra-kit)
- 🐛 [Issues](https://github.com/omed0/infra-kit/issues)
- 💬 [Discussions](https://github.com/omed0/infra-kit/discussions)