npm package: @kadi.build/file-manager
Description: Complete local and remote file management with watching and compression
Latest version: 1.1.0

# @kadi.build/file-manager

> Foundational file management package providing local and remote file operations, file watching, compression utilities, and file streaming.

## Features

- **Local File Operations** — Upload, download, copy, move, rename, delete files and folders
- **Remote File Operations** — SSH/SFTP-based remote operations with bastion/jump host support
- **File Watching** — Real-time file system monitoring with configurable debouncing and filtering
- **Compression** — ZIP and TAR.GZ compression/decompression with progress tracking and batch operations
- **File Streaming** — Memory-efficient streaming with range request support, ETag generation, and download tracking

## Installation

```bash
npm install @kadi.build/file-manager
```

## Quick Start

```javascript
import { createFileManager } from '@kadi.build/file-manager';

// Create a file manager instance
const fm = await createFileManager();

// Local file operations
await fm.uploadFile('./source.txt', './destination.txt');
const info = await fm.getFileInfo('./myfile.txt');
const files = await fm.listFiles('./my-directory');

// Compression
await fm.compressFile('./folder', './backup.zip', { format: 'zip' });
await fm.decompressFile('./backup.zip', './extracted/');

// File watching
const watcher = await fm.startWatching('./watched-dir');
fm.on('fileEvent', (event) => {
  console.log(`${event.type}: ${event.path}`);
});
```

## API Reference

### FileManager

The main orchestrator that provides a unified interface to all providers.

```javascript
import { FileManager, ConfigManager } from '@kadi.build/file-manager';

const config = new ConfigManager();
await config.load();
const fm = new FileManager(config);
```

#### Connection & Validation

| Method | Description |
|--------|-------------|
| `testConnection(providerName)` | Test connectivity for a specific provider (`'local'`, `'watch'`, `'compression'`, `'remote'`) |
| `validateProvider(providerName)` | Validate provider configuration |
| `getSystemInfo()` | Get system information (platform, memory, disk space) |

#### File Operations

| Method | Description |
|--------|-------------|
| `uploadFile(source, dest, providerName?)` | Copy/upload a file |
| `downloadFile(source, dest, providerName?)` | Copy/download a file |
| `getFileInfo(filePath, providerName?)` | Get file metadata |
| `listFiles(directory, providerName?, options?)` | List files in a directory |
| `deleteFile(filePath, providerName?)` | Delete a file |
| `renameFile(oldPath, newPath, providerName?)` | Rename a file |
| `copyFile(source, dest, providerName?)` | Copy a file |
| `moveFile(source, dest, providerName?)` | Move a file |
| `searchFiles(directory, pattern, options?)` | Search for files matching a pattern |

#### Folder Operations

| Method | Description |
|--------|-------------|
| `createFolder(folderPath, providerName?)` | Create a directory (recursive) |
| `listFolders(directory, providerName?)` | List subdirectories |
| `deleteFolder(folderPath, options?)` | Delete a directory |
| `renameFolder(oldPath, newPath)` | Rename a directory |
| `copyFolder(source, dest)` | Copy a directory recursively |
| `moveFolder(source, dest)` | Move a directory |
| `getFolderInfo(folderPath)` | Get directory metadata |

#### Compression

| Method | Description |
|--------|-------------|
| `compressFile(input, output, options?)` | Compress a file or directory |
| `decompressFile(archive, outputDir, options?)` | Decompress an archive |
| `compress(input, output, options?)` | Alias for `compressFile` |
| `decompress(archive, outputDir, options?)` | Alias for `decompressFile` |
| `compressMultipleFiles(files, outputDir, options?)` | Batch compress multiple files |
| `decompressMultipleFiles(archives, outputDir, options?)` | Batch decompress multiple archives |
| `getCompressionStatus()` | Get compression provider status |
| `getCompressionProvider()` | Get the underlying compression provider |

**Compression Options:**
```javascript
{
  format: 'zip' | 'tar.gz',  // Archive format (default: 'zip')
  level: 1-9,                  // Compression level (default: 6)
  includeRoot: true,           // Include root directory in archive
  overwrite: true              // Overwrite existing files during extraction
}
```

#### File Watching

| Method | Description |
|--------|-------------|
| `startWatching(path, options?)` | Start watching a path for changes |
| `stopWatching(watchIdOrPath)` | Stop a specific watcher |
| `stopAllWatching()` | Stop all active watchers |
| `listActiveWatchers()` | List all active watchers |
| `getWatcherInfo(watchId)` | Get details about a watcher |
| `getWatchingStatus()` | Get overall watching status |

**Watch Options:**
```javascript
{
  recursive: true,         // Watch subdirectories
  events: ['add', 'change', 'unlink'],  // Event types to listen for
  ignoreDotfiles: false,   // Ignore dotfiles
  debounceMs: 100          // Debounce interval in ms
}
```

**Events:**
```javascript
fm.on('fileEvent', (event) => {
  // event.type: 'add' | 'change' | 'unlink' | 'addDir' | 'unlinkDir'
  // event.path: string
  // event.watchId: string
});
```

#### Remote Operations

| Method | Description |
|--------|-------------|
| `connectRemote()` | Connect to the remote server |
| `disconnectRemote()` | Disconnect from the remote server |
| `syncToRemote(localPath, remotePath)` | Upload files to remote |
| `syncFromRemote(remotePath, localPath)` | Download files from remote |

**Remote Configuration (via environment variables):**
```
KADI_REMOTE_HOST=example.com
KADI_REMOTE_PORT=22
KADI_REMOTE_USERNAME=user
KADI_REMOTE_PASSWORD=pass
KADI_REMOTE_PRIVATE_KEY_PATH=~/.ssh/id_rsa
```

### Individual Providers

For advanced use cases, you can import providers directly:

```javascript
import {
  LocalProvider,
  WatchProvider,
  CompressionProvider,
  RemoteProvider,
  FileStreamingUtils,
  FileStreamer,
  DownloadTracker,
  PathUtils
} from '@kadi.build/file-manager';
```

### Factory Functions

```javascript
import { createFileManager, compressFile, decompressFile, watchDirectory } from '@kadi.build/file-manager';

// Quick file manager creation
const fm = await createFileManager(options);

// One-shot compression
await compressFile('./input', './output.zip', { format: 'zip' });

// One-shot decompression
await decompressFile('./archive.zip', './output/');

// Quick directory watching
const watcher = await watchDirectory('./path', (event) => {
  console.log(event);
});
```

## Configuration

Configuration is loaded from multiple sources (in priority order):

1. Constructor options
2. Environment variables (prefixed with `KADI_`)
3. `config.yml` file (walks up from CWD — looks for `file-manager:` section)
4. `.env` file (walks up from CWD — legacy fallback)

### config.yml

Place a `config.yml` in your project root (or any parent directory):

```yaml
file-manager:
  local_root: ./
  default_upload_directory: ./uploads
  default_download_directory: ./downloads
  default_temp_directory: ./temp
  max_file_size: 1073741824
  watch_debounce_ms: 100
  watch_max_watchers: 10
  compression_format: zip
  compression_level: 6
```

### Environment Variables (overrides)

| Variable | Description | Default |
|----------|-------------|---------|
| `KADI_LOCAL_ROOT` | Root directory for local operations | `.` |
| `KADI_WATCH_DEBOUNCE_MS` | Debounce interval for file watching | `100` |
| `KADI_WATCH_MAX_WATCHERS` | Maximum concurrent watchers | `10` |
| `KADI_COMPRESSION_FORMAT` | Default compression format | `zip` |
| `KADI_COMPRESSION_LEVEL` | Default compression level | `6` |
| `KADI_REMOTE_HOST` | Remote SSH host | — |
| `KADI_REMOTE_PORT` | Remote SSH port | `22` |
| `KADI_REMOTE_USERNAME` | Remote SSH username | — |
| `KADI_REMOTE_PASSWORD` | Remote SSH password | — |
| `KADI_REMOTE_PRIVATE_KEY_PATH` | Path to SSH private key | — |

## Testing

```bash
# Run all tests
npm test

# Run individual test suites
npm run test:local        # Local file operations (33 tests)
npm run test:watch        # File watching (30 tests)
npm run test:compression  # Compression (42 tests)
npm run test:streaming    # File streaming (12 tests)
npm run test:remote       # Remote operations (18 tests)
```

**Total: 135 tests across 5 suites**

## Project Structure

```
packages/file-manager/
├── package.json
├── README.md
├── src/
│   ├── index.js              # Main entry point & exports
│   ├── FileManager.js        # Main orchestrator
│   ├── ConfigManager.js      # Configuration management
│   ├── providers/
│   │   ├── LocalProvider.js       # Local file operations
│   │   ├── WatchProvider.js       # File watching (chokidar)
│   │   ├── CompressionProvider.js # ZIP & TAR.GZ compression
│   │   └── RemoteProvider.js      # SSH/SFTP remote operations
│   └── utils/
│       ├── FileStreamingUtils.js  # Streaming, MIME detection, ETags
│       └── PathUtils.js           # Path validation & normalization
└── tests/
    ├── run-all-tests.js
    ├── test-utils.js
    ├── test-helpers/
    │   └── index.js
    ├── test-local-operations.js
    ├── test-file-watching.js
    ├── test-compression.js
    ├── test-file-streaming.js
    └── test-remote-operations.js
```

## License

MIT