npm package: @hyperdrive.bot/serverless-composer-plugin
Description: Serverless Framework plugin for dynamic configuration loading, transformation, and module-scoped deployments
Author: DS Team
Latest version: 1.0.2

# Serverless Composer Plugin

A comprehensive Serverless Framework plugin that dynamically composes Serverless Framework services by loading configuration fragments from structured directories. Now includes module-scoped deployment functionality for deploying specific modules within your Serverless application.

## Features

### Core Composer Features
- ✅ **Dynamic configuration loading**: Load functions, resources, outputs, etc. from structured directories
- ✅ **Cross-version compatibility**: Works with Serverless Framework v2, v3 & v4  
- ✅ **Per-category transformers**: Optional transformation functions for each category
- ✅ **Variable resolution**: Cross-version variable resolution before merging
- ✅ **Multiple file formats**: Supports YAML, JSON, and JavaScript/TypeScript files

### Module-Scoped Deployment Features
- ✅ **Module-scoped deployments**: Deploy only specific modules instead of the entire service
- ✅ **Complete isolation**: Module deployments are completely isolated from other modules
- ✅ **Configurable module directory**: Customize where modules are located
- ✅ **CLI command integration**: Simple `deployModule` command with shorthand options
- ✅ **Automatic stack naming**: Follows `{service}-{module}-{stage}` pattern

## Installation

### Production Install
```bash
# Install stable version
npm install @hyperdrive.bot/serverless-composer

# Or install alpha version (latest features)
npm install @hyperdrive.bot/serverless-composer@alpha
```

Add the plugin to your `serverless.yml` file:

```yaml
plugins:
  - '@hyperdrive.bot/serverless-composer'
```

### Development Install (Local)

```bash
cd packages/serverless/composer
npm run build
```

## Configuration

Configure the plugin in your `serverless.yml`:

```yaml
custom:
  composer:
    modulesDir: serverless/modules  # Default: serverless/modules (for module deployments)
    # Optional category-specific configurations
    functions:
      directory: custom/functions  # Custom directory (optional)
      transformer: path/to/functions-transformer.js
    resources:
      directory: custom/resources  # Custom directory (optional)
      transformer: path/to/resources-transformer.js
    stepFunctions:
      directory: custom/stepFunctions  # Custom directory (optional)
      transformer: path/to/stepfunctions-transformer.js
```

## Directory Structure

### Regular Composer Structure

For regular deployments, organize your configuration files under `serverless/`:

```
project/
├── serverless.yml
├── serverless/
│   ├── functions/
│   │   ├── users.yml
│   │   ├── orders.yml
│   │   └── notifications.yml
│   ├── resources/
│   │   ├── dynamodb.yml
│   │   ├── s3.yml
│   │   └── api-gateway.yml
│   ├── stepFunctions/
│   │   └── order-processing.yml
│   └── outputs/
│       └── api-endpoints.yml
```

### Module-Based Structure

For module-scoped deployments, organize modules under the configured `modulesDir`:

```
project/
├── serverless.yml
├── serverless/
│   └── modules/
│       ├── auth/
│       │   ├── functions/
│       │   │   ├── login.yml
│       │   │   └── register.yml
│       │   ├── resources/
│       │   │   └── user-table.yml
│       │   └── outputs/
│       │       └── auth-outputs.yml
│       ├── payment/
│       │   ├── functions/
│       │   │   ├── process-payment.yml
│       │   │   └── refund.yml
│       │   └── resources/
│       │       └── payment-queue.yml
│       └── notifications/
│           ├── functions/
│           │   └── send-notification.yml
│           └── stepFunctions/
│               └── notification-workflow.yml
```

## Usage

### Regular Deployment

For regular deployments that compose all configuration fragments:

```bash
serverless deploy
```

This will load and merge all configuration files from the structured directories.

### Module-Scoped Deployment

Deploy only the `auth` module:

```bash
serverless deployModule --module auth
```

Or using the shorthand:

```bash
serverless deployModule -m auth
```

**What happens during module deployment:**
1. **Stack name set** to `{originalService}-{module}-{stage}` (e.g., `my-service-auth-dev`)
2. **Complete isolation** - clears existing config to prevent interference
3. **Composes only** the configuration from `serverless/modules/auth/`
4. **Automatically triggers** `serverless deploy` to deploy the composed module
5. **Shows progress logs** with `[composer]` prefix

### Module-Scoped Removal

Remove only the `auth` module:

```bash
serverless removeModule --module auth
```

Or using the shorthand:

```bash
serverless removeModule -m auth
```

**What happens during module removal:**
1. **Stack name set** to `{originalService}-{module}-{stage}` (e.g., `my-service-auth-dev`)
2. **Complete isolation** - clears existing config to prevent interference
3. **Composes only** the configuration from `serverless/modules/auth/`
4. **Automatically triggers** `serverless remove` to delete the composed module
5. **Shows progress logs** with `[composer]` prefix

### Deploy Multiple Modules

To deploy multiple modules, run the command for each module:

```bash
serverless deployModule -m auth
serverless deployModule -m payment
serverless deployModule -m notifications
```

### Remove Multiple Modules

To remove multiple modules, run the command for each module:

```bash
serverless removeModule -m auth
serverless removeModule -m payment
serverless removeModule -m notifications
```

## Supported Categories

The plugin supports the following configuration categories:

- **`functions`**: Lambda function definitions
- **`resources`**: CloudFormation resources
- **`stepFunctions`**: Step Functions state machines
- **`lambdaRoleStatements`**: IAM role statements for Lambda functions
- **`outputs`**: CloudFormation outputs
- **`userRoleStatements`**: Custom user role statements
- **`modules`**: Module-specific configurations

Each category can be customized with:
- **`directory`**: Custom path for the category files (defaults to `serverless/{categoryName}`)
- **`transformer`**: Optional function to transform loaded data before merging

## File Format Support

The plugin supports multiple file formats in each category directory:

- **YAML**: `.yml`, `.yaml`
- **JSON**: `.json`
- **JavaScript**: `.js`, `.cjs`, `.mjs`
- **TypeScript**: `.ts`

### Example Function Definition

```yaml
# serverless/functions/users.yml
getUser:
  handler: src/handlers/users.getUser
  events:
    - http:
        path: /users/{id}
        method: get

createUser:
  handler: src/handlers/users.createUser
  events:
    - http:
        path: /users
        method: post
```

### Example Resource Definition

```yaml
# serverless/resources/dynamodb.yml
UserTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: ${self:service}-users-${opt:stage}
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: userId
        AttributeType: S
    KeySchema:
      - AttributeName: userId
        KeyType: HASH
```

## Advanced Configuration

### Custom Module Directory

```yaml
custom:
  composer:
    modulesDir: my-custom-modules-path
```

### Custom Category Directories

You can customize the directory path for each category:

```yaml
custom:
  composer:
    functions:
      directory: my-custom-functions-dir
    resources:
      directory: my-custom-resources-dir
    stepFunctions:
      directory: my-custom-stepfunctions-dir
    outputs:
      directory: my-custom-outputs-dir
```

### Category-Specific Transformers

Transformers allow you to modify configuration fragments before they're merged:

```yaml
custom:
  composer:
    functions:
      directory: custom/functions  # Optional custom directory
      transformer: ./transformers/functions.js
    resources:
      directory: custom/resources  # Optional custom directory
      transformer: ./transformers/resources.js
```

### Transformer Example

```javascript
// transformers/functions.js
module.exports = (payload, serverless) => {
  // Add environment variables to all functions
  const functionsWithEnv = {}
  
  for (const [name, config] of Object.entries(payload)) {
    functionsWithEnv[name] = {
      ...config,
      environment: {
        ...config.environment,
        STAGE: serverless.service.provider.stage,
        SERVICE_NAME: serverless.service.service
      }
    }
  }
  
  return functionsWithEnv
}
```

### TypeScript Transformer Example

```typescript
// transformers/resources.ts
import { ServerlessInstance } from 'serverless'

interface ResourceConfig {
  [key: string]: any
}

export default (payload: ResourceConfig, serverless: ServerlessInstance): ResourceConfig => {
  // Add common tags to all resources
  const resourcesWithTags = {}
  
  for (const [name, config] of Object.entries(payload)) {
    if (config.Type && config.Properties) {
      resourcesWithTags[name] = {
        ...config,
        Properties: {
          ...config.Properties,
          Tags: [
            ...(config.Properties.Tags || []),
            { Key: 'Service', Value: serverless.service.service },
            { Key: 'Stage', Value: serverless.service.provider.stage }
          ]
        }
      }
    } else {
      resourcesWithTags[name] = config
    }
  }
  
  return resourcesWithTags
}
```

## How It Works

### Regular Composer Mode

1. **Plugin initialization**: Detects Serverless Framework version
2. **Directory scanning**: Scans configured directories for files
3. **File loading**: Loads and parses YAML, JSON, and JavaScript files
4. **Content merging**: Deep merges all content within each category
5. **Transformation**: Applies optional transformers
6. **Variable resolution**: Resolves Serverless variables (v2 only)
7. **Service composition**: Merges categories into the service configuration

### Module-Scoped Mode

When you run `deployModule --module myModule`:

1. **Stack naming**: Sets stack name to `{service}-{module}-{stage}`
2. **Configuration clearing**: Removes existing functions, resources, etc. for isolation
3. **Directory scoping**: Redirects all category paths to `{modulesDir}/{module}/`
4. **Module processing**: Loads only configuration from the specified module
5. **Composition**: Merges module-specific configuration into service
6. **Deployment**: Automatically triggers `serverless deploy`

## Version Compatibility

- **Serverless v2**: Uses hooks with `populateObject` for variable resolution
- **Serverless v3+**: Uses constructor-time processing with `extendConfiguration`

The plugin automatically detects your Serverless Framework version and uses the appropriate integration method.

## Troubleshooting

### Module Directory Not Found

```
Error: Module directory not found: /path/to/serverless/modules/myModule
```

**Solution**: Ensure the module directory exists and contains the expected category subdirectories.

### Module Directory Not Found (Remove Command)

```
Warning: Module directory not found: /path/to/serverless/modules/myModule
Proceeding with removal in case resources still exist in AWS...
```

**Note**: This is expected behavior for `removeModule`. The command will proceed with removal even if the local module directory doesn't exist, as the AWS resources may still need to be cleaned up.

### Transformer Errors

```
Transformer error in 'functions': Cannot read property 'stage' of undefined
```

**Solution**: Check your transformer function - ensure it handles the serverless instance correctly and doesn't assume properties exist.

### Variable Resolution Issues (v2)

If variables aren't resolving correctly, ensure:
1. Variables are valid Serverless Framework syntax
2. Referenced resources exist in the service configuration
3. The plugin is loaded before other plugins that might conflict

### Remove Command Not Triggering

```
⚠️ Cannot trigger automatic removal. Please run 'serverless remove' manually.
```

**Solution**: The plugin couldn't access the Serverless plugin manager. Run `serverless remove` manually to complete the module removal.

## Migration from Module-Composer

If you were previously using `serverless-module-composer-plugin`:

1. **Update plugin reference**:
   ```yaml
   plugins:
     # Remove these lines:
     # - serverless-composer-plugin
     # - serverless-module-composer-plugin
     
     # Add this line:
     - @hyperdrive.bot/serverless-composer
   ```

2. **Update configuration**:
   ```yaml
   custom:
     # Change from 'moduleComposer' to 'composer'
     composer:
       modulesDir: serverless/modules
   ```

3. **Commands work identically**:
   ```bash
   # Deploy commands (unchanged)
   serverless deployModule -m myModule
   
   # Remove commands (new functionality)
   serverless removeModule -m myModule
   ```

4. **All functionality preserved**: No breaking changes to existing workflows

## Development

### Building the Plugin

```bash
npm run build
```

### Running Tests

```bash
npm test
```

### Linting

```bash
npm run lint
npm run lint:fix
```

## License

MIT