npm package: @miridih-de/n8n-nodes-databricks
Description: n8n community node for Databricks SQL and Workflow execution
Author: suhyun
Latest version: 0.1.12

# n8n-nodes-databricks

[한국어](README.ko.md)

n8n community node for executing SQL queries and triggering jobs on Databricks.

## Features

- **SQL Execution**: Run SQL queries on Databricks SQL Warehouse or All-Purpose Cluster
- **Large Result Support**: Handle results up to 100GB with EXTERNAL_LINKS disposition (SQL Warehouse only)
- **Workflow Trigger**: Execute Databricks Jobs with parameters
- **Flexible Output**: Choose between individual rows or DataFrame format
- **Resource Locator**: Easy selection of warehouses, clusters, and jobs via dropdown
- **Multiple Auth Methods**: OAuth2 and Personal Access Token support

## Installation

### Community Nodes (Recommended)

1. Go to **Settings > Community Nodes**
2. Select **Install**
3. Enter `n8n-nodes-databricks`
4. Select **Install**

### Manual Installation

```bash
npm install n8n-nodes-databricks
```

## Credentials

This node supports two authentication methods:

### OAuth2 (Recommended)
Configure OAuth2 credentials for secure authentication with Databricks.

### Personal Access Token
| Field                     | Description                                                               |
| ------------------------- | ------------------------------------------------------------------------- |
| **Workspace URL**         | Your Databricks workspace URL (e.g., `https://xxx.cloud.databricks.com`)  |
| **Personal Access Token** | Token generated from Databricks User Settings > Developer > Access tokens |

## Operations

### SQL Resource

#### Execute SQL

Run SQL queries on Databricks.

**Parameters:**

| Parameter               | Description                                                                     |
| ----------------------- | ------------------------------------------------------------------------------- |
| Execution Mode          | `SQL Warehouse` or `Shared Cluster`                                             |
| SQL Warehouse / Cluster | Select from dropdown or enter ID                                                |
| SQL Statement           | The SQL query to execute                                                        |
| Output Format           | `Individual Rows` (n8n standard) or `DataFrame` (single object with rows array) |
| Timeout                 | Maximum wait time in seconds (default: 300)                                     |
| Large Result Support    | Enable for results > 25MB (SQL Warehouse only, up to 100GB)                     |

> **Note**: Shared Cluster mode has a ~1000 row limit due to Command API 1.2 limitations. For large datasets, use SQL Warehouse with Large Result Support.

**Output Format Examples:**

_Individual Rows:_

```json
[
	{ "id": 1, "name": "Alice" },
	{ "id": 2, "name": "Bob" }
]
```

_DataFrame:_

```json
{
	"rows": [
		{ "id": 1, "name": "Alice" },
		{ "id": 2, "name": "Bob" }
	],
	"rowCount": 2
}
```

### Workflow Resource

#### Run Now

Trigger a Databricks job immediately.

**Parameters:**

| Parameter           | Description                          |
| ------------------- | ------------------------------------ |
| Job                 | Select from dropdown or enter Job ID |
| Job Parameters      | Optional JSON parameters for the job |
| Wait for Completion | Whether to wait for job to finish    |
| Timeout             | Maximum wait time in seconds         |

## Configuration

### Large Result Support

When handling very large result sets (millions of rows), you may need to increase n8n's payload size limit:

```yaml
# docker-compose.yml
environment:
  - N8N_PAYLOAD_SIZE_LIMIT=2048  # MB, default is 16
```

## Development

### Local Development with Docker

```bash
# Build the node
npm run build

# Start n8n with Docker
docker-compose up -d

# Access n8n at http://localhost:5678
```

### Build Commands

```bash
npm run build       # Build for production
npm run dev         # Start development mode
npm run lint        # Check for linting errors
npm run lint:fix    # Fix linting errors
```

## License

[MIT](LICENSE.md)

## Links

- [n8n Community Nodes Documentation](https://docs.n8n.io/integrations/community-nodes/)
- [Databricks REST API Reference](https://docs.databricks.com/api/workspace/introduction)