npm package: @lumidb/lumidb
Description: The official LumiDB SDK for the browser and node.js.
Author: LumiDB Authors
Latest version: 0.7.2

# LumiDB SDK for JavaScript/Typescript

The official LumiDB SDK for the browser and node.js.

> [!CAUTION]
> The LumiDB API is currently in active development and should be considered unstable.
> While we strive to maintain functionality, breaking changes may occur between package versions.
> The API does not yet provide backward compatibility guarantees, and endpoints, request/response
> formats, and behavior may change significantly between versions.

## Installation

```bash
npm install --save @lumidb/lumidb
```

## API

An instance of `LumiDB` provides the following functionality:

**Authentication**

-   **`setAuthentication`** Configure the authentication method used for queries.

**Table Management**

-   **`listTables`** Retrieve a list of all accessible tables in the database.

**Data Operations**

-   **`query`** Query for point data.
-   **`getMetadata`** List the metadata for all multiple tables with optional filtering.
-   **`getTableMetadata`** Get complete metadata for a single table, including table-level information and metadata for all original files with optional filtering.
-   **`getDownloadUrls`** Get download URLs for original files.
-   **`get3DTileset`** Get a [3D Tiles](https://github.com/CesiumGS/3d-tiles/tree/main/specification) tileset description.
-   **`get3DTile`** Get a single 3D tile from a tileset.

**Exports**

-   **`createExport`** Start a data export operation.
-   **`getExportStatus`** Poll for export progress.
-   **`downloadExportUrl`** Download a finished export.

## Usage example

```js
import { LumiDB } from "@lumidb/lumidb";

const lumidb = new LumiDB({ baseUrl: "<LUMIDB_URL>", auth: { apikey: "<LUMIDB_APIKEY>" } });

// Get a list of accessible database tables
const tables = await lumidb.listTables();

// Query points from a table
const response = await lumidb.query({
    tableName: tables[0].tableName,
    queryBoundary: {
        Polygon: [
            // polygon boundary
            [
                [
                    [2777342, 8438905],
                    [2777342, 8439315],
                    [2777573, 8439315],
                    [2777573, 8438905],
                    [2777342, 8438905],
                ],
            ],
            // optional z range
            [0, 100],
        ],
    },
    queryProj: "EPSG:3857",
    maxPoints: 5_000_000,
    maxDensity: 10.0,
    sourceFileFilter: null, // see below
    classFilter: null,
    outputType: "threejs", // "laz" or "e57" for exporting to other software
});

console.log(`LumiDB responded with ${response.pointCount} points and ${response.bytes} bytes`);

// Load the points into three.js:

import * as THREE from "three";
import { PointCloudMaterial } from "@lumidb/lumidb";

const scene = new THREE.Scene();

const material = new PointCloudMaterial({ pointSize: 3.0 });
const points = new THREE.Points(response.pointGeometry, material);

scene.add(points);
```

For a more complete example, check out [a more complete example](https://github.com/lumidb/example-threejs-viewer) of
integrating LumiDB into a three.js -based viewer.

## Authentication

### Option 1: Server-Created Tokens (Recommended)

Enhanced security with fine-grained access control. Token-based auth gives you full control of what each client is allowed to access.

You must implement a custom authentication endpoint on your server and configure the LumiDB client to use it for fetching tokens.

```typescript
// Example callback function:
async function getLumiDbTokenExample(): string {
    // Call a custom endpoint on your server. You can re-use existing logic for authenticated API calls.
    const response = await fetch("/path/to/your/endpoint", {
        headers: {
            /* your auth headers */
        },
        body: JSON.stringify({
            /* extra info when using POST */
        }),
    });

    // The server should:
    // 1. Run it's own auth logic
    // 2. If request is allowed, call the lumidb /api/create_token. See HTTP API documentation further down.
    // 3. Pass the lumidb token back to with the reponse

    // Parse the token from your server response
    const body = await response.json();
    const token = body.lumidb_token;

    return token;
}

// Register the callback
lumidb.setAuthentication({ getTokenCallback: getLumiDbTokenExample });

// The client ill now automatically refresh tokens as needed before making API calls.
```

### Option 2: Direct API key

Simplest approach, but the client is provided full access to all resources.

```
lumidb.setAuthentication({ apikey: "<LUMIDB_APIKEY>"});
```

#### ⚠️ Security Considerations:

-   Grants full scope access to your tables
-   Not recommended for client-side applications
-   Appropriate for protected server environments only

## Filtering

Query filtering based on file metadata is a central concept in the API. You can get the metadata for a table using
`getMetadata` or `getTableMetadata`, which returns a `TableMetadata` object containing both table-level and file-level metadata. Each file entry contains the metadata given by the user at the time of import, in addition to metadata augmented by LumiDB automatically, such as file names, sizes, point counts and so on. The file metadata is stored in a map where the keys correspond to each files **source file ID**.

These IDs can then be passed in an array as the `sourceFileFilter` whenever calling `LumiDB.query()` to filter the
point data on the server to contain only points coming from files corresponding to the given IDs.

The same mechanism will work on any metadata, such as scanner type, author name, vehicle identifier etc.

### Example: By date

As a practical example, let's say you want to filter the points based on a scan time/date range. For each
file, at import you would provide a metadata field, such as `scanDate` with a value corresponding to the time at
which the file was scanned.

Then in your application, you'd call `getTableMetadata()` and choose the valid source file IDs that are within
your specified time/date range of interest. Then, you'd pass this list of IDs in the `sourceFileFilter` when
calling `query()`, and the resulting set of points will only contain points scanned within the specified time span.

## LumiDB HTTP API

### POST /api/create_token

Create a LumiDB token that can be used for authentication. The token will have
access to the specified tables.

#### Request

##### Headers

| Header        | Value                  |
| ------------- | ---------------------- |
| Content-Type  | application/json       |
| Authorization | apikey <LUMIDB_APIKEY> |

##### Body Parameters

| Parameter | Type     | Required | Description                                         |
| --------- | -------- | -------- | --------------------------------------------------- |
| `tables`  | string[] | Yes      | List of tables that the token gives read access to. |

##### Example Request

```json
{
    "tables": ["tokyo", "new_york"]
}
```

#### Response

```json
{
    "lumidb_token": "<LUMIDB_TOKEN>"
}
```

### POST /api/assets/upload

Create a presigned upload URL for uploading an asset to LumiDB. This endpoint generates a unique asset ID and returns a presigned URL that can be used to upload the asset file directly.

> [!NOTE]
> The API only supports only uploading files up to 5GB.

#### Request

##### Headers

| Header        | Value                  |
| ------------- | ---------------------- |
| Content-Type  | application/json       |
| Authorization | apikey <LUMIDB_APIKEY> |

##### Body Parameters

| Parameter  | Type                | Required | Description                                    |
| ---------- | ------------------- | -------- | ---------------------------------------------- |
| `name`     | string              | Yes      | Name of the asset being uploaded.              |
| `metadata` | object              | No       | Optional metadata associated with the asset.   |

##### Example Request

```json
{
    "name": "my-pointcloud-data.laz",
    "metadata": {
        "source": "drone_survey",
        "date": "2024-01-15",
        "location": "helsinki"
    }
}
```

#### Response

```json
{
    "upload_url": "<PRESIGNED UPLOAD URL>",
    "asset_id": "01961997-4e23-7962-aeac-f0ddc706b402",
    "expiration_seconds": 3600
}
```

### GET /api/assets

List all assets available in the LumiDB instance. Returns metadata for all uploaded assets.

#### Request

##### Headers

| Header        | Value                  |
| ------------- | ---------------------- |
| Authorization | apikey <LUMIDB_APIKEY> |

#### Response

```json
{
    "assets": [
        {
            "asset_id": "01961997-4e23-7962-aeac-f0ddc706b402",
            "name": "my-pointcloud-data.laz",
            "started_at": "2024-01-15T10:30:00Z",
            "metadata": {
                "source": "drone_survey",
                "date": "2024-01-15",
                "location": "helsinki"
            }
        }
    ]
}
```

### POST /api/tables/import

Import a set of files into a LumiDB table.

> [!NOTE]
> Calling import on an already existing table will create a new version of the table, overwriting the previous version.

#### Request

##### Headers

| Header        | Value                  |
| ------------- | ---------------------- |
| Content-Type  | application/json       |
| Authorization | apikey <LUMIDB_APIKEY> |

##### Body parameters

```typescript
{
	"table_name": string,
	"table_proj": string,
	
	// If true, the import will fail if any of the input files have incompatible projections.
	// Defaults to false.
	"strict_mode": false,
	
	// Optional valid bounds for the table. Any assets outside these bounds will be rejected during import.
	// This can be useful to skip inputs where the projection information is incorrect.
	// Depending on `strict_mode` setting, the import will either fail or assets outside the bounds will be ignored.
	// Defaults to null
	"valid_bounds": {
	    // [xmin, ymin, xmax, ymax]
	    "extent": [24, 61, 26, 63],
		"proj": "EPSG:4326"
	},
	
	"inputs": {
	    // NOTE: The `src` and `id` fields are mutually exclusive. You must provide either a source URL or an asset ID.

		// Reference to an asset ID if the file is already uploaded to LumiDB.
		"asset_id": "<ASSET_ID>",

		// Location for the input file (e.g. a presigned S3 url).
		// LumiDB will automatically fetch the file from this URL.
		"src": string,

		// File projection. EPSG code or PROJ.4 string.
		"proj": string,

		// rest of the per-file fields are treated as user-defined metadata
		// common ones could be `date`, `license`, `producer`, etc..
	}[]
}
```

##### Example Request

```json
{
    "table_name": "helsinki",
    "table_proj": "EPSG:3879",
    "inputs": [
        {
            "src": "https://ptp.hel.fi/DataHandlers/Lidar_kaikki/Default.ashx?q=673496b&y=2021",
            "proj": "EPSG:3879",
            "date": "2017",
            "license": "Helsingin kaupunki"
        }
    ]
}
```

##### Response

```json
{
    "status": "queued",
    "table_version": "01961997-4e23-7962-aeac-f0ddc706b402"
}
```

### GET /api/tables/import_status/:version_id

Get the status of an import job. Can be polled to track the progress of an ongoing import.
The `version_id` parameter is the ID returned in the response of the `/api/tables/import` endpoint.

#### Request

##### Headers

| Header        | Value                  |
| ------------- | ---------------------- |
| Authorization | apikey <LUMIDB_APIKEY> |

#### Response

```typescript
{
  status: "queued" | "processing" | "failed" | "ready",
  progress?: number
  error_msg?: string
}
```

##### Example

```json
{
    "status": "processing",
    "progress": 72.25
}
```

## Changelog

### v0.7.2

-   Support non-ASCII characters in table names.
-   Support `glTF` format for 3D Tiles.
-   Add `patchTableUserData()` for updating user-defined metadata.
-   `get3DTileset`: Support creating a single 3D tileset from multiple datasets.
-   `createExport`: Add `outputOrigin` option to modify the georeferencing of exported data.

### v0.7.1

- Fix incorrect JWT caching behaviour when authenticating with server-created tokens.

### v0.7.0

-   `getMetadata`: New function for fetching metadata for multiple tables with optional spatial filtering.
-   `getTableMetadata`: Added support for spatial filtering.
-   Restructured table metadata format:
    - Added more system-created metadata fields
    - Separated user-defined metadata into its own dedicated object
    - Table boundaries now returned as GeoJSON MultiPolygon

### v0.6.0

-   Improve TypeScript configuration for better compatibility across tools and bundlers.
-   New Export API
-   getTableMetadata: return table boundaries as GeoJSON MultiPolygons
-   getTableMetadata: return asset id's for all files in a table

### v0.5.4
-   Update license to [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0.txt)
-   Added validation to all `LumiDB` methods.

### v0.5.3

-   get3DTileset: Add `tileId` for each 3D tile.
-   get3DTileset: Require outputProj to be defined. Use `null` with Cesium.
-   Remove unnecessary console logging.

### v0.5.2

-   Add support for [3D Tiles](https://github.com/CesiumGS/3d-tiles/tree/main/specification)
-   `listTables`, `getTableMetadata`: Return table bounding box.

### v0.5.1

-   Fix example code in the README

### v0.5.0

-   Support for token-based authentication
-   Projections can be defined as PROJ.4 strings in addition to EPSG codes.
-   Add LumiDBError and LumiDBHttpError

### v0.4.0

-   Change HTTP query parameters to be compatible with updated HTTP API.
-   query: Add option to limit Z range when query area is a polygon.
-   query: Return the bounding box with query responses.
-   query: Add processing time to the JSON header.
-   listMetadata: return sensor_poses (for E57 files)

### v0.3.0

-   Add JSON header to point data response to enable better backward compatibility for future updates.
-   Return intensity range and classification list with point response.

### v0.2.3

-   Allow newer version of three.js to be used as peer dependency
-   Return query result offset as an array instead of THREE.Vector3

### v0.2.2

-   Make three.js peer dependency version limit less strict (from `^0.169.0` to `^0.152.0`)

### v0.2.1

-   Add missing auth header to `getDownloadUrl`

### v0.2.0

-   First public release