npm package: @qtsurfer/lastra
Description: TypeScript reader for the Lastra columnar time series format
Author: QTSurfer
Latest version: 0.9.1

# @qtsurfer/lastra

[![CI](https://github.com/QTSurfer/lastra-ts/actions/workflows/ci.yml/badge.svg)](https://github.com/QTSurfer/lastra-ts/actions/workflows/ci.yml)
[![npm](https://img.shields.io/npm/v/@qtsurfer/lastra)](https://www.npmjs.com/package/@qtsurfer/lastra)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)

TypeScript reader for the [Lastra](https://github.com/QTSurfer/lastra-java) columnar time series format.

## Features

- Read `.lastra` files written by the [Java LastraWriter](https://github.com/QTSurfer/lastra-java)
- Selective column access — only requested columns are decompressed
- Built-in decoders: ALP, Gorilla, Pongo (doubles), delta-varint (timestamps), ZSTD/gzip (binary)
- **Apache Arrow interop** — convert to Arrow Table for use with DuckDB-WASM, Perspective, Arquero, etc.
- Browser + Node.js compatible
- Zero-copy `Float64Array` output for decoded doubles

## Install

```bash
npm install @qtsurfer/lastra
```

## Usage

```typescript
import { LastraReader } from '@qtsurfer/lastra';

// From fetch response
const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// Series data
const ts = reader.readSeriesLong('ts');           // Float64Array (timestamps)
const close = reader.readSeriesDouble('close');    // Float64Array (prices)

// Column metadata
const meta = reader.getSeriesColumn('ema1').metadata;
// { indicator: 'ema', periods: '10' }

// Events (signals with independent timestamps)
const signalTs = reader.readEventLong('ts');
const types = reader.readEventBinary('type');      // (Uint8Array | null)[]
const data = reader.readEventBinary('data');       // ZSTD-compressed JSON

// Decode event strings
const decoder = new TextDecoder();
types.forEach((t, i) => {
  if (t) console.log(decoder.decode(t), data[i] ? JSON.parse(decoder.decode(data[i]!)) : null);
});
```

## API

### `LastraReader`

| Property / Method | Returns | Description |
|---|---|---|
| `seriesRowCount` | `number` | Number of series rows |
| `eventsRowCount` | `number` | Number of event rows |
| `seriesColumns` | `ColumnInfo[]` | Series column descriptors |
| `eventColumns` | `ColumnInfo[]` | Event column descriptors |
| `readSeriesLong(name)` | `Float64Array` | Decode a LONG series column |
| `readSeriesDouble(name)` | `Float64Array` | Decode a DOUBLE series column |
| `readSeriesBinary(name)` | `(Uint8Array\|null)[]` | Decode a BINARY series column |
| `readEventLong(name)` | `Float64Array` | Decode a LONG event column |
| `readEventDouble(name)` | `Float64Array` | Decode a DOUBLE event column |
| `readEventBinary(name)` | `(Uint8Array\|null)[]` | Decode a BINARY event column |
| `readEventBinaryAsync(name)` | `Promise<...>` | Async decode (native gzip in browser) |
| `getSeriesColumn(name)` | `ColumnInfo` | Get column descriptor with metadata |
| `getEventColumn(name)` | `ColumnInfo` | Get event column descriptor |

### `ColumnInfo`

```typescript
interface ColumnInfo {
  name: string;
  dataType: DataType;
  codec: Codec;
  metadata: Record<string, string>;
}
```

## Codec Support

| Codec | Type | Description | Bundle cost |
|---|---|---|---|
| `DELTA_VARINT` | LONG | Delta-of-delta + zigzag varint (timestamps) | 0 kB |
| `ALP` | DOUBLE | Adaptive lossless floating-point (~3-4 bits/value) | 0 kB |
| `GORILLA` | DOUBLE | XOR compression (Facebook VLDB 2015) | 0 kB |
| `PONGO` | DOUBLE | Decimal-aware erasure + Gorilla XOR (best for prices) | 0 kB |
| `VARLEN` | BINARY | Variable-length encoding | 0 kB |
| `VARLEN_ZSTD` | BINARY | Variable-length + ZSTD | ~3.8 kB gz |
| `VARLEN_GZIP` | BINARY | Variable-length + gzip (async, native `DecompressionStream`) | 0 kB |
| `RAW` | LONG/DOUBLE | Uncompressed / `Float64Array` zero-copy | 0 kB |

## Arrow Interop

Convert Lastra data to an Apache Arrow Table for integration with any Arrow-compatible tool.

```typescript
import { LastraReader, lastraToArrow, lastraToArrowColumns } from '@qtsurfer/lastra';

const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// All series columns
const table = lastraToArrow(reader);

// Selected columns only (others are not decompressed)
const partial = lastraToArrowColumns(reader, ['t', 'cls', 'vol']);

// One-liner: buffer → Arrow Table
import { readLastraAsArrow } from '@qtsurfer/lastra';
const table2 = readLastraAsArrow(buffer);
```

The resulting Arrow Table can be registered in DuckDB-WASM, used with Perspective, Arquero, Observable Plot, or any tool that accepts Arrow IPC.

## Cross-Language Testing

Test fixtures are generated by the [Java LastraWriter](https://github.com/QTSurfer/lastra-java) and read by the TS reader, ensuring wire format compatibility. Includes real BTC/USDT ticker data with mixed codecs (ALP + Gorilla + Pongo).

## License

Copyright 2026 Wualabs LTD. Apache License 2.0 — see [LICENSE](LICENSE).