npm package: @layer-drone/image-matching
Description: Image matching utilities for Layer Drone
Latest version: 0.0.5

# @layer-drone/image-matching

A TypeScript library for matching drone flight images with their provenance data. This package matches captured drone images (with EXIF metadata) to their intended flight path data, enabling validation of drone imagery.

## Overview

The `ImageMatcher` class processes drone flight provenance data and matches it with actual captured images. It supports multiple DJI drone models and handles different filename formats and path structures.

**Key Features:**

- Match drone images to flight path actions using EXIF data
- Support for DJI Mini 2, 3, 3 Pro, and 4 Pro
- File size-based disambiguation for multiple matches
- Export matches as structured data

### Matching Steps

1. **Parse Flight Plan** - Extract features (products) and their actions from `FlightPathIntended`
2. **Match Controller Logs** - Link `ControllerRequestLog` and `ControllerResponseLog` entries by tag and action index
3. **Generate Paths** - Convert DJI file indices to expected file paths based on drone model
4. **Create Match Entries** - Combine intended actions with actual controller logs
5. **Associate EXIF Data** - Match captured image metadata to specific response logs using filename patterns and file sizes

## Installation

```bash
pnpm add @layer-drone/image-matching
```

## Usage

### Basic Example

```typescript
import { createMatcher } from "@layer-drone/image-matching";

// Load your provenance data
const provenanceData = {
  FlightPathIntended: {
    type: "FeatureCollection",
    features: [...],
    metadata: {
      flightPlanType: "panorama",
      flightPlanId: 1,
      zoneHash: "abc123",
      imageCount: 17
    }
  },
  ControllerResponseLogs: [...],
  // ... other optional data
};

// Create matcher for DJI Mini 3 Pro
const matcher = createMatcher(provenanceData, "3");

// Get all matches
const matches = matcher.getMatches();

// Match images with EXIF data
const exifData = [
  {
    filename: "DJI_0851.JPG",
    fileSize: 5077938,
    location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
    timeOfCreation: "2025-10-03T14:12:54Z"
  }
];

const { unmatchedExifs } = matcher.setExifData(exifData);
console.log(`Unmatched: ${unmatchedExifs.length}`);

// Find a specific match
const result = matcher.getMatchFromExifData(exifData[0]);
if (result) {
  const { matchEntry, responseLogIndex } = result;
  console.log(`Matched to ${matchEntry.responseLog[responseLogIndex].path}`);
}
```

## API Reference

### `ImageMatcher`

Main class for performing image matching operations.

#### Constructor

```typescript
new ImageMatcher(provenanceData: ProvenanceData, droneModel: "1" | "2" | "3" | "4")
```

**Parameters:**

- `provenanceData` - Flight provenance data containing flight path and controller logs
- `droneModel` - DJI drone model:
  - `"1"` - DJI Mini 2
  - `"2"` - DJI Mini 3
  - `"3"` - DJI Mini 3 Pro
  - `"4"` - DJI Mini 4 Pro

#### Methods

##### `getDroneModel(): { droneModel: DroneModelType }`

Returns the drone model type.

```typescript
const { droneModel } = matcher.getDroneModel();
// => { droneModel: "DJI_MINI_3_PRO" }
```

##### `matchProvenanceData(): Map<string, MatchEntry>`

Processes provenance data and creates matches between flight actions and controller response logs. Called automatically in constructor.

##### `getMatches(): Map<string, MatchEntry> | undefined`

Returns all matched entries. Each key is a unique identifier like `"pano-1.0"` (product-action).

##### `setExifData(exifData: ExifData[]): { unmatchedExifs: ExifData[] }`

Matches and associates EXIF data from captured images to their corresponding response logs in provenance entries. Returns a list of EXIF entries that could not be matched.

```typescript
const { unmatchedExifs } = matcher.setExifData([
  {
    filename: "DJI_0851.JPG",
    fileSize: 5077938,
    location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
    timeOfCreation: "2025-10-03T14:12:54Z",
  },
]);

if (unmatchedExifs.length > 0) {
  console.log(`Failed to match ${unmatchedExifs.length} images`);
}
```

##### `getMatchFromExifData(exifData: ExifData): { matchEntry: MatchEntry; responseLogIndex: number } | undefined`

Finds a specific response log match for the given EXIF data. Returns an object containing the matched entry and the index of the response log, or `undefined` if no match or ambiguous match.

```typescript
const result = matcher.getMatchFromExifData(exifData);
if (result) {
  const { matchEntry, responseLogIndex } = result;
  const responseLog = matchEntry.responseLog[responseLogIndex];
  console.log(`Matched to: ${responseLog.path}`);
  console.log(`File size: ${responseLog.fileSize}`);
}
```

**Matching Logic:**

- **DJI Mini 4 Pro**: Matches pattern `DJI_*_XXXX_D.JPG` by extracting the 4-digit number
- **Other Models**: Matches `DJI_XXXX.JPG` pattern, handles timestamp prefixes
- Uses fileSize as tiebreaker when multiple response logs match the same pattern

##### `getResponseLogsWithoutExif(): ControllerResponseLogWithPath[]`

Returns response logs from provenance data that have no associated EXIF data.

##### `getZoneHash(): string`

Returns the zone hash from the flight path metadata.

##### `static djiIndexToPath(index: number, droneModel: keyof typeof DroneModel): string`

Converts DJI file index to expected file path format. Can be called without instantiating the class. The `droneModel` parameter must be one of: `"1"`, `"2"`, `"3"`, or `"4"`.

##### `getFilePaths(): string[]`

Returns all file paths from matched entries.

##### `getActionsWithoutMatches(): Map<string, MatchEntry>`

Returns actions that have no controller response logs.

##### `getActionsWithSingleMatch(): Map<string, MatchEntry>`

Returns actions that have exactly one controller response log.

##### `getActionsWithMultipleMatches(): Map<string, MatchEntry>`

Returns actions that have multiple controller response logs.

##### `getProvenanceData(): ProvenanceData`

Returns the complete provenance data used by the matcher.

### `createMatcher(provenanceData: ProvenanceData, droneModel: "1" | "2" | "3" | "4", exifData: ExifData[]): ImageMatcher`

Factory function to create a new ImageMatcher instance.

## Data Structures

### Tags

Each entry has an ID composed of three parts: the intended product name, a sequential ID referring to the action's location, and (for actions requiring multiple images, like a pano) a local index number within that action.

`<product-type>-<secuence-id>.<local-id>`

### MatchEntry

Represents a matched flight action with its associated data.

```typescript
interface MatchEntry {
  intendedAction: {
    productType: string; // e.g., "pano", "map", "gridMap"
    productId: number; // Product identifier
    geom?: Geometry; // GeoJSON geometry of the action point
    action: FlightAction; // Flight action details (type, yawAngle, gimbalPitchAngle)
  };
  responseLog: ControllerResponseLogWithPath[] | undefined; // Controller responses with file paths and EXIF data
  requestLog: ControllerRequestLog | undefined; // Controller request log
}

interface ControllerResponseLogWithPath extends ControllerResponseLog {
  path: string; // Generated file path based on DJI index
  exifData?: ExifData; // Associated EXIF data from captured image (if matched)
}
```

**Example:**

```typescript
{
  intendedAction: {
    productType: "pano",
    productId: 1,
    geom: {
      type: "Point",
      coordinates: [-71.415057, 41.840082, 82.8]
    },
    action: {
      type: "photo",
      yawAngle: 0,
      gimbalPitchAngle: -90
    }
  },
  responseLog: [
    {
      upLinkQuality: 100,
      timeCreated: 1696348374000,
      tag: "pano-1",
      mediaType: "photo",
      index: 6815745,
      fileSize: 5077938,
      path: "/DCIM/104MEDIA/DJI_0851.JPG",
      gimbalAttitude: { yaw: 0, roll: 0, pitch: -90 },
      aircraftLocation: { longitude: -71.415057, latitude: 41.840082, altitude: 82.8 },
      aircraftAttitude: { yaw: 0, roll: 0, pitch: 0 },
      actionIndex: 0,
      exifData: {
        filename: "DJI_0851.JPG",
        fileSize: 5077938,
        location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
        timeOfCreation: "2025-10-03T14:12:54Z"
      }
      // ... other fields
    }
  ],
  requestLog: {
    tag: "pano-1",
    actionIndex: 0,
    type: "photo",
    // ... other fields
  }
}
```

### ProvenanceData

Flight provenance data structure containing all information about the intended flight and captured data.

```typescript
interface ProvenanceData {
  FlightPathIntended: {
    type: "FeatureCollection";
    features: Array<{
      type: "Feature";
      properties: {
        actions: FlightAction[]; // Actions to perform at this point
        desiredFlyingHeightAboveGround: number;
        flyingHeightRelativeToTakeOff: number;
        takeOffOffset: number;
        flightSpeedMs: number;
        isFlyThrough: boolean;
        isFlightLineEnd: boolean;
        isFlightLineStart: boolean;
        absoluteAltitudeAtGroundLevel: number;
        takeOffCoordinates: Coordinates;
        intendedAltitudePoints?: IntendedAltitudePoint[];
      };
      geometry: {
        type: "Point";
        coordinates: [number, number, number]; // [lon, lat, alt]
      };
    }>;
    metadata: {
      flightPlanType: string; // e.g., "panorama", "map", "gridMap"
      flightPlanId: number;
      zoneHash: string;
      imageCount: number;
    };
  };
  ImageList?: ImageList; // Optional image metadata
  FlightMetadata?: FlightMetadata; // Optional flight metadata
  ControllerRequestLogs?: ControllerRequestLog[]; // Optional request logs
  ControllerResponseLogs?: ControllerResponseLog[]; // Optional response logs
}
```

**Example:**

```typescript
{
  FlightPathIntended: {
    type: "FeatureCollection",
    features: [
      {
        type: "Feature",
        properties: {
          actions: [
            {
              type: "photo",
              yawAngle: 0,
              gimbalPitchAngle: -90
            }
          ],
          desiredFlyingHeightAboveGround: 50,
          flyingHeightRelativeToTakeOff: 50,
          takeOffOffset: 0,
          flightSpeedMs: 5,
          isFlyThrough: false,
          isFlightLineEnd: false,
          isFlightLineStart: false,
          absoluteAltitudeAtGroundLevel: 100,
          takeOffCoordinates: {
            longitude: -71.415057,
            latitude: 41.840082,
            altitude: 100
          }
        },
        geometry: {
          type: "Point",
          coordinates: [-71.415057, 41.840082, 150]
        }
      }
    ],
    metadata: {
      flightPlanType: "panorama",
      flightPlanId: 1,
      zoneHash: "abc123def456",
      imageCount: 17
    }
  },
  ControllerResponseLogs: [
    {
      upLinkQuality: 100,
      timeCreated: 1696348374000,
      tag: "pano-1",
      mediaType: "photo",
      index: 6815745,
      fileSize: 5077938,
      // ... other fields
    }
  ]
}
```

### ExifData

EXIF metadata extracted from captured images.

```typescript
interface ExifData {
  location: {
    lat: number; // Latitude in decimal degrees
    lon: number; // Longitude in decimal degrees
    alt: number; // Altitude in meters
  };
  timeOfCreation: string; // ISO 8601 timestamp
  fileSize: number; // File size in bytes
  filename: string; // Original filename
}
```

**Example:**

```typescript
{
  location: {
    lat: 41.840082361111115,
    lon: -71.41505761111112,
    alt: 82.8
  },
  timeOfCreation: "2025-10-03T14:12:54Z",
  fileSize: 5077938,
  filename: "DJI_0851.JPG"  // or "1759531051938_DJI_0851.JPG" with timestamp prefix
}
```

## Drone Model Differences

### DJI Mini 4 Pro (Model "4")

- **Path Format**: `/DCIM/DJI_XXX/DJI_*_YYYY_D.JPG`
- **Filename Format**: `TIMESTAMP_DJI_TIMESTAMP_YYYY_D.JPG`
- **Matching**: Extracts 4-digit number (YYYY) from both path and filename

### Other Models (DJI Mini 2, 3, 3 Pro)

- **Path Format**: `/DCIM/XXXmedia/DJI_YYYY.JPG`
- **Filename Format**: `DJI_YYYY.JPG` or `TIMESTAMP_DJI_YYYY.JPG`
- **Matching**: Extracts `DJI_YYYY.JPG` pattern from both

## Development

### Build

```bash
pnpm build
```

### Development Mode (watch)

```bash
pnpm dev
```

### Run Tests

```bash
pnpm test
```

### Run Tests with Coverage

```bash
pnpm coverage
```

### Type Check

```bash
pnpm type-check
```

### Lint

```bash
pnpm lint
```

## License

MIT