npm package: @joelzangue/react-native-silver-train
Description: On-device ML inference for React Native - Run ONNX and CoreML models locally
Author: Joel Zangue
Latest version: 0.1.0

# react-native-silver-train

On-device ML inference for React Native. Run **any** ONNX or CoreML model locally on iOS with full Neural Engine support.

[![npm version](https://badge.fury.io/js/react-native-silver-train.svg)](https://badge.fury.io/js/react-native-silver-train)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Features

✨ **Universal Model Support** - Run ANY ONNX or CoreML model
- 🤖 **Text Generation** - LLMs (TinyLlama, GPT-2, Llama, etc.)
- 🖼️ **Image Classification** - MobileNet, ResNet, EfficientNet
- 🎯 **Object Detection** - YOLO, SSD, Faster R-CNN
- 📊 **Embeddings** - Sentence-BERT, CLIP, custom models
- 🔧 **Custom Models** - Bring your own ONNX/CoreML model

⚡ **Performance Optimized**
- iOS Neural Engine acceleration
- Native Swift implementation for CoreML
- ONNX Runtime integration
- Real-time performance monitoring

🎯 **Developer Friendly**
- Full TypeScript support
- Two APIs: Simple text generation + Generic tensor inference
- React hooks for easy integration
- Comprehensive documentation

## Installation

```bash
npm install react-native-silver-train
# or
yarn add react-native-silver-train
```

### Install peer dependencies:

```bash
npm install @react-native-async-storage/async-storage onnxruntime-react-native react-native-fs
# or
yarn add @react-native-async-storage/async-storage onnxruntime-react-native react-native-fs
```

### iOS Setup

```bash
cd ios && pod install
```

## Quick Start

### Text Generation (LLMs)

```typescript
import { MLInferenceManager } from 'react-native-silver-train';

const manager = MLInferenceManager.getInstance();

// Load your model
await manager.loadModel({
  id: 'tinyllama',
  name: 'TinyLlama',
  path: '/path/to/model.onnx',
  type: 'onnx',
  size: 1400000000,
});

// Run inference
const result = await manager.runInference('tinyllama', 'Hello, how are you?', {
  temperature: 0.8,
  maxTokens: 50,
});

console.log(result.text);
```

### Generic Tensor Inference (Any Model)

```typescript
import { MLInferenceManager } from 'react-native-silver-train';
import type { TensorInput } from 'react-native-silver-train';

const manager = MLInferenceManager.getInstance();

// Load image classification model
await manager.loadModel({
  id: 'mobilenet',
  name: 'MobileNet V2',
  path: '/path/to/mobilenet.onnx',
  type: 'onnx',
  size: 14000000,
  metadata: {
    modelType: 'image-classification'
  }
});

// Prepare input tensor (e.g., normalized image data)
const imageData = new Float32Array(224 * 224 * 3);
// ... fill with your preprocessed image data ...

const inputs: TensorInput[] = [{
  name: 'input',
  data: imageData,
  shape: [1, 224, 224, 3],
  type: 'float32'
}];

// Run inference
const result = await manager.runTensorInference('mobilenet', inputs);
const predictions = result.outputs['output'].data;
```

### Using React Hooks

```typescript
import { useMLInference, useSystemMetrics } from 'react-native-silver-train';

function MyComponent() {
  const { models, loadModel, loadState } = useMLInference();
  const metrics = useSystemMetrics(5000); // Update every 5 seconds

  return (
    <View>
      <Text>Battery: {metrics?.battery.level}%</Text>
      <Text>Memory: {(metrics?.memory.used / 1024 / 1024).toFixed(2)} MB</Text>
      <Text>Thermal: {metrics?.thermal.state}</Text>
    </View>
  );
}
```

## API Documentation

### Two Inference APIs

#### 1. Text Generation API (Optimized for LLMs)

```typescript
runInference(
  modelId: string,
  input: string,
  options?: InferenceOptions
): Promise<InferenceResult>
```

**Best for:** Language models, text generation, chat

#### 2. Generic Tensor API (Works with ANY model)

```typescript
runTensorInference(
  modelId: string,
  inputs: TensorInput[] | Record<string, TensorInput>
): Promise<GenericInferenceResult>
```

**Best for:** Image classification, object detection, embeddings, custom models

### Core Services

- **`MLInferenceManager`** - Main inference service
- **`ModelCacheManager`** - Model caching and management
- **`TinyLlamaTokenizer`** - Built-in tokenizer for Llama models

### React Hooks

- **`useMLInference()`** - Model loading and inference state
- **`useSystemMetrics(interval)`** - Real-time system monitoring

### Native Modules

- **`coreMLInference`** - CoreML model execution
- **`coreMLTextGeneration`** - Optimized text generation with CoreML
- **`gpt2Tokenizer`** - Native Swift BPE tokenizer (10x faster)
- **`performanceMonitor`** - System metrics monitoring

## Model Configuration

```typescript
interface ModelConfig {
  id: string;                   // Unique identifier
  name: string;                 // Display name
  path: string;                 // File path to model
  type: 'coreml' | 'onnx';     // Model type
  size: number;                 // File size in bytes

  // Optional: For text generation models
  maxTokens?: number;
  temperature?: number;
  topP?: number;

  // Optional: Model shape information
  inputShape?: number[];
  outputShape?: number[];
  inputNames?: string[];
  outputNames?: string[];

  metadata?: {
    modelType?: 'text-generation' | 'image-classification' | 'object-detection' | 'embedding' | 'custom';
    description?: string;
    version?: string;
  };
}
```

## Supported Model Types

| Model Type | Examples | API |
|------------|----------|-----|
| **Text Generation** | TinyLlama, GPT-2, Llama 2/3 | `runInference()` |
| **Image Classification** | MobileNet, ResNet, EfficientNet | `runTensorInference()` |
| **Object Detection** | YOLO, SSD, Faster R-CNN | `runTensorInference()` |
| **Embedding** | Sentence-BERT, CLIP | `runTensorInference()` |
| **Custom** | Your ONNX/CoreML model | `runTensorInference()` |

## Examples

### Image Classification

```typescript
// MobileNet example
const result = await manager.runTensorInference('mobilenet', [{
  data: imageData,
  shape: [1, 224, 224, 3],
  type: 'float32'
}]);

const predictions = result.outputs['output'].data as Float32Array;
const topClass = predictions.indexOf(Math.max(...predictions));
```

### Object Detection

```typescript
// YOLO example
const result = await manager.runTensorInference('yolo', [{
  data: imageData,
  shape: [1, 3, 640, 640],
  type: 'float32'
}]);

const boxes = result.outputs['boxes'].data;
const scores = result.outputs['scores'].data;
const classes = result.outputs['classes'].data;
```

### Embeddings

```typescript
// Sentence-BERT example
const result = await manager.runTensorInference('sentence-bert', {
  'input_ids': {
    data: new BigInt64Array(tokenIds),
    shape: [1, tokenIds.length],
    type: 'int64'
  },
  'attention_mask': {
    data: new BigInt64Array(attentionMask),
    shape: [1, attentionMask.length],
    type: 'int64'
  }
});

const embedding = result.outputs['last_hidden_state'].data;
```

## Advanced Usage

### Performance Monitoring

```typescript
// Check device capabilities
const capabilities = await manager.getDeviceCapabilities();
console.log('Neural Engine:', capabilities.hasNeuralEngine);

// Monitor during inference
const metrics = await manager.getSystemMetrics();
if (metrics.thermal.shouldThrottle) {
  console.warn('Device is hot, reducing inference frequency');
}
```

### Streaming Text Generation

```typescript
await manager.runStreamingInference(
  'tinyllama',
  'Write a story about',
  { maxTokens: 100 },
  (token, isDone) => {
    console.log(token);
    if (isDone) {
      console.log('Generation complete!');
    }
  }
);
```

### Model Caching

```typescript
import { ModelCacheManager } from 'react-native-silver-train';

const cacheManager = ModelCacheManager.getInstance();

// Get all cached models
const models = cacheManager.getAllCachedModels();

// Clear cache
await cacheManager.clearCache();

// Get cache size
const size = await cacheManager.getCacheSize();
```

## Documentation

- 📖 **[Generic Inference Guide](./GENERIC_INFERENCE.md)** - Complete guide for any model type
- 🎯 **[API Reference](#api-documentation)** - Full API documentation
- 💡 **[Examples](#examples)** - Code examples for common use cases

## Requirements

- iOS 13.0 or higher
- React Native 0.70+
- CocoaPods

## Performance

- **Neural Engine**: Automatic acceleration on iOS devices with Neural Engine (A11+)
- **Native Tokenizers**: 10x faster than JavaScript implementations
- **Optimized Inference**: Efficient memory management and thermal monitoring

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on:
- [Development workflow](CONTRIBUTING.md#development-workflow)
- [Sending a pull request](CONTRIBUTING.md#sending-a-pull-request)
- [Code of conduct](CODE_OF_CONDUCT.md)

## License

MIT

## Acknowledgments

- Built with [ONNX Runtime](https://onnxruntime.ai/)
- Powered by Apple's [CoreML](https://developer.apple.com/documentation/coreml)
- Created with [create-react-native-library](https://github.com/callstack/react-native-builder-bob)

---

**Made with ❤️ for the React Native community**