open-index/open-npm huggingface dataset:
npm package: @microfox/datamotion
Description: A TypeScript SDK for Datamotion.
Latest version: 1.0.1

# Datamotion

<!-- Add your project description here -->
A TypeScript SDK for Datamotion.

## Installation

\`\`\`bash
npm install @microfox/datamotion
\`\`\`

## Quick Start

\`\`\`typescript
import { Datamotion } from '@microfox/datamotion';

// Initialize the SDK
const sdk = new Datamotion({
  apiKey: 'your-api-key',
  // Add other config options
});

// Example usage
const result = await sdk.hello('World');
console.log(result.data);
\`\`\`

## Configuration

\`\`\`typescript
interface DatamotionConfig {
  apiKey: string;        // Required: Your API key
  baseUrl?: string;      // Optional: Custom base URL
  name?: string;         // Optional: SDK instance name
  version?: string;      // Optional: API version
}
\`\`\`

## Methods

### hello(name: string)

Example method - replace with your own API methods.

\`\`\`typescript
const result = await sdk.hello('World');
// Returns: { data: "Hello, World!", success: true, status: 200, message: "Success" }
\`\`\`

<!-- 
## TODO: Add your API documentation here

### getData(id: string)
\`\`\`typescript
const data = await sdk.getData('123');
\`\`\`

### createItem(item: object)
\`\`\`typescript
const result = await sdk.createItem({ name: 'Example' });
\`\`\`
-->

## Development

\`\`\`bash
# Install dependencies
npm install

# Build the project
npm run build

# Run tests
npm test

# Lint code
npm run lint
\`\`\`

## License

MIT
npm package: @microfox/db-upstash
Description: A TypeScript SDK for Db Upstash.
Latest version: 1.2.0

# @microfox/db-upstash

A TypeScript SDK for interacting with Upstash Redis, providing helpful utilities and classes.

## Features

- **Paginator**: A generic class for managing the state of long-running pagination or indexing tasks.

## Installation

```bash
npm install @microfox/db-upstash @upstash/redis
```

## Usage

### Paginator

The `Paginator` class helps you track the progress of background jobs, scrapers, or any task that involves pagination. It can store, update, pause, and resume stateful progress information in Redis.

#### Example

```typescript
import { Redis } from '@upstash/redis';
import { Paginator } from '@microfox/db-upstash';

// Define the structure for your task's progress
interface ScraperProgress {
  page: number;
  itemsScraped: number;
}

const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL!,
  token: process.env.UPSTASH_REDIS_REST_TOKEN!,
});

const scraperId = 'user-profiles-scraper';
const paginator = new Paginator<ScraperProgress>(redis, scraperId);

async function runScraper() {
  // Start a new scraping task or resume a previous one
  let status = await paginator.getCurrentStatus();
  if (!status) {
    console.log('Starting new scraper job...');
    status = await paginator.startNewIndexing({ page: 1, itemsScraped: 0 });
  } else if (status.status === 'paused') {
    console.log('Resuming scraper job...');
    status = await paginator.resumeIndexing();
  }

  if (!status || status.status === 'completed' || status.status === 'failed') {
    console.log('Job is already completed or has failed.');
    return;
  }

  console.log(`Scraping page ${status.progress.page}...`);

  // Simulate scraping
  await new Promise((resolve) => setTimeout(resolve, 2000));
  const newItems = 10;

  // Update the progress
  const updatedStatus = await paginator.updateIndexingStatus({
    page: status.progress.page + 1,
    itemsScraped: status.progress.itemsScraped + newItems,
  });

  console.log('Progress updated:', updatedStatus.progress);

  // Complete the job after a few pages
  if (updatedStatus.progress.page > 5) {
    await paginator.completeIndexing();
    console.log('Scraping job completed!');
  }
}

runScraper().catch(console.error);
```

For detailed API documentation, please refer to the files in the `docs/` directory.