open-index/open-npm huggingface dataset:
npm package: @shivamkj/shell-util
Description: Extremely simple, small, and opinionated helpers that make it easy to write complex shell scripts in NodeJS
Author: Shivam Kumar Jha
Latest version: 1.0.0

# Shell Util

[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](https://opensource.org/license/mit)

Extremely simple, small, and opinionated helpers that make it easy to write complex shell scripts in NodeJS. Just, a tiny utility function around child_process to reduce boilerplate for running commands from NodeJS script. Also, includes some commonly used packages like "fast-glob", "minimist", "node-fetch" etc.

### Contributing

We welcome and appreciate contributions from the community! Whether you're fixing a bug, implementing a new feature, or improving documentation, your efforts make this project better for everyone. Please follow the existing code style and formatting conventions. Use a consistent style throughout the project.

### Copyright

Copyright (©) 2024 - Shivam Kumar Jha

MIT Licensed. Check LICENSE for more details.
npm package: @shivamsharma11/readme-live
Description: A lightweight CLI for 1:1 GitHub-flavored README previews with live-reload
Author: Shivam Sharma
Latest version: 1.0.4

# @shivam/readme-live 

A lightweight, zero-config CLI tool to preview your GitHub README files locally with **instant live-reload**. See exactly how your README will look on GitHub without constant committing and pushing.



##  Features
* **GitHub-Flavored Markdown (GFM):** Uses the actual GitHub API for 1:1 accuracy.
* **Live Reload:** Instant updates in the browser using WebSockets.
* **Dark Mode:** Built-in GitHub Dark Theme styling.
* **Token Support:** Support for GitHub Personal Access Tokens to avoid rate limits.

##  Installation
You can run it instantly without installation:
```bash
npx @shivamsharma11/readme-live
npm package: @shivanandasai/emoji-tokenizer
Description: A simple emoji tokenizer for encoding and decoding text into emoji sequences.
Author: Shivananda Sai
Latest version: 1.1.1

# Custom Tokenizer JS

A JavaScript tokenizer that learns vocabulary from text, supports ENCODE/DECODE, and handles special tokens. Includes a CLI demo with emoji sequence encoding for large vocabularies.

https://github.com/user-attachments/assets/a4da855b-978e-4abf-988f-a9d6434cd28b

## Features

- Learns vocab from any input text
- Handles special tokens (customizable)
- ENCODE: Converts text to token IDs and emoji sequences
- DECODE: Converts token IDs or emoji sequences back to text
- CLI demo for interactive usage
- Supports large vocabularies using repeated emoji sequences

## Setup

1. Clone the repo:
   ```sh
   git clone <your-repo-url>
   cd emoji-tokenizer
   ```
2. Install dependencies (none required for core demo):
   ```sh
   pnpm install
   # or
   npm install
   ```

## Usage

### CLI Demo

Run the demo:

```sh
pnpm start
# or
node demo.js
```

#### Example CLI

- Learn vocab from text
- Encode text to emoji sequences (e.g., "⚡⚡ 💎💎💎")
- Decode emoji sequences back to text
- Show vocab table with token, ID, and emoji sequence
- Vocab is auto-saved to `vocab.json`

### API Usage

```js
import CustomTokenizer from "@shivanandasai/emoji-tokenizer";

const tokenizer = new CustomTokenizer();
tokenizer.learnVocabFromText("hello world");
const { rawIds, funEncoded } = tokenizer.encode("hello world");
console.log(rawIds, funEncoded); // e.g., [1,2], ["⚡", "💎"]
const text = tokenizer.decode(rawIds); // decode from IDs
const textFromEmoji = tokenizer.decode(funEncoded); // decode from emoji sequences
console.log(text, textFromEmoji);
```

## Customization

- Change special tokens in `custom-tokenizer.js`.
- Extend vocab learning logic for more advanced tokenization.
- Emoji mapping logic can be customized for different visual styles.

## License

MIT