open-index/open-npm huggingface dataset:
npm package: @porifa/tokenizer
Description: Regex Based Tokenizer used in parsers developed by Porifa
Author: Aamir Sohail
Latest version: 0.2.0

# @porifa/tokenizer

A Regex Based Tokenizer for Parsers

## Description

This package provides a flexible and efficient tokenizer that utilizes regular expressions to break down text into tokens for parsing. It's primarily designed for use within parsers developed by [Porifa](https://github.com/porifa), but can be adapted for other parsing tasks as well.

## Features

-   **Regex-based tokenization**: Define token patterns using regular expressions for precise control.
-   **Token types**: Assign a unique type to each token for identification and processing.
-   **Trivia handling**: Attach additional data to tokens, such as comments or whitespace, for custom handling.
-   **Skippable tokens**: Specify token types to be ignored during parsing.
-   **Customizable tokenization logic**: Inject custom tokenization logic through a trivia function.
-   **EOF handling**: Detects the end of the input stream.
-   **Peeking**: Preview upcoming tokens without consuming them.
-   **Utility functions**: Build regular expressions from special characters for convenience.

## Installation

```bash
npm install @porifa/tokenizer
```

## Usage

1. **Import the Tokenizer class**:

```ts
import { Tokenizer, TokenDefinition } from '@porifa/tokenizer';
```

2. **Define token definitions**:

```ts
enum TokenKind {
    EOF = 'eof',
    UNRECOGNIZED = 'unrecognized',
    WHITESPACE = 'whitespace',
    IDENTIFIER = 'identifier',

    IF = 'if',
    ELSE = 'else',
    WHILE = 'while',
    FOR = 'for',
}

const keywordMap: Record<string, TokenKind> = {
    if: TokenKind.IF,
    else: TokenKind.ELSE,
    while: TokenKind.WHILE,
    for: TokenKind.FOR,
};

const tokenDefinitions: TokenDefinition<TokenKind>[] = [
    { regex: /if|else|while|for/, tokenMap: keywordMap },
    { regex: /[a-zA-Z_][a-zA-Z0-9_]*/, kind: 'identifier' },
    { regex: /\s+/, kind: 'whitespace' },
    // ... more token definitions
];
```

3. **Create a tokenizer instance**:

```ts
const tokenizer = new Tokenizer<TokenKind, { text: string }>(
    tokenDefinitions,
    [TokenKind.WHITESPACE],
    TokenKind.EOF,
    TokenKind.UNRECOGNIZED,
    (token, tokenizer) => ({ text: tokenizer.code.substring(token.start, token.start + token.length) })
);
```

4. **Provide input code**:

```ts
tokenizer.setInput(code);
```

5. **Iterate through tokens**:

```ts
while (!tokenizer.isEndOfFile()) {
    const token = tokenizer.nextToken();
    console.log(token.kind, token.triviaData?.text);
}
```

## Additional Information

-   Explore the Token and Tokenizer classes for detailed properties and methods.
-   Refer to the examples directory for usage in different parsing scenarios.
-   Consider contributing to the package for enhancements and bug fixes.

## Contributing

We welcome contributions to this package! Please follow our contribution guidelines.

## License

This package is licensed under the MIT License.
npm package: @poriyaalar/telemetry
Description: A lightweight telemetry service
Latest version: 0.1.1

📊 Telemetry Service for Firebase & Sentry

A lightweight telemetry service for React apps, supporting Firebase Analytics, Firebase Performance, and Sentry error tracking using the Firebase Modular SDK.

🚀 Features

✅ Firebase Analytics (logEvent, setUserId)
✅ Firebase Performance Monitoring
✅ Sentry Error Logging (captureException, setUser)
✅ Supports both JavaScript and TypeScript React Apps
✅ Tree-shakable (Reduces Bundle Size)

📦 Installation

Install via NPM:

npm install your-npm-package-name

🔥 Usage

1️⃣ Import the Package

import Telemetry from "your-npm-package-name";

2️⃣ Initialize Telemetry

const telemetry = new Telemetry({
  firebase: {
    apiKey: "YOUR_FIREBASE_API_KEY",
    authDomain: "YOUR_FIREBASE_AUTH_DOMAIN",
    projectId: "YOUR_FIREBASE_PROJECT_ID",
    storageBucket: "YOUR_FIREBASE_STORAGE_BUCKET",
    messagingSenderId: "YOUR_FIREBASE_MESSAGING_SENDER_ID",
    appId: "YOUR_FIREBASE_APP_ID",
    measurementId: "YOUR_FIREBASE_MEASUREMENT_ID",
  },
  sentry: {
    dsn: "YOUR_SENTRY_DSN",
    environment: "production",
  },
});

3️⃣ Log Events

telemetry.logEvent("button_click", { buttonName: "Sign Up" });

4️⃣ Set User Details

telemetry.setUserDetails({
  uid: "12345",
  email: "user@example.com",
  username: "exampleUser",
});

5️⃣ Log Errors

try {
  throw new Error("Something went wrong!");
} catch (error) {
  telemetry.logError(error);
}

✅ Works with JavaScript React Apps

If you’re using JavaScript instead of TypeScript, just import and use it as:

import Telemetry from "your-npm-package-name";

const telemetry = new Telemetry({
  firebase: { /* Firebase Config */ },
  sentry: { dsn: "..." }
});

🧪 Running Unit Tests

This package includes unit tests using Jest.

1️⃣ Install Development Dependencies

npm install --save-dev jest ts-jest @types/jest

2️⃣ Run Tests

npm test

✅ Expected Output:

PASS  src/Telemetry.test.ts
✓ should initialize Firebase and Sentry
✓ should log an event
✓ should set user details
✓ should log an error

🤝 Contributing
	1.	Clone this repo:

git clone https://github.com/your-username/telemetry-service.git


	2.	Install dependencies:

npm install


	3.	Run tests:

npm test


	4.	Open a Pull Request to contribute 🎉

📝 License

This project is licensed under the MIT License.

🚀 Future Improvements

✅ Add more telemetry providers (e.g., LogRocket, Amplitude)
✅ Add custom logging levels
✅ Optimize Firebase performance metrics

Now, your README.md is professional, clear, and ready for NPM! 🚀 Would you like a GitHub workflow (CI/CD) setup for automated testing & publishing? 🎯