semvec/pypi-packages huggingface dataset:
Chaf 2.0.0
Chaf is a Python library designed to automatically detect and decode encrypted text using the Chaf web interface. You simply provide an encoded string, and the library processes it to identify the encryption type and return the decoded result in structured JSON format.
License: MIT
Requires Python: >=3.6

<p align="center">
  <img align="center" width="320" src="https://github.com/user-attachments/assets/8a02b9b7-e6e5-4cb2-b065-26bab530d476" />
  <h3 align="center"></h3>
</p>


<p align="center">

<a href="https://pypi.org/project/SyncAi/">
    <img src="https://img.shields.io/pypi/v/Chaf?color=red&logo=pypi&logoColor=red">
  </a>

  <a href="https://t.me/Pycodz">
    <img src="https://img.shields.io/badge/Telegram-Channel-blue.svg?logo=telegram">
  </a>
  
  <a href="https://t.me/DevZ44d" target="_blank">
    <img alt="Telegram Owner" src="https://img.shields.io/badge/Telegram-Owner-red.svg?logo=telegram" />
  </a>
</p>



> **⚠️ Notice:** This is an early prototype of the library. Future updates will include broader recipe support, faster execution, and more advanced decoding capabilities.

> **Chaf** is a Python library designed to **automatically** detect and decode encrypted text using the **[CyberChef](https://gchq.github.io/CyberChef/)** web interface. You simply provide an encoded string, and the library processes it to identify the encryption type and return the decoded result in structured **JSON format**.

### Installation and Development 🚀

### Via **PyPi** ⚡️
```shell
# via PyPi
pip install Chaf -U
```

### 🚀 Quick Start 
```python
from Chaf import CyberChef
c: CyberChef = CyberChef("")   # ZGVlcA==
c.get()
```

### 💬 Help & Support .
- Follow updates via the **[Telegram Channel](https://t.me/Pycodz)**.
- For general questions and help, join our **[Telegram chat](https://t.me/PyChTz)**.
linkwalker 0.9.0
Deep URL crawler for Python supporting dynamic and static content, domain restrictions, and callbacks.
License: MIT
Requires Python: >=3.10
Keywords: crawler, web-scraping, url, deep-crawl, automation

# LinkWalker

LinkWalker is a Python library for **deep URL crawling and walking**. It supports both **dynamic browser-based crawling** (using Playwright) and **static HTTP crawling**, allowing you to traverse websites, extract links, and filter URLs with ease. Perfect for developers building scrapers, bots, or web analyzers.

## Features

- **Dynamic crawling** with Playwright (handles JavaScript-heavy pages)
- **Static HTTP crawling** using aiohttp for lightweight scraping
- **Deep crawling** with configurable max depth
- **URL filtering**:
  - Include or exclude URLs based on substrings
  - Clean URLs by removing query parameters
  - Blacklist certain file extensions
  - HTTPS-only option
- **Domain control**: restrict crawling to specific domains or subdomains
- **Callbacks**: execute custom logic on each page visited
- **Concurrency control** with adjustable max parallel pages

## Installation

```bash
pip install linkwalker
```

## Example Usage

### Dynamic Browser Walker

```python
import asyncio
from linkwalker.spider.dynamic import BrowserWalker
from linkwalker.spider._types import BrowserWalkOptions
from playwright.async_api import Page

async def on_page(page: Page, html):
    print("Visited:", page.url)

async def main():
    walker = BrowserWalker(headless=True, max_pages=4)
    await walker.start()

    options: BrowserWalkOptions = {
        "https_only": False,
        "clean_url": True,
        "max_depth": 2,
        "on_page": on_page,
        "allow_all_domains": False,
    }

    urls = await walker.walk(origin_url="https://example.com", options=options)
    print(f"Found {len(urls)} URLs")

    await walker.close()

asyncio.run(main())
```

### Static HTTP Walker

```python
import asyncio
from linkwalker.spider.static import HTTPWalker
from linkwalker.spider._types import HTTPWalkOptions

async def on_page(url, html):
    print("Visited:", url)

async def main():
    walker = HTTPWalker(max_pages=5)
    await walker.start()

    options: HTTPWalkOptions = {
        "https_only": False,
        "clean_url": True,
        "max_depth": 2,
        "on_page": on_page,
        "allow_all_domains": False,
        "url_must_contain": ["/tag/", "/author/"],
        "url_must_not_contain": ["/page/"]
    }

    urls = await walker.walk(origin_url="https://quotes.toscrape.com", options=options)
    print(f"Found {len(urls)} URLs")

    await walker.close()

asyncio.run(main())
```

## Contributing

Feel free to submit issues or pull requests. Contributions to improve crawling efficiency, filtering, or feature support are welcome.

## License

MIT License