semvec/pypi-packages huggingface dataset:
antyr 1.0.2
A compact web crawling toolkit
Requires Python: >=3.11

<div align="center">
    <img width="300" height="300" alt="antyr-logo" src="https://github.com/user-attachments/assets/91a3347d-4863-43d7-a2a2-d581e5ede72c" />
</div>

# antyr

This project focuses on **core crawling primitives**: making HTTP requests, consuming responses as streams, and persisting streamed content with explicit, cancellation-safe lifetimes.

Unlike full-featured frameworks, `antyr` does not implement an end-to-end scraping pipeline. Parsing, extraction logic, data modeling, retries, scheduling, and storage are left to the caller.

> If you want a batteries-included scraping framework, consider Scrapy.

## Installation

Install via pip:

```bash
pip install antyr
```

Or using [uv](https://github.com/astral-sh/uv):

```bash
uv add antyr
```

Please note that the following packages will be installed alongside `antyr`:

- `trio` – structured concurrency runtime
- `httpx[socks]` – HTTP client with SOCKS proxy support
- `stem` – Tor control port integration

## Quickstart

The examples below show how to fetch a resource and either process its contents as a stream or persist it to disk.

### Fetch and process a response as a stream

Instead of buffering the entire response in memory, the response can be processed incrementally as it is received.

```python
import trio
from antyr import HttpCrawler

async def main() -> None:
    async with HttpCrawler("https://httpbin.org") as crawler:
        stream = await crawler.fetch("/json").content_stream()

        async for chunk in stream:
            # process chunk

trio.run(main)
```

If the response body is an archive, it can be extracted before processing by calling `extract()`. The extracted content is exposed through the same streaming interface.

```python
import trio
from antyr import HttpCrawler

async def main() -> None:
    async with HttpCrawler("https://example.com") as crawler:
        stream = await crawler.fetch("/archive.zip").extract()

        async for chunk in stream:
            # process chunk

trio.run(main)
```

### Stream to disk

Stream the response body directly to disk.

```python
import trio
from antyr import HttpCrawler

async def main() -> None:
    async with HttpCrawler("https://httpbin.org") as crawler:
        await crawler.fetch("/image/png").save("downloads")

trio.run(main)
```

The target filename is derived from the response headers or URL and normalized for filesystem safety.
skilleter-modules 0.0.11
Modules used by my various Python projects (and hopefully useful to other people)
Requires Python: >=3.8

# skilleter-modules

A collection of Python modules used in my various other Python projects (readable, skilleter-thingy, skilleter-extras)

## colour.py

Functions for writing coloured text using ANSI control code sequences

## dc_curses.py, dc_defaults.py, dc_util.py, dircolors.py

Colour filesystem objects similar to ls.

Large parts of this code are Copyright 2019 Allen Wild <allenwild93@gmail.com> released under the Apache-2.0 licence.

## files.py

Miscellaneous file-related functions - backup creation, file-type detection, etc.

## git.py

A git library

## path.py

Filesystem path-related functions

## popup.py

Popup window class for curses

## run.py

Wrapper around the subprocess module, providing additional functionality

## tfm_pane.py

Console-pane-handling code for TFM - will probably be moved into the TFM project when that is split out from skilleter-thingy

## tidy.py

Functions for tidying log files - this may be moved into the skilleter-readable project at some point
codenose 0.4.1
Code smell detection for LLMs - sniffs out duplicate logic, architecture violations, and other blind spots
Requires Python: >=3.11
Keywords: code-quality, code-smell, linting, llm, static-analysis

# CodeNose 👃

Code smell detection for LLMs - sniffs out duplicate logic, architecture violations, and other blind spots.

## Installation

```bash
pip install codenose
```

## Usage

```python
from codenose import CodeNose

# Quick scan a file
result = CodeNose.quick_scan_file("/path/to/file.py")
print(f"Found {len(result.smells)} smells")

# Scan a directory
result = CodeNose.quick_scan_directory("/path/to/project")
print(f"Cleanliness: {result.cleanliness_score:.0%}")

# With custom configuration
nose = CodeNose(
    max_file_lines=500,
    max_function_lines=50,
    canonical_filenames={"utils.py", "core.py", "models.py"}
)
result = nose.scan_file("/path/to/file.py")
```

## Smell Types

| Type | Emoji | Description |
|------|-------|-------------|
| syntax | 🔴 | Syntax errors |
| syspath | 💀 | sys.path manipulation |
| traceback | ☠️ | Exception without traceback |
| arch | 🏗️ | Non-canonical filename |
| facade | 🧅 | Logic in facade layer |
| dup | 👯 | Duplicate code blocks |
| long | 📏 | File/function too long |
| log | 📝 | Missing logging |
| import | 📦 | Duplicate imports |

## Configuration

```python
nose = CodeNose(
    # Architecture settings
    canonical_filenames={"utils.py", "core.py", ...},
    exempt_dirs={"tests", "__pycache__", ...},
    facade_files={"api.py", "cli.py", "mcp_server.py"},

    # Thresholds
    max_file_lines=400,
    max_function_lines=33,
    min_dup_block_size=3,
)
```

## License

MIT