semvec/pypi-packages huggingface dataset:
qcrawl 0.3.5
Fast async web crawler & scraping framework, supporting deduplication, and extensible middleware.
License: MIT
Requires Python: >=3.11
Keywords: async, asyncio, crawler, scraper, spider, web-scraping

<img src="https://www.qcrawl.org/assets/crawl.svg" alt="qCrawl Logo" style="min-width:75%;" />

[![PyPI Version](https://img.shields.io/pypi/v/qcrawl.svg?style=for-the-badge)](https://pypi.org/project/qcrawl)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/qcrawl.svg?style=for-the-badge)](https://pypi.org/project/qcrawl)
[![Codecov](https://img.shields.io/codecov/c/github/crawlcore/qcrawl/main?style=for-the-badge)](https://codecov.io/gh/crawlcore/qcrawl)

[qcrawl](https://www.qcrawl.org) is a fast async web crawling & scraping framework for Python to extract structured data from web-pages.
It is cross-platform and easy to install via `pip` or `conda`.

Follow the [documentation](https://www.qcrawl.org/).


### qCrawl features

1. Async architecture - High-performance concurrent crawling based on asyncio
2. Performance optimized - Queue backend on Redis with direct delivery, messagepack serialization, connection pooling, DNS caching
3. Powerful parsing - CSS/XPath selectors with lxml
4. Middleware system - Customizable request/response processing
5. Flexible export - Multiple output formats including JSON, CSV, XML
6. Flexible queue backends - Memory or Redis-based (+disk) schedulers for different scale requirements
7. Item pipelines - Data transformation, validation, and processing pipeline
8. Pluggable downloaders - HTTP (aiohttp), Camoufox (stealth browser) for JavaScript rendering and anti-bot evasion
promptsearch 0.1.0
Automated Reverse Prompt Engineering: Evolve system prompts from target outputs.
License: MIT
Requires Python: >=3.8

# PromptSearch

[![PyPI version](https://badge.fury.io/py/promptsearch.svg)](https://badge.fury.io/py/promptsearch)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Automated Reverse Prompt Engineering: Evolve system prompts from target outputs.

## Installation

```bash
pip install promptsearch
```

## Quick Start

```python
from promptsearch import PromptSearcher

# Define your target output (what you want the LLM to produce)
target_output = {
    "phone": "555-1234",
    "name": "John Doe"
}

# Define your initial system prompt
initial_prompt = "Extract information from the text."

# Create the searcher
searcher = PromptSearcher(
    target_output=target_output,
    initial_prompt=initial_prompt
)

# Run optimization
train_input = "Contact John Doe at 555-1234 for more information."
result = searcher.optimize(train_input=train_input, generations=5)

print(f"Best Prompt: {result['best_prompt']}")
print(f"Best Score: {result['best_score']:.3f}")
print(f"Best Output: {result['best_output']}")
```

## How It Works

PromptSearch uses evolutionary hill climbing to optimize system prompts:

1. **Generate**: Test the current prompt with your input
2. **Score**: Compare output to target using semantic similarity
3. **Mutate**: Use an LLM to rewrite the prompt based on failure analysis
4. **Repeat**: Continue until convergence or max generations

## Requirements

- Python 3.8+
- OpenAI API key (set `OPENAI_API_KEY` environment variable)

## License

MIT License - see LICENSE file for details.