semvec/pypi-packages huggingface dataset:
url2llm 0.3.3
The easiest way to crawl a website and produce LLM ready markdown files
Requires Python: >=3.12

# url2llm

I needed a **super simple tool to crawl a website** (or the links in a *llms.txt*) into a formatted markdown file (without headers, navigation etc.) **to add to Claude or ChatGPT project documents**.

I haven't found an easy solution, there is some web based tool with a few free credits, but if you are already paying for some LLM with an api, why pay also someone else?

## Quickstart

#### With uv (recommended):

Thanks to uv, you can easily run it from anywhere without installing anything:

```bash
uvx url2llm \
   --depth 1 \
   --url "https://modelcontextprotocol.io/llms.txt" \
   --instruction "I need documents related to developing MCP (model context protocol) servers" \
   --provider "gemini/gemini-2.5-flash-preview-04-17" \
   --api_key ${GEMINI_API_KEY}
```

Then drag `./model-context-protocol-documentation.md` into ChatGPT/Claude!

> [!TIP]
> You can invoke it with `url2llm` as a properly installed cli tool after running `uv tool install url2llm`.

#### With pip (alternative):

```
pip install url2llm
```

## What it does

The script uses Crawl4AI:

1. For each url in the crawling, the script produces a markdown
2. Then it asks the LLM to extract from each page only the content relevant to the given instruction.
3. Merge all pages into one and save the merged file.

## Command args and hints

- To use **another LLM provider**, just change `--provider` to eg. `openai/gpt-4o`
   - always set `--api-key`, it is not always inferred correctly fron env vars
- Provide a **clear goal** to `--instruction`. This will guide the LLM to filter out irrelevant pages.
- Recommended **depth** (default = `2`):
   - `2` or `1` for normal website
   - `1` for llms.txt
- Provide `--output_dir` to change where files are saved (default = `.`)
- If you need the single pages, use `--keep_pages True` (default = `False`)
- You can specify the **concurrency** with `--concurrency` (default = `16`)
- The scripts deletes files **shorter** than `--min_chars` (default = `1000`)

> [!CAUTION]
> If you need to do more complex stuff use Crawl4AI directly and build it yourself: https://docs.crawl4ai.com/
mongorepo 3.1.2
Simple lib that dynamically creates repositories for MongoDB
Requires Python: <4.0,>=3.12

# mongorepo
Simple lib for python &amp; mongodb, provides auto repository factory based on DTO type

## Example with **BaseMongoRepository**
```python
from mongorepo.classes import BaseMongoRepository
from mongorepo import use_collection


def mongo_client(mongo_uri: str = 'mongodb://mongodb:27017/') -> pymongo.MongoClient:
    client: pymongo.MongoClient = pymongo.MongoClient(mongo_uri)
    return client


@dataclass
class UserDTO:
    username: str = ''
    password: str = ''


@use_collection(mongo_client().users_db.users)
class SimpleMongoRepository(BaseMongoRepository[UserDTO]):
    ...


repo = SimpleMongoRepository()

new_user = UserDTO(username='admin', password='1234')
repo.add(new_user)

user = repo.get(username='admin')
print(user)  # UserDTO(username='admin', password='1234')
```

## Example with **mongorepo.async_repository**
```python
import mongorepo


def async_mongo_client(mongo_uri: str = 'mongodb://mongodb:27017/') -> AsyncIOMotorClient:
    async_client = AsyncIOMotorClient(mongo_uri)
    return async_client


@dataclass
class Person:
    id: str
    name: str
    skills: list[str] = field(default_factory=list)    


@mongorepo.async_repository(array_fields=['skills'])
class MongoRepository:
    class Meta:
        dto = Person
        collection = async_mongo_client().people_db.people


repo = MongoRepository()


admin = Person(id='289083', name='admin', skills=['python', 'c++', 'java', 'rust'])

await repo.add(admin)
await repo.skills__append('c', id='289083')
await repo.skills__remove('python', id='289083')

user = await repo.get(id='289083')
print(user.skills)  # ['c++', 'java', 'rust', 'c']
```

## Example with **implement** decorator
```py
from mongorepo.implement import implement
from mongorepo.implement.methods import GetMethod, AddMethod
from mongorepo import use_collection


def async_mongo_client(mongo_uri: str = 'mongodb://mongodb:27017/') -> AsyncIOMotorClient:
    async_client = AsyncIOMotorClient(mongo_uri)
    return async_client


@dataclass
class Author:
    name: str


@dataclass
class Message:
    id: str
    body: str
    author: Author


class MessageRepository(typing.Protocol):
    async def add_message(self, message: Message):
        ...

    async def get_message(self, id: str) -> Message | None:
        ...


@implement(
    GetMethod(MessageRepository.get_message, filters=['id']),
    AddMethod(MessageRepository.add_message, dto='message')
)
@use_collection(async_mongo_client().messages_db.messages)
class MongoMessageRepository:
    class Meta:
        dto = Message


repo: MessageRepository = MongoMessageRepository()


await repo.add_message(
    message=Message(id='1', body='hello', author=Author(name='friend'))
)

message = await repo.get_message(id='1')
print(message)  # Message(id='1', body='hello', author=Author(name='friend'))
```
burst-waveform 0.4.1
A Python package for gravitational wave burst waveform
Requires Python: >=3.8
Keywords: ligo, sine-gaussian, white noise burst, gravitational waves, burst, waveform model

# Burst waveform

[![Build Status](https://git.ligo.org/yumeng.xu/burst_waveform/badges/main/pipeline.svg)](https://git.ligo.org/yumeng.xu/burst_waveform/-/pipelines)
[![PyPI version](https://badge.fury.io/py/burst-waveform.svg)](https://badge.fury.io/py/burst-waveform)
[![License](https://img.shields.io/badge/license-GPLv3-blue)](https://git.ligo.org/yumeng.xu/pycwb/-/blob/main/LICENSE)

This package hosts the python version of the burst waveform used in coherentWaveBurst(cWB) search.

## Installation

### Using pip

```bash
pip install burst-waveform
```

### from source

```bash
make install
```

## Usage

The `Burst-Waveform` package provides four burst waveform models: `SineGaussian`, `SineGaussianQ`, `WhiteNoiseBurst`,
and `Ringdown`. 

The waveforms can be generated by calling the instance of the waveform model after initializing it with the desired
parameters.

For example, to generate a `SineGaussianQ` waveform,

```python
from burst_waveform.models import SineGaussianQ
from matplotlib import pyplot as plt

params = {
    "amplitude": 1.0,
    "frequency": 300.0,
    "Q": 9
}

model = SineGaussianQ(params)
strain = model()

plt.plot(strain)
plt.show()
```

For `WhiteNoiseBurst` waveform,

```python
from burst_waveform.models import WhiteNoiseBurst
import matplotlib.pyplot as plt

params = {
    'frequency': 300,
    'bandwidth': 50,
    'duration': 0.005,
    'inj_length': 1,
    'mode': 1
}

WNB = WhiteNoiseBurst(params)
wnb_strain = WNB()

plt.plot(wnb_strain)
plt.xlim(0.5 - 10 * params['duration'], 0.5 + 10 * params['duration'])
plt.show()
```

For `Ringdown` waveform,

```python
from burst_waveform.models import Ringdown
from matplotlib import pyplot as plt

params = {
    "tau": 0.3,
    "frequency": 10.0,
    "iota": 90.0,
}

model = Ringdown(params)
hp, hc = model()

plt.plot(hp)
plt.plot(hc)
plt.show()
```
cli-dependency-graph 0.1.0
CLI tool to statically analyze and visualize Click/Typer command trees.
Requires Python: >=3.9

# CLI Dependency Graph

Statically analyze and visualize the structure of Python CLI apps built with Click or Typer.

## Features
- Detects `click.Group`, `click.command`, and Typer apps (`Typer()`) via static AST parsing (no code execution)
- CLI commands:
  - `scan <path>`: Discover command tree
  - `show`: Print ASCII tree
  - `export --format {dot,svg}`: Save graph file (planned)
- Renders CLI structure as ASCII (terminal) or DOT/SVG (export)

## Installation

```sh
pip install .
```

## Usage

### 1. Scan a Python CLI app

```sh
python -m cli_dependency_graph scan example_click_app.py
```

### 2. Show the discovered CLI structure

```sh
python -m cli_dependency_graph show
```

### 3. (Optional) Export the CLI graph

```sh
python -m cli_dependency_graph export --format dot --output cli_graph.dot
```

## Example CLI App

```python
import click

@click.group()
def cli():
    pass

@cli.command()
def hello():
    """Say hello"""
    print("Hello!")

@cli.command()
def goodbye():
    """Say goodbye"""
    print("Goodbye!")

if __name__ == "__main__":
    cli()
```

## Development

- Source code: `src/cli_dependency_graph/`
- Tests: `tests/unit/`

### Run tests

```sh
pytest
```

## Requirements
- Python 3.9+
- [Typer](https://typer.tiangolo.com/) >= 0.12.3
- [Click](https://click.palletsprojects.com/) >= 8.1.7
- [rich](https://rich.readthedocs.io/) >= 13.7.1
- [graphviz](https://graphviz.readthedocs.io/) >= 0.20.3
- [pytest](https://docs.pytest.org/) (for tests)

---
Contributions welcome!