semvec/pypi-packages huggingface dataset:
tsilva_notebook_utils 0.0.104
Utility functions for Jupyter/Colab notebooks
Requires Python: >=3.6

# 🧰 tsilva-notebook-utils

🔬 Handy utilities for enhancing your Jupyter and Google Colab notebooks

## 📖 Overview

`tsilva-notebook-utils` is a collection of utility functions designed to make working with Jupyter and Google Colab notebooks more efficient. It provides tools for video rendering, notification systems, and Colab-specific features like automatic disconnection after idle periods.

## 🛠️ Usage

### Video Rendering

```python
from tsilva_notebook_utils import render_video

# Render a simple video from frames
frames = [frame1, frame2, frame3]  # List of numpy arrays
video = render_video(frames, fps=30, scale=1.5)
display(video)

# Render frames with labels
labeled_frames = [(frame1, "Start"), (frame2, "Middle"), (frame3, "End")]
video = render_video(labeled_frames, fps=24)
display(video)

# Compare multiple videos side by side
from tsilva_notebook_utils import render_videos
render_videos([(video1_frames, "Original"), (video2_frames, "Processed")])
```

### Google Colab Utilities

```python
from tsilva_notebook_utils import disconnect_after_timeout

# Automatically disconnect Colab after 5 minutes of inactivity
disconnect_after_timeout(timeout_seconds=300)
```

### Notifications

Send notifications to [PopDesk](https://github.com/tsilva/popdesk) notification server:

```python
from tsilva_notebook_utils import send_popdesk_notification

# Send a notification when your long-running notebook task completes
send_popdesk_notification(
    url="https://your-popdesk-url",
    auth_token="your-auth-token",
    title="Training Complete",
    message="Your model has finished training with 95% accuracy"
)
```

## 📄 License

This project is licensed under the [MIT License](LICENSE).
gleaner 0.1.3
Web scraper that finds all pages on a domain

# gleaner

A simple web scraper that finds all the pages on a domain using breadth-first search.

## Installation

```bash
pipx install gleaner
```

## Usage

```bash
glean https://example.com -o output.txt
```

This will

- Start at given URL
- Search through HTML to find pages on the same domain
- Follow those links and repeat the process
- Save all discovered URLs to output.txt

## Options

For more help on options

```bash
glean --help
```

- `-o, --output` - Output file (default: `urls.txt`)
- `-r, --rps` - Maximum requests per second (default: 10)
- `-v, --verbose` - Enable verbose output

## Limitations

- Rate limiter only works for rps >= 1
- Only follows `<a>`-tag links
- Doesn't read `robots.txt`
- No user-agent header
- Doesn't handle 429 errors (too many requests)
- Doesn't parse sitemap.xml
- No export formats beyond `.txt` (json? csv?)
- Only works for entire domains. You can't glean an individual page and all sub-pages
- Doesn't store progress for larger sites where scraping might last more than a few minutes
- No concurrent processing