semvec/pypi-packages huggingface dataset:
careful 0.4.0
a small library for writing resilient, well-behaved HTTP code
License: BSD-2-Clause
Requires Python: >=3.11

# careful

<img src="https://jpt.sh/projects/careful/carefully-3681327.svg" width=100 height=100 alt="logo of a warning sign">

**careful** is a Python library for writing resilient, well-behaved HTTP clients.

**Code**: <https://codeberg.org/jpt/careful>

**Docs**: <https://jpt.sh/projects/careful/>

![PyPI - Version](https://img.shields.io/pypi/v/careful)
[![status-badge](https://ci.codeberg.org/api/badges/15185/status.svg)](https://ci.codeberg.org/repos/15185)

Call one function to enchant an
**[httpx.Client](https://www.python-httpx.org)**, making your HTTP connections more resilient and better mannered.

- Configure **throttling** to avoid accidental Denial-of-Service / risking getting banned.
- **Retries** help overcome intermittent failures on flaky sites or long crawls.
- **Development caching** Cache persists between runs during development, reduces redundant requests made while iterating on your crawlers & scrapers.

### Example

```python
from httpx import Client
from careful.httpx import make_careful_client

# the only function you need to call is make_careful_client
# this wraps your existing `httpx.Client` with your preferred
# careful behaviors

client = make_careful_client(
    client=Client(headers={'user-agent': 'spiderman/1.0'}),

    # retries are configurable w/ exponential back off
    retry_attempts=2,
    retry_wait_seconds=5,

    # can cache to process memory, filesystem, or SQLite
    cache_storage=MemoryCache(),

    # easy-to-configure throttling
    requests_per_minute=60,
)

# methods on client are called as they always are
# configured behaviors occur without further code changes
client.get("https://example.com")
```


---

Logo licensed from [Adrien Coquet via Noun Project](https://thenounproject.com/icon/carefully-3681327/)