semvec/pypi-packages huggingface dataset:
astrawler 0.0.5
A lightweight Selenium crawler
License: MIT
Requires Python: <4.0,>=3.10

# Astrawler: A Robust and Flexible Web Crawler

[![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/codastr/astrawler/blob/main/LICENCE)

Astrawler is a Python library built on top of Selenium, designed to simplify web crawling and data extraction tasks.  It addresses the common challenges of dealing with dynamic web content and asynchronous loading without requiring boilerplate code.

The core idea is to provide a streamlined API for:

* **Navigating webpages:** Opening URLs and interacting with elements.
* **Locating elements:**  Using CSS Selectors or XPath to pinpoint specific elements on a page.
* **Extracting data:**  Retrieving text, attributes, or other data from those elements.
* **Handling Dynamic Content:**  Built-in waiting mechanisms manage pages that load content after the initial HTML is received.
* **Error Resilience:** Includes retry logic to automatically handle transient network errors.

Astrawler aims to be a pragmatic solution for developers who need to reliably scrape data from websites, especially those that heavily utilize JavaScript.  It's a lightweight alternative to more complex scraping frameworks when you need the full browser automation capabilities of Selenium but want a cleaner, more focused API.  

It's intended for tasks like:

* **Data mining:**  Extracting specific data points from websites.
* **Web monitoring:**  Tracking changes on web pages.
* **Automated testing:**  Simulating user interactions for testing purposes.
## Installation

```bash
pip install astrawler
```

## Usage
```python
with Crawler(headless=False) as crawler:
    crawler.open_url("google.com")
    doodles_element: WebElement = crawler.wgetx("/html/body/div[1]/div[3]/form/div[1]/div[1]/div[3]/center/input[2]", single=True,
                                                        wait_seconds=5)
    crawler.click(doodles_element)
```

## Motivation

I created Astrawler because I kept finding myself writing nearly the same web crawling code over and over again in different projects. Instead, I wanted a single, reusable library.

Astrawler is designed to handle common web crawling tasks – in a clean and simple way. 

I’m hoping that by sharing this library, it can also simplify the crawling experience for others and save them the effort of re-implementing these common tasks.

## License

[MIT License](https://github.com/codastr/astrawler/blob/main/LICENCE)
uniswap-viewer 0.1.10
Lightweight Python library for reading Uniswap V3 prices and tick data via Web3.
License: MIT
Requires Python: >=3.8
Keywords: uniswap v3 web3 defi ethereum blockchain async

# uniswap-viewer

A lightweight Python library for accessing Uniswap V3 on-chain data.

`uniswap-viewer` provides a clean and efficient interface for reading price and 
tick information directly from Uniswap V3 smart contracts using `web3.py`. It 
supports both synchronous and asynchronous workflows and is ideal for 
developers, analysts, and algorithmic traders.

## Features

- Supports both sync and async Web3 clients  
- Fetches token prices using `sqrtPriceX96` and token decimals  
- Retrieves detailed tick data via ABI-decoded contract calls  
- Computes tick index, tick spacing, and tick slot  
- Includes block streaming generator to track new blocks in real time  
- Built-in utilities for loading ABIs and checking Ethereum addresses  

## Modules

- `viewer_sync.py` – synchronous Uniswap V3 viewer using `web3.Web3`
- `viewer_async.py` – asynchronous Uniswap V3 viewer using `web3.AsyncWeb3`
- `utils.py` – helper functions (ABI loading, tick keys, address filters)

## Usage

### Sync example

```python
from uniswap_viewer import ViewerSync
from web3 import Web3

w3 = Web3(Web3.HTTPProvider("<your_provider_url>"))

viewer = ViewerSync(w3, "usdt", "weth", fee=3000)
viewer.init()

price = viewer.get_price()

tick = viewer.calc_tick(price, normalize=True)
tick_data = viewer.get_tick_data(tick)
liquidity = tick_data['liquidityGross']
```

### Async example

```python
from uniswap_viewer import ViewerAsync
from web3 import AsyncWeb3

w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider("<your_provider_url>"))

viewer = ViewerAsync(w3, "usdt", "weth", fee=3000)
await viewer.init()

price = await viewer.get_price()

tick = viewer.calc_tick(price, normalize=True)
tick_data = await viewer.get_tick_data(tick)
liquidity = tick_data['liquidityGross']
```

### Stream example

```python
from uniswap_viewer.viewer_async import Viewer as ViewerAsync, stream_new_blocks
from web3 import AsyncWeb3

w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider("<your_provider_url>"))

viewer = ViewerAsync(w3, "usdt", "weth", fee=3000)
await viewer.init()

async for block_num in stream_new_blocks(w3, timeout=3):
    price = await viewer.get_price(block_num=block_num)

    tick = viewer.calc_tick(price, normalize=True)
    tick_data = await viewer.get_tick_data(tick, block_num=block_num)
    liquidity = tick_data['liquidityGross']

    print(price, liquidity)
```

## Requirements

- Python 3.8+
- `web3` library (sync or async)

## Motivation

This library is designed for:

- Developers integrating Uniswap data directly from the blockchain
- Quants and analysts building custom tooling
- Avoiding reliance on third-party APIs like The Graph

## License

MIT