semvec/pypi-packages huggingface dataset:
tq-scroll-scrape 4.0
TQ-scroll-scrape is a library for downloading web pages via Selenium.
License: MIT
Requires Python: >=3.9

# Scroll-Scraper

Python library for automating scrolling and downloading web pages via Selenium. This is especially useful for pages that
utilize infinite scrolling.

## Usage

## Using ChromeDriver

Download ChromeDriver from https://chromedriver.chromium.org/downloads. Choose the version that matches the Chrome
browser running on your system.

## Using GeckoDriver for Firefox

Download GeckoDriver for Firefox from https://github.com/mozilla/geckodriver/releases.

### Install Package

Install the package by running `pip install tq-scroll-scrape`.

### Use the Package

Here is sample code demonstrating how to download a page:

```python
from tq_scroll_scrape.scroll_and_scrape import ScrollAndScrape

url = "https://www.espn.com/"
driver_path = "{PATH TO DRIVER EXECUTABLE}"
scroll_scraper = ScrollAndScrape(driver_path)
scroll_scraper.download(url)
scroll_scraper.driver.close()
scroll_scraper.driver.quit()
```

This library will scroll the page until it reaches the end. The scroll height can be controlled by passing
the `scroll_by` kwarg. If omitted the value of `document.body.scrollHeight` is used.

To give items the chance to render, the library will wait before scrolling again. The wait time can be controlled via
the `sleep_after_scroll_seconds` parameter. If omitted, the default value is two seconds.

The example below scrolls the page by 500 pixels, waiting 5 seconds between each scroll.

```python
from tq_scroll_scrape.scroll_and_scrape import ScrollAndScrape

driver_path = "{PATH TO DRIVER EXECUTABLE}"
scroll_scraper = ScrollAndScrape(driver_path)
downloader.download(url, sleep_after_scroll_seconds=5, scroll_by=500)
downloader.driver.close()
downloader.driver.quit()
```

#### Post-download Processing

The `download` function accepts a callback that executes after the page is downloaded. The page source is passed into
the callback. The example below saves the downloaded page to an html file.

```python
from tq_scroll_scrape.scroll_and_scrape import ScrollAndScrape


def save_file(source: str):
    with open("espn.html", "w") as file:
        file.write(source)


url = "https://www.espn.com/"
driver_path = "{PATH TO DRIVER EXECUTABLE}"
scroll_scraper = ScrollAndScrape(driver_path)
downloader.download(url, save_file)
downloader.driver.close()
downloader.driver.quit()
```
dbgidbotbuilder 0.0.2
a simple script to create bot API using HTTP server
License: MIT
Requires Python: >=3.9

# dbgidbotbuilder
`dbgidbotbuilder` is a simple module that can help you to make bot HTTP API requests.
Currently support `GET and POST` method only, please pay attention currently this module is `pre-release` so is under development.
**GET and POST** I think enought to your build API bot.
In the future this module may can added some HTTP requests method like `PUT, PATCH, DELETE or CUSTOM` defined by your choice :) .
So you guys I hope you can enjoy by this first time release, some feature are described below

# Feature
- Indonesian Fake Data
- GET or POST support HTML Inspector by CSS Selector
- Storage Utils (append, write)

# Class Method
> DbgHttp

This method to make new client of http client request.
# Usage
```python
from dbgidbotbuilder import DbgHttp

with DbgHttp('url here',verbose=False) as client:
 #get method
 
 client._get(headers={'accept':'*/*})
 
 #post method support output json/html
 
 client._post({'accept':'*/*','content-type':'application/x-www-form-urlencoded'},{'body':'this is body string'},json=False)
 
 #get text element by css class, support span,p,div
 
 client._get_get_text_element_by_class({'accept':'*/*'},'span','class-text')
 
 #get input value by name
 
 client._get_get_input_value_by_name({'accept':'*/*'},'email')
 
 #get input value by id
 
 client._get_get_input_value_by_id({'accept':'*/*},'email')
 
 #json post data
 
 client.__post_json({'accept':'application/json'},{'key':'value'})
```

> DbgStorage

This method for save data to local storage
# Usage
```python
from dbgidbotbuilder import DbgStorage

#save append file

DbgStorage.save('filename.txt','a','string to save')

#read file

DbgStorage.read('filename.txt')

#read as binary

DbgStorage.readBinary('filename.txt')

#read file each line

DbgStorage.readLine('filename.txt')

#read json file
DbgStorage.readJson('filename.json')
```

> DbgFaker

This method can generate indonesian data fake

# Usage

```python
from dbgidbotbuilder import DbgFaker

fake = DbgFaker()
print(fake.getData()) #return dict data
```

Insyallah in next update will rich feature added soon.
If you have bug issues don't hestitate to tell me guys