semvec/pypi-packages huggingface dataset:
SiteCrawler 0.1
A web scraper that downloads tables, images, and text from a webpage

# generic-crawler

This project contains a Python script for a web scraper that extracts tables, images, and text from a given website.

## Requirements

- Python 3.7 or later
- Selenium
- Beautiful Soup
- Pandas
- tqdm
- Requests

## Installation
### pip 

pip install WebScraper

### github
1. Clone this repository:

git clone https://gitlab.kaisens.fr/kaisensdata/apps/4inshield/drivers/generic-crawler/-/tree/asaid

2. Install the required Python packages:

pip install -r requirements.txt


## Usage

1. Download the appropriate [chromedriver](https://sites.google.com/a/chromium.org/chromedriver/downloads) for your system and add it to your system's PATH or specify the path when initializing the `WebScraper` class.

2. Use the following example code to run the scraper:

```python
from scraper import WebScraper

chrome_driver_path = "<path_to_your_chromedriver>"
url = "https://example.com"
scraper = WebScraper(chrome_driver_path)
scraper.process_website(url)
dbseeder 0.0.4.3
A package to make seeds for some relational databases
Requires Python: >=3.7

# Seeder for Mysql
[![gh-action-pip-audit](https://github.com/conlacda/auto-seeder/actions/workflows/gh-action-pip-audit.yml/badge.svg)](https://github.com/conlacda/auto-seeder/actions/workflows/gh-action-pip-audit.yml)

> Load database then make seeds for it

## Install

```shell
$ pip install dbseeder
```

## Usage
```python
from dbseeder import Database
db = Database(host="localhost", user="root", password="", database="seed")
# Make seed without deleting the existence data
db.makeSeed(rows_num=100000)
# Delete data, then make seeds
db.clearAndMakeSeed(rows_num=100000)
```
OR
```shell
$ python -m dbseeder --host localhost --user root --password= --database seed --rows_num 100 --drop
```

## TODO
* Load relationship
* Add test
* Add argparser

## Test

```shell
python -m pytest
```

## Packing
* [Tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/)
* [Package on Pypi](https://pypi.org/project/dbseeder/)

## My note
To publish new version:
* Change version in `pyproject.toml`
* Build: `python -m build`
* Upload to testpypi: `py -m twine upload --repository testpypi dist/*`
* Upload to pypi: `py -m twine upload --repository pypi dist/`