semvec/pypi-packages huggingface dataset:
voliboli-pdf-scraper 1.8
Voliboli PDF Scraper
License: MIT

# Voliboli PDF Scraper

Main repository for scraping data of off installed statistics that are in PDF format. Data is stored in a Postgres database with the help of Flask server and GraphQL protocol.

![image](https://user-images.githubusercontent.com/48418580/233640399-525d336e-ad3f-449a-b311-060489326123.png)

## Tests

You can run test from the root of this repository using:

    pipenv run python tests/test_main.py

## Distribute

To generate the distribution archives, run:

    python3 -m pip install --upgrade build
    python3 -m build

You can then install this package locally from any other project using:

    pip install -e /path/to/root

where `root` is the top-level directory of this project.

## New Release

To deploy a new version you can use the following commands:

    twine upload --skip-existing dist/*
Clotho 0.1.0
Proxy requests to AWS public endpoints and filter them using an allowlist
License: MIT
Requires Python: >=3.9
Keywords: aws proxy cross-account security

# PyClotho

A python implementation using [mitmproxy](https://mitmproxy.org/) of [Clotho](https://github.com/ClothoProxy/Clotho)

The proxy will only accept requests to amazonaws.com with a valid Authorization header.
It will filter AWS accounts, regions, and services based on the [configuration](./config.yaml.example)


## How to run

On the server
```sh
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

```


```

or using Docker 

```sh
docker run --rm -v $(pwd):/home/mitmproxy/.mitmproxy \
-p 8080:8080 mitmproxy/mitmproxy:10.2.2 mitmproxy \
-s /home/mitmproxy/.mitmproxy/clotho.py
```

On the client

```sh
export http_proxy=http//serverip:8080
export https_proxy=http://serverip:8080
aws s3 ls --no-verify-ssl
```


## Development

```sh
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

```