semvec/pypi-packages huggingface dataset:
vigorish 0.7.0
Hybrid Python/Node.js web scraper for Major League Baseball (MLB) data.
License: MIT
Requires Python: >=3.6

[![PyPI version](https://badge.fury.io/py/vigorish.svg)](https://badge.fury.io/py/vigorish) ![PyPI - Downloads](https://img.shields.io/pypi/dm/vigorish?color=%234DC71F) ![PyPI - License](https://img.shields.io/pypi/l/vigorish?color=%25234DC71F) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/vigorish) [![Maintainability](https://api.codeclimate.com/v1/badges/4a1753c77add039c3850/maintainability)](https://codeclimate.com/github/a-luna/vigorish/maintainability) [![codecov](https://codecov.io/gh/a-luna/vigorish/branch/master/graph/badge.svg)](https://codecov.io/gh/a-luna/vigorish)

## vigorish

`vigorish` is a hybrid Python/Node.js application that scrapes MLB data from mlb.com, brooksbaseball.net and baseball-reference.com.

My goal is to capture as much data as possible &mdash; ranging from PitchFX measurements at the most granular level to play-by-play data (play descriptions, substitutions, manager challenges, etc) and individual player pitch/bat stats at the highest level.

### Requirements

-   Python 3.6+
-   Node.js 10+ (Tested with Node.js 11-13)
-   Xvfb
-   AWS account (optional but recommended, used to store scraped data in S3)

### Project Documentation

For a step-by-step install guide and instructions for configuring/using `vigorish`, please visit the link below:

[Vigorish: Hybrid Python/Node.Js Web Scraper](https://aaronluna.dev/projects/vigorish/)

### Credits

`vigorish` either relies on the following projects listed below directly or as a dev dependency. It would not have been possible for me to create `vigorish` without these projects, thanks to all of the creators/maintainers for making these available (projects are listed alphabetically):

-   [aenum](https://github.com/ethanfurman/aenum)
-   [bullet](https://github.com/bchao1/bullet)
-   [click](https://github.com/pallets/click)
-   [dacite](https://github.com/konradhalas/dacite)
-   [dataclass-csv](https://github.com/dfurtado/dataclass-csv)
-   [enlighten](https://github.com/Rockhopper-Technologies/enlighten)
-   [events](https://github.com/pyeve/events)
-   [halo](https://github.com/manrajgrover/halo)
-   [lxml](https://github.com/lxml/lxml)
-   [naked](https://github.com/chrissimpkins/naked)
-   [rapidfuzz](https://github.com/maxbachmann/rapidfuzz)
-   [tabulate](https://github.com/astanin/python-tabulate)
census-parquet 0.0.9
Tools for generating Parquet files from US Census 2020
License: MIT

# census-parquet
Python tools for creating and maintaining Parquet files from [US 2020 Census Data](https://www.census.gov/programs-surveys/decennial-census/decade/2020/2020-census-main.html).


## Installation

To use the data download shell script files first install [wget](https://en.wikipedia.org/wiki/Wget).

To install the census-parquet package use
```
pip install census-parquet
```

This will also install the required Python dependencies which are:
1. [click](https://github.com/pallets/click)
2. [dask](https://docs.dask.org/en/latest/install.html)
3. [dask_geopandas](https://github.com/geopandas/dask-geopandas)
4. [geopandas](https://geopandas.org/getting_started/install.html)
5. [numpy](https://numpy.org/install/)
6. [openpyxl](https://openpyxl.readthedocs.io/en/stable/#installation)
7. [pandas](https://pandas.pydata.org/docs/getting_started/install.html)
8. [pyarrow](https://arrow.apache.org/docs/python/install.html)

## Usage
To run the census-parquet code simply use
```
run_census_parquet
```

This runs the following scripts in order:
1. `download_boundaries.sh` - This script downloads the Census Boundary data needed to run `process_boundaries.py`
2. `download_population_stats.sh` - This script downloads population stat data needed for process_blocks.py
3. `download_blocks.sh` - This script downloads the Census Block data needed to run process_blocks.py
4. `process_boundaries.py` - This script processes the Census Boundary data and creates parquet files. The parquet files will be output into a `boundary_outputs` folder.
5. `process_blocks.py` - This script processes Census Block data and creates parquet files. The final combined parquet file will have the name `tl_2020_FULL_tabblock20.parquet`.