semvec/pypi-packages huggingface dataset:
supersat 0.0.5
A python package to super-resolve your remote sensing imagery
Requires Python: <4.0,>=3.9

# supersat

[![Release](https://img.shields.io/github/v/release/csaybar/supersat)](https://img.shields.io/github/v/release/csaybar/supersat)
[![Build status](https://img.shields.io/github/actions/workflow/status/csaybar/supersat/main.yml?branch=main)](https://github.com/csaybar/supersat/actions/workflows/main.yml?query=branch%3Amain)
[![codecov](https://codecov.io/gh/csaybar/supersat/branch/main/graph/badge.svg)](https://codecov.io/gh/csaybar/supersat)
[![Commit activity](https://img.shields.io/github/commit-activity/m/csaybar/supersat)](https://img.shields.io/github/commit-activity/m/csaybar/supersat)
[![License](https://img.shields.io/github/license/csaybar/supersat)](https://img.shields.io/github/license/csaybar/supersat)

A python package to super-resolve your remote sensing imagery

- **Github repository**: <https://github.com/csaybar/supersat/>
- **Documentation** <https://csaybar.github.io/supersat/>

## Getting started with your project

First, create a repository on GitHub with the same name as this project, and then run the following commands:

```bash
git init -b main
git add .
git commit -m "init commit"
git remote add origin git@github.com:csaybar/supersat.git
git push -u origin main
```

Finally, install the environment and the pre-commit hooks with

```bash
make install
```

You are now ready to start development on your project!
The CI/CD pipeline will be triggered when you open a pull request, merge to main, or when you create a new release.

To finalize the set-up for publishing to PyPi or Artifactory, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/publishing/#set-up-for-pypi).
For activating the automatic documentation with MkDocs, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/mkdocs/#enabling-the-documentation-on-github).
To enable the code coverage reports, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/codecov/).

## Releasing a new version

- Create an API Token on [Pypi](https://pypi.org/).
- Add the API Token to your projects secrets with the name `PYPI_TOKEN` by visiting [this page](https://github.com/csaybar/supersat/settings/secrets/actions/new).
- Create a [new release](https://github.com/csaybar/supersat/releases/new) on Github.
- Create a new tag in the form `*.*.*`.

For more details, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/cicd/#how-to-trigger-a-release).

---

Repository initiated with [fpgmaas/cookiecutter-poetry](https://github.com/fpgmaas/cookiecutter-poetry).
dfschema 0.0.12
lightweight pandas.DataFrame schema
Requires Python: <4.0,>=3.8

# DFS (aka Dataframe_Schema)

**DFS** is a lightweight validator for `pandas.DataFrame`. You can think of it as a `jsonschema` for dataframe. 

Key features:
1. **Lightweight**: only dependent on `pandas`  and `pydantic` (which depends only on `typing_extensions`)
2. **Explicit**: inspired by `JsonSchema`, all schemas are stored as json (or yaml) files and can be generated or changed on the fly.
3. **Simple**: Easy to use, no need to change your workflow and dive into the implementation details. 
4. **Comprehensive**: Summarizes all errors in a single summary exception, checks for distributions, works on subsets of the dataframe 
5. **Rapid**: base schemas can be generated from given dataframe or sql query (using `pd.read_sql`).
6. **Handy**: Supports command line interface (with `[cli]` extra).
7. **Extendable**: Core idea is to validate *dataframes* of any type. While now supports only pandas, we'll add abstractions to run same checks on different types of dataframes (CuDF, Dask, SparkDF, etc )

## QuickStart

### 1. Validate DataFrame

Via wrapper
```python
import pandas as pd
import dfschema as dfs


df = pd.DataFrame({
  "a": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  "b": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
})

schema_pass = {
  "shape": {"min_rows": 10}
}

schema_raise = {
  "shape": {"min_rows": 20}
}


dfs.validate(df, schema_pass)  # won't raise any issues
dfs.validate(df, schema_raise) # will Raise DataFrameSchemaError
```
Alternatively (v2 optional), you can use the root class, `DfSchema`:
```python
dfs.DfSchema.from_dict(schema_pass).validate(df)  # won't raise any issues
dfs.DfSchema.from_dict(schema_raise).validate(df)  # will Raise DataFrameSchemaError
```

### 2. Generate Schema

```python
dfs.DfSchema.from_df(df)
```
### 3. Read and Write Schemas
  
```python
schema = dfs.DfSchema.from_file('schema.json')
schema.to_file("schema.yml")
```

### 4. Using CLI
> Note: requires [cli] extra as relies on `Typer` and `click`

#### Validate via CLI
```shell
dfschema validate --read_kwargs_json '{delimiter="|"}' FILEPATH SCHEMA_FILEPATH
```
Supports
- csv
- xlsx
- parquet
- feather

#### Generate via CLI
```shell
dfs generate --format 'yaml' DATA_PATH > schema.yaml
```

## Installation

WIP

## Alternatives
- [TableScheme](https://pypi.org/project/tableschema/)
- [GreatExpectations](https://greatexpectations.io/). Large and complex package with Html reports, Airflow Operator, connectors, etc. an work on out-of-memory data, SQL databases, parquet, etc
- [Pandera](https://pandera.readthedocs.io/en/stable/) - awesome package, great and suitable for type hinting, compatible with `hypothesis`
  - [great talk](https://www.youtube.com/watch?v=PI5UmKi14cM)
- [Tensorflow validate](https://www.tensorflow.org/tfx/guide/tfdv)
- [DTF expectations](https://github.com/calogica/dbt-expectations)

## Changes
- [[changelog]]

## Roadmap
- [ ] Add tutorial Notebook
- [ ] Support tableschema
- [ ] Support Modin models
- [ ] Support SQLAlchemy ORM models
- [ ] Built-in Airflow Operator?
- [ ] Interactive CLI/jupyter for schema generation