semvec/pypi-packages huggingface dataset:
ConfigPaerserExt 0.0.0
Insert project description here
Requires Python: >=3.6

.. image:: https://img.shields.io/pypi/status/ConfigPaerserExt
    :alt: PyPI - Status

.. image:: https://img.shields.io/pypi/wheel/ConfigPaerserExt
    :alt: PyPI - Wheel

.. image:: https://img.shields.io/pypi/pyversions/ConfigPaerserExt
    :alt: PyPI - Python Version

.. image:: https://img.shields.io/github/v/release/hendrikdutoit/ConfigPaerserExt
    :alt: GitHub release (latest by date)

.. image:: https://img.shields.io/github/license/hendrikdutoit/ConfigPaerserExt
    :alt: License

.. image:: https://img.shields.io/github/issues-raw/hendrikdutoit/ConfigPaerserExt
    :alt: GitHub issues

.. image:: https://img.shields.io/pypi/dm/ConfigPaerserExt
    :alt: PyPI - Downloads

.. image:: https://img.shields.io/github/search/hendrikdutoit/ConfigPaerserExt/GitHub
    :alt: GitHub Searches

.. image:: https://img.shields.io/codecov/c/gh/hendrikdutoit/ConfigPaerserExt
    :alt: CodeCov
    :target: https://app.codecov.io/gh/hendrikdutoit/ConfigPaerserExt

.. image:: https://img.shields.io/github/workflow/status/hendrikdutoit/ConfigPaerserExt/Pre-Commit
    :alt: GitHub Actions - Pre-Commit
    :target: https://github.com/hendrikdutoit/ConfigPaerserExt/actions/workflows/pre-commit.yaml

.. image:: https://img.shields.io/github/workflow/status/hendrikdutoit/ConfigPaerserExt/CI
    :alt: GitHub Actions - CI
    :target: https://github.com/hendrikdutoit/ConfigPaerserExt/actions/workflows/ci.yaml

.. image:: https://img.shields.io/pypi/v/ConfigPaerserExt
    :alt: PyPi

Project Short Description (default ini)

    Project long description or extended summary goes in here (default ini)

=======
Testing
=======

This project uses ``pytest`` to run tests and also to test docstring examples.

Install the test dependencies.

.. code-block:: bash

    $ pip install -r requirements_test.txt

Run the tests.

.. code-block:: bash

    $ pytest tests
    === XXX passed in SSS seconds ===

==========
Developing
==========

This project uses ``black`` to format code and ``flake8`` for linting. We also support ``pre-commit`` to ensure these have been run. To configure your local environment please install these development dependencies and set up the commit hooks.

.. code-block:: bash

    $ pip install black flake8 pre-commit
    $ pre-commit install

=========
Releasing
=========

Releases are published automatically when a tag is pushed to GitHub.

.. code-block:: bash

    # Set next version number
    export RELEASE = x.x.x
    
    # Create tags
    git commit --allow -empty -m "Release $RELEASE"
    git tag -a $RELEASE -m "Version $RELEASE"
    
    # Push
    git push upstream --tags
cocoa-system 0.1.0
COCOA: COrrelation COefficient-Aware Data Augmentation
Requires Python: >=3.6

# COCOA
### COrrelation COefficient-Aware Data Augmentation

## Table of Contents
  - [Prerequisites](#prerequisites)
    - [Python version and packages](#python-version-and-packages)
    - [Database schemas](#database-schemas)
  - [Installation](#installation)
  - [Usage](#usage)

## Prerequisites
### Python version and packages
This packages requires python version >= 3.6. Additionally, the following packages have to be installed:
```
pandas
numpy
```

### Database schemas
To run the system, the following set of tables need to be created in the database (e.g. Postgres):
- ```main_tokenized```: Inverted index, tokenized -> table, col, row
- ```distinct_tokens```: Like ```main_tokenized```, but maps only distinct tokenized -> table_col_id to compute
overlap faster
- ```order_index```: Index for each column. The index is created offline and stored in the db.
- ```max_column```: Maps table-> number of columns for quicker access

Schemas:
```sql
CREATE TABLE main_tokenized (
    tokenized TEXT,
    tableid INT NOT NULL,
    rowid INT NOT NULL,
    table_col_id TEXT NOT NULL,
);

CREATE TABLE distinct_tokens (
    tokenized TEXT, 
    table_col_id TEXT NOT NULL
);

CREATE TABLE order_index (
    table_col_id TEXT NOT NULL
    is_numeric BOOLEAN,
    min_index INT NOT NULL,
    order_list TEXT,
    binary_list TEXT,
);

CREATE TABLE max_column (
    tableid INT NOT NULL,
    max_col_id INT NOT NULL,
    PRIMARY KEY (tableid)
);
```

To fill ```distinct_tokens``` and ```max_column```, run the following queries:
```sql
INSERT INTO distinct_tokens
SELECT DISTINCT tokenized, table_col_id
FROM main_tokenized;

INSERT INTO max_column 
SELECT tableid, MAX(colid)
FROM main_tokenized
GROUP BY tableid;
```

The order index can be created by calling
```python
COCOA.create_index(values)
```
where values is a column (list of values). The index then has to be stored in the ```order_index``` table in the db for 
every column in the dataset.

## Installation
Run the following command to install COCOA to your python environment:
```
pip install cocoa
```

## Usage
```python
import pandas as pd
import psycopg2
from cocoa-system import DataAugmentation

CONN_INFO = {
    'host': '127.0.0.1',
    'dbname': 'db',
    'user': 'postgres',
    'password': 'password',
}

DB_TABLES = {
    'dt': 'distinct_tokens',
    'mt': 'main_tokenized_overlap',
    'mc': 'max_column',
    'oi': 'order_index',
}

DATASET_PATH = 'datasets/query.csv'
dataset = pd.read_csv(DATASET_PATH)

conn = psycopg2.connect(**CONN_INFO)
cocoa = DataAugmentation.COCOAHandler(conn, DB_TABLES)
result = cocoa.enrich(dataset, k_c, k_t, 'my_query_column', 'my_target_column')
```