semvec/pypi-packages huggingface dataset:
layerz 0.5
Module to create layers of dependencies
License: MIT

Layerz
======

A simple module for creating layers of dependencies.

Installation
------------

Just use pip!::

    $ pip install layerz

Usage
-----

.. code-block:: python

    from layerz import Layers

    dep1 = type("dep1", (object, ), {"dependencies": lambda s, all_deps: []})()
    dep2 = type("dep2", (object, ), {"dependencies": lambda s, all_deps: ["dep3"]})()
    dep3 = type("dep3", (object, ), {"dependencies": lambda s, all_deps: ["dep1"]})()
    dep4 = type("dep4", (object, ), {"dependencies": lambda s, all_deps: ["dep3"]})()
    dep5 = type("dep1", (object, ), {"dependencies": lambda s, all_deps: ["dep4", "dep2"]})()

    layers = Layers({"dep1": dep1, "dep2": dep2, "dep3": dep3, "dep4": dep4, "dep5": dep5})
    layers.add_to_layers("dep5")
    for layer in layers.layered:
        # might get something like
        # [("dep5", dep5)]
        # [("dep4", dep4), ("dep2", dep2)]
        # [("dep3", dep3)]
        # [("dep1", dep1)]

When we create the layers, it will do a depth first addition of all dependencies
and only add a dep to a layer that occurs after all it's dependencies.

Cyclic dependencies will be complained about.

Tests
-----

Install locally::

    $ pip install -e .
    $ pip install -e ".[tests]"

And then use the helper script::

    $ ./test.sh
InternetArgumentCorpus 1.0.1
The Internet Argument Corpus (IAC) version 2 is a collection of corpora for research in political debate on internet forums.
License: MIT
Keywords: corpus

Internet Argument Corpus
========================
The **Internet Argument Corpus (IAC)** version 2 is a collection of corpora for research in political debate on internet forums. The data is provided in a MySQL database (`download <https://nlds.soe.ucsc.edu/iac2>`_). There is also Python code for accessing/creating the database (`here <https://bitbucket.org/nlds_iac/internet-argument-corpus-2>`_).

Dependencies
------------
Data:
  * `MySQL <https://www.mysql.com/>`_ (Or `MariaDB <https://mariadb.org/download/>`_)
    (Server for hosting, client for access)

Code:
  * `Python 3 <https://www.python.org/>`_
  * Python libraries (pip3 install <whatever>):

    * sqlalchemy
    * inflect
    * mysqlclient (or other interface such as oursql)

Install (Code)
--------------
Either clone the git repository::

    git clone git@bitbucket.org:nlds_iac/internet-argument-corpus-2.git

Or install via pip::

    pip3 install InternetArgumentCorpus

Install (Data)
--------------
Restoring from a sql dump::

    mysql --user=root -p createdebate < createdebate_20xx_xx_xx.sql

Note that you may need to create the database first::

    drop database createdebate;
    SET GLOBAL innodb_file_format=Barracuda;  # in case it isn't already
    CREATE SCHEMA createdebate DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_bin;


Backing up::

    mysqldump createdebate -r createdebate_$(date +%Y_%m_%d).sql

Or potentially faster but much more complicated (How *I* do it)::

    dir=$(date "+%Y-%m-%d_%Hh%Mm");
    mkdir -m 777 -p /tmp/$dir
    date
    for db in convinceme fourforums createdebate createdebate_released; do 
        echo $db; 
        mkdir -m 777 /tmp/$dir/$db; 
        mysqldump --tab=/tmp/$dir/$db $db; 
        rm /tmp/$dir/$db/*.sql; 
        mysqldump --no-data $db -r /tmp/$dir/$db/$db.sql;
        echo "compressing";
        tar -czf /tmp/$dir/"$db"_$(date +%Y_%m_%d).tgz -C /tmp/$dir/ $db;
        rm -rf /tmp/$dir/$db;
    done; mv /tmp/$dir .; date;

    cd $dir
    date
    for db in convinceme fourforums createdebate createdebate_released; do 
        echo $db; 
        mysql -u root -p -e "drop database $db; CREATE SCHEMA $db DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_bin; SET GLOBAL foreign_key_checks=0"; 
        mysql -u root -p $db < $db/$db.sql;
        mysqlimport -u root -p --use-threads=4 --local $db $db/*.txt; 
        mysql -u root -p -e "SET GLOBAL foreign_key_checks=1"; 
    done;date;


Use
---
Python code:

.. code-block:: python

    from iacorpus import load_dataset

    dataset = load_dataset('fourforums')
    print(dataset.dataset_metadata)
    for discussion in dataset:
        print(discussion)
        for post in discussion:
            print(post)
            exit()

Contributing
------------
I welcome suggestions, pull requests, bug reports, etc.!