semvec/pypi-packages huggingface dataset:
hub-dataflow 0.9.4
Snark Hub
Requires Python: >=3
Keywords: snark-hub

# DataFlow &middot; [![SnarkAI](https://circleci.com/gh/snarkai/dataflow.svg?style=shield&circle-token=529446a9a8d3b2e209900afa0359e85fcf430b40)](https://circleci.com/gh/snarkai/dataflow)

Prototyping for Hub Dataflow

# Setup
1. Install the package
```
> pip3 install git+https://github.com/snarkai/dataflow@v0.2.2
```
Make sure gdal and opencv is installed as required by `image_io.py`

2. Setup credentials in the local folder and don't forget to put those in .gitignore
```
.secrets/agmri.cfg
.secrets/intelinair #AWS credentials
```

3. Example to get started

```
from dataflow import hub
import torch

hub.init()

df = hub.agmri().get_polygons_by_type(hub.agmri.UIUC_ENDROW, max=1) 
ds = hub.polygon_sampler(df, sample_count=32, shape=(1024,1024,4))
ds[0]
```

# Troubleshooting 

## Install GDAL and OpenCV
If it throws error on gdal or opencv install those manually

```
./bin/prepare.sh
./bin/gdal.sh
./bin/opencv.sh
```

## Credentials examples
`intelinair` file should look like this
```
[default]
aws_access_key_id = ...
aws_secret_access_key = ...
region = us-east-1

```
`agmri.cfg`
```
[production]
admin_username = ...
admin_password = ...
```

## Known Issues
[ ] Credentials are not flexible where to be stored


## Push a new version
```
git commit -a -m 'current changes'
git tag -a 'v0.2.2' -m 'some message here'
git push origin 'v0.2.2' 
```


## Automated integration testing
To run pytest before git push we need to do create .git/hooks/pre-push file with the following content:
```
#!/bin/bash
docker-compose run test
```
And make it executable
```
chmod +x .git/hooks/pre-push
```

Also whenever updating requirements.txt and options.txt please rebuild the test
```
docker-compose build test
```

## Formatting and Linting
Hub uses Black and Flake8 to ensure a consistent code format throughout the project.
Replace .vscode/settings.json content withthe following:
```
{
    "[py]": {
        "editor.formatOnSave": true
    },
    "python.formatting.provider": "black",
    "python.linting.flake8Enabled": true,
    "python.linting.flake8Path": "flake8",
    "python.linting.flake8Args": [
        "--max-line-length=80",
        "--select=B,C,E,F,W,B950",
        "--ignore=E203,E501,W503"
    ],
    "python.linting.pylintEnabled": false,
    "python.linting.enabled": true,
}
```
subsheets 0.8
utility to create subsheets in excel file
Requires Python: >=3.6
Keywords: subsheets sheets

# subsheets

> Utility to create subsheets in excel file.

## Usage

### Install
`
    pip install subsheets
`

and then
`
    subsheets <new_file_name>
`

the program will ask for the source directory where the files are present. The
new file will be created in the same directory.

## Contribute

### Clone the source
`
    https://github.com/CruiseDevice/subsheets
`

### Navigate inside the subsheets folder
`
    cd subsheets/
`

### Install the dependencies
`
    pip install -r requirements.txt
`

### Navigate inside the src/ folder
`
    cd src/
`

### Usage
`
    python subsheets.py <new-file_name>
`

#### Note: Make the changes in the different branch and send the PR.
print_structure 0.0.5
print_structure prints out the structure of any arbitrary python object.

# print_structure

## Description
Print structure of python object.
Most python object types are supported.
Please let me know if you would like some types added.

## Installation

Run the following to install:
```
pip install print_structure
```

## Attributes
object : any type of python object

## Output
There is no output for this function. It only prints the information in the terminal.

One line of print-out contains follwing information:
```
|___<level> [<obj_type>] (<obj_shape>)
```


## Usage

### small example
```python
from print_structure import print_str
test_obj = [1,2,3,["a","b",b"bytes",2+3j]]

print_str(object_atr = test_obj)
```
print-out
```python
|___0 [list] (4)
  |___1 [list] (4)
```
There is as well the possibility to look into every element by using "print_elements = True".
Careful with large object!
```python
print_str(object_atr = test_obj, print_elements = True)
```
print-out
```python
|___0 [list] (4)
  |___1 [int] 
  |___1 [int] 
  |___1 [int] 
  |___1 [list] (4)
    |___2 [str] 
    |___2 [str] 
    |___2 [bytes] 
    |___2 [complex] 
 ```


### large example

```python
from print_structure import print_str
import pandas as pd
import numpy as np

# pandas data
df2 = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]),columns=['a', 'b', 'c'])
data = np.array(['a','b','c','d'])
s = pd.Series(data)

# numpy data
np_array = np.array([[1, 2], [3, 4]])

# dict
dict_varm = {"a":1,"b":2}
# complex numbers
complex_n = 2+3j
# boolean
boolean_var = True
# tuple
tuple_var = (1,3,4)
# set
set_var = set((1,2,4))
# range
range_var = range(0,3)

### bytes
simple_bytes_string = b"test_test"
string = "Pyth"
bytes_arr = bytearray(string, 'utf-8')
mem_var = memoryview(simple_bytes_string)

test_obj = [df2, [s], np_array, complex_n, boolean_var, tuple_var, set_var, range_var, \
            dict_varm, [simple_bytes_string, [bytes_arr, mem_var]]]

print_str(object_atr = test_obj)
```
print-out
```python
|___0 [list] (10)
  |___1 [pandas.core.frame.DataFrame] (3, 3)
  |___1 [list] (1)
    |___2 [pandas.core.series.Series] (4,)
  |___1 [numpy.ndarray] (2, 2)
  |___1 [tuple] (3)
  |___1 [set] (3)
  |___1 [range] (3)
  |___1 [dict] (2)
  |___1 [list] (2)
    |___2 [list] (2)
      |___3 [bytearray] (4)
```
quickrepo 1.0.5
Command-line tool to automate git and github repository creation.
Requires Python: >=3.6

# quickrepo
![fury](https://badge.fury.io/py/quickrepo.svg)
![travis](https://travis-ci.org/silverhairs/quickrepo.svg?branch=master)
![pipy](https://pypip.in/d/quickrepo/badge.png)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

It is quite boring everytime typing `git init` in your local computer then go to github and initialize a new repository, take the remote url and add it to the local... I am already tired just by explaining. Thanks God **quickrepo** exists. <br/>
<br/>quickrepo is a command-line application that automates initializing a new repository both locally and on GitHub. The user can either initialize the repository in the directory they are currently working in, or generate a brand new folder which will come initialized as a git and github repository.

<img src="https://github.com/silverhairs/quickrepo/blob/master/demo.gif" alt="demo">

## Requirements
  - Python 3.6+
## Installation
<b>install the official package from PyPI</b>
```
pip install quickrepo
```
If you have multiple versions of Python installed in your system, use `pip3 install quickrepo` instead.

<b>or install editable source code</b>
```
git clone github.com/silverhairs/quickrepo.git
cd quickrepo
pip install --editable .
```
## Usage
Run `quickrepo` to list all the available commands.

### Initialize a new Git & Github repository
To generate a brand new project both locally and on Github, open your terminal/command prompt and run: </br>
```
quickrepo new
```
### Initialize current working directory as Git & Github repository
To initialize a Git repository in the current working directory and push the content on Github, open your terminal/command prompt and run </br>
```
quickrepo here
```