semvec/pypi-packages huggingface dataset:
visionner 0.0.7
Turn raw image dataset into numpy array ; more suitable for deep learning tasks
Keywords: Computer-vision,preprocessing,Images,Dataset,visionner

# Installation
- step 1 

```bash
git clone https://github.com/charleslf2/visionner
```  

- step 2

```bash
cd visionner
```
- step 3 

```bash
py setup.py install
```


# Usage

```python3

>>> from visionner.core import DatasetImporter

>>> your_dataset=DatasetImporter("path/to/your/dataset/", size=(28, 28))

```

```python3

>>> from visionner.core import SupervisedImporter

>>> features, labels= SupervisedImporter("path/to/your/dataset", categories=["cat", "dog"], size=(28,28))

```

```python3

### normalize your dataset

>>> from visionner.core import DatasetNormalizer

>>> your_normalized_dataset=DatasetNormalizer(your_dataset)

```

```python3

### create a trainset and a testset

>>> from visionner.core import TrainTestSpliter

>>> x_train, x_test=TrainTestSpliter(dataset, test_size=0.2)

```

```python3

### visualize the first image of your dataset

>>> import matplotlib.pyplot as plt 

>>> plt.imshow(your_dataset[0])
>>> plt.show()

```

```python3

### save your dataset

>>> from visionner.core import DatasetSaver

>>> DatasetSaver("my_saved_dataset", your_dataset)

```

```python3

### open your dataset

>>> from visionner.core import DatasetOpener

>>> my_saved_dataset=DatasetOpener("my_saved_dataset.npy") 

```
test-bench-control 0.0.3
For controlling our test bench camera and motor
Requires Python: >=3.7

_____________________________________________________________________________
Package for controlling our test bench with:

- Raspberry Pi 4B (Raspberry Pi OS 64-bit) 
- Nema 23 stepper motor with TB6600 Driver
- 2 IMX477 Raspberry Pi Cameras and Arducam 4x Multiplexer V2.2
_____________________________________________________________________________
How to install and setup:

- pip install test_bench_control
- in config.txt change 'camera_auto_detect=1' to 'camera_auto_detect=0'
- in config.txt add 'dtoverlay=camera-mux-4port,cam0-imx477,cam1-imx477'
_____________________________________________________________________________
How to use:

from test_bench_control import camera,motor

motor.rotate(degree=-8.6,slope=0.2,rpm_max=200)
camera.take_picture(0,"road/to/pic","my_pic.jpg")
DataPipelineExecutor 0.0.5
A package to execute Copy and Stored Procedure tasks on data

# DataPipelineExecutor

This package contains a pipeline which executes two tasks: Copying data from source (Kusto Cluster DB) to the sink (SQL Sever DB), and Executing a simple SQL Stored Procedure on the copied data. These tasks are completely configurable and the pipeline has following key features:
- Restartability with configurability.
- Logging - on top of logging on console, you can pass a dedicated filepath as a parameter if you wish to store the logs. Otherwise a *logger.log* file will be created in the same directory and all logs will be stored there until deleted.

## Installation
Use pip to install the package:

```shell
pip install DataPipelineExecutor
```

Run the following commands to install dependencies:
```shell
pip install numpy
pip install pandas
pip install azure-kusto-data
pip install azure-kusto-ingest
pip install pyodbc
pip install papermill
```

## Usage

To execute the pipeline run the following:
```shell
import main from DataPipelineExecutor
main('config.txt', 'logger.log')
# OR
main('config.txt')
```

Note that logger file is optional and if no paramater is passed it will automatically be created in the same directory.

## Configuration

Three config files are required for this pipeline to run:
- config: This contains parameters (refer to the format below) needed for task executions.
- source_config: This file contains parameters needed to establish connection to the source server. The path to this file is passed as a parameter in the primary_config file.
- sink_config: This file contains parameters needed to establish connection to the sink server. The path to this file is passed as a parameter in the primary_config file.

Format for the primary_config:
```shell
[Watermark]
Table_Name =
Column1 = 
Column2 = 
watermark_col_basis_name = 
DateTime = 

[CopyData]
SourceType = 
SinkType = 
SourceConfig = 
SinkConfig = 
SQLTable =
KustoTable = 
Query =
BatchSize = 

[TaskSet]
Sequence = 

[StoredProcedure]
ProcedureName = 
ParameterName = 
ParameterType = 
TargetColumn = 
TargetValue = 

[Notebook]
Path = 
OutputPath = 
Param1 = 
Param2 =
```

Format for source_config:
```shell
[Kusto]
Cluster = 
Database =
ClientID =
ClientSecret = 
AuthorityID = 
```

Format for sink_config:
```shell
[SQL]
Server = 
Database = 
Username =
Password =
Driver = 
```