semvec/pypi-packages huggingface dataset:
parallel-conveyor 0.1.3
Simple, intuitive pipelining in Python
Requires Python: >=3.8

# Conveyor - Intuitive Python pipelines

Conveyor is a multiprocessing framework for creating intuitive data pipelines. With Conveyor, you can easily create stream-based pipelines to efficiently perform a series of operations on data, a task especially useful in the fields of machine learning and scientific computing. Creating a pipelined job is as easy as writing a function.

### Why use Conveyor?
The answer is simple: throughput.  It's like putting a second load of laundry in the washer while a previous load is in the dryer. By breaking down a problem into smaller serial tasks, we perform the smaller tasks in parallel and increase the efficiency of whatever problem we're trying to solve.

### Installation
Install Conveyor with `pip install parallel-conveyor`.

### A quick and trivial example

Let's say we wanted to build a short pipeline that computed the fourth root of a number (done in two steps) and the cube of a number (in one step). In this case, we would describe this pipeline visually as such:

![](https://parallel-conveyor.readthedocs.io/en/latest/_images/Fork-and-join.png)

To express it with Conveyor, we simply build the pipeline as follows

```python
from conveyor.pipeline import Pipeline
from conveyor.stages import Processor, Pipe, ReplicatingFork, Join
from math import sqrt

def square_root(arg):
    return sqrt(arg)

def cube(arg):
    return arg ** 3

with Pipeline() as pl:
    # Duplicate the input
    pl.add(ReplicatingFork(2))

    # On first copy, compute the sqrt, on the second, the cube
    pl.add(Processor(square_root), Processor(cube))

    # On first copy, compute the sqrt, on the second, do nothing
    pl.add(Processor(square_root), Pipe())

    # Join the two data streams
    pl.add(Join(2))

    # Print the results
    pl.add(Processor(print))

    # Run the pipeline with three different inputs
    pl.run([16, 3, 81])
```

```console
$ python3 sample.py
2.0
1.3160740129524924
3.0
4096
27
531441
```

## Other links

* [Documentation](https://parallel-conveyor.readthedocs.io/en/latest/)
* [PyPI project page](https://pypi.org/project/parallel-conveyor/)

## A note on stability
Conveyor is currently considered in alpha. Specifications will change, potentially in breaking ways.

## Testing
To run the tests, ensure nose is installed and run nosetests from the project directory

`pip3 install nose && nosetests`

## Building from source
To build the distribution archives, you will need the latest version of setuptools and wheel.

`python3 -m pip install --user --upgrade setuptools wheel`

Run `setup.py` to build using the following command:

`python3 setup.py sdist bdist_wheel`

The compiled `.whl` and `.tar.gz` files will be in the `/dist` directory.
dnnpy 0.0.1
Pytorch-like Neural Network framework
Requires Python: >=3.6

# dnnpy Framework

## Installation
````bash
pip install dnnpy
````
## Example usage
```python
from dnnpy.activations import ReLU
from dnnpy.data import make_regression_data
from dnnpy.layers import Sequential, Dense, Dropout
from dnnpy.loss_functions import MAELoss
from dnnpy.optimizers import Adam
from dnnpy.train import train
from dnnpy.utils import split_data
import matplotlib.pyplot as plt

n_inputs = 10
hidden_units = 32
n_outputs = 1

x, y = make_regression_data(n_samples=1000, n_features=n_inputs, n_labels=1)

(x_train, y_train), (x_test, y_test) = split_data(x, y, ratio=0.7)

model = Sequential(Dense(in_features=n_inputs, out_features=hidden_units, activation=ReLU()),
                   Dropout(0.3),
                   Dense(in_features=hidden_units, out_features=n_outputs))

opt = Adam(model.parameters(), lr=1e-3)
loss_func = MAELoss()

train_loss, valid_loss = train(data=(x_train, y_train), network=model, loss=loss_func, optimiser=opt, epochs=30,
                               batch_size=16)

plt.plot(train_loss, label='train')
plt.plot(valid_loss, label='val')
plt.legend()
plt.show()
```