semvec/pypi-packages huggingface dataset:
vlab-ipam-api 2019.7.3
A RESTful API for automated IPAM of a personal lab in vLab

#############
vLab IPAM API
#############

This service replaces the existing static gateway used in vLab.

The goal of this service is to make it easier for users to access their lab
by removing the need for the jumpbox. This is achieved by exposing a RESTful API
on the gateway that can dynamically port-forward through the NAT firewall that
runs on the gateway. Additionally, vLab clients can leverage this API to programmically
*connect* users to a given resource. For example the vLab CLI client can potentially:

1. Expose an interface like ``vlab connect windows --name <name of instance> --protocol=RDP``
#. Call the main vLab server to *get the IP of the gateway*
#. Call the API on the gateway to *get the port* that maps to the specific component and protocol
#. Evoke an application *on the user's machine* that understands the specific protocol, and pre-populate the connection information.

Clients can also programmically create the port forwarding rules upon component
creation. This would remove the need for users to manually port forwarding
rules, but clients should not prevent users from inputting their own port forwarding
rules.

Background Services
###################

The IPAM service has two background processes:

vlab-worker
***********

Periodically pings IPs stored in the IPAM database. This allows the service to
identify "bad records" and relay that information to the user.

vlab-log-sender
***************

Forwards firewall logs to a remote server. The default iptables config will
log every time a package is FORWARDed. By forwarding the logs for remote processing,
admins of vLab can answer business questions like, *"Do they use that resource?"*


vlab-ddns-updater
*****************

Runs on a regular cycle to send Dynamic DNS updates to the vLab DNS service.
dampr 0.2.3
Data Processing implementation in Python

Dampr - Pure Python Data Processing
===

Dampr is intended for use as single machine data processing: it's natively out of core, supports map and reduce side joins, associative reduce combiners, and provides a high level interface for constructing Dataflow DAGs.

It's reasonably fast, easy to get started, and scales linearly by core.  It has no external dependencies, making it extremely lightweight and easy to install.  It has reasonable REPL support for data analysis, though there are better tools for the job for it.

Features
---

* Self-Contained: No external dependencies and simple to install
* High-Level API: Easy computation
* Out-Of-Core: Scales up to 100s of GB to TBs of data.  No need to worry about Out of Memory errors!
* Reasonably Fast: Linearly scales to number of cores on the machine
* Powerful: Provides a number of advanced joins and other functions for complex workflows

Setup
---

```
pip install dampr
```

or

```
python setup.py install
```

API
---
[docs/dampr/index.html](http://htmlpreview.github.io/?https://github.com/Refefer/Dampr/blob/master/docs/dampr/index.html)

Examples
---

Look at the `examples` directory for more complete examples.

Similarly, the tests are intended to be fairly readable as well.  You can view them in the `tests` directory.

## Example - WC

```python
import sys 

from dampr import Dampr

def main(fname):

    wc = Dampr.text(fname) \
            .map(lambda v: len(v.split())) \
            .a_group_by(lambda x: 1) \
            .sum()

    results = wc.run("word-count")
    for k, v in results:
        print("Word Count:", v)

    results.delete()

if __name__ == '__main__':
    main(sys.argv[1])
```

Why not Dask for data processing?
---
Dask is great!  I'd highly recommend it for fast analytics and datasets which don't need complex joins!

However.

Dask is really intended for in-memory computation and more analytics processing via interfaces like DataFrames.  While it does have a reasonable `bag` implementation for data processing, it's missing some important features such as joins across large datasets.  I have routinely run into OOM errors when processing datasets larger than memory when trying more complicated processes.

In that sense, Dampr is attempting to bridge that gap of complex data processing on a single machine and heavy-weight systems, geared toward ease of use.

Why not PySpark for data processing?
---
PySpark is great!  I'd highly recommend it for extremely large datasets and cluster computation!

However.

PySpark requires large amounts of setup to really get going.  It's the antithesis of "light-weight" and really geared for large scale production deployments.  I personally don't like it for proof of concepts or one-offs; it requires just a bit too much tuning to get what you need.