semvec/pypi-packages huggingface dataset:
storify 0.0.11
A lightweight database system for Python
License: MIT

# Storify

Storify is a dead-simple, lightweight Python database library that uses msgpack for efficient data serialization. 

It provides a framework for data storage and retrieval, with an optional ORM model for interaction with data. 

Storify supports automatic backups and error handling, ensuring data integrity and reliability. 

## Features

- Create and manage multiple msgpack-based databases with ease.
- Lightweight ORM-esque model for easy interaction with data.
- Automatic backups and data flushing to prevent data loss.
- Built-in error handling to automatically recover from database corruption and loading issues.
- Configurable save intervals for optimized performance.

## Installation

```bash
pip install storify
```

## Usage

Here's a simplified example of how to use the `storify` package:

```python
from storify import Storify

# Initialize Storify
storify = Storify(root="example_data")
db_name = "example_db"

# Check if the database exists
if storify.db_exists(db_name):
    print(f"Database '{db_name}' exists. It will be loaded.")

# Initialize a new database
 db = storify.get_db(name=db_name)

# Add some data to the database 
db["key1"] = "value1" 
db["key2"] = "value2"

# Get some data from the database
print(f"key1: {db['key1']}")
print(f"key2: {db['key2']}")

# Flush all databases (force save)
storify.flush()
```

For more complete examples, including how to use the ORM, please check the `examples` folder.
deepgnn-ge 0.1.66
Graph engine - distributed graph engine to host graphs.
License: MIT
Requires Python: >=3.7

# DeepGNN Overview

DeepGNN is a framework for training machine learning models on large scale graph data. DeepGNN contains all the necessary features including:

* Distributed GNN training and inferencing on both CPU and GPU.
* Custom graph neural network design.
* Online Sampling: Graph Engine (GE) will load all graph data, each training worker will call GE to get node/edge/neighbor features and labels.
* Automatic graph partitioning.
* Highly performant and scalable.

Project is in alpha version, there might be breaking changes in the future and they will be documented in the changelog.

## Usage

Install pip package:
```bash
python -m pip install deepgnn
```
If you want to build package from source, see instructions in [`CONTRIBUTING.md`](CONTRIBUTING.md).

Train and evaluate a graphsage model with pytorch on cora dataset:
```bash
cd examples/pytorch
python sage.py
```

## Migrating Scripts

We provide a python module to help you upgrade your scripts to new deepgnn versions.

```bash
pip install google-pasta
python -m deepgnn.migrate.0_1_56 --script_dir directory_to_migrate
```

See [`CHANGELOG.md`](CHANGELOG.md) for full change details.
mlos-viz 0.6.2
Visualization Python interface for benchmark automation and optimization results.
License: MIT
Requires Python: >=3.10
Keywords: autotuning, benchmarking, optimization, systems, visualization

# mlos-viz

The [`mlos_viz`](https://github.com/microsoft/MLOS/tree/main/mlos_viz/./) module is an aid to visualizing experiment benchmarking and optimization results generated and stored by [`mlos_bench`](https://github.com/microsoft/MLOS/tree/main/mlos_viz/../mlos_bench/).

## Overview

Its core API is `mlos_viz.plot(experiment)`, initially implemented as a wrapper around [`dabl`](https://github.com/dabl/dabl) to provide a basic visual overview of the results, where `experiment` is an [`ExperimentData`](https://github.com/microsoft/MLOS/tree/main/mlos_viz/../mlos_bench/mlos_bench/storage/base_experiment_data.py) objected returned from the [`mlos_bench.storage`](https://github.com/microsoft/MLOS/tree/main/mlos_viz/../mlos_bench/mlos_bench/storage/) layer APIs.

In the future, we plan to add more automatic visualizations, interactive visualizations, feedback to the `mlos_bench` experiment trial scheduler, etc.

It's available for `pip install` via the pypi repository at [mlos-viz](https://pypi.org/project/mlos-viz/).

## See Also

- <https://microsoft.github.io/MLOS/autoapi/mlos_viz/> for more documentation details
mlos-core 0.6.2
MLOS Core Python interface for parameter optimization.
License: MIT
Requires Python: >=3.10
Keywords: autotuning, optimization

# mlos-core

This [directory](https://github.com/microsoft/MLOS/tree/main/mlos_core/./) contains the code for the `mlos-core` optimizer package.

It's available for `pip install` via the pypi repository at [mlos-core](https://pypi.org/project/mlos-core/).

## Description

`mlos-core` is an optimizer package, wrapping other libraries like FLAML and SMAC to use techniques like Bayesian optimization and others to identify & sample tunable configuration parameters and propose optimal parameter values with a consistent API: `suggest` and `register`.

These can be evaluated by [`mlos-bench`](https://github.com/microsoft/MLOS/tree/main/mlos_core/../mlos_bench/), generating and tracking experiment results (proposed parameters, benchmark results & telemetry) to update the optimization loop, or used independently.

## Features

Since the tunable parameter search space is often extremely large, `mlos-core` automates the following steps to efficiently generate optimal task-specific kernel and application configurations.

1. Reduce the search space by identifying a promising set of tunable parameters
    - Map out the configuration search space: Automatically track and manage the discovery of new Linux kernel parameters and their default values across versions.
    Filter out non-tunable parameters (e.g., not writable) and track which kernel parameters exist for a given kernel version.
    - Leverage parameter knowledge for optimization: Information on ranges, sampling intervals, parameter correlations, workload type sensitivities for tunable parameters are tracked and currently manually curated.
    In the future, this can be automatically maintained by scraping documentation pages on kernel parameters.
    - Tailored to application: Consider prior knowledge of the parameter's impact & an application's workload profile (e.g. network heavy, disk heavy, CPU bound, multi-threaded, latency sensitive, throughput oriented, etc.) to identify likely impactful candidates of tunable parameters, specific to a particular application.
2. Sampling to warm-start optimization in a high dimensional search space
3. Produce optimal configurations through Bayesian optimization
    - Support for various optimizer algorithms (default Bayesian optimizer, Flaml, SMAC, and random for baseline comparison), that handle multiple types of constraints.
    This includes cost-aware optimization, that considers experiment costs given current tunable parameters.
    - Integrated with `mlos-bench`, proposed configurations are logged and evaluated.

## See Also

- <https://microsoft.github.io/MLOS/autoapi/mlos_core/> for more documentation details