semvec/pypi-packages huggingface dataset:
ibm-data-engine 0.4.0
Feast offline feature store implementation backed by the IBM Cloud Data Engine
License: Apache-2.0
Requires Python: <3.12,>=3.9

[Feast](https://feast.dev/) plugin for IBM Cloud.  
This plugin implements Feast's offline store backed using [IBM Cloud Data Engine](https://www.ibm.com/cloud/data-engine) and [IBM Cloud Object Storage](https://www.ibm.com/cloud/object-storage)

# Installation

Project dependencies can be installed in a dedicated virtual environment
by running the following command:

```bash
poetry install
```

# Testing and Linting

```bash
poetry run pytest tests/
poetry run pylint ibm_data_engine
```

# Test with Feast

You use it with [Feast](https://feast.dev/) by defining your offline store and data sources.
The instructions below illustrate how it can be used in
[feast-ibm-quickstart](https://github.com/IBM/feast-ibm-quickstart).

## Define dependency

This library is currently not published in [PyPI](https://pypi.org/); you will have to
point to the repository directly. The easiest way to do it is to clone
the repository, and define the dependency as a path in `feast-ibm-quickstart`.

```toml
ibm-data-engine = { path = "/path/to/ibm-data-engine" }
```

After running `poetry update`, you should be able to use the IBM Cloud
Data Engine offline store.

## Define data source

You can modify the `src/feature_repo/example_repo.py` file to use the new data
source. Below is the minimal example of the file:

```python
from ibm_data_engine import DataEngineDataSource
driver_stats_source = DataEngineDataSource(
    name="driver_hourly_stats_source",
    table="driver_stats_demo",
    timestamp_field="event_timestamp",
)
```

## Define offline store

Then, `feature_repo/feature_store.yaml` must configure the offline store.

```yaml
project: test_plugin
entity_key_serialization_version: 2
registry: data/registry.db
provider: local
online_store:
    type: redis
    connection_string: ${REDIS_HOST}:${REDIS_PORT},username=${REDIS_USERNAME},password=${REDIS_PASSWORD},ssl=true,ssl_ca_certs=${REDIS_CERT_PATH},db=0

offline_store:
    type: ibm_data_engine.DataEngineOfflineStore
    api_key: ${DATA_ENGINE_API_KEY}
    instance_crn: ${DATA_ENGINE_INSTANCE_CRN}
    target_cos_url: ${IBM_CLOUD_OBJECT_STORE_URL}
```

Notice that you must define the environment variables:
 * `IBM_CLOUD_OBJECT_STORE_URL`
 * `REDIS_HOST`
 * `REDIS_PORT`
 * `REDIS_PASSWORD`
 * `REDIS_CERT_PATH`
 * `DATA_ENGINE_API_KEY`
 * `DATA_ENGINE_INSTANCE_CRN`

## Apply

To apply the definitions to the registry, run:

```
poetry run feast -c ./feature_repo apply
```

## Training

Run training by retrieving historical feature information from feature store
```
poetry run python training.py
```
## Materialize

To materialize to Redis, run:

```
poetry run feast -c ./ materialize '<START_TIMESTAMP>'  '<END_TIMESTAMP>'
```
## Inference

```
poetry run python inference.py
```
aperol 0.0.0a0
Aperol: A featherweight framework for configuration of Python objects
License: Apache-2.0
Requires Python: >=3.10
Keywords: aperol, config, configuration, machine, learning


# Aperol Config

Aperol is a featherweight framework for configuration of Python objects based on dependency injection.

## Usage

Requires Python >= 3.10.

Install Aperol from PyPI:

```bash
$ pip install aperol
```

Define a basic configuration file `dummy.yaml`:

```yaml
imports:
  - random

a: -1
b.type: math.pi
c.type: uniform
d:
  type: uniform()
  a: 10
  b: 11
```

Python objects are specified by the `type` keyword.
In the example above, `uniform` is resolved to the `random` module specified in the `imports` section.
On the other hand, `pi` is explicitly imported
from the `math` module.

Load the config file using Aperol and use the configured objects:

```python
>>> import random
>>> random.seed(0)  # seed set for reproducibility
>>> import aperol
>>> configured = aperol.parse_config("dummy.yaml")
>>> configured["a"]
-1
>>> configured["b"]
3.141592653589793
>>> configured["c"]
_DelayedConstructor(factory=<bound method Random.uniform of <random.Random object at ...>>, init=False, kwargs={'a': -1, 'b': 3.141592653589793})
>>> configured["d"]
10.844421851525048
>>> uniform = configured["c"]
>>> uniform()
2.1391383869735945
>>> uniform()
0.7418361694776736
```

Package imports can be specified in one of three ways:
1. Within the config `imports` section,
2. passing `search_pkgs` to `aperol.parse_config`, or
3. registering imports globally with `aperol.register_imports`

In all three cases, each import must either be a string module name `"X.Y"` corresponding to `import X.Y`, or a tuple `("X.Y", "Z")` corresponding to `import X.Y as Z`.

Configuration file locations can be registered using `aperol.register_config_path`.

## Syntax

TODO (and TBC).

## Related

Aperol is inspired by [Gin Config](https://github.com/google/gin-config).
In comparison to Gin, Aperol allows configuration to be define using YAML (or any format which can be parsed to a nested dict tree). Thus it does not require learning a completely new syntax. Aperol is also simpler and does not provide all of the functionality of Gin; however, it is powerful enough to configure any Python object enabling flexible configuration.