semvec/pypi-packages huggingface dataset:
flytekitplugins-whylogs 1.16.14
Enable the use of whylogs profiles to be used in flyte tasks to get aggregate statistics about data.
License: apache2
Requires Python: >=3.9

# Flytekit whylogs Plugin

whylogs is an open source library for logging any kind of data. With whylogs,
you are able to generate summaries of datasets (called whylogs profiles) which
can be used to:

- Create data constraints to know whether your data looks the way it should
- Quickly visualize key summary statistics about a dataset
- Track changes in a dataset over time

```bash
pip install flytekitplugins-whylogs
```

To generate profiles, you can add a task like the following:

```python
import whylogs as why
from whylogs.core import DatasetProfileView

import pandas as pd

from flytekit import task

@task
def profile(df: pd.DataFrame) -> DatasetProfileView:
    result = why.log(df) # Various overloads for different common data types exist
    profile_view = result.view()
    return profile
```

>**NOTE:** You'll be passing around `DatasetProfileView` from tasks, not `DatasetProfile`.

## Validating Data

A common step in data pipelines is data validation. This can be done in
`whylogs` through the constraint feature. You'll be able to create failure tasks
if the data in the workflow doesn't conform to some configured constraints, like
min/max values on features, data types on features, etc.

```python
from whylogs.core.constraints.factories import greater_than_number, mean_between_range

@task
def validate_data(profile_view: DatasetProfileView):
    builder = ConstraintsBuilder(dataset_profile_view=profile_view)
    builder.add_constraint(greater_than_number(column_name="my_column", number=0.14))
    builder.add_constraint(mean_between_range(column_name="my_other_column", lower=2, upper=3))
    constraint = builder.build()
    valid = constraint.validate()

    if valid is False:
        print(constraint.report())
        raise Exception("Invalid data found")
```

If you want to learn more about whylogs, check out our [example notebooks](https://github.com/whylabs/whylogs/tree/mainline/python/examples).
flytekitplugins-perian-job 1.16.14
Flyte connector for Perian Job Platform (perian.io)
License: apache2
Requires Python: >=3.9

# Flytekit Perian Job Platform Plugin

Flyte Connector plugin for executing Flyte tasks on Perian Job Platform (perian.io).

See the [official docs page](https://perian.io/docs/flyte-getting-started) for more details.
flytekitplugins-omegaconf 1.16.14
OmegaConf plugin for Flytekit
License: apache2
Requires Python: >=3.9

# Flytekit OmegaConf Plugin

Flytekit python natively supports serialization of many data types for exchanging information between tasks.
The Flytekit OmegaConf Plugin extends these by the `DictConfig` type from the
[OmegaConf package](https://omegaconf.readthedocs.io/) as well as related types
that are being used by the [hydra package](https://hydra.cc/) for configuration management.

## Task example
```
from dataclasses import dataclass
import flytekitplugins.omegaconf  # noqa F401
from flytekit import task, workflow
from omegaconf import DictConfig

@dataclass
class MySimpleConf:
    _target_: str = "lightning_module.MyEncoderModule"
    learning_rate: float = 0.0001

@task
def my_task(cfg: DictConfig) -> None:
    print(f"Doing things with {cfg.learning_rate=}")


@workflow
def pipeline(cfg: DictConfig) -> None:
    my_task(cfg=cfg)


if __name__ == "__main__":
    from omegaconf import OmegaConf

    cfg = OmegaConf.structured(MySimpleConf)
    pipeline(cfg=cfg)
```

## Transformer configuration

The transformer can be set to one of three modes:

`Dataclass` - This mode should be used with a StructuredConfig and will reconstruct the config from the matching dataclass
during deserialisation in order to make typing information from the dataclass and continued validation thereof available.
This requires the dataclass definition to be available via python import in the Flyte execution environment in which
objects are (de-)serialised.

`DictConfig` - This mode will deserialize the config into a DictConfig object. In particular, dataclasses are translated
into DictConfig objects and only primitive types are being checked. The definition of underlying dataclasses for
structured configs is only required during the initial serialization for this mode.

`Auto` - This mode will try to deserialize according to the Dataclass mode and fall back to the DictConfig mode if the
dataclass definition is not available. This is the default mode.

You can set the transformer mode globally or for the current context only the following ways:
```python
from flytekitplugins.omegaconf import set_transformer_mode, set_local_transformer_mode, OmegaConfTransformerMode

# Set the global transformer mode using the new function
set_transformer_mode(OmegaConfTransformerMode.DictConfig)

# You can also the mode for the current context only
with set_local_transformer_mode(OmegaConfTransformerMode.Dataclass):
    # This will use the Dataclass mode
    pass
```

```note
Since the DictConfig is flattened and keys transformed into dot notation, the keys of the DictConfig must not contain
dots.
```
flytekitplugins-huggingface 1.16.14
Hugging Face plugin for flytekit
License: apache2
Requires Python: >=3.9

# Flytekit Hugging Face Plugin
[Hugging Face](https://github.com/huggingface) is a community and data science platform that provides: Tools that enable users to build, train and deploy ML models based on open source (OS) code and technologies

This plugin supports `datasets.Dataset` as a data type with [StructuredDataset](https://docs.flyte.org/en/latest/user_guide/data_types_and_io/structureddataset.html).

To install the plugin, run the following command:

```bash
pip install flytekitplugins-huggingface
```
flytekitplugins-deck-standard 1.16.14
This Plugin provides more renderers to improve task visibility
License: apache2
Requires Python: >=3.9

# Flytekit Deck Plugin

This plugin provides additional renderers to improve task visibility within Flytekit.

## Installation

To install the plugin, run the following command:

```bash
pip install flytekitplugins-deck-standard
```

## Renderer Requirements

Each renderer may require additional modules.

The table below outlines the dependencies for each renderer:

| Renderer               | Required Module(s)          |
|------------------------|-----------------------------|
| SourceCodeRenderer      | `pygments`                  |
| FrameProfilingRenderer  | `pandas`, `ydata-profiling` |
| MarkdownRenderer        | `markdown`                  |
| BoxRenderer             | `pandas`, `plotly`          |
| ImageRenderer           | `pillow`    |
| TableRenderer           | `pandas`                    |
| GanttChartRenderer      | `pandas`, `plotly`  |

## Renderer Descriptions

### SourceCodeRenderer
Converts Python source code to HTML using the Pygments library.

### FrameProfilingRenderer
Generates a profiling report based on a pandas DataFrame using `ydata_profiling`.

### MarkdownRenderer
Converts markdown strings to HTML.

### BoxRenderer
Creates a box-and-whisker plot from a column in a pandas DataFrame.

### ImageRenderer
Displays images from a `FlyteFile` or `PIL.Image.Image` object in HTML.

### TableRenderer
Renders a pandas DataFrame as an HTML table with customizable headers and table width.

### GanttChartRenderer
Displays a Gantt chart using a pandas DataFrame with "Start", "Finish", and "Name" columns.
flytekitplugins-dbt 1.16.14
DBT Plugin for Flytekit
License: apache2
Requires Python: >=3.9

# Flytekit dbt plugin

Flytekit plugin for performing DBT tasks. Currently it supports  `dbt run` , `dbt test`, `dbt source freshness` tasks.

To install the plugin, run the following command:

```bash
pip install flytekitplugins-dbt
```

_Example coming soon!_

## Contributors

- [Gojek](https://www.gojek.io/)
flytekitplugins-data-fsspec 1.16.14
This is a deprecated plugin as of flytekit 1.5
License: apache2
Requires Python: >=3.9

# fsspec data plugin for Flytekit — Experimental

This plugin provides an implementation of the data persistence layer in Flytekit that uses fsspec. Once this plugin
is installed, it overrides all default implementations of the data plugins and provides the ones supported by fsspec. This plugin
will only install the fsspec core. To install all fsspec plugins, please follow the [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/).
flytekitplugins-dask 1.16.14
Dask plugin for flytekit
License: apache2
Requires Python: >=3.9

# Flytekit Dask Plugin

Flyte can execute `dask` jobs natively on a Kubernetes Cluster, which manages the virtual `dask` cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) `dask` cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.

To install the plugin, run the following command:

```bash
pip install flytekitplugins-dask
```

To configure Dask in the Flyte deployment's backed, follow
[these directions](https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html)ernetes/k8s_dask/index.html#step-2-environment-setup)

An [usage example](https://docs.flyte.org/en/latest/flytesnacks/examples/k8s_dask_plugin/index.html)
can be found in the documentation.