semvec/pypi-packages huggingface dataset:
reply-data-processing 0.0.9
Package for processing data in multiple formats for Reply AI services
Requires Python: >=3.7

# Data processing package

This package is dedicated to providing functionality for processing data in text, HTML formats and etc. for further usage in predictors and endpoints. The package consists of services EmailsProcessor, TextParser, HTMLParser and helpers such as TextValidator. 

## Package installation guide
To install package with pip use:
`pip install reply-data-processing`

To install package locally without usage of pip run this command in virtual env from the directory of the package:
`python3 setup.py install`

## Package release guide
1. Change package version in setup files (setup.py and setup.cfg)
2. Rebuild package after the changes (to update generated directories and egg-info files):
`pip install build`
`python -m build`
3. Push the changes to the repo and create a release in Releases tab. 
Create a new tag with new package version to generate release notes. The workflow described in `python-publish.yml` will be used for pushing new package version to pip.
myhydra 0.6
This Package can be used to insert Hydra Job Status and Data Quality Metrics to Elastic Open Search


This Package can be used to insert Hydra Job Status and Data Quality Metrics to Elastic Open Search

Intended Audience 
=================

Comcast BDT Team

Permissions
===========

AWS Role should have access to Service "events.amazonaws.com"


Available Functions
===================


add_project_metadata
-----------------------
Arguments -

project - 
           Name of the project Ex: RTDP
app     - 
           Name of the application Ex: V4 Device Lookup Model
dataset -
           Name of the Dataset Ex: X1_Households
jobid   -
           JobID  Ex: Databricks JobId
dayid   -
           dayid being processed (Optional Parameter)

post_job_status
-----------------------
Arguments -

environment -
              Provide environment value. It will take one of the value ['DEV', 'QA', 'STAG', 'PROD']. For any other value it throws ValueError Exception
status      - 
              Provide job status value. It will take one of the value ['START', 'SUCCESS', 'FAILURE']. For any other value it throws ValueError Exception
source      - 
              Data source value. Example: RTDP D2A
input       - 
              Optional Parameter
output      - 
             Optional Parameter

add_dq_metric
-----------------------
Arguments -

type      - 
            "COUNT" / "DISTINCT" / "NULL"
Name      - 
            Name of the column + type ("Col_1_Count")
value     - 
            Metric value
status    - 
            True / False
threshold - 
            Optional Parameter (Value Ex: 0.5)
percent   - 
            Optional Parameter (Value Ex: 0.0)

post_dq_metrics
-----------------------
Arguments -

status    -
            True / False (if all metrics status are True then True else False)



Change Log
==========

Version 0.1 (2023-09-22)
------------------------
Initial Version 

Functions - 
           update_project_metadata, insert_hydra_job_status

Version 0.2 (2023-09-23)
------------------------
Changes -
          README.md and CHANGELOG.txt added

Version 0.3 (2023-09-23)
------------------------
Changes -
          README.md and CHANGELOG.txt updated

Version 0.4 (2023-09-25)
------------------------
Changes -
          README.md and CHANGELOG.txt updated

Version 0.5 (2023-09-25)
------------------------
Changes -
          Renamed functions to add_project_metadata and post_job_status
          Added new functions add_dq_metric and post_dq_metrics