semvec/pypi-packages huggingface dataset:
cap_upload_validator 1.6.0
Python tool for validating H5AD AnnData files before uploading to the Cell Annotation Platform.
Requires Python: >=3.9

# cap-validator

[![PyPI version](https://img.shields.io/pypi/v/cap-upload-validator)](https://pypi.org/project/cap-upload-validator/)  [![License](https://img.shields.io/github/license/cellannotation/cap-validator)](https://github.com/cellannotation/cap-validator/blob/main/LICENSE)  [![Build Status](https://github.com/cellannotation/cap-validator/actions/workflows/unit_testing.yml/badge.svg)](https://github.com/cellannotation/cap-validator/actions)


## Overview

Python tool for validating H5AD AnnData files before uploading to the Cell Annotation Platform. The same validation code is used in [Cell Annotation Platform](https://celltype.info/) following requirements from the CAP-AnnData schema published [here](https://github.com/cellannotation/cap-data-schema/blob/main/cap-anndata-schema.md).

Full documentation could be found in the [GitHub Wiki](https://github.com/cellannotation/cap-validator/wiki)

## Features
- ✨ Validates all upload requirements and returns results at once
- 🚀 RAM efficient
- 🧬 Provides a full list of supported ENSEMBL gene IDs for *Homo sapiens* and *Mus musculus*


## Installation
```bash
pip install -U cap-upload-validator
```

## Usage

### Basic usage

```python
from cap_upload_validator import UploadValidator

h5ad_path = "path_to.h5ad"

uv = UploadValidator(h5ad_path)
uv.validate()
```


### CLI interface
```console
$ capval tmp/tmp.h5ad
CapMultiException: 
AnnDataMissingEmbeddings: 
        The embedding is missing or is incorrectly named: embeddings must be a [n_cells x 2] 
        numpy array saved with the prefix X_, for example: X_tsne, X_pca or X_umap.
        
AnnDataMisingObsColumns: 
            Required obs column(s) missing: file must contain 
            'assay', 'disease', 'organism' and 'tissue' fields with valid values.
        
For details visit: 
        https://github.com/cellannotation/cap-validator/wiki/Validation-Errors

$ capval --help
usage: capval [-h] adata_path

CLI tool to validate an AnnData H5AD file before uploading to the Cell Annotation Platform.
The validator will raise CAP-specific errors if the file does not follow the CAP AnnData Schema,
as defined in:
https://github.com/cellannotation/cell-annotation-schema/blob/main/docs/cap_anndata_schema.md

Full documentation, including a list of possible validation errors, is available at:
https://github.com/cellannotation/cap-validator/wiki

Usage Example:
    `capval path/to/anndata.h5ad`

positional arguments:
  adata_path  Path to the AnnData h5ad file.

optional arguments:
  -h, --help  show this help message and exit
```

## License
[BSD 3-Clause License](LICENSE)
mlguardx 0.1.0
ML data validation, profiling, drift detection and data contracts framework
Requires Python: >=3.7

# MLGuardian
- Smart Schema Inference
- Data Contracts
- Drift Detection
- Data Profiling


<!-- utils.py functionality -->
1. Tracks if data changed or not
2. Detect schema changes
3. Reduce memory usage
4. Automatically detect CSV format
5. Preparing datasets for production ML

why?
1. Schema change
2. Data Corruption
3. Memory overload

<!-- Profiling script -->
Pandas alternative, as it automatically analyzes the dataset structure, quality, correlations, and drift and produces warnings and actionable ML recommendations.

<!-- Schema.py -->
This is a schema management system for ML datasets.
It does 4 major things:
- Automatically infer dataset schema
- Detect semantic meaning (email, URL, UUID, etc.)
- Validate new data against stored schema
- Handle schema evolution & drift


<!-- problems I faced -> when working -> ML projects 
-->

1. dataset schema changes -> model crashes
2. more NANs or NULLS -> model fails
3. Accuracy drops

what this pip will do?

1. Automatically infers column types

2. Detects target column

3. Detects semantic types (email, uuid, etc.)

4. Stores min/max/mean

5. Saves schema to JSON/YAML

6. Validates new data against schema

7. Calculates null %

8. Detects high cardinality

9. Detects constant columns

10. Finds correlated features

11. Produces HTML report

12. Suggests recommendations
## Installation
pip install mlguardian