semvec/pypi-packages huggingface dataset:
pytalite 1.0.0
Python/Pyspark Package for Model-agnostic Evaluation and Diagnosis.
License: MIT
Keywords: pytalite

# <img src="icons/logo_thumb.png" alt="logo" height="100" />
# Pytalite: Model-agnostic diagnostics and evaluation tool for Python and Pyspark

## Overview
Evaluating model performance and diagnosing models are important steps in machine learning model development. Pytalite provides a straightforward wrapper for visualizing model performance and diagnosing machine learning models. It enables users to visualize key snapshots of model performance, e.g. discrete precision-recall curves, probability density plots, model-agnostic feature importance, partial dependence plots, accumulative local effect plots, and feature correlation plots.

## Requirements
Pytalite for python is developed under python 3.7, but is compatible with python 2.7. Pytalite for pyspark is developed to support spark 2.0 and above.

matplotlib &ge; 2.2.x (although 1.4.3 also works, higher version is recommended)

numpy &ge; 1.9.x

scipy &ge; 0.15.x

multiprocess &ge; 0.70.4

## Package Structure
<center><img src='resources/package_structure.png'></center>

## Highlights
Pytalite provides the following model evaluation and diagnostic algorithms:

- Discrete Precision/Recall Plot (Binary-Classification only)
- Feature Correlation Plot (Binary-Classification only)
- Probability Density Plot (Binary-Classification only)
- Feature Importance Plot (Binary-Classification only)
- Accumulated Local Effect Plot (Binary-Classification / Regression, numerical feature)
- Partial Dependence Plot (Binary-Classification / Regression)

## Examples
See `examples` folder for usage examples.

## Gallery
### Discrete Precision/Recall Plot

<center><img src='resources/decile_plot.png'></center>

---

### Feature Importance Plot

<center><img src='resources/feature_importance_plot.png'></center>

---

### Feature ALE Plot for numerical features

<center><img src='resources/feature_ale_plot.png'></center>
brep 1.0.1
Binary Search in plaintext and Gzip files.
License: MIT
Requires Python: >=3.5

# BREP : Binary Search in plaintext and gzip files
Search large files in O(log n) time using binary search.  
We support plaintext and Gzipped files.

## Benchmark : 8x faster than `grep` on a 2GB dataset !
`brep` is usually faster than `grep` for >1GB datasets.

Check `tests/benchmark.py` to reproduce the results.
```
grep ^777 test.txt : 1.594 s (15 runs)
brep 777 test.txt : 206.8 ms (15 runs)
```

## Installation
`pip install brep` or `pip install .` from this repo

## Index your file
In order to conduct binary search, your file needs to be sorted.    
We recommend `GNU sort`, as it's multithreaded and supports large files.  
`LC_ALL=C sort -u -o output_file input_file`

BREP supports compressed file in the GZIP format.  
We recommend `pigz` for quick multicore compression :
`pigz file`

## Usage
Provide 1 prefix search term and 1 filepath  
`brep 77777 test/large.gz`

You can also search from our Python class  
```python
from brep import Search

for result in Search("77777", "test/large.gz"):
    print(result)
```

## Contribute
PRs are welcome!

Install dev dependencies: `pip install -e .[dev]`  
Test and lint before submitting: `pytest && flake8`  

## Todo
- Reimplement in Rust
- Faster gz size estimation
- Search multiple strings at once