semvec/pypi-packages huggingface dataset:
evmspec 0.5.0
A collection of msgspec.Struct definitions for use with the Ethereum Virtual Machine
License: MIT
Requires Python: <3.14,>=3.10

## evmspec

A collection of [msgspec.Struct](https://jcristharif.com/msgspec/structs.html) definitions for use with the Ethereum Virtual Machine

[Why msgspec?](https://jcristharif.com/msgspec/why.html)

Inspired by [snek3](http://github.com/banteg/snek3), I took the structs and extended them with some extra functionality. 

#### Type checking
- Run `poetry run mypy` (uses the `pyproject.toml` configuration).


#### Modules:
- [block](https://bobthebuidler.github.io/evmspec/source/evmspec.html#module-evmspec.structs.block)
- [log](https://bobthebuidler.github.io/evmspec/source/evmspec.html#module-evmspec.log)
- [receipt](https://bobthebuidler.github.io/evmspec/source/evmspec.html#module-evmspec.receipt)
- [trace](https://bobthebuidler.github.io/evmspec/source/evmspec.structs.trace.html)
- [transaction](https://bobthebuidler.github.io/evmspec/source/evmspec.html#module-evmspec.transaction)
- and more
sas2parquet 1.0.9
SAS → Parquet Hybrid Converter & Validator
License: MIT
Requires Python: <4.0,>=3.11
Keywords: sas, parquet, etl, data, pyarrow

# sas2parquet

[![PyPI version](https://badge.fury.io/py/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
[![Python versions](https://img.shields.io/pypi/pyversions/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
[![License](https://img.shields.io/pypi/l/sas2parquet.svg)](LICENCE)

**The ultimate SAS (.sas7bdat) to Parquet converter** — built to handle files that fail with standard tools.

`sas2parquet` automatically detects encodings, repairs schemas, infers correct data types, and performs **pixel-perfect validation** between SAS and Parquet outputs.

---

## ✨ Features

| Feature | Description |
|-------|-------------|
| 🔄 **Auto Encoding** | Detects UTF-8, Latin1, CP1252 from metadata or fallback |
| 🧠 **Smart Types** | Infers datetime, numeric, string with 20+ retry strategies |
| ✅ **Validation** | Chunk-by-chunk comparison (metadata, counts, values) |
| 📊 **Memory Safe** | Chunked processing (96GB RAM optimized, configurable) |
| 💾 **ZSTD Compression** | Level-6 ZSTD for efficient Parquet storage |
| 📝 **Detailed Logs** | Full conversion trace + mismatch reports |
| 🎯 **Two Modes** | Single file or recursive directory processing |

---

## 🚀 Quick Start

### Install
```bash
pip install sas2parquet
```

---

## ✅ Usage

### Convert a directory (recommended)

```bash
sas2parquet path/to/sasdata/
```

- Converts **all `.sas7bdat` files recursively**
- Creates `parquetdata/` and `logging/` next to `sasdata/`

---

### Convert a single file

```bash
sas2parquet path/to/file.sas7bdat
```

Output (default):
```text
path/to/file.parquet
```

---

### Specify output location

#### Directory mode — custom output directory
```bash
sas2parquet path/to/sasdata/ --out path/to/parquetdata/
```

#### File mode — custom output file
```bash
sas2parquet path/to/file.sas7bdat --out path/to/output.parquet
```

---

### Custom log directory (directory mode)

```bash
sas2parquet path/to/sasdata/ --log-dir path/to/logs/
```

---

## 📁 Directory Mode Behavior

```text
your-project/
├── sasdata/
│   ├── file1.sas7bdat
│   └── subfolder/
│       └── nested.sas7bdat
├── parquetdata/
│   ├── file1.parquet
│   └── subfolder_parquet/
│       └── nested.parquet
└── logging/
    └── conversion_20260205_1145.log
```

---

## 🛠️ CLI Reference

```bash
sas2parquet --help
```

---

## ⚙️ Configuration (Advanced)

Edit constants in:

```text
src/sas2parquet/convert.py
```

---


## 👥 Authors

This package was built collaboratively.
- Zaman Ziabakhshganji — creator and maintainer
- Anete Kristiana Zonnenberga — co-author and contributor
- Farshad Radman — co-author and contributor
- Jos van Dongen — co-author and contributor

## 📄 License

MIT License