semvec/pypi-packages huggingface dataset:
hadoop-fs-wrapper 0.7.2
Python Wrapper for Hadoop Java API
License: MIT
Requires Python: <3.13,>=3.11

# Hadoop FileSystem Java Class Wrapper 
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

Typed Python wrappers for [Hadoop FileSystem](https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/FileSystem.html) class family.

## Installation
You can install this package from `pypi` on any Hadoop or Spark runtime:
```commandline
pip install hadoop-fs-wrapper
```

Select a version that matches hadoop version you are using:

| Hadoop Version / Spark version | Compatible hadoop-fs-wrapper version |
|--------------------------------|:------------------------------------:|
| 3.2.x / 3.2.x                  |                0.4.x                 |
| 3.3.x / 3.3.x                  |             0.4.x, 0.5.x             |
| 3.3.x / 3.4.x                  |                0.6.x                 |
| 3.5.x / 3.5.x                  |                0.7.x                 |

## Usage
Common use case is accessing Hadoop FileSystem from Spark session object:

```python
from hadoop_fs_wrapper.wrappers.file_system import FileSystem

file_system = FileSystem.from_spark_session(spark=spark_session)
```

Then, for example, one can check if there are any files under specified path:
```python
from hadoop_fs_wrapper.wrappers.file_system import FileSystem

def is_valid_source_path(file_system: FileSystem, path: str) -> bool:
    """
     Checks whether a regexp path refers to a valid set of paths
    :param file_system: pyHadooopWrapper FileSystem
    :param path: path e.g. (s3a|abfss|file|...)://hello@world.com/path/part*.csv
    :return: true if path resolves to existing paths, otherwise false
    """
    return len(file_system.glob_status(path)) > 0
```

## Contribution

Currently basic filesystem operations (listing, deleting, search, iterative listing etc.) are supported. If an operation you require is not yet wrapped,
please open an issue or create a PR.

All changes are tested against Spark 3.4 running in local mode.
liteopt 0.1.6
Lightweight gradient descent optimizer implemented in Rust
License: MIT
Requires Python: >=3.8

# liteopt

A lightweight optimization library written in Rust with Python bindings.

## Installation

```bash
pip install liteopt
```

## Usage

```python
import liteopt

def f(x):
    x0, x1 = x
    return (1.0 - x0)**2 + 100.0 * (x1 - x0**2)**2

def grad(x):
    x0, x1 = x
    df_dx = -2.0 * (1.0 - x0) - 400.0 * x0 * (x1 - x0**2)
    df_dy = 200.0 * (x1 - x0**2)
    return [df_dx, df_dy]

x0 = [-1.2, 1.0]
x_star, f_star, converged = liteopt.gd(f, grad, x0, step_size=1e-3, max_iters=200_000, tol_grad=1e-4)
print(converged, x_star, f_star)
```