semvec/pypi-packages huggingface dataset:
pywhdfs 1.1.3
pywhdfs: python Web HDFS Client.
License: MIT

PyWHDFS
==================================

API and interactive command line interface for Web HDFS.

```bash
  $ pywhdfs -c prod

  ---------------------------------------------------
  ------------------- PYWEBHDFS 1.0.0 ------------------
  -- Welcome to WEB HDFS interactive python shell. --
  -- The WEB HDFS client is available as `CLIENT`. --
  --------------------- Enjoy ! ---------------------
  ---------------------------------------------------


  >>> CLIENT.list("/")
  [u'admin', u'data', u'group', u'solr', u'system', u'tmp', u'user']
```

Functionalities
--------

* Python Library for interacting with WebHDFS and HTTFS Rest API
* Support both secure (Kerberos,Token) and insecure clusters
* Supports HA cluster and handle namenode failover
* Supports HDFS federation with multiple nameservices and mount points.
* Json format clusters configuration. 
* Command line interface to interactively interact with WebHDFS API
  on a python shell.
* Support concurency on uploads and downloads.


Getting started
---------------

```bash
  $ easy_install pywhdfs
```

Some dependencies require the following packages to be also installed :
- krb5-devel krb5-libs
- gcc
- python-devel

Configuration
---------------

PyWHDFS uses a json configuration file that define the connection parameters
for the different clusters.
A simple configuration file looks like:

```json
  {
    "clusters": [
      {
        "name": "prod",
        "auth_mechanism": "GSSAPI",
        "verify": false,
        "truststore": "trust/store/path.jks",
        "nameservices": [
          {
            "urls": ["http://first_namenode_url:50070" , "http://second_namenode_url:50070"],
            "mounts": ["/"]
          }
         ]
      }
    ]
  }
```

The configuration file is validated against a [ schema file ](pywhdfs/resources/config_schema.json)

The default location of configuration file is "~/.webhdfs.cfg" but can can be overwritten using
WEBHDFS_CONFIG environement variable.

USAGE
-------

The interactive python shell client is the easiest way to use pywhdfs, but you can also instanciate
the client manually :

```python 
 >>>import pywhdfs.client as pywhdfs
 >>>CLIENT = pywhdfs.WebHDFSClient(nameservices=[{'urls':[ "http://host1.hadoop.domain:50070" , "http://host2.hadoop.domain:50070"],'mounts':['/']}], auth_mechanism="GSSAPI", verify=False)
 >>>CLIENT.list("/")
```

The interacctive shell requires the connection parameters for the cluster to be setup in the configuration file,
and the cluster name needs to match the name you pass as argument. 

Contributing
------------

Feedback and Pull requests are very welcome!
The-Arbiter 1.0.0
An argument validation library.
License: MIT
Requires Python: >=3
Keywords: validation,arguments

# The Arbiter
The arbiter is a library for writing validations using a simple syntax. What kind of validations? Originally it was intended to validate named arguments. But use your imagination. It is really, really flexible.

The syntax has simple rules:  
- Validators must be a list, or a dictionary.
- Lists are treated as "AND" conditions.
- Dictionaries are used for "OR" and "XOR" conditions. Requirements:
    - Keys _must_ be strings and start with "OR" or "XOR" (case insensitive). Beyond that, you can use any suffix you like for clarity.
    - Values _must_ be a list. 
- Values must be strings

You can nest things all you want. For example:
```python
contact_validator = Arbiter([
    'name', 'address',                                   # must have name and address
    {'or_contact': ['phone', 'email']},                  # also need phone or email, or both
    {'xor_location': ['street_address', 'coordinates']}  # and, street_address or coordinates, but not both
])
```

Using dictionaries and lists you can create simple or complex validation rules. 

Why "The Arbiter?" Because an arbiter is considered the ultimate authority when it comes to settling a dispute. I thought it was more fun than "argument validator."  

# Current Limitations
This does not do any type validation on arguments. It only verifies whether or not it exists.

# Code Examples
### Verifying a simple named argument.
```python
from thearbiter.validator import Arbiter

required_favorites = Arbiter(['animal'])

def hi_there(name, **favorites):
    # this will raise an AttributeException if the "animal" keyword is not found. 
    required_favorites.validate_args(**favorites)
```

More examples to come.