semvec/pypi-packages huggingface dataset:
progenitor-probability 1.1
Returns the joint progenitor probability (probability that a galaxy is a spheroid at z=0) for a given redshift, mass, environment (percentile) and star-formation rate.
License: MIT



[![PyPI](https://img.shields.io/pypi/v/progenitor-probability.svg)](https://pypi.python.org/pypi/progenitor-probability/) [![Python](https://img.shields.io/pypi/pyversions/progenitor-probability.svg)](https://pypi.python.org/pypi/progenitor-probability/)
## *README* for `tabulate_progenitor_probability.py`

### Reference:
[***Martin et al. 2018a***](https://doi.org/10.1093/mnras/stx3057 "Martin+18a"),
*Martin, G., Kaviraj, S., Devriendt J. E. G., Dubois Y., Pichon C. and Laigle C., MNRAS, 474, 3140*


### Contact:
[g.martin4@herts.ac.uk](mailto:g.martin4@herts.ac.uk "email")


### Purpose:
Reads `fractions.dat` binary file and returns the joint progenitor probability for given redshift, mass, environment (percentile) and star-formation rate (i.e. the probability that a galaxy with given properties has spheroidal morphology at z=0). If one or more dimensions are not specified, the joint progenitor probability is returned with missing dimensions marginalised out.


### Prerequisites:
* `fractions.dat`
* numpy
* scipy

### Installation:

    pip install progenitor-probability


### Usage:

#### Using the built-in script:

```
tabulate_progenitor_probability.py -z 0.4 -m 10.8
0.266751184855
```    
```    
tabulate_progenitor_probability.py --help

usage: tabulate_progenitor_probability.py [-h] [-z REDSHIFT] [-m MASS]
                                          [-p DENSITY] [-s SFR]

optional arguments:
  -h, --help            show this help message and exit
  -z REDSHIFT, --redshift REDSHIFT
                        Redshift
  -m MASS, --mass MASS  Log10 stellar mass/M_sun
  -p DENSITY, --density DENSITY
                        Local number density percentile [0,100]
  -s SFR, --SFR SFR     SFR in M_sun/yr
```    

**-z** *redshift*

**-m** *log<sub>10</sub>(stellar mass / M<sub>sun</sub>)*

**-p** *percentile of local number density* in the range [0,100] (see [Martin et al. 2018a](https://doi.org/10.1093/mnras/stx3057 "Martin+18a"))

**-s** *star formation rate in M<sub>sun</sub>/yr*

#### Importing the package

```
from progenitor_probability import progenitor_probability
>>> progenitor_probability(redshift=0.4, mass=10.8)
0.266751184855
```
pyyml 0.0.2
Use python in yaml
License: MIT

# pyyml
Use python in yaml
## 基本使用
在python中使用：
```python
from pyyml import load

with open('conf.yml') as f:
    raw_conf = f.read()
config = load(raw_conf)
print(config)
```
在yaml中 `${...}` 中的内容将会被python执行：

yaml内容
```yaml
sum: ${1 + 1}
```
实际解析内容
```python
{'sum': 2}
```
通过在文件开头第一行注释来导入所需的包：

```yaml
# libs:['os', 'os.path:path', 'this']
```
通过 [包名]:[别名] 取别名导入包

`'os.path:path' 等同于 python 中的 from os import path`

例如yaml文件内容
```yaml
# libs:['os', 'os.path:path', 'this']
os_name: ${os.name}
base_dir: ${path.abspath(path.dirname(__file__))}
```
metric-client 1.1.0
metric client for python
License: MIT

# metric-client

python版的指标收集客户端

## 安装

```sh
pip install metric-client
```

## 使用

```python
from metric_client import MetricClient

# send_api: 接收推送数据的API
# token：负责appcode的身份验证，不能为空
token = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2ZXJzaW9uIjoxLCJhcHBjb2RlIjoib3BzX21ldHJpY2d3IiwidG9fdXNlciI6Inh4eHguemhhbyIsImlhdCI6MTU0NTczNTczMH0.Aj8srWIjyFwxhcMrZlCxyNlP44uLG0iiR31ynyYd4Bw'
send_api = 'http://localhost:6066/v1/metric/send'
metric = MetricClient(send_api, token)
metric.set('set_metric_name', 344, ts=1554704683)
metric.counter('counter_metric_name', 100)
metric.max('max_metric_name', 100)
metric.min('min_metric_name', 100)
metric.avg('avg_metric_name', 100)
metric.timing('timing_metric_name', 100, ts=1554704683)
metric.summary('summary_metric_name', 100)
metric.summary('summary_low_metric_name', 20, percentiles=[1, 5, 10])
# percentiles 表示输出的指标中包含的百分位数列表，默认是 [50, 90, 95, 99]
```

## 指标类型

### set

set 类型的指标，在一个指标周期（比如1分钟）中只会取最后一个值，是最普通的的指标类型。

该指标不需要在一个周期内做任何聚合操作，总是记录最后一个值。

### counter

counter 类型的指标，可用于记录qps等场景。

在一个指标周期（比如1分钟）可以发生变化，可以增加，也可以减小，该指标最后输出一个汇总值（即周期内所有值的求和）。

### max

max 类型的指标，可用于记录一分钟内的最大值。该指标需要在一个周期内做聚合操作，最后输出一个最大值。

### min

min 类型的指标，可用于记录一分钟内的最小值。该指标需要在一个周期内做聚合操作，最后输出一个最小值。

### timing

timing 类型的指标，可用于记录代码执行时间。

每次update指定一个value和count（默认为1），该指标需要在一个周期内做聚合操作，最后输出一组汇总值（最大值，最小值，总和，总数量，平均值）。


### summary

summary 是一种更加复杂的统计类型指标，该指标需要在一个周期内做聚合操作，会输出一组汇总值，如平均值，总和，总数量，百分位数（P90，P99等）。

该指标的的输入支持两种类型：元数据（raw）和t-digest序列化字符串（tdigest）
pyspark-sugar 0.4.1
SparkUI enchancements with pyspark
License: MIT
Keywords: pyspark

# pyspark-sugar

Set python traceback on dataframe actions, enrich spark UI with actual business logic stages of spark application.


## Installation

Via pip:
```
pip install pyspark-sugar
```

Directly from github:
```
pip install git+https://github.com/sashgorokhov/pyspark-sugar.git@master
```

## Usage


Consider this synthetic example
```python
import random

import pyspark
import pyspark_sugar
from pyspark.sql import functions as F


# Set verbose job description through decorator
@pyspark_sugar.job_description_decor('Get nulls after type casts')
def get_incorrect_cast_cols(sdf, cols):
    """
    Return columns with non-zero nulls amount across its values.

    :param cols: Subset of columns to check
    """
    cols = set(cols) & set(sdf.columns)
    if not cols:
        return {}
    null_counts = sdf.agg(*[F.sum(F.isnull(col).cast('int')).alias(col) for col in cols]).collect()[0]
    return {col: null_counts[col] for col in cols if null_counts[col]}


def main():
    sc = pyspark.SparkContext.getOrCreate()  # type: pyspark.SparkContext
    ssc = pyspark.sql.SparkSession.builder.getOrCreate()
    
    # Define a job group. All actions inside job group will be grouped.
    with pyspark_sugar.job_group('ETL', None):
        sdf = ssc.createDataFrame([
            {'CONTACT_KEY': n, 'PRODUCT_KEY': random.choice(['1', '2', 'what', None])}
            for n in range(1000)
        ]).repartition(2).cache()

        sdf = sdf.withColumn('PRODUCT_KEY', F.col('PRODUCT_KEY').cast('integer'))
        
        # Collect action inside this function will have nice and readable description
        print(get_incorrect_cast_cols(sdf, ['PRODUCT_KEY']))

        sdf = sdf.dropna(subset=['PRODUCT_KEY'])
    
    # You can define several job groups one after another.
    with pyspark_sugar.job_group('Analytics', 'Check dataframe metrics'):
        sdf.cache()
        
        # Set job description for actions made inside context manager.
        with pyspark_sugar.job_description('Get transactions count'):
            print(sdf.count())

        sdf.show()

    sc.stop()


if __name__ == '__main__':
    # Patch almost-all dataframe actions with code that will set stack trace to job details description
    with pyspark_sugar.patch_dataframe_actions():
        main()

```

This how SparkUI pages will look like:

![SparkUI jobs page screenshot](docs/screenshots/screenshot_jobs.png)
![SparkUI sql page screenshot](docs/screenshots/screenshot_sql.png)

Now SparkUI is full of human-readable descriptions so even you manager can read it! Sweet.
nblogic 1.0.20
NBLOGIC for the win
License: MIT
Requires Python: >=3.6.0


# NBLOGIC

The integration of KLOGIC and ICIT service for using information in various way.
This program run on Python 3 and terminal use-only 
JSON output is provided for back-end calling


## Prerequisites

This program needs Python 3 and following libraries

* [MechanicalSoup](https://mechanicalsoup.readthedocs.io/en/stable/) - A Python library for automating website interaction
* [art](https://pypi.org/project/art/) - Used to generate ASCII art


### Installing

First, install this project by pip

```
pip install nblogic
```

### Note

Add apiMode and JSON language (1.0.9)

## Authors

* **Thanawat Petchuen** - [Thanawat(GitHub)](https://github.com/thanawatpetchuen) - [Thanawat(Bitbucket)](https://bitbucket.org/thanawatpetchuen/) 


## License

This project is licensed under the MIT License