semvec/pypi-packages huggingface dataset:
migration-lint 0.2.13
Tool for lint operations in DB migrations SQL
Requires Python: <4.0,>=3.9

# Migration Lint

![ci status](https://github.com/pandadoc/migration-lint/actions/workflows/ci.yml/badge.svg)
![Coverage](https://raw.githubusercontent.com/pandadoc/migration-lint/coverage-badge/coverage.svg)
![Py Version](https://img.shields.io/pypi/pyversions/migration-lint.svg)

`migration-lint` is the modular linter tool designed
to perform checks on database schema migrations
and prevent unsafe operations.

Features:

- Works with [Django migrations](https://docs.djangoproject.com/en/5.1/topics/migrations/),
  [Alembic](https://alembic.sqlalchemy.org/en/latest/) and raw sql files.
- Easily extensible for other frameworks.
- Can identify Backward Incompatible operations
  and check if they are allowed in the current context.
- Can identify "unsafe" operations, e.g. operations that acquire locks
  that can be dangerous for production database.

## Installation

```shell linenums="0"
poetry add "migration-lint"
```

```shell linenums="0"
pip install "migration-lint"
```

## Documentation

Read the docs on [GitHub Pages](https://pandadoc.github.io/migration-lint/)
allianceauth-discord-multiverse 0.0.1b3
Service module for managing an unlimited number of discord servers from a single auth instance. It can run side by side with the core auth discord service, or completely standalone.
Requires Python: >=3.8
Keywords: allianceauth, eveonline

# Discord Multiverse

Service module for managing an unlimited number of discord servers from a single auth instance. It can run side by side with the core auth discord service, or completely standalone.

Most of the code is borrowed from Alliance Auth's core Discord Service Module[link] and re-purposed to be guild agnostic.

Active Devs:

- [AaronKable](https://github.com/pvyParts)

```diff
-                 THIS APPLICATION IS NOT PRODUCTION READY
```

MID PRIO:

- TODO: Check the performance with massive servers/counts...
- TODO: ensure no cache conflicts
- TODO: Instructions and documentation
- TODO: Maybe custom client/secrets at a server level?

HIGH PRIO!

- TODO: Check AuthBot to see what it does.

### Installation

1.  pip install `package`
2.  Add `'aadiscordmultiverse',` to your `INSTALLED_APPS` in your projects `local.py`
    #TODO add redirect to sso app
    #TODO add settings for redirect or make it auto
    #TODO add the other step...
3.  Run migrations, collectstatic and restart auth.
4.  Setup your permissions as documented below

### Access Control and Server Permissions

- First and foremost to access **ANY** server a user must have this permission;
  - `aadiscordmultiverse | discord managed server | Can access the Discord Multiverse services`
- Access control to each server is managed by the Server Model
  - You can grant access by adding one of the following to the managed server in admin.
    - User State
    - User Groups
    - Main Characters Faction
    - Main Characters Alliance
    - Main Characters Corporation
    - Main Character

### Adding a Guild to Auth

1.  Add a new `DISCORD MANAGED SERVER` in admin
2.  Set the guild id to match your new server
3.  set any access control settings you need
4.  Set the included groups for the server. These are the only groups that will not be synced to this discord server. Enable the "managed groups" option if you want the auto corp/ali groups to sync magically too.
5.  Click Save
6.  Restart Auth
7.  Goto Services in the main auth site
8.  Click "Link Discord" on the new server and add your auth bot to the correct server.
9.  People can now join as required
cocup 0.1.2
Thom's COokie CUtter for Python
Requires Python: >=3.10

# Thom's COokie CUtter for Python (CoCuP)

## Description
A 'cookie cutter' to create a blank template for a python package in the target directory.

`cocup` will create a skeleton Python package with the following architechture:

```
my_package/
├── setup.py
├── LICENSE
├── README.md
├── .gitignore
├── my_package/
│   ├── __init__.py
│   ├── __main__.py
│   ├── main.py
│   ├── parser/
│   │   ├── __init__.py
│   │   └── parser.py
│   ├── utils/
│   │   ├── __init__.py
│   │   └── errors.py
├── example_data/
│   ├── example_in/
│   └── example_out/
└── tests/
```
Minimal scripts will be generated to produce a functioning skeleton package with a simple parser and basic logging functions.

## Installation
Either clone this repository, or download and install with PyPI:

`pip install cocup`

## Usage
### Minimal Usage
At minimum `cocup` requires two positional arguments: the project name and description. 
For example, to create a package called `my_package`, navigate to an empty directory and run:

`cocup my_package 'a description of my package'`

**IMPORTANT:** By default, `cocup` creates a GPL3.0 license for the package. If you want to use a different license,
you can either select a different license with `-l`, or delete this license and replace it with you own. **You alone
are responsible for licensing your software!** You have been warned!

### Recommended Usage
I recommend to use `cocup` with additional parameters to help you populate the package with information about the author.
Specifically:

`cocup my_package 'a description of my package' -a 'John Smith' -e john.smith@email.com`

### Other Parameters
There are a few other parameters that might be useful:

`-l` change the license, **IMPORTANT:** Defaults to GPL3.0, see above.

`-r` add requirements to the setup.py (e.g. `-r numpy,matplotlib`)

`-h` print the help text for the parser

## Citation
None - go crazy!

## Version Log
- 0.1.0 
  - initial version
- 0.1.1
  - fixed a bug when creating a setup.py with no requirements
- 0.1.2 
  - fixed printing and logging errors in main.py template
  - fixed inconsistent indentation in parser.py template
table-file-parser 0.1.5
load table files and parse them into parquet or pyarrow  format
Requires Python: >=3.12

# 代码结构说明

## ReadAndSplit_Chunk.py

设定读取文件的路径，输出路径，输出格式（非必要），分块大小

```
FormParser类
convert_to_output_format 转换输出
```

```
def processing_single_file(input_file_path : str, output_dir_path : str, output_format : str, max_size : int) -> list[dict]

入参：转换文件所在文件夹路径，文件输出路径, 文件输出格式（可选，默认parquet）, 分块大小（可选，默认1gb）
出参：sciendb标准化info
```

## ./huggingface

### my_huggingface_descriptive_statistics

根据parquet文件生成统计描述文件

```
DescriptiveStatisticsGenerator
generate_and_save_json（）
```



## ./utils

### my_pandas_type_confirming_utils

字符串类型数据推断

```
MyPandasTypeConfirmingUtils
```

### Invaild_Chars

文件名/路径名非法字符过滤

### split_file_name_format_checker

创建duckdb索引遵循：大文件分块后只处理part00，其余跳过；不分块直接创建duckdb索引
此工具用于识别哪些文件需要创建duckdb索引

```
FormatChecker.check_index_generate_format()
```

###  header_detector.py

检测csv等文件表头列名工具类

### csv_chunk_read_util

分块读取csv工具类，用于大文件读取并分块



## ./dataLoader

### base_file_loader.py

文件读取基类，包含```read_and_convert```读取转换抽象方法，需要子类实现

```
get_invalid_chars # 获取文件命名非法字符
_split_dataframe_by_size # df分割
```

### file_loader_factory

文件读取类的工厂类，根据传入需要读取的类型生成对应的数据loader

### excel_loader

excel类型读取转换逻辑

### hdf5_loader

hdf5类型文件读取转换

### single_table_loader

```
'.csv', '.sav', '.tsv', '.ods', '.parquet', '.tab' 类型转换
```



## ./dataWriter

### base_writer_utils

基础输出写文件类

包含通用方法

```
_analyze_all_schemas # 分析分块schema
auto_convert_column # 更安全的类型转换逻辑
clean_dataframe # 清洗dataframe
write_dataframe_in_chunks # 分块写入文件
save_as_output_format_with_chunks #输出
```

### parquet_writer.py

parquet输出

### pyarrow_writer.py

pyarrow输出



## ./despatch

## ./duckDBUtils

### my_parquet_fts_indexer_upgrade

读取parquet文件，识别字符串数据类型数据，并创建duckdb全文索引，输出duckdb文件
```
DuckDBFTSIndexer.create_fts_index()
```
每个duckdb中数据表名称固定为： data
生成的全文索引名称固定为：fts_main_data

# 统计文件statics.json类型
```
    "class_label",
    "float",
    "int",  
    "string_label",
    "string_text",
    "bool",
    "list",
    "datetime",
    "unknown"
```



# 文本类文件转Markdown

## ProcessText2MD.py

直接调用

process_text_file(input_file_path, output_path)

即可，目前支持的文件类型:

```
pdf
pptx
ppt
docx
doc
html
epub
```

使用的marker需要安装的依赖:

```
conda install weasyprint -c conda-forge
conda install Pango -c conda-forge
conda install fontTools -c conda-forge
conda install -c conda-forge fontconfig
conda install -c conda-forge pillow
conda install -c conda-forge freetype libpng
conda install transformers -c conda-forge
conda install tensorflow -c conda-forge
# 核心依赖
pip install marker-pdf[full]

```
displaycal 3.9.17
Display calibration and profiling with a focus on accuracy and versatility
Requires Python: >=3.9

![license](https://img.shields.io/badge/License-GPL%20v3-blue.svg)
![pyversion](https://img.shields.io/pypi/pyversions/DisplayCAL.svg)
![pypiversion](https://img.shields.io/pypi/v/DisplayCAL.svg)
![wheel](https://img.shields.io/pypi/wheel/DisplayCAL.svg)

DisplayCAL Python 3 Project
===========================

This project intended to modernize the DisplayCAL code including Python 3 support.

Florian Höch, the original developer, did an incredible job of creating and maintaining
DisplayCAL for all these years. But, it seems that, during the pandemic, very
understandably, he lost his passion to the project. Now, it is time for us, the
DisplayCAL community, to contribute back to this great tool.

This project is based on the ``HEAD`` of the Sourceforge version, which had 5 extra
commits that Florian has created over the ``3.8.9.3`` release on 14 Jan 2020.

Thanks to all the efforts put by the community DisplayCAL is now working with Python
3.9+:

![image](screenshots/DisplayCAL-screenshot-GNOME-3.9.5-running_on_python3.10.png)

Installation Instructions
=========================

Follow the instructions depending on your OS:

- [Windows](docs/install_instructions_windows.md)
- [MacOS](docs/install_instructions_macos.md)
- [Linux](docs/install_instructions_linux.md)

Have fun!
hpylib 0.1.0
A small Python library for simple parallel programming (spawn, finish, pmap).
Requires Python: >=3.8

# hpylib

**hpylib** is a small Python library that provides simple building blocks for parallel programming with threads. It is designed to make it easy to run tasks asynchronously, coordinate them, and perform parallel map operations, all with minimal boilerplate.  

It provides three main functions:

1. **`spawn(fn)`** — run a function asynchronously in a worker thread.  
2. **`finish()`** — context manager to wait for all asynchronous tasks submitted inside its block.  
3. **`pmap(fn, iterable)`** — parallel map: apply a function to each element of an iterable in parallel, returning the results as a list.

---

## Installation

```bash
pip install hpylib
```

---

## Usage

### 1. Asynchronous Tasks with `spawn` and `finish`

```python
from hpylib import spawn, finish
import time

def work(x):
    time.sleep(1)
    print(f"Processed {x}")

with finish():
    for i in range(5):
        spawn(lambda i=i: work(i))

print("All tasks completed!")

```

* spawn(fn) submits a task to run in the background.

* finish() waits for all tasks submitted inside its block to complete.

### 2. Parallel Map with `pmap`

```python
from hpylib import pmap
from math import isqrt

def is_prime(n):
    if n < 2:
        return False
    for i in range(2, isqrt(n)+1):
        if n % i == 0:
            return False
    return True

numbers = range(1, 100_000)
results = pmap(is_prime, numbers)
total_primes = sum(results)
print(f"Number of primes: {total_primes}")

```

* pmap(fn, iterable) automatically splits the iterable into chunks and distributes the work across the thread pool.

* Returns a list of results in the same order as the input.

* Blocks until all tasks are complete.
scm20367104703 0.1.1
This is package . A example package to find Roots of eqaution and solves an ordinary differential equation (ODE).
Requires Python: >=3.8

# scm20367104703

`scm20367104703`is A example package to find Roots of eqaution and solves an ordinary differential equation (ODE).

---

## Features
- **Fixed-Point Iteration**: use a function `fixed_point()` to use Fixed_point method
- **Secant Method**: use a function `secant()` to use Secant method
- **2nd-order Runge-Kutta**: Use function `rk2()`solves an ordinary differential equation (ODE) of the form 
$$
\frac{dy}{dx} = f(t, y)
$$
using the 2nd-order Runge-Kutta method (Midpoint Method).
- **5nd-order Runge-Kutta** : Use `rk5()` Solves the same type of ODE using a more accurate 5th-order Runge-Kutta method.
---

## Installation

You can install the package directly from PyPI:

```
pip install scm20367104703

```
---

## Example1
```
    def g(x):
        return (x**2 + 3) / 5
    initial_guess = 0.5
    root, iterations = fixed_point(g, initial_guess)
    print(f"\nApproximate: {root}")
```
---
## Example2
```
    def f(x):
        return x**3 - x - 2
    root, iterations = secant(f, 1, 2)
    print(f"\nApproximate : {root}")

```
---
## Example3
```
    def f(t, y):
        return y - t**2 + 1
    t0 = 0.0
    y0 = 0.5
    h = 0.5
    n_steps = 8
    rk2(f, t0, y0, h, n_steps)
```
---
## Example4
```
    def f(t, y):
        return y - t**2 + 1
    t0 = 0.0
    y0 = 0.5
    h = 0.5
    n_steps = 8
    rk5(f, t0, y0, h, n_steps)
```
pytroll-collectors 0.20.0
Pytroll data collectors
Requires Python: >=3.11

pytroll-collectors
==================


[![Build status](https://github.com/pytroll/pytroll-collectors/workflows/CI/badge.svg?branch=main)](https://github.com/pytroll/pytroll-collectors/workflows/CI/badge.svg?branch=main)

[![Build status](https://ci.appveyor.com/api/projects/status/5lm42n0l65l5o9xn?svg=true)](https://ci.appveyor.com/project/pytroll/pytroll-collectors)

[![Coverage Status](https://coveralls.io/repos/github/pytroll/pytroll-collectors/badge.svg?branch=main)](https://coveralls.io/github/pytroll/pytroll-collectors?branch=main)

[![PyPI version](https://badge.fury.io/py/pytroll-collectors.svg)](https://badge.fury.io/py/pytroll-collectors)

A set of modules and functions to support real-time processing of satellite
data with pytroll. It requires the Posttroll library for messaging. Satellite
data processing is often done in chunks, usually referred to as granules or
segments. Several chunks are normally required to cover a given area of
interest. When these chunks of data are processed and a Posttroll message is
sent modules in Pytroll-collectors support the collection of such chunks into
datasets, allowing other (Pytroll) processes to start processing on all the
chunks in one batch once all relevant chunks are available. This is, however,
only one example of what this package provides. There are also functionality to
list and stalk files in an object store like AWS S3 for instance.

[Documentation on Readthedocs](https://pytroll-collectors.readthedocs.io/en/latest/).
fiestaEM 0.1.0
Fast inference of electromagnetic signals with JAX
Requires Python: >=3.10
Keywords: sampling, inference, astrophysics, kilonovae, gamma-ray bursts, afterglow, surrogates

# fiesta 🎉

`fiesta`: **F**ast **I**nference of **E**lectromagnetic **S**ignals and **T**ransients with j**A**x

![fiesta logo](docs/fiesta_logo.jpeg)

**NOTE:** `fiesta` is currently under development. We have some basic documentation available under `./docs`. Feel free to contact us for any questions.

## Installation

`fiesta` can be installed from pip via 
```
pip install fiesta
```

Alternatively you can install it directly from source by cloning 
```
git clone git@github.com:nuclear-multimessenger-astronomy/fiestaEM.git
```
and then run 
```
pip install -e .
```
in the cloning directory. 

Note, that by default only the cpu version of `jax` is installed. If you want to use GPU acceleration, run 
```
pip install fiesta[gpu]
```
or install `jax[cuda12]` as indicated on the [`jax` webpage](https://docs.jax.dev/en/latest/installation.html#installation) manually.

Also, due to the file size limit on pypi, the pypi distribution only contains a reduced amount of built-in surrogates. If you want all built-in surrogates, we recommend *editable* installation from source or to download the `.pkl` files manually and store them in the `surrogates` folder of the site-package.
You can check which built-in surrogates are available by running 
```
python -c "from fiesta.inference.lightcurve_model import list_built_in_surrogates; list_built_in_surrogates()"
```


## Training surrogate models

To train your own surrogate models, have a look at some of the example scripts in the repository for inspiration. You can find them under `./surrogates/GRB/` and `./surrogates/KN/` in the respective model folders. The example section on training is currently work in progress. 

## Examples

We have example scripts for running an inference on AT2017gfo + GRB170817A. They can be found in `./examples/inference/`. We also plan to add an example section on training surrogates in the future.

## Acknowledgements

The logo was created by [ideogram AI](https://ideogram.ai/).