semvec/pypi-packages huggingface dataset:
bibmanager 1.4.13
A BibTeX manager for LaTeX projects
Requires Python: >=3.9

# bibmanager
> The Next Standard in BibTeX Management

[![Tests](https://github.com/pcubillos/bibmanager/actions/workflows/python-package.yml/badge.svg?branch=master)](https://github.com/pcubillos/bibmanager/actions/workflows/python-package.yml)
[![Documentation Status](https://readthedocs.org/projects/bibmanager/badge/?version=latest)](https://bibmanager.readthedocs.io/en/latest/?badge=latest)
[![PyPI](https://img.shields.io/pypi/v/bibmanager.svg)](https://pypi.org/project/bibmanager)
[![Conda Version](https://img.shields.io/conda/vn/conda-forge/bibmanager.svg)](https://anaconda.org/conda-forge/bibmanager)
[![GitHub](https://img.shields.io/github/license/pcubillos/bibmanager.svg?color=blue)](https://bibmanager.readthedocs.io/en/latest/license.html)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.2547042.svg)](https://doi.org/10.5281/zenodo.2547042)

### Install as:
```
pip install bibmanager
```
or:
```
conda install -c conda-forge bibmanager
```

### Docs at:
<https://bibmanager.readthedocs.io/en/latest/>

### Cite as:
```bibtex
  @MISC{Cubillos2020zndoBibmanager,
         author = {{Cubillos}, Patricio E.},
          title = "{bibmanager: A BibTeX manager for LaTeX projects, Zenodo, doi 10.5281/zenodo.2547042}",
           year = 2020,
          month = feb,
   howpublished = {Zenodo},
            eid = {10.5281/zenodo.2547042},
            doi = {10.5281/zenodo.2547042},
      publisher = {Zenodo},
         url    = {https://doi.org/10.5281/zenodo.2547042},
         adsurl = {https://ui.adsabs.harvard.edu/abs/2020zndo...2547042C},
        adsnote = {Provided by the SAO/NASA Astrophysics Data System},
  }
```

# <img alt="bibmanager" src="https://github.com/pcubillos/bibmanager/blob/master/docs/infograph.png" width="535">
llm-data-processor 0.1.2
A processor for LLM tasks
Requires Python: >=3.8

# LLM Data Processor

[![PyPI version](https://badge.fury.io/py/llm-data-processor.svg)](https://badge.fury.io/py/llm-data-processor)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

LLM Data Processor 是一个专为大语言模型(LLM)数据清洗和预处理设计的工具包，帮助研究人员和开发者高效地准备高质量训练数据。

## 特性

- 🧹 **高效数据清洗**：去除噪声、重复内容和低质量文本
- 🔍 **智能内容过滤**：基于规则和启发式方法过滤不相关内容
- 🧠 **格式标准化**：将不同来源的数据转换为统一的训练格式
- 📊 **数据质量评估**：提供数据质量指标和可视化工具
- ⚡ **批量处理**：支持大规模数据集的并行处理
- 🛠️ **可扩展性**：易于添加自定义处理器和过滤器

## 安装

```bash
pip install llm-data-processor
```

## 快速开始

### 基本用法

```python
from llm_data_processor import DataCleaner

# 初始化数据清洗器
cleaner = DataCleaner()

# 清洗单个文本
cleaned_text = cleaner.clean("这是一个包含HTML标签<div>和重复重复内容的文本。")
print(cleaned_text)  # 输出: "这是一个包含HTML标签和重复内容的文本。"

# 批量处理文件
cleaner.process_file("input.txt", "output.txt")
```

### 处理数据集

```python
from llm_data_processor import DatasetProcessor

# 初始化数据集处理器
processor = DatasetProcessor()

# 处理整个数据集
processor.process_dataset(
    input_path="raw_data/",
    output_path="processed_data/",
    filters=["duplicate", "low_quality", "non_chinese"],
    num_workers=4
)
```

### 自定义过滤器

```python
from llm_data_processor import DataCleaner, Filter

# 创建自定义过滤器
class MyCustomFilter(Filter):
    def apply(self, text):
        # 实现自定义过滤逻辑
        if "关键词" in text:
            return None  # 返回None表示过滤掉该文本
        return text  # 返回处理后的文本

# 使用自定义过滤器
cleaner = DataCleaner()
cleaner.add_filter(MyCustomFilter())
result = cleaner.clean("这是包含关键词的文本")
print(result)  # 输出: None (被过滤)
```

## 高级功能

### 配置文件支持

```python
from llm_data_processor import DataCleaner

# 从配置文件加载设置
cleaner = DataCleaner.from_config("config.yaml")
cleaner.process_directory("input_dir/", "output_dir/")
```

### 质量评估

```python
from llm_data_processor import QualityEvaluator

# 评估数据质量
evaluator = QualityEvaluator()
metrics = evaluator.evaluate("processed_data.jsonl")
print(f"数据质量分数: {metrics['quality_score']}")
print(f"重复率: {metrics['duplication_rate']}%")
```

### 命令行使用

```bash
# 处理单个文件
llm-data-processor clean --input input.txt --output output.txt

# 处理目录
llm-data-processor process --input-dir raw_data/ --output-dir clean_data/ --workers 4

# 评估数据质量
llm-data-processor evaluate --input dataset.jsonl --report quality_report.json
```

## 配置示例

`config.yaml` 示例:
upload-tm-records 1.0.1
Upload your trackmania records to a trackmania-records server!
Requires Python: <4,>=3.7
Keywords: trackmania, gaming

# trackmania-records - Uploader (client)

This is the uploader for trackmania-records, a Trackmania 2020 records sharing/comparison tool.

The uploader extracts your records for the training maps of the latest Trackmania version (from 2020)
and uploads them to the web application server so you can easily share your records with friends.

Install (requires Python 3 on your system):

	pip3 install upload-tm-records

Update:

	pip3 install --upgrade upload-tm-records

Usage:

	upload-tm-records http://example.com/upload.php
hydropostgis 0.1.1
水文地理信息服务平台
Requires Python: >=3.8

# HydroPostGIS - 水文地理信息服务平台

## 项目简介

HydroPostGIS是一个基于PostGIS的水文地理信息服务平台，旨在提供行政区、流域面、测站点等地理信息的存储、维护和服务化能力。本项目是知汛平台的基础服务之一，提供地理信息的服务化解决方案。

## 技术栈

- **数据存储**：PostgreSQL + PostGIS
- **空间数据格式**：GeoPackage, Shapefile, GeoJSON
- **服务发布**：GeoServer
- **开发工具**：Python, GDAL
- **客户端支持**：QGIS, ArcGIS

## 功能模块

### 1. 行政区管理

- 行政区数据的PostGIS存储
- 通过QGIS、ArcGIS进行维护
- 页面展示
- GeoServer同步
- 面平均雨量计算

### 2. 流域面管理

- 流域面数据的PostGIS存储
- 通过QGIS、ArcGIS进行维护
- 页面展示
- GeoServer同步
- 流域DEM的CDN缓存
- 面平均雨量计算

### 3. 测站点管理

- 测站点数据的PostGIS存储
- 与流域面的空间关系计算
- 基于权重的面平均雨量计算
- 数据异常检验

## 项目结构
HydroPostGIS/
├── api/                    # API接口层，提供RESTful服务
├── config.py               # 全局配置文件
├── core/                   # 核心业务逻辑
│   └── gis_service/        # GIS服务相关功能
│       ├── basin_region_relation.py    # 流域与行政区关系
│       ├── basin_station_relation.py   # 流域与测站关系
│       └── region_station_relation.py  # 行政区与测站关系
├── docs/                   # 项目文档
├── service/                # 服务层，连接API和核心层
├── utils/                  # 工具函数
│   ├── format_transform/   # 格式转换工具
│   │   └── geojson_to_gpkg.py  # GeoJSON转GeoPackage
│   ├── id_generator.py     # ID生成器
│   ├── json_util.py        # JSON处理工具
│   ├── minio/              # MinIO对象存储工具
│   │   ├── hres.py         # 高分辨率文件处理
│   │   └── minio_util_with_cert.py  # MinIO工具(带证书)
│   └── postgres_util.py    # PostgreSQL数据库工具
├── .env                    # 环境变量配置
└── requirements.txt        # 项目依赖

## 安装与配置

### 环境要求
- Python 3.8+
- PostgreSQL 12+ with PostGIS 3.0+
- GeoServer 2.19+
sep2tools 0.3.2
Useful functions for working with IEEE 2030.5 (SEP2)
Requires Python: >=3.11

# SEP2-Tools

This library provides some useful functions for working with IEEE 2030.5 (SEP2).

## Install

```sh
pip install sep2tools
```

## Certificate Inspection

Get the LFDI for a certificate. It will also do some validation checks.

```sh
python -m sep2tools cert-lfdi certs/dev-ABC-cert.pem
```

## Helper Functions

### Generating IDs

```python
from sep2tools.ids import generate_mrid

EXAMPLE_PEN = 1234
mrid = generate_mrid(EXAMPLE_PEN)
print(mrid)  # 2726-D70C-C6C2-40DB-B78E-9B38-0000-1234
```

### Bitmap Hex Mappings

Some helper functions are provided for calculating the hex representation of SEP2 bitmap fields.

```python
from sep2tools.hexmaps import get_role_flag

binval, hexval = get_role_flag(is_mirror=1, is_der=1, is_submeter=1)
print(binval)  # 0000000001001001
print(hexval)  # 0049
```

## Certificate Creation

Note the below CLI commands are only appropriate for testing purposes.
For production certificates, additional functions and policies may be required.

### Signing Certificates

Create a SERCA, and a MICA.

```sh
python -m sep2tools create-serca
python -m sep2tools create-mica certs/serca.pem certs/serca.key
```

### Device Certificates

To create a device certificate, first create a Key and Certificate Signing Request (CSR).

```sh
python -m sep2tools create-key --key-file certs/dev-ABC.key
```

Once you have the CSR, build the cert by signing with the MICA.

```sh
python -m sep2tools create-cert certs/dev-ABC.csr certs/mica.pem certs/mica.key --pen 12345 --serno ABC
```