semvec/pypi-packages huggingface dataset:
example-package-Jsqfylune 0.0.1
A python package which can help you visually track your training process of machine-learning
Requires Python: >=3.7

# BlackSwanlab 用户使用指南

为了帮助您更好地理解如何使用我们的`Swanlab`库进行模型训练过程中的损失和准确度实时跟踪，这里提供了详细的步骤。

## 1. 安装 BlackSwanlab

首先，您需要安装`BlackSwanlab`库。可以使用以下命令通过`pip`进行安装：

```Bash
pip install BlackSwanlab
```

## 2. 导入 BlackCat 类

在您的训练代码中，首先需要做的是导入我们提供的`BlackCat`类。您可以通过以下方式进行：

```Python
from BlackSwanlab.tracker import BlackCat
```

## 3. 定义训练函数

在开始训练之前，您需要定义一个训练函数。这是一个典型的训练函数例子：

```Python
def my_training_function(epoch):
    # 在这里插入您的训练代码
    # 根据您的模型和数据计算损失和精度
    # 假设我们在这里随机生成一些数据
    import random
    loss = random.uniform(0, 1)
    accuracy = random.uniform(0, 1)
    return loss, accuracy
```

**注意**: 您的训练函数必须接受一个名为`epoch`的参数，并返回两个值：`loss`和`accuracy`。

## 4. 实例化和使用 BlackCat 类

接下来，您可以实例化`BlackCat`类，并将您的训练函数作为参数传给它。然后，只需调用`track()`方法即可启动训练：

```Python
tracker = BlackCat(epochs=100, user_training_func=my_training_function)
tracker.track()
```

在训练开始后，Swanlab将会启动一个Flask服务器，实时更新并提供训练的损失和精度数据。

## 5. 查看实时训练指标

最后，您可以使用任意的web浏览器，访问 http://localhost:5000/metrics 来查看训练的实时指标。

**注意**: 如果您运行的主机有端口访问限制，您可能需要对您的网络配置进行相应的调整。

这就是所有您需要了解的训练和实时监控训练过程的步骤。如果您遇到任何问题，欢迎提交问题或联系我们。Swanlab团队祝您使用愉快！
multiprocessing-mq 0.5
在多进程内维护一个消息队列，更方便的使用多进程

# Convenient multiprocessing with Message Queues
[pypi](https://pypi.org/project/multiprocessing-mq/) \
[中文](./README-cn.md)
## Features
- Lightning-fast startup: New processes launch in just 10ms.
- Lightning-fast execution: Sending code (with parameters) takes only 0.8ms.
- Easy to use: Thanks to `interactivity.py`, minimal modifications are required.
- Automatically suspends when idle, conserving resources.

## Usage
(We recommend referring to `example.py` for guidance)
### Creating Processes
```python
import multiprocessing_mq as mq
my_pro = mq.Process(init=init_code, suspend=True, rest_time=0)
```
#### Parameters
- `init`: Initialization code.
- `suspend`: Whether to suspend.
- `rest_time`: Sleep time.
- `process_events`: Performs event loop while waiting for process to resume, preventing UI from freezing.

### Usage
There are multiple ways to send tasks to child processes. Among them, using `inter` is the simplest.
#### inter
Create a virtual class to simulate the environment of a child process. For example, to retrieve the value of `a` inside the child process, you only need `my_pro.inter.a`. Running a function only requires `my_pro.inter.a()`.
#### run_com
Run a function and wait for a return value.
```python
run_com(self, code: str, args: dict = {}, process_events=None)
```
- `code`: The code to execute.
- `args`: Arguments (**Note:** `args` won't automatically pass parameters; you need to pass them yourself within `code`).
- `process_events`: Performs an event loop while waiting for the process result, preventing the UI from freezing.

**Note:** `run_com` uses `eval` to execute functions, so you cannot use it to modify variable values (you can use `run_without_return` instead).
#### run_without_return
```python
run_without_return(self, code: str, args: dict = {})
```
Run a function without waiting for a return value; internally uses `exec`.
threadinglogger 0.0.6
Logger using a thread.
Requires Python: >=3.9

# threadinglogger
Logger using a thread.  
Log Type : print console, write file, insert postgresql

## Start and Stop
```python
import threadinglogger
threadinglogger.start()
# code...
threadinglogger.stop()
```

## Log
Default is print console only.
```python  
threadinglogger.information("Hello, world!")
threadinglogger.exception("Hello, world!")
# exception() output 'traceback.format_exc()' next line of message if exists.
threadinglogger.signal("Hello, world!")
threadinglogger.other("OTHER", "Hello, world!")
```

## Initialize Print console.
Default is enable.  
Set disable auto if process run background.  
Excute before 'start()'.  
```python  
import threadinglogger
threadinglogger.enable_print_console()
# or
threadinglogger.disable_print_console()
```
## Initialize Write file.
Default is disable.  
Set enable auto if success run 'init_write_file()'.  
Excute before 'start()'.  
```python  
# {write_file_name}.logging.log
threadinglogger.init_write_file(write_file_name= "test")
# change path. default is '.'(current path)
threadinglogger.init_write_file(write_file_name= "test", write_file_path='./logs')
# change extension. default is 'log'
threadinglogger.init_write_file(write_file_name= "test", write_file_path='./logs', write_file_extension= 'txt')
```
## Initialize Insert PostgreSQL.
Default is disable.  
Set enable auto if success run 'init_postgresql()'.  
Excute before 'start()'.  
```python
def init_postgresql(host:str, port:int, database_name:str, table_name:str, user_id:str, password:str):
```

## Change Format
Change format string.  
No sync with logging.  

Format Variables:  
log_type  
log_type_length  
date_str  
thread_id  
thread_id_length  
process_id  
file_name  
file_name_length  
file_lineno  
file_lineno_length  
text  
  
Default:
```python  
"{date_str} {thread_id:0{len_thread_id}d}:TID {log_type:{len_log_type}} {file_name:>{maxlen_file_name}}:{file_lineno:<{maxlen_file_lineno}} {text}"
```
Output:  
'2100-01-01 18:02:47.047984 123456:TID INFO /dir:272 Message'  

Change:
```python
output_format = "{date_str} {log_type:{log_type_length}} {text}"
threadinglogger.change_print_format(output_format)
threadinglogger.change_write_format(output_format)
```
Output:  
'2100-01-01 18:02:47.047984 INFO Message'  

### Change Date Format
Default is '%Y-%m-%d %H:%M:%S.%f'
```python  
threadinglogger.change_date_format_print_console('%H:%M:%S')
threadinglogger.change_date_format_write_file('%H:%M:%S')
```

### Change Thread ID Lenght Format
Default is 6
```python  
threadinglogger.change_thread_id_length_print_console(4)
threadinglogger.change_thread_id_length_write_file(4)
```  

### Change Log Type Length
Default is 4
```python
threadinglogger.change_log_type_length_print_console(5)
threadinglogger.change_log_type_length_write_file(5)
```
fundrive-oss 0.11.29
fundrive-oss

# 概述

1.支持百度云盘、蓝奏云盘的上传、下载  
2.百度云盘分享链接经常失效、蓝奏云永久有效

# 使用场景

数据产出后直接上传备份  
1.网上爬取的数据  
2.模型训练的结果  
3.大一点的数据集

# 分类

| 序号 | 网盘         | 文档                               | 实例                                   | 支持内容   |
| :--: | :----------- | :--------------------------------- | :------------------------------------- | :--------- |
|  1   | 基础下载     | [doc](./fundrive/base/README.md)   | [example](./example/base_example.py)   | 多线程下载 |
|  2   | 百度云       | [doc](./fundrive/baidu/README.md)  | [example](./example/baidu_example.py)  | 上传、下载 |
|  3   | 蓝奏云(推荐) | [doc](./fundrive/lanzou/README.md) | [example](./example/lanzou_example.py) | 上传、下载 |

# 安装

```bash
pip install git+https://github.com/Leon406/lanzou-gui.git --no-deps
pip install fundrive
```

或者直接从 GitHub 安装

```bash
pip install git+https://github.com/notechats/fundrive.git
```

# 为啥要做这个

1.学习中经常用的数据集，尤其是国外数据集，下载速度贼慢  
2.网上找数据比较麻烦  
3.预借助蓝奏云，将一些国内外公开数据集放到一起，整理到[notedata](https://github.com/notechats/notedata)

#参考
百度云盘的 python-api，[官方 API](https://openapi.baidu.com/wiki/index.php?title=docs/pcs/rest/file_data_apis_list)  
蓝奏云的 python-api [参考](https://github.com/zaxtyson/LanZouCloud-API)