semvec/pypi-packages huggingface dataset:
instant-clip-tokenizer 0.1.1
Fast text tokenizer for the CLIP neural network
License: MIT

![Cover logo](./cover.svg)

# Instant CLIP Tokenizer: a fast tokenizer for the CLIP neural network

[![Documentation](https://docs.rs/instant-clip-tokenizer/badge.svg)](https://docs.rs/instant-clip-tokenizer/)
[![Crates.io](https://img.shields.io/crates/v/instant-clip-tokenizer.svg)](https://crates.io/crates/instant-clip-tokenizer)
[![PyPI](https://img.shields.io/pypi/v/instant-clip-tokenizer)](https://pypi.org/project/instant-clip-tokenizer/)
[![Build status](https://github.com/instant-labs/instant-clip-tokenizer/workflows/CI/badge.svg)](https://github.com/instant-labs/instant-clip-tokenizer/actions?query=workflow%3ACI)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE-MIT)

Instant CLIP Tokenizer is a fast pure-Rust text tokenizer for [OpenAI's CLIP model](https://github.com/openai/CLIP). It is intended to be a replacement for the original Python-based tokenizer included in the CLIP repository, aiming for 100% compatibility with the original implementation. It can also be used with [OpenCLIP](https://github.com/mlfoundations/open_clip) and other implementations using the same tokenizer.

In addition to being usable as a Rust crate it also includes Python bindings built with [PyO3](https://pyo3.rs/) so that it can be used as a native Python module.

For the microbenchmarks included in this repository, Instant CLIP Tokenizer is ~70x faster than the Python implementation (with preprocessing and caching disabled to ensure a fair comparison).

## Using the library

### Rust

```toml
[dependencies]
instant-clip-tokenizer = "0.1.0"
# To enable additional functionality that depends on the `ndarray` crate:
# instant-clip-tokenizer = { version = "0.1.0", features = ["ndarray"] }
```

### Python **(>= 3.9)**

```sh
pip install instant-clip-tokenizer
```

Using the library requires `numpy >= 1.16.0` installed in your Python environment (e.g., via `pip install numpy`).

### Examples

```rust
use instant_clip_tokenizer::{Token, Tokenizer};

let tokenizer = Tokenizer::new();

let mut tokens = Vec::new();
tokenizer.encode("A person riding a motorcycle", &mut tokens);
let tokens = tokens.into_iter().map(Token::to_u16).collect::<Vec<_>>();
println!("{:?}", tokens);

// -> [320, 2533, 6765, 320, 10297]
```

```python
import instant_clip_tokenizer

tokenizer = instant_clip_tokenizer.Tokenizer()

tokens = tokenizer.encode("A person riding a motorcycle")
print(tokens)

# -> [320, 2533, 6765, 320, 10297]

batch = tokenizer.tokenize_batch(["A person riding a motorcycle", "Hi there"], context_length=5)
print(batch)

# -> [[49406   320  2533  6765 49407]
#     [49406  1883   997 49407     0]]
```

## Testing

To run the tests run the following:

```sh
cargo test --all-features
```

You can also test the Python bindings with:

```sh
make test-python
```

## Acknowledgements

The vocabulary file and original Python tokenizer code included in this repository are copyright (c) 2021 OpenAI ([MIT-License](https://github.com/openai/CLIP/blob/main/LICENSE)).
streamlit-embeded 0.0.2
Common embeded for Streamlit.
License: MIT
Requires Python: >=3.6

# 📈 Streamlit Embeded

[![GitHub][github_badge]][github_link] [![PyPI][pypi_badge]][pypi_link] 

## Installation

```sh
pip install streamlit-embeded
```

## Getting started

```python
import streamlit as st

from streamlit_embeded import st_embeded

source = """

"""

st_embeded(source)

```
NeteaseCloudMusicApi 0.1.2
网易云音乐API NeteaseCloudMusicApi项目的 Python SDK
License: MIT
Requires Python: >=3.6.0


# NeteaseCloudMusic_PythonSDK
> 基于 [ NeteaseCloudMusicApi](https://github.com/Binaryify/NeteaseCloudMusicApi) 封装的 Python SDK。
> 网易云API Python版本。
> 现已同步原项目接口且测试通过的有200多个
> 已发布到PyPi，可直接使用pip安装
> 项目地址：[GitHub](https://github.com/2061360308/NeteaseCloudMusic_PythonSDK)

![](https://img.shields.io/badge/py_mini_racer-@0.6.0-red.svg)
![License](https://img.shields.io/badge/license-MIT-yellow)

### 依赖于
- [ NeteaseCloudMusicApi](https://github.com/Binaryify/NeteaseCloudMusicApi)
- [ NeteaseCloudMusicApi_V8 ](https://github.com/2061360308/NeteaseCloudMusicApi_V8)

### 原理
- 通过 `py_mini_racer` 调用 `NeteaseCloudMusicApi_V8` 的 `js` 方法。进一步进行了简单封装。

### 使用
- 安装 `pip install NeteaseCloudMusic`
- 导入API进行使用(具体查看`test.py`中的示例)
```python
from NeteaseCloudMusic import NeteaseCloudMusicApi, api_help, api_list
import os

netease_cloud_music_api = NeteaseCloudMusicApi()  # 初始化API
netease_cloud_music_api.cookie = os.getenv("COOKIE")  # 设置cookie
response = netease_cloud_music_api.request("song_url_v1", {"id": 33894312, "level": "exhigh"})  # 调用API

# 获取帮助
print(api_help())
print(api_help('song_url_v1'))
# 获取API列表
print(api_list())
```

> 注意： request(self, name, query=None) 的第一个参数为API名称，第二个参数为API参数，具体API名称和参数请参考 [NeteaseCloudMusicApi文档](https://docs.neteasecloudmusicapi.binaryify.com)，name支持`/song/url/v1`和`song_url_v1`两种写法。


### 开发
- 克隆项目 `git clone git@github.com:2061360308/NeteaseCloudMusic_PythonSDK.git`
- 安装依赖 `pip install -r requirements.txt`
- 目录/文件说明
├── package  项目包根目录
├── test_gender  生成测试代码的脚本
├── test.py  手动测试/ 使用示例


### 改进
> 下列API未支持
>
- apicache.js
- memory-cache.js
- request_reference.js
- avatar_upload.js
- cloud.js
- playlist_cover_update.js
- voice_upload.js
- register_anonimous.js
- verify_getQr.js

> 以下api未测试(这些接口测试起来比较繁琐)
> 
- /user/replacephone
- /audio/match
- /rebind
- /nickname/check
- /activate/init/profile
- /cellphone/existence/check
- /register/cellphone
- /captcha/verify
- /captcha/sent
- /login/refresh
- /logout
- /user/update
- /pl/count
- /playlist/update
- /playlist/desc/update
- /playlist/name/update
- /playlist/tags/update
- /event/forward
- /event/del
- /share/resource
- /send/text
- /send/playlist
- /playlist/create
- /playlist/tracks
- /daily_signin
- /fm_trash

### 欢迎提交PR