---
language:
- zh
tags:
- search
---
# Cross Language Search
## Search cliassical CN with modern ZH
* In some cases, Classical Chinese feels like another language, I even trained 2 translation models ([1](https://huggingface.co/raynardj/wenyanwen-chinese-translate-to-ancient) and [2](https://huggingface.co/raynardj/wenyanwen-ancient-translate-to-modern)) to prove this point.
* That's why, when people wants to be savvy about their words, we choose to quote our ancestors. It's exactly like westerners like to quote Latin or Shakespeare, the difference is we have a much bigger pool to choose.
* This model helps you **find** text within **ancient Chinese** literature, but you can **search with modern Chinese**
# 跨语种搜索
## 博古搜今
* 我不记得是谁, 哪个朝代,我只记得大概这么一个事儿,我就能模糊找到原文
* 我不记得原文, 但是我只记得原文想表达的现代汉语意思, 希望能找出来引用一下。
* 我在写文章, 有个观点, 我想碰运气看看古人有没有提过同样类似的说法。
* 我只是想更有效率地阅读古文
推荐的使用通道如下,当然, cosine距离搜索相关的框架和引擎很多, 大家自己看着适用的选
装包
```shell
pip install -Uqq unpackai
pip install -Uqq SentenceTransformer
```
搜索语句的函数
```python
from unpackai.interp import CosineSearch
from sentence_transformers import SentenceTransformer
import pandas as pd
import numpy as np
TAG = "raynardj/xlsearch-cross-lang-search-zh-vs-classicical-cn"
encoder = SentenceTransformer(TAG)
# all_lines is a list of all your sentences
# all_lines 是一个你所有句子的列表, 可以是一本书, 按照句子分割, 也可以是很多很多书
all_lines = ["句子1","句子2",...]
vec = encoder.encode(all_lines, batch_size=32, show_progress_bar=True)
# consine距离搜索器
cosine = CosineSearch(vec)
def search(text):
enc = encoder.encode(text) # encode the search key
order = cosine(enc) # distance array
sentence_df = pd.DataFrame({"sentence":np.array(all_lines)[order[:5]]})
return sentence_df
```
将史记打成句子以后, 搜索效果是这样的:
```python
>>> search("他是一个很慷慨的人")
```
```
sentence
0 季布者,楚人也。为气任侠,有名於楚。
1 董仲舒为人廉直。
2 大将军为人仁善退让,以和柔自媚於上,然天下未有称也。
3 勃为人木彊敦厚,高帝以为可属大事。
4 石奢者,楚昭王相也。坚直廉正,无所阿避。
```
```python
>>> search("进入军营,必须缓缓牵着马骑")
```
```
sentence
0 壁门士吏谓从属车骑曰:将军约,军中不得驱驰。
1 起之为将,与士卒最下者同衣食。卧不设席,行不骑乘,亲裹赢粮,与士卒分劳苦。
2 既出,沛公留车骑,独骑一马,与樊哙等四人步从,从间道山下归走霸上军,而使张良谢项羽。
3 顷之,上行出中渭桥,有一人从穚下走出,乘舆马惊。
4 元狩四年春,上令大将军青、骠骑将军去病将各五万骑,步兵转者踵军数十万,而敢力战深入之士皆属骠骑。
```
## 其他资源清单
* [项目源代码 🌟, 欢迎+star提pr](https://github.com/raynardj/yuan)
* [跨语种搜索 🔎](https://huggingface.co/raynardj/xlsearch-cross-lang-search-zh-vs-classicical-cn)
* [现代文翻译古汉语的模型 ⛰](https://huggingface.co/raynardj/wenyanwen-chinese-translate-to-ancient)
* [古汉语到现代文的翻译模型, 输入可以是未断句的句子 🚀](https://huggingface.co/raynardj/wenyanwen-ancient-translate-to-modern)
* [断句模型 🗡](https://huggingface.co/raynardj/classical-chinese-punctuation-guwen-biaodian)
* [意境关键词 和 藏头写诗🤖](https://huggingface.co/raynardj/keywords-cangtou-chinese-poetry)