semvec/pypi-packages huggingface dataset:
beautifier 0.5.5
Cleanup and beautifier emails, urls and patterns
License: MIT

Beautifier
------

Simple library to cleanup and prettify url patterns, domains and so on.
Library helps to clean unicodes, special charecters and unnessesary 
redirection patterns from the urls and gives you clean date.
 
 
## Documentation

### Installation

```bash
pip install beautifier
```

## Basic Usages

#### Email Function

EMAIL cleanup API's

```python
from beautifier import Email
email = Email('me@imsach.in')

>>> email.domain
'imsach.in'

>>> email.username
 'me'

>>> email.is_free_email
 False
``` 

#### Url Function

URL cleanup API's


```python
from beautifier import Url
url = Url('https://in.linkedin.com/in/sachinphilip?authtoken=887nasdadasd6hasdtg21&secret=98jy766yhhuhnjk')

>>> url.cleanup
'https://in.linkedin.com/in/sachinphilip'

>>> url.domain
 'in.linkedin.com'

>>> url.param
 ['authtoken': '887nasdadasd6hasdtg21',
  'secret': '98jy766yhhuhnjk' ]
```
pydomaintextfeature 0.1.1
Generate text feature of domain for machine learn
License: MIT
Keywords: pydomaintextfeature

Overview
========
This lib generate text feature sparse vector for a given domain

Usage
=====
```
import pydomaintextfeature 
if __name__ == '__main__':
    while True:
        iput = raw_input()
        f = Ppydomaintextfeature.yDomainTextFeature({'ngram':3})
        print f.range()
        print f.feature(iput)
```
pydnsfeature 0.1.1
Get the dns feature for machine learning
License: MIT
Keywords: dns

Overview
========
This lib generate dns feature for machine learning

Usage
=====
```
import pydnsfeature
if __name__ == '__main__':
    f = pydnsfeature.PyDnsFeature()
    while True:
        iput = raw_input()
        info = f.feature(iput)
        print info
```
xinci 1.2.0
Chinese words extraction and new words discovery
License: MIT
Keywords: NLP,Chinese words extraction,New words discovery

# xinci 新词 & 抽词
xinci is a Python interface for chinese words extraction & new words extraction.
[https://pypi.org/project/xinci/]

## Requirements
Python >= 2.7

## Installation
### 1. using pip
```shell
pip install xinci
```
### 2. using setup.py
``` shell
git clone git@github.com:Lapis-Hong/xinci.git  
cd xinci 
pip setup.py install
```

## Usage
This package has two main use cases: words extraction and
find new words. 

### 1. command line
```shell
cd xinci
python word_extraction.py 
```
or 
```
./run.sh
```

### 2. python package
```python 
import xinci

# if you want to see logging events.
import logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s : %(levelname)s : %(message)s')

# init default dictionary or user dic,
dic = xinci.Dictionary()
# load vocab, vocab is a python set.
vocab = dic.load()  # or dic.dictionary
print(vocab)

# add words to dic
dic.add(['神马'])  # or dic.add_from_file('user.dic')
# remove words from dic
dic.remove(['神马'])  # or dic.remove_from_file('user.dic')

# extract new words, xc is a set
xc = xinci.extract('corpus.txt')
for w in xc:
    print(w)
# extract all words, c is a set
c = xinci.extract('corpus.txt', all_words=True)
for w in xc:
    print(w)
```
result
```angular2html
发现5个新词如下:
@新词	@词频
祛斑	13
后再	7
今日头条	9
洗净切	7
蛋液	9
```
### Notes: Iteratively add "not seems to new words" in result to common dic will improve a lot. 


## API documentation
```python
xc = xinci.extract(params)

```
List of available `params` and their default value:
```angular2html
corpus_file:           string, input corpus file (required)
common_words_file:     string, common words dic file [common.dic]
min_candidate_len:     int, min candidate word length [2]
max_candidate_len:     int, max candidate word length [5]
least_cnt_threshold:   int, least word count to extract [5]
solid_rate_threshold:  float, solid rate threshold [0.018]
entropy_threshold:     float, entropy threshold [1.92]
all_words: bool,       set True to extract all words mode [False]
save_file: string,     output file [None]
```

## References
The code is based on this java version
[https://github.com/GeorgeBourne/grid]