semvec/pypi-packages huggingface dataset:
sgnlp 0.4.0
Machine learning models from Singapore's NLP research community
Requires Python: >=3.7
Keywords: NLP,machine learning,deep learning

# SG-NLP

Machine learning models from Singapore's natural language processing (NLP) research community.

`sgnlp` is a Python package that allows you to easily get started on using various (NLP) models implemented using the
Pytorch and Transformers frameworks.

We have an accompanying [demo site](https://sgnlp.aisingapore.net/) where you can interact with our models and get a
better understanding on how they work.

## Installation

* Python >= 3.8

```
pip install sgnlp
```

## Documentation

Visit our [documentation](https://sgnlp.aisingapore.net/docs/) for tutorials.

## License

Code and models from this project are released under the MIT License unless otherwise stated. If a model's code is under
a separate license, it can be found in the respective model's folder.
gec-noise4korean 0.0.1
Noise Generator for Korean Text
Requires Python: >=3.7

# Noise Generator for Korean Text Grammar Error Correction Model

### This model is using python-mecab-ko as the main tokenizer.
### If you need to know the token types check the website below
### https://m.blog.naver.com/PostView.naver?isHttpsRedirect=true&blogId=aramjo&logNo=221404488280

# Requirements

## Python >= 3.7 

```consol
pip install wget
pip install konlpy
pip install hangul_utils
pip install hangul_jamo
pip install inko
pip install g2pk
```
 Currently not available in Windows OS.<br />


# How to use
## Preparation
```consol
pip install gec_noise4korean
```

```
import project
nsgec = noise_generate.NoiseGenerate("text data directory", "error Type")
The result will appear in results file. 
```

Requires a Text file with its directory as the First argument when calling the function.<br />
Text data should be in txt file and each sentence should be organized through each line.<br />

# Functions (Error Type) (second argument)

* spacing_error
> Creates Random spacing error according to rule written below<br />
> (명사, 관형어) + (명사 , 관형어 의존명사) 인 경우 띄어쓰기를 만든다<br />
> 조사와 접사를 띄어써서 생기는 오류<br />
> 단어 가운대에 띄어쓰기를 해서 생기는 오류<br />

* punctuation_error
> Converts puntuation within the same type<br />
> 문장속의 토큰중 품사가 “S-” 로 시작하는 랜덤하게 고른 토큰을 같은 품사표 내에서 랜덤하게 변경한다.<br />

* punctuation_error
> Converts puntuation within the same type<br />
> 문장속의 토큰중 품사가 “S-” 로 시작하는 랜덤하게 고른 토큰을 같은 품사표 내에서 랜덤하게 변경한다.<br />

* phonetic_first_error or phonetic_last_error
> Converts the first or last character if it exists in phonetic data list<br />
> 여 > 녀 / 율 > 률<br />

* remove_josa_error
> Randomly remove josa from sentence<br />
> Remove token which type starts with "J"<br />

* addition_error
> Randomly add consonant to character which has "ㅇ" as its starting consonant or doens't have the last consonant<br />
> 아기 > 바기 / 다치 > 닫치<br />

* separation_error
> From the randomly selected word, decompose the letter to consonant and vowel<br />
> 할 > ㅎㅏㄹ<br />

* typing_language_error
> Convert Korean text to English text regarding to the same keyboard position<br />
> 한글 > gksrmf / 고양이 > rhdiddl<br />

* postposition_diff_josa_error or postposition_same_josa_error
> Convert a Josa to either different or same type of Josa from the Josa dataset.
> 를 > 을 / 에게 > 할

* busa_error
> Convert busa "이" to "히" either way
> 부단히 > 부단이 / 같이 > 같히

* middle_shiot_error
> If there is "ㅅ" as the last consonant in a word longer than 2 characters, erase the last consonant "ㅅ"
> 숫자 > 수자 / 찻잔 > 차잔
noise-generator4korean 0.0.1
Noise Generator for Korean Text
Requires Python: >=3.7

# Noise Generator for Korean Text Grammar Error Correction Model

### This model is using python-mecab-ko as the main tokenizer.
### If you need to know the token types check the website below
### https://m.blog.naver.com/PostView.naver?isHttpsRedirect=true&blogId=aramjo&logNo=221404488280

# Requirements

* Python >= 3.7 

> Currently not available in Windows OS.<br />
> Correct Text dataset that has each sentence organized through each line.<br />
> Remind that this is an Noise Generator for Korean Text.<br />

# How to use
## in terminal
```consol
pip install noise-generator4korean
```
## in python
```
import project
nsgec = noise_generate.NoiseGenerate("text data directory", "error Type")
The result will appear in results file. 
```

    

## Need a Text file with its directory as the First argument when calling the function.



# Functions (Error Type) (second argument)

* spacing_error
> Creates Random spacing error according to rule written below<br />
> (명사, 관형어) + (명사 , 관형어 의존명사) 인 경우 띄어쓰기를 만든다<br />
> 조사와 접사를 띄어써서 생기는 오류<br />
> 단어 가운대에 띄어쓰기를 해서 생기는 오류<br />

* punctuation_error
> Converts puntuation within the same type<br />
> 문장속의 토큰중 품사가 “S-” 로 시작하는 랜덤하게 고른 토큰을 같은 품사표 내에서 랜덤하게 변경한다.<br />

* punctuation_error
> Converts puntuation within the same type<br />
> 문장속의 토큰중 품사가 “S-” 로 시작하는 랜덤하게 고른 토큰을 같은 품사표 내에서 랜덤하게 변경한다.<br />

* phonetic_first_error or phonetic_last_error
> Converts the first or last character if it exists in phonetic data list<br />
> 여 > 녀 / 율 > 률<br />

* remove_josa_error
> Randomly remove josa from sentence<br />
> Remove token which type starts with "J"<br />

* addition_error
> Randomly add consonant to character which has "ㅇ" as its starting consonant or doens't have the last consonant<br />
> 아기 > 바기 / 다치 > 닫치<br />

* separation_error
> From the randomly selected word, decompose the letter to consonant and vowel<br />
> 할 > ㅎㅏㄹ<br />

* typing_language_error
> Convert Korean text to English text regarding to the same keyboard position<br />
> 한글 > gksrmf / 고양이 > rhdiddl<br />

* postposition_diff_josa_error or postposition_same_josa_error
> Convert a Josa to either different or same type of Josa from the Josa dataset.
> 를 > 을 / 에게 > 할

* busa_error
> Convert busa "이" to "히" either way
> 부단히 > 부단이 / 같이 > 같히

* middle_shiot_error
> If there is "ㅅ" as the last consonant in a word longer than 2 characters, erase the last consonant "ㅅ"
> 숫자 > 수자 / 찻잔 > 차잔