semvec/pypi-packages huggingface dataset:
entropymethod 0.3.12
Estimate the number of clusters in your data set.
Requires Python: >=3.6

The Entropy Method is an algorithm to estimate the number of clusters in a data set.

The method relies on the stability principle. Subsamples of your data set are repeatedly clustered into different numbers of clusters; the algorithm then assesses which number of clusters provides the most stable clustering.

For more details, see the pre-print (link to be provided)
kmeseg 0.1.1
Useful AI library for chemistry.
Requires Python: >=3.5, <4
Keywords: AI,tensorflow,development

![TensorFlow](https://img.shields.io/badge/TensorFlow-%23FF6F00.svg?style=flat&logo=TensorFlow&logoColor=white) 
![Keras](https://img.shields.io/badge/Keras-%23D00000.svg?style=flat&logo=Keras&logoColor=white) 
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/2480da2b4a684a198c75db6ed249edda)](https://www.codacy.com/gh/namirinz/KME-WordSegmentation/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=namirinz/KME-WordSegmentation&amp;utm_campaign=Badge_Grade)
# KME Word Segmentation
AI for tokenize chemical IUPAC name using tensorflow and keras.

## Prepare training dataset
#### What to have
1. CHAR_INDICES: dictionary with key is character [string], value is number [int] *(use to preprocess text to number)*

2. Dict_cut: Input text with determined (by '|' ) where to be cut (use for create label)
```
Cyclo|prop|ane| |Non|a|-|1|,|8|-|di|yne| 
|1|,|3|-|di|chlorocyclo|hex|ane| |Hept|a|-|1|,|5|-|di|ene
```

3. Dict: Input text (raw) (use for train model)
```
Cyclopropane Nona-1,8-diyne 1,3-dichlorocyclohexane Hepta-1,5-diene
```

## Create dataset
1. Make JSON value as array of chemical name (dataset, dataset_cut)
2. Split array for **training dataset (90%)** (dataset_train, dataset_cut_train) and **validation dataset (10%)** (dataset_val, dataset_cut_val) 
3. Join item in each array together into text
4. Create dataset using **create_dataset function** that **take dataset_cut** then return **X_train**  (size: [text_length, look_back]) (dataset_cut that have been cut '|') and **label** (position where to cut 1 = cut, 0 = not cut)
5. Use tf.data.Dataset.from_tensor_slices((X, y)).batch_size(128) to make data easy to be train

## Create Model
1. We use 1x**Embedding** layer, 1x**Bidirection LSTM** layer, **Dense** Layer
2. Compiled **model optimizer = Adam**, **loss_function = Categorical Crossentropy** (becase we classify 2 label output 1 = cut, 0 = not cut) **call_back = [EarlyStopping, ModelCheckpoint]**
*Early stopping : Stop train model if validation_loss is being increase*
*ModelCheckpoint : Save model that has minimum validation_loss*

## After Train Model
- The output of the model is array (size: [batch_size, 2] determined which position to be cut (value = 0 -> not cut ; 1 -> cut))
```
[1 1 1 1 1 0 1 1 0 0 0 0 1 1 1 1 1 0 0 0 1 0 1 0 0 0 1 0 0]
```
- Tokenize dataset (text which **didn't** determined where to be cut) with label (output from model) using **word_tokenize function** that return **array of text that has been cut**
``` 
['1', ',', '2', '-', 'di', 'h', 'ydrox', 'y', '-', '2', '-', 'meth', 'yl', 'prop', 'ane']
```