semvec/pypi-packages huggingface dataset:
meta-msg 0.1
Meta-msg it's an additional layer for your binary exchange systems. It lets you add status/error codes to your messages and raise/handle errors between microservices in traditional (pythonic) way.
License: MIT
Keywords: message,nats,rabbitmq

Meta-msg it's an additional layer for your binary exchange systems. It lets you add status/error codes to your messages and raise/handle errors between microservices in traditional (pythonic) way.
bnaug 1.1.2
bnaug is a text augmentation tool for Bangla text.
License: MIT
Requires Python: >=3.7

# bnaug (Bangla Text Augmentation)
__bnaug__ is a text augmentation tool for Bangla text.

## Installation
```
pip install bnaug
```
- Dependencies
    - pytorch >=1.7.0
    
## Demo Notebook
- [bnaug demo](https://github.com/sagorbrur/bnaug/blob/main/notebook/bnaug_demo.ipynb)

## Necessary Model Links
- [word2vec](https://huggingface.co/sagorsarker/bangla_word2vec/resolve/main/bangla_word2vec_gen4.zip)
- [glove vector](https://huggingface.co/sagorsarker/bangla-glove-vectors/resolve/main/bn_glove.300d.zip)

## Sentence Augmentation
### Token Replacement
- Mask generation based augmentation

    ```py
    from bnaug.sentence import TokenReplacement

    tokr = TokenReplacement()
    text = "আমি ঢাকায় বাস করি।"
    output = tokr.masking_based(text, sen_n=5)
    ```

- Word2Vec based augmentation

    ```py
    from bnaug.sentence import TokenReplacement

    tokr = TokenReplacement()
    text = "আমি ঢাকায় বাস করি।"
    model = "msc/bangla_word2vec/bnwiki_word2vec.model"
    output = tokr.word2vec_based(text, model=model, sen_n=5, word_n=5)
    print(output)
    ```

- Glove based augmentation

    ```py
    from bnaug.sentence import TokenReplacement

    tokr = TokenReplacement()
    text = "আমি ঢাকায় বাস করি।"
    vector = "msc/bn_glove.300d.txt"
    output = tokr.glove_based(text, vector_path=vector, sen_n=5, word_n=5)
    print(output)
    ```

### Back Translation
Back translation based augmentation first translate Bangla sentence to English and then again translate the English to Bangla.

```py
from bnaug.sentence import BackTranslation

bt = BackTranslation()
text = "বাংলা ভাষা আন্দোলন তদানীন্তন পূর্ব পাকিস্তানে সংঘটিত একটি সাংস্কৃতিক ও রাজনৈতিক আন্দোলন। "
output = bt.get_augmented_sentences(text)
print(output)

```

### Text Generation
- Paraphrase generation

```py
from bnaug.sentence import TextGeneration

tg = TextGeneration()
text = "বিমানটি যখন মাটিতে নামার জন্য এয়ারপোর্টের কাছাকাছি আসছে, তখন ল্যান্ডিং গিয়ারের খোপের ঢাকনাটি খুলে যায়।"
output = tg.parapharse_generation(text)
print(output)
```

### Random Augmentation
- Random remove part and generate new sentence

    At present it's removing word, stopwords, punctuations, numbers and generate new sentences

    ```py
    from bnaug.sentence import RandomAugmentation

    raug = RandomAugmentation()
    sentence = "আমি ১০০ বাকি দিলাম"
    output = raug.random_remove(sentence)
    print(output)

    ```

    or apply individually

    ```py
    from bnaug import randaug

    text = "১০০ বাকি দিলাম"
    output = randaug.remove_digits(text)
    print(output)

    text = "১০০! বাকি দিলাম?"
    output = randaug.remove_punctuations(text)
    print(output)

    text = "আমি ১০০ বাকি দিলাম"
    randaug.remove_stopwords(text)
    print(output)

    text = "আমি ১০০ বাকি দিলাম"
    randaug.remove_random_word(text)
    print(output)

    text = "আমি ১০০ বাকি দিলাম"
    randaug.remove_random_char(text)
    print(output)
    ```

## Inspired from
- [nlpaug](https://github.com/makcedward/nlpaug)
- [amitness blog post](https://amitness.com/2020/05/data-augmentation-for-nlp/)
adcirc-rom 0.0.0
Add a short description here!
License: MIT

# ADCIRC ROM
ADCIRC ROM (Reduced-Order-Modeling) is suite of tools for developing surrogate machine learning models of storm surge.
The tools can be used both in an HPC and a single-threaded environment.

## Designsafe Quickstart

1. Start a Jupyter lab session.

2. Use Jupyter lab to launch a terminal, and in the terminal run the following:
```
pip install netCDF4 sklearn global-land-mask xgboost geopandas scipy h5py fire
git clone https://github.com/UT-CHG/adcirc-rom.git
cd adcirc-rom
python3 dataset.py setup
```

This will create a `data` folder with the subdirectories `datasets`, `storms` and `models`.
These subdirectories are needed for the model development workflow. The `datasets` directory is used for storing
machine-learning ready datasets. The `storms` directory will contain the raw ADCIRC input (note when run within the DesignSafe
environment this directory will be prepopulated with a dataset of 446 synthetic ADCIRC simulations).
Finally, the `models` dataset is used for storing saved ML models and predictions.

3. To generate a dataset, run the command 
```
python3 dataset.py create default
```

This will create a dataset named 'default' in the directory `data/datasets`.
This dataset can be used to train machine learning models.
The `dataset.py` script takes a number of options that control the size and scope of the generated dataset,
as well as the included features.

Note: with the default settings, dataset
creation in the Designsafe environment will take a few hours due to the lack of MPI support and the
size of the data to be processed. The dataset generation script supports parallization with MPI - and is significantly
faster when run on HPC resources such as TACC.
 

4. To train and save a new model named 'xgb_base', using the dataset named default, run the command
```
python3 model.py train xgb_base --dataset=default
```

This will create a new model named 'xgb_base'. During, training, a portion of the dataset is set aside
for testing purposes - predictions are generated for the test dataset and saved alongside the model binary.
Additional model training parameters can be passed to the script.

To perform cross-validation, run the command

```
python3 model.py cv --dataset=default
```

This will also print out feature importances. Note that all model training parameters that are supported by the
`train` are supported by the `cv` command as well. This allows for testing out of parameters before training a model.

Finally, to generate predictions on a new dataset using a saved model, run
```
python3 model.py predict [modelname] --dataset=[datasetname]
```

All predictions can be accessed in the folder `data/datasets/[modelname]`.

Please reach out with any questions or bug reports to Benjamin Pachev <benjamin.pachev@gmail.com>.