semvec/pypi-packages huggingface dataset:
nmt 0.0.6
Neural Machine Translation for NLPIA 2nd Edition
License: mit

# Neural Machine Translation (NMT)

## Description

This is the Neural Machine Translation package for NLPIA 2nd Edition. Currently support Spanish-English Seq2Seq model using 1-layer GRU with Bag-of-Word accuracy

## Installation

If you just want to install nmt package from PyPI channel:

```console
$ pip install nmt==0.0.4
```

If you want to modify the source code to run experiments you'll need to install dependencis in an environment and then install the package in `--editable` mode.


### Environment

Dependencies:

- NLTK
- editdistance

Create a conda environment where you can install all the dependencies like pytorch, pandas, nltk, spacy, and scikit-learn.
Jupyter is also installed so developers can experiment in `jupyter console` (ipython) and Data Scientists can use `jupyter notebook`.

```console
$ conda update -y -n base -c defaults conda
$ conda create -y -n nmt 'python=3.7.9'
$ conda env update -n nmt -f environment.yml
$ conda activate nmt || source activate nmt
```

## Usage

### Train an NMT model

1. Activate conda env with the `nmt` package installed
2. `nmt --config ${model_hyperparameter_json} --epochs ${num_epoch} --data_path ${training_file} --model_checkpoint_dir ${export_path} --metrics_dir ${metrics_path}`

### Parameters

- Model Hyperparameter Json: Name of the config file (under the experiment subdirectory)
- Epoch: Number of Epoch
- Training Text File: Directory of the training corpus (.txt)
- Model Checkpoint Path: Directory to save model checkpoint
- Metric Directory: Directory to save learning curve and model metrics

## Roadmap
- [ ] 0. [Add badge for unittests](https://docs.gitlab.com/ee/user/project/badges.html) to README.md
- [ ] 0. Push release to pypi: `git tag -a 0.0.6 -m 'toy_problem.py works!' && python setup.py sdist bdist_wheel upload`
- [x] 1. Set up a simple decoder-encoder model using GRU cells, BLEU score as evaluation metrics
- [x] 2. Conduct hyperparameter search
- [x] 3. Add Attention Mechanism to Decoder-Encoder module
- [ ] 4. Incorporate transfer learning from BERT or other models


## Directory structure

Code Structure within source directory:
- experiments: submodule where hyperparameters are stored in json format and retrieved as config
- models: submodule where Decoder, Encoder, Seq2Seq models are stored
- utils: submodule where Word Dictionary and Data Preprocessing functions are found
- main_script.py: script to kick start model training
- training.py: script to walk through the whole training process



## Credits/References:

- [Benjamin Etienne's repo](https://github.com/b-etienne/Seq2seq-PyTorch/)
- [PyTorch's documentation on Seq2Seq](https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html)