Package: trecover
Welcome to Text Recovery Project 👋A python library for training a Transformer neural network to solve theRunning Key Cipher, widely known in the field of cryptography.🚀 ObjectiveThe main goal of the project is to study the possibility of using Transformer neural network to “read” meaningful text in columns that can be compiled for aRunning Key Cipher. You can read more about the problemhere.In addition, the second rather fun 😅 goal is to train a large enough model so that it can handle the case described below.
Let there be an original sentence:Hello, my name isZendayaMaree Stoermer Coleman but you can just call meZendaya.The columns for this sentence will be compiled in such a way that the last seven contain from ten to thirteen letters of the English alphabet, and all the others from two to five. Thus, the last seven characters will be much harder to "read" compared to the rest. However, we can guess from the meaning of the sentence that this is the nameZendaya.
In other words, the goal is also to train a model that can understand and correctly “read” the last word.⚙ InstallationTrecover requires Python 3.8 or higher and supports both Windows and Linux platforms.Clone the repository:gitclonehttps://github.com/alex-snd/TRecover.git&&cdtrecoverCreate a virtual environment:Windows:python-mvenvvenvLinux:python3-mvenvvenvActivate the virtual environment:Windows:venv\Scripts\activate.batLinux:sourcevenv/bin/activateInstall the package inside this virtual environment:Just to run the demo:pipinstall-e".[demo]"To train the Transformer:pipinstall-e".[train]"For development and training:pipinstall-e".[dev]"Initialize project's environment:trecoverinitFor more options use:trecoverinit--help👀 Demo🤗 Hugging FaceYou can play with a pre-trained model hostedhere.🐳 Docker ComposePull from Docker Hub:docker-compose-fdocker/compose/scalable-service.ymlupBuild from source:docker-compose-fdocker/compose/scalable-service-build.ymlup💻 Local (requires docker)Download pretrained model:trecoverdownloadartifactsLaunch the service:trecoverup🗃️ DataTheWikiTextandWikiQAdatasets
were used to train the model, from which all characters except English letters were removed.You can download the cleaned dataset:trecoverdownloaddata💪 TrainTo quickly start training the model, open theJupyter Notebook.🕸️ DistributedTODO💻 LocalAfter the dataset is loaded, you can start training the model:trecover train local \
--project-name {project_name} \
--exp-mark {exp_mark} \
--train-dataset-size {train_dataset_size} \
--val-dataset-size {val_dataset_size} \
--vis-dataset-size {vis_dataset_size} \
--test-dataset-size {test_dataset_size} \
--batch-size {batch_size} \
--n-workers {n_workers} \
--min-noise {min_noise} \
--max-noise {max_noise} \
--lr {lr} \
--n-epochs {n_epochs} \
--epoch-seek {epoch_seek} \
--accumulation-step {accumulation_step} \
--penalty-coefficient {penalty_coefficient} \

--pe-max-len {pe_max_len} \
--n-layers {n_layers} \
--d-model {d_model} \
--n-heads {n_heads} \
--d-ff {d_ff} \
--dropout {dropout}For more information usetrecover train local --help✔️ Related workTODO: what was done, tech stack.🤝 ContributingContributions, issues and feature requests are welcome.Feel free to checkissues pageif you want to contribute.👏 Show your supportPlease don't hesitate to ⭐️ this repository if you find it cool!📜 LicenseCopyright © 2022Alexander Shulga.This project isApache 2.0licensed.