Package: decimer
DECIMER Image Transformer: Deep Learning for Chemical Image Recognition using Efficient-Net V2 + TransformerAbstractThe DECIMER 2.2 [5] (Deep lEarning for Chemical ImagE Recognition) project [1] was launched to address the OCSR problem with the latest computational intelligence methods to provide an automated open-source software solution.The original implementation of DECIMER[1] using GPU takes a longer training time when we use a bigger dataset of more than 1 million images. To overcome these longer training times, many implement the training script to work on multiple GPUs. However, we tried to step up and implemented our code to use Google's Machine Learning hardwareTPU(Tensor Processing Unit)[2]. You can learn more about the hardwarehere.Method and model changesThe DECIMER now uses EfficientNet-V2[3] for Image feature extraction and a transformer model [4] for predicting the SMILES.The SMILES used during training and predictionsChanges in the training methodWe converted our datasets intoTFRecordFiles, A binary file system the TPUs can read in a much faster way. Also, we can use these files to train on GPUs. Using the TFRecord helps us train the model fast by overcoming the bottleneck of reading multiple files from the hard disks.We moved our data toGoogle Cloud Buckets. An efficient storage solution provided by the google cloud environment where we can access these files from any google cloud VMs easily and in a much faster way. (To get the highest speed, the cloud storage and the VM should be in the same region)We adopted the TensorFlow data pipeline to load all TFRecord files to the TPUs from Google Cloud Buckets.We modified the main training code to work on TPUs usingTPU strategyintroduced in Tensorflow 2.0.How to use DECIMER?Python packageDocumentationModel library could be found here:We suggest using DECIMER inside a Conda environment, which makes the dependencies install easily.Conda can be downloaded as part of theAnacondaor theMinicondaplatforms (Python 3.7). We recommend installing miniconda3. Using Linux, you can get it with:$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.shInstructionsPython Package InstallationUse a conda environment for clean installation$sudoaptupdate
$sudoaptinstallunzip
$condacreate--nameDECIMERpython=3.10.0
$condaactivateDECIMER
$condainstallpip
$python3-mpipinstall-UpipInstall the latest code from GitHub with:$pipinstallgit+https://github.com/Kohulan/DECIMER-Image_Transformer.gitInstall in development mode with:$gitclonehttps://github.com/Kohulan/DECIMER-Image_Transformer.gitdecimer
$cddecimer/
$pipinstall-e.Where-emeans "editable" mode.Install from PyPi$pipinstalldecimerHow to use inside your own python scriptfromDECIMERimportpredict_SMILES# Chemical depiction to SMILES translationimage_path="path/to/imagefile"SMILES=predict_SMILES(image_path)print(SMILES)Install tensorflow == 2.10.1 if you do not have an Nvidia GPU (On Mac OS)License:This project is licensed under the MIT License - see theLICENSEfile for detailsCitationRajan K, Brinkhaus HO, Agea MI, Zielesny A, Steinbeck C (2023) DECIMER.ai - An open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications. ChemRxiv. doi:https://10.26434/chemrxiv-2023-xhcx9Rajan, K., Zielesny, A. & Steinbeck, C. DECIMER 1.0: deep learning for chemical image recognition using transformers. J Cheminform 13, 61 (2021).https://doi.org/10.1186/s13321-021-00538-8ReferencesRajan, K., Zielesny, A. & Steinbeck, C. DECIMER: towards deep learning for chemical image recognition. J Cheminform 12, 65 (2020).https://doi.org/10.1186/s13321-020-00469-wNorrie T, Patil N, Yoon DH, Kurian G, Li S, Laudon J, Young C, Jouppi N, Patterson D (2021) The Design Process for Google's Training Chips: TPUv2 and TPUv3. IEEE Micro 41:56–63Tan M, Le QV (2021) EfficientNetV2: Smaller Models and Faster Training. arXiv [cs.CV]Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention Is All You Need. arXiv [cs.CL]Rajan, K., Zielesny, A. & Steinbeck, C. DECIMER 1.0: deep learning for chemical image recognition using transformers. J Cheminform 13, 61 (2021).https://doi.org/10.1186/s13321-021-00538-8AcknowledgementWe thankCharles Tapley Hoytfor his valuable advice and help in improving the DECIMER repository.Research supported with Cloud TPUs from Google's TPU Research Cloud (TRC)Author:KohulanProject Website:A web application implementation is available atdecimer.ai, implemented byOtto BrinkhausResearch Group