Package: paragraph2actions
Extraction of actions from experimental proceduresThis repository contains the code forAutomated Extraction of Chemical Synthesis Actions from Experimental Procedures.OverviewSystem RequirementsInstallation GuideTraining the transformer model for action extractionData augmentation exampleAction post-processing exampleOverviewThis repository contains code to extract actions from experimental procedures. In particular, it contains the following:Definition and handling of synthesis actionsCode for data augmentationTraining and usage of a transformer-based modelA trained model can be freely used online athttps://rxn.res.ibm.comor with the Python wrapper availablehere.Links:GitHub repositoryPyPI packageSystem RequirementsHardware requirementsThe code can run on any standard computer.
It is recommended to run the training scripts in a GPU-enabled environment.Software requirementsOS RequirementsThis package is supported formacOSandLinux. The package has been tested on the following systems:macOS: Catalina (10.15.4)Linux: Ubuntu 16.04.3PythonA Python version of 3.6 or greater is recommended.
The Python package dependencies are listed insetup.cfg.Installation guideTo use the package, we recommended to create a dedicatedcondaorvenvenvironment:# Condacondacreate-np2apython=3.8
condaactivatep2a# venvpython3.8-mvenvmyenvsourcemyenv/bin/activateThe package can then be installed from Pypi:pipinstallparagraph2actionsFor local development, the package can be installed with:pipinstall-e.[dev]The installation should not take more than a few minutes.Training the transformer model for action extractionThis section explains how to train the translation model for action extraction.General setupFor simplicity, set the following environment variable:exportDATA_DIR="$(pwd)/test_data"DATA_DIRcan be changed to any other location containing the data to train on.
We assume thatDATA_DIRcontains the following files:src-test.txtsrc-train.txtsrc-valid.txttgt-test.txttgt-train.txttgt-valid.txtSubword tokenizationWe train a SentencePiece tokenizer on the train split:exportVOCAB_SIZE=200# for the production model, a size of 16000 is usedparagraph2actions-create-tokenizer-i$DATA_DIR/src-train.txt-i$DATA_DIR/tgt-train.txt-m$DATA_DIR/sp_model-v$VOCAB_SIZEWe then tokenize the data:paragraph2actions-tokenize-m$DATA_DIR/sp_model.model-i$DATA_DIR/src-train.txt-o$DATA_DIR/tok-src-train.txt
paragraph2actions-tokenize-m$DATA_DIR/sp_model.model-i$DATA_DIR/src-valid.txt-o$DATA_DIR/tok-src-valid.txt
paragraph2actions-tokenize-m$DATA_DIR/sp_model.model-i$DATA_DIR/tgt-train.txt-o$DATA_DIR/tok-tgt-train.txt
paragraph2actions-tokenize-m$DATA_DIR/sp_model.model-i$DATA_DIR/tgt-valid.txt-o$DATA_DIR/tok-tgt-valid.txtTrainingConvert the data to the format required by OpenNMT:onmt_preprocess\-train_src$DATA_DIR/tok-src-train.txt-train_tgt$DATA_DIR/tok-tgt-train.txt\-valid_src$DATA_DIR/tok-src-valid.txt-valid_tgt$DATA_DIR/tok-tgt-valid.txt\-save_data$DATA_DIR/preprocessed-src_seq_length300-tgt_seq_length300\-src_vocab_size$VOCAB_SIZE-tgt_vocab_size$VOCAB_SIZE-share_vocabTo then train the transformer model with OpenNMT:onmt_train\-data$DATA_DIR/preprocessed-save_model$DATA_DIR/models/model\-seed42-save_checkpoint_steps10000-keep_checkpoint5\-train_steps500000-param_init0-param_init_glorot-max_generator_batches32\-batch_size4096-batch_typetokens-normalizationtokens-max_grad_norm0-accum_count4\-optimadam-adam_beta10.9-adam_beta20.998-decay_methodnoam-warmup_steps8000\-learning_rate2-label_smoothing0.0-report_every1000-valid_batch_size32\-layers4-rnn_size256-word_vec_size256-encoder_typetransformer-decoder_typetransformer\-dropout0.1-position_encoding-share_embeddings-valid_steps20000\-global_attentiongeneral-global_attention_functionsoftmax-self_attn_typescaled-dot\-heads8-transformer_ff2048Training the model can take up to a few days in a GPU-enabled environment.
For testing purposes in a CPU-only environment, the same command with-save_checkpoint_steps 10and-train_steps 10will take only a few minutes.FinetuningFor finetuning, we first generate appropriate data in OpenNMT format by following the steps described above.
We assume that the preprocessed data is then available as$DATA_DIR/preprocessed_finetuningWe then use the same training command with slightly different parametersonmt_train\-data$DATA_DIR/preprocessed_finetuning\-train_from$DATA_DIR/models/model_step_500000.pt\-save_model$DATA_DIR/models/model\-seed42-save_checkpoint_steps1000-keep_checkpoint40\-train_steps530000-param_init0-param_init_glorot-max_generator_batches32\-batch_size4096-batch_typetokens-normalizationtokens-max_grad_norm0-accum_count4\-optimadam-adam_beta10.9-adam_beta20.998-decay_methodnoam-warmup_steps8000\-learning_rate2-label_smoothing0.0-report_every200-valid_batch_size512\-layers4-rnn_size256-word_vec_size256-encoder_typetransformer-decoder_typetransformer\-dropout0.1-position_encoding-share_embeddings-valid_steps200\-global_attentiongeneral-global_attention_functionsoftmax-self_attn_typescaled-dot\-heads8-transformer_ff2048Extraction of actions with the transformer modelExperimental procedure sentences can then be translated to action sequences with the following:# Update the path to the OpenNMT model as requiredexportMODEL="$DATA_DIR/models/model_step_520000.pt"paragraph2actions-translate-t$MODEL-p$DATA_DIR/sp_model.model-s$DATA_DIR/src-test.txt-o$DATA_DIR/pred.txtEvaluationTo print the metrics on the predictions, the following command can be used:paragraph2actions-calculate-metrics-g$DATA_DIR/tgt-test.txt-p$DATA_DIR/pred.txtData augmentationThe following code illustrate how to augment the data for existing sentences and associated action sequences.fromparagraph2actions.augmentation.compound_name_augmenterimportCompoundNameAugmenterfromparagraph2actions.augmentation.compound_quantity_augmenterimportCompoundQuantityAugmenterfromparagraph2actions.augmentation.duration_augmenterimportDurationAugmenterfromparagraph2actions.augmentation.temperature_augmenterimportTemperatureAugmenterfromparagraph2actions.miscimportload_samples,TextWithActionsfromparagraph2actions.readable_converterimportReadableConverterconverter=ReadableConverter()samples=load_samples('test_data/src-test.txt','test_data/tgt-test.txt',converter)cna=CompoundNameAugmenter(0.5,['NaH','hydrogen','C2H6','water'])cqa=CompoundQuantityAugmenter(0.5,['5.0 g','8 mL','3 mmol'])da=DurationAugmenter(0.5,['overnight','15 minutes','6 h'])ta=TemperatureAugmenter(0.5,['room temperature','30 °C','-5 °C'])defaugment(sample:TextWithActions)->TextWithActions:sample=cna.augment(sample)sample=cqa.augment(sample)sample=da.augment(sample)sample=ta.augment(sample)returnsampleforsampleinsamples:print('Original:')print(sample.text)print(converter.actions_to_string(sample.actions))for_inrange(5):augmented=augment(sample)print('  Augmented:')print(' ',augmented.text)print(' ',converter.actions_to_string(augmented.actions))print()This script can produce the following output:Original:
The reaction mixture is allowed to warm to room temperature and stirred overnight.
STIR for overnight at room temperature.
  Augmented:
  The reaction mixture is allowed to warm to -5 °C and stirred overnight.
  STIR for overnight at -5 °C.
  Augmented:
  The reaction mixture is allowed to warm to room temperature and stirred 15 minutes.
  STIR for 15 minutes at room temperature.
[...]Action post-processingThe following code illustrate the postprocessing of actions.fromparagraph2actions.postprocessing.filter_postprocessorimportFilterPostprocessorfromparagraph2actions.postprocessing.noaction_postprocessorimportNoActionPostprocessorfromparagraph2actions.postprocessing.postprocessor_combinerimportPostprocessorCombinerfromparagraph2actions.postprocessing.wait_postprocessorimportWaitPostprocessorfromparagraph2actions.readable_converterimportReadableConverterconverter=ReadableConverter()postprocessor=PostprocessorCombiner([FilterPostprocessor(),NoActionPostprocessor(),WaitPostprocessor(),])original_action_string='NOACTION; STIR at 5 °C; WAIT for 10 minutes; FILTER; DRYSOLUTION over sodium sulfate.'original_actions=converter.string_to_actions(original_action_string)postprocessed_actions=postprocessor.postprocess(original_actions)postprocessed_action_string=converter.actions_to_string(postprocessed_actions)print('Original actions     :',original_action_string)print('Postprocessed actions:',postprocessed_action_string)The output of this code will be the following:Original actions     : NOACTION; STIR at 5 °C; WAIT for 10 minutes; FILTER; DRYSOLUTION over sodium sulfate.
Postprocessed actions: STIR for 10 minutes at 5 °C; FILTER keep filtrate; DRYSOLUTION over sodium sulfate.