# Language Models for Predicting Organic Synthesis Procedures
## 7 Discussion
** Lastly, to improve the understanding of the synthesis duration prediction accuracy, we analyzed the average errors in synthesis duration. The synthesis duration encompasses the combined durations of both _Stir_ and _Reflux_ actions within the procedure. All the time that is given by the actions is converted to hours and added together. We calculate the average error using the testing dataset where it is available (about 90-95% of the testing dataset). The _SDE_ is calculated using Formula (7), where \(n\) is the number of samples, \(d_{p}\) is the predicted temperature, and \(d_{g}\) is the ground truth. \[SDE=\frac{1}{n}\sum\lvert d_{p}-d_{g}\rvert\] (7) The average duration of reactions was around 13.3 h, and the range of the duration was from 5 min to 10 days. The _SDE_ for the top models was as follows: _FLAN-T5-base-CC_ (10.04), _moT5-base-PP_ (10.26), and _moT5-large-PP_ (10.04). The average error of about 10 h is quite high, and there certainly is room for improvement. The _BART-base-CC_ had the top score by a small margin (9.98, best overall) and an average error of less than 3 h in 38% of all suitable instances and less than 1 h in 23.5% of cases. Generally, the length of the synthesis is variable because it depends on the nature of the chemicals, the selection of solvents, the reaction temperature, and the scale of the synthesis. In addition, the end of the synthesis operations is usually monitored using analytical equipment; therefore, if incorrect predictions are made, the operation's duration can be adjusted accordingly. In fact, a regular or automated reaction analysis may render the duration prediction irrelevant. Additionally, we should acknowledge that if the predicted temperature is higher and appropriate for the reaction, the reaction rate will naturally increase, necessitating an adjustment to the duration. However, our analysis of the four additional metrics reveals that future research should focus on addressing the synthesis duration, given the higher level of errors compared to other aspects of the procedure. The analysis of common and uncommon metrics can be difficult because of the specificity of reaction procedures. To illustrate some of the challenges, two examples in Table 5 are given from the gold standard subset of the _BART-base-CC_ predictions with the matching ground truth. In the first example, we see a shorter ground truth procedure than the prediction. The prediction itself is similar in the beginning and correctly adds HATU, DIPEA, and DMF into one mixture. It incorrectly predicts the duration but also includes a workup sequence (extraction, washing, drying, and filtration). When comparing the two with most metrics, we should see lower scores; however, in practical terms, the core of the ground truth procedure has been predicted correctly. If the workup is not needed, it simply can be skipped. If it is needed, the predicted version suggests a possible path. In example two, the inverse is true, and the ground truth includes a longer workup. The core of the reaction is quite accurate; only the _Purify_ action is perfromed after the synthesis is complete. It is important to address this aspect of chemical reaction procedure prediction in future research to have greater control over prediction generation. While the length of the generation can be affected by the length penalty constants, it is not necessarily the solution, since the length is not what matters most. For example, the action _Purify_ calls for chromatographic purification, which can be used in place of an entire sequence of workup actions. Also, certain laboratories might not have the required equipment for the predictedsequence of actions; so, a generated alternative ending of the procedure could also be very beneficial for scientists, which currently is not possible. In addition, some actions do not require parameters, like _Purify_, but it would be useful to generate them.