user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Using Deep Reinforcement Learning to Generate Rationales for Molecules

Paper content:
Deep learning algorithms are increasingly used in modeling chemical processes.
However, black box predictions without rationales have limited used in practical applications, such as drug design.
To this end, we learn to identify molecular substructures -- rationales -- that are associated with the target chemical property (e.g., toxicity).
The rationales are learned in an unsupervised fashion, requiring no additional information beyond the end-to-end task.
We formulate this problem as a reinforcement learning problem over the molecular graph, parametrized by two convolution networks corresponding to the rationale selection and prediction based on it, where the latter induces the reward function.
We evaluate the approach on two benchmark toxicity datasets.
We demonstrate that our model sustains high performance under the additional constraint that predictions strictly follow the rationales.
Additionally, we validate the extracted rationales through comparison against those described in chemical literature and through synthetic experiments.
Recently, deep learning has been successfully applied to the development of predictive models relating chemical structures to physical or biological properties, outperforming existing methods BID8 BID14 .
However, these gains in accuracy have come at the cost of interpretability.
Often, complex neural models operate as black boxes, offering little transparency concerning their inner workings.Interpretability plays a critical role in many areas including cheminformatics.
Consider, for example, the problem of toxicity prediction.
Over 90% of small molecule drug candidates entering Phase I trials fail due to lack of efficacy or due to adverse side effects.
In order to propose a modified compound with improved properties, medicinal chemists must know which regions of the molecule are responsible for toxicity, not only the overall level of toxicity BID1 .
We call the key molecular substructures relating to the outcome rationales.
In traditional cheminformatics approaches such as pharmacophore mapping, obtaining such a rationale behind the prediction is an intrinsic part of the model BID24 BID7 BID12 In this paper, we propose a novel approach to incorporate rationale identification as an integral part of the overall property prediction problem.
We assume access to the same training data as in the original prediction task, without requiring annotated rationales.
At the first glance, the problem seems solvable using existing tools.
For instance, attention-based models offer the means to highlight the importance of individual atoms for the target prediction.
However, it is challenging to control how soft selections are exploited by later processing steps towards the prediction.
In this sense, the soft weighting can be misleading.
In contrast, hard selection confers the guarantee that the excluded atoms are not relied upon for prediction.
The hard selection of substructures in a molecule is, however, a hard combinatorial problem.
Prior approaches circumvent this challenge by considering a limited set of predefined substructures (typically of 1-6 atoms), like the ones encoded in some molecular fingerprints BID7 .
Ideally, we would like the model to derive these structures adaptively based on their utility for the target prediction task.We formulate the problem of selecting important regions of the molecule as a reinforcement learning problem.
The model is parametrized by a convolutional network over a molecular graph in which the atoms and bonds are the nodes and edges of the graph, respectively.
Different from traditional reinforcement learning methods that have a reward function provided by the environment, our model seeks to learn such a reward function alongside the reinforcement learning algorithm.
More generally, our model works as a search mechanism for combinatorial sets, which readily expands to applications beyond chemistry or graphs.Our iterative construction of rationales provides several advantages over standard architectures.
First, sequential selection enables us to incorporate contextual features associated with past selections, as well as global properties of the whole molecule.
Second, we can explicitly enforce desirable rationale properties (e.g., number of substructures) by including appropriate regularization terms in the reward function.
We test our model on two toxicity datasets: the Tox21 challenge dataset, which is a series of 12 toxicity tests, and the human ether-a-go-go-related gene (hERG) channel blocking.
The reinforcement learning model identifies the structural components of the molecule that are relevant to these toxicity prediction tasks while simultaneously highlighting opportunities for molecular modification at these sites.
We show that by only selecting about 40-50% of the atoms in the molecules, we can create models that nearly match the performance of models that use the entire molecule.
By comparing selected regions with rationales described in chemical literature, we further validate the rationales extracted by the model.
We present a model that treats the problem of selecting rationales from molecules as a reinforcement learning problem.
By creating an auxiliary prediction network, we use a learned reward structure to facilitate the selection of atoms in the molecule that are relevant to the prediction task, without significant loss in predictive performance.
In this work, we explore the applicability of rationales in the chemistry domain.
Through various experiments on the Tox21 and hERG datasets, we demonstrate that our model successfully learns to select important substructures in an unsupervised manner, requiring the same data as an end-to-end prediction task, which is relevant to many applications including drug design and discovery.
Molecules are far more complicated to reason about as compared to images or text due to complex chemical theories and a lack of definitive ground truth rationale labels.
As deep learning algorithms continue to permeate the chemistry domain, it will be ever more important to consider the interpretability of such models.

assistant: text: We use a reinforcement learning over molecular graphs to generate rationales for interpretable molecular property prediction.