Reinforced Molecule Generation with Heterogeneous States De novo molecular design and generation are frequently prescribed in the field of chemistry and biology, for it plays a critical role in maintaining the prosperity of the chemical industry and benefiting the drug discovery. In recent years, reinforcement learningbased methods leverage graphs to represent molecules and generate molecules as a decision making process. However, this vanilla graph representation may neglect the intrinsic context information with molecules and limits the generation performance accordingly. In this paper, we propose to augment the original graph states with the SMILES context vectors. As a result, SMILES representations are easily processed by a simple language model such that the general semantic features of a molecule can be extracted; and the graph representations perform better in handling the topology relationship of each atom. Moreover, we propose a framework that combines supervised learning and reinforcement learning algorithm to take a solid consideration of these two heterogeneous state representations of a molecule, which can fuse the information from both of them and extract more comprehensive features so that more sophisticated decisions can be made by the policy network. Our model also introduces two attention mechanisms, i.e., actionattention, and graphattention, to further improve the performance. We conduct our experiments on a practical dataset, ZINC, and the experiment results demonstrate that our framework can outperform other baselines in the learning performance of molecule generation and chemical property optimization.

Molecular Graph Generation with Deep Reinforced Multitask Network and Adversarial Imitation Learning Molecular graph generation aims to design molecules with desired biochemical properties, which is promising in drug discovery. Existing methods typically combine deep reinforcement models with adversarial training. However, the reinforced rewards in molecule generation are delayed and sparse while adversarial training suffers mode collapse issue. Moreover, they optimize multiple properties with a linear combination manner, where the models get distracted by potentially conflicting objectives. To tackle the above challenges, we propose a Deep Reinforced framework with Adversarial Imitation and Multitask learning (DRAIM). First, the reinforced agent generates discrete molecular graphs via deep Qlearning, where the trajectories of high rewards are cached as experts. Then policies are extracted directly from expert trajectories by adversarial imitation learning, in which the discriminator delivers behavior distribution signals to the agent as dense rewards. Second, we propose to realize multigoal molecule generation as a multitask learning process, where different property optimizations are treated as different tasks to be trained jointly. Extensive experiments demonstrate the effectiveness of DRAIM in molecule generation.