PROTEIN SEQUENCE DESIGN WITH A LEARNED POTENTIAL

The task of protein sequence design is central to nearly all rational protein engineering problems, and enormous effort has gone into the development of energy functions to guide design. We investigate the capability of a deep neural network model to automate design of sequences onto protein backbones, having learned directly from crystal structure data and without any human-specified priors. The model generalizes to native topologies not seen during training, producing experimentally stable designs. We evaluate the generalizability of our method to a de novo TIM-barrel scaffold. The model produces novel sequences, and high-resolution crystal structures of two designs show excellent agreement with the in silico models. Our findings demonstrate the tractability of an entirely learned method for protein sequence design. PROTEIN SEQUENCE DESIGN WITH A LEARNED POTENTIAL Computational protein design has emerged as a powerful tool for rational protein design, enabling significant achievements in the engineering of therapeutics [1, 2, 3], biosensors [4, 5, 6], enzymes [7, 8], and more [9, 10, 11]. Key to such successes are robust sequence design methods that minimize the folded-state energy of a pre-specified backbone conformation, which can either be derived from existing structures or generated de novo. This difficult task [12] is often described as the “inverse” of protein folding—given a protein backbone, design a sequence that folds into that conformation. Current approaches for fixed-backbone design commonly involve specifying an energy function and sampling sequence space to find a minimum-energy configuration [13, 14, 15], and enormous effort has gone into the development of carefully modeled and parameterized energy functions to guide design, which continue to be iteratively refined [16, 17]. With the emergence of deep learning systems and their ability to learn patterns from high-dimensional data, it is now possible to build models that learn complex functions of protein sequence and structure, including models for protein backbone generation [18, 19, 20] and protein structure prediction [21, 22]; as a result, we were curious as to whether an entirely learned method could be used to design protein sequences on par with energy function methods. Recent experimentally validated efforts for machine learning-based sequence generation have focused on sequence representation learning, requiring fitting to data from experiments or from known protein families to produce functional designs [23]. We hypothesized that by training a model that conditions on local backbone structure and chemical environment, the network might learn residue-level patterns that allow it to generalize without fine-tuning to new backbones with topologies outside of the training distribution, opening up the possibility for generation of de novo designed sequences with novel structures and functions. Moreover, for rational protein design we are often interested in designing structures, interfaces, or residue interactions with angstrom-level specificity. As such, we explored a method in which the neural network explicitly builds side-chain conformers and evaluates full atom structural models, an approach not reported to date. Conventional energy functions used in sequence design calculations are often composed of pairwise terms that model inter-atomic interactions. Given the expressivity of deep neural networks, or their ability to approximate a rich class of functions, we predicted that a model conditioned on chemical context could learn higher order (multi-body) interactions relevant for sequence design (e.g., hydrogen bonding networks). Furthermore, most energy functions are highly sensitive to specific atom placement, and as a result designed sequences can be convergent for a given starting backbone conformation. For most native proteins, however, the existence of many structural homologs with low sequence identity suggests that there is a distribution of viable sequences that can adopt a target fold, but the discovery of these sequences given a fixed-backbone reference structure is difficult. We hypothesized that a learned model could operate as a “soft” potential that implicitly captures backbone flexibility, producing diverse sequences for a fixed protein backbone. In this study, we explore an approach for sequence design guided only by a neural network that explicitly models side-chain conformers in a structure-based context (Fig. 1A), and we assess its generalization to unseen native topologies and to a de novo protein backbone. We are interested in sampling from the true distribution of n-length sequences of amino acids Y ∈ {1 . 20} conditioned on a fixed protein backbone. The backbone is fully specified by the positions of each residue’s four N − Cα − C −O atoms and the C-terminal oxygen atom, whose positions are encoded as X ∈ R(4n+1)×3; thus, the final conditional distribution we are interested in modeling is: P (Y |X) = p(yi=1, . . . , yn|X) (1) Due to the local nature of the physical interactions within proteins, we can expect that the likelihood of a given side-chain identity and conformation will be dictated by neighboring residues. Defining envi as the combination of the backbone atoms X and neighboring residues yNB(i), the conditional side-chain distribution at a given position i can be factorized sequentially as follows: p(yi|envi) =p(ri|envi) 4 ∏ j=1 p(χji|χ1:j−1i, ri, envi) (2) where ri ∈ {1 . 20} is the amino acid type of residue i and χ1i, χ2i, χ3i, χ4i ∈ [−180◦, 180◦] are the torsion angles for the side-chain. We train a deep neural network conditional model θ to learn these conditional distributions from data. Conditioning on the local environment, the network autoregressively predicts distributions over residue types pθ(ri|envi) and rotamer angles pθ(χ1j | ˆ χ1:j−1i, r̂i, envi) (Fig. 1B). Our design algorithm involves iteratively sampling side-chains conditioned on their local chemical environments from these network-predicted distributions. We can approximate the joint probability of a sequence P (Y |X) by the pseudo-log-likelihood (PLL) [24] of the sequence under the model PLL(Y |X) = ∑ i log pθ(yi|envi) (3)