SlowGuess/NeurIPS_2023_OCR huggingface dataset:
FoldX [67] leverages an empirical force field to estimate the change in stability. It models the Van Der Waals interactions, solvation and hydrogen bonding interactions, changes in entropy, and other physical values.
PROSTATA [74] is a sequence model for stability prediction built on the ESM2 model. It takes native and mutant protein sequences as input and produces a  $\Delta \Delta G$  value.
Stability Oracle [18] is a structure-based graph-transformer model for stability prediction.
Trancection [49] is a specialized autoregressive transformer that uses grouped attention to encourage specialization across heads.
B.1 Higher-order Mutation Results on PTMul Breakdown
ProTherm Multiple (PTMul) contains 846 mutations, of which 536 are double mutations and 310 mutations with more than two mutations. We breakdown the performance on PTMul into double mutations and higher-order mutations (greater than 2) in Table 6. While our model performs similarly against the additive baseline on double mutations, Mutate Everything outperforms the additive baseline on higher-order mutations.
Method	Double Mutations	More Mutations
rs	AUC	MCC	RMSE↓	rs	AUC	MCC	RMSE↓
DDGun	0.28	0.63	0.22	2.23	0.15	0.57	0.12	2.19
DDGun3D	0.29	0.61	0.17	2.25	0.19	0.60	0.17	2.20
Additive Baseline	0.52	0.76	0.30	1.95	0.49	0.79	0.45	2.14
Mutate Everything (ours)	0.50	0.75	0.34	2.08	0.60	0.86	0.58	1.99
Table 6: Results on PTMul splits containing double mutations and more than 2 mutations. Mutate Everything outperforms existing methods, especially when there are multiple mutations.
Method	Forward	Reverse
rs	AUC	MCC	RMSE↓	rs	AUC	MCC	RMSE↓
PROSTATA-IFML [18, 74]	0.53	0.75	0.31	1.75	0.53	0.75	0.32	1.75
Stability Oracle [18]	0.62	0.81	0.39	1.65	0.62	0.81	0.39	1.65
Mutate Everything (Ours)	0.65	0.79	0.39	1.59	0.52	0.72	0.28	1.75
Table 7: Comparisons on Single Mutation Thermodynamic Stability prediction in T2837. We report regression and classification metrics on the forward and reverse datasets.  $r_s$  refers to Spearman correlation coefficient. Our training proteins have at most  $34\%$  sequence similarity with those in T2837. Mutate Everything is state-of-the-art on T2837 forward.
Test Set	#Mutations	\( {\mathrm{r}}_{p} \)	\( {\mathrm{r}}_{s} \)	RMSE \( \downarrow \)	MCC	AUROC	Precision	Recall	Accuracy
T2837 Orig	2835	0.6274	0.6472	1.5877	0.3888	0.7956	0.5615	0.4791	0.7894
T2837 TR	2835	0.5301	0.5208	1.7563	0.2802	0.7229	0.8297	0.8572	0.7517
T2837 Orig + TR	5670	0.7229	0.7282	1.6741	0.5482	0.8433	0.7765	0.7701	0.7705
Table 8: Mutate Everything regression and classification metrics on T2837 and its reverse datasets. TR refers to Thermodynamic Reversibility.  $\mathbf{r}_p$  is Pearson correlation coefficient.  $\mathbf{r}_s$  is Spearman correlation coefficient. Mutations that failed our data pipeline are excluded.
B.2 Single Mutation Results on T2837
We include our results on the newly proposed T2837 dataset in Table 7 [18]. Mutate Everything achieves a Spearman correlation of 0.65 compared to 0.62 of Stability Oracle, outperforming existing methods on the forward dataset. PROSTATA-IFML [18] fine-tunes PROSTATA [74] on the cDNA proteolysis dataset for more direct comparison. Mutate Everything performs competitively with other sequence models on the reverse dataset even without training on reverse mutations. We show our performance on additional metrics for T2837 in Table 8.