SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zero-shot Table A5 shows the results for our zero-shot DMS substitutions benchmark. We report Spearman's rank correlations and bootstrapped standard error estimates for forty baseline models. Table A6 breaks down our substitution DMS by MSA depth, , Table A7 by function type, Table A8 by taxa, and Table A9 by mutational depth. To compute the final Spearman's rank correlation reported in Table A5, we first average all the assays for a particular function type together, resulting in five average values (one each for Activity, Binding, Expression, Organismal Fitness, and Stability). The average of these five numbers is the final reported value.
**Clustering zero-shot substitution models** We clustered the zero-shot models using hierarchical clustering on the vector of NDCG metrics for each dataset in the DMS substitutions (Fig. A1). We find that models with the same architecture tend to cluster together (e.g., RITA models), however, there are exceptions (e.g., ESM-2 models). We also observe that the alignment-based models tend to cluster together, suggesting that training on the same MSA may promote similar scoring behavior.
Supervised Table A10 shows the results for our supervised DMS substitutions benchmark. We report Spearman's rank correlations for 10 baseline models.
Figure A1: Hierarchical clustering of zero-shot models by NDCG performance Heatmap colored by the Pearson correlation of the NDCG@  $10\%$  values for each DMS assay for each pair of zero-shot models. Lighter color corresponds to higher correlation. The ordering and dendogram were produced by hierarchical clustering of the correlation values.
A.5.2 DMS indel benchmarks
Zero-shot Table A14 shows the results for our zero-shot DMS indels benchmark, and Table A15 shows Spearman's rank correlations for each indel DMS dataset and model. Figure A2 compares each model's aggregate performance between the Library and Designed DMS sets (numbers provided in Table 4). More detailed performance files are available in the repository.
Figure A2: Performance comparison of indel baselines on different types of assays Spearman's rank correlation over unbiased libraries vs model-designed sequences biased towards natural sequences.
Supervised Table A16 ranks the performance of each model on the supervised indel DMS benchmark.
A.5.3 Clinical substitution benchmarks
As discussed in § 4, since the performance of zero-shot models is on par – or higher – than their supervised counterparts we subsume the Clinical zero-shot and supervised rankings into a combined rankings, available in Table. A17. Although supervised models trained on ClinVar labels (such as ClinPred) perform well on the clinical benchmark, unsupervised models (such as TranceptEVE) provide better performance on the subset of DMS assays assessing the clinical effect of variants in humans, and competitive performance on the clinical benchmark without being subject to the same label biases (see Fig. 2).
A.5.4 Clinical indel benchmarks
Table A18 shows model performance on the ClinVar datasets, and Figure A3 shows the combined performance on the DMS and ClinVar indel benchmarks.
Figure A3: Performance comparison of indel baselines on the indel benchmarks. AUC over ClinVar with gnomAD controls (x axis) and Spearman's rank correlation over functional assay benchmarks (y axis).
Ranking	Model	Type	Spearman	Std. error
1*	TrancaptEVE L	Hybrid model	0.456	0.000
1*	TrancaptEVE M	Hybrid model	0.455	0.004
1*	GEMME	Alignment-based model	0.455	0.007
4	TrancaptEVE S	Hybrid model	0.452	0.004
5	EVE (ensemble)	Alignment-based model	0.439	0.006
6	VESPA	Protein language model	0.436	0.006
7*	Trancception L	Hybrid model	0.434	0.004
7*	MSA Transformer (ensemble)	Hybrid model	0.434	0.009
9	EVE (single)	Alignment-based model	0.433	0.005
10	Trancception M	Hybrid model	0.427	0.005
11	ESM-IF1	Inverse folding model	0.422	0.011
12	MSA Transformer (single)	Hybrid model	0.421	0.009
13	DeepSequence (ensemble)	Alignment-based model	0.419	0.008
14	Trancception S	Hybrid model	0.418	0.006
15	ESM2 (650M)	Protein language model	0.414	0.012
16*	DeepSequence (single)	Alignment-based model	0.407	0.008
16*	ESM-1v (ensemble)	Protein language model	0.407	0.012
18	ESM2 (3B)	Protein language model	0.406	0.011
19*	MIF-ST	Inverse folding model	0.401	0.010
19*	ESM2 (15B)	Protein language model	0.401	0.010
21	EVmutation	Alignment-based model	0.395	0.006
22*	ESM-1b	Protein language model	0.394	0.010
22*	VESPAI	Protein language model	0.394	0.007
24	ProGen2 XL	Protein language model	0.391	0.008
25	ESM2 (150M)	Protein language model	0.387	0.013
26	MIF	Inverse folding model	0.382	0.011
27	ProGen2 L	Protein language model	0.380	0.008
28	ProGen2 M	Protein language model	0.379	0.008
29	ProGen2 Base	Protein language model	0.378	0.009
30*	Trancception L no retrieval	Protein language model	0.374	0.008
30*	ESM-1v (single)	Protein language model	0.374	0.013
32	WaveNet	Alignment-based model	0.373	0.012
33	RITA XL	Protein language model	0.372	0.009
34	CARP (640M)	Protein language model	0.368	0.011
35	RITA L	Protein language model	0.365	0.009
36	Site-Independent	Alignment-based model	0.359	0.010
37	RITA M	Protein language model	0.350	0.010
38	Trancception M no retrieval	Protein language model	0.348	0.009
39	Unirep evotuned	Hybrid model	0.347	0.009
40	ProGen2 S	Protein language model	0.336	0.011
41	CARP (76M)	Protein language model	0.328	0.012
42	ESM2 (35M)	Protein language model	0.321	0.015
43	RITA S	Protein language model	0.304	0.011
44	Trancception S no retrieval	Protein language model	0.303	0.012
45	CARP (38M)	Protein language model	0.279	0.014
46	ProteinMPNN	Inverse folding model	0.258	0.011
47	ESM2 (8M)	Protein language model	0.226	0.015
48	UniRep	Protein language model	0.190	0.016
49	ProtGPT2	Protein language model	0.188	0.011
50	CARP (600K)	Protein language model	0.106	0.016
Table A5: ProteinGym - Zero-shot substitution DMS benchmark Ranking based on corrected average of Spearman's rank correlation between experimental assay measurement and model prediction. The standard error reported corresponds to the non-parametric bootstrap standard error of the difference between the Spearman performance of a given model and that of the best overall model (i.e., TrancaptEVE), computed over 10k bootstrap samples from the set of proteins in the ProteinGym DMS substitution benchmark.
Model type	Model name	Spearman by MSA depth (↑)
Low	Medium	High	All
Alignment-based	Site-Independent	0.426	0.373	0.320	0.373
WaveNet	0.299	0.389	0.452	0.380
EVmutation	0.403	0.423	0.410	0.412
DeepSequence (ens.)	0.383	0.428	0.473	0.428
EVE (ens.)	0.425	0.453	0.481	0.453
GEMME	0.455	0.470	0.497	0.474
Protein language	UniRep	0.181	0.161	0.209	0.184
CARP (640M)	0.314	0.375	0.428	0.372
ESM-1b	0.350	0.398	0.482	0.410
ESM-2 (15B)	0.357	0.414	0.473	0.415
RITA XL	0.315	0.382	0.412	0.370
ESM-1v (ens.)	0.326	0.418	0.502	0.415
ProGen2 XL	0.354	0.405	0.444	0.401
VESPA	0.427	0.455	0.484	0.455
Hybrid	UniRep evotuned	0.330	0.344	0.372	0.349
MSA Transformer (ens.)	0.404	0.450	0.488	0.447
Tranception L	0.432	0.438	0.473	0.448
TranceptEVE L	0.451	0.467	0.492	0.470
Inverse Folding	ESM-IF1	0.300	0.431	0.544	0.425
MIF-ST	0.376	0.403	0.456	0.412
ProteinMPNN	0.173	0.280	0.434	0.296
Table A6: ProteinGym - Zero-shot substitution DMS benchmark by MSA depth Average Spearman's rank correlation between model scores and experimental measurements by MSA depth on the ProteinGym substitution benchmark. Alignment depth is measured by the ratio of the effective number of sequences  $N_{\mathrm{eff}}$  in the MSA, following Hopf et al. [2017], by the length covered  $L$  (Low:  $N_{\mathrm{eff}} / L < 1$ ; Medium:  $1 < N_{\mathrm{eff}} / L < 100$ ; High:  $N_{\mathrm{eff}} / L > 100$ ). The All column is the average across the 3 depths.
Model type	Model name	Spearman by Function Type (↑)
Activity	Binding	Expression	Organismal Fitness	Stability	All
Alignment-based	Site-Independent	0.369	0.345	0.351	0.382	0.358	0.361
WaveNet	0.379	0.325	0.350	0.365	0.449	0.374
EVmutation	0.440	0.322	0.382	0.411	0.430	0.397
DeepSequence (ens.)	0.455	0.368	0.396	0.413	0.476	0.422
EVE (ens.)	0.464	0.394	0.406	0.447	0.491	0.440
GEMME	0.482	0.387	0.443	0.452	0.519	0.457
Protein language	UniRep	0.182	0.203	0.230	0.141	0.210	0.193
CARP (640M)	0.395	0.274	0.419	0.364	0.414	0.373
ESM-1b	0.428	0.289	0.427	0.351	0.500	0.399
ESM-2 (15B)	0.405	0.318	0.425	0.388	0.488	0.405
RITA XL	0.366	0.303	0.416	0.381	0.398	0.373
ESM-1v (ens.)	0.414	0.320	0.456	0.387	0.500	0.415
ProGen2 XL	0.402	0.302	0.423	0.387	0.445	0.392
VESPA	0.468	0.365	0.410	0.440	0.500	0.437
Hybrid	UniRep evotuned	0.355	0.304	0.366	0.346	0.366	0.347
MSA Transformer (ens.)	0.469	0.343	0.439	0.421	0.495	0.433
Trancection L	0.465	0.351	0.455	0.436	0.471	0.436
TrancptEVE L	0.487	0.381	0.456	0.460	0.500	0.457
Inverse Folding	ESM-IF1	0.368	0.392	0.403	0.324	0.624	0.422
MIF-ST	0.390	0.323	0.432	0.373	0.486	0.401
ProteinMPNN	0.197	0.165	0.198	0.165	0.566	0.258
Table A7: ProteinGym - Zero-shot substitution DMS benchmark by function type Corrected average of Spearman's rank correlation between model scores and experimental measurements on the ProteinGym substitution benchmark, separated into five functional categories (Activity, Binding, Organismal Fitness, Stability and Expression). 'All' is the average of all the categories.
Model type	Model name	Spearman by Taxa (↑)
Human	Other Eukaryote	Prokaryote	Virus	All
Alignment-based	Site-Independent	0.379	0.385	0.316	0.383	0.366
WaveNet	0.391	0.410	0.427	0.328	0.389
EVmutation	0.409	0.444	0.422	0.388	0.416
DeepSequence (ens.)	0.442	0.469	0.460	0.344	0.429
EVE (ens.)	0.453	0.487	0.468	0.428	0.459
GEMME	0.468	0.510	0.473	0.469	0.480
Protein language	UniRep	0.213	0.219	0.165	0.057	0.164
CARP (640M)	0.416	0.386	0.390	0.273	0.366
ESM-1b	0.434	0.475	0.455	0.241	0.401
ESM-2 (15B)	0.431	0.449	0.459	0.313	0.413
RITA XL	0.394	0.384	0.353	0.402	0.383
ESM-1v (ens.)	0.458	0.446	0.454	0.289	0.412
ProGen2 XL	0.384	0.442	0.439	0.391	0.414
VESPA	0.438	0.492	0.490	0.432	0.463
Hybrid	UniRep evotuned	0.355	0.363	0.346	0.349	0.353
MSA Transformer (ens.)	0.437	0.505	0.463	0.414	0.455
Tranception L	0.453	0.483	0.431	0.432	0.450
TranceptEVE L	0.471	0.498	0.473	0.453	0.474
Inverse Folding	ESM-IF1	0.415	0.497	0.507	0.374	0.448
MIF-ST	0.404	0.415	0.463	0.396	0.420
ProteinMPNN	0.282	0.395	0.354	0.248	0.320
Table A8: ProteinGym - Zero-shot substitution DMS benchmark by taxa Average Spearman's rank correlation between model scores and experimental measurements on the ProteinGym substitution benchmark, separated by taxon. 'All' is the average across the taxa.
Model type	Model name	Spearman by Mutational Depth (↑)
1	2	3	4	5+	All
Alignment-based	Site-Independent	0.336	0.235	0.226	0.267	0.350	0.283
WaveNet	0.357	0.204	0.250	0.217	0.293	0.264
EVmutation	0.376	0.274	0.324	0.301	0.394	0.334
DeepSequence (ens.)	0.405	0.264	0.313	0.309	0.378	0.334
EVE (ens.)	0.428	0.273	0.308	0.298	0.355	0.332
GEMME	0.447	0.274	0.321	0.324	0.414	0.356
Protein language	UniRep	0.175	0.071	0.111	0.141	0.191	0.138
CARP (640M)	0.390	0.213	0.187	0.164	0.162	0.223
ESM-1b	0.384	0.227	0.187	0.149	0.270	0.243
ESM-2 (15B)	0.407	0.204	0.239	0.172	0.234	0.251
RITA XL	0.356	0.139	0.136	0.154	0.233	0.204
ESM-1v (ens.)	0.403	0.221	0.186	0.151	0.203	0.233
ProGen2 XL	0.385	0.184	0.280	0.219	0.280	0.270
VESPA	0.434	0.183	0.357	0.302	0.328	0.321
Hybrid	UniRep evotuned	0.319	0.154	0.250	0.226	0.294	0.249
MSA Transformer (ens.)	0.426	0.238	0.384	0.366	0.408	0.364
Tranception L	0.423	0.258	0.352	0.318	0.387	0.348
TranceptEVE L	0.446	0.280	0.350	0.320	0.382	0.356
Inverse Folding	ESM-IF1	0.439	0.345	0.290	0.289	0.358	0.344
MIF-ST	0.430	0.265	0.334	0.298	0.298	0.325
ProteinMPNN	0.292	0.257	0.171	0.186	0.278	0.237
Table A9: ProteinGym - Zero-shot substitution DMS benchmark by mutational depth Spearman's rank correlation between model scores and experimental measurements on the ProteinGym substitution benchmark, separated by mutational depths of 1,2,3,4, and 5 or more. The All column is the average across the 5 depths.
Ranking	Model name	Model type	Spearman
1	ProteinNPT	NPT	0.613
2	Tranception	Embeddings	0.571
3	MSA Transformer	Embeddings	0.568
4	ESM-1v	Embeddings	0.542
5	TrancaptEVE	OHE	0.477
6	Tranception	OHE	0.458
7	MSAT	OHE	0.453
8	DeepSequence	OHE	0.440
9	ESM-1v	OHE	0.417
10	OHE w/o augmentation	OHE	0.224
Table A10: ProteinGym - Supervised substitution DMS benchmark Ranking based on corrected average of Spearman's rank correlation between experimental assay measurement and model prediction.
Model type	Model name	Spearman by MSA depth (↑)
Low	Medium	High	All
NPT	ProteinNPT	0.701	0.587	0.608	0.632
Embeddings	Trancection	0.621	0.556	0.561	0.579
MSAT	0.685	0.518	0.567	0.590
ESM-1v	0.653	0.465	0.541	0.553
One-hot encoding	TrancaptEVE	0.503	0.483	0.468	0.485
Trancection	0.490	0.455	0.445	0.463
MSAT	0.500	0.441	0.448	0.463
DeepSequence	0.482	0.422	0.426	0.443
ESM-1v	0.496	0.338	0.400	0.411
No Augmentation	0.246	0.204	0.227	0.226
Table A11: Supervised substitution DMS benchmark by MSA depth Average Spearman's rank correlation between model scores and experimental measurements by MSA depth on the ProteinGym substitution benchmark. Alignment depth is measured by the ratio of the effective number of sequences  $N_{\mathrm{eff}}$  in the MSA, following Hopf et al. [2017], by the length covered  $L$  (Low:  $N_{\mathrm{eff}} / L < 1$ ; Medium:  $1 < N_{\mathrm{eff}} / L < 100$ ; High:  $N_{\mathrm{eff}} / L > 100$ )
Model type	Model name	Spearman by Function Type (↑)
Activity	Binding	Expression	Organismal Fitness	Stability	All
NPT	ProteinNPT	0.577	0.536	0.637	0.545	0.772	0.613
Embeddings	Trancection	0.520	0.529	0.613	0.519	0.674	0.571
MSAT	0.547	0.470	0.584	0.493	0.749	0.569
ESM-1v	0.487	0.450	0.587	0.468	0.717	0.542
One-hot encoding	TrancptEVE	0.502	0.444	0.476	0.470	0.493	0.477
Trancection	0.475	0.416	0.476	0.448	0.473	0.458
MSAT	0.480	0.393	0.463	0.437	0.491	0.453
DeepSequence	0.467	0.418	0.424	0.422	0.471	0.440
ESM-1v	0.421	0.363	0.452	0.383	0.463	0.416
No Augmentation	0.213	0.212	0.226	0.194	0.273	0.224
Table A12: Supervised substitution DMS benchmark by function type Average Spearman's rank correlation between supervised model scores and experimental measurements on the ProteinGym substitution benchmark, separated into five functional categories. Assays are split into one of Activity, Binding, Organismal Fitness, Stability and Expression. The All column is the average of all the categories
Model type	Model name	Spearman by Taxa (↑)
Human	Other Eukaryote	Prokaryote	Virus	All
NPT	ProteinNPT	0.649	0.628	0.668	0.580	0.631
Embeddings	Trancection	0.569	0.582	0.594	0.568	0.578
MSAT	0.634	0.579	0.648	0.521	0.596
ESM-1v	0.565	0.579	0.617	0.433	0.548
One-hot encoding	TrancaptEVE	0.481	0.490	0.475	0.478	0.481
Trancection	0.457	0.472	0.453	0.456	0.460
MSAT	0.482	0.459	0.468	0.448	0.464
DeepSequence	0.451	0.460	0.455	0.383	0.437
ESM-1v	0.426	0.444	0.452	0.292	0.404
No Augmentation	0.236	0.217	0.233	0.238	0.231
Table A13: Supervised substitution DMS benchmark by taxa Average Spearman's rank correlation between model scores and experimental measurements on the ProteinGym substitution benchmark, separated into four taxon categories. Assays are split into one of Human, Prokaryote, Other Eukaryote, or Virus. The All column is the average across the categories.
Ranking	Model	Type	Spearman	Std. error
1	ProGen2 M	Protein language model	0.465	0.000
2	ProGen2 Base	Protein language model	0.464	0.010
3	RITA L	Protein language model	0.457	0.034
4	Trancception M no retrieval	Protein language model	0.453	0.036
5*	RITA XL	Protein language model	0.449	0.037
5*	ProGen2 L	Protein language model	0.449	0.011
7	Trancception L no retrieval	Protein language model	0.437	0.041
8	RITA M	Protein language model	0.436	0.030
9	ProGen2 XL	Protein language model	0.431	0.035
10*	TrancceptEVE M	Hybrid model	0.424	0.045
10*	ProGen2 S	Protein language model	0.424	0.025
12	TrancceptEVE L	Hybrid model	0.412	0.046
13	Trancception S no retrieval	Protein language model	0.410	0.036
14	RITA S	Protein language model	0.397	0.032
15	Trancception L	Hybrid model	0.395	0.043
16	Trancception M	Hybrid model	0.394	0.047
17	HMM	Alignment-based model	0.389	0.045
18	WaveNet	Alignment-based model	0.368	0.067
19	TrancaptEVE S	Hybrid model	0.357	0.049
20	PROVEAN	Alignment-based model	0.347	0.046
21	Trancception S	Hybrid model	0.340	0.053
22	ProtGPT2	Protein language model	0.191	0.053
23	UniRep	Protein language model	0.169	0.060
Table A14: ProteinGym - Zero-shot indel DMS benchmark Ranking based on corrected average of Spearman's rank correlation between experimental assay measurement and model prediction. The standard error reported corresponds to the non-parametric bootstrap standard error of the difference between the Spearman performance of a given model and that of the best overall model (ie., Tran-ceptEVE), computed over 10k bootstrap samples from the set of proteins in the ProteinGym DMS indel benchmark.
Dataset	Tranception	TrancptEVE	ProGen2	RITA	WN.	PRO.	HMM
M	L	M+Ret	L+Ret	M	L	M	XL	L
DMS Assays
amyloid β	0.466	0.416	0.442	0.444	0.459	0.466	0.478	0.462	0.438	0.512	0.381	-0.207
β-Lac	0.365	0.344	0.379	0.296	0.401	0.342	0.619	0.409	0.334	0.437	0.385	0.347
AAV	0.371	0.338	0.126	0.210	0.419	0.416	-0.100	0.167	0.103	-0.007	0.177	0.057
Kir2.1	0.437	0.440	0.412	0.391	0.444	0.431	0.432	0.387	0.383	0.408	0.386	0.368
TP53	0.536	0.362	0.579	0.395	0.560	0.399	0.428	0.354	0.383	0.031	0.273	0.482
PTEN	0.678	0.546	0.700	0.563	0.708	0.602	0.552	0.402	0.504	0.697	0.237	0.668
MtrA	0.612	0.395	0.615	0.375	0.562	0.374	0.403	0.348	0.380	0.244	0.278	0.472
OCT1	0.379	0.458	0.447	0.466	0.442	0.453	0.546	0.383	0.542	0.087	0.301	0.280
Tsuboyama	0.169	0.234	0.434	0.463	0.198	0.265	0.511	0.533	0.489	0.475	0.333	0.364
Assays of Natural and Designed Sequences
AAV	0.677	0.709	0.362	0.691	0.726	0.736	-0.466	0.492	0.543	0.666	0.683	0.607
CM	0.344	0.326	0.219	0.223	0.357	0.340	0.380	0.379	0.337	0.438	0.372	0.398
HIS3	0.611	0.655	0.687	0.707	0.678	0.695	0.702	0.713	0.677	0.687	0.701	0.548
Table A15: Spearman's rank correlation between model scores and individual deep mutational scans of indels. WN and PRO are shorthands for the WaveNet and PROVEAN models respectively.
Ranking	Model	Type	Spearman
1	ESM-1v	Embeddings	0.752
2	Trancection	Embeddings	0.735
3	MSAT	Embeddings	0.689
Table A16: ProteinGym - Supervised indel DMS benchmark Ranking based on corrected average of Spearman's rank correlation between experimental assay measurement and model prediction.
Ranking	Model	Type	AUC
1	ClinPred	Supervised	0.981
2	MetaRNN	Supervised	0.977
3	BayesDel (addAF)	Supervised	0.972
4	VEST4	Supervised	0.929
5	REVEL	Supervised	0.928
6	BayesDel (noAF)	Supervised	0.925
7	VARITY (R)	Supervised	0.921
8	TrancaptEVE	Unsupervised	0.920
9	GEMME	Unsupervised	0.919
10	VARITY (ER)	Supervised	0.918
11	EVE	Unsupervised	0.917
12	gMVP	Supervised	0.914
13	CADD	Supervised	0.905
14	PolyPhen2 (HVAR)	Supervised	0.896
15	DEOGEN2	Supervised	0.894
16	ESM-1b	Unsupervised	0.892
17	PROVEAN	Unsupervised	0.886
18	MPC	Supervised	0.881
19	PolyPhen2 (HDIV)	Supervised	0.879
20	SIFT	Unsupervised	0.878
21	SIFT4G	Unsupervised	0.877
22	MutationAssessor	Unsupervised	0.877
23	MutPred	Unsupervised	0.875
24	PrimateAI	Unsupervised	0.855
25	LIST-S2	Unsupervised	0.842
26	MutationTaster	Supervised	0.816
27	DANN	Supervised	0.812
28	LRT	Unsupervised	0.805
29	FATHMM	Supervised	0.723
Table A17: ProteinGym - Clinical substitution benchmark Ranking based on AUROC between model prediction and ClinVar benign/pathogenic annotation.
Model Type	Model Name	AUROC(↑)	AUPRC (↑)
Alignment-based models	HMM	0.679	0.775
PROVEAN	0.926	0.947
WaveNet	-	-
Protein language models	UniRep	0.395	0.600
RITA XL	0.923	0.954
ProGen2 XL	0.846	0.889
Tranception L (no retrieval)	0.877	0.938
Tranception M (no retrieval)	0.858	0.929
ProtGPT2	0.655	0.779
Hybrid models	Tranception L	0.857	0.920
Tranception M	0.844	0.909
TranceptEVE	0.857	0.916
Table A18: ProteinGym - Clinical indels benchmark Results for indel-compatible baselines on our ClinVar/gnomAD indel benchmark. AUPRC is area under the precision recall curve, and AUROC is area under the receiver-operating characteristic curve. Bold denotes best method, with the runner-up underlined.
Table A19: List of substitution datasets See the reference file in the GitHub repo for other info (UniProt ID, taxon, DOI, and more assay details).
Dataset	Reference
β-Lactamase	Jacquier et al. [2013]
β-Lactamase	Stiffler et al. [2015]
β-Lactamase	Firnberg et al. [2014]
β-Lactamase	Deng et al. [2012]
β-Lactamase VIM-2	Chen et al. [2020]
β-Glucosidase	Romero et al. [2015]
AAV	Sinai et al. [2021]
ACE2	Chan et al. [2020]
ADRB2	Jones et al. [2020]
APH(3')II, neo	Melnikov et al. [2014]
APP	Seuma et al. [2021]
Activation-induced deaminase	Gajula et al. [2014]
Aliphatic amidase	Wrenbeck et al. [2017]
Alpha-synuclein	Newberry et al. [2020]
Amyloid β	Gray et al. [2019]
Amyloid β	Seuma et al. [2022]
Ancestral spleen tyrosine kinase	Hobbs et al. [2022]
Anti-CRISPR protein AcrIIA4	Stadelmann et al. [2021]
Antitoxin ParD3	Ding et al. [2023]
Antitoxin ParD3	Aakre et al. [2015]
Arrestin-1	Ostermaier et al. [2014]
BRCA1	Findlay et al. [2018]
BRCA2	Erwood et al. [2022]
CALM1	Weile et al. [2017]
CARD11	Meitlis et al. [2020]
CASP3	Roychowdhury and Romero [2022]
CASP7	Roychowdhury and Romero [2022]
CBS (cystathionine beta-synthase)	Sun et al. [2020]
CCR5	Gill et al. [2023]
CD19	Klesmith et al. [2019]
CVB3 capsid	Mattenberger et al. [2021]
CXCR4	Gill et al. [2023]
Chalcone synthase	Wrenbeck et al. [2019]
Cytochrome P450 2C9	Amorosi et al. [2021]
Cytochrome P450 2C9	Amorosi et al. [2021]
D-amino acid oxidase	Vanella et al. [2023]
DHFR reductase	Nguyen et al. [2023a]
DHFR reductase	Thompson et al. [2020]
DNA methylase HaeIII	Rockah-Shmuel et al. [2015]
Dengue virus NS5	Suphattrakul et al. [2023]
Dlg4, (PSD95_PDZ3)	McLaughlin et al. [2012]
EfrC	Meier et al. [2023]
EfrD	Meier et al. [2023]
EnvZ	Ghose et al. [2023]
ErbB2 membrane domain	Elazar et al. [2016]
EstA	Nutschel et al. [2020]
GAL4	Kitzman et al. [2015]
GB1	Wu et al. [2016]
GB1	Olson et al. [2014]
GDI1	Silverstein et al. [2021]
GFP	Sarkisyan et al. [2016]
GMR (aacC1)	Dandage et al. [2018]
GRB2-SH3	Faure et al. [2022]
Gcn4	Staller et al. [2018]
Glucokinase regulatory protein	Gersing et al. [2023]
Glucokinase regulatory protein	Gersing et al. [2022]
Glycophorin A membrane domain	Elazar et al. [2016]
Green fluorescent protein amacGFP	Gonzalez Somermeyer et al. [2022]
Green fluorescent protein cgreGFP	Gonzalez Somermeyer et al. [2022]
Green fluorescent protein ppluGFP2	Gonzalez Somermeyer et al. [2022]