SlowGuess/NeurIPS_2023_OCR huggingface dataset:
VHH Substantiation The gene sequences encoding each selected VHH clone, which were connected with a  $4 \times$  (GGGGS) linker for expression as a tandem dimer, were codon-optimized and synthesized (Eurofins Genomics Inc., Tokyo, Japan). The synthesized genes were subcloned into the pMES4 vector to express N-terminal PelB signal peptide-conjugated and C-terminal  $6 \times$  Hist-tagged VHHs. BL21 (DE3) E. coli cells transformed with the plasmids were plated on LB agar with ampicillin and incubated at  $37^{\circ} \mathrm{C}$  overnight. Growth colonies were picked and cultured at  $37^{\circ} \mathrm{C}$  to reach an OD of 0.6 AU, and the cells were then cultured at  $37^{\circ} \mathrm{C}$  for three hours with  $1 \mathrm{mM}$  of IPTG (isopropyl- $\beta$ -D-thiogalactopyranoside, Nacalai Tesque). Lastly, the cultured cells were pelleted by centrifugation and stored in a freezer until use. VHHs were eluted from the periplasm by soaking in TES buffer (200 mM Tris, 0.125 mM EDTA, 125 mM sucrose, and pH 8.0) at  $4^{\circ} \mathrm{C}$  for one hour. They were further incubated with a  $2 \times$  volume of  $0.25 \times$  diluted TES buffer with a trace amount of benzonase nuclease (Merck) at  $4^{\circ} \mathrm{C}$  for 45 minutes. The supernatants were centrifuged  $(20,000 \times \mathrm{g}, 4^{\circ} \mathrm{C}$  for 10 minutes), sterilized by adding gentamicin (Thermo), and passed through a  $0.22 \mu \mathrm{m}$  filter (Sartorius AG, Gottingen, Germany). The filtered supernatants were then applied to a HisTrap HP nickel column (Cytiva) on an ÅKTA pure HPLC system, and the bound His-tagged VHHs were eluted with  $300 \mathrm{mM}$  of imidazole. The eluted fraction was collected and concentrated with a VIVAspin 3000-molecular-weight cutoff filter column (Sartorius) and applied to a Superdex75 10/300 GL gel-filtration column (Cytiva) on an ÅKTA pure HPLC system. Finally, the protein purity was measured via Coomassie brilliant blue (CBB) staining (Rapid Stain CBB Kit, Nacalai Tesque).
Immunofluorescence Staining Analysis HEK293T cells were transiently transfected with a plasmid-encoding C-terminally HA-tagged wild-type IL-6 protein by using Lipofectamine 3000 (Thermo) according to the manufacturer's instructions. The next day, the cells were seeded on collagen type-I-coated culture plates (IWAKI, AGC TECHNO GLASS CO., LTD., Shizuoka, Japan) and cultured for 24 hours before being fixed with  $2\%$  paraformaldehyde (PFA) at  $4^{\circ}\mathrm{C}$  overnight. After three washes with PBST (PBS with  $0.005\%$  Tween 20), the cells were blocked with PBST containing  $2\%$  goat serum (blocking solution) at room temperature for one hour. Each well was soaked with  $100~\mu \mathrm{L}$  of the blocking solution containing  $100\mathrm{ng}$  of purified VHH at  $4^{\circ}\mathrm{C}$  overnight. After washing with PBST, 1:3000-diluted anti-His-tag rabbit antibodies and 1:100-diluted anti-HA 7C9 mouse
monoclonal antibodies (ChromoTek GmbH, Planegg-Martinsried, Germany) in blocking buffer were added and reacted at room temperature for one hour. Finally, after washing, Alexa-Fluor-conjugated anti-rabbit IgG (594 nm emission) antibodies at 1:3000 dilution and Alexa-Fluor-conjugated antimouse IgG (488 nm emission) antibodies at 1:3000 dilution in blocking buffer were added to the wells, and the fixed cells were labeled at room temperature for one hour before washing three times with PBST. The cell nuclei were visualized with  $4^{\prime},6$ -diamidino-2-phenylindole (DAPI). The stained cells were imaged with an 8-ms exposure time (594 nm emission), a 40-ms exposure time (488 nm emission), or an automatically adjusted exposure time (DAPI) by using an IX71S1F-3 microscope (Olympus Corporation, Tokyo, Japan) with the cellSens Standard 1.11 application (Olympus). Each full observed field corresponding to a  $165~\mu \mathrm{m} \times 220~\mu \mathrm{m}$  square was photographed.
Kinetic Assays via Biolayer Interferometry (BLI) Real-time binding experiments were performed using an Octet Red96 instrument (forteBIO, Pall Life Science, Portsmouth, NH). Each purified VHH clone was biotinylated with EZ-Link Sulfo-NHS-LC-Biotin (Thermo) according to the manufacturer's protocol; uncoupled biotin was excluded with a size exclusion spin column (PD SpinTrap G-25, Cytiva) in PBS  $(\mathrm{pH}7.4)$ . Assays were performed at  $30^{\circ}\mathrm{C}$  with shaking at 1000 rpm. Biotin-conjugated clones at  $10~\mu \mathrm{g / mL}$  were captured on a streptavidin-coated sensor chip (SA, forteBIO) to reach the signals at  $1\mathrm{nm}$ . One unrelated VHH P17-coated sensor chip was monitored as a baseline. The loaded concentration of the wild-type IL-6 was  $200~\mu \mathrm{g}$ , corresponding to  $0.625~\mu \mathrm{M}$ . Assays were performed with PBS containing  $0.005\%$  Tween 20 (Nacalai Tesque). After baseline equilibration for  $180\mathrm{s}$  in the buffer, association and dissociation were each performed for  $180\mathrm{s}$ . The data were then subtracted from the baseline data and analyzed with fortBIO data analysis software 9.0.
A.3.2 Additional Results
As listed in Table 3, we selected 12 binder-labeled, six non-binder-labeled, and five noise-labeled clones for substantiation. Of the 12 binder-labeled clones, two could not be isolated by the E. coli
(a)
Figure 5: Immunofluorescence staining analysis using (a) 10 binder-labeled, (b) six non-binder-labeled, and (c) four noise-labeled VHHs. HA-tagged IL-6 proteins were introduced into the cells and stained with His-tagged VHHs. Subsequently, HA-tags were visualized in green and His-tags in red.
(a) Binder
(b) Non-binder
(c) Noise
Figure 6: Biolayer interferometry analysis using (a) 10 binder-labeled, (b) six non-binder-labeled, and (c) four noise-labeled VHHs. Each line represents the association and dissociation curve of each VHH on the wild-type IL-6. When a line moves up from the base point in the association phase, the VHH is considered a binder.
protein synthesis system because of the limitations of the phage display method. Even if a protein of interest functioned as a fusion protein with the g3p protein on a phage, it was not always possible to express the protein alone in a soluble form with function [16]. However, if a sufficient signal was observed in the mother library, then the clones must have been truly present as heavy-chain antibodies, at least in the alpaca body. The remaining 10 clones all showed binding to the wild-type IL-6 protein, as shown in Figure 5(a). The immunofluorescence staining analysis showed strong to weak signals, which probably reflected avidity differences between the clones. Note that the calculated p-values did not correlate with the staining intensity by a simple inverse relationship. The biolayer interferometry (BLI) analysis revealed that the clones positively associated with the wild-type IL-6 protein with different association curves  $(\mathrm{K}_{on})$ , dissociation curves  $(\mathrm{K}_{off})$ , and KDs  $(\mathrm{K}_{off} / \mathrm{K}_{on})$ , as shown in Figure 6(a), although the sensitivity was relatively lower than that of the immunostaining analysis. All six non-binder-labeled clones showed negative results in both the immunostaining and BLI analyses, as shown in Figures 5(b) and 6(b), respectively. Of the five noise-labeled clones, one could not be isolated. The immunostaining analysis showed that the remaining four clones likely had nonspecific binding, as shown in Figure 5(c), but not to the wild-type IL-6 protein, as confirmed by the BLI analysis results shown in Figure 6(c). Accordingly, the sensitivity and specificity of our labeling method can be considered sufficiently high.
A.4 Benchmarks
A.4.1 Data Splitting
For a test set, we randomly selected 15 mutants: P42A, T48A, E51A, I57A, I60A, K69A, C78A, S81A, E87A, L120A, L126A, L129A, Q144A, D162A, and T165A. The remaining 15 mutants and the wild type were used for model training. First, we trained the models by using only the wild-type IL-6 protein and evaluated the model performance on the test set. Then, we randomly selected one mutant from the mutants not contained in the test set and added it to the training set to evaluate each model's predictive performance on the test set. By repeating this procedure, we tracked each model's predictive performance for unknown mutants contained only in the test set. Because the order of adding mutants to the training set affected the model performance, we ran the same experiment five
times in shuffled order, and we report the averaged results in section 4.3. Table 4 summarizes the order in which mutants were added and the number of samples in each set.
A.4.2 Model Implementations
We adopted AbAgIntPre because it is a state-of-the-art model designed for the same task setting as ours of predicting interactions solely from antigen and antibody sequences. At present, fewer studies have focused on developing machine learning models for predicting antigen-antibody interactions based only on amino acid sequences, as compared to PPI. However, antigen-antibody interactions that ignore non-protein antigens can be considered a subset of PPI, meaning that models designed for PPI are also applicable to antigen-antibody interactions. Thus, we adopted PIPR as a representative neural-network-based model designed for PPI. In addition, we used MLP as a simpler, shallower neural network model than AbAgIntPre and PIPR. Lastly, we chose LR as a classical machine learning model other than neural networks. The implementations of all the benchmark models are available at https://github.com/cognano/AVIDa-hIL6.
- AbAgIntPre [20]. We used the implementation<sup>3</sup> that is provided by AbAgIntPre's developers and released under Apache License 2.0 for the composition of k-spaced amino acid pairs (CKSAAP) [11] encoding. The calculation was performed using  $\mathrm{k} = 0, 1, 2, 3$ , thus yielding a 1600-dimensional vector for each amino acid sequence. We also used the original PyTorch [33] implementation released under Apache License 2.0 for the AbAgIntPre model. We used the model parameters reported in the original paper [20].
- PIPR [10]. We reimplemented PIPR by using PyTorch with reference to the original implementation<sup>4</sup> released under Apache License 2.0. We changed the number of RCNN units from five to three, while the other parameters were the same as in the original paper [10]. Each RCNN unit had a one-dimensional max pooling with a kernel size of three, which shortened the sequence length by a third. Our dataset's maximum sequence length is 218, and the application of five RCNN units would have resulted in a sequence length shorter than one; thus, we reduced the number of units. In addition, we used the pretrained embeddings published by PIPR's developers in their original implementation as "vec5_CTC.txt."
- Multi-Layer Perceptron (MLP). We implemented one-hot encoding and an MLP with one hidden layer of 512 neurons and the rectified linear unit (ReLU) activation function by using PyTorch. The one-hot vectors of the VHHs and IL-6 proteins were flattened and concatenated for input to the MLP. We used zero padding to match the dimensions, thus yielding 8000-dimensional vectors for each VHH and IL-6 protein pair.
- Logistic Regression (LR). We implemented LR by using scikit-learn [34]. We used the default settings of scikit-learn v1.2.2 as model parameters. As with the MLP, the one-hot vectors of the VHHs and IL-6 proteins were flattened and concatenated for input to the LR model. We used zero padding to match the dimensions, thus yielding 8000-dimensional vectors for each VHH and IL-6 protein pair.
A.4.3 Model Training
All three neural network-based models were trained on one NVIDIA Tesla V100 GPU on Google Colaboratory. The models were trained for 100 epochs with a batch size of 256. We used the Adam optimizer with a fixed learning rate of 0.0001 and no weight decay. Also, we used the binary cross-entropy with sigmoid activation as the loss function. During the model training, the GPU memory consumptions for AbAgIntPre, PIPR, and MLP were approximately 1.4, 1.6, and 1.1 GB, respectively. When using 16 IL protein types for training, the training times for AbAgIntPre, PIPR, and MLP were approximately 0.5, 1, and 0.5 hours, respectively.
The LR model, with L2 penalization, was trained using one Intel(R) Xeon(R) CPU @ 2.20 GHz on Google Collaboratory. The solver was the LBFGS algorithm, with 1,000 as the maximum number of iterations for convergence. When using 16 IL protein types for training, the training time for LR was approximately 0.5 hours. More detailed training information is available at https://github.com/cognano/AVIDa-hIL6.
3AbAgIntPre: https://github.com/emersON106/AbAgIntPre
4PIPR: https://github.com/muhaochen/seq_ppi
Table 4: Details of the data splitting.
Experiment	#Antigen types	Added antigen	#Samples
Binder	Non-binder	Total
Run 1	1	Wild-type	540	12,480	13,020
2	G63A	1,074	39,629	40,703
3	F153A	2,232	49,940	52,172
4	T117A	2,967	64,508	67,475
5	Q45A	3,799	80,574	84,373
6	E108A	4,424	99,729	104,153
8	C72A, F102A	5,503	148,208	153,711
10	P93A, E138A	7,249	178,334	185,583
12	D54A, S135A	8,737	222,447	231,184
14	D168A, D99A	9,452	257,265	266,717
16	L90A, G105A	10,564	282,279	292,843
Run 2	1	Wild-type	540	12,480	13,020
2	E138A	1,942	22,052	23,994
3	C72A	2,406	47,106	49,512
4	L90A	2,656	59,150	61,806
5	F153A	3,814	69,461	73,275
6	E108A	4,439	88,616	93,055
8	D99A, G105A	5,737	124,168	129,905
10	D168A, D54A	6,509	165,155	171,664
12	G63A, P93A	7,387	212,858	220,245
14	Q45A, S135A	9,214	244,286	253,500
16	F102A, T117A	10,564	282,279	292,843
Run 3	1	Wild-type	540	12,480	13,020
2	G63A	1,074	39,629	40,703
3	L90A	1,324	51,673	52,997
4	G105A	2,186	64,643	66,829
5	F102A	2,801	88,068	90,869
6	Q45A	3,633	104,134	107,767
8	D99A, D54A	4,562	155,467	160,029
10	E138A, E108A	6,589	184,194	190,783
12	S135A, C72A	8,048	224,610	232,658
14	P93A, D168A	8,671	257,400	266,071
16	F153A, T117A	10,564	282,279	292,843
Run 4	1	Wild-type	540	12,480	13,020
2	F102A	1,155	35,905	37,060
3	D99A	1,591	58,487	60,078
4	L90A	1,841	70,531	72,372
5	G105A	2,703	83,501	86,204
6	D54A	3,196	112,252	115,448
8	E138A, P93A	4,942	142,378	147,320
10	C72A, E108A	6,031	186,587	192,618
12	F153A, G63A	7,723	224,047	231,770
14	Q45A, T117A	9,290	254,681	263,971
16	D168A, S135A	10,564	282,279	292,843
Run 5	1	Wild-type	540	12,480	13,020
2	D99A	976	35,062	36,038
3	E108A	1,601	54,217	55,818
4	F102A	2,216	77,642	79,858
5	G63A	2,750	104,791	107,541
6	D168A	3,029	117,027	120,056
8	F153A, Q45A	5,019	143,404	148,423
10	T117A, G105A	6,616	170,942	177,558
12	D54A, C72A	7,573	224,747	232,320
14	P93A, S135A	8,912	260,663	269,575
16	L90A, E138A	10,564	282,279	292,843