# Capturing Protein Domain Structure and Function Using Self-Supervision on Domain Architectures
## 3 Materials and Methods
### Quantitative Intrinsic Evaluation  
In the following, we use the metadata for the most characteristic properties of domains, in order to evaluate the learned embedding space for various hyper-parameters of _word2 vec_. We propose four intrinsic evaluation approaches for domain embeddings:  
Figure 2: Non-overlapping and non-redundant domain architectures of the _Diphthine synthase_ protein. domain hierarchy based on the family/subfamily relation, SCOPe secondary structure class, EC primary class, and GO molecular function annotation. We refer to the embedding space learned by _word2 vec_ for a particular set of hyperparameters as \(V_{emb}\). The \(k\) nearest neighbors of a domain \(d\) is found by using the Euclidean distance, and it is denoted as \(C_{nearest}^{d}\). To inspect the relative performance of \(V_{emb}\) on each of the following evaluations, we _randomized_ all domain vectors and ran each evaluation task. That is, we assigned to each domain vector a newly created random vector, for each unique dimensionality of embedding space, irrespective of all other embedding method parameters. Domain hierarchy  
InterPro defines a strict family-subfamily relationship among domains. This relationship is based on sequence similarity of the domain signatures. We refer to the children of domain \(p\) as \(S_{p}\). We use these relationships to evaluate an embedding space, posing the following research question,  
\(RQ_{hierarchy}\): _Did vectors of hierarchically close domains form clusters in the \(V_{emb}\)?_  
**Evaluation** We predicted the closest \(|S_{p}|\) domains on cosine similarity of their vector to the parent vector, and we denote this predicted set as \(\hat{S}_{p}\). For all learned embedding spaces, we measured their recall performance, \(Recall_{liter}\), defined as follows:  
\[Recall_{liter}=\sum_{p}\frac{|S_{p}\cap\hat{S}_{p}|}{|S_{p}|}. \tag{1}\]  
SCOPe Secondary Structure Class  
We extracted the secondary structure of Interpro domains from the SCOPe database and formed the following research question,  
\(RQ_{SCOPe}\): _Did vectors of domains, with same secondary structure class, form clusters in the \(V_{emb}\)?_  
**Evaluation** We evaluated \(V_{emb}\) by retrieving \(C_{nearest}^{d}\) of each domain. Then, we applied stratified 5-fold cross-validation and measured the performance of a \(k\)-nearest neighbor classifier to predict the structure class of each domain. The intrinsic evaluation performance metric is the average accuracy across all folds, \(Accuracy_{SCOPe}\). The enzymatic activity of each domain is given by its primary EC class [32] and we pose the following research question,  
\(RQ_{EC}\): _Did vectors of domains, with the same enzymatic primary class, form clusters in the \(V_{emb}\)?_  
**Evaluation** We again evaluate \(V_{emb}\) using \(k\) nearest neighbors in a stratified 5-fold cross-validation setting. The average accuracy across all folds, \(Accuracy_{EC}\), is again used to quantify the intrinsic quality of the embedding space. For our last intrinsic evaluation, we aimed to assess \(V_{emb}\) using the molecular function GO annotation. We extracted all molecular function GO annotations associated with each domain. In order to account for differences in specificity of different GO annotations, we always used the depth-1 ancestor of each annotation, that is, children of the root molecular function term, GO:0003674. Since model organisms have the most-annotated proteins, we created GO molecular function data sets for one example of prokaryote (_Escherichia coli_, denoted _E. coli_); one example of a simple eukaryote (_Saccharomyces cerevisiae_, denoted _S.cerevisiae_); and one complex eukaryote (_Homo sapiens_, denoted _Human_). To also assess our embeddings for not highly annotated organisms, we included a molecular function data set for an example of a human pathogen (_Plasmodium falciparum_, denoted as _Malaria_).