Semisupervised Neural Proto-Language Reconstruction (2024)

Abstract: Existing work implementing comparative reconstruction of ancestral languages (protolanguages) has usually required full supervision. However, historical reconstruction models are only of practical value if they can be trained with a limited amount of labeled data. We propose a semisupervised historical reconstruction task in which the model is trained on only a small amount of labeled data (cognate sets with proto-forms) and a large amount of unlabeled data (cognate sets without proto-forms). We propose a neural architecture for comparative reconstruction (DPDBiReconstructor) incorporating an essential insight from linguists’ comparative method: that reconstructed words should not only be reconstructable from their daughter words, but also deterministically transformable back into their daughter words. We show that this architecture is able to leverage unlabeled cognate sets to outperform strong semisupervised baselines on this novel task1.

heading: 1 Introduction
text: In the 19th century, European philologists made a discovery that would change the direction of the human sciences: they discovered that languages change in systematic ways and that, by leveraging these systematic patterns, it was possible to reproducibly reconstruct ancestors of families of languages (protolanguages) even when no record of those languages survived. This technique, called the comparative method, provided an unprecedented window into the human past—its cultures, its migrations, and its encounters between populations.
The assumption that historical changes in pronunciation (“sound changes”) are regular, known as ‘the regularity principle’ or ‘the Neogrammarian hypothesis’, is fundamental to the comparative
1The code is available at https://github.com/ cmu-llab/dpd.
method (Campbell, 2021). As the 19th century Neogrammarians Hermann Osthoff and Karl Brugmann put it:
“Every sound change, in so far as it proceeds mechanically, is completed in accordance with laws admitting of no exceptions; i.e. the direction in which the change takes place is always the same for all members of a language community, apart from the case of dialect division, and all words in which the sound subject to change occurs in the same conditions are affected by the change without exception.” (Morphologische Untersuchungen auf dem Gebiete der indogermanischen Sprachen i, Brugmann and Osthoff, 1878, p. xiii, translated and quoted in Szemerényi, 1996)
The comparative method, however, is challenging for humans to apply. This is true largely because it involves dealing with large volumes of data and modeling numerous interactions between competing patterns. One must balance the need for phonetic similarity between reconstructed words and their descendants (reflexes) with the need to be able to deterministically derive reflexes from
14715
reconstructed words with a single set of sound changes. It imposes a heavy cognitive load. For this reason, researchers have long aspired to implement the comparative method computationally.
With some exceptions (He et al., 2023; Akavarapu and Bhattacharya, 2023), recent attempts at automatic reconstruction models mostly take the form of neural sequence-to-sequence transduction models similar to those used for machine translation, trained with full supervision on a dataset where every cognate set is paired with a gold reconstruction (Meloni et al., 2021; Chang et al., 2022; Fourrier, 2022; Cui et al., 2022; Kim et al., 2023).
The development of supervised reconstruction systems has given the field insights into how reconstruction of protolanguages (unattested ancestor languages) can be modelled computationally. However, in a realistic scenario, these models only become usable once the hardest part of reconstruction has been done (since they rely on the linguist having already identified enough of the sound changes in the data to reconstruct a substantial part of the lexicon). Computational models of comparative reconstruction are most useful if they can be deployed without training data, or if only a small volume of labeled data is needed to prime the comparative pump.
We introduce a semisupervised protoform (reconstructed parent word) reconstruction task wherein the reconstruction model has access at training time to both a small number of cognate sets (sets of daughter words—reflexes—of a single parent) labeled with a protoform and a large number of unlabeled cognate sets, mirroring the situation of historical linguists early in their reconstruction of a protolanguage. Though similar to semisupervised machine translation, the semisupervised reconstruction formulation entails the absence of target-side monolingual data. Most semisupervised machine translation techniques rely on monolingual data in the target language, such as back-translation (Edunov et al., 2018; Sennrich et al., 2016) and pre-trained target-side language models (Skorokhodov et al., 2018; Gülçehre et al., 2015). In contrast, in this task, models have access only to cognate sets (no monolingual text), meaning the structure of the problem is quite different.
In this paper, we propose to incorporate the comparative method into a semisupervised reconstruction model via end-to-end multi-task learning. Our
proposed model, named DPD-BiReconstructor, learns to improve its reconstructions by performing reflex predictions on an intermediate representation of its predicted reconstructions. Reflex prediction losses are propagated into the reconstruction network, allowing the model to train on cognate sets without protoform labels. A hypothetical example from three Tangkhulic languages is shown in Table 1. In this example, the phonetic information in the ‘grandchild’ and ‘bone’ sets is insufficient to reconstruct ‘laugh’ with a distinct vowel, so daughter-to-proto (D2P)models will typically reconstruct the two words identically. Models incorporating reflex prediction, however, are able—in principle—learn to reconstruct words like ‘laugh’ as distinct from words like ‘breast’. Experiments show that DPD is an attractive approach for semisupervised reconstruction, and a combination of DPD with existing semisupervised strategies performs significantly better than baseline strategies in almost all situations. Additionally, analyses show some indication that DPD-based models could help improve supervised reconstruction.

heading: 2 Methods

heading: 2.1 Model
text: We propose a multi-task reconstruction strategy that learns to recover reflexes from its own reconstructions, effectively utilizing unlabeled cognate sets. Our model comprises a reconstruction subnetwork (D2P for daughter-to-protoform) and a reflex-prediction sub-network (P2D for protoformto-daughter) with shared phoneme embeddings. On labeled data, the model learns sound changes from accurate reconstructions to reflexes, in addition to learning reconstruction. In the absence of labeled protoforms, the reflex prediction subnetwork acts as a weak supervision by informing the reconstruction sub-network on whether correct reflexes can be derived from proposed reconstructions. This workflow directly mirrors the comparative method’s constraint that reconstruction must yield protoforms such that reflexes can be derived from them through regular sound changes. We refer to our training strategy and its architectural realization as Daughters-to-Protoform-to-Daughters Bidirectional Reconstructor (DPD).
For D2P to learn from P2D, we propagate gradients from P2D into D2P. It is not feasible, however, to simply feed token predictions from D2P into P2D, as token outputs are not differentiable.
P2D (for incorrectly
predicted protoforms) are propagated into both sub-networks. An additional cosine similarity loss LBRIDGE is used to train the bridge network. The overall loss is calculated as a weighted sum of all the losses, with the weights being tunable hyperparameters. Reflexes in the same cognate are concatenated together (with separaters) into one sequence and embedded with both phoneme and language embedding as in Meloni et al. (2021) and Kim et al. (2023) (and an additional positional embedding for Transformer). Phoneme embedding is shared among D2P and P2D, whereas language embedding is only used in D2P.
Instead, we represent reconstruction outputs as a continuous latent space, inspired by end-to-end spoken language understanding systems in which a semantic understanding sub-network receives a latent representation of text transcriptions from a speech-recognition sub-network (Saxon et al., 2021; Arora et al., 2022). In particular, D2P’s finallayer decoder output is connected to P2D’s encoder via a trainable dense network, referred to as the bridge network (See Figure 1).
There is no easy way to transform gold protoforms used to train P2D so that they match the bridge network’s output latent representation. To encourage a consistent input representation for P2D, we add a cosine similarity loss between the bridge network’s output and the actual phoneme embeddings of D2P’s predictions, effectively training the bridge network to serve as an alternative embedding for reconstructed protoforms.
As a final training objective, we discourage P2D from producing correct reflexes given incorrect protoforms so that D2P can be better informed when producing incorrect reconstructions. This is achieved via a CRINGE loss on the P2D outputs, designed to penalize negative examples (Adolphs et al., 2022). For negative examples, the CRINGE loss contrasts each negative token with
an estimated positive token (by sampling a nonnegative token in the model’s top-k predictions) and applies a contrastive learning objective (crossentropy loss) to lower the probability of the negative token. When training the P2D sub-network, a correctly predicted reflex is considered a negative example if the input was an incorrect protoform.
The DPD strategy does not specify the architecture of the D2P and P2D sub-networks. In this paper, we adapt existing encoder-decoder architectures proposed for neural reconstruction, including GRU sub-networks based on Meloni et al. (2021)2 and Transformer sub-networks based on Kim et al. (2023).

heading: 2.2 Semisupervised Strategies
text: Aside from our DPD strategy, a naïve approach is to discard unlabeled data and perform supervised training. We take this, the supervisedonly strategy, as our first baseline. To compare with established semisupervised machine learning techniques, we implement two more strategies: Bootstrapping and Π-models, representing
2We use Chang et al. (2022)’s PyTorch reimplementation obtain from https://github.com/cmu-llab/ middle-chinese-reconstruction, with minor modifications to support multi-layer.
the proxy-labelling and consistency regularization approaches respectively.
Bootstrapping adds the model’s most confident predictions on unlabeled data to the train set as pseudo-labels (Lee, 2013). In our Bootstrapping setup, the model’s most confident (i.e. probable) protoform reconstructions for unlabeled cognate sets, filtered by a minimum confidence threshold and capped at a maximum number of top predictions per epoch, are added as pseudo-labels to the training set at the end of each epoch starting from a set number of warmup epochs (See Appendix C for Bootstrapping hyperparameters).
Π-model optimizes the model’s consistency by creating two stochastically augmented inputs from the same training example, feeding both of augmented inputs into the model, and minimizing the mean square difference between the two outputs (Laine and Aila, 2017). For continuous inputs, stochastic augmentation could be simple noise. Stochastic changes to phonemes, however, would defy protoform reconstruction’s goal of finding regular sound changes. Instead, we implement stochastic cognate set augmentation by randomly permuting the order of reflexes and randomly dropping daughter languages.
Observe that some of the above strategies can be combined: Bootstrapping can always be used on top of every other strategy, while our proposed DPD architecture can be merged with Π-model into a model that performs both reflex prediction and consistency regularization (See Appendix O for detail). In total, we test 8 strategies: supervised only (SUPV), Bootstrapping (BST), Πmodel (ΠM), Π-model with Bootstrapping (ΠMBST), DPD-BiReconstructor (DPD), DPD with Bootstrapping (DPD-BST), DPD merged with Πmodel (DPD-ΠM), and DPD-ΠM with Bootstrapping (DPD-ΠM-BST). Among the 8 strategies, we consider single baseline strategies (SUPV, BST, ΠM) to be weak baselines, and the combination of non-DPD semisupervised techniques (ΠM-BST) to be the strong baseline. Combined with 2 encoder-decoder architectures, GRU and Transformer, we experiment with 16 strategyarchitecture combinations which we identify by an architecture prefix (GRU- or Trans-) followed by the strategy name.

heading: 2.3 Experiments
text: Datasets: We test both Romance and Sinitic languages, represented by the phonetic version3 of Meloni et al. (2021)’s Romance dataset (Romphon) for Latin reconstruction and Chang et al. (2022)’s WikiHan dataset for Middle Chinese reconstruction4. We simulate the semisupervised reconstruction scenario by hiding a random subset of protoform labels from the fully labeled train set. We refer to the percentage of labels retained after label removal as the labeling setting. Cross-strategy comparisons: The primary interest of our experiments is how reconstruction performance differs between strategies with a fixed percentage of labels. We fix the labeled percentage at 10%, which entails approximately 516 and 870 labeled cognate sets for WikiHan and Romphon respectively. Due to randomness in semisupervised dataset generation, we repeat the experiment on four randomly generated semisupervised trained sets for each of WikiHan and Rom-phon, labeled group 1 to group 4. Each of the 16 strategyarchitecture combinations is tested 10 times in each group. We then compare the performance of strategies within the same architecture. Cross-labeling setting comparisons: We test all strategies on datasets with 5%, 10%, 20%, and 30% of labels to study the relationship between the labeling setting and the model’s performance5. With these labeling settings, the numbers of labeled cognate sets ranging from 181 to 1,084 for WikiHan and 304 to 1,821 for Rom-phon6. Randomly selecting labels for each of the labeling settings—especially on already small datasets— introduces variations on the learnable information contained in the labels and could introduce noise. To mitigate this, we enforce a monotonic subset selection constraint: given the complete train label set L and a semisupervised label set Lpi ⊆ L retaining pi per cent of the labels, semisupervised sets Lp1 , Lp2 of increasing percentages (p1 ≤ p2) must satisfy Lp1 ⊆ Lp2 (See Figure 2). The 10% labeled
3We focus on the phonetic version due to a large number of semisupervised strategies and since DPD is primarily motivated by phonetic reconstruction.
4Rom-phon is not licenced for redistribution; WikiHan is licenced under Creative Commons CC0. For more dataset details, see Appendix A.
5It is often the case that, in manual reconstruction projects, the majority of sound changes can be discovered from a minority of the available cognate sets. We hope to capture this observation with our choice of labeling settings.
6See Table 6 for detail.
dataset we use in cross-labeling setting comparison corresponds to group 1 in cross-strategy comparison (See Appendix B for detail).
Hyperparameters: We use Bayesian search to tune hyperparameters for all 16 strategyarchitectures combinations on a fixed 10% labeled semisupervised dataset7. For each strategyarchitecture combination, for 100 iterations, we select the hyperparameter leading to the best validation phoneme edit distance. See Appendix C for details on hyperparameters. Evaluationmetrics: Evaluation metrics we use follow directly from supervised reconstruction literature, including token edit distance (TED), a count of the number of insertion, deletion, or substitution operations between the prediction and the target (Levenshtein, 1966); token error rate (TER), a length-normalized token edit distance (Cui et al., 2022); accuracy (ACC); feature error rate (FER), a measure of phonetic distance by PanPhon (Mortensen et al., 2016); and B-Cubed F Score (BCFS), a measure of structural similarity between the prediction and the target (Amigó et al., 2009; List, 2019). Statistical tests: For each combination of dataset group, labeling setting, and architecture, we test all strategies against each other for differences. We use both the Wilcoxon Rank-Sum test (Wilcoxon, 1992) and Bootstrap test for mean difference (Sivaganesan, 1994) to test for significance in performance difference. We use a α = 0.01 significance threshold and consider results to be significant if indicated by both tests.
7Due to a large number of strategy-architecture combinations, tuning the model on every labeling settings is costly. We fix the dataset generation seed to 0 for tuning.

heading: 3 Results
text: Cross-strategy comparisons: Table 2 shows the performance of all strategy-architecture combinations on four 10% labeled datasets. For both architectures on WikiHan, DPD-ΠM-BST performs the best and significantly better than all baselines on all metrics. Transformer trained with DPD attains similar performance to DPD-ΠM-BST, outperforming all baseline strategies. GRU trained with DPD performs similarly to ΠM-BST, both of which perform better than weak baselines in a majority of situations. On Rom-phon, Transformer performed the best when trained with DPD-ΠMBST while GRU performed the best when trained with DPD-BST, both of which are significantly better than all baselines across all metrics.
Interestingly, while Kim et al. (2023) finds that a supervised Transformer model outperforms Meloni et al. (2021)’s GRU model on Rom-phon but not WikiHan, we observe the opposite in a 10% labeled semisupervised reconstruction setup, with Transformer outperforming GRU on WikiHan but not Rom-phon under most strategies. This appears to contradict Kim et al. (2023)’s hypothesis that a Transformer reconstruction model requires more data compared to an RNN.
Consistent with our hypothesis that access to different subsets of labels affects learning outcomes, we observe high variations in performance between dataset groups. Figures 6 and 7 visualize the performance of strategy-architecture combination by group, revealing that most strategies tend to do better on some dataset seeds. Performance for varied labeling settings: Despite only being tuned with 10% of labels, DPDbased strategies generalized to other labeling settings, often outperforming strong baseline strategies on all metrics on at least one of the architectures (See Appendix F for detail). We see a nonlinear scaling between performance and the percentage of labels and notice higher performance variations between strategies for lower percentages of labels. At a 5% labeling setting, for example, GRU-DPD-ΠM-BST almost doubles the accuracy of GRU-SUPV on WikiHan. At a 30% labeling setting, some strategies attain accuracy close to existing fully supervised reconstruction models, with Trans-DPD-ΠM 5.14 percentage points behind Meloni et al. (2021)’s supervised GRU on WikiHan (Chang et al., 2022) and GRU-DPD-BST 7.30 percentage points behind Kim et al. (2023)’s
supervised Transformer on Rom-phon (See Table 15 and Appendix J for detail).

heading: 4 Analysis

heading: 4.1 DPD Training
text: We investigate the interaction between D2P and P2D during training. Figure 3 shows the train and validation accuracy trajectories of reflexes reconstructed from the intermediate representation of protoform reconstructions. We perform a CRINGE loss ablation by repeating the same run but with CRINGE loss disabled (same hyperparameter and model parameter initialization). In both cases, reflexes are more accurately predicted from correct protoform reconstructions, corroborating our motivation that better reflex reconstruction should promote better protoform reconstruction. CRINGE loss appears to lead to slightly lower reflex accuracy given incorrect reconstruction at earlier epochs, albeit at the expense of slightly lower reflex accuracy given correct reconstruction. In practice, we find the CRINGE loss weight to be a relatively insignificant hyperparameter.

heading: 4.2 Reflex Prediction Performance
text: While the DPD architecture is designed for reflex prediction-assisted reconstruction, we observe good reflex performance in some situations. Table 3 shows the strategy-architecture combinations with the most accurate P2D sub-network when evaluated on gold protoforms8. Compared to reconstruction, we obtain semisupervised reflex prediction performance that are much closer to supervised performance, even with small percentages of labels—consistent with the assumption that sound changes in the proto-to-daughters direction are easier to model.
It is worth noting, however, that reflex prediction performance based on gold protoforms depends on the weight of the corresponding loss. In fact, some of the best DPD models perform poorly
8It is arguably more interesting to evaluate reflex prediction performance based on the model’s latent protoform representation. Unfortunately, obtaining latent representations of correct protoforms is not always possible.
when the P2D sub-network is evaluated on gold protoforms. We conclude that P2D’s ability to assist D2P during training is not contingent on P2D’s performance on gold protoforms as discrete input.

heading: 4.3 Learned Phonetic Representations
text: Inspired by Meloni et al. (2021), we probe the model’s learned embeddings for a hierarchical organization of phonemes using sklearn’s Ward variance minimization algorithm (Ward, 1963). Figure 4 shows the results for two selected daughter languages on the most accurate model in each dataset (GRU-DPD-BST for Rom-phon and TransDPD-ΠM-BST for WikiHan) and the best model from their non-DPD counterpart (GRU-BST and Trans-ΠM-BST respectively).
For French, phoneme embeddings trained with DPD-BST reveals a clear division between vowels and consonants similar to Meloni et al. (2021)’s supervised reconstruction model. Except for [ø], nasal vowels are grouped together. Specific
phoneme pairs with minimal difference in features are also placed together, such as the alveolar fricatives [s] and [z], post-alveolar fricatives [ʃ] and [ʒ], and velar plosives [k] and [g], and all of which only differ in voicing. In the embeddings trained with BST, some but not all vowels are clustered together, and sister groups (i.e. immediate relative in the tree) are less interpretable, such as [m] and [ɛ]̃ as well as [b] and [ә]. For Cantonese, we see a similar pattern where vowels and consonants have a clearer division when trained with DPDΠM-BST. Additionally, tones are organized into the same cluster by DPD-ΠM-BST but not by ΠMBST. In Appendix N, we extend our probe to set of all phonemes present in daughter languages9 and find that the above observations generalize to phoneme organization beyond a single language, which Meloni et al. (2021) does not consider as part of their analysis.
We conclude from phonetic probing that DPDbased strategies are better at capturing linguistically meaningful representations of phonemes. It is possible that DPD-based strategies need good phonetic representations to perform well on multiple phonology-intensive tasks, which could in turn better inform protoform reconstruction.

heading: 4.4 Ablation on Unlabeled Data
text: To study whether the performance gains of semisupervised strategies are because of their effective use of unlabeled cognate sets, we perform ablation experiments at a 10% labeling setting but with all
9Phonemes present only in the protolanguage are not included because non-DPD do not update their embeddings.
unlabeled training data removed10, effectively creating a small supervised training set11.
We find that, in the absence of unlabeled data, ΠM, DPD, and DPD-ΠM can sometimes perform significantly better than SUPV, but almost always perform significantly worse than when unlabeled data is used or when unlabeled data is used in conjunctionwith Bootstrapping (see Tables 13 and 14). This seems to suggest that ΠM, DPD, and DPDΠM learn effectively from both labeled and unlabeled data. It is possible that, on labeled data, the P2D sub-network in DPD can still inform the D2P sub-subnetwork, and the stochastic data augmentation in our implementation of Π-model can augment labeled training examples.

heading: 4.5 Applicability in Supervised Reconstruction
text: Seeing some indication that the semisupervised reconstruction strategies are applicable for supervised reconstruction on a small subset of the training set (see Section 4.4), we test whether their advantages generalize to supervised reconstruction on the full training set12.
Table 15 compares the supervised reconstruction performance of ΠM, DPD, and DPD-ΠM with existing supervised reconstruction methods, including Meloni et al. (2021)’s GRU model, Kim et al. (2023)’s Transformer model, and Lu et al. (2024)’s state-of-the-art reranked reconstruction systems. We find that, with the right architecture,
10We exclude strategies with Bootstrapping because it has no effect when there is no unlabeled data.
11See Appendix I for experimental details. 12See Appendix J for experimental details.
ΠM, DPD, and DPD-ΠM can often outperform SUPV. On average, Trans-DPD-ΠM performs the best on WikiHan for all metrics, and GRU-DPD performs the best on Rom-phon for all metrics except FER. OnWikiHan, Trans-DPD-ΠM performs significantly better than Lu et al. (2024) on FER only. On Rom-phon, GRU-DPD and GRU-DPDΠM both perform significantly better than Lu et al. (2024) on ACC only. We conclude that, despite being motivated by semisupervised reconstruction, ΠM and DPD could be useful for supervised reconstruction. We leave it to future work to understand the role of data augmentation and the DPD architecture in supervised settings.

heading: 5 Related Work
text: Computational Historical Linguistics: Protoform reconstruction and reflex prediction are two central tasks in computational historical linguistics. Protoform reconstruction predicts the protoform given reflexes in a cognate set, while reflex prediction models the changes from the protoform to its reflexes. Computational reconstruction and reflex prediction methods vary and include rule-based systems (Heeringa and Joseph, 2007; Marr and Mortensen, 2020, 2023), probabilistic models operating on phylogenetic trees (BouchardCôté et al., 2007b,a, 2009, 2013), automated alignment systems (Bodt and List, 2022; List et al., 2022), and more recently, neural networks (Fourrier, 2022).
Supervised Reconstruction and reflex prediction: Ciobanu and Dinu (2018) and Ciobanu et al. (2020) formulate protoform reconstruction as a sequence labelling task and use conditional random field to reconstruct protoform phonemes at each position in the daughter sequences. Meloni et al. (2021) reformulates the reconstruction task as a sequence-to-sequence task by concatenating reflexes into a single input sequence separated by language tags and uses GRU to reconstruct Latin on a new Romance dataset. A group of subsequent researchers refined this task with additional datasets (Chang et al., 2022) and improved neural methods (Kim et al., 2023; Akavarapu and Bhattacharya, 2023; Lu et al., 2024).
Reflex prediction can be viewed as sequenceto-sequence transduction in the reverse direction. Cathcart and Rama (2020) propose an LSTM encoder-decoder model to infer Indo-Aryan reflexes from Old Indo-Aryan, aided by semantic
embedding. Arora et al. (2023) replicates Cathcart and Rama (2020)’s experiments on South Asia languages with both GRU and Transformer models. As reflex prediction maps from protolanguage to multiple daughter languages, a prompting token is often attached to the input to specify the target daughter language.
Non-Supervised Computational Historical Linguistics: Past work in which comparative reconstruction was performed without full supervision included Bouchard-Côté et al. (2009) and He et al. (2023). To the best of our knowledge, no research focuses specifically on semisupervised neural reconstruction.
Semisupervised Learning: Effective semisupervised learning should utilize unlabeled training data. One approach is proxy-labelling, by which synthesized labels are added to unlabeled training examples via heuristics (Ouali et al., 2020). Another approach is consistency regularisation, largely based on the smoothness assumption that is applicable regardless of whether labels are present: similar input data in high-density regions should have similar labels, whereas input data separated by low-density regions should not (Tsai and Lin, 2019; Ouali et al., 2020; Luo et al., 2018).
Our task is related to semisupervised machine translation (Edunov et al., 2018; Sennrich et al., 2016; Skorokhodov et al., 2018; Gülçehre et al., 2015; Cheng et al., 2016). However, it differs crucially in that the model has no access to monolingual text, only labeled and unlabeled cognate sets.

heading: 6 Conclusion
text: We introduce the task of semisupervised reconstruction, marking a step forward toward a practical computational reconstruction system that can assist early-stage protolanguage reconstruction projects. We design the DPD-BiReconstructor architecture to implement historical linguists’ comparative method, yielding performance that surpasses existing sequence-to-sequence reconstruction models and established semisupervised learning techniques, especially when protoform labels are scarce.
Limitations
Due to a large number of possible strategies, we have limited our focus of semisupervised reconstruction experiments with DPD to 10% labeled
WikiHan andRom-phon datasets. It is left to future work to expand the research on semisupervised reconstruction to other datasets.
Though DPD has a clear motivation and demonstrates superior empirical performance, interpretations of what sound changes DPD learns from the unlabeled cognate sets that enable its better reconstruction performance are less clear. Our theory is that neural networks within the system are sufficiently expressive to learn better reconstruction in a bidirectional manner, but we have not yet obtained evidence that the model’s reasoning matches that of a linguist beyond having a better representation of phonemes and taking the step to derive the reflexes from the reconstruction. Nevertheless, we demonstrate that inferring reflexes from reconstructions is not just a powerful methodology for historical linguistics, but also for computational historical linguistics.
Although we observe less accurate reflexes being predicted from incorrect protoforms compared to correct protoforms, reflexes derived from incorrect protoforms are still highly accurate. Future work could explore ways to mitigate this issue and improve the reflex prediction sub-network’s ability to discriminate between correct and incorrect protoforms.
It is also notable that our implementation of the Π-Model included only reflex permutation and daughter deletion as noising strategies. It is possible that other possible strategiesmay have strengthened this baseline.
The protoform reconstruction task is far from solved—with humans’ success at the reconstruction of ancient languages, a truly intelligent reconstruction system should in the future be able to perform reconstruction without the help of labels.
Ethics
Historical reconstruction involves very limited risks to humans. The risks that do exist are both individual and political. On the one hand, we cannot guarantee that all of the data used in this study were collected in an ethical fashion. However, we did not do any data collection and relied upon existing resources which, to the best of our knowledge, were collected under standard scholarly and scientific protocols. On the other hand, the results of historical reconstruction can be politically fraught. For example, historical reconstructions can be used to show, in some cases, that linguis-
tic boundaries between people groups do not align with cultural and political loyalties. This can be disruptive and may even be associated with violence. Because our work does not concentrate directly on phylogeny—the main source of political complications in comparative reconstruction—we believe that the risks are minimal.

heading: Acknowledgements
text: This work is supported by Carnegie Mellon University’s SURF grant. We would like to thank ChenxuanCui, Kalvin Chang, andGrahamNeubig for helpful ideas and discussions. We are grateful to our anonymous reviewers for many comments that helped improve the writing and informed additional experiments and analyses.

heading: A Dataset Details
text: Both WikiHan and Rom-phon are split by 70%, 10%, and 20% into train, validation, and test sets. We remove labels from the supervised train set to create semisupervised train sets. The validation and test sets are left unmodified. The splits for WikiHan (Chang et al., 2022) match the original work. The splits and preprocessing for Meloni et al. (2021) and matches Kim et al. (2023). Table 4 shows the number of cognate sets in each split. WikiHan includes 8 daughter languages: Cantonese, Gan, Hakka, Jin, Mandarin, Hokkien, Wu, and Xiang. Rom-phon includes 5 daughter languages: French, Italian, Spanish, Romanian, and Portuguese.

heading: B Dataset Groups
text: Semisupervised reconstruction datasets are generated pseudo-randomly based on a seed, which is
itself chosen randomly. That is, a dataset seed deterministically generates a semisupervised dataset. To generate a semisupervised dataset, we initialize PyTorch’s pseudo-random number generator with the dataset seed, assign a uniformly distributed number between 0 and 1 to each training example with torch.rand in the same order as they appear in the dataset, and keep the protoform label on cognate sets whose assigned number is above a threshold such that the desired percentage of labels remain. The dataset seed is independent of seeds used to initialize model parameters in the experiments. Table 5 details the dataset seed used to select the subset of training labels to include for each experiment setup, and Table 6 details the number of labels at each labeling setting. We use group 1 for comparisons between labeling settings, with the same dataset seed ensuring label sets are monotonic subsets with respect to labeling settings with increasing percentages of labels. For the 10% labeled setting, four distinct dataset seeds simulate variations in the training data.

heading: C Hyperparameters
text: We tune hyperparameters using Bayesian search on WandB (Biewald, 2020) with 100 runs for each strategy-architecture combination. We validate the model every 3 epochs and use early stopping if no improvement is made after 24 epochs. The dataset used for tuning semisupervised models is a 10% labeled train set generated using 0 as the dataset seed, making it different from the semisupervised datasets used in the experiments. Hyperparameters for semisupervised models can be found in Tables 16 and 17. Additional hyperparameter tunings for analysis purposes (Sections I and J) follow the same procedure but remove unlabeled training data or use a different labeling setting.

heading: D Training
text: Parameter counts for the models can be found in Table 22. Hyperparameter tuning and experiments
are performed on a mix of NVIDIA GeForce GTX 1080 Ti, NVIDIAGeForce RTX 2080 Ti, NVIDIA RTX 6000 Ada Generation, NVIDIA RTX A6000, Quadro RTX 8000, and Tesla V100-SXM2-32GB GPUs for a total of 411 GPU days.

heading: E Package Usage
text: Our model is implemented in PyTorch (See the code for details at https://github.com/ cmu-llab/dpd). Sequence alignment is done using lingpy (List and Forkel, 2021) with default parameters. Hierarchical clustering is done using AgglomerativeClustering from sklearn with Ward linkage and distance threshold set to 0. Bootstrap tests are done using scipy with random state set to 0, a 99% confidence interval, and 9,999 resamples (default). Wilcoxon Rank-Sum tests are done using scipy at default parameter. Hierarchical clustering is visualized using scipy. Plots are created using Matplotlib.

heading: F Additional Performance Data
text: Tables 8 and 9 show the performance of each strategy on group 1 for the 5%, 20%, and 30% labeling settings, along with indicators of statistical significance.

heading: G Transductive Evaluation
text: Evaluation for semisupervised learning can be categorized as transductive and inductive (Ouali et al., 2020). Inductive evaluation tests the model’s performance on unseen data in a test set, whereas transductive evaluation tests the model’s ability to predict labels for the unlabeled data in the train set. In the context of semisupervised protoform reconstruction, transductive evaluation corresponds to the early stage of a real-world reconstruction project where cognate sets in higher abundance than known protoforms. In this section, we report the transductive performance of all the strategies.
We find that transductive performance differences between strategies are largely similar to industive performance on the test set. Significance as to whether our DPD strategies perform better than baseline is also similar to that of inductive evaluation on the test set. For detailed transductive performance and their statistical significance, see Tables 10 (10% labeled WikiHan and Rom-phon), 11 (5%, 20%, and 30% labeled WikiHan), and 12 (5%, 20%, and 30% labeled Rom-phon).
We observed no clear pattern as to which strategies perform better on transductive evaluation compared to inductive evaluation. On 10% labeled WikiHan, average transductive accuracies are all 0.0-1.14% higher than their test accuracies. On 10% labeled Rom-phon, transductive accuracies for all strategies are 1.20-1.90% worse than their test accuracies. Given that differences between test and transductive performance are relatively consistent across strategies, including for supervised strategies in which the unlabeled portion of the train set effectively acts as another test set, our hypothesis is that the fixed train-test split played a role in the evaluation.

heading: H Aligned Error Analysis
text: As an additional analysis, we align the protoform predictions and their targets on the test set using lingpy (List and Forkel, 2021) and identify errors made by reconstruction models. Consistent with Meloni et al. (2021)’s (supervised reconstruction) error analysis on Rom-phon, tense-lax errors occur most frequently for both the best DPD strategy and its non-DPD counterpart, with [i]/[ɪ], [e]/[ɛ], [o]/[ɔ], and [u]/[ʊ] being the top-four exchange er-
rors13. The top errors in WikiHan include inserting or deleting [j] and [w], vowel height exchange errors between pairs such as [e]/[i], [o]/[u], and [æ]/[a], tone errors, along with [a]/[o] errors. Exchange errors make up 68% and 63% of all errors for WikiHan and Rom-phon respectively.
We compare the average number of exchange errors between the best DPD-based strategy and their non-DPD counterpart. Figure 5 shows the most prominent exchange and non-exchange (i.e. insertion or deletion) error differences between the strategies. On average, GRU-DPD-BST makes 130 fewer mistakes than GRU-BST among 1731 test entries on Rom-phon, with a majority (72%) being insertion or deletion error reductions. TransDPD-ΠM-BST makes 144 fewer mistakes than Trans-ΠM-BST on WikiHan among 1033 test entries, with a majority (74%) being exchange error reductions. Some error differences between DPD and its non-DPD counterpart appear highly dependent on the dataset, such as GRU-DPD-BST making on average about 13 less [o]/[ɔ] errors on group 4 but more such errors on all other groups. Differences between groups could indicate that the distribution of additional unlabeled cognate sets in the training data plays an important role in the error patterns of DPD-BST strategies.
On Rom-phon, DPD is better at deciding whether to insert [m] in the reconstruction. This is the exact type of problem DPD is designed to handle: in situations where it is not apparent from the reflexes whether a phoneme absent in the reflexes should be added, it is often the case that one of the decisions will lead to lost information in the reconstruction. In DPD, the reflex prediction subnetwork should be able to detect such information
13The [i]/[ɪ] exchange error, for example, refers to [i] being predicted in place of [ɪ] or vice versa.
loss, which would otherwise lead to reflexes not being inferrable from the reconstruction.
We compare errors made by the best GRU-DPDBST against GRU-BST (within group 1 at 10% labeling setting). Among test examples where only one of GRU-DPD-BST and GRU-BST is correct, we see 1214 instances where GRU-DPD-BST produces the correct reconstruction but GRU-BST makes a mistake on [m] insertion or deletion. The opposite is true in only 2 instances15. Table 7 shows three test examples where GRU-DPD-BST performs better at deciding whether to add an [m] in the reconstruction as well as one example where it fails.
We hypothesize that since DPD makes better use of unlabeled data, an error is less likely to occur if there is an abundance of unlabeled examples involving information about the underlying
14excluding two false positives by lingpy 15excluding one false positive by lingpy
sound change pattern governing the context of a possible error. However, without explicit rules being learned by the models, it is difficult to assess what additional information is available to DPD in the unlabeled portion of the dataset and how the model learns from it. In an attempt to test this hypothesis, we estimate the abundance of learning resources in the unlabeled dataset by counting the number of examples involving the same sound correspondences (using lingpy Meloni et al. (2021)’s multi-sequence alignment) as a heuristic. We find no indication that count differences in exchange, insertion, or deletion errors are correlated with a higher abundance of training examples with the same sound correspondence in the unlabeled portion of the train set. Previous work has observed that statistical baselines can perform better than neural models at identifying the sound correct correspondences Fourrier and Sagot (2022), implying that sound correspondence is not necessarily a
proxy for sound change rules. It is likely that our DPD models learnt to use unlabeled cognate sets via means other than just sound correspondences.

heading: I Details on Unlabeled Data Ablation Experiments
text: We perform ablations on unlabeled training at a 10% labeling setting using the group 1 dataset seed, effectively keeping the same subset of labeled data as semisupervised group 1 experiments. This is equivalent to supervised reconstruction but with only 362 and 607 training examples for WikiHan and Rom-phon respectively.
We perform additional hyperparameter tuning for non-trivial semisupervised strategies16 to account for a large difference in dataset size. The hyperparameter we use are reported in Tables 18 and 19. We then perform 10 runs (random seed) for each strategy other than SUPV.
Table 13 compares the performance of various strategies after ablation, and Table 14 compares the performance with base strategies including ΠM, DPD, and DPD-ΠM under the configuration of whether to exclude unlabeled data, include unla-
16For SUPV, including unlabeled data has no effect, so we reuse hyperparameters and runs from semisupervised reconstruction experiments.
beled data, or include and pseudo-label unlabeled data (+BST, include unlabelled).
In many situations, the non-trivial semisupervised strategies still perform significantly better than SUPV despite the lack of unlabeled data. On WikiHan, DPD-ΠM performs the best on all metrics except Trans-DPD-ΠM on BCFS. On RomPhon, ΠM performs the best on all metrics. For strategies involving DPD but not ΠM, performance is always higher when unlabeled cognate sets are used. Interestingly, strategies involving ΠM sometimes perform worse when using unlabeled cognate sets on Rom-phon. In almost all situations, configurations that include and pseudolabel unlabeled cognate sets perform significantly better than when unlabeled data is excluded.

heading: J Details on 100% Labeled Supervised Reconstruction Experiments
text: Similar to the setup for unlabeled data ablation at 10% labeling setting (see Section I), we tune additional hyperparameters for ΠM, DPD, and DPDΠM to account for a difference in data. The hyperparameters we obtain are reported in Tables 20 and 21.
The SUPV strategy is equivalent to existing reconstruction models: GRU-SUPV is equivalent to
Meloni et al. (2021) and Trans-SUPV is equivalent Kim et al. (2023). Existing hyperparameters and checkpoints exist for SUPV: for WikiHan, we obtain 10 checkpoints from Kim et al. (2023) and 10 additional checkpoints from Lu et al. (2024); for Rom-phon, we obtain 20 checkpoints from Lu et al. (2024) (for hyperparameters, refer to Kim et al. (2023) and Lu et al. (2024)). For all other strategies, we perform 10 runs (random seed).
We compare the results of ΠM, DPD, and DPDΠM (both GRU and Transformer) against existing supervised reconstruction systems in Table 15. We use Meloni et al. (2021) and Kim et al. (2023) as SUPV baselines, and Lu et al. (2024)’s stateof-the-art supervised reconstruction systems (both GRU-BS + GRU Reranker and GRU-BS + Trans. Reranker) as strong baselines. We obtain evaluation results for 20 runs per setup from Lu et al. (2024).

heading: K Sample Outputs at Different Labeling Settings
text: We compare the protoform predictions of the best-performing strategy-architecture combination (best at a 10% labeling setting) trained on different labeling settings (with group 1 dataset seed)17. We present sample predictions stratified by the following categories:
• The predictions are correct at all % • The predictions are correct only above a certain % threshold
• The predictions are correct only below a certain % threshold
• The predictions are incorrect at all % • All other patterns Table 23 shows the distribution of test examples in these categories. Tables 25 and 24 show sample protoform predictions in each of these categories (proportionate sampling). For both datasets, correct only above a certain % threshold is the most common category when the predictions differ between labeling setting.

heading: L Sample Outputs from Different Strategies
text: We show sample predictions from the best performing runs in group 1. Tables 28 and 27 show predictions of the best-performing strategy-architecture
17At a 100% labeling setting, BST has no effect, so TransDPD-ΠM-BST is equivalent to Trans-DPD-ΠM and GRUDPD-BST is equivalent to GRU-DPD.
combination and its non-DPD counterpart for each dataset, proportionately sampled by the confusion matrices in Table 26. Tables 30 and 29 compares predictions across all 16 strategy-architecture pairs for 4 randomly selected test examples among those with the most diverse (approximately in the upper quartile18) protoform predictions.

heading: M Performance Data Visualizations
text: Figures 6 and 7 show the distribution of semisupervised reconstruction performance by group on 10% labeled datasets. Figures 8 and 9 visualize the group 1 performance of each strategy-architecture combination on datasets with different percentages of label.

heading: N Additional Phonetic Probing Results
text: A complete set of hierarchical clusterings of learned phoneme embeddings can be found in Figures 10 (Rom-phon) and 11 (WikiHan). Similar to French and Cantonese, phoneme embeddings learned using a DPD-based strategy also appear more interpretable for other languages. For instance, Trans-DPD-ΠM-BST creates a big cluster of tones within all phonemes present in daughter languages in WikiHan, and GRU-DPD-BST creates a cluster encompassing most palatalized consonants in Romanian.

heading: O DPD-ΠM Implementation
text: In a hybrid model combining the DPD architecture with Π-model, we first augment the input cognate set to obtain two augmented inputs A and B. Input A is used to train DPD as described in Figure 1. Input B is fed through D2P (and not P2D). A mean square difference loss then is used to minimize the difference between the D2P classifier logits given A and B as inputs.

heading: P Responsible Use of AI
text: GitHub Copilot has been used as a coding assistant for our model implementation and data analysis. All code generated by Github Copilot is checked manually.
18The precise cutoff is top 25.65% for WikiHan and top 24.91% for Rom-phon, accounting for ties and integer divisions.
Performance distribution by group for 10% labeled WikiHan
Performance distribution by group for 10% labeled Rom-phon
G R U -SU PV G R U -Π M G R U -B ST G R U -Π M -B ST G R U -D PD G R U -D PD -Π M G R U -D PD -B ST G R U -D PD -Π M -B ST Trans-SU PV Trans-Π M Trans-B ST Trans-Π M -B ST Trans-D PD Trans-D PD -Π M Trans-D PD -B ST Trans-D PD -Π M -B ST batch size 128 64 128 64 128 128 64 64 128 128 256 256 64 256 64 64 m ax epochs 221 247 238 253 345 268 384 253 205 288 374 341 261 256 390 259 w arm up epochs(learning rate) 29 12 22 16 37 10 2 27 26 6 17 26 18 7 31 22 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0009315 0.0007376 0.0008285 0.0005646 0.0007678 0.0009974 0.0008380 0.0008118 0.0007602 0.0005026 0.0009074 0.0008765 0.0007032 0.0009268 0.0006642 0.0008475 w eightdecay (A dam ) 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000e-07 0.000 1.000e-07 1.000e-07 1.000e-07 0.000 0.000 0.000 D PD protoform reconstruction w eight 1.100 0.7723 0.7696 0.5396 0.6270 0.5239 1.338 0.6952 D PD bridge netw ork w eight 0.3119 0.06223 0.6173 0.3858 0.5003 0.5839 0.1296 0.2182 D PD reflex prediction from gold protoform w eight 0.04734 0.5011 0.09115 0.6411 0.6265 0.4270 0.4811 0.2464 D PD reflex prediction from reconstruction w eight 1.498 1.249 0.9654 1.468 1.196 1.475 0.8612 1.322 D PD C R IN G E lossw eightα 0.7015 0.3820 0.5720 0.3732 0.3769 0.2332 0.6782 0.3056 D PD C R IN G E losstop k 3 4 1 3 1 4 3 3 D PD shared em bedding size 128 256 128 256 256 128 256 256 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 D 2P dropout 0.3117 0.2189 0.2336 0.3137 0.3228 0.2997 0.1999 0.2471 0.2539 0.1951 0.1949 0.1877 0.1594 0.2649 0.1574 0.1692 D 2P inference decode m ax length 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 D 2P feedforw ard dim ension 512 512 512 512 512 512 512 512 512 512 512 512 512 512 512 512 D 2P em bedding size 384 128 128 128 256 384 256 128 D 2P m odelsize 128 64 128 128 64 64 64 64 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 D 2P num berofheads 8 8 8 8 8 8 8 8 D 2P m ax inputlength 128 128 128 128 128 128 128 128 P2D encoderlayerscount 2 2 2 2 2 2 2 2 P2D dropout 0.1728 0.3136 0.3145 0.1546 0.3374 0.2532 0.1752 0.3339 P2D inference decode m ax length 15 15 15 15 15 15 15 15 P2D feedforw ard dim ension 512 512 512 512 512 512 512 512 P2D em bedding size P2D m odelsize 128 64 128 128 P2D encoderlayerscount 2 2 2 2 P2D num berofheads 8 8 8 8 P2D m ax inputlength 128 128 128 128 B ootstrapping starting epoch 38 7 29 24 16 4 25 11 B ootstrapping log probability threshold -0.006900 -0.008731 -0.007431 -0.005348 -0.002607 -0.009312 -0.003975 -0.007042 B ootstrapping m ax new pseudo-labelsperepoch 75 87 41 49 54 79 86 35 Π -m odelconsistency ram p-up epochs 1 11 8 23 24 8 25 27 Π -m odelm ax consistency scaling 382.0 222.1 275.1 181.7 223.3 184.5 207.5 88.70 Table 16:H yperparam etersforsem isupervised reconstruction experim entson W ikiH an.
G R U -SU PV G R U -Π M G R U -B ST G R U -Π M -B ST G R U -D PD G R U -D PD -Π M G R U -D PD -B ST G R U -D PD -Π M -B ST Trans-SU PV Trans-Π M Trans-B ST Trans-Π M -B ST Trans-D PD Trans-D PD -Π M Trans-D PD -B ST Trans-D PD -Π M -B ST batch size 256 256 64 64 256 256 128 128 256 128 256 128 256 256 128 128 m ax epochs 316 283 289 383 224 334 371 373 219 266 382 344 305 206 257 216 w arm up epochs(learning rate) 39 15 20 29 34 29 26 13 33 4 36 37 37 29 10 32 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0006089 0.0005475 0.0007060 0.0009183 0.0006284 0.0005706 0.0008698 0.0006257 0.0007210 0.0005128 0.0009196 0.0006014 0.0005052 0.0006181 0.0005023 0.0008986 w eightdecay (A dam ) 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000e-07 0.000 0.000 1.000e-07 0.000 1.000e-07 0.000 1.000e-07 D PD protoform reconstruction w eight 1.358 0.5163 0.6345 0.8579 0.6164 1.033 0.5132 0.5674 D PD bridge netw ork w eight 0.4293 0.3316 0.5736 0.5268 0.7455 0.4612 0.6321 0.5607 D PD reflex prediction from gold protoform w eight 0.6922 0.6284 0.1439 0.4346 0.2108 0.5989 0.3991 0.3401 D PD reflex prediction from reconstruction w eight 1.317 0.9957 1.476 0.9865 1.201 0.8703 1.294 1.421 D PD C R IN G E lossw eightα 0.02017 0.002749 0.5966 0.5339 0.03967 0.3295 0.6362 0.02023 D PD C R IN G E losstop k 5 1 5 5 5 2 4 1 D PD shared em bedding size 256 384 384 384 384 384 384 256 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 D 2P dropout 0.3342 0.2129 0.2680 0.2261 0.3300 0.1774 0.1970 0.2402 0.2838 0.3484 0.1963 0.2587 0.2667 0.3453 0.2627 0.2551 D 2P inference decode m ax length 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 D 2P feedforw ard dim ension 512 512 512 512 512 512 512 512 512 512 512 512 512 512 512 512 D 2P em bedding size 384 128 256 128 256 128 384 384 D 2P m odelsize 128 128 64 64 64 64 128 128 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 D 2P num berofheads 8 8 8 8 8 8 8 8 D 2P m ax inputlength 128 128 128 128 128 128 128 128 P2D encoderlayerscount 2 2 2 2 2 2 2 2 P2D dropout 0.2214 0.2429 0.3359 0.3346 0.3470 0.3168 0.2199 0.2197 P2D inference decode m ax length 30 30 30 30 30 30 30 30 P2D feedforw ard dim ension 512 512 512 512 512 512 512 512 P2D em bedding size P2D m odelsize 128 64 64 64 P2D encoderlayerscount 2 2 2 2 P2D num berofheads 8 8 8 8 P2D m ax inputlength 128 128 128 128 B ootstrapping starting epoch 37 20 28 16 4 28 39 39 B ootstrapping log probability threshold -0.005144 -0.009153 -0.003401 -0.005465 -0.004997 -0.003632 -0.001733 -0.007130 B ootstrapping m ax new pseudo-labelsperepoch 51 95 86 37 48 69 73 46 Π -m odelconsistency ram p-up epochs 18 14 29 12 2 1 23 18 Π -m odelm ax consistency scaling 166.8 168.4 198.2 211.6 393.6 248.6 301.3 192.2 Table 17:H yperparam etersforsem isupervised reconstruction experim entson R om -phon.
G R U -SU PV G R U -Π M G R U -D PD G R U -D PD -Π M Trans-SU PV Trans-Π M Trans-D PD Trans-D PD -Π M batch size 128 128 64 256 128 64 64 64 m ax epochs 221 206 323 363 205 383 217 363 w arm up epochs(learning rate) 29 17 5 11 26 26 35 37 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0009315 0.0008794 0.0005477 0.0009955 0.0007602 0.0009179 0.0008318 0.0009033 w eightdecay (A dam ) 0.000 0.000 0.000 0.000 1.000e-07 1.000e-07 0.000 1.000e-07 D PD protoform reconstruction w eight 1.195 0.6016 0.6045 0.5390 D PD bridge netw ork w eight 0.01908 0.2512 0.2167 0.3800 D PD reflex prediction from gold protoform w eight 0.4224 0.1871 0.1942 0.7438 D PD reflex prediction from reconstruction w eight 1.140 0.7006 1.326 1.089 D PD C R IN G E lossw eightα 0.4218 0.03425 0.1233 0.6634 D PD C R IN G E losstop k 2 5 1 2 D PD shared em bedding size 384 256 384 128 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 D 2P dropout 0.3117 0.3494 0.2859 0.2805 0.2539 0.3113 0.2902 0.2546 D 2P inference decode m ax length 15 15 15 15 15 15 15 15 D 2P feedforw ard dim ension 512 512 512 512 512 512 512 512 D 2P em bedding size 384 128 256 384 D 2P m odelsize 128 64 64 128 D 2P encoderlayerscount 2 2 2 2 D 2P num berofheads 8 8 8 8 D 2P m ax inputlength 128 128 128 128 P2D encoderlayerscount 2 2 2 2 P2D dropout 0.1934 0.2122 0.2663 0.2011 P2D inference decode m ax length 15 15 15 15 P2D feedforw ard dim ension 512 512 512 512 P2D em bedding size P2D m odelsize 64 128 P2D encoderlayerscount 2 2 P2D num berofheads 8 8 P2D m ax inputlength 128 128 B ootstrapping starting epoch B ootstrapping log probability threshold B ootstrapping m ax new pseudo-labelsperepoch Π -m odelconsistency ram p-up epochs 11 24 28 4 Π -m odelm ax consistency scaling 352.9 257.3 202.9 197.1
Table 18:A dditionalhyperparam etersfor10% labeled W ikiH an w hen unlabeled cognate setsare excluded.
G R U -SU PV G R U -Π M G R U -D PD G R U -D PD -Π M Trans-SU PV Trans-Π M Trans-D PD Trans-D PD -Π M batch size 256 256 128 128 256 64 64 256 m ax epochs 316 210 357 373 219 355 221 310 w arm up epochs(learning rate) 39 17 29 14 33 31 22 33 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0006089 0.0009945 0.0006927 0.0005954 0.0007210 0.0007154 0.0007520 0.0009266 w eightdecay (A dam ) 0.000 0.000 0.000 0.000 1.000e-07 0.000 0.000 0.000 D PD protoform reconstruction w eight 0.9617 0.8749 0.6145 0.7346 D PD bridge netw ork w eight 0.07922 0.5155 0.2638 0.4678 D PD reflex prediction from gold protoform w eight 0.2101 0.6491 0.5297 0.1037 D PD reflex prediction from reconstruction w eight 1.248 0.6504 0.9362 1.036 D PD C R IN G E lossw eightα 0.6470 0.2729 0.6543 0.1272 D PD C R IN G E losstop k 1 2 3 4 D PD shared em bedding size 384 384 384 384 D 2P encoderlayerscount 2 2 2 2 2 2 2 2 D 2P dropout 0.3342 0.2882 0.3309 0.2913 0.2838 0.3305 0.2617 0.3448 D 2P inference decode m ax length 30 30 30 30 30 30 30 30 D 2P feedforw ard dim ension 512 512 512 512 512 512 512 512 D 2P em bedding size 384 128 256 384 D 2P m odelsize 128 128 64 64 D 2P encoderlayerscount 2 2 2 2 D 2P num berofheads 8 8 8 8 D 2P m ax inputlength 128 128 128 128 P2D encoderlayerscount 2 2 2 2 P2D dropout 0.3284 0.2994 0.3333 0.2712 P2D inference decode m ax length 30 30 30 30 P2D feedforw ard dim ension 512 512 512 512 P2D em bedding size P2D m odelsize 64 128 P2D encoderlayerscount 2 2 P2D num berofheads 8 8 P2D m ax inputlength 128 128 B ootstrapping starting epoch B ootstrapping log probability threshold B ootstrapping m ax new pseudo-labelsperepoch Π -m odelconsistency ram p-up epochs 21 25 24 6 Π -m odelm ax consistency scaling 266.8 145.5 370.4 334.1
Table 19:A dditionalhyperparam etersfor10% labeled R om -phon w hen unlabeled cognate setsare excluded.
G R U -Π M G R U -D PD G R U -D PD -Π M Trans-Π M Trans-D PD Trans-D PD -Π M batch size 256 64 256 128 128 64 m ax epochs 283 269 275 382 232 315 w arm up epochs(learning rate) 4 7 4 22 40 22 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0006593 0.0005142 0.0008696 0.0007581 0.0007199 0.0007997 w eightdecay (A dam ) 0.000 0.000 0.000 0.000 0.000 1.000e-07 D PD protoform reconstruction w eight 0.8200 0.9314 0.5602 1.427 D PD bridge netw ork w eight 0.3285 0.5967 0.4405 0.5442 D PD reflex prediction from gold protoform w eight 0.5770 0.3550 0.5421 0.4134 D PD reflex prediction from reconstruction w eight 0.8717 0.6495 1.009 0.8214 D PD C R IN G E lossw eightα 0.3174 0.2638 0.001917 0.2851 D PD C R IN G E losstop k 5 3 4 4 D PD shared em bedding size 128 128 128 128 D 2P encoderlayerscount 2 2 2 2 2 2 D 2P dropout 0.2404 0.2346 0.1761 0.1685 0.2935 0.3055 D 2P inference decode m ax length 15 15 15 15 15 15 D 2P feedforw ard dim ension 512 512 512 512 512 512 D 2P em bedding size 128 256 D 2P m odelsize 128 64 128 D 2P encoderlayerscount 2 2 2 D 2P num berofheads 8 8 8 D 2P m ax inputlength 128 128 128 P2D encoderlayerscount 2 2 2 2 P2D dropout 0.1692 0.3488 0.3083 0.1804 P2D inference decode m ax length 15 15 15 15 P2D feedforw ard dim ension 512 512 512 512 P2D em bedding size P2D m odelsize 128 64 P2D encoderlayerscount 2 2 P2D num berofheads 8 8 P2D m ax inputlength 128 128 B ootstrapping starting epoch B ootstrapping log probability threshold B ootstrapping m ax new pseudo-labelsperepoch Π -m odelconsistency ram p-up epochs 4 22 14 14 Π -m odelm ax consistency scaling 257.4 156.4 200.2 142.0 Table 20:A dditionalhyperparam etersfor100% labeled W ikiH an.
G R U -Π M G R U -D PD G R U -D PD -Π M Trans-Π M Trans-D PD Trans-D PD -Π M batch size 256 128 64 64 64 128 m ax epochs 379 309 307 295 218 365 w arm up epochs(learning rate) 39 37 7 33 13 3 β 1 (A dam ) 0.9000 0.9000 0.9000 0.9000 0.9000 0.9000 β 2 (A dam ) 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 ε(A dam ) 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 1.000e-08 learning rate 0.0008681 0.0008735 0.0005797 0.0006855 0.0006586 0.0006777 w eightdecay (A dam ) 0.000 0.000 0.000 1.000e-07 0.000 1.000e-07 D PD protoform reconstruction w eight 0.5814 1.184 0.5771 1.473 D PD bridge netw ork w eight 0.7379 0.5496 0.5159 0.1906 D PD reflex prediction from gold protoform w eight 0.3473 0.2647 0.3375 0.5105 D PD reflex prediction from reconstruction w eight 1.147 0.5560 0.5103 1.405 D PD C R IN G E lossw eightα 0.1322 0.5651 0.4757 0.09542 D PD C R IN G E losstop k 4 4 5 1 D PD shared em bedding size 256 384 384 256 D 2P encoderlayerscount 2 2 2 2 2 2 D 2P dropout 0.2888 0.2923 0.3016 0.1566 0.2144 0.2418 D 2P inference decode m ax length 30 30 30 30 30 30 D 2P feedforw ard dim ension 512 512 512 512 512 512 D 2P em bedding size 256 128 D 2P m odelsize 128 128 128 D 2P encoderlayerscount 2 2 2 D 2P num berofheads 8 8 8 D 2P m ax inputlength 128 128 128 P2D encoderlayerscount 2 2 2 2 P2D dropout 0.1589 0.2117 0.1996 0.2921 P2D inference decode m ax length 30 30 30 30 P2D feedforw ard dim ension 512 512 512 512 P2D em bedding size P2D m odelsize 64 64 P2D encoderlayerscount 2 2 P2D num berofheads 8 8 P2D m ax inputlength 128 128 B ootstrapping starting epoch B ootstrapping log probability threshold B ootstrapping m ax new pseudo-labelsperepoch Π -m odelconsistency ram p-up epochs 9 15 30 29 Π -m odelm ax consistency scaling 83.25 92.97 67.85 64.89 Table 21:A dditionalhyperparam etersfor100% labeled R om -phon.
C antonese G an
H akka H okkien Jin M andarin W u
X iang M iddle C hinese ˆ M iddle C hinesenon-D PD ˆ M iddle C hineseD PD ˆ
C antoneseD PD ˆ
G an D PD ˆ H akka D PD ˆ H okkien D PD Ĵin D PD
ˆ M andarin D PD Ŵ u D PD ˆ X iang D PD
pɐi̯˨ pʰi˩˩
pi˥ pe˧
pi˦ pi˥
b̥i˨ pe̞i̯˦
bjej去 bjej去
bjej去 pɐi̯˨
pʰi˧ pi˥
pe˧ pi˦
pi˥ b̥i˨ pi˦ fɔːŋ˨ - - haŋ˥ - faŋ˧ - - bjw aŋ平 bjw aŋ平 bjw aŋ平 fɔːŋ˨ - - hɔŋ˨ - faŋ ˧ - - pʊk˨ - pʰuk˨ pɔk˥ pʰxaʔ˥ pʰu˧ b̥ʊʔ˩ pʰu˨ bukʷ入 bukʷ入 bu平 pʊk˨ - pʰuk˨ pʰɔk˥ pxu̯әʔ˥ pʰu ˧ b̥ʊʔ˩ pʰu ˨ paː˥ - pa˨ pe˧ - pa˥ pa̱˥ pa̠˨ bw a上 pa平 pæ平 paː˥ - pa˨ pe˥ - pa˥ pa̱˥ pa̠˧˧ pʰuːy̯˨ - pʰi˩ pu̯e˨ - pʰei̯˧ b̥e̞˨ - bw oj平 bw oj平 bw oj平 pʰuːy̯˨ - pʰi˩ pu̯e˨ - pʰei̯˧ b̥e̞˨ - pʰaːi̯˨ - - pai̯˥ - pʰai̯˧ - - bɛj平 boj平 bɛj平 pʰaːi̯˨ - - pe˨ - pʰei̯˧ - - taːn˧ - tan˥ tã˧ - tan˥ - - dan上 tan去 tan去 taːn˧ - tan˥ ta˧ - tan ˥ - - t ͡sʰɔːi̯˨ - t ͡sʰoi̯˩ t ͡sai̯˨ - t ͡sʰai̯˧ z̥e̞˨ - d͡zoj平 d͡zoj平 d͡zoj平 t͡sʰɔːi̯˨ - t ͡sʰa̯i˩ t͡sai̯˨ - t͡sʰai̯˧ z̥e̞˨ - t ͡sɔː˨ - t ͡sʰo˥ t ͡sɤ˧ - t ͡su̯ɔ˥ z̥v̩ʷ˨ - d͡zw a去 d͡zw a去 d͡zw a去 t ͡sʰɔː˨ - t͡sʰo˥ t͡sɤ˧ - t͡su̯ɔ˥ z̥v̩ʷ˨ - sɵy̯˨ - - su̯i˨ - ʈ ͡ʂʰu̯ei̯˧ - - d͡ʑw e平 d͡ʑw ij平 d͡ʑw ij平 sɵy̯˨ - - su̯i˨ - ʂu̯ei̯˧ - - jiː˨ - - i˨ - i˧ - - ji平 jij平 je平 jiː˨ - - i˨ - i˧ - - kʰɐi̯˥ - - ke˥ - t ͡ɕi˥ - - kej平 kej平 kej平 kʷɐi̯˥ - - ke˥ - t͡ɕi˥ - - kɐm ˥ t ͡ɕin˦ kim ˨ kin˥ t ͡ɕĩŋ˩˩ t ͡ɕin˥ t ͡ɕɪɲ˥ t ͡ɕin˧˧ kim平 kim平 kim平 kɐm ˥ t͡ɕin ˦ kim ˨ kim ˥ t͡ɕĩŋ˩˩ t͡ɕin˥ t͡ɕɪɲ ˥ t͡ɕin˧˧ kei̯˧ - ki˧ ku̯i˥ - t ͡ɕi˨ t ͡ɕi˧ - kjɨj上 kij上 kjɨj上 kʷei̯˧ - ki˧ ki˥ - t͡ɕi˨ t͡ɕi˧ - kuː˧ - - kɔ˥ - ku˨ - - ku上 ku上 ku去 kuː˧ - - kɔ˥ - ku˨ - - liːu̯˨ - - li̯au̯˨ - li̯oʊ̯ ˥ - - lew平 lew去 ljew平 liːu̯˨ - - li̯au̯˧ - li̯oʊ̯ ˥ - - liːp˨ - - li̯ap˥ - li̯ɛ˥ - - ljep入 lep入 ljep入 liːp˨ - - li̯ap˥ - li̯ɛ˥ - - lou̯˨ - - - - lu˧ lv̩ʷ˨ - lu平 lu平 lu平 lou̯˨ - - - - lu ˧ lv̩ʷ˨ - m ei̯˨ - - bi˧ - m ei̯˥ - - m ij去 m ij去 m je去 m ei̯˨ - - bi˧ - m ei̯˥ - - m iːn˩ - m i̯en˧ bi̯ɛn˥ - m i̯ɛn˨ m i˨ - m jen上 m jen上 m jen上 m iːn ˩ - m i̯en ˧ bi̯ɛn ˥ - m i̯ɛn˨ m i˨ - m ɐn˨ - - bun˧ - u̯әn˨ - - m jun去 m jun上 m jun上 m ɐn˩ - - bun˥ - u̯әn˨ - - m ou̯˨ u˦ m u˩ bu˨ vu˩˩ u˥ v̥v̩ʷ˨ u˧˧ m ju平 m u平 m u平 m ou̯˨ u˧ m u˩ bɔ˨ vu˩˩ u˧ v̥v̩ʷ˨ ɸ˩ m ʊŋ˨ - - bɔ˨ - m ɤŋ˧ - - m uŋʷ平 m uŋʷ平 m uŋʷ平 m ʊŋ˨ - - bɔŋ - m ɤŋ ˧ - - m aː˨ - m a˥ m e˧ - m a˥ m o˨ - m æ上 m æ去 m æ去 m aː˨ - m a˥ be˧ - m a˥ m o˨ - pɪŋ˥ - pen˨ pi̯әŋ˥ - piŋ˥ - - piŋ平 pjeŋ平 pjeŋ平 pɪŋ˥ - pin˨ pi̯әŋ˥ - piŋ˥ - - piːt˧ pʰi̯ɛt˥ pi̯et˨ piʔ˧ pi̯әʔ˨ pi̯ɛ˥ pi̯ɪʔ˥ pʰi̯e̞˨ pjet入 pʰet入 pʰjet入 piːt˧ pi̯ɛt˥ pi̯et˨ piʔ˧ pi̯әʔ˨ pi˥ pi̯ɪʔ˥ pi̯e̞˨ piːu̯˥ - - pi̯au̯˥ - pi̯au̯˥ - - pjew平 pjew平 pjew平 piːu̯˥ - - pi̯au̯˥ - pi̯au̯˥ - - paː˧ - pa˧ pe˥ - pa˨ - - pæ去 pæ上 pæ上 paː˧ - pa˧ pa˥ - pa˨ - - pʰiːu̯˥ - - - - pʰi̯au̯˥ pʰi̯ɔ˥ - pʰjew平 pʰjew平 pʰjew平 pʰiːu̯˥ - - - - pʰi̯au̯˥ pʰi̯ɔ˥ - pʰʊk˥ - pʰu˥ pʰɔk˧ - pʰu˥ pʰʊʔ˥ - pʰukʷ入 pʰukʷ入 pʰukʷ入 pʰʊk˥ - pʰuk pʰɔk˧ - pʰu˥ pʰʊʔ˥ - pɔːk˥ - pʰuk˨ pʰɔk˧ - pʰu˥ - - pʰukʷ入 bukʷ入 pʰu平 pʰɔːk˧ - pʰuk˨ pʰɔk˧ - pʰu˧ - - tʰaːi̯˧ - - tʰai̯˧ - tʰai̯˥ - - tʰaj去 tʰoj去 tʰoj去 tʰɔːi̯˧ - - tʰai̯˧ - tʰai̯˥ - - t ͡sʰɔːi̯˧ - t ͡sʰa̯i˧ t ͡sʰai̯˥ - t ͡sʰai̯˨ t ͡sʰe̞˧ - t ͡sʰoj上 ʈ ͡ʂʰæ j上 ʈ ͡ʂʰoj上 t͡sʰɔːi̯˧ - t͡sʰa̯i˧ t͡sʰai̯˥ - t͡sʰai̯˨ t͡sʰe̞˧ - t ͡siː˧ - - - - ʈ ͡ʂz̩˥ t ͡sz̩˧ - t ͡ɕi去 t ͡ɕe去 t ͡ɕe去 t͡siː˧ - - - - t ͡sz̩˥ t͡sz̩˧ - t ͡sɐu̯˥ t ͡sɨ̯u˦ t ͡su˨ t ͡ɕi̯u˥ t ͡sәu̯˩˩ ʈ ͡ʂoʊ̯˥ t ͡sɜ˥ ʈ ͡ʂә̯u˧˧ t ͡ɕuw平 t͡ɕuw平 t͡ɕuw平 t͡sɐu̯˥ t͡sɨ̯u ˦ t͡su ˨ t͡ɕi̯u˥ t͡sәu̯˩˩ ʈ͡ʂoʊ̯˥ t͡sɜ˥ ʈ͡ʂә̯u˧˧ hei̯˥ - - hi˥ - ɕi˥ - - xi平 xij平 xje平 hɵy̯˥ - - hi˥ - ɕy˥ - - hœ ː˥ ɕy̯e˦ hi̯o˨ hi̯a˥ ɕy̯e˩˩ ɕyɤ˥ ɕi̯o˥ ɕy̯a̠˧˧ xjw a平 xja平 xja平 hœ ː˥ ɕi̯a˦ hi̯o˨ hi̯a˥ ɕi̯ɒ̃˩˩ ɕi̯aɤ˥ ɕi̯a̱˥ ɕi̯a̠˧˧ syːn˨ - - su̯an˧ - ɕyan˥ - - zjw en平 d͡ʑw an去 sw en去 syːn˨ - - su̯an˧ - ɕyan ˥ - - sɪk˥ sz̩it˥ sɨt˨ ɕi̯әk˧ sәʔ˨ ʂz̩˥ sәʔ˥ ʂʐ̩˦ ɕik入 ɕek入 ɕek入 sɪk˥ sz̩it˥ sɨt˨ ɕi̯әk˧ sәʔ˨ ʂz̩˥ sәʔ˥ ʂʐ̩˨ sɐu̯˧ - su˧ ɕi̯u˥ - ʂoʊ̯˨ - - ɕuw上 ɕuw上 ɕuw上 sɐu̯ ˧ - se̯u˧ ɕi̯u ˥ - ʂoʊ̯˨ - - t ͡sɪk˨ - tat˥ ti̯әk˥ - ʈ ͡ʂz̩˧ z̥әʔ˩ - ɖi去 ʈik入 ʈik入 t͡sɪk˨ - t ͡sɨk˨ ti̯әk˥ - ʈ ͡ʂz̩˥ z̥әʔ˩ - t ͡sʊk˨ - t ͡sʰuk˥ ti̯ɔk˥ - ʈ ͡ʂu˧ - - ɖjukʷ入 ʈjukʷ入 ɖjukʷ入 t͡sʊk˨ - t ͡sʰuk˨ ti̯ɔk˧ - ʈ͡ʂu ˧ - - hɔːn˩ - hon˨ ũ̯ã˨ xæ̃˦ xan˥ ɦø˨ - ɣan上 ɣan上 ɣan上 hɔːn˨ - hon˥ h˧ xæ̃˦ xan ˥ ɦø˨ - siː˥ sz̩˦ sɨ˨ sai̯˥ sz̩˩˩ ʂz̩˥ sz̩˥ sz̩˧˧ ʂij平 si平 ɕe平 siː˥ sz̩˦ sɨ˨ ɕu˥ sz̩˩˩ ʂz̩˥ sz̩˥ ʂz̩˧˧ siː˥ - - su˥ - ʂz̩˥ - - ʂje平 si平 ɕe平 siː˥ - - ɕu˥ - ʂz̩˥ - - t ͡siː˥ t ͡sz̩˦ ti˨ t ͡sai̯˥ t ͡sz̩˩˩ ʈ ͡ʂz̩˥ t ͡sz̩˥ ʈ ͡ʂʐ̩˧˧ ʈje平 t ͡ɕe平 t ͡ɕe平 t͡siː˥ t͡sz̩˦ t ͡sɨ˨ t ͡ɕe˥ t͡sz̩˩˩ ʈ͡ʂz̩˥ t͡sz̩˥ ʈ͡ʂʐ̩˧˧ t ͡syːt˧ - t ͡sʰot˨ t ͡sʰeʔ˧ - ʈ ͡ʂʰu̯ɔ˥ t ͡sʰʊʔ˥ - ʈjw ej去 t ͡sʰw at入 ɖjw et入 t ͡sʰyːt˧ - t ͡sʰut˨ t ͡sʰu̯aʔ˧ - ʈ͡ʂʰu̯ɔ˥ t ͡sʰәʔ˥ - t ͡sʰaː˥ - t ͡sʰa˨ t ͡sʰa˥ - ʈ ͡ʂʰa˥ t ͡sʰo˥ - ʈ ͡ʂʰje平 ʈ ͡ʂʰæ平 ʈ ͡ʂʰæ平 t͡sʰaː˥ - t͡sʰa˨ t͡sʰa˥ - ʈ͡ʂʰa˥ t͡sʰo˥ - ɔːi̯˥ ŋai̯˦ a̯i˨ ai̯˥ - ai̯˥ e̞˧ ŋai̯˩ ʔoj平 ʔaj平 ʔoj平 ɔːi̯˥ ŋai̯˦ oi̯˨ ai̯˥ - ai̯˥ e̞˥ ŋai̯˧˧
Table 27:Sam ple reconstruction predictionsby Trans-Π M -B ST (denoted D PD )and Trans-D PD -Π M -B ST (denoted non-D PD )and sam ple reflex prediction predictionsby TransD PD -Π M -B ST (w ith latentreconstruction asinput)forW ikiH an (proportionate sam pling according to the confusion m atrix in Table 26). ˆ L anguage:prediction forL anguage; bold:correctprediction;‘-’:the datasetdoesnotcontain thisreflex.
R om anian French
Italian Spanish
Portuguese L atin
ˆ L atin non-D PD
ˆ L atin D PD
ˆ R om anian D PD
ˆ French D PD
ˆ Italian D PD
ˆ Spanish D PD
ˆ PortugueseD PD - adikt addetto aðikto ɐditʊ addɪktʊm addɪktʊm addɪktʊm - adikt addetto aðikto ɐditʊ - apɑ̃dʁ appɛndere apendeɾ - appɛndɛrɛ appɛndɛrɛ appɛndɛrɛ - apɑ̃de appɛndere apendeɾ - ardɔaɾe aʁdœ ʁ ardore aɾðoɾ ɐɾәdoɹ ardorɛm ardorɛm ardɔrɛm ardoaɾe aʁdoʁ ardore aɾðoɾ ɐɾәdoɹ - - arpinate - - arpinatɛm arpɪnatɛm arpɪnatɛm - - arpinate - - - bɛstjɛʁ bestjario bestjaɾjo - bestɪarɪʊm bɛstɪarɪʊm bɛstɪarɪʊm - bɛstjɛʁ bestjario bestjaɾjo - bovin bovɛ̃ bovino boβino buvinʊ bɔw inʊm bɔw ɪnʊm bɔw ɪnʊm bovin bonɛ̃ bovino boβino buvinʊ - - - - dɨbɐkaɹ debakkʰari debakarɛ debakarɛ - - - - dɨbɐkaɹ debil - dɛbile deβil dɛbil debɪlɛm debɪlɛm debɪlɛm debile - dɛbile deβile dɛbilɨ - delɛk̃ɑ̃s delinkw ɛntsa delinkw ɛnθja dɨliŋkw eɪŋsjɐ delɪŋkw ɛntɪam delɪŋkw ɛntɪam delɪŋkw ɛntɪam - delɛk̃ɑ̃s delinkw ɛntsa delinkw ɛnθja dɨliŋkw eɪŋsjɐ - - dilidʒere - - dilɪɡɛrɛ dɪlɪdɛrɛ dɪlɪɡɛrɛ - - dilidʒere - - flw id flyid fluido fluiðo fluɪdʊ flʊɪdʊm fludʊm fluɪdʊm fluid flyid fluido fluiðo fluɪdʊ - fɛʁm әte - - - fɪrm ɪtatɛm fɛrm atɛrɛ fɛrm atɛm - fɛʁm әte - - - tʃeapә siv - - - kepam kɪw am kipam tʃeam ә siva - - - - kw adʁilatɛʁ kw adrilatero kw aðɾilateɾo - kw adrɪlatɛrʊm kw adrilatɛrʊm kw adrɪlatɛrʊm - kw adʁilatɛʁ kw adrilatero kw aðɾilateɾo - - - - - kɛɾulʊ kw ɛrʊlʊm kɛrʊlʊm kɛrʊlʊm - - - - kɛɾulʊ - - kom m izerare - - kɔm m ɪsɛrari kɔm ɪserarɛ kɔm m isɛrarɛ - - kom m izerare - - konsubstantsjal kɔ̃ sybstɑ̃sjɛl konsustantsjale konsustanθjal kuŋsubstɐ̃ŋsiɑl kɔnsʊbstantɪalɛm kɔnsʊstantɪalɛm kɔnsʊbtantɪalɛm konsubstantsjal kɔ̃sybstɑ̃tjɛl konsustantsjale konsustanθjal kuŋsubstɐ̃ŋsiɑl kum nat koɲa koɲato kuɲaðo - kɔŋnatʊm kʊɪatʊm kɔnatʊm kunnat kuɲa koɲato koɡato - labil labil labile laβil labil labɪlɛm labɪlɛm labɪlɛm labile labil labile laβile labilɨ - m itiʒe m itiɡare m itiɣaɾ - m itɪɡarɛ m itɪɡarɛ m ɪtɪɡarɛ - m itiɡe m itiɡare m itiɣaɾ - - - m endatʃe - m eɪŋdasɨ m ɛndakɛm m ɛndakɛm m ɛndakɛm - - m endatʃe - m eɪŋdaʃɨ nodos - nodozo nuðoso nudozʊ nodosʊm nɔdosʊm nɔdɔsʊm nodus - nodozo noðoso nuduzʊ notә nɔt nɔta nota nɔtɐ notam nɔtam nɔtam notә nom nɔta nota nɔtɐ oɾatsje oʁɛzɔ̃ oratsione oɾaθjon ɔɾɐsɐ̃ʊ̃ oratɪonɛm ɔratɪonɛm ɔratɪonɛm oɾatsje oʁasj oratsione oɾaθjon ɔɾɐsɐ̃ʊ̃ prefektuɾә pʁefɛktyʁ prefettura pɾefektuɾa pɹɨfeɪtuɾɐ praɪfɛkturam prɛfɛktʊram prɛfɛktʊram prefektuɾә pʁefɛktyʁ preffttura pɾefektuɾa pɹɨfeɪtuɾɐ - pʁesede pretʃedere pɾeθeðeɾ - praɪkedɛrɛ praɪkɛderɛ praɪkɛdɛrɛ - pʁesede pretʃedere pɾeθeðeɾ - protʃes pʁosɛ protʃɛsso pɾoθeso pɹusɛsʊ prokɛssʊm prokɛssʊm prokɛssʊm protʃes pʁosɛs protʃɛsso pɾoθeso pɹusɛsʊ puɾifika pyʁifje purifikare puɾifikaɾ puɾifikaɹ purɪfɪkarɛ purɪfɪkarɛ purɪfɪkarɛ puɾifika pyʁifie purifikare puɾifikaɾ puɾifikaɹ pekuliw pekyl pekulio pekuljo pekuliʊ pɛkulɪʊm pɛkʊlɪʊm pɛkʊlɪʊm pekuliw pekyl pekulio pekuljo pekuljʊ - - - puðoɾoso - pʊdorosʊm pʊdɔrosʊm pudorosʊm - - - puðoɾoso - - - - pultaθeo - pʊltɛm pʊltatɛʊm pʊltatɛʊm - - - pultad - - sanasjɔ̃ sanatsione sanaθjon - sanatɪonɛm sanatɪonɛm sanatɪonɛm - sanasjɔ̃ sanatsione sanaθjon - sartʃinә - sartʃina saɾθina - sarkɪnam sarkɪnam sarkinam sartʃinә - sartʃina saɾθina - - - spem e - - spɛm spɛm ɛm spɛm ɛm - - spem e - - - sɛʁvityd - seɾβiðum bɾe seɾәvidɐ̃ʊ̃ sɛrw ɪtudɪnɛm sɛrw ɪtʊm sɛrw ɪtʊm brɛm - sɛʁvitym - seɾβiðunbɾe seɾәvituʊ̃ - tʁɑ̃spɔʁte trasportare tɾanspoɾtaɾ tɹɐ̃ŋspuɾәtaɹ transpɔrtarɛ transportarɛ transpɔrtarɛ - tʁɑ̃spɔʁte trasportare tɾanspoɾtaɾ tɹɐ̃ŋspuɾәtaɹ tristetsә tʁistɛs tristetssa tɾisteθa tɹiʃtezɐ tristɪtɪam trɪstɛtam trɪstɛtɪam tristetsә tʁistɛt tristetssa tɾisteθj tɹiʃtɨsɐ tub tœ b tubo tuβo tubʊ tʊbʊm tʊbʊm tʊbʊm tub tym tubo tuβo tubʊ venjal venjɛl venjale benjal veniɑl w ɛnɪalɛm w ɛnɪalɛm w ɛnɪalɛm venjal venjal venjale benjal veniɑl - vɛʁtiʒinø vertidʒinozo beɾtixinoso veɾәtiʒinozʊ w ɛrtiɡɪnosʊm w ɛrtɪɡɪnosʊm w ɛrtɪɡɪnosʊm - vɛʁtiʒinø vertidʒinozo beɾtixinoso veɾәtiʒinozʊ vәduvә vøv vedova bjuða viuvɐ w ɪdʊam w ediw am w ɛdʊw am veduw ә vøv vuduvo bjuβo veuvʊ ivoɾiw ivw aʁ avɔrio eβoɾe - ɛbʊr ɪw w arɪʊm aw orɪʊm evɔɾek evw aʁ avɔrea eβoɾ - - ɛkspyʁʒe spurɡare ekspuɾɣaɾ - ɛkspʊrɡarɛ spʊrɡarɛ ɛkspʊrɡarɛ - ɛkspyʁɡe spurɡare ekspuɾɣaɾ - - ɛkssikasjɔ̃ essikkatsione - - ɛkssɪkkatɪonɛm ɛkssɪkatɪonɛm ɛkssɪkkatɪonɛm - ɛksiikasjɔ̃ essikkatsione - - episkopat episkopa episkopato episkopaðo epiʃkupadʊ ɛpiskɔpatʊm ɛpɪskɔpatʊm ɛpɪskɔpatʊm episkopat episkopa episkopato episkopaðo epiʃkupadʊ ɡladjum ɡlɛv ɡjado ɡlaðjo ɡladjʊ ɡladɪʊm ɡladɪʊm ɡladɪʊm ɡladw uә ɡlad ɡjadu ɡlaðw o ɡlɐduʊ ɡratsjos ɡʁasjø ɡratsiozo ɡɾaθjoso ɡɹɐsiozʊ ɡratɪosʊm ɡratɪosʊm ɡratɪosʊm ɡratsjos ɡʁasjø ɡratsiozo ɡɾaθjoso ɡɹɐsiozʊ - - dʒeom etria xeom etɾia - ɡɛom ɛtrɪam jɛɔm ɛtriam ɡɛɔm ɛtriam - - dʒeom etria xeom etɾia - - - inalbare - - ɪnalbarɛ ɪnalbarɛ ɪnalbarɛ - - inalbare - - - - indaɡare indaɣaɾ - ɪndaɡarɛ ɪndaɡarɛ ɪndaɡarɛ - - indaɡare indaɣaɾ -
Table 28:Sam ple reconstruction predictionsby G R U -D PD -B ST (denoted D PD )and G R U -B ST (denoted non-D PD )and sam ple reflex prediction predictionsby G R U -D PD -B ST (w ith latentreconstruction as input)forR om -phon (proportionate sam pling according to the confusion m atrix in Table 26). ˆ L anguage:prediction forL anguage;bold:correct prediction;‘-’:the datasetdoesnotcontain thisreflex.

heading: Future Work
text: It is left to future work to expand the research on semisupervised reconstruction to other datasets. Though DPD has a clear motivation and demonstrates superior empirical performance, interpretations of what sound changes DPD learns from the unlabeled cognate sets that enable its better reconstruction performance are less clear. Our theory is that neural networks within the system are sufficiently expressive to learn better reconstruction in a bidirectional manner, but we have not yet obtained evidence that the model’s reasoning matches that of a linguist beyond having a better representation of phonemes and taking the step to derive the reflexes from the reconstruction. Nevertheless, we demonstrate that inferring reflexes from reconstructions is not just a powerful methodology for historical linguistics, but also for computational historical linguistics. Although we observe less accurate reflexes being predicted from incorrect protoforms compared to correct protoforms, reflexes derived from incorrect protoforms are still highly accurate. Future work could explore ways to mitigate this issue and improve the reflex prediction sub-network’s ability to discriminate between correct and incorrect protoforms. It is also notable that our implementation of the Π-Model included only reflex permutation and daughter deletion as noising strategies. It is possible that other possible strategiesmay have strengthened this baseline. The protoform reconstruction task is far from solved—with humans’ success at the reconstruction of ancient languages, a truly intelligent reconstruction system should in the future be able to perform reconstruction without the help of labels