Title: Permutree Process
Venue: NeurIPS 2024
Accepted: no
Keywords: Bayesian nonparametrics

Permutree

Marked point process
Abstract: The permutree is an innovative mathematical concept recently introduced in the field of combinatorics. It encompasses permutations, trees, partitions, and binary sequences as its special cases, while also allowing for interpolations between them. In this paper, we present the permutree notion within the context of Bayesian machine learning. We exploit the fact that permutrees have a one-to-one correspondence with special permutations to propose a stochastic process on permutrees, and further propose two data modeling strategies analogous to the stick-breaking process and Chinese restaurant process that are frequently used in Bayesian nonparametrics. 
Permutations, trees, partitions, and binary sequences frequently appear as building blocks in Bayesian nonparametric models, and these models have been studied and developed independently. However, in practical situations, there are many complicated problems that require master craftsmanship to combine these individual models  into a  single giant model. Our models provide a framework for modeling such complicated tasks in a unified manner. As a significant application, we demonstrate the potential of our models for phylogenetic analysis of lineages, which involve coalescence, recombination, multiple ancestors, and mutation.

Abstract

1 Introduction

Various combinatorial structures -Permutations , trees , partitions , and binary sequences have been frequently utilized in Bayesian modeling, and conventionally, various models have been studied separately for each subject. Permutations have been used in a wide range of applications such as Bayesian ranking [101, 63, 110, 73], matrix reordering [70, 81, 99], and the traveling salesman problem [102, 17, 105]. Various random permutation models, such as the Mallows model [54, 12, 16], the permuton models [37, 7, 51, 6] and the modified Chinese restaurant process [57], have been employed in Bayesian modeling. Trees are typically used for hierarchical clustering [22, 21] and multiple resolution regression [47, 25, 20]. In the Bayesian literature, the Dirichlet diffusion tree [62, 45], the Mondrian process [88, 87] and the Pólya tree [56, 48, 27, 15] are particularly popular models. Partitions and binary sequences are fundamental tools in machine learning, with numerous examples of their usage in clustering, factor analysis, feature selection, and more. For the modeling of partitions and binary sequences, the Dirichlet process mixture model [23, 79, 100, 59], the Pitman-Yor process mixture model [76], the Chinese restaurant process [74], and the stick-breaking process [89] for random partitions, and the beta-Bernoulli process and the Indian buffet process [29, 95, 93] for random binary sequences have frequently been employed.

Combination of different combinatorial structures - In real-world applications of machine learning, it is often a useful strategy to combine several different combinatorial structures to model data, rather than using only one combinatorial structure. For example, the combination of partitioning and factor models is particularly popular, including the infinite factorial hidden Markov models [24, 97], the subset infinite relational model [39], the infinite latent factor model with the infinite mixuture model [111, 112] and the kernel beta process [83]. As another example, the combination of tree structures

and partitioning have also been actively studied, including the hierarchical Dirichlet process [94], the 39 nested Dirichlet process [84, 64], their hybrid models [2, 71, 50], the infinite context-free grammar 40 [49] and the tree-structured stick-breaking process [1, 65]. Furthermore, permutations are occasionally 41 employed in conjunction with clustering to analyze relational data [61]. As we have discussed so far, 42 this kind of strategy of combining multiple models into a single model is one promising direction for 43 research and development. However, advancing research in this direction necessitates the evaluation 44 of an enormous number of models in a combinatorial fashion, which becomes infeasible due to the 45 exponentially increasing number of potential combinations. Consequently, we are striving to initiate 46 a paradigm shift towards exploring an entirely new approach capable of unifying these models. 47

48

49

50

51

52

53

Key insight - In our pursuit of creating a unified model capable of encompassing permutations, trees, partitions, and binary sequences, we are incorporating the concept of permutrees [75], which has recently emerged in the field of combinatorics, into the realm of Bayesian nonparametric (BNP) machine learning. Permutrees not only serve as a framework that includes permutations, trees, partitions, and binary sequences as distinct cases but also exhibit intriguing properties of interpolation between them. Figure 1 (a)-(d) provides a concise visual representation of the key characteristics.

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

Our contributions - The main contribution of this paper is to produce, by using the concept of permutrees, a stochastic process that can represent combinatorial structures such as permutations, binary trees, partitions and binary sequences in a unified manner for the first time. Section 3 exploits the one-to-one correspondence between permutations and certain permutations using a two-dimensional marked point process to construct this process, which we call a permutree process . Section 4 derives a data modelling strategy using this stochastic process by analogy with the stickbreaking process that is frequently used in BNP machine learning. Section 5 demonstrates the application of phylogenetic analysis of DNA sequence data dealing with multiple biological events such as coalescence, recombination, mutation and multiple ancestry in a unified manner.

2 Preliminaries: Permutree and related objects

Permutree [75] - A permutree is a new mathematical tool invented recently in the field of combinatorics, which not only represent permutations, trees, partitions, and binary sequences as special cases, but can also interpolate between them [75]. Let us begin with the definition of a permutree. We consider a directed tree T with a vertex set V of n ( n ∈ N ) vertices of degree at least 2 , and a set of terminal nodes of degree 1 (See also Figure 1 (a)). For technical reasons (discussed immediately below), we dare to pay particular and explicit attention here to the set V of the 'interior vertices' (i.e., vertices of degree at least 2 ) other than the terminal nodes. Each vertex v ∈ V is assigned a natural number p ( v ) as a label, using the bijective vertex labeling (one-to-one correspondence) p : V → [ n ] := { 1 , 2 , . . . , n } based on the following permutree requirements (Definition 1 in [75]):

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

(C2) If a vertex v has a left parent (or child), then all labels in the subtree of the left ancestor (or descendant) of v are smaller than p ( v ) . If v has a right parent (or child), then all labels in the subtree of the right ancestor (or descendant) of v are greater than p ( v ) .

A directed tree T that satisfies the above requirements can be expressed more intuitively and clearly by introducing the notion of decorations to the vertices V . See also Figure 1 (e). We introduce the n -tuple decorations δ ( T ) := ( δ ( T ) 1 , . . . , δ ( T ) n ) ∈ { GLYPH<17> , GLYPH<15> , 6 , 7 } n , defined as follows: (i) δ ( T ) p ( v ) = GLYPH<17> if v has one parent and one child, (ii) δ ( T ) p ( v ) = GLYPH<15> if v has two parents and two children, (iii) δ ( T ) p ( v ) = 6 if v has one parent (lower in Figure 1 (e)) and two children (upper), and (iv) δ ( T ) p ( v ) = 7 if v has two parents (lower) and one child (upper). The symbolic feature of permutrees can represent various combination objects in a unified manner as follows:

Remark 2.1. (See Example 4 in [75].) Permutation - Permutrees with decoration GLYPH<17> n have a one-toone correspondence with permutations of [ n ] . For example, by reading the horizontal labels in the order of the natural number of vertical labels, Figure 1 (i) represents a permutation 436152 . Binary tree - Permutrees with decoration 7 n have a one-to-one correspondence with rooted planar binary trees on n vertices. See Figure 1 (j) for an example. Cambrian tree - Permutrees with decoration { 6 , 7 } n are exactly the Cambrian trees proposed in [82, 13]. See Figure 1 (k) for an example. Binary sequence - Permutrees with decoration GLYPH<15> n have a one-to-one correspondence with binary sequences with length n -1 . The i th element of the binary sequence is determined according to the following procedure: for any i ∈ [ n -1] , there exists p ( v ) = i and p ( w ) = i +1 , and if v is the parent of w , output 1 , otherwise output 0 . See Figure 1 (l) for 10010 as an example.

Now that we have summarized the important property of permutrees, we will describe the findings necessary to construct a stochastic process on a permutree, which is the main focus of this paper. As a motivation for describing the following findings, imagine actually drawing an instance of permutree on a hand-drawn blackboard. At this point, we notice that the horizontal positional relationship of vertices V is explicitly given by the natural number label p ( · ∈ V ) , however, the vertical positional relationship is still ambiguous (In Figure 1, (f) is identical to (g) in terms of the permutree, but distinct in terms of the increasing tree). Hence, in order to construct a stochastic process on permutrees in a concise and clear manner, a mechanism to control the vertical positioning of the vertices of the permutrees is required. With this motivation in mind, we introduce two useful notions, an increasing tree (Figure 1 (g)) and a leveled permutree (Figure 1 (h)).

Leveled permutree - To define the leveled permutree, we start by introducing an additional notion of an increasing tree . We consider a directed tree T with vertex set V . Each vertex v ∈ V is assigned a natural number label q ( v ) , using the bijective vertex labeling (one-to-one correspondence) q : V → [ n ] such that, if v ∈ V is the parent of w ∈ V , then q ( v ) < q ( w ) is satisfied. Intuitively, the function q serves to label the vertices V from 1 to n vertically from bottom to top (Figure 1 (g)). Then, a leveled permutree is a directed tree T with a vertex set V endowed with two bijective vertex labelings p, q : V → [ n ] which respectively define a permutree and an increasing tree. By using two types of labels p and q , the horizontal and vertical arrangement of the vertices V can be explicitly specified, as shown in Figure 1 (h). The leveled permutree is a useful tool when considering the generative model of the permutree in Section 3, because its specification is clear.

The notion of a leveled permutree so far has improved the prospects for dealing with permutrees. However, leveled permutrees are still combinatorial and geometric, and are not yet easy to handle computationally (in terms of Bayesian modeling, which is the main objective of this paper). Finally, we would like to wrap up this section by revealing one of the most important aspects of leveled permutrees: their relationship to decorated permutations .

Decorated permutation - For the description of decorated permutations, the notion of a permutation table should be prepared first. A permutation table is a geometrical representation of a permutation σ with n length by the ( n × n ) -table, with rows labeled by positions from bottom to top and columns labeled by values from left to right, and with a dot at column i and row σ ( i ) for all i ∈ [ n ] [9]. Figure 2 (left) shows an example for a permutation 536214 . Now that we are ready, we move on to the description of a decorated permutation. A decorated permutation is a permutation table where each dot is decorated by GLYPH<17> , GLYPH<15> , 6 , or 7 . Figure 2 (left bottom) shows an illustration of a decorated permutation. One of the important properties of decorated permutations is shown below.

Proposition 2.2. (See Proposition 8 in [75].) There exists one-to-one correspondence between decorated permutations with decorations ˆ δ ∈ { GLYPH<17> , GLYPH<15> , 6 , 7 } n and leveled permutrees with δ ( T ) = ˆ δ .

Now that we have reviewed the permutree findings, the next and subsequent sections will address 129 three challenges: (i) How can we construct a stochastic process that can represent any permutree (in 130 Section 3)? (ii) How can we construct a BNP prior model of the data using the stochastic process on 131 the permutree (in Section 4)? (iii) What likelihood models can we combine the BNP prior with in 132 actual machine learning applications (in Section 5)? 133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

3 Permutree processes

The goal of this section is to construct a stochastic process that can represent any permutree; ideally, as is the basic philosophy of BNP, that stochastic process should also be able to simultaneously represent randomness with respect to complexity (in the context of permutrees, the number of vertices). In fact, our construction below can represent every permutree with an unlimited number of finite or infinite number of vertices in a unified manner, depending on certain hyperparameters. One thing to note in advance is that the stochastic process described in this section does not refer to any modeling of data. We will discuss data modeling in more detail in the next Section 4.

Key insight - Our strategy is to use point processes. Recall that, as discussed in Section 2, permutrees can be represented through leveled permutrees (surjection), and furthermore, leveled permutrees have a one-to-one correspondence (bijection) with decorated permutations (Proposition 2.2). Thanks to these facts, instead of dealing directly with permutrees (seemingly difficult to handle), we can obtain a model of permutrees indirectly by considering a model of decorated permutations. So how can we model decorated permutations? We represent the random decorated permutations as a marked point process by considering random permutations as a point process and random decorations as marks .

Marked point process for decorated permutations - We consider a marked point process consisting of a point process and associated marks, which can be expressed as { ( l i , m i ) : i = 1 , 2 , . . . } , where l 1 , l 2 , . . . are locations and m 1 , m 2 , . . . are associated marks. Specifically, we employ the following Poisson process on a 2 -dimensional plane [0 , 1] × [0 , 1] with discrete marks (Figure 2 right):

Random locations - We draw the random locations l 1 , l 2 , . . . from a Poisson point process on the plane [0 , 1] × [0 , 1] with the intensity function λ : [0 , 1] × [0 , 1] → R + , where R + = { r : r > 0 , r ∈ R } . Although not essential, for the sake of simplicity, we use a homogeneous Poisson

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

point process, that is, λ ( A ) = µ · Leb ( A ) for all measurable subset A of [0 , 1] × [0 , 1] , where Leb ( · ) indicates the Lebesgue measure, and 0 < µ < ∞ is a tunable variable. For convenience, let l i = ( l i, 1 , l i, 2 ) , where l i, 1 and l i, 2 are the horizontal and vertical positions, respectively.

Random marks - We draw the random marks m 1 , m 2 , . . . , m n independently from a categorical distribution on { GLYPH<17> , GLYPH<15> , 6 , 7 } : Categorical ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) , where c ∗ ≥ 0 ( ∗ ∈ { GLYPH<17> , GLYPH<15> , 6 , 7 } ) denotes the probability that decoration ∗ is adopted.

Transformation to leveled permutree - The above marked point process can immediately lead to a random leveled permutree with the following procedure. Recall that, as discribed in Section 2, the leveled permutree is defined by (i) the decorations on the vertices V and (ii) the two bijective vertex labelings p, q : V → [ n ] . For the decoration of vertices, we consider the point set of the marked point process as the vertex set V , and the mark m i assigned to the i -th point as the decoration of the i -th vertex v i ∈ V . Thus, the remainder to be considered is the setting of two functions p and q . By construction, we can obtain the indices a 1 , . . . , a n so that the random positions l 1 , . . . , l n are in ascending order in the horizontal direction, that is, l a 1 , 1 < l a 2 , 1 < · · · < l a n , 1 (Recall that l i = ( l i, 1 , l i, 2 ) , and l i, 1 represents the horizontal position). Similarly, in the vertical direction, we can obtain the indices b 1 , . . . , b n so that l b 1 , 2 < l b 2 , 2 < · · · < l b n , 2 . Now, if we choose to set p ( v a i ) = i and q ( v b i ) = i for i = 1 , 2 , . . . , n , then p and q satisfy the requirement of bijective functions. By the above, we have seen that indeed the marked point process provides us with what we need to define a leveled permutree, that is, the vertex decorations and two bijective functions p and q . Finally, Figure 2 (right) show the procedure for explicitly converting a marked point process to a random leveled permutree. Inheriting Proposition 2.2 and the result (with the proof procedure) of [75, Proposition 8], we can confirm that this transformation is well defined (See Appendix E for details).

4 Data modeling with permutree process

The purpose of this section is to show how the permutree process described earlier can be used for modeling actual data. More specifically, this consists of the following two issues:

How to represent data using permutrees : As permutrees themselves are simply mathematical objects, we must be clear about how we relate them to data modeling and analysis. In fact, there are many possible ways to describe data by permutrees. We consider the situation where a data path (a lineage to describe the data in conjunction with some likelihood model, such as the evolutionary model in Section 5) from one of the lower terminal nodes to one of the upper terminal nodes on the permutree is assigned to each data (Figure 3 (a), top). For example, if we restrict the permutree to one of its special cases, the binary tree, this data path is attributed to the path from the root to the terminal node, which is a situation commonly used in hierarchical clustering (Figure 3 (a), bottom). 1 We show a strategy to represent this random data path using a special variant of the nested Chinese restaurant process [10].

Howto 'implement' a permutree process : In the previous section, we have shown that a marked point process with an intensity function λ : [0 , 1] × [0 , 1] → R + can be used for the stochastic process on permutrees. On the other hand, another important topic is to clarify how to implement models (or more practically, what intensity function λ to use) suitable for data analysis. Our strategy is to use the analogy of the stick-breaking process [89] to represent the infinite number of marked points generated from the marked point process, which is the entity of the permutree process. This can be viewed as a special case of using beta intensity in the horizontal direction and uniform intensity in the vertical direction as the intensity function λ of the permutree process.

Experts in the BNP field might remind themselves that there are many other strategy options for the above topics in the light of the various findings that have emerged in the history of the development of the BNP method over the last 20 years. We will, for the sake of space, summarize in Appendix D the various ideas and their respective advantages and disadvantages with respect to those historical findings, including whether it is possible to extend the conventional tool of the 'ordered' Chinese restaurant process [77, 85] for random binary trees to random permutrees. The main body of this paper focuses on the most straighforward strategy.

...

...

...

Permutree of infinite size - We first generate random positions l 1 , l 2 , . . . , l k = ( l k, 1 , l k, 2 ) , . . . in 206 the point process of the permutree process, as shown in Figure 3 (b), using the stick-breaking process: 207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

where α > 0 is the concentration parameter. As in the original permutree process, each point mark m k ( k = 1 , 2 , . . . ) is generated from a categorical distribution: m k ∼ Categorical ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) . As mentioned earlier, by the procedure in Figure 2 (right), we can transform this sample (i.e., a set of infinite number of marked points) drawn from the permutree process into a uniquely single permutree.

Data assignments to bottom terminal nodes - Next, we can represent data modeling by the paintbox scheme for the random permutree generated from the marked stick-breaking process described earlier. We associate one uniform random variable U j for each data indexed by j = 1 , 2 , . . . , N ( N ∈ N ) : U j ∼ Uniform([0 , 1]) . Similar to Kingman's representation to the exchangeable partitions, called paintbox schemes [44, 11], we choose which terminal node on the lower edge of the permutree to assign the j th data to, depending on which stick in the stick-breaking process this random variable U j is located on [0 , 1] , as shown in Figure 3 (b).

Data path modeling - Finally, we model the path assignment for each data by choosing a path that starts at this assigned lower terminal node and reaches one of the upper terminal nodes through the following two-table Chinese restaurant process (i.e., variant of two-class Dirichlet allocation):

6 - We break up the set of data flowing in, following the left-right table-assignement operation below 2 : the first data is chosen uniformly at random from either the left or the right table. For the n th data, the left table is chosen with probability ( N Left + γ/ 2) / ( n + γ ) and the right table with probability ( N Right + γ/ 2) / ( n + γ ) , where γ > 0 is a hyperparameter, and N Left and N Right are the number of data allocated so far to the left and right tables respectively.

- We merge the sets of data flowing from the two lower branches and feed them into the upper.

7 · GLYPH<15> - It would be straightforward to perform operations whose marks are 6 and 7 together. Another promising option is the representation of data flowing from the left parent to the left child and from the right parent to the right child. This can be interpreted as giving the mark GLYPH<15> the ability to partitioning . This interpretation also plays an important role in the validity of finite truncation , which will be discussed below.

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

GLYPH<17> - We pass on the whole data set that flows in, all the way to the top.

For notational simplicity, we will denote the random variable for the j th data path by Z j . For a sample z of data paths between the upper and lower terminal nodes of the permutree (specified by a sequence of edges), the above generative probabilistic model allows us to evaluate the probability P [ Z j = z ] of the j th data choosing a data path sample z .

Property #1: Exchangeability - Random data paths based on the generative probability model described above have exchangeability , an important property common to most BNP models [3, 36, 41]. Simply put, the model probability is invariant to the indexing of the data. As a result, it follows the philosophy of BNP models that even if the actual data to be observed is finite, the model itself, with infinite complexity, can reflect the uncertainty due to unobserved data. More specifically, this can be summarised as the following statement:

Proposition 4.1 (Exchangeability) . For any permutation σ of length N ( N ∈ N ) , we have P [ Z 1 = z 1 , Z 2 = z 2 , . . . , Z N = z N ] = P [ Z σ (1) = z 1 , Z σ (2) = z 2 , . . . , Z σ ( N ) = z N ] , where z j ( j ∈ [ N ]) is a sample of paths of random permutrees. (See Appendix A.1 for proof.)

Property #2: Validity of finite truncation - The above generative probability model requires in principle an infinite number of random variables for its description, but finite truncation works reasonably well for a finite number of actual observed data. This poses an inherently non-trivial challenge that is not present in the validity of approximating the stick-breaking process [89] for the Dirichlet process [23] with a finite number of stick-breaking procedures, which is a typical topic in the past [94, 87, 67]. The reason for this non-triviality is that the substructure of a permutree with infinite size is, in principle, affected by an infinite numnber of all marked vertices. Therefore, restricting the structure of the permutree to only some marked vertices may have a significant impact on the structure of the permutree. However, as the following statement shows, the substructure of the permutree has the good property that it depends only on a subset of marked vertices.

Proposition 4.2 (Finite truncation) . In the above generative probability model of data indexed by j = 1 , 2 , . . . , N ( N ∈ N ) , we consider an event that all random variables U j ( j = 1 , . . . , N ) , representing the horizontal position of the j th data, falls in the range [0 , 1 -ϵ ) as a situation with a sufficiently high probability P [ ∧ N j =1 0 ≤ U j < 1 -ϵ ] = ∏ N j =1 P [0 ≤ U j < 1 -ϵ ] > 1 -ϵ · O ( N ) , where ϵ > 0 is a tiny real value. In this situation, there exists some natural number K < ∞ , and all data paths are assigned with probability 1 only to paths on the finite-size random permutree generated from the random marked points l 1 , l 2 , . . . , l K . (See Appendix A.2 for proof.)

5 Application to phylogenetic permutree analysis

This section presents an application example of using the prior model representation of data using permutrees, which has been described in Section 4, in conjunction with a likelihood model in a specific application. One of the most promising applications of permutrees would be phylogenetic tree analysis for DNA molecular sequence data (e.g., CAGTC ). DNA sequences from one or more populations are related by a branching structure known as genealogy. The complex correlative structure of a collection of DNA sequences can be represented as a phylogenetic tree, a record of coalescence , recombination , and mutation events in the history of the target organism: coalescence refers to the event in which two sequences are attributed to a common ancestor, recombination refers to the event in which a lineage splits into two sub-lineages when looking back in time from the present to the past, and mutation refers to the change of each letter of a DNA sequence over time.

Challenges of conventional methods - The most standard structure that has been used in phylogenetic analysis is the binary tree [66, 78, 58, 103, 98, 113, 107, 106, 60]. In fact, binary trees are very well suited to represent coalescence events in genealogy. However, one drawback of binary tree models is that they are not suitable for representing recombination events in a way that is compatible with coalescence events. To circumvent this drawback, the ancestral recombination graphs (ARGs) have sometimes been used as models that can represent both coalescence and recombination at the same time [52, 42, 80, 72, 90, 28]. However, it is not easy to model or infer ARGs directly, and often indirect ways of representing models by other perspectives (e.g., the fragmentation-coagulation process [92, 19]) have been explored, or approximate models (e.g., the coalescent hidden Markov model [34, 53] and the sequentially Markov coalescent model [80]) have been considered. Moreover, conventional phylogenetic tree analysis, including not only ARGs but also binary tree models,

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

GAATCCTGTA

X

GCATATCGCT

X

AGTCCCGGTA

X

GCATCCGGTA

GGATCCGCTA

GGATCTGGAA GCGTCAGGTA

GCATCCGGTA

GGATCCGCTA

GGATCCGCTA

GGATCCGCTA

GCATCTGCTA

generally imposes a strong assumption that observed DNA sequences or observed taxa have a single ancestor. In other words, this implies that the inferred phylogenetic tree should be a strongly connected graph. Needless to say, such an assumption is reasonable for taxa that have been carefully selected by biologists. On the other hand, when we want to use a large number of taxa that are too large to be selected by experts as observation data (i.e., the situation that BNP methods are really aiming for), a mechanism that allows multiple ancestors to be inferred in a data-driven manner will be very useful. In light of the above, phylogenetic tree analysis requires a model that can represent coalescence, recombination, multiple ancestors, and mutation in a unified manner.

Phylogenetic permutree - As input observation data, we used DNA (molecular) sequences observed at letter length S over N species. For example, the sequence GAGTAC (i.e., N = 1 species) has length S = 6 . We regard these DNA sequences as following a phylogenetic permutree . Specifically, we represent coalescence, recombination, multiple ancestry, and mutation events in genealogy by combining the four types of the decorations 7 , 6 , GLYPH<15> , GLYPH<17> with the following interpretations. We note that, to be consistent with the traditional notation of phylogenetic tree analysis, the past (upper) to present (lower) direction as biological events is the opposite of the parent (lower) to child (upper) direction of the permutree as a purely mathematical object that we have used in the diagrams so far.

Coalescence 7 - A coalescence event represents two lineages (bottom side of Figure 4 (b)) having a common ancestral lineage (top side).

Recombination 6 - A recombination event represents the joining of two exclusive subsequences of two lineages (top side of Figure 4 (c)) by one lineage (bottom).

Partition GLYPH<15> - We give the decoration GLYPH<15> the role of division so that a single permutree can represent a phylogenetic tree with multiple ancestors. Specifically, as shown in Figure 4 (d), we connect the two left edges and connect the two right edges resulting in two tree structures unconnected to each other on either side of decoration GLYPH<15> .

Backward in time GLYPH<17> (optional) - We assume that no mutation occurs while going back in time from a vertex to a vertex with GLYPH<17> (Figure 4 (e)). This allows us to set the mutation rate in the evolutionary model as a single parameter common to all branches, and the mutation rate can be adjusted according to the permutree itself.

Evolutionary models on permutrees - Statistical models of gene mutation have a history of more than half a century, and a vast number of models have been proposed. An excellent recent review article can be found, for example, in [4]. For simplicity, we adopt two of the most popular models, the Jukes-Cantor model (JC) [40] and the generalized time reversible model (GTR) [91], for DNA sequences (i.e., words with A, G, C , and T as letters of the alphabet { A,G,C,T } , such as CCTAAG ). JC is defined as a Markov process in which (1) all letters are independently generated from a uniform categorical distribution on { A,G,C,T } as initialization and (2) one letter (e.g., A ) changes to another letter (e.g., G ) after t seconds with probability (1 -exp( -4 αt )) / 4 or does not mutate with probability (1 + 3 exp( -4 αt )) / 4 , where α ( > 0) is a hyperparameter representing the mutation rate. Simply put, JC means that the transition probabilities of letters in mutation are fixed. GTR, on the other hand, can be regarded as a more flexible version of the JC model, in which the letter transition probabilities themselves are also estimated from the data as hidden parameters.

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

RMPS

GTR

1

1.05

1.1

1.15

1.2

Perplexity

HDPHMM

FCP

MrBayes

ppHMC

ncSMC

MSBP-bTree

MSBP-cTree

MSBP-pTree

HWB

GTR

1

1.1

1.2

1.3

1.4

HDPHMM

FCP

MrBayes

ppHMC

ncSMC

MSBP-bTree

MSBP-cTree

MSBP-pTree

ZB

GTR

1

1.05

1.1

1.15

HDPHMM

FCP

MrBayes

ppHMC

ncSMC

MSBP-bTree

MSBP-cTree

MSBP-pTree

COMB

GTR

1

1.05

1.1

1.15

1.2

1.25

HDPHMM

FCP

MrBayes

ppHMC

ncSMC

MSBP-bTree

MSBP-cTree

MSBP-pTree

Demonstration - We use the following three benchmark datasets [60] for DNA sequences: RMPS ( N = 64 species, S = 1008 length) [86], HWB ( N = 41 , S = 1137 ) [33], and ZB ( N = 50 , S = 1133 ) [108]. In addition, to establish a situation where the permutree notion would be useful (i.e., multiple ancestry derived from exclusive disconnected graphs), we extract the sequences of these datasets by S = 1000 length from the beginning and mix them to create a dataset we call COMB ( N = 155 ). We use the marked stick-breaking process (referred to as MSBP; Section 4) as our proposed model. Since MSBP can easily adjust the representational capabilities of of its own model, as ablation studies, we use MSBP-bTree as the one restricted to binary trees (with the prior ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) ∼ Dirichlet( ϵ/ 2 , 0 , 0 , ϵ/ 2) ), MSBP-cTree as the one restricted to Cambrian trees (with Dirichlet( ϵ/ 3 , 0 , ϵ/ 3 , ϵ/ 3) ), and MSBP-pTree as the main proposal permutrees (with Dirichlet( ϵ/ 4 , ϵ/ 4 , ϵ/ 4 , ϵ/ 4) ), where we set ϵ = 0 . 01 . For the evolutionary model, we employ the mutation rate α ∼ Gamma( ϵ ′ , ϵ ′ ) , where ϵ ′ = 0 . 1 . We only present the case of K = 100 as the truncation level here, while we report the other cases in Appendix C. We compare these models to the hierarchical Dirichlet process hidden Markov model (HDPHMM) [8, 94, 5], the fragmentation-coagulation process (FCP) [92], and the binary tree model with the MrBayes [38, 46], the probabilistic path Hamilton Monte Carlo (ppHMC) [18], and the nested combinatorial sequential Monte Carlo (ncSMC) [60]. It is noted that HDPHMM and FCP do not use evolutionary models because they represent sequence data directly without tree structure. We held out 20% letters of the input sequences for testing, and each model was trained using the remaining 80% of the letters. Each inference method uses MCMC to estimate the posterior distribution by the following 100 samples: each method extracts 5 MCMCruns with different random numbers, and each MCMC run is sampled every 50 iterations after 2000 burn-in until 3000 iterations. We evaluate the models using perplexity as a criterion: perplexity( · ) = exp( -(log p ( · )) /E ) , where E is the number of missing letters in the input sequences. Figure 5 shows the comparison of the prediction performance of each method for the four sets of data. As an overall trend, it can be seen that the Cambrian tree and permutree models show better prediction performance than the binary tree model, which has limited expressive power.

6 Discussion and limitation

This paper (i) imports the notion of permutrees, recently invented in combinatorics, to Bayesian analysis, (ii) proposes the stochastic process that can represent various models such as permutations, trees, partitions, and factors in a unified manner, (iii) and applies it to phylogenetic permutree analysis.

Limitations - While our proposed permutree process can represent various combinatorial structures in a unified 'prior model,' the likelihood model that describes the data (as we have shown in the context of phylogenetic tree analysis in Section 5, for example) must be prepared separately by the user or engineer. Thus, while the permutree process is a tool that allows data-driven inference of the model structure as a broad framework, the design of the likelihood model needs to be carefully done manually. In the near future, the exploration of representing this likelihood model in some kind of black box function model would be an important research direction.

Remaining challenge - In the technical context of the BNP field, an important topic is whether a marginalized representation of the marked stick-breaking process, an infinite-dimensional intermediate random variable in the representation of data paths with exchangeability described in Section 4, can be obtained. This topic is a question closely related to the Aldous-Hoover-Kallenberg representation theorem for exchangeability in general [3, 36, 41]. As a more familiar analogy, it corresponds to the fact that if we marginalise the stick-breaking process representation in a Dirichlet process infinite mixture model, then we obtain the Chinese restaurant process representation. Our strategy and budding attempts on this question are summarized in Appendix D.

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

References

Ryan Prescott Adams, Zoubin Ghahramani, and Michael I. Jordan. Tree-structured stick breaking for hierarchical data. In Advances in Neural Information Processing Systems , pages 19-27, 2010.

Priyanka Agrawal, Lavanya Sita Tekumalla, and Indrajit Bhattacharya. Nested hierarchical Dirichlet process for nonparametric entity-topic analysis. In Machine Learning and Knowledge Discovery in Databases , volume 8189, pages 564-579, 2013.

David J. Aldous. Representations for partially exchangeable arrays of random variables. Journal of Multivariate Analysis , 11:581-598, 1981.

Miguel Arenas. Trends in substitution models of molecular evolution. Frontiers in Genetics , 6, 2015.

Marius Bartcus, Faicel Chamroukhi, and Hervé Glotin. Hierarchical Dirichlet process hidden Markov model for unsupervised bioacoustic analysis. In 2015 International Joint Conference on Neural Networks , pages 1-7. IEEE, 2015.

Frédérique Bassino, Mathilde Bouvel, Valentin Féray, Lucas Gerin, Mickaël Maazoun, and Adeline Pierrot. Universal limits of substitution-closed permutation classes. Journal of the European Mathematical Society , 22(11):3565-3639, 2019.

Frédérique Bassino, Mathilde Bouvel, Valentin Féray, Lucas Gerin, and Adeline Pierrot. The Brownian limit of separable permutations. The Annals of Probability , 46(4):2134--2189, 2018.

Matthew J. Beal, Zoubin Ghahramani, and Carl Edward Rasmussen. The infinite hidden Markov model. In Advances in Neural Information Processing Systems 14 , pages 577-584. MIT Press, 2001.

Anders Björner and Michelle L. Wachs. Permutation statistics and linear extensions of posets. Journal of Combinatorial Theory, Series A , 58(1):85-114, 1991.

D. M. Blei, M. I. Jordan, T. L. Griffiths, and J. B. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. pages 17-24, 2003.

Tamara Broderick, Jim Pitman, and Michael I. Jordan. Feature allocations, probability functions, and paintboxes. Bayesian Analysis , 8(4):801 - 836, 2013.

Róbert Busa-Fekete, Dimitris Fotakis, Balázs Szörényi, and Emmanouil Zampetakis. Identity testing for Mallows model. In Advances in Neural Information Processing Systems , pages 23179-23190, 2021.

Grégory Chatel and Vincent Pilaud. Cambrian Hopf algebras. arXiv:1411.3704 , 2014.

Anand Vir Singh Chauhan, Shivshankar Reddy, Maneet Singh, Karamjit Singh, and Tanmoy Bhowmik. Deviation-based marked temporal point process for marker prediction. In Machine Learning and Knowledge Discovery in Databases. Research Track , volume 12975, pages 289-304, 2021.

William Cipolli and Timothy Hanson. Supervised learning via smoothed Pólya trees. Advances in Data Analysis and Classification , 13(4):877-904, 2019.

Fabien Collas and Ekhine Irurozki. Concentric mixtures of Mallows models for top-k rankings: sampling and identifiability. In International Conference on Machine Learning , volume 139, pages 2079-2088, 2021.

Tiago Tiburcio da Silva, Antônio Augusto Chaves, Horacio Hideki Yanasse, and Henrique Pacca Loureiro Luna. The multicommodity traveling salesman problem with priority prizes: a mathematical model and metaheuristics. Computational and Applied Mathematics , 38(4), 2019.

Vu Dinh, Arman Bilge, Cheng Zhang, and Frederick A. Matsen IV. Probabilistic path hamiltonian monte carlo. In International Conference on Machine Learning , volume 70, pages 1009-1018, 2017.

Lloyd T. Elliott and Yee Whye Teh. Scalable imputation of genetic data with a discrete fragmentationcoagulation process. In Advances in Neural Information Processing Systems , pages 2861-2869, 2012.

Xuhui Fan, Bin Li, Ling Luo, and Scott A. Sisson. Bayesian nonparametric space partitions: A survey. In International Joint Conference on Artificial Intelligence , pages 4408-4415, 2021.

Xuhui Fan, Bin Li, and Scott A. Sisson. The binary space partitioning-tree process. In International Conference on Artificial Intelligence and Statistics , pages 1859-1867, 2018.

Xuhui Fan, Bin Li, Yi Wang, Yang Wang, and Fang Chen. The Ostomachion Process. In AAAI Conference 418 on Artificial Intelligence , pages 1547-1553, 2016. 419

Thomas Ferguson. Bayesian analysis of some nonparametric problems. Annals of Statistics , 2(1):209-230, 420 1973. 421

422

423

Jurgen Van Gael, Yee Whye Teh, and Zoubin Ghahramani. The infinite factorial hidden Markov model. In Advances in Neural Information Processing Systems , pages 1697-1704, 2008.

424

425

426

427

Shufei Ge, Shijia Wang, Yee Whye Teh, Liangliang Wang, and Lloyd Elliott. Random tessellation forests. In Advances in Neural Information Processing Systems , pages 9575-9585. 2019.

Weina Ge and Robert T. Collins. Marked point processes for crowd counting. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pages 2913-2920, 2009.

Bernhard Gittenberger and Veronika Kraus. The degree profile of random Pólya trees. Journal of 428 Combinatorial Theory, Series A , 119(7):1528-1557, 2012. 429

Robert C. Griffiths and Paul Marjoram. Ancestral inference from samples of DNA sequences with 430 recombination. Journal of Compututational Biology , 3(4):479-502, 1996. 431

Thomas L. Griffiths and Zoubin Ghahramani. Infinite latent feature models and the Indian buffet process. 432 In Advances in Neural Information Processing Systems , 2005. 433

Ruocheng Guo, Jundong Li, and Huan Liu. INITIATOR: noise-contrastive estimation for marked temporal 434 point process. In International Joint Conference on Artificial Intelligence, , pages 2191-2197. ijcai.org, 435 2018. 436

Tobias Hatt and Stefan Feuerriegel. Early detection of user exits from clickstream data: A Markov 437 modulated marked point process model. In WWW: The Web Conference , pages 1671-1681, 2020. 438

Xiaoyu He and Matthew Kwan. Universality of random permutations. arXiv:1911.12878 , 2019. 439

440

441

Daniel A. Henk, Alex Weir, and Meredith Blackwell. Laboulbeniopsis termitarius, an ectoparasite of termites newly recognized as a member of the laboulbeniomycetes. Mycologia , 95(4):561-564, 2003.

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

Asger Hobolth, Ole F Christensen, Thomas Mailund, and Mikkel H Schierup. Genomic relationships and speciation times of human, chimpanzee, and gorilla inferred from a coalescent hidden Markov model. PLOS Genetics , 3(2):1-11, 02 2007.

Sujun Hong and Hirotaka Hachiya. Multi-stream based marked point process. In Asian Conference on Machine Learning , volume 157, pages 1269-1284, 2021.

Douglas N. Hoover. Relations on probability spaces and arrays of random variables. Technical report, Institute of Advanced Study, Princeton, 1979.

Carlos Hoppen, Yoshiharu Kohayakawa, Carlos Gustavo Moreira, Balazs Rath, and Rudini Menezes Sampaio. Limits of permutation sequences. Journal of Combinatorial Theory, Series B , 103(1):93--113, 2013.

John P. Huelsenbeck and Fredrik Ronquist. MrBayes: bayesian inference of phylogenetic trees. Bioinform. , 17(8):754-755, 2001.

Katsuhiko Ishiguro, Naonori Ueda, and Hiroshi Sawada. Subset infinite relational models. In International Conference on Artificial Intelligence and Statistics , volume 22, pages 547-555, 2012.

Thomas H. Jukes and Charles R. Cantor. Chapter 24 - evolution of protein molecules. In Mammalian Protein Metabolism , pages 21-132. Academic Press, 1969.

Olav Kallenberg. Symmetries on random arrays and set-indexed processes. Journal of Theoretical Probability , 5(4):727-765, 1992.

Steven Kelk. Review of recombinatorics: The algorithmics of ancestral recombination graphs and explicit phylogenetic networks by dan gusfield. SIGACT News , 47(1):12-15, 2016.

Hideaki Kim, Tomoharu Iwata, Yasuhiro Fujiwara, and Naonori Ueda. Read the silence: Well-timed 462 recommendation via admixture marked point processes. In AAAI Conference on Artificial Intelligence , 463 pages 132-139, 2017. 464

John Frank Charles Kingman. The representation of partition structures. Journal of the London Mathe465 matical Society , s2-18(2):374-380, 1978. 466

David A. Knowles, Jurgen Van Gael, and Zoubin Ghahramani. Message passing algorithms for the 467 Dirichlet diffusion tree. In International Conference on Machine Learning , pages 721-728, 2011. 468

Lidia Kuan, Frederico Pratas, Leonel Sousa, and Pedro Tomás. MrBayes sMC 3 . International Journal of 469 High Performance Computing Applications , 32(2):246-265, 2018. 470

Balaji Lakshminarayanan, Daniel Roy, and Yee Whye Teh. Mondrian forests: Efficient online random 471 forests. In Advances in Neural Information Processing Systems , 06 2014. 472

Michael Lavine. Some Aspects of Pólya Tree Distributions for Statistical Modelling. The Annals of 473 Statistics , 20(3):1222 - 1235, 1992. 474

Percy Liang, Slav Petrov, Michael I. Jordan, and Dan Klein. The infinite PCFG using hierarchical 475

476

Dirichlet processes.

In

Joint Conference on Empirical Methods in Natural Language Processing and

Computational Natural Language Learning , pages 688-697, 2007. 477

Tengfei Ma, Issei Sato, and Hiroshi Nakagawa. The hybrid nested/hierarchical Dirichlet process and its 478 application to topic modeling with word differentiation. In AAAI Conference on Artificial Intelligence , 479 pages 2835-2841, 2015. 480

Mickaël Maazoun. On the Brownian separable permuton. Combinatorics, Probability and Computing , 481 29(2):241-266, 2019. 482

Ali Mahmoudi, Jere Koskela, Jerome Kelleher, Yao-ban Chan, and David J. Balding. Bayesian inference 483 of ancestral recombination graphs. PLoS Computational Biology , 18(3), 2022. 484

485

486

487

Thomas Mailund, Julien Y. Dutheil, Asger Hobolth, Gerton Lunter, and Mikkel H. Schierup. Estimating divergence time and ancestral effective population size of bornean and sumatran orangutan subspecies using a coalescent hidden Markov model. PLoS Genetics , 7(3):1-15, 03 2011.

Colin L. Mallows. Non-null ranking models. Biometrika , 44:114-130, 1957. 488

Luca Mancini and Anna Maria Paganoni. Marked point process models for the admissions of heart failure 489 patients. Statistical Analysis and Data Mining , 12(2):125-135, 2019. 490

Daniel Mauldin, William D. Sudderth, and Stanley C. Williams. Pólya trees and random distributions. 491 The Annals of Statistics , 20(3):1203 - 1221, 1992. 492

Peter Mccullagh. Random permutations and partition models, 2010. 493

Gráinne McGuire, Michael C. Denham, and David J. Balding. MAC5: Bayesian inference of phylogenetic 494 trees from DNA sequences incorporating gaps. Bioinform. , 17(5):479-480, 2001. 495

Jeffrey W. Miller and Matthew T. Harrison. Mixture models with a prior on the number of components. 496 Journal of the American Statistical Association , 113(521):340-356, 2018. 497

Antonio Khalil Moretti, Liyi Zhang, Christian A. Naesseth, Hadiah Venner, David M. Blei, and Itsik 498 Pe'er. Variational combinatorial sequential Monte Carlo methods for Bayesian phylogenetic inference. In 499 Thirty-Seventh Conference on Uncertainty in Artificial Intelligence , volume 161, pages 971-981, 2021. 500

Masahiro Nakano, Akisato Kimura, Takeshi Yamada, and Naonori Ueda. Baxter permutation process. In 501 Advances in Neural Information Processing Systems , 2020. 502

Radford M. Neal. Density modeling and clustering using Dirichlet diffusion trees. Bayesian Statistics , 503 7:619-629, 2003. 504

Quoc Phong Nguyen, Sebastian Tay, Bryan Kian Hsiang Low, and Patrick Jaillet. Top-k ranking bayesian 505 optimization. In AAAI Conference on Artificial Intelligence , pages 9135-9143, 2021. 506

Kai Ni, Lawrence Carin, and David B. Dunson. Multi-task learning for sequential data via ihmms and the 507 nested dirichlet process. In International Conference on Machine Learning , volume 227, pages 689-696, 508 2007. 509

510

[65]

Lukasz P. Olech, Michal Spytkowski, Halina Kwasnicka, and Zbigniew Michalewicz.

Hierarchical

data generator based on tree-structured stick breaking process for benchmarking clustering methods. 511

Information Sciences , 554:99-119, 2021. 512

Gary J. Olsen, Hideo Matsuda, Ray Hagstrom, and Ross A. Overbeek. fastdnaml: a tool for construction 513 of phylogenetic trees of DNA sequences using maximum likelihood. Computer Applications in the 514 Biosciences , 10(1):41-48, 1994. 515

Peter Orbanz. Projective limit random probabilities on polish spaces. Electronic Journal of Statistics , 5, 516 2011. 517

518

519

Mathias Ortner, Xavier Descombes, and Josiane Zerubia. Building outline extraction from digital elevation models using marked point processes. International Journal of Computer Vision , 72(2):107-132, 2007.

520

521

522

523

524

525

526

527

528

529

530

Mathias Ortner, Xavier Descombes, and Josiane Zerubia. A marked point process of rectangles and segments for automatic analysis of digital elevation models. IEEE Transactions on Pattern Analysis and Machine Intelligence , 30(1):105-119, 2008.

Daniel Osei-Kuffuor, Ruipeng Li, and Yousef Saad. Matrix reordering using multilevel graph coarsening for ILU preconditioning. SIAM Journal on Scientific Computing , 37(1), 2015.

John W. Paisley, Chong Wang, David M. Blei, and Michael I. Jordan. Nested hierarchical Dirichlet processes. IEEE Transactions on Pattern Analysis and Machine Intelligence , 37(2):256-270, 2015.

Laxmi Parida. Ancestral recombinations graph: A reconstructability perspective using random-graphs framework. Journal of Compututational Biology , 17(10):1345-1370, 2010.

Agyemang Paul, Zhefu Wu, Kai Liu, and Shufeng Gong. Robust multi-objective visual Bayesian personalized ranking for multimedia recommendation. Applied Intelligence , 52(4):3499-3510, 2022.

Rainer Picard and Jim Pitman. Combinatorial Stochastic Processes: Ecole d'Eté de Probabilités de 531 Saint-Flour XXXII - 2002 . Lecture Notes in Mathematics. Springer Berlin Heidelberg, 2006. 532

Vincent Pilaud and Viviane Pons. Permutrees. Electronic Notes in Discrete Mathematics , 61:987-993, 533 2017. 534

535

536

J. Pitman and M. Yor. The two-parameter poisson-dirichlet distribution derived from a stable subordinator. IBM Journal of Research and Development , 2(25):855-900, 1997.

537

538

539

540

541

542

543

544

545

Jim Pitman and Matthias Winkel. Regenerative tree growth: Binary self-similar continuum random trees and poisson-dirichlet compositions. The Annals of Probability , 37(5):1999-2041, 2009.

Andrew Rambaut and Nicholas C. Grassly. Seq-gen: an application for the monte carlo simulation of DNA sequence evolution along phylogenetic trees. Computer Applications in the Biosciences , 13(3):235-238, 1997.

Carl Edward Rasmussen. The infinite Gaussian mixture model. In Advances in Neural Information Processing Systems , 2000.

Matthew D. Rasmussen, Melissa J. Hubisz, Ilan Gronau, and Adam Siepel. Genome-wide inference of ancestral recombination graphs. PLoS Genetics , 10(5):1-27, 05 2014.

Parisa Rastin and Basarab Matei. Incremental matrix reordering for similarity-based dynamic data sets. 546 In Neural Information Processing , volume 10638, pages 76-84, 2017. 547

Nathan Reading. Cambrian lattices. Advances in Mathematics , 205(2):313-353, 2006. 548

Lu Ren, Yingjian Wang, David B. Dunson, and Lawrence Carin. The kernel beta process. In Advances in 549 Neural Information Processing Systems , pages 963-971, 2011. 550

Abel Rodriguez and Kaushik Ghosh. Nested partition models. Technical report, JackBaskin School of 551 Engineering, 2009. 552

Dane Rogers and Matthias Winkel. A Ray-Knight representation of up-down Chinese restaurants. 553 Bernoulli , 28(1):689--712, 2021. 554

Amy Y. Rossman, John M. McKemy, Rebecca A. Pardo-Schultheiss, and Hans-Josef Schroers. Molecular 555 studies of the bionectriaceae using large subunit rDNA sequences. Mycologia , 93(1):100-110, 2001. 556

Daniel Roy. Computability, inference and modeling in probabilistic programming . PhD thesis, Mas557 sachusetts Institute of Technology, 2011. 558

Daniel Roy and Yee Whye Teh. The Mondrian process. In Advances in Neural Information Processing 559 Systems , 2009. 560

Jayaram Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica , 4:639-650, 1994. 561

Kyung-Ah Sohn and Eric P. Xing. Hidden markov dirichlet process: Modeling genetic recombination in 562 open ancestral space. In Advances in Neural Information Processing Systems , pages 1305-1312. MIT 563 Press, 2006. 564

Simon Tavaré. Some probabilistic and statistical problems in the analysis of dna sequences. 1986. 565

Yee Whye Teh, Charles Blundell, and Lloyd T. Elliott. Modelling genetic variations using fragmentation566 coagulation processes. In Advances in Neural Information Processing Systems , pages 819-827, 2011. 567

568

569

570

Yee Whye Teh, Dilan Grür, and Zoubin Ghahramani. Stick-breaking construction for the Indian buffet process. In International Conference on Artificial Intelligence and Statistics , volume 2, pages 556-563. PMLR, 21-24 Mar 2007.

571

572

573

574

575

576

577

578

579

580

581

582

Yee Whye Teh, Michael I. Jordan, Matthew Beal, and David Blei. Hierarchical Dirichlet processes. Journal of the AmericanStatistical Association , 101:1566-1581, 2006.

Romain Thibaux and Michael I. Jordan. Hierarchical beta processes and the Indian buffet process. In International Conference on Artificial Intelligence and Statistics , volume 2, pages 564-571, 2007.

Ákos Utasi and Csaba Benedek. A 3-D marked point process model for multi-view people detection. In IEEE Conference on Computer Vision and Pattern Recognition , pages 3385-3392, 2011.

Isabel Valera, Francisco J. R. Ruiz, and Fernando Pérez-Cruz. Infinite factorial unbounded-state hidden Markov model. IEEE Transactions on Pattern Analysis and Machine Intelligence , 38(9):1816-1828, 2016.

Ivan Vogel, Frantisek Zedek, and Pavel Ocenasek. Constructing phylogenetic trees based on intra-group analysis of human mitochondrial DNA. In Human Interface and the Management of Information , volume 6771 of Lecture Notes in Computer Science , pages 165-169. Springer, 2011.

Gauthier Van Vracem and Siegfried Nijssen. Iterated matrix reordering. In Machine Learning and 583 Knowledge Discovery in Databases , volume 12977, pages 745-761, 2021. 584

S. Walker. Sampling the Dirichlet mixture model with slices. Communications in Statistics Simulation 585 and Computation , 36(1):45-54, 2007. 586

Chao Wang, Hengshu Zhu, Chen Zhu, Chuan Qin, and Hui Xiong. Setrank: A setwise bayesian approach 587 for collaborative ranking from implicit feedback. In AAAI Conference on Artificial Intelligence , pages 588 6127-6136, 2020. 589

Yong Wang and Jeffrey B. Remmel. A binomial distribution model for the traveling salesman problem 590 based on frequency quadrilaterals. Journal of Graph Algorithms and Applications , 20(2):411-434, 2016. 591

Dan Wei and Qingshan Jiang. A DNA sequence distance measure approach for phylogenetic tree 592 construction. In Fifth International Conference on Bio-Inspired Computing: Theories and Applications , 593 pages 204-212. IEEE, 2010. 594

Weichang Wu, Junchi Yan, Xiaokang Yang, and Hongyuan Zha. Decoupled learning for factorial marked 595 temporal point processes. In International Conference on Knowledge Discovery & Data Mining , pages 596 2516-2525, 2018. 597

Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. NeuroLKH: Combining deep learning model with 598 lin-kernighan-helsgaun heuristic for solving the traveling salesman problem. In Advances in Neural 599 Information Processing Systems , pages 7472-7483, 2021. 600

Cheng Zhang. Improved variational Bayesian phylogenetic inference with normalizing flows. In Advances 601 in Neural Information Processing Systems , 2020. 602

Cheng Zhang and Frederick A. Matsen IV. Variational Bayesian phylogenetic inference. In International 603 Conference on Learning Representations . 604

Ning Zhang and Meredith Blackwell. Molecular phylogeny of dogwood anthracnose fungus (discula 605 destructiva) and the diaporthales. Mycologia , 93(2):355-365, 2001. 606

Ping Zhang, Rishabh K. Iyer, Ashish Tendulkar, Gaurav Aggarwal, and Abir De. Learning to select 607 exogenous events for marked temporal point process. In Advances in Neural Information Processing 608 Systems , pages 347-361, 2021. 609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

Qian Zhang and Fuji Ren. Prior-based bayesian pairwise ranking for one-class collaborative filtering. Neurocomputing , 440:365-374, 2021.

Mingyuan Zhou, Haojun Chen, John W. Paisley, Lu Ren, Guillermo Sapiro, and Lawrence Carin. Non-parametric bayesian dictionary learning for sparse image representations. In Advances in Neural Information Processing Systems , pages 2295-2303, 2009.

Mingyuan Zhou, Hongxia Yang, Guillermo Sapiro, David B. Dunson, and Lawrence Carin. Dependent hierarchical beta process for image interpolation and denoising. In International Conference on Artificial Intelligence and Statistics , volume 15, pages 883-891, 2011.

Quan Zou, Shixiang Wan, and Xiangxiang Zeng. Hptree: Reconstructing phylogenetic trees for ultra-large unaligned DNA sequences via NJ model and hadoop. In IEEE International Conference on Bioinformatics and Biomedicine , pages 53-58. IEEE Computer Society, 2016.

A Properties of marked stick-breaking process

This section provides proofs of Propositions 4.1 and 4.2 concerning two properties of the marked stick-breaking process omitted in Section 4 of the main text.

A.1 Exchangeability

One of the most important properties of random data paths drawn from the generative probabilistic model described in Section 4 is exchangeability , that is, the model probability is invariant to the indexing of the data. More specifically, this can be summarised as the following statement:

Proposition A.1 (Exchangeability; Proposition 4.1) . For any permutation σ of length N ( N ∈ N ) , we have P [ Z 1 = z 1 , Z 2 = z 2 , . . . , Z N = z N ] = P [ Z σ (1) = z 1 , Z σ (2) = z 2 , . . . , Z σ ( N ) = z N ] , where z j ( j ∈ [ N ]) is a sample of paths of random permutrees.

Proof. Broadly as a whole, we will check two following facts:

(i) The random data assignments to bottom terminal nodes by the stick-breaking process [89] and the Kingman's paintbox scheme [44] are themselves exchangeable.

(ii) The selection of data paths by the two-table Chinese restaurant process is exchangeable.

Exchangeability of data assignments to bottom terminal nodes - We denote the index of the stick of the stick-breaking process to which the j th ( j = 1 , . . . , N ) data is assigned by the random variable Z bottom j . It follows from the model construction that, given a random partition of [0 , 1] drawn from the stick-breaking process, the random variable U j ( j = 1 , . . . , N ) is independent. As a result, for any permutation σ of length N , we have

where s j ( j = 1 , . . . , N ) is a sample of stick indices ( ∈ N )

Exchangeability of data path selection - Given the assignment of data to the terminal nodes, the choice of data paths follows a chain of the two-table Chinese restaurant process (see Figure 3 (c) in the main text) according to the decoration of each inner vertex of the permutree. It should be noted that in the two-table Chinese restaurant process, the data paths are chosen deterministically when the decorations are GLYPH<17> , GLYPH<15> , and 7 . Therefore, we only need to focus on the case of table partitioning (Figure 3 (c), top) when the decoration is 6 . It immediately from the model construction that the probability of partitioning the data when the decoration is is obtained as follows:

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

where the variable s j is the index of bottom terminal nodes (i.e., the stick index of the stick-breaking process on [0 , 1] ) included in the path sample z j , the variable n ( r ) represents the number of data flowing to the r th permutree vertex from the bottom in the vertical direction in the collection of data path samples z 1 , . . . , z N , and N ( r ) Left and N ( r ) Right represent the number of data to be partitioned into the left and right tables at the r thth vertex (if the decoration at that vertex is 6 ), respectively. It is important to note that the probability of selecting this datapath depends only on the number of data in the division of the table at each vertex. That is, in other words, it does not depend on the index of the data as follows:

for any permutation σ with length N . Thus, it can be checked that the selection of data paths is exchangeable. From Equations 2 and 4, we have completed our proof.

A.2 Validity of finite truncation

The generative probability model (described in Section 4) requires in principle an infinite number of random variables for its description, but finite truncation works reasonably well for a finite number of actual observed data. More specifically, we can summarize this property as follows:

Proposition A.2 (Finite truncation; Proposition 4.2) . In the generative probability model (described in Section 4) of data indexed by j = 1 , 2 , . . . , N ( N ∈ N ) , we consider an event that all random variables U j ( j = 1 , . . . , N ) , representing the horizontal position of the j th data, falls in the range [0 , 1 -ϵ ) as a situation with a sufficiently high probability P [ ∧ N j =1 0 ≤ U j < 1 -ϵ ] = ∏ N j =1 P [0 ≤ U j < 1 -ϵ ] > 1 -ϵ · O ( N ) , where ϵ > 0 is a tiny real value. In this situation, there exists some natural number K < ∞ , and all data paths are assigned with probability 1 only to paths on the finite-size random permutree generated from the random marked points l 1 , l 2 , . . . , l K .

Proof. It follows from the construction that the uniformly random random random variables U j ( j = 1 , . . . , N ) are independent, so that the probability of an event for which all those random variables fall within the range [0 , 1 ϵ ) can be checked as follows.

Then, from the construction of the marked stick-breaking process on [0 , 1] , since there are countably infinite number of marked points in the range [1 -ϵ, 1] × [0 , 1] , the probability that there exists some natural number K < ∞ and the corresponding decoration of it is GLYPH<15> is 1 . That is, we have

From the construction of the two-table Chinese restaurant process (described in Section 4) and the permutree requirement (C2) (described in Section 2), the data path assigned to the 1 st to K th bottom terminal nodes in the stick-breaking process never reaches the ( K +1) th and subsequent indexed permutree vertices. Therefore, each have a data path only on the edges of the finite permutree consisting of the 1 st to K th marked vertices. From the above, we have completed the proof.

B Relationship between permutree process and other stochastic processes

The purpose of this section is to provide additional information to help the reader better understand the characteristics of the permutree process as marked point process.

We clarify the relationship between the permutree process and other existing stochastic processes. Specifically, the permutree process can lead to the uniform random permutations and the Mondrian process as its special cases. These relationships can be derived immediately from the fact that each can be expressed as a Poisson process of some sort. We will discuss each of these in specific detail

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

below. First of all, for self-containment, the core of the permutree process is restated, although it is the same as that detailed in the body of this paper.

Permutree process - We consider a marked point process consisting of a point process and associate marks, which can be expressed as { ( l i , m i ) : i = 1 , 2 , . . . } , where l 1 , l 2 , . . . are locations and m 1 , m 2 , . . . are associated marks. Specifically, we employ the following Poisson process on a 2 -dimensional plane [0 , 1] × [0 , 1] with discrete marks:

Random locations - Draw the random locations l 1 , l 2 , . . . from a Poisson point process on the plane [0 , 1] × [0 , 1] with the intensity function λ : [0 , 1] × [0 , 1] → R + , where R + = { r : r > 0 , r ∈ R } . For notational convenience, we use l i = ( l i, 1 , l i, 2 ) ( ∈ R 2 ) , where l i, 1 and l i, 2 are the horizontal and vertical positions, respectively.

Random marks - Draw the random marks m 1 , m 2 , . . . , m n independently from a categorical distribution on { GLYPH<17> , GLYPH<15> , 6 , 7 } : Categorical ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) , where c ∗ ( ∗ ∈ { GLYPH<17> , GLYPH<15> , 6 , 7 } ) denotes the probability that decoration ∗ is adopted.

Connection to uniform random permutation - If we restrict the decoration weights for marks to ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) = (1 , 0 , 0 , 0) and make the Poisson process homogeneous (i.e., make the intensity function λ uniform), the permutree process leads to a stochastic process that generates a uniform random permutation . This fact can be easily derived by interpreting the permutation process as follows. See also Figure 6 (a). By construction, we can obtain the indices a 1 , . . . , a n so that the random positions l 1 , . . . , l n are in ascending order in the horizontal direction, that is, l a 1 , 1 < l a 2 , 1 < · · · < l a n , 1 . If we choose to set p ( v a i ) = i for the i -th vertex v i ( i = 1 , 2 , . . . ) of the resulting permutree, then p can lead to a permutation. The following fact shows that p corresponds to a uniform random permutation:

Proposition B.1. (See, for example, Lemma 2 . 2 in [32].) A uniform random permutation σ with length n can be obtained via a sequence of n i.i.d. Uniform([0 , 1]) random variables W 1 , . . . , W n (Note that their values are distinct with probability 1 ), by taking σ to be the unique permutation for which W σ (1) < · · · < W σ ( n ) .

Connection to Mondrian process - If we restrict the decoration weights to ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) = (0 , 0 , 1 , 0) and set λ ( · ) = µ · Leb( · ) , the permutree process leads to a stochastic process that simulates a Mondrian process [88, 87] on [0 , 1] with the intensity µ and the budget 1 . This fact can be easily derived by interpreting the permutation process as follows. In the above setup, the sample generated by the permutation process can be restricted to a binary tree by following the procedure described in

(d)

(

c

, c

, c

, c

) = (1

/

4

,

1

/

4

,

1

/

4

,

1

/

4)

with

µ

= 10

(top),

µ

= 20

(middle), and

µ

= 40

(bottom).

下齐

心房斋

GLYPH<17>

GLYPH<15>

6

7

Markov process with an intensity µ and a time limit of 1 . Furthermore, if we consider the horizontal 722 location of the marked point process as the position where the line segment of length 1 is cut, we can 723 consider this time evolution as a hierarchical partition of the line segment. Therefore, this can be 724 regarded as a Mondrian process. 725

726

C Bayesian inference for phylogenetic permutree (omitted in Section 5)

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

This section reveals the Bayesian inference algorithm for phylogenetic permutree analysis using our permutree processes. For the sake of generality, we will use the marked point process representation described in Section 3 in particular as a permutree process. This argument can also be applied, with minor modifications, to the special case of the marked stick-breaking process (with its finite truncation) described in Section 4.

Overview - Standard Bayesian inference algorithms such as Markov chain Monte Carlo (MCMC) methods can be realized by sequentially iterating the following two update rules: (i) updating the permutree process (ii) updating the evolutionary model. Since the latter can be supported by standard inference methods to evolutionary models, it is the updating method of the former that is particularly important here. For the former, various inference algorithms that have been proposed for generic marked point processes and their extensions [68, 69, 26, 96, 43, 104, 30, 55, 31, 14, 35, 109] would be applicable, since the entity of the permutree process is a marked point process as shown in Section 3 of the main text. This section describes a useful inference method that exploits an important property of Poisson processes, namely, that a certain Poisson process can be obtained by thinning operations from another Poisson process with higher intensity. Section C.1 provides a brief description of the thinning operation for the Poisson process as a preliminary to our MCMC method. Then, Section C.2 once again writes down the whole generative probabilistic model, since it should be possible to see at a glance what the parameters to be inferred are in the permutree process and its phylogenetic tree described in Section 5. Finally, Section C.3 describes the MCMC inference algorithm.

C.1 Preliminaries: thining operations for Poisson processes

Our MCMC method uses important properties of Poisson processes. Specifically, we will discuss how to represent a certain Poisson process via another Poisson process with higher intensity.

Homogeneous Poisson process - In this paper, we mainly consider homogeneous Poisson processes on [0 , 1] × [0 , 1] , i.e., where the intensity function is given by a constant. A Poisson process on [0 , 1] × [0 , 1] with intensity µ (where 0 < µ < ∞ ) is a stochastic process for a random set of points, where the number of points belonging to ( x 1 , x 2 ] × ( y 1 , y 2 ] follows a Poisson distribution with the parameter µ ( x 2 -x 1 )( y 2 -y 1 ) for any 0 ≤ x 1 < x 2 ≤ 1 , 0 ≤ y 1 < y 2 ≤ 1 . For notational simplicity, we will denote a Poisson process on [0 , 1] × [0 , 1] with intensity µ by PP( µ, [0 , 1] × [0 , 1]) . We also recall that in the main text, we defined this homogeneous Poisson process as the Poisson process with the intensity function λ ( · ) = µ · Leb( · ) as an equivalent expression, where Leb( · ) indicates the Lebesgue measure.

Thinning operation on Poisson processes - One of the most interesting properties of Poisson processes is that a Poisson process with a certain intensity can be obtained by applying the thinning operation from a Poisson process with a higher intensity. More specifically, a Poisson process with intensity µ can be constructed as follows:

(i) We generate a random set of points from a Poisson process with intensity ν ( > µ ) .

(ii) For each point generated, independently decide whether or not to accept it with probability µ/ν .

(iii) The set of only accepted points can be regarded as following the Poisson process with intensity µ .

C.2 Full description of phylogenetic permutree with permutree process

As input observation data, we consider DNA (molecular) sequences x j ( j = 1 , . . . , N ) observed at 768 letter length S over N species. For example, the sequence x j = GAGTAC has length S = 6 . Figure 769 8 (top) shows the observation DNA sequences as an S × N matrix. The four colors represented 770

ObservationDNAsequences

Sequence length

50㎡

200

10

20

30

40

50

60

70

80

90

100

Numberofobservationsequences

Sequencesassignedtoleafnodes

Sequence length

5m

200

10

20

30

40

50

60

70

80

90

100

Numberofobservationsequences

by each element of the matrix correspond to the four different letters A,C,G , and T . We regard 771 these DNA sequences as following a phylogenetic tree based on a permutree. First, we generate the 772 marked points { ( l i , m i ) : i = 1 , 2 , . . . n } and the corresponding permutree T from the permutree 773 process. We recall that the transformation from marked points to permutree can be performed by 774 the transformation in Figure 9 (which we will call MPP2PT). Then, we represent coalescence, 775 recombination, multiple ancestry, and mutation events in genealogy by combining the four types of 776 the decorations 7 , 6 , GLYPH<15> , GLYPH<17> with the following interpretations: 777

778

779

Coalescence 7 - A coalescence event represents two lineages having a common ancestral lineage.

780

781

782

783

784

785

786

787

Recombination 6 - A recombination event represents the joining of two exclusive subsequences of two lineages by one lineage.

Partition GLYPH<15> - We give the decoration GLYPH<15> the role of division so that a single permutree can represent a phylogenetic tree with multiple ancestors. Specifically, we connect the two left edges and similarly connect the two right edges to lead to two unconnected tree structures on either side of decoration GLYPH<15> .

GLYPH<17>

Backward in time GLYPH<17> (optional) - We suppose that no mutation occurs when going back in time from a vertex to a vertex with .

802

803

804

805

806

807

808

809

810

811

812

for i = 1 , 2 , . . . , n and j = 1 , . . . , N , where T | Z j refers to a phylogenetic tree (a tree consisting of one edge and two vertices at either end) from which only the leaf nodes and their children are extracted from the phylogenetic tree T . Since the variables ϵ and ϵ ′ are hyperparameters for the non-informative prior distributions, it is standard to use them fixed to tiny values. Can we then consider how to directly infer the above generative probability model? Certainly, it is possible in principle to infer the above generative probability model as it is by direct updating of the permutree process as shown in Figure 13. However, in such direct inference, the complexity n is often strongly affected by bad local modes, and often the Markov chain is entangled in the local optima, resulting in slow convergence. Therefore, using the properties of Poisson processes described in the preparation, a method can be considered to reduce the influence of such local optima by taking the dare to have redundant model parameters. Equation (7) and (9) can be rewritten as follows.

for i = 1 , . . . , K . The above is the full phylogenetic tree model based on the permutree. One point to 813 recall here is that permutrees includes binary trees and Cambrian trees as special cases (as discussed 814 in Remark 2 . 1 of the main text). Therefore, the permutree process can be attributed to various 815 models by adjusting the prior distribution for the decoration weights ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) . Figure 14 (left) 816 shows the permutree process with ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) ∼ Dirichlet( ϵ/ 2 , 0 , 0 , ϵ/ 2) as BINARYTREE 817 (restricting the expressive power to binary trees). Figure 14 (right) shows ( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) ∼ 818 Dirichlet( ϵ/ 3 , 0 , ϵ/ 3 , ϵ/ 3) as CAMBRIANTREE (restricting it to Cambrian trees). 819

820

821

Joint probability

Z

:=

(

Z

1

, . . . , Z

- For notational convenience, we use

N

)

,

b

:=

(

b

1

, . . . , b

K

)

,

ˆ

L

:=

ˆ

(

l

1

X

, . . . ,

ˆ

l

K

:= (

x

)

,

ˆ

m

1

, . . . ,

:=

( ˆ

x

m

1

N

)

,

H

, . . . ,

ˆ

m

:= (

h

K

)

v

, and

)

v

∈T

c

,

=

( c GLYPH<17> , c GLYPH<15> , c 6 , c 7 ) . We obtain the following joint probability density function: 822

823

824

825

826

where P obs is the probability density function (PDF) of Equation (15), P evo is PDF of Equation (13), P PP is PDF of Equation (16), and subsequent terms are PDFs of the standard distributions. The posterior distribution of the parameters H , b , w , z , ˆ L , ˆ m , c to be estimated is proportional to this joint probability density.

827

828

829

830

831

832

833

C.3 Bayesian inference algorithm for phylogenetic permutree

We can construct the MCMC algorithm by iteratively repeating the following update rules for the DNA evolution H on the phylogenetic permutree, the binary indicators b , the leaf node weights w , the observation assignments z , the redundant locations ˆ L , the redundant marks ˆ m , and the decoration weights c .

Update rule for DNA evolution H - We recall that each element of the matrix H = ( H s,j ) S × N consists of one of the letters A , C , G , or T . Using Equation (21), we calculate the joint probability that

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

38

88

37

36

36

35

35

34

33

31

31

30

30

29

29

28

28

27

27

26

25

25

24

24

Z

23

22

22

21

21

20

20

19

18

181

17

15

16

1b

14

CL

CL

12

11

10

10

each element H s,j is A , C , G , or T , respectively, and let p A , p C , p G , or p T denote them respectively. Then we obtain the following Gibbs update rule:

Update rule for binary indicators b - For each i = 1 , . . . , K , we can obtain the Gibbs update rule derived by calculating the posterior probability ratio for b i = 0 and b i = 1 using Equation (21). Specifically, we suppose that the value of the joint density for b i = 0 is π 0 and the value for b i = 1 is π 1 , and then we obtain the following update rule:

Update rule for leaf node weights w - From the conjugacy of the Dirichlet and Categorical distributions, we obtain the following Gibbs update rule:

where N i ( i = 1 , . . . , |LN ( T ) | ) indicates the number of the observation sequences x j ( j = 1 , . . . , N ) which is assigned to the i th leaf node of the phylogenetic permutree T .

Update rule for observation assignments z - Using Equation (21), we calculate the joint probability that each observation sequence x j ( j = 1 , . . . , N ) is assigned to the i th ( i = 1 , . . . , |LN ( T ) | ) leaf node of the phylogenetic permutree T , and let ¯ w i denote it. Then we obtain the following Gibbs update rule:

Update rule for redundant locations ˆ L - We use the simple Metropolis-Hastings (MH) method. For each position, we generate a new candidate sample from the normalized probability measure ˆ λ of the intensity function λ , that is, ˆ l i ∼ ˆ λ ( i = 1 , . . . , K ) , and decide whether to adopt it through the MHacceptance/rejection scheme using Equation (21).

Update rule for redundant marks ˆ m - Using Equation (21), we calculate the joint probability that each element ˆ m i has 7 , 6 , GLYPH<15> , GLYPH<17> , respectively, and let p 7 , p 6 , p GLYPH<15> , p GLYPH<17> denote them respectively. Then we obtain the Gibbs update rule:

CAGTGTGGAT

CGCATAACTT

GCAGGTGGGT

GCTCGGTTCT

CTACGAGGTT

GGGCAGTACT

CAACCAAAGA

TAAATACAGC

dTATCTCTAG

CAACATGCGC

CTGGCGCAAT

TGGTGTAAAT

AAGCACGCGG

TGAGATGAGC

AGGGAGAATC

TGCCCCTCAG

GGACGAATTT

AGGTGATCGG

CCAAGATACC

CCAAGATACC

AGCACCTATT

TACACTTATA

TGGTTGCTCA

CCGTATAAAC

GGCCCTTCGC

CTGCCCGGCC

AGAAATACGT

GGGCAGTACT

AGCATGTCTA

GGATTTCGCA

CGCTGGCCCTI

ACATTACGCA

GTGGCGTTTC

ACAGTTAACC

ACCGTTAATA

GACTCACCTT

GCGTAGCAAT

CCTGATACAT

TCCGGGTCCC

CCTCCCAGAG

CTAGAGATCTI

CAGAGGTGCC

CTAACGATOT

GCAGTTCTGT

ACGTTTTGAA

TAGTGCGCCT

GTTTGACTAT

GATAGAGCAT

TCCGGGTCCC

Update rule for decoration weights c - From the conjugacy of the Dirichlet and Categorical 855 distributions, we obtain the following Gibbs update rule: 856

where N ∗ ( ∗ ∈ { 7 , 6 , GLYPH<15> , GLYPH<17> } ) indicates the number of the marks ˆ m i ( i = 1 , . . . , K ) which has the 857 decoration ∗ . 858

859

C.4 Empirical impact of finite truncation

To investigate the empirical impact of finite censoring on the marked stick-breaking process described 860 in Section 4, we report in Figure 15 the prediction performance for different levels of finite censoring, 861 K = 25 , 50 , 100 and 150 , in the same experimental setup as in the main text (Section 5). It can be 862 seen that when the level of finite truncations is extremely restricted, the prediction performance has 863 been reduced, while when some level of censoring is ensured, the prediction performance is not so 864 reduced. This can be seen as reflecting the fact that the marked stick-breaking process can adjust its 865 own real model complexity in a data-driven manner according to the observation data. 866

GCCAGGTGTC

ATCGCACGCC

CATACTCGGC

1

AGTCGATTTT

1

1

AGTCGATTTT

AACAAGGCCA

1

AACAAGGCCA

1

ATOGCACGCC

CATACTCGGA

TGGCACTGAT

TTCAGTGCTG

1

CAGTAAGTTT

AGTCGATTTT

TTCAGTGCTG

GCCAGGTATC

TTCAGTGCTG

TTCAGTGCTG

AACAAGGCTG

CATACTCGAA

1

AGOTTGATCC

■

AACAAGGCCA

TTCAGTGCTG

TGGCTCTGAT

ACACAGGCCTI

TTGTTGCCCC

GCCAGGGCCA

TTGTTGTGAT

AGCTTGATCC

CAGTAAGTTT

TTCAGTGCTG

GCCAGGGCCA

AGTCGATTTT

CATACGGCCT

TTCAGTGATG

TTCTTGATCC|

AGCAAGGCCA

AGCAAGGCCA

AACAAGGCTG

ATCGCACGCC

CATACGGCCT

TTCAGTGTTG

CATACGCGCC

CAGTAAGTTT

CTATTGATCC

AGTCGATTTT

CCGTAAGTTT

CCCAGGGCCA

AGACGATTTT

CATACGGCCT

十

D Remaining challenges 867

The main difficulty in applying the permutree process to data modeling is how to handle its unlimited 868 finite or infinite model complexity (i.e., number of vertices). Roughly speaking, it is not possible 869 in principle to naively implement a model with infinite parameters on current computers. This is a 870

ObservationDNAsequences

ObservationDNAsequences

Sequence length

Sequencelength

5

50

100

150

200

200

10

20

30

40

50

60

70

80

90

10

20

30

40

50

60

70

80

90

100

Number of observation sequences

Number of observation sequences

Sequences assigned toleaf nodes

Sequencesassigned toleaf nodes

Sequence length

Sequencelength

100

150

200

200

10

20

30

40

50

60

70

80

90

1

10

20

30

40

50

60

70

80

90

100

Number of observation sequences

Number of observation sequences

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

Perplexity

RMPS

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

HWB

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

ZB

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

COMB

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

Perplexity

RMPS

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

HWB

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

ZB

0

50

100

150

Truncation level

1

1.1

1.2

1.3

1.4

1.5

COMB

central topic in the BNP field, and we have historically had two policies. One is to represent models 871 of infinite complexity such that finite truncation works reasonably well. This corresponds just to 872 the representation methods for the stick-breaking process [89] in Dirichlet process infinite mixture 873 models [79] and the beta-Bernoulli process in infinite factor models [95, 93]. The other method is a 874 model representation in which, in conjunction with the finite amount of observed data, the model 875 activates only as many of the potentially infinite number of parameters as necessary. This corresponds 876 to the Chinese restaurant process [74, 76] in the mixure models or the Indian buffet process [29] in 877 the factor models. While Section 4 focuses on the former policy, this section will explore the latter. 878

879

D.1 Preliminary: ordered Chinese restaurant process

We begin our discussion with a representation using oCRP for binary trees, a special case of 880 permutrees. Let θ > 0 be the concentration parameters and α > 0 the discount parameter. We will 881 now take a so-called spinal decomposition (Figure 16 left) of the binary tree. In the metaphor of CRP, 882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

△

0

△

△

6+0

A

A

一

2α+θ

6+0

6+0

4

6+0

一

6

+θ

position

2

6+0

2

一

6+0

6

+6

θ

the customer can be viewed as as seeking a dish at the terminal node of the binary tree. The new customer can then either proceed to the existing subtree or create a new branch on one of the edges, according to the proportions shown in Figure 16 right. This can be viewed as CRP with random ordering of the CRP tables.

Ordered Chinese restaurant process (oCRP) [77, 85] - This stochastic process is a generative prbabilistic model that constructs a random binary tree by means of a recursive structure as follows. Let α and θ be the discount parameter and the concentration parameter, respectively.

The first customer goes straight from the root to form one terminal node.

The second customer forms a split between the root and the terminal node where the first customer is located. At this stage, the advanced subtree of the second customer is assigned a weight of 1 -α and each edge of the split spinal cord is assigned a weight of α and θ , respectively.

The third and subsequent guests determine their own destination according to the proportion of weights assigned to the subtree and each edge on the spinal cord. If it chooses an edge on the spinal cord, it creates a new branch there to become a subtree and assigns weight α to the newly created edge on the spinal cord. If it moves on to an existing subtree, it recursively determines its own destination according to the nested oCRPs on that subtree and adds 1 to the weight of the subtree.

One will notice that this stochastic process is very similar to the standard Chinese restaurant process (CRP). If each subtree is considered a table, the probability that a new customer will sit at an existing table is proportional to the weight of the number of customers already sitting at that table minus the discount parameter α . It will be seen that this is the same situation as in the standard CRP corresponding to the well-known Pitman-Yor process [76]. However, it differs from the standard CRP in that when a new table is seated, that table is determined by reference to the order of the existing tables. For this reason, this stochastic process is called an 'ordered' CRP.

One of the most important properties of oCRPs is exchangeablity :

Theorem D.1 (Proposition 1 (a) [77]) . A random binary tree generated by the nested oCRPs with the discount parameter α and the concentration parameter θ has exchangeable leaf labels for all n = 1 if and only if α = θ = 1 / 2 .

D.2 Our attempt: Chinese restaurant street

Strategy sketch and advance notice - Recalling the requirements of (C1) and (C2) for the definition of permutrees (Section 2), we could introduce the following metaphor of CRP (See also Figure 17):

Each customer is looking for a dish in a Chinese restaurant street and prefers a street that is popular with other customers, but is also willing to explore new streets on a whim. The development of the streets, with one customer after another searching for a dish, represents the permutree evolution.

The Chinese restaurant street has a recursive structure. The boulevard (main street) has side streets , and each side street becomes the next boulevard, with its own next side streets recursively. This recursive structure would be reminiscent of an existing oCRP or nested CRP that recursively calls smaller CRPs in the overall process. When a side street becomes a cross street, it represents a vertex with GLYPH<15> . If the side street extends only one way, it corresponds to a vertex with 6 or 7 .

̸

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

Chinese restaurant street (CRS) - CRS is given by the recursive structure of the streets, consisting of boulevards and side streets. Let θ 1 > 0 and θ 2 ≥ 0 be the concentration parameters and α > 0 the discount parameter. Figure 18 illustrates the situation where a new boulevard is a small CRS in a large CRS with the recursive structure. A CRS at a certain level consists of a boulevard and side streets, where each side of the boulevard is weighted by the concentration parameters θ 1 , θ 2 and the discount parameter α , and each side street is assigned a weight equal to the number of customers who proceeded to it minus the discount parameter α . When the next customer enters this boulevard, the next destination is determined according to the ratio of those weights. It would have been a wishful idea if this vanilla CRS could be used as a permutree model, but unfortunately, it does not satisfy the requirements (C1) and (C2) as it is.

Properties - (1) The most important feature of CRS is that it is an extension of the existing oCRP. Specifically, in the case of θ 2 = 0 (i.e., a situation where both boulevards and side streets grow only to one side), CRS is equivalent to oCRP for random binary trees. (2) Another important property is exchangeability (i.e., invariance of probabilities with respect to the order in which customers enter the process), which is often the case with variants of CRPs. For our CRS, we can show that it is exchangeable in the case of α = θ 1 + θ 2 = 1 / 2 , inheriting the result of exchangeability [77, Proposition 1] of oCRP. This property would be helpful in Bayesian inference.

̸

Theorem D.2. Arandom tree generated CRS with the discount parameter α and the two concentration parameters θ 1 and θ 2 (described in Section 3 . 2 of the main text) has exchangeable leaf labels for all n = 1 if and only if α = θ 1 + θ 2 = 1 / 2 .

Proof. This can be verified by inheriting the exchangeability of oCRPs described in Theorem D.1. We shall consider each subtree in oCRP as a table and assign natural numbers of labels to the tables, starting from 1 according to the order in which the tables were generated. By reducing the resolution of the leaf labels in Theorem D.1 to table labels, the ordered tables generated by oCRP are also exchangeable. Then, for the random tree generated by CRS, if we consider each subtree as a table and set θ 1 + θ 2 = θ , this is also attributed to the random ordered tables of oCRP with the discount parameter α and the concentration parameter θ . Thus, by repeatedly applying the fact that the table labels of oCRP are exchangeable only when α = θ = θ 1 + θ 2 = 1 / 2 , until each table eventually becomes a leaf node, we can confirm that the CRS has exchangeable leaf labels.

E Validity of transformation from marked points to permutree

This section verifies that the marked points { ( l i , m i ) : i = 1 , 2 , . . . n } generated from the marked point process are correctly transformed into permutrees by the algorithm in Figure 9 (Algorithm 1 of the main text). This can be verified by the following procedure, which is similar to the method in the proof of Proposition 8 in [75].

(i) There is exactly one strand in each section separated along the auxiliary line (the red dotted line in Figure 9). This can be shown by mathematical induction on the number of nodes in the permutree.

(ii) The graph created by the algorithm in Figure 9 has no cycles. This can be shown by using the proof by contradiction. If the graph had cycles, it would cross the red dotted line. However, by construction, the graph never crosses the red dotted line.

963

964

965

...

...

...

(iii) From (i) and (ii), the graph generated by the algorithm in Figure 9 is a tree, and furthermore, since the red dotted line separates the left and right ancestor and descendant sub-trees, this is a leveled permutree.

NeurIPS Paper Checklist 966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: Section 6 clarifies the limitations and remaining challenges.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Appendix A provides all details for theoretical statements in Section 4.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Appendix C provides all details for our Bayesian inference algorithms.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

Justification: In general, phylogenetic tree analysis can sometimes have significant impacts in our daily lives. For example, during the SARS-CoV-2 pandemic, it has recently been used to analyze the spread and evolution of pathogens [52, 60]. We believe that the permutree process proposed in this study could be an important elemental technology for such phylogenetic tree analysis in the future. In order to focus on permutree methodology in machine learning, this paper uses the classical model for the DNA evolution model as is. However, it will be necessary to improve the model to a more sophisticated model that can represent real-world phenomena more precisely in order to provide some insights for real-world applications.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

1047

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article adapts the recently-developed combinatoric concept of the “Pemutree” into a machine learning context, making links with existing methods in Bayesian nonparametrics, and providing a pathway for how to make the abstract mathematical concept relevant to data-driven approaches and inference. The theory is explained, and an example application in phylogenetic analysis is performed.\n\nSTRENGTHS:\nThis is an ambitious paper! Making links with recent combinatorial research and machine learning is a good thing to do! It has a large vision of building one grand unifying theory of discrete Bayesian nonparametrics, which (it is claimed) can be done with the framework presented. The review of discrete BNP at the start of the article is thorough. \n\nThe potentially difficult subject matter of the abstract mathematical objects is explained fairly clearly through the use of figures and well-chosen notation. The theoretical aspects are explained thoroughly, and the application pursued emerges naturally from the framework developed earlier in the article: the coalescent analysis and the Mondrian process are good results to have. The scientific writing is of a fairly high standard, with only a couple of surprising vocabulary choices.\n\nWEAKNESSES:\nThe subsequent developments of the theory and applications don’t quite live up to the grand vision set out earlier in the article. The authors struggle to represent the most widely-used discrete BNP object of the Dirichlet Process in this supposedly all-encompassing framework: perhaps this is doable in the future and represents work yet to be done, but the initial claims about the generality of permutree processes made in the article are not fully followed through on.\n\nThe article also (necessarily) spends a lot of time introducing the theoretical framework, quite heavily at the expense of presenting the data applications properly later in the manuscript. Squeezing all of the experimental results into “Demonstration” in half a page is really too brief to be very convincing, although there is a lot of interesting material in Appendix C that would ideally be in the main text. Many of the potentially thorny issues concerning inference and computation are therefore overlooked.\n\nSome of the figures could be better designed: I found the visual interpretation of permutrees key to developing some understanding them, so making Figure 1 bigger and more prominent would be a help (I think figure 1 is more crucial than Figures 2/3/4 in this respect). The representation of the data in greenscale Figures 8 and 14 is very confusing: I don’t think I really learned anything from that representation.\n\nSome of the language choices are a bit strange: line 69: “we dare to pay particular and explicit attention here”, line 869: “Roughly speaking, it is not possible in principle to naively implement a model with infinite parameters on current computers”, \n\nLine 250: “as an overall trend…” The analysis of the experimental results is not rigorous enough. You have real values and uncertainties for the perplexity. Do some tests or similar to establish more clearly the differences in performances rather than painting broad brushstrokes.\n\nLine 80: some of the symbols used have already established meanings in a machine learning context, i.e. \\otimes meaning kronecker product. Maybe make clear that this is a new notation that overrides any previous perceptions.\n\nQUESTIONS:\nLine 219: “Data path modelling” the introduction of the two-table Chinese restaurant process is instructive, but might reveal the limitations of the model framework: are the only methods that can be captured by a permutree some variation on iterative applying of CRP-style binary splits and merges? Do you really think that this is a large enough “basic vocabulary” of operations to cover the entire world of discrete BNP methods out there? Is that a provable statement?\nLine 336: epsilon = 0.01: where does that come from?\n\nAppendix C.3: How would you describing performing the inference for this model? Did it need much fine tuning? Did it need lots of mixing time? Did it need a lot of computational resources? What about the comparison methods? Is this this “fancy but expensive” model? No shame if so, but it’s good to know this.\n\nLine 887: “ordered Chinese Restaurant Process” - what are the material, practical implications of adding the ordering of tables on top of the standard chinese restaurant process? Can this model still answer the same type of statistical problems that a standard CRP does? Do the  categories/clusters observed in the data need to have some sort of ordering for this to be valid? Can the permutree represent a totally standard CRP without this ordering? If the permutree cannot represent the most widely used discrete BNP model then the claims towards the start of the paper about this being the grand unified framework need toning down.\n\nLine 912: “Chinese Restaurant Street” - i worry that between the indian buffets and the Chinese Restaurant Franchises then the underlying metaphor is being overextended in this community. Why exactly have you proposed this model formulation? Do you consider it to be a useful new potential BNP model that has been underexplored so far? Is this the closest thing you can derive within this framework to a standard chinese restaurant process? This is unclear to me.\n\nRandom non-crucial question from an interested non-combinatorist: Can you define permutrees in a >2d space? This will surely change the topologically allowable relationships between nodes.\n\nLIMITATIONS:\nThe practical models that are (presently) successfully captured in this framework are not the most widely used BNP models out there. The whole permutree framework seems best suited to the coalescent-type models pursued, and the other BNP models that have been successfully described, such as Mondrian processes, are interesting but not in widespread use.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAfter giving an introduction to permutrees, a stochastic process on them is constructed by sampling the nodes according to an intensity function on the 2d unit interval and uniformly assigning the marks. It is shown how to add the edges to meet the requirements for the object being a labeled permutree. Paths from terminal nodes to other terminal nodes in the permutree can be used to represent sequential data. Finally, the model is used for an inference task involving DNA sequences.\n\nSTRENGTHS:\n* Sections 1-4 are very concise but nonetheless clear and easy to follow. This paper does a great job explaining the complex concepts of permutrees and permutree processes. \n* Figure 1-3 are very helpful\n* the concept generalizes popular processes used in ML, e.g., the Mondrian processes\n\nWEAKNESSES:\n* As a reader unfamiliar with phylogenetic analysis, I did not immediately understand what the task in this setting is (and I am still unsure if I fully got it): Do you have a set of DNA sequences where some of the letters are masked, and you want to predict the masked letters? \n* Besides not really understanding what the goal of this task is, I think Section 5 does not provide enough explanation of how this goal then is achieved, i.e., the length of the paper/ the level of detail in Section 5 is a problem. It's ok to defer details to the appendix as long as one can still follow the main section without them, but I struggle with that. Example given: I find it crucial to know the likelihood function when it comes to a Bayesian inference task, which is not mentioned in the main section.\n\nQUESTIONS:\n* What is the task in the phylogenetic analysis application? \n* What would be other tasks where inference on permutrees is relevant in machine learning?\n\nLIMITATIONS:\nyes, there is a dedicated paragraph on limitations and I think it captures the limitations of the suggested concept appropriately.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a prior for Bayesian nonparameterics called the permutree. They apply it to model complex phylogenetic data with both coalescence and recombination, a setting that previous processes such as the Kingman could not model; the model seems to perform state-of-the-art phylogenetic inference. In principle the process could also be used to model other combinatorial objects such as permutations or trees however this is not demonstrated.\n\nSTRENGTHS:\nModeling recombination is challenging and this model proposes a method to do so.\n\nWEAKNESSES:\nThe writing is very challenging. There are multiple points where the writing is strange, for example the use of \"dare\" in \"For technical reasons (discussed immediately below), we dare to pay particular and explicit attention here to the set V of the “interior vertices” (i.e., vertices of degree at least 2) other than the terminal nodes.\" \". The exposition is also very verbose and proposition is challenging to understand without reading the proof. Figures 3(c) and 1(b)-(d) are never mentioned in the text, the later is quite confusing since it seems to suggest that the permutree can model many combinatorial objects.\n\nIt seems that there is a disconnect between the description of the methods in sections 3, 4, and the experiment in section 5. See questions.\n\nThe ultimate goal of phylogenetics is to infer ancestry which is not identical to maximizing likelihood. To validate a bone fide phylogenetic inference method that handles recombination, one should show that inferred recombination events are realistic.\n\nQUESTIONS:\nIn section 3 you state that you generate $l$ from a Poisson process then in section 4 you use a stick-breaking procedure. Which do you use in practice? In the genetics case, $l_k$ is stated to be drawn from a point process in eqn 7. if this is the case, is the data still modeled as part of an infinite exchangeable process? How could I query the posterior predictive for example?\n\nIn the genetics case, could you explain the role of the two table Chinese restaurant process? It seems that sequences are clustered by node and that the path does not affect their likelihood.\n\nWhat is the form of the likelihood for recombination?\n\nLIMITATIONS:\nDiscussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

comments: SUMMARY:
The authors describe the concept of permutrees, how to sample permutrees in a stochastic process, and how to model data with permutrees. They apply it to tracking DNA changes.

STRENGTHS:
Interesting new model that unifies permutations, trees, partitions, and binary sequences 

Strong mathematical foundation

Practical applications

well-written

WEAKNESSES:
the figures are small and hard to read when printed in gray-scale

QUESTIONS:
I do not know what a stick breaking process or Chinese restaurant process is?

>Fig.1 (l) 

It is not obvious how the binary number is generated.
Could one say if the line from one node to the next goes up it yields 1 and if it goes down it yields 0?

>p7 253 numnber 

number ?

>p7 261 "tiny real value"

how tiny is tiny ?

>p16 669 uniformly random random random variables Uj

is that more random than just a random variable?

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12