Collaborative and Confidential Junction Trees for Hybrid Bayesian Networks
Authors: Roberto Gheda
Abele Mălan
Thiago Guzella
Carlo Lancia
Robert Birke
Lydia Y. Chen
Venue: NeurIPS 2025 poster ()
Keywords: Bayesian Networks
Collaboration
Secure Multi-Party Computation
Primary area: probabilistic_methods

Abstract: Bayesian Network models are a powerful tool to collaboratively optimize production processes in various manufacturing industries.
When interacting, collaborating parties must preserve their business secrets by maintaining the confidentiality of their model structures and parameters.
While most realistic industry scenarios involve hybrid settings, handling both discrete and continuous data, current state-of-the-art methods for collaborative and confidential inference only support discrete data and have high communication costs.
In a centralized setting, Junction Trees enable efficient inference even in hybrid scenarios without discretizing continuous variables, but no extension for collaborative and confidential scenarios exists.
To address this research gap, we introduce Hybrid CCJT, the first framework for confidential multiparty inference in hybrid domains with semi-honest, non-colluding adversaries, comprising:
(i) a method to construct a strongly-rooted Junction Tree across collaborating parties through a novel construct of interface cliques; and,
(ii) a protocol for confidential inference built upon multiparty computation primitives comprising a one-time alignment phase and a belief propagation system for combining the inference results across the Junction Tree cliques.
Extensive evaluation on nine datasets shows that Hybrid CCJT improves the predictive accuracy of continuous target variables by 32% on average compared to the state-of-the-art, while reducing communication costs by a median 10.4x under purely discrete scenarios.

Collaborative and Confidential Junction Trees for Hybrid Bayesian Networks
Roberto Gheda" Delft University of Technology r. gheda@tudelft.
Abele Mälan University of Neuchâtel abele mww.mmunine..n.mmmmmmmmmmmm
Thiago Guzella
ASML The Netherlands B.V. thi thiago. 1aazzell.aam.... com
Carlo Lancia
Robert Birke University of Turin robert. birke@unito.it
Lydia Y. Chen
ASML The Netherlands B.V.
Delft University of Technology 10hhnn10@
carlo. lam.ia@asml....................m com
Abstract
Bayesian Network models are a powerful tool to collaboratively optimize produc- tion processes in various manufacturing industries. When interacting, collaborating parties must preserve their business secrets by maintaining the confidentiality of their model structures and parameters. While most realistic industry scenarios involve hybrid settings, handling both discrete and continuous data, current state-of- the-art methods for collaborative and confidential inference only support discrete data and have high communication costs. In a centralized setting, Junction Trees enable efficient inference even in hybrid scenarios without discretizing continuous variables, but no extension for collaborative and confidential scenarios exists. To address this research gap, we introduce Hybrid CCJT, the first framework for con- fidential multiparty inference in hybrid domains with semi-honest, non-colluding adversaries, comprising: (i) a method to construct strongly-rootes Junetion Tree across collaborating parties through a novel construct of interface cliques; and, (ii) a protocol for confidential inference built upon multiparty computation primitives comprising one-time alignment phase and belief propagation system for com- bining the inference results across the Junction Tree cliques. Extensive evaluation nine datasets shows that Hybrid CCJT improves the predictive accuracy of continuous target variables by 32% on average compared to the state-of fftte-of-the-eeee the-art, while reducing communication costs by median 10.4x under purely discrete scenarios.
Introduction
Bayesian Networks (BNs) are probabilistic models that incorporate domain knowledge while retain- ing interpretability [1]. This makes them a powerful tool for optimizing manufacturing processes in numerous industrial domains 22 3 4] One example is semiconductor manufacturing, which involves complex procedures and machines. Production process optimization within the field in- creasingly requires the collaboration between parties like manufacturers (with domain knowledge on the product design and manufacturing steps) and equipment vendors (with expertise on the equip- ment performance) (5). However, such collaboration must preserve the confidentiality of the BNs' structure and parameters, which encode sensitive industrial knowledge and the parties' evidence. Moreover, most real-world applications require models to handle discrete (A) and continuous (T) data simultaneously [6, achieved using Hybrid Bayesian Networks 7
Work partly done while interning at ASML,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Chipmaker
Equipment Vendor
Evidence
Model
Query Overley
Model
Evidence
Eishing
A
1.2mm
A
Wafer Table Wear Low MB
Figure I: The equipment vendor and the chip manufacturer observe status of etching rate and wafer table wear separately (evidences), and model relationship between variables with BNs (models). Equipment vendor predicts the overlay error (query), leveraging evidence of wafer table wear (discrete value) and etching rate (continuous value).
Collaborative Confidential Bayesian Networks tackle the aforementioned challenges by treating local parties' BNs as a larger global network while concealing these parties' knowledge from each other. Figure exemplifies confidential inference in the semiconductor industry.
Non-collaborative BNs can be queried via the Variable Elimination algorithm 77 which eliminates variables one by one by marginalization. CCBNet S extends this algorithm to collaborative confi- dential BNs. However, CCBNet has main drawbacks. First, it only supports discrete variables, leaving out continuous ones. Second, it does not scale well, as inference time is worst-case expo- nential in the number of network nodes, and high communication costs exchange partial results between parties. Such drawbacks hinder its real-worke adoption in scenarios with continuous or mixed-variable types, and BNs with as few as 15 nodes.
To make inference on larger BNs tractable, 99 divides the network into smaller sub-networks, called cliques, and organizes them into Junction Tree (JT) structure. Subsequently, inference is performed via the Variable Elimination algorithm each clique, and the results are combined via Belief Propagation [0]]. Conveniently, when Junction Trees are strongly-rooted, they can be used to carry-out inference over Hybrid BNs [1I]. The strongly-rooted property requires a strong order eliminating continuous variables before discrete ones [12]. Constructing such tree in distributed and confidential fashion is an unexplored challenge.
To tackle this, propose Hybrid CCJT (Hybrid Collaborative Confidential Junction Trees), the first peer-to-peer framework that runs collaborative inference over Hybrid BNs confidentially with no need for central coordinator. Hybrid CCJT consists of two main components:
(i) A collaborative strongly-rooter Junction Tree. We introduce interface cliques in the tree structure, type of clique that models distributions of variables common among parties, and preserves a strong order which is key for inference hybrid BNs, detailed Section 3.3
(ii) A confidential inference protocol for Hybrid BNs. To perform inference, parties propagate their clique partial results over the built strongly-rooted JT, starting with the continuous variables and ending with the discrete ones. Within this protocol, we present two novel procedures to ensure confidentiality during inference when processing interface cliques: Collaborative Inference and Collaborative I Inference. In particular, the former eliminates more variables before pessage-passingg than the sfate-of-the-ar CCBNet for discrete BNs, drastically reducing communication costs. The latter enables handling continuous variables, detailed in Section 3.4
Code is available ggithbb.oom--ghhcaahyyi----e-eee Our contributions are as follows:
We propose Hybrid CCJT, the first framework for Collaborative Confidential Hybrid BN inference.
We design a collaborative strongly- JT protocol, the first method to build collaborative, strongly-rooted Junction Tree based on interface cliques to run accurate inference in hybrid domains.
We prove the correctness of this novel strongly-rooted JT structure and its construction protocol.
We propose a confidential inference protocol, which uses HE and SMC to make parties' BNs collaborate, while protecting their structure and parameters of and posterior of private variables.
We evaluate our method against nine different datasets and report improvements compared to non-hybrid confidentiality-preseeee
methods. We obtain 32% average decrease in mean squared error and up to 86x reduction in communication costs. Furthermore, our method uses up to 331 smaller communication costs under purely discrete scenarios.

---

2 Related Studies and Background
2.1 Hybrid CLG Bayesian Networks
Bayesian Networks T are directed acyclic graphs whose nodes are random variables and whose edges correspond to the direct influence of one variable on another. The conditional probability distribution (CPD) of variable X given its parents, PIX pa(X)] is called its factor. The tables that summarize such probability distributions are called CPD tables. Traditionally, BNs only allow variables to be discretely valued (7]. However, such a requirement limits the representation quality for variables that naturally describe continuous-valun data [3]. Moreover, exact inference in discrete BNs is harhard, while other continuous representations, such as Conditional Linear Gaussian (CLG), can perform exact inference with polynomial cost in the network size 771.
Hybrid Bayesian Networks 13] allow to model probability distributions with both discrete (A) and continuous variables (T). One of the most common classes of hybrid models is the set of Hybrid Conditional Linear Gaussian (Hybrid CLG) distribution models [4]. In this class, continuous variables are Gaussian-shaped and cannot have any discrete children. The factor of continuous variable X E I with discrete parents مa and continuous parents ær is given by:
[ = N(X;a((s) + =( (=a))
where and B are the coefficients that depend the discrete state combination of 2Λ. If the state combination of a is fixed, X is Gaussian-shaped Otherwise, f(X) is mixture of O(2) Al) Gaussian distributions. In general, even representing the correct marginal distribution in a Hybrid CLG network requires space exponential in the size of the network 77]. Furthermore, even approximate inference for simple model structures such as polytrees NP-hard in Hybrid CLG networks [5]
Inference Lauritzen and Jensen [11] propose an algorithm to carry out accurate inference in Hybrid CLG BNs by leveraging strong elimination order. In such an order, continuous nodes get eliminated from the graph before discrete ones [12..
2.2 Collaborative and Confidential Bayesian Networks
Collaborative confidential BNs aim to hide each party's own BN structures and parameters from one another during distributed inference. CCBNet S is the current state-of-the-art for collaborative confidential BN inference. Networks) and SAVE (Share Aggregation Variable Elimination). CABN privately performs alignment of factors of common variables, while SAVE performs distributed inference based on a Variable Elimination and BN merging scheme inspired by Del Sagrado and Moral [16] and Feng et al. [17].
based on two protocols: CABN (Confidentially Augmented Bayesian
Despite being significant step forward in the Confidential BN literature, CCBNet falls short in several aspects. It only supports discrete variables, has high communication costs, and reveals marginals for some private variables from peers to the party executing a query.
Multiply-sectioned Bayesian Networks 18 proposed a method for distributed inference on Bayesian Networks leveraging Belief Propagation. However, these methodologies require ana- lyzing the shared BN structure, breaking confidentiality requirements. Furthermore, loopy belief propagation entails sending more messages (due the iterative nature of the algorithm) and is proven to converge only on graphs with at most one cycle [19].
3 Hybrid CCJT
3.1 Preliminaries on Junction Trees
Exact inference in discrete BNs is NP-hard as its cost grows exponentially in the network's number of variables (nodes). A widely used technique to make inference on larger instances tractable is to build Junction Tree 9 Similarly to tree decomposition, the original network is decomposed into a tree-like graph where each node contains a sub-graph of the original BN-called clique. After computing inference in each small clique (e.g.. by variable elimination), Belief Propagation (7). message-passing algorithm proven to converge in linear time on trees, can combine the results. As such, the size of the largest clique bounds the cost of inference.
3

---

In Figure 2 we showcase the popular ASIA network with nodes as an example. Here, running Variable Elimination requires Ra 2 operations to perform exact inference. contrast, on the corresponding Junction Tree in Figure 2/, the size of the largest clique (4 in our example) bounds the inference cost. Thus, it requires only ~ 2 operations.
E,O.E,3
Formally, Junction Tree of a Bayesian Network over variables X' with set of factors & is computational graph whose nodes C, termed cliques, are tuples (X; _ X' S D). Edges, termed separators, are associated with a set of variables called sepse! Stj = X, O X,. These edges connect the cliques to form tree. To be valid, a Junction Tree must satisfy the following rules:
(a)
(b)
Figure 2: Construction of a Junction Tree: (a) Example of discrete Bayesian Net- work. (b) A corresponding Junction Tree consisting of 4 cliques.
Family preservation: Each factor ø E o must be associated with one clique C₁ such that Scope[ø] S
X;.
momning-interseeen property: For each pair C2 and Cy. every clique on the path between them contains X; ( X--
Exact inference can run on Junction Trees via message passing schemes exchanging factors with joint probabilities, one of the most popular being the sum cum-product algorithm [1O]. Let us define the potential of each clique C, as (C,) Il,,e0: øj. Sum-product requires cliques to send messages
through the tree towards a root as follows:
PC-c, =
Σ D
((C.)
II
MC
EnEnCCC))
where MCC+CC message sent from clique Ck to clique C₁ in the form of a CPD table with scope Sk.i. Σ represents the variable marginalization operator: a fundamental operator of BN inference which removes a of variables from factor. The outcome of such an operation is called marginal.
Strongly-Rootes Junction Trees Regular Junction Trees lack a strong elimination order 12 They break the requirement of strictly eliminating continuous nodes before discrete nodes for correctly running inference on Hybrid CLG models.
Definition 3.1. A Junction Tree over a set of discrete variables A and CLG variables is strongly- rooted if it has distinguished clique Cr, called strong rool, such that for every couple of neighboring cliques (Ci, Cý), with C2 being closer to Cr than Cj. holds that:
C. CnCc, CA V Cr
Lauritzen and Jensen 11 shows that strongly-rooted Junction Trees allow computing exact posteriors of all discrete variables (strong marginalization) and exact first and second moments of all continuous variables (weak marginalization) via multivariate Gaussian approximation.
Party
Shared Interface
Party
Party
Party 2
Party 1
Shared Interface
Party
Colloberative Inforrnce
Local PC X
Collaboruive
Local PV/D
T2 X Y.C.
Tr
[X:C]
B
D
Query CAB.C.IN--
Colinberative C. Didh
A
PNN n.c.
(C.A5
(a) Partics' input BNs
(b) Collaborative JT
(c) Confidential Inference Protocol
Figure 3: Overview of Hybrid CCJT. Red represents discrete, blue represent continuous. (a) Parties' secret BNs are taken as input. (b) Junction Tree is constructed. (c) Confidential inference is performed. Arrows represent input.

---

3.2 Hybrid CCJT Overview
Hybrid CCJT is framework to run collaborative, confidential queries in Hybrid CLG domains. It uses two main protocols: (i) Collaborative strongly-rooted JT Protocol for parties to collaboratively
generate a strongly-rooter Junction Tree; and (ii) Collaborative Inference Protocol for parties to jointly perform exact inference leveraging homomorphic encryption (HE) [20] and secure multi-party computation (SMC) [2 to preserve confidentiality. Figuure 3 showcases the steps of Hybrid CCJT. Further, Appendix H provides a detailed diagram that explains the steps of inference in a two parties
scenario.
We assume that variables from different parties have the same name only if they represent the same concept, and that considering the distinct parents for the same node across parties to be independent reasonably approximates the ground truth. We model the joint probability distribution as the union of each party's s BN. Our adversarial model includes semi-honest parties that follow the protocol while trying to abuse gained information [22] but do not collude. No trusted third party is assumed. The goal is to protect the structure and parameters of parties' BNs, and to hide marginals and posteriors of private variables. We prove our confidentiality achievements in Appendix
In centralized settings, Junction Tree construction requires using algorithms that analyze the network structure 77]. Doing so goes against our confidentiality requirements and does not consider minimizing communication costs. Our collaborative strongly-rooted JT Protocol allows the construction of strongly-rooted Junction Tree without sharing such confidential information. In this regard, two key novel steps are the construction of interface cliques across parties and modeling common variable distributions. Centralized inference requires parties to reveal the network structure and marginals within the interface clique. Our Confidential Inference Protocol allows computing the outcome of this inference without disclosing such sensitive information.
3.3 Collaborative Strongly-Roode Rootted JT Protocol
In centralized settings, building Junction Trees involves algorithms that analyze the full network structure [7]. Such an approach goes against our confidentiality requirements and does not account for the demand to reduce communication costs. Contrarily, our Collaborative Junction Tree protocol builds a strongly-rooted Junction Tree without sharing confidential information. First, we construct tree cliques and separators to define an elimination order valid for hybrid inference. We then model the posterior distribution of common variables by a process we call alignmens. The assumptions we make for providing a correct alignment are as follows: (i) variables have the same name when defining identical concepts; (ii) same variables have the same discrete states or continuous measurement units.
Every party defines an input BN (either Discrete, Continuous, or Hybrid CL.) describing their knowledge (Figure 3ap. Network structure and parameters can be defined by human experts or deduced via structure and parameter learning methods (23). Before running the protocol, we find the set of interface variables I (i.e., the common variables among parties). For confidentiality, we do so via Private Set Intersection (24.. The Junction Tree's 's definition relies on each party's 's input network and the global set of interface variables.
Junction Tree construction We construct the collaborative Junction Tree by building each party's ss local discrete and continuous cliques and their corresponding shared interface cliques. Finally, we connect all the cliques. For each party i, we define two local cliques: one discrete (A2) and one continuous (1')) (see red and blue nodes in Figure 3b). Then, we define two shared interface cliques, one discrete (Az) and one continuous (Tz). Next, we map variables and factors to each clique. Let ((X) denote the set of factors for each x E X, where X is a set of variables. Take A, and I, as the set of discrete and continuous variables in party i's input network. Let Scope, be the scope of variable x in party i's input network. Each party i defines its local cliques as A = =AA, (A,, \ Z)) and I'2 (T.U Scope, (TT \ I)). Define interface cliques as Az = (Scope [, A,l U (Tcope(T]nA) ((AnZ)) and Tz = (Scope Lau AA), (TTnZ)).
Cliques are then connected to form a tree. As shown in Figure 3b) discrete and continuous local cliques are connected with discrete and continuous shared interface cliques, respectively. The two interface cliques are connected, with the separator being the set of threshold variables T. That is, the set of discrete variables with continuous children (A O pT)))). The resulting graph is a valid Junction Tree, with any A being valid strong root. We show this in detail in Appendix A
5

---

Algorithm 1 Collaborative Inference Protocol
Input: discrete target T continuous target T evidence &, querier Q, peers P, overlap variables O
1: Cullaberatieeeommmmnotttteeeeeee (Q)
2: strong mangggalasnnlssa + Collaboraatveeissrreeeffffe P,
3: A-result Σ (A strong marginals
> Local Variable Elimination
4: T-result <- Waakkarriiilizzziooossssnngggg marginals, T")
Equation 10
5: return A-result, r-result
Interface alignment After finding consensus on how to model probability distributions of common variables in the interface clique, parties can perform all further collaborative inference queries.
To align the CPD of discrete interface clique, we use geometric average (also known as the Logarithmic Opinion Pool (25):
HEP P.[A = =a ayl": P a EnER(A) Hep P:[A = an)=: x a II P.[A= ay)"
where w; is publicly known weight of party i, which represents the confidence in its BN. Other foundational works on combining BNs [I6] and consensus belief (25) also study this approach. From Equation 4 the definition of the interface clique potential for discrete variables follows:
<(Ar)
II o( EEP
(5)
where a is the column normalization factor applied each table's column based on the alignment process. To implement it confidentially, parties allocate space in the interface variables' CPDs to account for parents managed by other parties with obfuscated names. Then, they collaboratively compute column normalization factors via HE [20]. This normalization is only applied during inference by the querying party, enabling peers to marginalize their private variables before message passing, where CPD entries are secretly shared via a multiplication-base scheme 22), enhancing both communication costs and privacy guarantees.
To align the continuous interface clique. we define the joint probability distribution of common continuous variables the weighted mixture of parties' (local) Gaussian distributions. Given fixed discrete state combination E I(T)):
P[X x]
Σ (x,N(X:14,))))000c EEP
At inference time, we approximate this distribution as a normal distribution with the same first and second moments of Equution 6
P = - Σ Wijti,s (X),
Σ,.Ee [wror, (X)]
Σ [un (px (x))))))
VEP
SEF
sEP
where 1a Σier mij4i,s(X) the mean of the PDF. The abovementioned step does not require sharing any parameters, maintaining confidentiality.
3.4 Collaborative Inference Protocol
We outline the protocol via pseudocode in Algorithm 1 and graphically in Figure 3c Let Q be the querying party. Belief propagates towards A through two steps. First, parties collaboratively run inference within the continuous domain to compute marginals over threshold variables (line I). Second, they run collaborative discrete inference (line 2). Subsequently party Q uses the computed marginals to perform the remaining part of the query (lines 3-4). Centralized inference requires parties to reveal the network structure and partial inference results inside of the interface clique. Our protocol provides a procedure to compute the outcome of this inference without disclosing such sensitive information. We discuss this in detail in Appendix B
Collaborative Continuous Inference During this step, we aim to merge parties' local continuous evidence to find the strong marginals over threshold variables. Algorithm 2 outlines its logic.

---

Algorithm 2 Collaborative Continuous Inference
Input: evidence E, parties P, overlap variables O
1: for p do
2: p.ovBelief <- PIO 8]
Di Local Continuous Inference
3: end for
4: ovBelief +- secretShare(Unpe p.ovBelief)
> Equation 8
5: for E P do
6: threshold p.r-vars pa(p.I-varr)
7: p.threshBelief + P Pithreshold ovBelief]
8: end for
Algorithm 3 Collaborative Discrete Inference
Input: evidence &, querier Q, peers P, overlap variables O
1: for P do
2: p.factors + p.threshBelief P[X Ea
3: p.msg E Lo
Local Discrete Inference
4: end for
5: Q.ovBelief < secretShare((p.nm) Vp P))
Equation 9
6: return Q.ovBelief IIxEA P[X] E
Messages from 1 to I'z are derived by each party without interaction (lines 1-3). When computing a message from I' to Аr, we aim to find strong marginals over discrete parents continuous variables. To do so, we merge the knowledge of continuous interface variables (line 4). Parties then marginalize all continuous variables to find strong marginals (lines 5-7). To merge continuous variable knowledge, parties collaboratively compute the mean and variance of EEuutiion 7
((X) Σ WX),,(X,, (X) Σ [w,or,,s + Σ SE
(8)
SP
RE
We additive secret-sharing with trusted third party preserve confidentiality 227 (line 4). Each party randomly splits its secret value into as many shares as the number of parties and sends them to each of them. Every party adds up the values it received. Finally, parties share their results and compute the sum to find the final output. To compute μu, each party i secretly shares (X). While to find a, each party secretly shares wx),(X) wi(p (X))²
Finding strong marginals requires integrating out all continuous variables while accounting for their evidences. This can be conveniently done in ([[[))) using canonical form representation (7), which we discuss in detail in Appendix C
Collaborative Discrete Inference After computing the marginals of threshold variables, parties final- ize strong marginalization by collaboratively calculating the message from AT to 22 Algorithm showeases the procedure. Each non-querying party finds posteriors over their discrete domain A,- Then, from Equation 2 and 5
we derive:
M MAr~Ag ad(A)) T
II
Σ
((A)) MAA
KEP zEAo
We show the derivation of Equation 9 in Appendix From the equation follows that each non- querying party i has to compute message m; = Ergao o(At)" PA;--> It is possible do so locally, as < ((A)) ) is the outcome of the discrete interface alignment procedure known to party i and Paa = Σrgr ((A)) can be computed without interaction between parties (lines 2-7).
Eventually, the product m; gives a factor over variables owned by the querying party Q that does not reveal any information about the posterior of variables owned by other parties. We prove the above statement in subsection B.2 To protect the content of these messages, we use a secret sharing scheme for multiplication [26]. A secret value gets split into shares distributed amongst parties. Parties perform the computation with their local share of each secret, and all aggregate their results to reconstruct the answer.

---

Table 1: Results on hybrid data: best in bold, second best Brier score underlined. Lower is better.
Datasct
Healtheare
Sangiovese
#Parties
Overlap
30%
10%
30%
10%
30%
30%
Brier
0.0496
0.036
0.0577
0,0856
0.019
0.0129
0.02746
0.015
Hybrid CCJT
MSB
4.7c+06
4.6c+05
4.8c+06
1,4c+07
0.0033
0,0018
0.00041
0.0045
Comm 4
4.7
16.6
11.4
139.7
43.5
87
219
596
Brier
0.0557
0.058
0.0651
0.128
0.0457
0.0138
0.0484
0.0125
3 States
MSB
5.9e406
4,9+05
5.4e406
1.7e407
0.044
0,021
0.083
0,0071
Comm 4
4.6
9.3
8.3
23,9
44.6
164.8
133.6
2654
Brier
0.0502
0.0578
0.0623
0.173
0.0243
0.0132
0.0476
0.013
A-CCJT
5 States
MSE
Se406
5.2e405
5.3e406
1.c4077
0.025
0,012
0.012
0,0218
Comm.
4.6
18.9
5.2
36,2
71
261.3
216.3
4174
Brier
0.0488
0.044
0.0597
0,112
0,0181
0.0129
0.0269
0,013
10 States
MSB
4.8c+06
4.9c+05
5c+06
16c+077
0.012
0.0036
0.0069
0.0049
Comm.
4.0
58.2
5.2
66.7
140.1
213.7
424.7
24013
Brier 1
0.0358
0.0568
0.0651
0.128
0.0496
0.0138
0.08769
0.0125
3 States
MSB
5.8c406
4.9+05
5.4+006
1.7e+07
0.044
0.02
0.064
0.007
Comm.
38.7
15.2
17.0
34.3
793.6
19044
20271
Brier
0,0502
0.0571
0.0623
0,172
0.0243
0.0132
0,0464
0.013
CCBNet
5 States
MSB
5e+06
5.2e405
5.3e+06
1.6e+07
0.025
0.011
0.011
0.0215
Comm 4
12.6
28.2
161.1
52.8
808.9
377341
Brier
0.0488
0.044
0.0897
0.113
0.0181
0.0129
0.0269
0.013
10 States
MSE
4.8c+06
4.9c+05
5c+06
16c+07
0.012
0.0037
0,0069
0.0049
Comm 4
9.7
135.2
9.6
560.2
154.1
1381.9
4074.6
189650
Weak Marginalization After computing discrete posteriors, following the weak marginalization procedure 7 yields the continuous ones. Thus, for any continuous variable X we get:
P[X]=N[
Σ
PlTo- (X),(X),
E
PlTo = s)fo + (H 14--am px())) (10)
#ED(AQ)
(EN(AQ)
where μ = ExEN(Ta) PiTo = s|p, (X) is the mean of the PDF in Equation 10
Proof of Confidentiality Below, we outline why both of Hybrid CCJT's collaborative phases preserve confidentiality during inference:
1. Collaborative Discrete Inference CPD tables of interface variables (say, X) are "aug- mented" allocate space for all parents of pa(X), some of which might be private. This step is required to ensure the correct outcome of HE, CPD product, and CPD normalization as Equation We show that this step does not leak any information about such private parent variables by proving that their marginals yield a uniform distribution (1 (pa(pp())))))) ())).. Where ((X) is the of states of variable X. We prove this in Proposition B.2 Then, we prove that, after message passing, the marginal over these parent variables remains uniform. This ensures no information about such private variables leaks at inference time. We prove this in Proposition B.5 Furthermore, note that parties encrypt the names of variables and their states to enhance confidentiality further.
2. Collaborative Continuous Inference Unlike discrete variables, continuous counterparts do not require computing and correctly applying normalization factors potentially defined over private factors. Thus, continuous private variables are inherently protected during alignment. At inference time, the means and variances of shared continuous variables are calculated using multi multi-party party secret sharing, which updates the parameters of such shared continuous interface variables. This approach inherently safeguards the information of private variables, as they are not involved in the update process.
4 Evaluation
We evaluate Hybrid CCJT on nine publicly available models (see Apeendix for details) whose data structures are hybrid or discrete only. We compare against the state-of-the-art on different types of queries. Apppenix F presents additional results on purely continuous data. [ppendix I and present further experiments on computational costs of cryptographie tools and insights on communication costs respectively.
8

---

Evaluation Metrics Our experiments assess the average predictive performance for discrete and continuous target variables, and the associated communication costs. For discrete variables, we evaluate the prediction quality using the Brier Score defined as 1 Σi= Σi= (fix - =2,²² where N N is the number of queries, R is the number of target variables state combinations and f and are the predicted and actual probabilities, respectively. We use the Mean Squared Error (MSE) of predicted means compared to the actual values for continuous variables. The average CPD and CLG parameter values transmitted per query give the communication costs.
We sample a dataset from each model. Then, we assign a subset of variables to each party. Fach party receives the vertical split of the sampled dataset corresponding to its assigned variables. From these vertical splits, the parties independently learn the input BNs. The network structures are learned via 2-phase Restricted Maximization [28]. Parameters are learned via Maximum Likelihood Estimator for conditional probabilities (for discrete variables) and least squares regression models (for CLG variables) 229
Dataset We consider the models listed in Appendix G
Baselines We compare Hybrid CCJT's performance against two baselines:
the current state-of-the-art for collaborative confidential BNs. A-CCJT: a simplified Hybrid CCJT with only the discrete inference from aoooAlgrithmm 3
CCBNet g]
Since none of the baselines can handle continuous data, we discretize such variables with different degrees of coarseness, ranging from 3 to 10 states per variable. Given the complexity of discrete exact inference (7] finer discretization implies significantly higher computational and communication costs, making it infeasible to run these algorithms with many states per variable.
4.1 Results on Hybrid Data
We consider three hybrid datasets: Healthcare and San- giovese (Table Tp. followed by Mehra (Table 2}. We test Hybrid CCJT with different combinations for the number of involved parties and overlap ratios. The overlap ratio denotes the fraction of variables assigned to more than one party. For each, we run 1000 queries with one dis- crete target variable and 1000 with one continuous target variable.
Table 2: Hybrid CCJT results the large hybrid dataset Mehra for parties.
Overlap 10%
30%
Brier MSE
0.00783
0.00772
7.4e+11
4.5e+12
Comm
186
6734
Predictive Accuracy In all experiments, Hybrid CCJT
outperforms all baselines in predictive accuracy of continuous target variables with average 32% improvement in MSE compared with the best-performing baseline. When targeting discrete variables, Hybrid CCJT is either the best performing solution or the second best performing solution with performance gap always under 10-3 in terms of Brier score. The only exception is Sangiovese with 4 parties and 30% overlap, where the deficit to the best model is 0.0025 (0.0125 versus 0.015). Note that Sangiovese has only one discrete variable, and this column returns a quasi-uniform posterior distribution regardless of the set of continuous evidence. The aforementioned explains why running hybrid inference on this data does not lead to any improvement over column discretization. As one would expect, the best performing baseline is the one with a finer discretization. Using coarse representation leads to drastic performance decay compared to our implementation with an MSE 26.9 times higher, and Brier score 42.8% higher on average. In summary. Hybrid CCJT brings notable improvements when targeting continuous variables, proving the benefit of natively handling continuous data. Besides, Hybrid CCJT matches the baselines when targeting discrete variables.
Communication Costs Hybrid CCJT demonstrates superior scalability in communication costs compared all discretized baselines. Although A-CCJT achieves the lowest communication costs on the smaller Healthcare dataset, this ad advantage diminishes with larger datasets where we can start to appreciate the improved scalability of Hybrid CCJT. Under Sangiovese with all continuous variables except one, the communication cost of A-CCIT with 10 states increases significantly faster than for Hybrid CCJT, reaching up to 40 times more communicated values per query. This happens because discrete CPD tables take more space than regular continuous posteriors. Further highlighting the advantage of handling continuous data natively. While discretizing with fewer states may reduce communication costs in specific scenarios, this comes at the expense of a sharp decline in predictive performance. For example, on Sangiovese with 4 parties and 10% overlap, A-CCJT exhibits nearly 2x the error for discrete targets and x the error for continuous ones, making

---

Hybrid CCJT the most desirable choice overall. The current state-of-the tht- CCBNet, exhibits the worst communication performance across all experiments. Its worst result averages almost 190K communicated values per query against only 596 used by Hybrid CCJT (i.e., a 318x reduction). This shows how our collahorative discrete inference approach alone significantly improves communication costs. We explore this further in subsection 4.2
Large dataset Mehra the largest dataset we consider, with 4 times the number of parameters of Munin, the largest discrete dataset. While Hybrid CCJT managed to complete all experiments (see Table 2). none of the discretized baselines finished within the timeouf This is due to the heavy computational requirements of aligning large CPD tables of discretized continuous variables. Despite the size, Hybrid CCJT achieves good Brier and MSE scores. while maintaining reasonable communication costs. Specifically, for a 30% overlap ratio in the smaller Sangiovese dataset, Hybrid CCJT communicates less than a third of the values compared A-CCJT with 10 states.
4.2 Results on Discrete Data
Since scalability of communication costs significant issue for CCBNet, we emphasize the improvement of Hybrid CCJT on six mod- els with purely discrete data. Table 3) show- cases the results. On the Child, Alarm, and Insurance datasets, we run experiments with 2 to 8 involved parties, and up to 128 on larger datasets. We perform 2000 different queries for each. Since these datasets lack any contin- uous variable, Hybrid CCJT degenerates into A-CCJT. In smaller experiments, with two par- ties, Hybrid CCJT reduces CCBNet commu- nication costs by 8 times, from an average of 125 to 15 communicated values. Improvement factors further increase when the number of in- volved parties grows. In larger-scale experi- ments, CCBNet's communication costs increase significantly, reaching as high 243K transmitted values for Munin. In contrast, Hybrid CCJT maintains a low communication overhead, transmitting 735 values (331x less). As expected, we did not measure any significant difference (2 0.001) in predictive accuracy.
Table 3: Results on discrete data: communication
costs on discrete datasets. 10% overlap. Lower is better for all. Predictive accuracy difference is negligible (< 10-3).
Improvement factor
Dataset
Parttis
CCBNet
Hybrid CCJT
67
Child
157
33
4.7x
429
58
7.4x
166
15
11.1x
Alarm
1959
40
49x
1886
79
23.9x
143
17
8.4x
Insurance
42.7x
473
42
11.3x
23175
6080
3.8x
Link
4455
459
9.7x
Munin #2
128
243474
735
331.3x
5 Conclusion
Motivated by the need of collaborative and confidential inference in manufacturing settings, we introduce Hybrid CCJT, novel framework enabling collaborative confidential inference on Hybrid Bayesian Networks. By addressing the scalability limitations of existing methods and their inability to model hybrid data, Hybrid CCJT facilitates secure collaborative inference while maintaining confidentiality of party models and of private variable posteriors. The proposed framework introduces two pivotal components: collaborative strongly-rooted JT for constructing strongly-rooted Junetion Tree, and a Confidential Inference Protocol to perform such Junction Tree. Our evaluation across nine datasets demonstrates Hybrid CCJT's superior predictive accuracy (by 32% better MSE on average) at reduced communication costs (up to 86x less sent values). In [Appendix K we discuss Hybrid CCJT limitations, like possible attacks on confidentiality or the ability to handle broader class of hybrid networks, and possible solutions.
inference that leverages
Acknowledgments
We thank the ASML High- Prrch-Perfrmancc Computing team for enabling some of the analyses con- ducted in this paper. This research is partly funded by the "Priv-GSyn: Preseerin Graph Synthesis" grant (200021F, _212004)2222 from the Swiss National Science Foundation, the "DEPMAT
TTimoout per experiment is set to 24 hours, 512GB RAM.
10

---

(Data Enhanced Physical Models to reduce material use)" project (P20-22a) from the "Perspectief" Dutch Research Council (NWO) research programme, and ASM International NV. This work has also been partially supported by the Spoke "FutureHPC & BigData" of the ICSC-Centro Nazionale di Ricerea High Performance Computing, Big Data and Quantum Computing, and hosting entity, funded by European Union Next GenerationEU and by the DYMAN project funded by the European Union European Innovation Council under G.A. n. 101161930.
References
[1] Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann, 1988.
[2] Saideep Nannapaneni, Sankaran Mahadevan, and Sudarsan Rachuri. Performance evaluation of a manufacturing process under uncertainty using bayesian networks. Journal of Cleaner Production, 13:947-959, 2016.
[3] Sun Jin, Changhui Liu, Xinmin Lai, Fei Li, and Bo He. Bayesian network approach for ceramic shell deformation fault diagnosis in the investment casting process. The International Journal of Advanced Manufacturing Technology, 88:663-674, 2017.
[4] Guodong Wang. Ramin M. Hasami, Yungang Zhu, and Radu Grosu, A novel bayesian network- based fault prognostic method for semiconductor manufacturing process. In 2017 IEEE International Conference on Industrial Technology (ICIT), pages 1450-1454, 2017. doi: 10.1 00.100000CT22221779155577
[5] Alexander Ypma, Adrianus Cornelis Matheus Koopman, and Scott Anderson Middlebrooks. Methods and apparatus for obtaining diagnostic information, methods and apparatus for control- ling industrial process, January 21 2020. US Patent 10,539,882.
[6] Nathan Hertlein, Sourabh Deshpande, Vysakh Venugopal, Manish Kumar, and Sam Anand. Prediction of selective laser melting part quality using hybrid bayesian network. Additive Manufacturiing 32: 101089, 2020.
[7] Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
[8] Abele Mälan, Jérémie Decouchant, Thiago Guzella, and Lydia Chen. Ccbnet: Confidential collaborative bayesian networks inference. arXiv preprint arXiv:2405. 15055, 2024.
[9] Steffen L Lauritzen and David Spiegelhalter. Local computations with probabilities on graphical structures and their application expert systems. Journal of the Royal Staristical Society: Series (Methodological), 50(2):157-194, 1988.
[10] Prakash Shenoy and Glenn Shafer. Axioms for probability and belief-function propagation. In Machine intelligence and pattern recognition, volume 9, pages 169-198. Elsevier, 1990.
[11] Steffen L Lauritzen and Frank Jensen. Stable local computation with conditional gaussian distributions. Statistics and Computing,
2001.
[12] Anders L Madsen. Belief update in clg bayesian networks with lazy propagation. International Journal of Approximate Reasoning, 49(2):503-521, 2008.
[13] Antonio Salmerón, Rafael Rumí, Helge Langseth, Thomas D Nielsen, and Anders L. Madsen. review of inference algorithms for hybrid bayesian networks. Journal of Artificial Intelligence Research, 62:799-825 828, 2018
[14] Steffen L Lauritzen and Nanny Wermuth. Graphical models for associations between variables, some of which are qualitative and some quantitative. The annals of Statistics, pages 31-57, 1989.
[15] Uri Lerner and Ronald Parr. Inference in hybrid networks: theoretical limits and practical algo- rithms. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pages 310-318, 2001.
11

---

[16] José Del Sagrado and Serafin Moral. Qualitative combination of bayesian networks. Interna- tional Journal of Intelligent Systems. 18(2):237-249, 2003.
[17] Guang Feng, Jia-Dong Zhang, and Stephen Shaoyi Liao. A novel method for combining bayesian networks, theoretical analysis, and its applications. Pattern Recognition, 47(5):2057- 2069, 2014.
[18] Yang Xiang and Victor Lesser. Justifying multiply sectioned bayesian networks. In Proceedings Fourth International Conference on MultiAgent Systems, pages 349-356. IEEE, 2000.
[19] Roie Zivan, Omer Lev, and Rotem Galiki. Beyond trees: Analysis and convergence of belief propagation in graphs with multiple cycles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7333-7340, 2020.
[20] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In Advances in SoptologyyASIACRII 2017: 23rd International Conference on the Theory and Applications of Cryptology and Information Security, Hong Kong, China, December 3-7, 2017, Proceedings, Part 23, pages 409-437. Springer, 2017.
[21] Ronald Cramer, Ivan Bjerre Damgård, and Jesper Buus Nielsen. Secure Multiparty Computation and Secret Sharing. Cambridge University Press, 2015.
[22] Oded Goldreich al. Foundations of cryptography primer. Foundations and Trends@ in Theoretical Computer Science, 1(1):1-116, 2005.
[23] Marco Scutari and Jean-Baptiste Denis. Bayesian networks: with examples in R. Chapman and Hall/CRC, 2021.
[24] Daniel Morales, Isaac Agudo, and Javier Lopez. Private set intersection: A systematic literature review. Computer Science Review, 49:100567, 2023.
(25) David M Pennock and Michael P Wellman. Graphical representations of consensus belief. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, pages 531-540, 1999.
[26] Niki Kilbertus, Adrià Gascón, Matt Kusner, Michael Veale, Krishna Gummadi, and Adrian Weller. Blind justice: Fairness with encrypted sensitive attributes. In International Conference on Machine Learning, pages 2630- 2639. PMLR, 2018
[27] Flavio D Garcia and Bart Jacobs. Privacy-friende friendly energy weerginnt via homomorphic encryption. In Security and Trust Management: 6th International Workshop, STM 2010, Athens, Greece, September 23-24, 2010, Revised Selected Papers 6, pages 226-238. Springer, 2011.
[28] Nir Friedman, Iftach Nachman, and Dana Peér. Learning bayesian network structure from massive datasets: the "sparse candidate" algorithm. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, pages 206-215, 1999.
[29] Laura Azzimonti, Giorgio Corani, and Marco Zaffalon. Hierarchical estimation of parameters in bayesian networks. Computational Statistics Data Analysis, 137:67-99 2019.
[30] Michael Kearns, Jinsong Tan, and Jennifer Wortman. Privacy-preserving belief propagation and sampling. Advances in Neural Information Processing Systems, 20, 2007.
[31] Alessandro Magrini, Stefano Di Blasi, and Federico Mattia Stefanini. A conditional linear gaussian network to assess the impact of several agronomic settings on the quality of tuscan sangiovese grapes. Biometrical Letters, 54(1):25 42, 2017.
[32] Claudia Vitolo, Marco Scutari, Mohamed Ghalaieny, Allan Tucker, and Andrew Russell. Model- ing air pollution, climate, and health data using bayesian networks: A case study of the english regions. Earth and Space Science, 5(4):66 88, 2018
[33] David Spiegelhalter. Learning in probabilistic expert systems. Bayesian statistics, 4:447-465,
1992.
12

---

[34] Ingo A Beinlich, Henri Jacques Suermondt, R Martin Chavez, and Gregory Cooper. The alarm monitoring system: A case study with two probabilistic inference techniques for belief networks. In AIME 89; Second European Conference on Artificial Intelligence in Medicine, London, August 29th-31st 1989, Proceedings, pages 247-256. Springer, 1989.
[35] John Binder, Daphne Koller, Stuart Russell, and Keiji Kanazawa. Adaptive probabilistic networks with hidden variables. Machine Learning, 29:213-244 1997.
[36] Cristina Conati, Abigail S Gertner, Kurt VanLehn, and Marek Druzdzel. On-line student modeling for coached problem solving using bayesian networks. In User Modeling: Proceedings of the Sixth International Conference UM97 Chia Laguna, Sardinia, Italy June 2-5 1997, pages 231-242 Springer, 1997.
[37] Claus Skaanning Jensen and Augustine Kong. Blocking gibbs sampling for linkage analysis in large pedigrees with many loops. The American Journal of Human Genetics, 65(3):885- 901, 1999.
[38] Steen Andreassen, Finn Jensen, Stig Kjær Andersen, B Falck, Kjærulff, Woldbye, AR Sørensen, A Rosenfalck, and F Jensen. Munin: expert emg assistant. In Computer-aided electromyography and expert systems, pages 255-277. Pergamon Press, 1989.
[39] Juliane Schäfer and Korbinian Strimmer. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical upplications in genetics and molecular biology, 4(1), 2005.
[40] Marco Scutari, Phil Howell, David J Balding, and Ian Mackay. Multiple quantitative trait analysis using bayesian networks. Genetics, 198()):22--377, 2014.
[41] Ahmad Al Badawi, Jack Bates, Flavio Bergamaschi, David Bruce Cousins, Saroja Erabelli, Nicholas Genise, Shai Halevi, Hamish Hunt, Andrey Kim, Yongwoo Lee, et al. Openfhe: Open-source fully homomorphic encryption library. In proceedings of the 10th workshop encrypted computing applied homomorphic cryptography, pages 53-63, 2022.
[42] Uri Lemer, Eran Segal, and Daphne Koller. Exact inference in networks with discrete children of continuous parents. In Proceedings of the Seventeenth conference on Uncertainty artificial intelligence, pages 319-328, 2001.
Nomenclature
Set all discrete variables
A Local discrete clique of party
Az Interface discrete clique
A
Set of discrete variables owned by party Set of all continuous variables
T
Local continuous clique of party
Tz
Interface continuous clique
T,
Set of continuous variables owned by party
Set of the parties
N(X; p1 o) Normal distribution over with mean μ and standard deviation
T
Set threshold variables, i.c., A O pa(T)
U(S) Uniform distribution over set
((X) Set of states over variable
*(S) Set of factors over variables in S
((X) Factor of variable X
CPD, Entry i.j of Conditional Probability Distribution
13

---

pa(X)
Set of parents of variable X
P[X
E
Probability of variable X conditioned on evidence E =
P[X
E]
Probability of variable conditioned on variable
P[X]
Probability of variable
A
Strongly-rooted Junction Tree Proof
Here, we prove the validity of the strongly-rooted Junction Tree T built following the Collaborative Junction Tree protocol. We first prove that 7 is a valid Junction Tree. Then, we proceed to show that any local discrete clique (A)) is valid strong root for T.
A.1 Family Preservation Property
Definition A.1. A Junction Tree T over set of factors & is said to be family preserving if each factor & is associated with one cluster c; T such that Scope[ø] S Xi. With X; being the set of variables over which G is defined.
Since in Hybrid CLG networks, continuous variable have discrete child. It follows that: Lemma A.2. Scope[A] C A
Let A, and I; be the set of discrete and continuous variables in party i's input network. It follows that:
Lemma A.3. Vr (A, UT;) \ I all parents of a are in A
Lemma A.4. (I =I
By definition, I the set of all variables that belong to more than one party. From this, Lemma A.5 and Lemma [.6
Lemma A.5. (A; I) (A, = Lemma A.6. (T, (Z) (T, II) = 0vi.j eP
Clearly, the set of discrete and continuous variables disjoint.
Lemma A.7. A Anr=1 Proposition A.8. is family preserving.
Proof. In T there are four possible types of cliques:
A = (A:, (AA Z))
1 (T' U Scope, (T ((T; [])
.
Az ((Scope n,, A, U (Scope (T) A), ((A
Iz = (Scope r (Scope( A, (T
Lemma A.2 Lemma A.3 Scope A
A is family preserving.
Trivially, Scope(L', I] C U Scope[Lrj] 1.1% is family preserving.
Trivially, Scope/A n] S (Scope A:) U (Scope(T) nA)
Az is family preserving.
Trivially, Scope[r Z] _ Scope [; r.] U (Scope T| n
.r is family preserving.
From Lemma A.4 + (Ulep A, \ r,
Each factor is associated with at least one clique.
14

---

Lemma
A.5
(1)) Vi
Lemma
= Vi
Trivially,
(Z)n(An))=0) ViEF
Trivially,
(
[)n(TTI)= =0 WEP
Lemma
A.7
7 (Tn)))(AnD)=
Lemma A.7
+
(T; ZI = WiE
Lemma
A.7
+ (A, DAIITTI
WiEP
Lemma
+ (A, ( I Vi,j
.aEc factor is associated with most one clique.
T is family preserving.
A.2 Running Intersection Property
We recall all cliques definitions given by construction following the Collaborative Junction Tree
protocol in subsection 3.3 Let (c) be the scope of variable in party i's input network.
Lemma A.9. D = A
Lemma A.10. Åz Scope n. A. (Scope[[) A)
Lemma A.11. I r, U Scope, (Ię) Lemma A.12. FT. ScopeiN, r:J (Scope(T) nA)
We recall the definition of the running intersection property:
Definition A.13. A Junction Tree T satisfies the running intersection property if, for all pairs of cliques (CI,C) T, C2 C2 also in every clique the path in T between C₁ and Cz.
From Definition A.13 it trivially follows that:
Lemma A.14. Every pair of neighhoring cliques satisfy the running intersection property.
Proposition A.15, T satisfies the rumning intersection property.
Proof. By Lemma A.14 all pairs of neighboring cliques satisfy the running intersection property. In T, there are five possible pairs of non-neighboring cliques:
Pi = (42. A with Az the only clique in the path.
P2 = (T2, r2),, with Pz the only elique the path.
P3 = (4), Izz, with Az the only clique in the path.
P4 = (T' Az), with T the only clique in the path.
*
Ps = ( )), with Az and I'z in the path.
Lemma A.10
A n n A
P1 satisees the running intersection property.
Lemma A.12
r n CT
P. satisfies the running intersection property.
Lemma A.9
Lemma A.12 +
A nTz S Scope(T) 04 A
Lemma
A.
ScopelT'r
Az
P3 satisfies the running intersection property.
Lemma A.11
Lemma A.
I n Az C Scope[L'] NA C
Lemma
10
Scope [
P. satisfies the running intersection property.
Lemma A.9
Lemma A.11 +
A n nr S Scope 0A
Lemma
A.TO
AT Scope []]
Lemma
A.12
C Scope A
Ps satisfies the running intersection property.
15

---

7 satisfies the running intersection property.
A.3 Strong Root Property
Following Definition 3.1 we prove that A a strong root of T.
From Lemma 1. and Lemma A.12 it follows that:
Lemma A.16. r 0 CTynA
From Lemma A.10 and Lemma A.2 it follows:
Lemma A.17. Âz CA
Proposition A.18. A a strong rool T.
Proof. In T there are three possible pairs of neighboring cliques:
Sj: (I"; 1iz), with I'z being closer to the root.
Sy: (T',, Az), with Az being closer to the root.
Sy: (A): АT), where either can be closer to the root depending on which party requests the
query.
Lemma
A.16
(T) 6 -
:1
Equation
Lemma
A.17
nTz CA
S.S2
Equation
Lemma
A.17
A
0 Ar CA
..Sa
Equation
A. is strong root of T.
B
Confidentiality of Hybrid CCJT
Here, we present the achievements in confidentiality preservation of Hybrid CCJT. Before proceed- ing to the discussion, we introduce some auxiliary concepts.
Confidentiality in Bayesian Networks When dealing with confidentiality in Bayesian Networks it worth noticing that some information leakage is typically inevitable. In fact, simply knowing the result of an inference query could be used to reverse-compute further information. For instance, [30] studied crivacy-preservee
applications of algorithms such Belief Propagation and Sampling. In Kearns' application, messages in the Belief Propagation process must be protected. However, leaf nodes can always reverse compute incoming messages from their neighbors from solely the inference outcome (30] To formalize this notion, 30 give the following definition of protocol:
Definition B.1. Let be any protocol for the parties to jointly compute the value y = f(11, 2x)) from their bitit private inputs. We say that II is that party can compute polynomial time in n following the execution of II, they could also compute in polynomial time given only their private input æ; and the value
if for every S i k, anything
In this section, we show that Hybrid CCJT abides Definition B.1 structure, parameters, and posteriors of other parties' BNs revealed after performing query with Hybrid CCJT.
That is, no information about
B.1 Structure and Parameters
Junction Tree Construction Hybrid CCJT defines collaborative junction tree without requiring any information disclosure between the parties. In fact, every party knows which of their variables to
16

---

allocate in the local and interface cliques. This is sufficient to carry out inference as in our protocol, but no party knows anything about the content nor the structure of other parties' local cliques.
Discrete Factor Alignment Parties' factors are updated for discrete interface factors only. As anticipated in subsectionn 3.3] this is done via securely computing the column normalization factor Let ,pD.,; denote party CPD column j. then a = O CPD+,lli is computed via homomorphic encryption. Our implementation relies on the CKKS (20] scheme for floating-poin addition and multiplication. This procedure allows to apply the geometric mean correctly without disclosing the parties' CPD entries. As a matter of fact, after computing the (scalar) normalization factor a, is not possible to decompose it into the cP o 2 values used to compute it in polynomial time, abiding to Definition B.1
B.2 Posterior of Private Variables
Hybrid CCJT executes two subroutines to perform collaborative inference: one on the discrete domain, one the continuous domain. We show that party gains any information about the posterior of other parties' private variables (i.e., that are not part of the interface).
Posterior of Discrete Variables We ensure that only the querying party handles the normalization process of interface factors. As shown in Equation 9| this allows each party to marginalize all variables that are not shared with the querying party prior to message passing. As such, the merging procedure will only reveal the posterior of the interface variables, which are owned by the querying party.
Let X be a discrete variable owned by two parties A and B, with A being the querying party.
Let pa, (X)(X)
(X) x) be the set of parent variables of X owned by party
Let PA[X], [X] be the probability distribution according to A before and after the message passing procedure respectively.
We show that after performing factor alignment (Proposition B.2 and mhessage-passingg (Proposition B.5). the querying party does not gain any additional information about the true probability distribution of variables in pX))
Proposition B.2. Pa]pap(X)) = U(S((pa (X)))
Proof. By construction, PA obtained by duplicating each entry to allocate space for variables in pag (X). It follows that:
Σ
Palpag(X) =
= =1== [ppa (X))
Vi E (Xpa
(11)
(X))
Thus.
PA =x] = (((((pap(())
(12)
Let S = pa (X) (X) pap (X)
Let S' = 5 (X).
The following a corollary of Proposition B.2
Corollary B.3. Ejes' PA[S j. [X)a((X]]]
Vi E (X)a
From Equation it follows:
Lemma B.4. PA[S] PA[S]: Evorepag()) PA[S]
Let P/- = =vvarepyy PB Proposition B.S. Palpap U(S((pap = Pa (x u({(pap(x)))
Proof.
Phlpag(XX) (X)= =]= Σjen(s) (X)= PA[S jl
=
Vi E ((pag(X)) (13)
ExEn(s) PAIS
17

---

PA(S j,t].
EjEa())
LEE (x)) PB.S - j,
E (((ppp())()) (14)
ME PaLS - ki ml
E
mmns(pan (X)) PB(S = K, nl
PA(S" = í1.
Ejea(s))
[12(pan (X))
PB =
vi ((pa((X)) (15)
PA =k]
EvEN(s")
P = k)
PA (S' = 5|. P P =
EjEn(s))
Vi ((pag(X))
(16)
[(pa
PA(S' = k]-
EnEn(S))
P = k)
1
Wi ((pap(X))
(17)
(X)a
Thus,
P
(18)
L
Generalization to N parties Proposition B.5 can be generalized to N parties by substituting (x))()g(()) Sa[S=10 with IIEE)(()) (x)) P.[S i, and P3 with ILEEA(A) P/-
Posterior of Continuous Variables As anticipated in subsectiion 3.3 continuous variables mean and variances are computed via multi-party secret sharing updating the parameters of continuous interface variables. This process inherently protects private variables' information as they not involved in this update operation.
C Canonical Representation of Conditional Linear Gaussian CPDs
common representation used for CLG BNs is the canonical form 77. which represents CLG factors log-quadratic form exx()(XX) where Q is some quadratic function.
Definition C.1. A canonical form C(X; K, f. g) is defined as
C(X;K,h, g) = exp (- x-xxTXx***Xx
(19)
We can represent every Gaussian in the canonical form
N(r C(K,h,9)
(20)
where
K k= h h-Σ マリ T -ー log (22)
(21)
C.1 Operations on Canonical Form
Factor Product The product of two canonical forms over the same scope can easily computed as:
C(Ki, h1, C(K2, h2, g2) C(Ki K2, h h2, 91 + 922
(22)
Whenever two factors not defined over the same scope, they be expanded by simply padding the K, h parameters with zeros.
Factor Marginalization We can marginalize canonical form onto subset of its variables. Let C(X, Y; K, h, g) be some canonical form over (X,Y Y) where:
Kxx
Kxy
hx
K
(23)
Kyx
Kyy
by
18

---

The marginalized factor """,,",,'',g')) SC(,, g) dY is computed as: hx - Kxy Kyyhy,
K' = Xxx - Kxy g - log [ 2 -
(24)
Factor Reduction Finally, is possible to reduce canonical form based on a certain evidence. Let C(X, K; K, h, g) be some canonical form over {X, Y] represented as in Equation 23 Then, setting = y results in the canonical form C(;; K", H., g'),, where:
K' Kxx, N'' = hx KxYy. g' g hyy 20" Kyyy
(25)
Computational Costs Importantly all factor operations can be done in polynomial time in the scope of the factor. In particular, the product or division of factors requires quadratic time. Factor marginalization, which requires matrix inversion, can be done naively in ((N9), and more efficiently using advanced methods.
C.2 Canonical Form in Hybrid CLG BNs
hybrid BNs, using both discrete (A) and continuous (I) variables, factors with both types of variables can be represented using a canonical table 77.
Definition C.2. A canonical table & over D, X with D C A and X C I is table with entry for each d E (D) where each entry contains a canonical form ((d) = C(X; Ka, ha,9a).
Note that the canonical table representation subsumes both the canonical form and the table factors used in the context of discrete networks. For the former, D) 0, have only single canonical form over X. For the latter, X = 0, the parameters Ka, ha are vacuous, and we remain with a canonical form ((d) exp(9a) for each entry.
D Strong Elimination Order Example
alimination
elimination
X,Y,O
XYY
GnOEE
CAO,ET <
G.O,ER Er x
B
C
C,D.A
X,0,D
GOGEAV
x
D
D,
(a) An example Bayesian Network
(b) Two possible elimination orders over the network. Root cliques are highlighted with thicker borders.
Figure 4: (a) shows an example Bayesian Network. (b) shows two possible elimination orders over Only the left one is strongly-rooted, allowing accurate inference in hybrid domains.
E Collaborative Inference for Discrete Query Messages
As introduced in Section aim at finding P A2 Following the sum- cum-product message- pesssag-ppasing formulation EEquatin 2) [00:
when performing collaborative confidential inference (Algorithm 3
MA PAI-A2
((Az)
II
PAi-Az
(26)
zgAq
(EP(()
We follow the formulation based the geometrie mean for o( ((Ar) Equation 5) and rewrite the equation:
MAr+ = Σ a II
II (EP((Q)
PA
(27)
TEP
19

---

Σ rgAq
ad
A mq
II 6(AZ) sEP
(28)
Since the marginalizzaton operator does require eliminate any variable in Aq. rewrite the equation as:
PAI-~a (6(4))
EI o(A))
(29)
REAQ
Furthermore, when multiplying CPD tables, variables that are not common in both tables can be marginalized prior to the product without affecting the result.
MAI+ a ma
II Σ (EP)(Q) TEAQ
((A)) 10 Hay- -Az
(30)
Results on Continuous Data
Table 4: Result continuous data: mean squared errors.
Dataset
Ecoli70
Magic-Niab
Hybrid CCJT
0.070
0.287
States
1,411
0.521
Discretized BN
States
0.367
0.533
10 States
0.368
N/A
Here, we extend our experiments on purely continuous datasets, i.e., Ecoli70 and Magic-Niab and summarize the results in [Table 4] Since these datasets do not contain any categorical variable, we only run collaborative continuous inference as shown in Algorithm 2 Our baseline the exact inference discretized datasets with different levels of coarseness. Duc poor scalability of CCBNet and A-CCJT when dealing with discretized datasets, we use a centralized discrete network instead with no communication cost. This provides upper bound the predictive performance of a discretization--ue approach. Under both experiments, we ran 10000 queries, with parties and 10% overlap.
Across all experiments, Hybrid CCJT achieves the highest predictive accuracy. Under the Ecoli70 dataset, Hybrid CCJT attains MSE of 0.07, which is five times lower than the optimal discrete counterpart and 20 times better than the non-optimal one. Under the Magie-Niab dataset, our model achieves an MSE of 0.287, outperforming the discretized counterpart, which has an MSE of 0.521. Furthermore, the discretized model failed to run inference a reasonable amount of time with 10 states on Magic-Niab.
G Datasets Details
Table Overview used datasets.
Type
Dataset
#Discrete nodes
#Continuous nodes
#Ares
#Params
Source
Healthcare
3
4
9
42
Hybrid CLG
Sangiovese
14
55
259
Mehra
8
16
71
324423
32
Child
20
25
230
33
Alarm
37
46
509
Insurance
27
52
1008
35
Discrete
Andes
223
338
1157
36
Link
724
1125
14211
37
Munin #2
1003
1244
69431
38
Ecoli70 Magic-Niab
46
70
162
39
Continuous
44
66
154
40
In Table 5 we provide the characteristics about the datasets used in our experiments. We used three Hybrid CLG datasets (Healthcare, Sangiovese, Mehra), six discrete datasets (Child, Alarm, Insurance. Andes, Link, Munin #2), and two CLG ones (Ecoli70, Magic-Niab).
20

---

H Hybrid CCJT Diagram
To graphically showcase the Hybrid CCJT inference pipeline, Figure and 6 provide a detailed flowchart of collaborative inference with two parties.
Lucal Margiuaizattnn
Local Clazeeis Infweece
Party
Local Comtimnu
Inference
As Tjá, A]J,,orrenlap ovenlap) Pforenlapl
Psfoerraapi threshold,
Secret
E Pioeeree 4)
Plovenag
Party 2
Local Norghallation
Local Diserete inference
Local Comtimunus Inference
PTTIA RTTA,,malll menlayl Pleverlopl
Pipheesteld
7sjorenlpp shreehold,
Figure 5: Flowchart of Hybrid CCJT collaborative continuous inference procedure with two parties.
Local Discrete inference
Party
P. [overlap] Mhreshuld, 4]
Pt (overtopp
Scorct Sharing
Pl
Ploveriop)
Local Disrrcee Nargholizatin on
Party 2
Psjnerdapp Mireshuld, J]
Local Discreto Inference
Pajmerkay
PAAA overlapl Plpoeriayi
Local Continuces Merg nol Lotton
Figure 6: Flowchart of Hybrid CCJT collaborative discrete inference procedure with two parties.
We will prepare more detailed explanation of the inference protocol in the methodology section and a more detailed pseudocode and diagram to be included in the appendix. The inference is divided in three steps (of which two are collaborative) and follows the logic of the Lauritzen algorithm [Л].
Collaborative Continuous Inference The first step requires marginalizing continuous variables and finding the strong marginals over the threshold variables. Recall that threshold variables are the discrete variables with least one continuous child.
short, we aim at computing P[T e]. where T is the set of threshold variables and I' = e is some continuous evidence. To do so, parties collaboratively compute the updated parameters of interface continuous variables given the set of continuous evidences as in
((X)=
Σ (X),(X), (XX)= SEP
Σ [w:o,; Σ [w: (u (X))² REP
SEP
where the parameters H4; and o; are obtained locally by computing P[ c). Once these new parameters obtained, they can help compute the marginals over the threshold variables by just marginalizing all the continuous variables.
Collaborative Discrete Inference Once parties find the marginals of threshold variables, the querying party needs to find the strong marginals over the remaining discrete variables. For this, parties compute P[ A e). the posterior of discrete interface variables given their discrete evidence. The querying party receives these posteriors and merge them via secret sharing to find the geometric mean:
P|AS = II PM
TEP
The querying party can then use this factor to update its local probability distribution via sum-product message passing [10) (Equation 2).
Local Query At this point in the protocol, the querying party can find the posterior of any variable owns. For discrete variables, the posterior found by marginalizing non-target variables, as in normal junction tree. For continuous variables, the posterior is found by marginalizing all non non-threshold variables and performing weak marginalization:
P|x]=N(x;
Σ SEN(AR)
P(Te = xX),(X);
Σ sEn(Ag)
P(Te = <)(((XX) + (o-x((X))²))
21

---

I Computational Costs of Cryptographic Tools
The computation complexity of homomorphic encryption in solving an overlap is O(L N (Ng(N)), bounded by L chained multiplications of a size N ciphertext. L is the maximum number of parties in an overlap. N is a power of two (in the range 2 - derived from the desired precision, the maximum output scale, and Successive multiplications increase complexity linearly [20]. The cost of single multiplication is linearithmic [4]] In our implementation, solving private set intersection between two parties (m (o)(m)) time, where m is the maximum number of variables among the two parties.
The protocol relies private intersection, secret sharing (addittive/rroduct-basell and homomor- phic encryption (HE) at different points in its lifecycle. Among the enumerated cryptographic tools, only HE involves significant computational overhead, while the primary bottleneck for the others is network latency. HE is used exclusively during discrete factor alignment. The results below split the HE computation time across:
1. Context Generation: Generation of HE keys and parameters.
2. Encrypted Column Summation: The actual HE-encrypted computation; used to compute column normalization factors (i.e., in Equation 4).
3. CPD Combination: Alignment of party CPDs based on the HE-computation outputs.
We report the per-factor mean and standard deviation of each HE substep and the total time required for all factors.
Table Results on discrete datasets (10% overlap)
Dataset
#Parties
Context Gen. (ms)
Col. Sums (ms)
CPD comb. (ms)
Total (ms)
153+
18.7 0
0.27 0
176
Asia
644 0
802 0
0.53 0
1449
2155+t
767 0
0.746 0
2926
1364 5.7
514011: 82611
9.57 14.6
206191
Alarm
4
672 50
36394 4780
0.749 0.4
17250
8
1900 150
96745+ 134754
4.19 4.69
394601
143
1441 84
0.33 0.034
577
Child
703 26
17151 9176
1.78 0.6
35713
1984 57
83010 79570
3.9 2.9
169999
Link
64
133+ 7
324.9+ 381
0.4+ 0.1
431291
As expected, substep 2, i.c., encrypted column summation, is the most come-consuming one, taking up 96 seconds in the worst-case scenario, but remains under second in 6 out of 10 experiment scenarios. Its time requirements increase overall with the number of parties sharing factors. In contrast, substeps and i.e., context generation and CPD combination, and one millisecond, respectively. Moreover, we note that the HE procedure represents a one-off setup cost, after which parties may perform unlimited queries until of them updates their local network.
in under
second
When dealing with hybrid domains, notice that encryption takes significantly more time the discretized variants. While Hybrid CCJT maintains the entire encryption process under one second for both Sangiovese and Healthcare datasets, A-CCJT's encryption cost grows as the discretization coarseness is reduced (i.e., more states). These results also reaffirm the gains in memory and communication costs of natively handling CLG variables compared discretizing them.
J Communication Costs Analysis
We repeated the experiments from Table 1 with new variable splits to derive further insights in the communication costs of Hybrid CCJT. Table isolates the communication costs of Discrete and Continuous collaborative inference. This allows us to better analyze the communication costs
22

---

Table 7: Results on hybrid datasets (30% overlap)
Dataset
#Partics
Context Gen, (ms)
Col. Sums (ms)
CPD comb. (ms)
Total (ms)
Healthcare (Hybrid)
2
127.1 0
26.1 0
0.23.1 0
154
Healthcare (Discrete, 3 states)
2
1424: 8.21
82 67
0.33 0.05
675
Healthcure (Discrete, 5 states)
71.354 13
71.42 62.4
0.2711 0.03
641
Hcaltheare (Discrete, 10 states)
139.7# 10.2
165.5# 152.6
0.316 0.04
918
Healthcare (Hybrid)
646 6.04
2444 90.5
0,434 0.02
1785
Hcalthcare (Discrete, 3 states)
678. 478.46444 49.3
300.2 139.2
0.482 0.06
2940
Hcalthcare (Discrete, 5 states)
4
682.9+ 46.7
3007.8= 3660
0.602+ 0.295
11076
Healtheare (Discrete, 10 states)
677+ 51
5956++ 7221.4
0.86+ 0.48
19905
Sungiovese (Hybrid)
2
127. 0
12.11 0
0.25
141
Sangiovese (Discrete. 3 states)
2
141: 9.7
827:1: 1080
0.461 0,21
2906
Sangiovese (Discrete, 5 states)
139.8+ 9.2
710.3+ 905
0.403+ 0.18
2553
Sangiovese (Discrete, states)
141:: 11.9
621: 489
0.424 0.09
2289
Sangiovese (Hybrid)
1274:
12.12
0.25
140
Sangiovese (Discrete, 3 states)
139
813.7# 1063
0,434
2861
Sangiovese (Discrete, 5 states)
4
659 019.3+ 49.6
188826. 7 239278.3
13.2+ 16.4
758015
Sangiovese (Discrete, 10 states)
4
838+ 51
1082339++ 1591371
51 + 72.8
4332927
gains obtained by natively handling continuous data. Furthermore, we report standard deviation measurement of communication costs.
Table 8: Communication costs analysis for discrete and continuous variables
Dataset
Healtheare
Sangiovese
4
Overlop
10%
30%
10%
30%
10%
30%
10%
30%
HybridCCIT
Discrete
11 4.2
Set 4.4
20: 11.6
48- 43.9
32:
32+
96+
96= 0
CLG
U::
21:t
3 States
11 3.9
21d 11.2
14th 5.6
70 20.2
21: 1.11
1774: 57.9
243 45.0
31134 1251.
A-CCIT
5 States
10- 3.8
20+ 6.7
10+ 1.6
121+ 78.8
38+ 1.1
225+ 36.0
279- 46.7
35591+
18277.8
10 States
8th
39sh
10sh
2294 148.7
27
3344 146.0
530 1002
51314 1842.
3 States
58.-27.3
61: 15.6
20ch 5.01
1204: 30.6
2404: 77.1
1021 298.8
25600 11412.6
285054 11896.8
CCBNet
5 States
14.74 8.7
72 20.8
10th: 1.6
668 428.7
664t: 7.01
1557 506.7
5210 1971.2
271063:
126798L1
States
11
200+
10th 1.8
2008+
160+ 40.8
900+
4766- 1438.7
577838+
206695.1
K Limitations and Future Work
Attacks on Hybrid CCJT When performing inference with Hybrid CCJT with N parties, secret shared messages can be reverse computed if - parties collude. This means that this kind of information leakage is inevitable when only two parties participate in the inference process. For instance, us look at secret sharing for continuous variables. The merged posterior of common continuous variables is weighted average of parties' locally computed posteriors. Then, party can reverse-compute its peer posterior using the secret sharing outcome and its own locally computed posterior. Similarly, the same goal can be achieved when secret sharing posteriors of common discrete variables, dividing the outcome by the CPD table of the locally computed posterior.
These two attacks abide our security definition, as structure and parameters of the local Bayesian Networks remain protected. Furthermore, the attack previously mentioned falls under Definition [.] of a privacy- -reraccirreeevinggg protocol, as the secret local posterior is found using the outcome of the protocol and prior knowledge. outcome detail of a query. Maximum A Posteriori (MAP) queries [7] allow to find the most-likely state combination of a set of variables given a set of evidence instead revealing their posterior distribution. Conveniently. MAP queries can be performed via Belief Propagation using the queries to collaborative Junction Trees.
order avoid this information leakage, it possible to limit
Junction Tree message-passinng procedure [7]. Future research will focus on applying MAP
Finding Common Nodes Hybrid CCJT assumes that the same variable is given the same name across multiple parties. This allows us to use Private Set Intersection to find common nodes among parties' networks. On the one hand, this can be strong assumption at times difficult to satisfy real use-cases. On the other hand, privately finding common variables from multiple Bayesian Networks a modular operation which can be easily replaced with more advanced techniques, without affecting the core implementation of the Hybrid CCJT framework.
23

---

Inconsistencies can appear at the level of variable definition (parties using different names for the same concept) their representation (state/measureemnn mismatches for discrete and continuous variables, respectively). Some possible mitigation strategies are:
parties giving synonyms for variable names;
normalizing variable names (e.g., as lowercase);
for state alignment mismatches in discrete variables, states not present in all parties could get mapped to a single miscellancous state;
for measurement unit mismatches in continuous variables, parties could update their local representations an agreed-upon unit.
Consensus on Representation of Common Nodes Another requirement of Hybrid CCJT that common nodes need to follow the same representation across multiple parties'. For instance, the same discrete variable must have the same set of states when represented by different parties. Common continuous variables must be represented using the same measure unit.
Furthermore, when dealing with hybrid domain, parties must find consensus on whether represent- ing a variable as discrete or continuous. On the other hand, when discretizing continuous variable, we introduce further consensus problem. That is, variables need to discretized with the same set of states.
Modeling limitations of Hybrid CLG BNs Our work addresses inference in Hybrid CLG BNs leveraging the Lauritzen algorithm [ In general, these networks do not allow continuous variables have discrete children. However, showcase how extend the Lauritzen algorithm (used in our work) to run exact inference in augmented Hybrid CLG BNs. Augmented Hybrid CLG BNs allow continuous variables have discrete children by modeling CPDs of these nodes as softmax functions. Let A be discrete node and Y,...., Yz be its (continuous) parents:
exp(b' + Et= w(y)
PIA = a.
(31)
En= exp(b + Eh w/ v1)
These variables are referred by 42 as CD-CPD (Continuous-Diserdee CPD) variables. Briefly, the algorithm proposed requires to:
Perform inference as in to compute the strong marginals of variables and weak marginals of T variables, without taking into account CD-CPD variables.
2. Insert CD-CPD variables in the Junction Tree, and re-compute marginals of all variables (Referred as tree re-calibration in the paper).
Our method allows to perform step of Lerner's algorithm. Extending Hybrid CCJT to perform step 2 of Lerner algorithm is interesting research gap that will be addressed by future work.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims presented in the abstract reflect the methodology presented in Section 3 and the experimented results showcased Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are introduced in Section 5 and further detailed in Appendix K
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

The assumptions are detailed in Section 3 the verification of assumptions and derivations of proof are provided Amppendix A, for exactness of computation, and Appendix BJ for confidentiality achievements.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We explain our novel algorithm and the experiment methodology in Section and Section respectively.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
26

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We made our code publicly available at atthhhulccco///gggood///yybbb.ad
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (httpss//nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https /nips
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Used datasets and experiments methodologies are stated in Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: While we report only average values, each experiment includes a large num- ber of queries. For predictive performance metrics, observe consistent patterns across all tested scenarios. In contrast, communication improvements are typically so substan- tiae-often by several orders of magnitude- -tht they are unlikely to be due statistical anomalies.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
27

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We mention the amount of system memory used for the experiments, which is the main limiting factor. We did not use any accelerators in our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp//////eerpppe.cpppppp//////
Answer: [Yes]
Justification: We hereby confirm that the conducted research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted.
28

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release any trained models and all used datasets come from existing public repositories.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We reference all assets used our evaluations in Table 5
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
29

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing nor research with humans subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve crowdsoureing nor research with humans subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
30

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not used in creating this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
31


Reviews:
[{"review_id": "hyi8oiJVgg", "reviewer": "Reviewer_rRs9", "summary": "This paper addresses the challenge of performing confidential, collaborative inference in Bayesian Networks (BNs) that span multiple parties and involve hybrid data (i.e., both discrete and continuous variables). The authors identify that existing privacy-preserving methods are typically limited to discrete-only scenarios and often incur high communication overhead.\n\nTo overcome these limitations, the paper introduces Hybrid CCJT, a novel framework for secure multi-party inference in hybrid BNs. The proposed method is twofold: first, it presents a novel technique for constructing a single, strongly rooted Junction Tree (JT) across all parties using \"interface cliques\" to connect their local network structures. Second, it develops a secure protocol based on Multi-Party Computation (MPC) to perform belief propagation on this shared JT, enabling inference while keeping model parameters and structures confidential.\n\nThe authors conduct an empirical evaluation of Hybrid CCJT on nine datasets of varying types (mixed, discrete, and continuous). They compare its performance against a state-of-the-art baseline (CCBNet) and an ablation variant of their method ($\\Delta$-CCJT), measuring prediction accuracy (using MSE for continuous and Brier score for discrete variables) and communication cost.", "questions": "- Could the authors clarify the computational overhead introduced by the cryptographic tools used in the protocol, and how they compare with communication costs in larger models?\n- Is the local marginalization in Equation 9 the primary reason for communication efficiency compared to Variable Elimination methods like CCBNet? A brief confirmation of this mechanism would be helpful.\n- How does the protocol handle inconsistencies in data representation (e.g., mismatched units or state definitions across parties)?", "limitations": "- The paper does not provide a detailed analysis of the computational cost, especially when using cryptographic primitives, which is crucial for practical implementations.\n- Data misalignment issues are not sufficiently addressed, and there is no discussion on the protocol's sensitivity to such inconsistencies.\n- Scalability concerns with larger datasets or more participants are not fully explored.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "* Strengths\nThe paper addresses a significant and previously unaddressed challenge: confidential inference in collaborative, hybrid Bayesian Networks. It is the first to extend the Junction Tree (JT) algorithm to this domain, enabling the native handling of both discrete and continuous variables without discretization. This greatly enhances the practical relevance of collaborative probabilistic modeling, especially for industrial applications. \n\nThe proposed Hybrid CCJT methodology is technically sound, introducing \"interface cliques\" to construct a strongly-rooted JT in a distributed fashion. This is a critical feature for valid inference in hybrid Conditional Linear Gaussian (CLG) models. The paper’s design is complete, including theoretical proofs and integration of cryptographic primitives to ensure confidentiality.\n\nThe empirical evaluation is comprehensive, covering nine datasets with varying characteristics and scales. The comparison against CCBNet and an ablation variant ($\\Delta$-CCJT) clearly demonstrates the contribution of the paper’s specific innovations. The reported results show substantial performance improvements, including a 32\\% reduction in Mean Squared Error (MSE) and up to a 331x reduction in communication costs, providing strong evidence of the method’s superiority.\n\n* Weaknesses\n- Security Guarantee Scope. The security of the protocol is based on a semi-honest, non-colluding adversary model. As noted in Appendix H, the protocol cannot withstand collusion among N-1 participants. While this is a common and explicitly stated assumption, it represents a significant limitation when deployed in environments with low trust or high incentives for collusion. Briefly mentioning this boundary condition in the main text would provide readers with a clearer understanding of the threat model.\n- Practical Assumptions for Data Alignment. The successful operation of the framework relies on two key practical assumptions: (a) shared variables between parties can be correctly identified (e.g., through private set intersections on names), and (b) there is consensus on the representation of these shared variables (e.g., the states of discrete variables, applicable units, and discretization schemes). In practice, achieving semantic interoperability is a significant challenge. The importance of this as a prerequisite could be emphasized more clearly.\n- Context of the 331x Communication Improvement Claim. The paper reports a communication cost reduction of up to 331x on the Munin #2 dataset (Table 3), which is a highly impressive result. However, this peak performance figure arises under a specific, best-case scenario for the proposed method (a large, purely discrete network where the communication overhead of the baseline is maximal). While the result is technically correct and showcases the potential of the method, highlighting this peak number so prominently (e.g., in the abstract) could be slightly misleading. The average improvement across all datasets and settings might be a more representative measure of the expected gains.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback.\nPlease see our reply to the limitations, weaknesses, and questions.\nWe look forward to any further comments.\n\n## Weaknesses\n\n**[W1] Underspecified security guarantee scope in main text**\n\nWe will update the abstract and introduction to mention the semi-honest, non-collusion setting.\nMoreover, in the second paragraph of subsection 3.2, we will visually highlight the \"semi-honest\" denomination alongside the existing one for the \"adversarial model\".\n\n**[W2] Underemphasized practical assumptions for data alignment in main text**\n\nWe will explicitly enumerate the assumptions for successful alignment. Namely, as correctly identified by the reviewer, shared variables across parties should have (i) the same name when defining identical concepts, and (ii) the same discrete states or continuous measurement units.\nFurthermore, we will also point to an appendix with a misalignment-related discussion based on the **Q3** answer below.\n\n**[W3] Undercontextualized $331\\times$ communication improvement claim**\n\nWe concur that the $331\\times$ communication improvement is in a scenario that heavily favours the proposed method.\nWe will additionally report the median ($10.4\\times$) communication improvement observed alongside the maximum in the abstract and introduction for a fairer presentation.\n\n## Questions\n\n**[Q1] What is the computational overhead of the cryptographic tools the protocol employs?**\n\nThe protocol relies on private set intersection, secret sharing (additive/product-based), and homomorphic encryption (HE) at different points in its lifecycle.\nAmong the enumerated cryptographic tools, only HE involves a significant computational overhead, while the primary bottleneck for the others is network latency.\nHE is used exclusively during discrete factor alignment.\nThe results below split the HE computation time across:\n1. *Context Generation*: Generation of HE keys and parameters.\n1. *Encrypted Column Summation*: The actual HE-encrypted computation; used to compute column normalization factors (i.e., $\\alpha$ in Equation 4 in the paper).\n1. *CPD combination*: Alignment of party CPDs based on the HE-computation outputs.\n\nWe report the **per-factor** mean and standard deviation of each HE substep and the total time required for **all factors**.\n\n*Table A: Results on discrete datasets (10% overlap):*\n|Dataset|#Parties|Context Gen. (ms)|Col. Sums (ms)|CPD comb. (ms)|Total (ms)|\n|---|---|---|---|---|---|\n|Asia|2|$153\\pm0$|$18.7\\pm0$|$0.27\\pm0$|176|\n||4|$644\\pm0$|$802\\pm0$|$0.53\\pm0$|1449|\n||8|$2155\\pm0$|$767\\pm0$|$0.746\\pm0$|2926|\n|Alarm|2|$136\\pm5.7$|$51401\\pm82611$|$9.57\\pm14.6$|206191|\n||4|$672\\pm50$|$3639\\pm4780$|$0.749\\pm0.4$|17250|\n||8|$1900\\pm150$|$96745\\pm134754$|$4.19\\pm4.69$|394601|\n|Child|2|$143\\pm5$|$144\\pm84$|$0.33\\pm0.034$|577|\n||4|$703\\pm26$|$17151\\pm9176$|$1.78\\pm0.6$|35713|\n||8|$1984\\pm57$|$83010\\pm79570$|$3.9\\pm2.9$|169999|\n|Link|64|$133\\pm7$|$324.9\\pm381$|$0.4\\pm0.1$|431291|\n\nAs expected, substep 2, i.e., encrypted column summation, is the most time-consuming one, taking up to **96** seconds in the worst-case scenario, but remains under one second in 6 out of 10 experiment scenarios.\nIts time requirements increase overall with the number of parties sharing factors.\nIn contrast, substeps 1 and 3, i.e., context generation and CPD combination, run in under one second and one millisecond, respectively.\nMoreover, we note that the HE procedure represents a one-off setup cost, after which parties may perform unlimited queries until one of them updates their local network.\n\n*Table B: Results on hybrid datasets (30% overlap):*\n\n|Dataset|#Parties|Context Gen. (ms)|Col. Sums (ms)|CPD comb. (ms)|Total (ms)|\n|---|---|---|---|---|---|\n|Healthcare (Hybrid)|2|$127\\pm0$|$26\\pm0$|$0.23\\pm0$|154|\n|Healthcare (Discrete, 3 states)|2|$142\\pm8.24$|$82.4\\pm67$|$0.3\\pm0.05$|675|\n|Healthcare (Discrete, 5 states)|2|$71.35\\pm13$|$71.4\\pm62.4$|$0.271\\pm0.03$|641|\n|Healthcare (Discrete, 10 states)|2|$139.7\\pm10.2$|$165.5\\pm152.6$|$0.316\\pm0.04$|918|\n|Healthcare (Hybrid)|4|$646.9\\pm6.04$|$244\\pm90.5$|$0.43\\pm0.02$|1785|\n|Healthcare (Discrete, 3 states)|4|$678.46\\pm49.3$|$300.2\\pm139.2$|$0.48\\pm0.06$|2940|\n|Healthcare (Discrete, 5 states)|4|$682.9\\pm46.7$|$3007.8\\pm3660$|$0.662\\pm0.295$|11076|\n|Healthcare (Discrete, 10 states)|4|$677\\pm51$|$5956\\pm7221.4$|$0.86\\pm0.48$|19905|\n|Sangiovese (Hybrid)|2|$127\\pm0$|$12.1\\pm0$|$0.25\\pm0$|141|\n|Sangiovese (Discrete, 3 states)|2|$141\\pm9.7$|$827\\pm1080$|$0.46\\pm0.24$|2906|\n|Sangiovese (Discrete, 5 states)|2|$139.8\\pm9.2$|$710.3\\pm905$|$0.403\\pm0.18$|2553|\n|Sangiovese (Discrete, 10 states)|2|$141\\pm11.9$|$621\\pm489$|$0.424\\pm0.09$|2289|\n|Sangiovese (Hybrid)|4|$127\\pm0$|$12.1\\pm0$|$0.25\\pm0$|140|\n|Sangiovese (Discrete, 3 states)|4|$139\\pm9.7$|$813.7\\pm1063$|$0.43\\pm0.24$|2861|\n|Sangiovese (Discrete, 5 states)|4|$659.3\\pm49.6$|$188826.7\\pm239278.3$|$13.2\\pm16.4$|758015|\n|Sangiovese (Discrete, 10 states)|4|$838\\pm51$|$1082339\\pm1591371$|$51\\pm72.8$|4332927|\n\nWhen dealing with hybrid domains, we notice that encryption takes significantly more time on the discretized variants.\nWhile Hybrid CCJT maintains the entire encryption process under one second for both Sangiovese and Healthcare datasets, $\\Delta$-CCJT's encryption cost grows as the discretization coarseness is reduced (i.e., more states).\nThese results also reaffirm the gains in memory and communication costs of natively handling CLG variables compared to discretizing them, as further highlighted in our answer to **Q2**.\n\nWe will add the above tables and discussion as a new appendix.\n\n**[Q2] Is local marginalization in Equation 9 the primary reason for improved communication efficiency?**\n\nEquation 9's local marginalization is the primary reason for improved communication **when only considering discrete variables**.\nWhen incorporating continuous variables, a significant portion of Hybrid CCJT's efficiency also comes from reducing the representation space from $O(2^N)$ states required by the discretization used in $\\Delta$-CCJT or CCBNet, down to $O(N^2)$ parameters required by representing continuous variables in the canonical form.\n\nFor more details on the canonical representation of CLG variables, please see Appendix C of the paper.\nTable 1 from the paper contains quantitative results on the improvements given by Equation 9 (i.e., $\\Delta$-CCJT vs CCBNet; for example, $15.5\\times$ on average for 5 states) and natively handling CLG variables (i.e., Hybrid CCJT vs $\\Delta$-CCJT; for example, $1.9\\times$ on average for 5 states).\n\n**[Q3] How does the protocol handle data representation inconsistencies?**\n\nWhile the current protocol assumes no data representation inconsistencies, the name-matching procedure is flexible in being updated to mitigate such issues.\nInconsistencies can appear at the level of variable definition (parties using different names for the same concept) or their representation (state/measurement mismatches for discrete and continuous variables, respectively).\n\nSome possible mitigation strategies are:\n* parties giving synonyms for variable names;\n* normalizing variable names (e.g., as lowercase);\n* for state alignment mismatches in discrete variables, states not present in all parties could get mapped to a single miscellaneous state;\n* for measurement unit mismatches in continuous variables, parties could update their local representations to an agreed-upon unit.\n\nWe will update Appendix H with the above discussion.\n\n## Limitations\n\n**[L1] No detailed computation time analysis (for cryptographic primitives)**\n\nThe computation complexity of homomorphic encryption in solving an overlap is $O(L \\cdot N \\log(N))$, bounded by $L$ chained multiplications of a size $N$ ciphertext.\n$L$ is the maximum number of parties in an overlap.\n$N$ is a power of two (in the range $2^{10}$--$2^{15}$), derived from the desired precision, the maximum output scale, and $L$.\nSuccessive multiplications increase complexity linearly [1].\nThe cost of a single multiplication is linearithmic [2].\n\nIn our implementation, solving a private set intersection between two parties is $O(m \\log(m))$ time, where $m$ is the maximum number of variables among the two parties.\n\nFor an empirical overview of homomorphic encryption overhead, please refer to our answer to **Q1**.\n\n**[L2] Data misalignment issues insufficiently addressed**\n\nPlease see the answer to **Q2**.\n\n**[L3] Scalability concerns not fully explored**\n\nWe acknowledge that larger-scale tests on continuous and hybrid networks would be beneficial. \nWe note, however, that the primary determiner of our method's scalability compared to a centralized approach is the total number of overlaps and the number of parties involved in each.\nThe sizes of each party's local network or the number of parties are not the bottleneck.\nFor time complexity, see the answer to **L1**.\nFor communication overhead, cost is predominantly related to processing discrete variables.\nAs evidenced by our main evaluation results and our reply to **L3** of Reviewer HuYN, continuous variables have much lower communication requirements, making them even less of an issue for scaling.\n\nMoreover, in our motivating example, we expect overlap percentages to be lower than 30% as covered in our experiments.\nAlso, in such settings with highly specialized participants, at most a handful of parties would model most concepts.\nThus, the insights from our experiments with up to 8 parties remain relevant.\nHowever, we acknowledge that other setups are also plausible (e.g., a central concept modeled by many parties).\n\n**References**\n\n[1] Cheon, Jung Hee, et al. \"Homomorphic encryption for arithmetic of approximate numbers.\" International conference on the theory and application of cryptology and information security (ASIACRYPT), 2017.\n\n[2] Al Badawi, Ahmad, et al. \"OpenFHE: Open-source fully homomorphic encryption library.\" 10th workshop on encrypted computing & applied homomorphic cryptography (WAHC), 2022."}, {"review_id": "70EkZA0rTB", "reviewer": "Reviewer_HuYN", "summary": "Bayesian Networks (BN) are a tool that allows different parties to collaboratively optimize their production process. State-of-the-art frameworks for collaborative and confidential inference only support discrete data and have high communication costs, while frameworks for hybrid scenarios (mix of discrete and continuous data) are limited to single-party settings. The authors solve this by providing a collaborative and confidential framework for BNs in the hybrid setting, constructing a strongly rooted Junction Tree then running a belief-propagation-like algorithm to combine inference results confidentially. Results show 32% average improvement in predictive accuracy and up to 331× reduction in communication costs.", "questions": "Please address weaknesses.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The authors' use of Junction Trees gives a clear advantage in terms of communication costs compared to vanilla belief propagation.\n2. Authors provide a proof of confidentiality under adversarial model and a clear collaborative strongly-rooted JT protocol.\n3. The experimental results are strong: clear improvement in accuracy in hybrid setting, communication cost reduction in discrete setting.\n\nWeaknesses:\n1. Could the authors provide a high-level proof sketch for Proof of confidentiality would help understand why the collaborative inference protocol always hides the marginal posteriors on private variables.\n2. The Collaborative inference protocol is quite sophisticated and high-level. More implementation details would be appreciated if possible.\n3. Limited communication cost improvement analysis in the hybrid setting compared to the extensive discrete results (though Table 1 does show some improvements for hybrid datasets).", "was_revised": true, "final_justification": "This is definitely a technically strong paper. Authors have addressed my concerns and I am satisfied with their response.\nI didn't recommend 5 (my score would be 4.5) because I do not have the domain expertise to judge if this is high-impact and which areas of machine learning this paper can heavily influence (confidence 2). That said, the other reviewers seem to agree that this is a \"significant and previously unaddressed challenge\".", "rebuttal": "Thank you for your valuable comments and feedback. It's helpful for us to further improve this work.\nPlease see our reply to the highlighted weaknesses.\nWe will be happy to follow up on any of the points.\n\n## Weaknesses\n\n**[W1] No high-level proof of confidentiality**\n\nBelow, we outline why both of Hybrid CCJT's collaborative phases preserve confidentiality during inference:\n\n1. **Collaborative Discrete Inference**\nCPD tables of interface variables (say, $X$) are \"augmented\" to allocate space for all parents of $X$ $\\text{pa}(X)$, some of which might be private. This step is required to ensure the correct outcome of HE, CPD product, and CPD normalization as in Equation 4.\nWe show that this step does not leak any information about such private parent variables by proving that their marginals yield a uniform distribution $\\mathcal U(\\Omega(\\text{pa}(X)))$. Where $\\Omega(X)$ is the set of states of variable $X$. We prove this in Proposition B.2. \nThen, we prove that, after message passing, the marginal over these parent variables remains uniform. This ensures no information about such private variables leaks at inference time. We prove this in Proposition B.5.\nFurthermore, note that parties encrypt the names of variables and their states to enhance confidentiality further.\n2. **Collaborative Continuous Inference**\nUnlike discrete variables, continuous counterparts do not require computing and correctly applying normalization factors potentially defined over private factors.\nThus, continuous private variables are inherently protected during alignment.\nAt inference time, the means and variances of shared continuous variables are calculated using multi-party secret sharing, which updates the parameters of such shared continuous interface variables. This approach inherently safeguards the information of private variables, as they are not involved in the update process.\n\n**[W2] Missing implementation details on the inference protocol**\n\nWe will prepare a more detailed explanation of the inference protocol in the methodology section and a more detailed pseudocode and a diagram to be included in the appendix.\nThe inference is divided in three steps (of which two are collaborative) and follows the logic of the Lauritzen algorithm [1].\n\n1. **Collaborative Continuous Inference**\nThe first step requires marginalizing continuous variables and finding the **strong marginals** over the threshold variables. Recall that threshold variables are the discrete variables with at least one continuous child.\nIn short, we aim at computing $P[T|\\Gamma = e]$, where $T$ is the set of threshold variables and $\\Gamma = e$ is some continuous evidence.\nTo do so, parties collaboratively compute the updated parameters of interface continuous variables given the set of continuous evidences as in\n$$\n    {\\mu}(X) = \\sum_{i \\in \\mathbb P} w_i\\mu_i(X), \\quad\n     \\sigma(X) = \\sum_{i \\in \\mathbb P} [w_i\\sigma_{i}(X)] + \\sum_{i \\in \\mathbb P}[w_i(\\mu - \\mu_{i}(X))^2\n    ]\n$$\nwhere the parameters $\\mu_i$ and $\\sigma_i$ are obtained locally by computing $P[I_\\Gamma | \\Gamma=e]$.\nOnce these new parameters are obtained, they can help compute the marginals over the threshold variables by just marginalizing all the continuous variables.\n\n2. **Collaborative Discrete Inference**\nOnce parties find the marginals of threshold variables, the querying party needs to find the strong marginals over the remaining discrete variables.\nFor this, parties compute $P[I_\\Delta | \\Delta = e]$, the posterior of discrete interface variables given their discrete evidence.\nThe querying party receives these posteriors and merge them via secret sharing to find the geometric mean:\n$$\n P[I_\\Delta] = \\prod_{i \\in \\mathbb P} P_i[I_\\Delta]^{w_i}\n$$\nThe querying party can then use this factor to update its local probability distribution via sum-product message passing [2] (Equation 2 from the paper). For a more detailed explanation on the sum-product message passing please refer to Section 10.2 in the book of Koller [3].\n\n3. **Local query**\nAt this point in the protocol, the querying party can find the posterior of any variable it owns. \nFor **discrete variables**, the posterior is found by marginalizing non-target variables, as in a normal junction tree.\nFor **continuous variables**, the posterior is found by marginalizing all non-threshold variables and performing weak marginalization:\n$$\n P[X] = \\mathcal N\\Bigl(X; \n    \\sum_{s \\in \\Omega(\\Delta_{Q})}P[\\mathcal T_Q=s] \\mu_{s}(X),\n    \\sum_{s \\in \\Omega(\\Delta_{Q})}P[\\mathcal T_Q=s] \\{\\sigma_{s}(X) + (\\mu - \\mu_{s}(X))^2\\}\n    \\Bigr)\n$$\n\n**[W3] Limited communication cost analysis on hybrid domains**\n\nWe repeated the experiments from Table 1 with new variable splits to derive further insights in the communication costs of Hybrid CCJT.\nIn Table C isolates the communication costs of Discrete and Continuous collaborative inference. This allows us to better analyze the communication costs gains obtained by natively handling continuous data.\nFurthermore, we report standard deviation measurement of communication costs.\n\n*Table C: Communication cost split for discrete and continuous variables*\n|Dataset||Healthcare||||Sangiovese||||\n|---|---|---|---|---|---|---|---|---|---|\n|#Parties||2|2|4|4|2|2|4|4|\n|Overlap||10%|30%|10%|30%|10%|30%|10%|30%|\n|HybridCCJT|Discrete|$11\\pm4.2$|$5\\pm4.4$|$20\\pm11.6$|$48\\pm43.9$|$32\\pm0$|$32\\pm0$|$96\\pm0$|$96\\pm0$|\n||CLG|$0\\pm0$|$21\\pm4.8$|$0\\pm0$|$192\\pm62.1$|$29\\pm7.5$|$59\\pm10.5$|$182\\pm44.9$|$541\\pm73.6$|\n|$\\Delta$-CCJT|3 States|$11\\pm3.9$|$21\\pm11.2$|$14\\pm5.6$|$70\\pm24.2$|$21\\pm1.0$|$177\\pm37.9$|$243\\pm45.0$|$3313\\pm1251.6$|\n||5 States|$10\\pm3.8$|$20\\pm6.7$|$10\\pm1.6$|$121\\pm78.8$|$38\\pm1.1$|$225\\pm36.0$|$279\\pm46.7$|$35591\\pm19277.8$|\n||10 States|$8\\pm3.1$|$39\\pm15$|$10\\pm1.8$|$229\\pm148.7$|$27\\pm2.2$|$334\\pm146.0$|$539\\pm100.2$|$5131\\pm1842.4$|\n|CCBNet|3 States|$58\\pm27.3$|$61\\pm15.6$|$20\\pm5.0$|$120\\pm30.6$|$240\\pm77.1$|$1021\\pm298.3$|$25609\\pm11412.6$|$28505\\pm11896.8$|\n||5 States|$14.7\\pm3.7$|$72\\pm29.8$|$10\\pm1.6$|$888\\pm428.7$|$55\\pm7.0$|$1557\\pm506.7$|$5210\\pm1971.2$|$271053\\pm125799.6$|\n||10 States|$11\\pm3.2$|$290\\pm139$|$10\\pm1.8$|$2068\\pm1043$|$169\\pm40.8$|$990\\pm247.0$|$4766\\pm1438.7$|$577838\\pm295695.1$|\n\n**References**\n\n[1] Lauritzen, Steffen L., and Frank Jensen. \"Stable local computation with conditional Gaussian distributions.\" Statistics and Computing (SC), 2001.\n\n[2] Shenoy, Prakash P., and Glenn Shafer. \"Axioms for probability and belief-function propagation.\" Machine intelligence and pattern recognition, 1990.\n\n[3] Koller, Daphne, and Nir Friedman. \"Probabilistic graphical models: principles and techniques\". MIT Press, 2009."}, {"review_id": "ZWe3x1dw9D", "reviewer": "Reviewer_6zT8", "summary": "Bayesian networks are a way to represent distributions over multivariate data, each node is a variable, and a directed edge between nodes shows dependence, the network is a DAG. Collaborative Bayesian networks are when multiple parties determine individual Bayesian networks which may contain some common variables, the goal is to combine networks and perform inference on the combined Bayesian network. However we may also require that agents cannot see each others private local variables. \n\nJunction trees are a standard technique for performing inference over (non-collaborative) Bayesian networks by clustering factors in a graphical model (e.g. a Bayesian network) into nodes called cliques, within each clique one may apply variable elimination followed by message passing across cliques for exact inference, the tree structure ensures that message passing will converge in a single pass. \n\nThis paper considers the problem of performing inference over a collaborative Bayesian network while preserving confidentiality of individual parties local variables and their distributions. When there are continuous and discrete variables. \n\nThe proposed solution is a custom protocol to build a junction tree that creates two cliques for each agent, one with all agents discrete variables, and one with all the agents continuous variables, and there are two central cliques, one for communal discrete variables and communal continuous variables, and these are connected to ensure that all discrete nodes have no continuous parents. \n\nThe method is applied to 9 different datasets synthetically generated from known Bayesian network models and shows improvements over baselines.", "questions": "- How does it work with 3+ agents? If I understand correctly, we make a star/hub and spoke network of discrete cliques with interface at the hub and one agent in each spoke, and similar hub+spokes for continuous variables cliques and link the hubs? \n\n- If the interface nodes are the only link from discrete to continuous variables, for an agent with a private discrete parent and private continuous child node, the necessary (private) message must still pass through the interface nodes. How are private threshold variables preserved if they still pass through the interface node? Is it safe to assume that they can be encrypted/ignored by all other agents as they are not required and just pass through? If so, is this equivalent to appending a private continuous clique to each private discrete clique? I understand this preserves Tree structure and strong rootedness. \n\n- What is the “#Arc”  field in the datasets table?", "limitations": "- Cost is exponential in the number of variables in each agent, (presumably one may use nested junction trees?) \n\n- In the two party case, each agent can infer information from the aligned result (which is a fundamental fact of HE and not a problem with Hybrid CCJT), I see this is mentioned in the limitations (Appendix H), this may be more prominently mentioned in the main paper.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "I am familiar with graphical models and junction trees, I am not familiar with all the topics covered in the paper (SMC and I may have misunderstood details and apologize in advance for any mistakes, particularly in . \n\nIn general, I am positive about the paper and the approach seems reasonable. My comments are mostly minor. \n\n## Strengths  \n- Seems intuitive clean simple idea \n\n## Weaknesses  \n- there is no mention of [Multiply Sections Bayesian Networks](https://ieeexplore.ieee.org/document/858473), presumably one could augment this with HE for interface cliques and perform loopy belief propagation? The continuous variables require a tree structure and strong rooting hence we cannot use such a method?\n- the interface cliques must contain shared variables as well as all threshold variables (both private and shared), is there a privacy leak or can we just treat private threshold variables as invisible to irrelevant agents (see questions)?\n- it is stated that the two agent case allows each agent to infer the other agents distribution over shared variables, there is a slight data leak? \n\nhttps://ieeexplore.ieee.org/document/858473  \n- Inherits the same exponential cost in largest clique as junction trees", "was_revised": true, "final_justification": "My original comments were mostly minor and my questions were answered accordingly and as expected.\n\nAfter reading other reviews, particularly rRs9 thoughts on impact and novelty and the issue being addresed, I don't see a clear reason to not accept and have raised my score accordingly.", "rebuttal": "Thank you for the insightful review.\nPlease see our reply to the outlined points.\nWe would be happy to clarify any unclear or unaddressed aspects.\n\n## Weaknesses\n\n**[W1] No mention of Multiply Sectioned Bayesian Networks**\n\nPerforming loopy belief propagation over Multiply Sectioned Bayesian Networks would (indeed) break the strong root constraint and lead to poor inference quality in hybrid domains.\nFurthermore, loopy belief propagation entails sending more messages (due to the iterative nature of the algorithm) and is proven to converge only on graphs with at most one cycle [1].\nWe will update the related work section with this information.\n\n\n**[W2] Is there a privacy leak over threshold variables?**\n\nPlease see the answer to **Q2**.\n\n**[W3] Possible data leak in two-party scenario**\n\nIndeed, when only two parties are involved, the proposed method does not protect the private posterior of interface variables, as doing so is impossible.\nIntuitively, consider computing the sum of $N$ secret numbers $x_1, ..., x_N$ (Equations 7 and 8 in the paper).\nIf $N>2$ a certain party $i$ knows $x_i$ and $y = \\sum^N_{j = 1}x_j$ from which party $i$ cannot reconstruct the values of $x_{j \\neq i}$. If $N=2$, party $i$ can reconstruct $x_j = (x_i + x_j) - x_i = y - x_i$.\nSimilar reasoning also holds for products.\n\nThe work from Kearns et al. [2] builds upon this basic idea to explore the limitations of confidentiality in Bayesian Networks and specific methodologies including sum-product message passing.\nNotably, that work establishes a generalized definition for confidentiality in Bayesian Network applications (Definition 1 from [2]):\n\n> Let $\\Pi$ be any protocol for the $k$ parties to jointly compute the value $y = f(x_1, ... ,x_k)$ from their $n$-bit private inputs. We say that $\\Pi$ is privacy-preserving if for every $1 \\leq i \\leq k$, anything that party $i$ can compute in polynomial time in $n$ following the execution of $\\Pi$, they could also compute in polynomial time given only their private input $x_i$ and the value $y$.\n\nWe abide by this definition which we include in Appendix B.\n\n## Questions\n\n**[Q1] How does the protocol work with 3+ parties?**\n\nThe junction tree forms 2 star networks corresponding to discrete and continuous variables, respectively (as also remarked by the reviewer), joined via their hubs (i.e., the interface cliques).\nPlease note that, as stated in the paper, the Junction Tree dictates the computational logic within the protocol, not the actual distribution of data across parties.\nThus, an interface clique is not centralized or controlled by any party.\nInstead, parties solve inference collaboratively over this clique.\n\n**[Q2] How are threshold variables protected?**\n\nUnlike interface variables, whose protection depends on homomorphic encryption and secret sharing during alignment and inference, threshold variables are not shared between parties and only used locally.\nWe will better emphasize this aspect in the methodology section.\n\n**[Q3] What is the “#Arc” field in the datasets table?**\n\n\"#Arc\" in Table 5 refers to the number of edges in the original network.\nIt describes the number of relationships between variables in the network.\nHowever, having more edges does not necessarily imply an increase in the number of parameters (#Params column) reported in the same table.\n#Params depends on the network's edge distribution and the representations of variables.\nFor instance, a discrete binary variable with two binary parents will require $2^3=8$ parameters. Whereas, one discrete variable with one parent both with three states will require $3^2=9$ parameters.\n\n## Limitations\n\n**[L1] Cost is exponential in the number of variables in each party**\n\nPer the nature of discrete Bayesian Networks, inference is NP-Hard.\nHowever, our algorithm is agnostic to the inference algorithm used locally by each party.\nThus, instead of the Variable Elimination (as employed in our experiments) parties could indeed autonomously harness auxiliary junction trees for local inference exponential in the tree-width instead of the number of variables.\nNote that using nested Junction Trees would not affect communication costs.\n\n**[L2] Undermentioned caveats in two-party case**\n\nAs correctly identified by the reviewer, there is an unavoidable confidentiality leak in the two-party case.\nWe will update the paper's method section to emphasize the phenomenon.\nMore specifically, the issue stems from the need to disclose the output of a bijective function with two inputs, with one of the inputs also being known beforehand.\nThus, no secure multiparty computation technique, including homomorphic encryption, can address the issue.\nPlease refer to the answer of **W3** for additional details.\n\n**References**\n\n[1] Zivan, Roie, Omer Lev, and Rotem Galiki. \"Beyond trees: Analysis and convergence of belief propagation in graphs with multiple cycles.\" AAAI Conference on Artificial Intelligence (AAAI), 2020.\n\n[2] Kearns, Michael, Jinsong Tan, and Jennifer Wortman. \"Privacy-preserving belief propagation and sampling.\" Advances in Neural Information Processing Systems 20 (NeurIPS), 2007."}]

Author rebuttal:

Meta-review: This paper considers a highly original setting wherein multiple parties (e.g., companies with proprietary data/IP) seek to collaboratively optimize a process (e.g., manufacturing) by performing inference in a large Bayesian network that is composed of separate party-specific Bayesian networks, each representing their respective system or domain. The key challenge in this setting is permitting collaboration while guaranteeing confidentiality---i.e., performing inference in the large Bayesian network while ensuring that parties cannot access the propriety knowledge encoded in other parties' Bayesian (sub-)networks. The paper introduces the "first framework for performing confidential multiparty inference in hybrid domains", where hybrid domains are those involving both discrete and continuous variables. 

The reviewers all appreciated the originality and novelty of the setting and the proposed framework. They found the methodology to be sound and compelling, and were satisfied with the empirical results. The following is a representative excerpt from one reviewer's comments:

"The paper addresses a significant and previously unaddressed challenge [...] The proposed methodology Hybrid CCJT is technically sound [...] The paper's design is complete [...] The empirical evaluation is comprehensive."

The authors were highly responsive during the discussion period and largely assuaged the reviewers' concerns, including responding in detail to questions about computational overhead, giving a proof sketch, and providing additional experimental results. The authors also presented a clear plan for revising the paper in response to the reviewers' comments, which I find reasonable and achievable for camera-ready.
Decision: Accept (poster)