Probabilistic Circuits
Probabilistic circuits (PCs) [ 44 , 3 ] model probability distributions with a structured computation graph. They are an umbrella term for a large family of tractable probabilistic models including arithmetic circuits [ 9 , 10 ] , sum-product networks (SPNs) [ 35 ] , cutset networks [ 36 ] , and-or search spaces [ 28 ] , and probabilistic sentential decision diagrams [ 21 ] .
The syntax and semantics of PCs are defined as follows.

A PC \mathcal{C}\!:=\!(\mathcal{G},\bm{\theta}) represents a joint probability distribution {p}(\mathbf{X}) over random variables \mathbf{X} through a directed acyclic (computation) graph (DAG) \mathcal{G} parameterized by \bm{\theta} . Similar to neural networks, each node in the DAG defines a computational unit. Specifically, the DAG \mathcal{G} consists of three types of units — input , sum , and product . Every leaf node in \mathcal{G} is an input unit; every inner unit n (i.e., sum or product) receives inputs from its children \mathsf{in}(n) , and computes output , which encodes a probability distribution {p}_{n} defined recursively as follows:

{{p}}_{n}(\bm{x}):=\begin{cases}f_{n}(\bm{x})&if $n$ is an input unit,\ \prod_{c\in\mathsf{in}(n)}{p}_{c}(\bm{x})&if $n$ is a product unit,\ \sum_{c\in\mathsf{in}(n)}\theta_{c|n}\cdot{p}_{c}(\bm{x})&if $n$ is a sum unit,\end{cases} (1)

where f_{n}(\bm{x}) is a univariate input distribution (e.g, Gaussian, Categorical), and \theta_{c|n} denotes the parameter that corresponds to edge (n,c) in the DAG. For every sum unit n , its input parameters sum up to one, i.e., \sum_{c\in\mathsf{in}(n)}\theta_{c|n}=1 .
Intuitively, a product unit defines a factorized distribution over its inputs, and a sum unit represents a mixture over its input distributions with weights \{\theta_{c|n}\!:\!c\in\mathsf{in}(n)\} . Finally, the probability distribution of a PC (i.e., {p}_{\mathcal{C}} ) is defined as the distribution represented by its root unit r (i.e., {p}_{r}(\bm{x}) ), that is, its output neuron. The size of a PC, denoted |\mathcal{C}|=|\bm{\theta}| , is the number of parameters in \mathcal{C} .
We assume w.l.o.g. that a PC alternates between layers of sum and product units before reaching its inputs. Figure 2 shows an example of a PC.

Computing the (log)likelihood of a PC \mathcal{C} given a sample \bm{x} is equivalent to evaluating its computation units in \mathcal{G} in a feedforward manner following Equation 1 .
The key property that separates PCs from other deep probabilistic models such as flows [ 14 ] and VAEs [ 19 ] is their tractability , which is the ability to exactly and efficiently answer various probabilistic queries. This paper focuses on PCs that support linear time (w.r.t. model size) marginal probability computation, as they are increasingly used in downstream applications such as data compression [ 26 ] and making predictions under missing data [ 18 ] , and also achieve on-par expressiveness [ 26 , 25 , 24 ] . To support efficient marginal inference, PCs need to be smooth and decomposable .

The scope \phi(n) of a PC unit n is the set of input variables that it depends on; then,
(1) a product unit is decomposable if its children have disjoint scope;
(2) a sum unit is smooth if its children have identical scope.
A PC is decomposable if all of its product units are decomposable; a PC is smooth if all of its sum units are smooth.

Decomposability ensures that every product unit encodes a well-defined factorized distribution over disjoint sets of variables; smoothness ensures that the mixture components of every sum units are well-defined over the same set of variables.
Both structural properties will be the key to guaranteeing the effectiveness of the structure learning algorithms proposed in the following sections.