# A class of multivariate conditional logistic distributions

Curtis Miller

► To cite this version:

Curtis Miller. A class of multivariate conditional logistic distributions. 2026. hal-05516096

HAL Id: hal-05516096

https://hal.science/hal-05516096v1

Preprint submitted on 18 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# A class of multivariate conditional logistic distributions

Curtis Miller *

University of New Mexico, College of Pharmacy.

February 18, 2026

## Abstract

A class of multivariate distributions is defined. If $X_1, \dots, X_m$ have such a distribution, then $X_j$ has a logistic distribution, given values for $X_\ell, \ell \neq j$. A parametrization is suggested. Some important symmetry properties are shown.

**Keywords:** Logistic distribution; conditionally specified distributions; multivariate distribution

## 1 Introduction

In this paper, a class of multivariate distributions is defined. These distributions have the property that each outcome variable, conditional on fixed values for the other outcomes, has a logistic distribution. These distributions can be defined for any number of outcomes, and are flexible enough to allow for a wide variety of relationships between outcomes or sets of outcomes. This includes independence between two sets of outcomes.

Multivariate logistic distributions have been studied for some time. Gumbel ([Gumbel (1961)]) examined a bivariate distribution such that the marginal distributions were logistic. Ali, Mikhail, & Haq ([Ali, Mikhail, and Haq (1978)] defined a modification of the Gumbel distribution with a flexible relation between the two outcomes. Malik and Abraham ([Malik and Abraham (1973)]) defined a multivariate logistic distribution in such a way that the marginal distributions were logistic, but they did not control interactions between the variables. More recently, Yeh ([Yeh (2010)]) gave several classes of multivariate distributions for which the individual outcomes have distributions that are similar to the logistic.

The multivariate distributions given in most references in the bibliography have the characteristic that the individual univariate variables have marginal distributions that are logistic (or logistic-like). Multivariate distributions for which the individual outcomes have conditional distributions that are logistic have not investigated nearly as much. Chapter 51 of [Kotz, Balakrishnan, and Johnson (2000)] discusses several varieties of multivariate logistic distributions, including a section on a bivariate conditionally specified logistic model. This was derived from a bivariate distribution introduced by [Arnold (1987)].

The logistic distribution has been used for modeling life test data (see Berkson ([Berkson (1944)]), Plackett ([Plackett (1958)]), among others). We will show, by example, that a multivariate logistic distribution can be used to model several outcomes that are not associated with survival data.

## 2 Basic definitions and concepts
or more generally

$$ \frac{c \exp \left( \frac{x_1 - \mu_1}{\theta_1} + \frac{x_2 - \mu_2}{\theta_2} \right)}{1 + \exp \left( \frac{x_1 - \mu_1}{\theta_1} \right) + \exp \left( \frac{x_2 - \mu_2}{\theta_2} \right) + C \exp \left( \frac{x_1 - \mu_1}{\theta_1} + \frac{x_2 - \mu_2}{\theta_2} \right)} \quad (2.1) $$

where C is some positive constant. Gumbel ([Gumbel (1961)]) looked at a similar function:

$$ F(x_1, x_2) = \frac{\exp\left(\frac{x_1 - \mu_1}{\theta_1} + \frac{x_2 - \mu_2}{\theta_2}\right)}{\exp\left(\frac{x_1 - \mu_1}{\theta_1}\right) + \exp\left(\frac{x_2 - \mu_2}{\theta_2}\right) + \exp\left(\frac{x_1 - \mu_1}{\theta_1} + \frac{x_2 - \mu_2}{\theta_2}\right)} \quad (2.2) $$

It is not hard to check that the conditionals of this ($F(x_1|x_2)$ or $F(x_2|x_1)$) are CDFs of univariate logistic distributions. So Equation (2.2) defines a bivariate logistic distribution. But when we use a bivariate logistic distribution, we may have two variables $X_1$ and $X_2$ in mind. When we fit a joint distribution for $X_1$ and $X_2$, we need to consider the correlation of $X_1$ and $X_2$. $X_1$ and $X_2$ may be independent. In Equation (2.1), if $c = 0$ (so that $\log c = 0$), the expression factors out into

$$ \frac{e^{(x_1 - \mu_1)/\theta_1}}{1 + e^{(x_1 - \mu_1)/\theta_1}} * \frac{e^{(x_2 - \mu_2)/\theta_2}}{1 + e^{(x_2 - \mu_2)/\theta_2}} $$

Equation (2.2) does not factor out, so it does not allow independence of $X_1$ and $X_2$. So, if we want a possibility that $X_1$ and $X_2$ might be independent, the denominator must include a constant.

If we are seeking a joint distribution for $X_1, \dots, X_m$, there is always a possibility that $X_j$ may be independent of $\{X_\ell, \ell \neq j\}$ for some $j \in \{1, \dots, m\}$, or that $\{X_\ell : \ell \in S\}$ may be independent of $\{X_\ell : \ell \in \{1, \dots, m\}/S\}$ for some $S \subset \{1, \dots, m\}$. To allow for this, the denominator should include a term of form $C \exp(\sum_{\ell \in S} (x_\ell - \mu_\ell)/\theta_\ell)$, for all $S \subset \{1, \dots, m\}$.

So far, all this has concerned cumulative distribution functions of logistic distributions. However, the logistic distribution has the notable feature that the density function has almost the same form as the cumulative density: Simply square the denominator. This observation, along with the preceding paragraph, suggests the following definition.

## 2.2 Preliminary definition

**Definition 2.1.** A multivariate conditional logistic distribution (MCL) is a joint distribution of variables $X_1, \dots, X_m$ that has probability density function
of form

$$ \frac{K \exp \left( \sum_{j=1}^{m} (x_j - \mu_j) / \theta_j \right)}{\left[ 1 + \sum_{j=1}^{m} \exp \left( (x_j - \mu_j) / \theta_j \right) + \sum \{ c_S \exp \left( \sum_{j \in S} (x_j - \mu_j) / \theta_j \right) : S \subset \{1, \dots, m\}, |S| > 1 \right]^{2}} \quad (2.3) $$

where $c_S > 0$ for all $S \subset \{1, \dots, m\}$ of size larger than one. Here $K$ is a normalizing constant. Another way to express this is

$$ \frac{K \exp \left( \sum_{j=1}^{m} (x_j - \mu_j) / \theta_j \right)}{\left[ \sum \{ c_S \exp \left( \sum_{j \in S} (x_j - \mu_j) / \theta_j \right) : S \subset \{1, \dots, m\} \} \right]^2} \quad (2.4) $$

where $c_S > 0$ for all $S \subset \{1, \dots, m\}$, $c_\emptyset = 1$, and $c_{\{j\}} = 1$, $1 \le j \le m$.

The denominator includes terms for all subsets of $\{1, \dots, m\}$. So there are $2^m$ terms in the denominator (before taking the square).

## 2.3 Conditional distributions

Choose a set $T_1 \in \{1, \dots, m\}$ and fix $x_j = x_j^*$ for $j \in T_1$. Let $T_2 := \{1, \dots, m\}/T_1$. For any $S \subset \{1, \dots, m\}$, $S = S_1 \cup S_2$, where $S_1 = S \cap T_1$ and $S_2 = S \cap T_2$. For any subset $S_2 \subset T_2$, the terms in the denominator of Definitions (2.4) with $x_j$ in the logarithm for all $j \in S_2$ and with no $x_j$ for $j \in T_2/S_2$ are of form

$$ c_{S_1 \cup S_2} \exp \left( \sum_{j \in S_2} (x_j - \mu_j) / \theta_j + \sum_{j \in S_1} (x_j^* - \mu_j) / \theta_j \right) $$

for some $S_1 \in T_1$. So the coefficient of $\exp \left( \sum_{j \in S_2} (x_j - \mu_j) / \theta_j \right)$ is $c_{S_2}^{[\mathbf{x}^*]} := \sum_{S_1} c_{S_1 \cup S_2} \exp(\sum_{j \in S_1} (x_j^* - \mu_j) / \theta_j)$, with the sum being over all $S_1 \subset T_1$. So density (2.3) becomes

$$ \begin{aligned} & K \frac{\exp \left( \sum_{j \in T_1} (x_j^* - \mu_j) / \theta_j \right) \exp \left( \sum_{j \in T_2} (x_j - \mu_j) / \theta_j \right)}{\left[ \sum \{ c_{S_2}^{[\mathbf{x}^*]} \exp \left( \sum_{j \in S_2} (x_j - \mu_j) / \theta_j \right) : S_2 \subset T_2 \} \right]^2} \\ & = K \frac{\exp \left( \sum_{j \in T_2} (x_j - \mu_j) / \theta_j \right)}{\left[ \sum \{ c_{S_2}^{[\mathbf{x}^*]} \exp \left( \sum_{j \in S_2} (x_j - \mu_j) / \theta_j \right) : S_2 \subset T_2 \} \right]^2} \end{aligned} $$

where $c_{S_2}^{[\mathbf{x}^*]} := c_{S_2}^{[\mathbf{x}^*]} \exp \left( -\frac{1}{2} \sum_{j \in T_1} (x_j^* - \mu_j) / \theta_j \right)$. Hence an MCL for $(X_1, \dots, X_m)$, conditional on $X_j = x_j^*$, $j \in T_1$, becomes on MCL in variables $\{X_j, j \in T_2\}$. In particular, if $T_1 = \{1, \dots, m\}/\{j\}$, then $T_2 = \{j\}$, and the p.d.f. of $X_j$ conditional on $X_\ell = x_\ell^*$, $\ell \ne j$ is proportional to

$$ \frac{\exp((x_j - \mu_j)/\theta_j)}{(A + B \exp((x_j - \mu_j)/\theta_j))^2} $$
where

$$A = \sum \{c_{S_1} \exp \left( \sum_{\ell \in S_1} (x_{\ell}^* - \mu_{\ell}) / \theta_{\ell} \right) : S_1 \subset \{1, \dots, m\} / \{j\} \}$$

$$B = \sum \{c_{S_1 \cup \{j\}} \exp \left( \sum_{\ell \in S_1} (x_{\ell}^* - \mu_{\ell}) / \theta_{\ell} \right) : S_1 \subset \{1, \dots, m\} / \{j\} \}$$

which in turn is proportional to

$$\frac{\exp((x_j - \mu_j)/\theta_j + \log(B/A))}{(1 + \exp((x_j - \mu_j)/\theta_j + \log(B/A)))^2}$$

so $X_j|X_\ell = x_\ell^*, \ell \neq j$ has a logistic distribution with variance $\theta_j^2\pi^2/3$ and mean $\mu_j - \theta_j \log(B/A)$.

## 2.4 Marginal distributions

Pick $j \in \{1, \dots, m\}$. The MCL density can be written

$$\frac{C e^{(x_j - \mu_j)/\theta_j}}{(A + B e^{(x_j - \mu_j)/\theta_j})^2}$$

where $A$ and $B$ are linear combinations of $\exp(\sum_{\ell \in S} (x_\ell - \mu_\ell)/\theta_\ell)$, for all $S \subset \{1, \dots, m\}/\{j\}$. (This includes a constant, for the empty set). It can be calculated that if we integrate out $x_j$, the result is

$$\frac{C \theta_j}{AB} \qquad (2.5)$$

So the marginal distribution of $\{X_\ell : \ell \neq j\}$ is a ratio with denominator $AB$. This is a weighted geometric mean of the densities of two MCL distributions in $m-1$ variables. Each has variances $\{\theta_\ell : \ell \neq j\}$; one has coefficients $\{c_S\}$ for terms in the denominator defined by subsets $S \in \{1, \dots, m\}/\{j\}$, the other density has coefficients $\{c_{S \cup \{j\}}\}$.

A derivation of (2.5) is given in Appendix A.

Choose another $k \in \{1, \dots, m\}$, $k \neq j$. Then $C/AB$ of Equation (2.5) can be written

$$\frac{C' e^{(x_k - \mu_k)/\theta_k}}{(A_1 + B_1 e^{(x_k - \mu_k)/\theta_k})(A_2 + B_2 e^{(x_k - \mu_k)/\theta_k})}$$

where $A_1, B_1, A_2, B_2$ are linear combinations of terms $\exp(\sum_{\ell \in S} (x_\ell - \mu_\ell)/\theta_\ell)$, for all $S \subset \{1, \dots, m\}/\{j, k\}$. For the exact formulas for $A_1, B_1, A_2$, and $B_2$, see Appendix A. Again, if we integrate out $x_k$, we obtain

$$\frac{C' \theta_k \log(A_1 B_2 / A_2 B_1)}{A_1 B_2 - A_2 B_1} \qquad (2.6)$$
This is the marginal distribution of {$X_\ell : \ell \neq j, k$}. A derivation of (2.6) is given in Appendix A.

To find marginal distributions of sets {$X_\ell : \ell \in S$} for $|S| < m - 2$, we would have to integrate the last expression in Equation (2.6) with respect to one or more $X_\ell$'s. There does not seem to be a closed form expression for such an integral. However, in Section 2.6 this will be taken up again.

## 2.5 A proposed parametrization

Consider the case $m = 2$. In that case, $\{1, 2\}/\{1, 2\} = \emptyset$. The expression in Equation (2.6) simplifies: $C' = K$, $A_1 = B_1 = A_2 = 1$, $B_2 = c_{12}$. Since the last expression is the result of integrating out all $X$ variables of a probability density, it must be one; so

$$1 = \frac{K \log c_{12}}{c_{12} - 1}$$

or

$$K = \frac{c_{12} - 1}{\log c_{12}}.$$

But clearly $c_{12}$ must be positive. To have a parameter that can take any real value, let $q_{12} := -\log c_{12}$. Then

$$K = \begin{cases} (e^{-q_{12}} - 1)/(-q_{12}) = (1 - e^{-q_{12}})/q_{12} & \text{if } q_{12} \neq 0, \\ 1 & \text{if } q_{12} = 0 \end{cases}$$

Let's look at $q_{12}$ another way. Consider the expression in (2.4). For $S \subset \{1, \dots, m\}$, let $b_S$ be the coefficient of $\exp(\sum_{\ell \in S} (x_\ell / \theta_\ell))$ in the denominator of (2.4). So $b_\emptyset = 1$, $b_j = e^{-\mu_j / \theta_j}$, and if $S \subset \{1, \dots, m\}$, $|S| > 1$, then

$$b_S = c_S \exp \left( - \sum_{\ell \in S} \frac{\mu_\ell}{\theta_\ell} \right).$$

If $m = 2$, then $b_{12} = c_{12}e^{-\mu_1/\theta_1 - \mu_2/\theta_2}$, so $b_{12}b_\emptyset/b_1b_2 = c_{12}$, and

$$q_{12} = -\log c_{12} = -\log \frac{b_{12}b_{\emptyset}}{b_1b_2} = -\log b_{12} + \log b_1 + \log b_2 - \log b_{\emptyset}.$$

This suggests a new class of parameters. If $m = 3$, define

$$\begin{aligned} q_{123} &= \log \frac{b_{12}b_{13}b_{23}b_{\emptyset}}{b_{123}b_1b_2b_3} \\ &= -\log b_{123} + \log b_{12} + \log b_{13} + \log b_{23} - \log b_1 - \log b_2 - \log b_3 + \log b_{\emptyset} \end{aligned}$$

and in general, for $S \subset \{1, \dots, m\}$, $|S| > 1$, define

$$q_S := - \sum \left\{ (-1)^{|S|-|S'|} \log b_{S'} : S' \subset S, |S'| \ge 1 \right\}. \quad (2.7)$$
Notice that

$$
\begin{align*}
\log b_{12} &= -q_{12} + \log b_1 + \log b_2 - \log b_\emptyset = -q_{12} + \sum_{j=1}^{2} (\mu_j / \theta_j) \\
\log b_{123} &= -q_{123} + \log b_{12} + \log b_{13} + \log b_{23} - \log b_1 - \log b_2 - \log b_3 \\
&= -q_{123} + q_{12} + q_{13} + q_{23} - \sum_{j=1}^{3} (\mu_j / \theta_j).
\end{align*}
$$

For any $S \subset \{1, \dots, m\}$, $|S| > 1$, we can think of $q_S$ as measuring the interaction between $\{X_j, j \in S\}$, over and above any interaction induced by lower level interactions among $\{X_j, j \in S\}$.

It is not hard to show that if $S \subset \{1, \dots, m\}$ and $|S| > 1$, then

$$
\log b_S = - \sum \{q_{S'} : S' \subset S, |S'| > 1\} - \sum_{j \in S} \mu_j / \theta_j. \quad (2.8)
$$

By (2.8), the formula (2.3) can now be written

$$
f(\mathbf{x}|\bar{\mu}, \mathbf{q}, \bar{\theta}) = \frac{K \exp \left( \sum_{j=1}^{m} (x_j - \mu_j) / \theta_j \right)}{\left[ 1 + \sum_{j=1}^{m} \exp \left( (x_j - \mu_j) / \theta_j \right) + \sum_{S \subset \{1, \dots, m\}, |S| > 1} \exp \left( \sum_{j \in S} (x_j - \mu_j) / \theta_j - \sum_{S' \subset S, |S'| > 1} q_{S'} \right) \right]^2} \quad (2.9)
$$

We can extend the definition (2.7) so that $q_S$ is defined for all subsets of
$\{1, \dots, m\}$, and also find a simpler formula for the density. For $S \subset \{1, \dots, m\}$,
define $b_S' := K^{-1/2} b_S$, so that the density becomes

$$
f(\mathbf{x}|\mathbf{b}', \bar{\mu}, \bar{\theta}) = \frac{\exp\left(\sum_{j=1}^{m} (x_j - \mu_j)/\theta_j\right)}{\left[\sum_{S \subset \{1, \dots, m\}} b'_S \exp\left(\sum_{j \in S} x_j / \theta_j\right)\right]^2}
$$

Define

$$
q_S := - \sum \left\{ (-1)^{|S|-|S'|} \log b'_{S'} : S' \subset S \right\}. \quad (2.10)
$$

By this definition, $q_\emptyset = -\log b'_\emptyset = \log(K^{1/2}) = (\log K)/2$; for $1 \le j \le m$,

$$
q_j = - \log b'_j + \log b'_0 = - \log(K^{-1/2} e^{-\mu_j / \theta_j}) - (\log K)/2 = \mu_j / \theta_j. \quad (2.11)
$$

Definitions (2.7) and (2.10) agree if $|S| > 0$, since $(b'_{S'}/b_{S'}) = K^{-1/2}$ for all $S' \subset \{1, \dots, m\}$. Then it is not hard to show that for all $S \subset \{1, \dots, m\}$,

$$
\log b'_{S} = - \sum \{q_{S'} : S' \subset S\}. \qquad (2.12)
$$
By (2.12), formula (2.4) can be written

$$f(\mathbf{x}|\mathbf{q}, \bar{\theta}) = \frac{\exp\left(\sum_{j=1}^{m} \left[(x_j/\theta_j) - q_j\right]\right)}{\left[\sum_{S \subset \{1, \dots, m\}} \exp\left(\sum_{j \in S} (x_j/\theta_j) - \sum_{S' \subset S} q_{S'}\right)\right]^2} \quad (2.13)$$

We will use formulas (2.9) and (2.13) from now on.

## 2.6 A change of variables and integration

Let

$$\log \frac{p_j}{1-p_j} := \frac{x_j - \mu_j}{\theta_j}$$

so that

$$x_j = \theta_j \log \frac{p_j}{1-p_j} + \mu_j \quad \text{and} \quad dx_j = \frac{\theta_j}{p_j(1-p_j)} dp_j. \quad (2.14)$$

## 2.7 Normalizing constant

Now look at the marginal distribution given in Equation (2.6). This can be written

$$g(\mathbf{x}) = K \theta_j \theta_k \exp \left( \sum_{\ell \neq j, k} \frac{x_\ell - \mu_\ell}{\theta_\ell} \right) \frac{\log (A_1(\mathbf{x}) B_2(\mathbf{x}) / A_2(\mathbf{x}) B_1(\mathbf{x}))}{A_1(\mathbf{x}) B_2(\mathbf{x}) - A_2(\mathbf{x}) B_1(\mathbf{x})} \quad (2.15)$$

where $A_1, B_1, A_2, B_2$ are the polynomials of Equation (2.6). Now apply the change of variables defined by (2.14). Then $g(\mathbf{x})$ becomes $\tilde{g}(\mathbf{p})$. Integrating $g(\mathbf{x})$ over $\mathbb{R}^{m-2}$ is the same as integrating $\tilde{g}(\mathbf{p})$ $[\prod_{\ell \neq j, k} (dx_\ell/dp_\ell)]$ over $(0, 1)^{m-2}$.

Now, for convenience, we will set $H := \{1, \dots, m\}/\{j, k\}$. Remember that $g(\mathbf{x})$, and $\tilde{g}(\mathbf{p})$ $[\prod_{\ell \neq j, k} (dx_\ell/dp_\ell)]$, are probability densities; their integral is one. We can define polynomials $A_1, B_1, A_2, B_2$ of $\mathbf{p} = \{p_\ell : \ell \in H\}$. These are derived from $A_1, B_1, A_2, B_2$ of Section 2.4, by change of variables. See Appendix A for derivation and formulas for $A_1, B_1, A_2, B_2$.

$$G(\mathbf{p}) := \begin{cases} \left(\prod_{\ell=1}^{m} \theta_{\ell}\right) \frac{\log (A_1(\mathbf{p}) B_2(\mathbf{p}) / A_2(\mathbf{p}) B_1(\mathbf{p}))}{A_1(\mathbf{p}) B_2(\mathbf{p}) - A_2(\mathbf{p}) B_1(\mathbf{p})} & \text{if } A_1(\mathbf{p}) B_2(\mathbf{p}) \neq A_2(\mathbf{p}) B_1(\mathbf{p}) \\ \left(\prod_{\ell=1}^{m} \theta_{\ell}\right) & \text{if } A_1(\mathbf{p}) B_2(\mathbf{p}) = A_2(\mathbf{p}) B_1(\mathbf{p}) \end{cases}$$

$G(\mathbf{p})$ is just $\tilde{g}(\mathbf{p})$ $[\prod_{\ell \in H} (dx_\ell/dp_\ell)]$. Then we see that

$$K = \left[ \int_{0}^{1} \cdots \int_{0}^{1} G(\mathbf{p}) \prod_{\ell \in H} dp_{\ell} \right]^{-1} \quad (2.16)$$

This is a practical formula for the normalizing constant. It is an integral of a well-defined function over a hypercube of dimension $m-2$; it should be amenable to numerical integration, if $m$ is not large.
## 2.8 Marginal distributions

Suppose we want to find the marginal distribution of {$X_j : j \in T$}, where $T \subset \{1, \dots, m\}$, and $1 \le |T| \le m-2$. Choose $j, k \in \{1, \dots, m\}/T$ and integrate out, to obtain the $g(\mathbf{x})$ of Equation (2.15). Again $H = \{1, \dots, m\}/\{j, k\}$.

We must still integrate out $x_j, j \in H/T$. If we apply the transformation of Equation (2.14) to {$x_j : j \in H/T$}, we must also multiply the resulting function by $[\prod_{\ell \in H/T}(dx_\ell/dp_\ell)]$ to obtain

$$ 
\begin{aligned}
& K \theta_j \theta_k \prod_{\ell \in H/T} \left( \frac{p_\ell}{1-p_\ell} \right) \exp \left( \sum_{\ell \in T} \frac{x_\ell - \mu_\ell}{\theta_\ell} \right) \frac{\log \left( A_1^{[T]}(\mathbf{p}, \mathbf{x}) B_2^{[T]}(\mathbf{p}, \mathbf{x}) / A_2^{[T]}(\mathbf{p}, \mathbf{x}) B_1^{[T]}(\mathbf{p}, \mathbf{x}) \right)}{A_1^{[T]}(\mathbf{p}, \mathbf{x}) B_2^{[T]}(\mathbf{p}, \mathbf{x}) - A_2^{[T]}(\mathbf{p}, \mathbf{x}) B_1^{[T]}(\mathbf{p}, \mathbf{x})} \\
& \quad * \prod_{\ell \in H/T} \frac{\theta_\ell}{p_\ell(1-p_\ell)} \\
&= K \left( \prod_{\ell \in \{1, \dots, m\}/T} \theta_\ell \right) \exp \left( \sum_{\ell \in T} \frac{x_\ell - \mu_\ell}{\theta_\ell} \right) \frac{\log \left( A_1^{[T]'}(\mathbf{p}, \mathbf{x}) B_2^{[T]'}(\mathbf{p}, \mathbf{x}) / A_2^{[T]'}(\mathbf{p}, \mathbf{x}) B_1^{[T]'}(\mathbf{p}, \mathbf{x}) \right)}{A_1^{[T]'}(\mathbf{p}, \mathbf{x}) B_2^{[T]'}(\mathbf{p}, \mathbf{x}) - A_2^{[T]'}(\mathbf{p}, \mathbf{x}) B_1^{[T]'}(\mathbf{p}, \mathbf{x})} \quad (2.17)
\end{aligned}
$$

where $A_1^{[T]}(\mathbf{p}, \mathbf{x})$ is $A_1(\mathbf{x})$ with $x_\ell$ transformed for $\ell \in H/T$, and $B_1^{[T]}$, $A_2^{[T]}$, and $B_2^{[T]}$ are defined similarly. The exact formulas for $A_1^{[T]}$, $B_1^{[T]}$, $A_2^{[T]}$, and $B_2^{[T]}$ are given in Appendix A. The expression in Equation (2.17) can be integrated with respect to {$p_\ell, \ell \in H/T$} over a hypercube of dimension $|H| - |T| = m - 2 - |T|$ to obtain the marginal density for {$X_\ell : \ell \in T$}.

## 3 Reflections and Symmetries

In this section we show some mathematical properties of the MCL. These will help to justify the parametrization described in Section 2.5.

**Definition 3.1.** Let $\mathbf{p}$ be a point in $\mathbb{R}^m$, and let $S \subset \{1, \dots, m\}$. The S-reflection around $\mathbf{p}$ is defined as follows: For any $x \in \mathbb{R}^m$, $R(\mathbf{x}, \mathbf{p}|S)$ is a point in $\mathbb{R}^m$ such that

$$ R(\mathbf{x}, \mathbf{p}|S)_j = \begin{cases} 2p_j - x_j & \text{if } j \in S \\ x_j & \text{if } j \notin S \end{cases} $$

The following theorem shows how the parametrization of an MCL, as defined in Section 2.5, changes under any reflection.

**Theorem 3.1.** Let $\mathbf{p} \in \mathbb{R}^m$, $\mathbf{q} = \{q_S : S \subset \{1, \dots, m\}\} \in \mathbb{R}^{2m}$, and $\bar{\theta} \in (\mathbb{R}^+)^m$. $\mathbf{q}$ is a set of location-interaction parameters as defined in Equation (2.13). (So $q_\emptyset$ is determined by the rest of $\mathbf{q}$). Let $T$ be a nonempty subset of $\{1, \dots, m\}$. Then for any $\mathbf{x} \in \mathbb{R}^m$, $f(R(\mathbf{x}, \mathbf{p}|T)|\mathbf{q}, \bar{\theta}) = f(\mathbf{x}|\mathbf{q}^*, \bar{\theta})$, where

$$ i) q_S^* = (-1)^{|S \cap T|} \sum_{S' \subset T/S} q_{S \cup S'} \text{ for } S \subset \{1, \dots, m\}, |S| > 1 $$
$$ii) q_j^* = \frac{2p_j}{\theta_j} - q_j - \sum_{S \subset T/\{j\}, |S|>0} q_{S \cup \{j\}} \quad \text{if } j \in T$$

$$iii) q_j^* = q_j + \sum_{S \subset T, |S|>0} q_{S \cup \{j\}} \quad \text{if } j \notin T$$

$$iv) q_0^* = q_0 + \frac{1}{2} \sum_{\ell \in \{1, \dots, m\} / T} \left( \sum_{S \subset T, |S|>0} q_{S \cup \{\ell\}} \right) + \frac{1}{2} \sum_{S \subset T, |S|>2} (2 - |S|) q_S.$$

**Proof.** See Appendix B.

Notice that if $T = \{1, \dots, m\}$, the sum $\sum_{\ell \in \{1, \dots, m\} / T}$ is empty, so that iv) reduces to $q_0^* = q_0 + \frac{1}{2} \sum_{S \in \{1, \dots, m\}, |S|>2} (2 - |S|) q_S$. So while the normalizing constant is determined by interactions $(\{q_S, |S|>1\})$, the change in the normalizing constant when reflecting around some point $\mathbf{p}$ is determined by higher-order interactions.

## 3.1 Symmetries

Let $\mathbf{p} \in \mathbb{R}^m$ and $T \subset \{1, \dots, m\}$.

**Definition 3.2.** A function $g$ on $\mathbb{R}^m$ is said to be symmetric with respect to the reflection $R(\cdot, \mathbf{p}|T)$ if $g(\mathbf{x}) = g(R(\mathbf{x}, \mathbf{p}|T))$ for all $\mathbf{x} \in \mathbb{R}^m$.

An MCL is determined by the location-interaction constants $\mathbf{q}$ and variances $\bar{\theta}$. Theorem 3.1 says that all reflections $R(\cdot, \mathbf{p}|T)$ preserve variances. So for an MCL to be symmetric with respect to $R(\cdot, \mathbf{p}|T)$, the reflection must preserve $\mathbf{q}$. That is, in the statement of Theorem 3.1, $\mathbf{q}^*$ can be replaced by $\mathbf{q}$. So we have: If $S \subset \{1, \dots, m\}$ and $|S| > 1$,

$$q_S = (-1)^{|S \cap T|} \sum_{S' \subset T/S} q_{S \cup S'} = (-1)^{|S \cap T|} \left( q_S + \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'} \right).$$

If $|S \cap T|$ is even, this is

$$q_S = q_S + \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'}, \quad \text{or} \quad \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'} = 0. \quad (3.1)$$

If $|S \cap T|$ is odd, we obtain

$$q_S = -q_S - \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'}, \quad \text{or} \quad q_S = -\frac{1}{2} \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'}.$$

If $j \in T$, then

$$q_j = \frac{2p_j}{\theta_j} - q_j - \sum_{S \subset T/\{j\} : |S|>0} q_{S \cup \{j\}}, \quad \text{or} \quad q_j = \frac{p_j}{\theta_j} - \frac{1}{2} \sum_{S \subset T/\{j\} : |S|>0} q_{S \cup \{j\}}.$$

If $j \notin T$, then

$$q_j = q_j + \sum_{S \subset T : |S|>0} q_{S \cup \{j\}}, \quad \text{or} \quad \sum_{S \subset T : |S|>0} q_{S \cup \{j\}} = 0.$$
Notice that this is Equation (3.1) with $S$ equal to $\{j\}$, but $|S \cap T| = |\{j\} \cap T| = 0$. So we have

**Corollary 3.1.** An MCL is symmetric with respect to a reflection $R(\cdot, \mathbf{p}|T)$ if and only if

1. For any $S \subset \{1, \dots, m\}$ such that $|S| > 0$ and $|S \cap T|$ is even,

$$ \sum_{S' \in T/S: |S'|>0} q_{S \cup S'} = 0. $$

2. For any $S \subset \{1, \dots, m\}$ such that $|S| > 1$ and $|S \cap T|$ is odd,

$$ q_S = -\frac{1}{2} \sum_{S' \in T/S: |S'|>0} q_{S \cup S'}. $$

3. For all $j \in T$,

$$ q_j = \frac{p_j}{\theta_j} - \frac{1}{2} \sum_{S \subset T / \{j\} : |S| > 0} q_{S \cup \{j\}}. $$

4. $\sum_{\ell \in \{1, \dots, m\} / T} \left( \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}} \right) + \sum_{S \subset T : |S| > 2} (2 - |S|) q_S = 0$.

The Corollary seems to impose rather elaborate conditions for symmetry. However, the next theorem shows that the requirements for symmetry are much simpler.

**Theorem 3.2.** Let $m$ be a positive integer and $T$ a nonempty subset of $\{1, \dots, m\}$. Let $\mathbf{q} = \{q_S : S \subset \{1, \dots, m\}\}$ be a set of location-interaction constants, and $\bar{\theta} = \{\theta_j : 1 \le j \le m\} \subset (\mathbb{R}^+)^m$ a vector of variances.

A) The MCL density $f(\cdot|\mathbf{q}, \bar{\theta})$ is symmetric with respect to $R(\cdot, \mathbf{p}|T)$ for some $\mathbf{p} = \{p_\ell : \ell \in T\}$ if and only if

I) For any subset $S \subset \{1, \dots, m\}$ such that $|S \cap T|$ is odd and $|S| > 1$,

$$ q_S = -\frac{1}{2} \sum_{S' \subset T/S: |S'|>0} q_{S \cup S'}. $$

If condition I) holds, then $\mathbf{p}$ is given by $p_j = \theta_j (q_j + \frac{1}{2} \sum_{S \subset T / \{j\}} q_{S \cup \{j\}})$, all $j \in T$.

B) Given $\mathbf{p} = \{p_\ell : \ell \in T\}$. The MCL density $f(\cdot|\mathbf{q}, \bar{\theta})$ is symmetric with respect to $R(\cdot, \mathbf{p}|T)$ if and only if these conditions are satisfied: Condition I), and

$$ \text{II) } q_j = \frac{p_j}{\theta_j} - \frac{1}{2} \sum_{S \subset T / \{j\}} q_{S \cup \{j\}}, \forall j \in T. $$

**Proof.** See Appendix B.

Part A) of Theorem 3.2 says that the MCL is symmetric with respect to the partial reflection defined by $T$ if condition I) holds; if so, the partial reflection
must have center **p** defined by {$q_j : j \in T$}. Part B) says that if we specify both T and **p** = {$p_j : j \in T$}, then conditions I) and II) must both be satisfied to have symmetry; that is, the location parameters for T, {$q_j : j \in T$} are now specified.

The proof of Theorem (3.2) shows that if condition 2) of the Corollary holds, then condition 1) will also hold, at least for higher-order interactions; and if conditions 1) and 2) hold, then condition 4) holds (since the normalizing constant is determined by higher-order interactions). Condition 1) puts no restriction on the value of $q_S$ if $|S \cap T|$ is even; Condition 2) says that if $S \cap T$ is odd, the interaction term $q_S$ is determined by {$q_{S'} : S \subset S'$}, that is, the higher-order interactions that ‘contain’ S. So Theorem 3.2 says that if we wish an MCL density to have symmetry with respect to a reflection defined by $T \subset \{1, \dots, m\}$, the interaction terms $q_S$ can be chosen freely, for all S such that $|S \cap T|$ is even. Then for S such that $|S \cap T|$ is odd and $|S| > 1$, $q_S$ is fixed by Condition I).

**Example.** Theorem 3.2 may seem abstract. Here is a simple example of how it may be applied. Let $m = 3$, and $T = \{1, 2, 3\}$. Then the only subset $S \subset \{1, 2, 3\}$ such that $|S \cap T|$ is odd and $|S| > 1$ is T itself. Then the set of subsets {$S' \subset T/S, |S'| > 0$} is empty, and

$$ \sum_{S' \subset T/S: |S'| > 0} q_{S \cup S'} = 0. $$

So Condition I) of Theorem 3.2 reduces to: the MCL distribution for $m = 3$ is symmetric about some point **p** $\in \mathbb{R}^3$ if and only if $q_{123} = 0$. If this is true, then the distribution is symmetric about the point

$$ \theta_1(q_1 + 0.5q_{12} + 0.5q_{13}), \theta_2(q_2 + 0.5q_{12} + 0.5q_{23}), \theta_3(q_3 + 0.5q_{13} + 0.5q_{13}) \\ = (\mu_1 + 0.5\theta_1(q_{12} + q_{13}), \mu_2 + 0.5\theta_2(q_{12} + q_{23}), \mu_3 + 0.5\theta_3(q_{13} + q_{23})). $$

## 3.2 A result for multiple reflections

The next result concerns sequences of points in $\mathbb{R}^m$. We will assume that such a sequence will go to infinity in all coordinates. However, we do not want to assume that the sequence is confined to a particular quadrant of $\mathbb{R}^m$, or that it follows one direction.

**Definition 3.3.** Let **x** and **p** be two points in $\mathbb{R}^m$. Define a point $\mathbf{x}^{*,\mathbf{p}}$ as follows: for $j = 1, \dots, m$,

$$ x_j^{*,\mathbf{p}} := \max(x_j, 2p_j - x_j). $$

$\mathbf{x}^{*,\mathbf{p}} = R(\mathbf{x}, \mathbf{p}|S)$ for the set $S = \{j \in \{1, \dots, m\} : 2p_j - x_j > x_j\}$. $\mathbf{x}^{*,\mathbf{p}}$ has been defined so that $\mathbf{x}^{*,\mathbf{p}} - \mathbf{p}$ is in the first quadrant (positive in all coordinates), even if $\mathbf{x}$ may not be in the first quadrant.
**Theorem 3.3.** Let {$\mathbf{x}_s : s = 1, 2, \dots, \}$} and {$\mathbf{p}_s : s = 1, 2, \dots, \}$} be sequences in $\mathbb{R}^m$ such that for all $j \in \{1, \dots, m\}$,

$$ 
\begin{aligned}
\max(x_{j,s}, 2p_{j,s} - x_{j,s}) & \to \infty \quad \text{as } s \to \infty \\
\min(x_{j,s}, 2p_{j,s} - x_{j,s}) & \to -\infty \quad \text{as } s \to \infty
\end{aligned}
\qquad (3.2) $$

Then for any nonempty set $U \subset \{1, \dots, m\}$,
(I) if $|U| > 1$, then

$$ 
\begin{aligned}
\lim_{s \to \infty} \sum \{(-1)^{|T \cap U|} \log f(R(\mathbf{x}_s^*, \mathbf{p}_s, \mathbf{p}_s|\{1, \dots, m\}/U \cup T)|\mathbf{q}, \bar{\theta}) : T \subset U\} \\
= -2q_U
\end{aligned}
$$

(II) If $U = \{j\}$, some $j \in \{1, \dots, m\}$, then

$$ 
\begin{aligned}
\lim_{s \to \infty} \left[ \sum \{(-1)^{|T \cap U|} \log f(R(\mathbf{x}_s^*, \mathbf{p}_s, \mathbf{p}_s|\{1, \dots, m\}/U \cup T)|\mathbf{q}, \bar{\theta}) : T \subset U\} + 2p_{j,s}/\theta_j \right] \\
= -2q_j
\end{aligned}
$$

and for (I) and (II), the rate of convergence is $O(\exp(-\min_j(x_{j,s} - p_{j,s})/\theta_j))$.

**Proof.** See Appendix B.

Theorem 3.3 says that taking a signed sum of logarithms of the density around vertices for a “face” of a hyperrectangle will yield a parameter associated with the subset of $\{1, \dots, m\}$ defined by that “face”. This result, however, is asymptotic. It will converge to the parameter value as the size of the hyperrectangle grows. Also, Theorem 3.3 does not assume a single center point $\mathbf{p}$. The conditions of the theorem allow {$\mathbf{p}_s$} to diverge also. But $|\mathbf{p}_s|$ cannot grow as fast as $|x_{j,s}|$. Also Theorem 3.3 does not require that $x_{j,s}$ diverge at a similar rate, for $j = 1, \dots, m$.

# 4 Example: An application to a data set

Use of the logistic distribution is usually restricted to certain methods, namely, logistic regression and (to a lesser extent) modeling life test data (see [Plackett (1958)]). However, we can make a simple transformation: instead of

$$ F(x) = \frac{\exp((x - \mu)/\theta)}{1 + \exp((x - \mu)/\theta)} $$

use

$$ F(x) = \frac{\exp((g(x) - \mu)/\theta)}{1 + \exp((g(x) - \mu)/\theta)} \qquad (4.1) $$

If a random variable $X$ follows a distribution of form 4.1, then $g(X)$ follows a logistic distribution.

We will use the case where

$$ g(x) = \logit x = \log\left(\frac{x}{1-x}\right). \qquad (4.2) $$
This distribution is described in [Tadikamalla and Johnson (1982)], along with distributions based on other choices for g. They did not give a name for the distribution based on 4.2. In the case that g(x) = log x, they named the resulting distribution the ‘log-logistic’. We will call the distribution defined by Equations 4.1 and 4.2 the ‘logit-logistic’. This is a distribution with (0, 1) as support, and is a potential alternative to the beta distribution. Just as we derived the multivariate conditional logistic distribution from the univariate logistic distribution, we can derive a multivariate conditional logit-logistic from the univariate logit-logistic distribution. See Appendix C for details.

## 4.1 The data set

We use a simulated data set. The data is based on 1000 random walks. Each random walk is generated with a set of four chosen parameters: $\kappa_1$, $\kappa_2$, $\mu$, and $K$. The three outcomes are proportions. Each is a property of a random walk. Each proportion can be considered an outcome, and the four parameters are the potential predictors. For a more detailed description of how the random walks were generated, and how the outcomes were calculated, see the supplementary file.

## 4.2 Modeling

We began analysis by modeling individual proportions. There exists a method for fitting a regression model for an outcome that is a proportion: Beta regression. See [Ferrari and Cribari-Neto (2004)]. We modeled each outcome, using the R package betareg (see [Cribari-Neto and Zeileis (2010)]). The potential predictors were the parameters $\kappa_1$, $\kappa_2$, $\mu$, and $K$. By trial and error, it was established that each outcome could be modeled using $\kappa_1$ and $K$, and $K$ was of little strength.

The three proportions were modeled using multivariate conditional logit-logistic (MCLL) distributions. For observations i, $p_i = (p_{i1}, p_{i2}, p_{i3})$ was assumed to be drawn from a MCLL. This distribution has three location parameters $\mu$, three variance parameters $\theta$, and four interactions. Further, the location parameters are modeled:

$$ \mu_i = X_i \beta $$

where $\beta$ has the regression coefficients for intercept, $\kappa_1$, and $\mu$ for each outcome, and $X_i$ has 1 (for intercept) but also values of $\kappa_1$ and $\mu$ for observation i. So there are nine regression coefficients, three variance parameters, and four interactions.

Statisticians may prefer to avoid using high order interactions when modeling data. Linear regression models in practice may often have second order interactions between predictor variables, but rarely third order interactions. So, when modeling multiple outcomes, we may prefer to not have higher order interactions among the outcomes. For the data and distributions of this example, this would mean no third order interaction among the three proportions. So
we first fit an MCLL model to the three proportions, with the condition that $q_{123} = 0$. The model was fitted by MCMC, using the Metropolis-Hastings algorithm. Parameter estimates are shown in Table 1. Standard errors were derived from the multiple estimates of each parameter produced by the MCMC.

Now if $(p_1, p_2, p_3)$ follow a multivariate conditional logit-logistic distribution with parameters $\mu, \theta, \mathbf{q}$, then (logit $(p_1)$, logit $(p_2)$, logit $(p_3)$) follow a multivariate conditional logistic distribution with the same parameters. So we are assuming that for observations $i$, (logit $p_{i1}$, logit $p_2$, logit $p_3$) follows an MCL distribution for which $q_{123} = 0$. By the example following Theorem 3.2, such a distribution is symmetric around a centerpoint given by

$$ \mathbf{c}_i = (\mu_1 + \theta_1(q_{12} + q_{13}), \mu_2 + \theta_2(q_{12} + q_{23}), \mu_3 + \theta_3(q_{13} + q_{23})). \quad (4.3) $$

Define

$$ \mathbf{D}_i = (\text{logit } \mathbf{p}_i) - \mathbf{c}_i. $$

Then

$$
\begin{align*}
\text{probability density of } \mathbf{D} &= (\text{logit } \mathbf{p}) - \mathbf{c} \text{ at } \mathbf{x} \\
&= \text{p.d.f of logit } \mathbf{p} \text{ at } \mathbf{c} + \mathbf{x} \\
&= \text{p.d.f of logit } \mathbf{p} \text{ at } 2\mathbf{p} - (\mathbf{c} + \mathbf{x}) = \mathbf{c} - \mathbf{x} \\
&\quad \text{(by symmetry of logit } \mathbf{p} \text{ around } \mathbf{c}) \\
&= \text{p.d.f of } \mathbf{D} = (\text{logit } \mathbf{p}) - \mathbf{c} \text{ at } -\mathbf{x}
\end{align*}
$$

Then the set of deviations ${\mathbf{D}_i}_{i=1}^{1000}$ are all drawn from a multivariate conditional logistic distribution that is symmetric about the origin (0,0,0). That is, ${\mathbf{D}_i}_{i=1}^{1000}$ should be a cloud of points in $\mathbb{R}^3$ that is symmetric about 0. So ${\mathbf{D}_i}_{i=1}^{1000}$ and $\{-\mathbf{D}_i\}_{i=1}^{1000}$ should be very similar. In Figure 1, the two clouds ${\mathbf{D}_i}$ and $\{-\mathbf{D}_i\}$ are plotted together. If our assumption is correct, the two clouds should be intermixed, with little separation. They are not. There are regions of +'s without *'s and regions of *'s without +'s. This strongly indicates that our assumption was wrong; there should be a third order interaction.

Another model was fit, this time with $q_{123}$ allowed to be nonzero. Parameter estimates are shown in Table 2.

## 4.3 Evaluation and interpretation

For comparison of the two models, we use loglikelihood and AIC. The model with zero third order interaction has in effect 15 parameters; that with $q_{123} \neq 0$ has sixteen.

<table><thead><tr><th></th><th>model with q<sub>123</sub> = 0</th><th>model with q<sub>123</sub> &ne; 0</th></tr></thead><tbody><tr><td>loglikelihood</td><td>2228.0</td><td>2234.4</td></tr><tr><td>AIC</td><td>-4426</td><td>-4436.8</td></tr></tbody></table>
So the model with a third order interaction is preferable.

What do the two MCL models tell us about relations between outcomes? The model in Table 1 has positive second order interactions between $p_1$ and $p_2$, and between $p_2$ and $p_3$. The interaction $q_{13}$ between $p_1$ and $p_3$ is not significantly different from zero. This would indicate that $p_1$ and $p_3$ are not associated. The model in Table 2 modifies all this. $q_{12}$ and $q_{23}$ are now larger. $q_{13}$ is smaller than the other second order interactions, but it too is positive and significant. The third order interaction, however, is strongly negative. How do we interpret this? In a model with no third order interaction, the association between $p_k$ and $p_\ell$ is determined by $q_{k\ell}$. If there is a third order interaction, however, the association between $p_k$ and $p_\ell$ is determined by $q_{k\ell} + Y_jq_{123}$, where $j$ is the index of the third outcome, and $Y_k = \text{logit } p_k$. So $q_{123} < 0$ implies: all pairs of outcomes tend to be positively associated; but as the third outcome increases, the association between $p_k$ and $p_\ell$ shrinks and may become negative.

**Goodness of fit.** To see how well a model fits the given data, we must have estimates $Y$ corresponding to outcomes $\mathbf{Y}$. Since we are modeling proportions, the outcomes are $\mathbf{p}$ and the estimates are $\hat{\mathbf{p}}$. We will use pseudo-$R^2$ as a measure of goodness of fit. This is defined in [Ferrari and Cribari-Neto (2004)], and is used in the implementation of beta regression models. The formula is

$$ \text{pseudo-}R^2 = \frac{\sum_{i} (Y_i - \hat{Y}_i)^2}{\sum_{i} (Y_i - \bar{Y})^2} \quad \text{where } Y_i = \text{logit } p_i, \hat{Y} = \text{logit } \hat{p}_i. $$

What do we use for $\hat{p}$? For both models, the assumption is that $Y_i = \text{logit } \hat{p}_i$ is drawn from an MCL distribution, all $i$. For a model with $q_{123} = 0$, this MCL distribution will be symmetric about $c_i$ defined by Equation (4.3). So this distribution has $c_i$ as both mean and median point; $c_i$ is a natural candidate for best estimate of a random draw from this distribution. So we set $\mathbf{Y}_i = c_i$. We also fit beta regression models for all three outcomes, using the same predictor variables. The following table shows values of pseudo-$R^2$ for all outcomes.

<table><thead><tr><th></th><th colspan="3">Proportion outcome</th></tr><tr><th></th><th>p<sub>1</sub></th><th>p<sub>2</sub></th><th>p<sub>3</sub></th></tr></thead><tbody><tr><th>β regression</th><td>0.0628</td><td>0.2475</td><td>0.1437</td></tr><tr><th>c from MCL model<br>with q<sub>123</sub> = 0</th><td>0.0624</td><td>0.2462</td><td>0.1414</td></tr></tbody></table>

So the MCL model with zero third order interaction models individual outcomes almost as well as a standard univariate modeling method. The MCL model also provides information about relations between the outcomes.

Now consider the MCL model with nonzero third order interaction. Since the common distribution for deviations does not have $q_{123} = 0$, this distribution is not symmetric. There is no natural center, so at present we do not have a formula for $\mathbf{Y}$. The AIC values indicate that this MCL model is superior to that with zero third order interaction, so it should be possible to extract values of
$Y$ that will give values of pseudo-$R^2$ that are larger (or at least no worse) than those for the model with zero third order interaction. But at present we do not have a formula for such $Y$.

# 5 Conclusion

We have shown that for any positive integer $m$, a multivariate distribution for variables $(Y_1, \dots, Y_m)$ may be constructed such that $Y_j|\{Y_\ell, \ell \neq j\}$ has a logistic distribution.

We have found a parametrization $\{q_S : S \subset \{1, \dots, m\}\}$ such that $q_S \in \mathbb{R}$, all $S$. By specifying the $\{q_S\}$, the relationships between the outcomes may be controlled. This includes possible independence between $\{Y_\ell : \ell \in S\}$ and $\{Y_\ell : \ell \notin S\}$.

The MCL has these theoretical properties:

* The MCL distribution may be symmetric with respect to some $\ell$-dimensional hyperplane $\{\mathbf{x} \in \mathbb{R}^m : x_j = p_j, j \in S\}$ for some $\mathbf{p} \in \mathbb{R}^m$ and $S \subset \{1, \dots, m\}$. The conditions for such a symmetry may be compactly expressed in terms of the parameters $\{q_S\}$ (see Theorem 3.2).

* For each $S \subset \{1, \dots, m\}$, $|S| > 1$, the interactions parameter $q_S$ may be derived by asymptotically measuring the MCL density at a set of reflected points $\{R(\mathbf{x}_j|\mathbf{p}_j, S') : S' \subset S\}$, where $\mathbf{x}_j \to \infty$.

The fact that symmetries may be so neatly expressed in Theorem 3.2 indicates that the parametrization defined in Section 2.5 is “natural” in some sense.

We do not yet know if a multivariate conditional logistic distribution, as defined, may be used for a real-world application. However, if we apply a transformation of form Equation 4.1, we can derive a multivariate distribution that could be used to model a set of outcomes of interest. The example in Section 4 was for three proportions. There are several areas of study (medicine, social science, etc.) where researchers may be interested in several variables that are proportions, and may want to fit a single joint model for a set of such outcomes, for greater understanding. But Equation 4.1 is not restricted to the case $f(p) = \text{logit } p$.

To be done:

* If we have symmetry about a hyperplane or point $\{x_j = p_j, j \in S \subset \{1, \dots, m\}\}$, then the mean (and median) of distribution for $j \in S$ is given by the hyperplane. But in general, an MCL distribution will not be symmetric about any hyperplane, and there is not a clear way to extract a point estimate $\hat{\mathbf{p}} = (\hat{p}_1, \dots, \hat{p}_m)$ without symmetry.

* The example in Section 4 was fitted using MCMC. This is time consuming. We need a fast and reliable method to fit MCL models.
* The interaction terms control the correlations between the outcome variables. However, the relation between, for example, $q_{ij}$ and $Corr(Y_i, Y_j)$ is not known.

* Theorem 3.2 gives the relation between interaction parameters and symmetries. In the example of Section 4, the analysis of the first model gives a simple instance how Theorem 3.2 might be used in data analysis. At present, however, it is not clear how the relation between symmetries and interactions given by the theorem could be used in data analysis and modeling.

## References

[Ali, Mikhail, and Haq (1978)] Ali, M., Mikhail, N, and Haq, M. (1978) A class of bivariate distributions including the bivariate logistic. *J. of Multivariate Analysis*, Vol. 8, pp. 405-412.

[Arnold (1987)] Arnold, B. (1987) Bivariate distributions with Pareto conditionals. *Statistics & Probability Letters*, Vol. 5, no. 4, pp. 263-266.

[Balakrishnan (1992)] Balakrishnan, N. (1992) *Handbook of the logistic distribution*. New York: Marcel Dekker.

[Berkson (1944)] Berkson, J. (1944) Application of the Logistic Function to Bio-Assay. *J. of the American Statistical Association*, Vol. 39, pp. 357-365.

[Cribari-Neto and Zeileis (2010)] Cribari-Neto, F. and Zeileis, A. (2010) Beta Regression in R. *J. of Statistical Software*, Vol. 34, no. 2.

[Ferrari and Cribari-Neto (2004)] Ferrari, S. and Cribari-Neto, F. (2004) Beta Regression for Modeling Rates and Proportion. *Journal of Applied Statistics*, Vol. 31, no. 7, pp. 799-815.

[Glonek and McCullagh (1995)] Glonek, G.F.V. and McCullagh, P. (1995) Multivariate Logistic Models. *J. of the Royal Statistical Society, Series B*, vol. 57, no. 3, pp. 533-546.

[Gumbel (1961)] Gumbel, E.J. (1961) Bivariate logistic distributions. *J. of the American Statistical Association*, Vol. 56, pp. 335-349.

[Kotz, Balakrishnan, and Johnson (2000)] Kotz, S., Balakrishnan, N., and Johnson, N. (2000) *Continuous Multivariate Distributions, Vol. 1: Models and Applications*. Wiley-Interscience.

[Malik and Abraham (1973)] Malik, H. and Abraham, B. (1973) Multivariate logistic distributions. *Ann. Statistics*, Vol 1, no. 3, pp. 588-590.

[Plackett (1958)] Plackett, R.L. (1958) Linear estimation from censored data. *Ann. Math. Stat.*, Vol. 29, no. 1, pp. 131-142.
[Tadikamalla and Johnson (1982)] Tadikamalla P. and Johnson, N. (1982) Systems of frequency curves generated by transformations of logistic variables. *Biometrika*, vol. 60, no. 2, pp. 461-5.

[Yeh (2010)] Yeh, H-C. (2010) Multivariate semi-logistic distributions. *J. of Multivariate Analysis*, Vol. 101, pp. 893-908.

<figure><img src="image_2.png" /><figcaption>Figure 1: Deviations derived from *logitp* and the center points *c*. The deviations are plotted with blue '*'; the negatives of the deviations are plotted with red '+'.</figcaption></figure>
<table><caption>Table 1: Parameters of multivariate conditional logit-logistic model for simulated data of example in Section 4. This is model with $q_{123} = 0$.</caption><thead><tr><th></th><th>estimate</th><th>SE</th><th>t-values</th><th>p-values</th></tr></thead><tbody><tr><td>intercept(p<sub>1</sub>)</td><td>-2.83820</td><td>0.08845</td><td>-32.08640</td><td>0.00000</td></tr><tr><td>κ<sub>1</sub>(p<sub>1</sub>)</td><td>8.49177</td><td>1.00958</td><td>8.41121</td><td>0.00000</td></tr><tr><td>K(p<sub>1</sub>)</td><td>-7.3991×10<sup>-03</sup></td><td>1.6663×10<sup>-03</sup></td><td>-4.4403</td><td>8.9825×10<sup>-06</sup></td></tr><tr><td>intercept(p<sub>2</sub>)</td><td>-4.29708</td><td>0.12314</td><td>-34.89644</td><td>0.00000</td></tr><tr><td>κ<sub>1</sub>(p<sub>2</sub>)</td><td>27.85075</td><td>1.40105</td><td>19.87849</td><td>0.00000</td></tr><tr><td>K(p<sub>2</sub>)</td><td>-0.01146</td><td>0.00085</td><td>-13.47336</td><td>0.00000</td></tr><tr><td>intercept(p<sub>3</sub>)</td><td>0.58283</td><td>0.06905</td><td>8.44009</td><td>0.00000</td></tr><tr><td>κ<sub>1</sub>(p<sub>3</sub>)</td><td>-8.13418</td><td>0.66731</td><td>-12.18946</td><td>0.00000</td></tr><tr><td>K(p<sub>3</sub>)</td><td>5.5891×10<sup>-03</sup></td><td>8.5469×10<sup>-04</sup></td><td>6.5393</td><td>6.1789×10<sup>-11</sup></td></tr><tr><td>log θ<sub>1</sub></td><td>-0.98047</td><td>0.02544</td><td>-38.54754</td><td>0.00000</td></tr><tr><td>log θ<sub>2</sub></td><td>-0.61952</td><td>0.02369</td><td>-26.14837</td><td>0.00000</td></tr><tr><td>log θ<sub>3</sub></td><td>-1.23823</td><td>0.02635</td><td>-46.99475</td><td>0.00000</td></tr><tr><td>q<sub>12</sub></td><td>4.18505</td><td>0.22705</td><td>18.43254</td><td>0.00000</td></tr><tr><td>q<sub>13</sub></td><td>0.13786</td><td>0.20796</td><td>0.66292</td><td>0.50738</td></tr><tr><td>q<sub>23</sub></td><td>1.6907</td><td>0.23156</td><td>7.3016</td><td>2.8444×10<sup>-13</sup></td></tr></tbody></table>

<table><caption>Table 2: Parameters of multivariate conditional logit-logistic model for simulated data of example in Section 4. This is model with nonzero $q_{123}$.</caption><thead><tr><th></th><th>estimate</th><th>SE</th><th>t-values</th><th>p-values</th></tr></thead><tbody><tr><td>intercept(p<sub>1</sub>)</td><td>-3.31792</td><td>0.15965</td><td>-20.78223</td><td>0.00000</td></tr><tr><td>κ<sub>1</sub>(p<sub>1</sub>)</td><td>8.76672</td><td>0.98275</td><td>8.92061</td><td>0.00000</td></tr><tr><td>K(p<sub>1</sub>)</td><td>-0.0083858</td><td>0.0029328</td><td>-2.8593361</td><td>0.0042453</td></tr><tr><td>intercept(p<sub>2</sub>)</td><td>-5.06148</td><td>0.21388</td><td>-23.66498</td><td>0.00000</td></tr><tr><td>κ<sub>1</sub>(p<sub>2</sub>)</td><td>28.12652</td><td>1.39896</td><td>20.10529</td><td>0.00000</td></tr><tr><td>K(p<sub>2</sub>)</td><td>4.0281×10<sup>-04</sup></td><td>1.2355×10<sup>-03</sup></td><td>032602</td><td>0.74441</td></tr><tr><td>intercept(p<sub>3</sub>)</td><td>0.15210</td><td>0.12964</td><td>1.17322</td><td>0.24071</td></tr><tr><td>κ<sub>1</sub>(p<sub>3</sub>)</td><td>-7.93675</td><td>0.65021</td><td>-12.20637</td><td>0.00000</td></tr><tr><td>K(p<sub>3</sub>)</td><td>4.5996×10<sup>-03</sup></td><td>6.7788×10<sup>-04</sup></td><td>6.7853e+00</td><td>1.1582×10<sup>-11</sup></td></tr><tr><td>log θ<sub>1</sub></td><td>-0.99160</td><td>0.02581</td><td>-38.41342</td><td>0.00000</td></tr><tr><td>log θ<sub>2</sub></td><td>-0.62651</td><td>0.02464</td><td>-25.42266</td><td>0.00000</td></tr><tr><td>log θ<sub>3</sub></td><td>-1.24546</td><td>0.02953</td><td>-42.17786</td><td>0.00000</td></tr><tr><td>q<sub>12</sub></td><td>5.97901</td><td>0.48991</td><td>12.20425</td><td>0.00000</td></tr><tr><td>q<sub>13</sub></td><td>1.9117</td><td>0.46113</td><td>4.1457</td><td>3.3874×10<sup>-05</sup></td></tr><tr><td>q<sub>23</sub></td><td>3.4534</td><td>0.47352</td><td>7.2931</td><td>3.0309×10<sup>-13</sup></td></tr><tr><td>q<sub>123</sub></td><td>-2.9207</td><td>0.61845</td><td>-4.7226</td><td>2.3280×10<sup>-06</sup></td></tr></tbody></table>
# Appendices

## Contents:

* A Computation for formulas in Section 2

* B Proofs of results in Section 3

* C Description of random walks used in example in Section 4

# A Computation for formulas in Section 2

## A.1 For Section 2.4

### Derivation of first marginal distribution

Pick $j \in \{1, \dots, m\}$. The MCL density can be written

$$ \frac{C e^{(x_j - \mu_j)/\theta_j}}{(A + B e^{(x_j - \mu_j)/\theta_j})^2} $$

where

$$ C = K \exp \left( \sum_{\ell \neq j} (x_{\ell} - \mu_{\ell}) / \theta_{\ell} \right) $$

$$ A = \sum \{c_S \exp \left( \sum_{\ell \in S} (x_{\ell} - \mu_{\ell}) / \theta_{\ell} \right) : S \in \{1, \dots, m\} / \{j\} \} $$

$$ B = \sum \{c_{S \cup \{j\}} \exp \left( \sum_{\ell \in S} (x_{\ell} - \mu_{\ell}) / \theta_{\ell} \right) : S \in \{1, \dots, m\} / \{j\} \} $$

Here the {$c_S$} are as in definition of the MCL given in Equation 2.4; they are defined for all subsets of {$1, \dots, m$}. It can easily be calculated that

$$ \begin{aligned} & \int_{-\infty}^{\infty} \frac{C e^{(x_j - \mu_j)/\theta_j}}{(A + B e^{(x_j - \mu_j)/\theta_j})^2} dx_j \\ &= \lim_{s \to \infty} \frac{C \theta_j}{B} \left( - \frac{1}{A + B e^{(x_j - \mu_j)/\theta_j}} \right) \Big|_{x_j = -s}^{x_j = s} = \frac{C \theta_j}{AB} \end{aligned} \quad (A.1) $$

This is the formula of Equation 2.5.
## Derivation of second marginal distribution

Choose another $k \in \{1, \dots, m\}$, $k \neq j$. Then $C/AB$ of Equation (2.5) can be written

$$ \frac{C'e^{(x_k - \mu_k)/\theta_k}}{(A_1 + B_1 e^{(x_k - \mu_k)/\theta_k})(A_2 + B_2 e^{(x_k - \mu_k)/\theta_k})}, $$

where

$$ C' = K \theta_j \exp \left( \sum \left\{ \frac{(x_l - \mu_l)}{\theta_l} : l \neq j, k \right\} \right) $$

$$ A_1 = 1 + \sum_{l \neq j, k} \exp \left( \frac{(x_l - \mu_l)}{\theta_l} \right) $$

$$ + \sum_S c_S \exp \left( \sum_{l \in S} \frac{(x_l - \mu_l)}{\theta_l} \right) $$

$$ B_1 = 1 + \sum_S c_{S \cup \{k\}} \exp \left( \sum_{l \in S} \frac{(x_l - \mu_l)}{\theta_l} \right) $$

$$ A_2 = 1 + \sum_S c_{S \cup \{j\}} \exp \left( \sum_{l \in S} \frac{(x_l - \mu_l)}{\theta_l} \right) $$

$$ B_2 = c_{jk} + \sum_S c_{S \cup \{j,k\}} \exp \left( \sum_{l \in S} \frac{(x_l - \mu_l)}{\theta_l} \right) \quad (A.2) $$

The sums for $A_1, B_1, A_2, B_2$ are over $\{S \subset \{1, \dots, m\} / \{j, k\} : |S| > 1\}$.

Now

$$ \frac{C' e^{(x_k - \ell_k)/\theta_k}}{(A_1 + B_1 e^{(x_k - \ell_k)/\theta_k})(A_2 + B_2 e^{(x_k - \ell_k)/\theta_k})} = \frac{C' \theta_k}{A_1 B_2 - A_2 B_1} \left[ \frac{B_2 e^{(x_k - \ell_k)/\theta_k}}{\theta_k(A_2 + B_2 e^{(x_k - \ell_k)/\theta_k})} - \frac{B_1 e^{(x_k - \ell_k)/\theta_k}}{\theta_k(A_1 + B_1 e^{(x_k - \ell_k)/\theta_k})} \right] $$

and

$$ \begin{align*} & \frac{C' \theta_k}{A_1 B_2 - A_2 B_1} \int_{-\infty}^{\infty} \left[ \frac{B_2 e^{(x_k - \ell_k)/\theta_k}}{\theta_k(A_2 + B_2 e^{(x_k - \ell_k)/\theta_k})} - \frac{B_1 e^{(x_k - \ell_k)/\theta_k}}{\theta_k(A_1 + B_1 e^{(x_k - \ell_k)/\theta_k})} \right] dx_k \\ &= \frac{C' \theta_k}{A_1 B_2 - A_2 B_1} \lim_{s \to \infty} \left[ \log(A_2 + B_2 e^{(x_k - \ell_k)/\theta_k}) - \log(A_1 + B_1 e^{(x_k - \ell_k)/\theta_k}) \right] \Big|_{x_k = -s}^{x_k = s} \\ &= \frac{C' \theta_k}{A_1 B_2 - A_2 B_1} \frac{\log(B_2/B_1) - \log(A_2/A_1)}{A_1 B_2 - A_2 B_1} = \frac{C' \theta_k \log(A_1 B_2 / A_2 B_1)}{A_1 B_2 - A_2 B_1}. \quad (A.3) \end{align*} $$

The last expression is the formula of Equation 2.6.
## A.2 For Section 2.6

For the normalizing constant: Set $H := \{1, \dots, m\}/\{j, k\}$. Then

$$ 
\begin{aligned}
& \tilde{g}(\mathbf{p}) \left[ \prod_{\ell \in H} (dx_{\ell} / dp_{\ell}) \right] \\
& = K \theta_j \theta_k \prod_{\ell \in H} \left( \frac{p_{\ell}}{1 - p_{\ell}} \right) \frac{\log (A'_1(\mathbf{p}) B'_2(\mathbf{p}) / A'_2(\mathbf{p}) B'_1(\mathbf{p}))}{A'_1(\mathbf{p}) B'_2(\mathbf{p}) - A'_2(\mathbf{p}) B'_1(\mathbf{p})} * \prod_{\ell \in H} \frac{\theta_{\ell}}{p_{\ell}(1 - p_{\ell})} \\
& = K \left( \prod_{\ell=1}^{m} \theta_{\ell} \right) \frac{\log (A'_1(\mathbf{p}) B'_2(\mathbf{p}) / A'_2(\mathbf{p}) B'_1(\mathbf{p}))}{A'_1(\mathbf{p}) B'_2(\mathbf{p}) - A'_2(\mathbf{p}) B'_1(\mathbf{p})}.
\end{aligned} 
\quad (A.4) $$

Here $A'_1(\mathbf{p})$ is $A'_1(\mathbf{x})$ of Equation (A.2) with the change of variables of Equation (2.14). That is,

$$ A'_1(\mathbf{p}) = 1 + \sum_{\ell \in H} \frac{p_{\ell}}{1 - p_{\ell}} + \sum_{S \subset H, |S| > 1} \left[ c_S \prod_{\ell \in S} \left( \frac{p_{\ell}}{1 - p_{\ell}} \right) \right]. $$

In the parametrization of Section (2.5), this is

$$ A'_1(\mathbf{p}) = 1 + \sum_{\ell \in H} \frac{p_{\ell}}{1 - p_{\ell}} + \sum_{S \subset H, |S| > 1} \left[ \exp \left( \sum_{S' \subset S, |S'| > 1} q_{S'} \right) \prod_{\ell \in S} \left( \frac{p_{\ell}}{1 - p_{\ell}} \right) \right]. $$

From this, we obtain the formula for $A'_1$ in the last line of Equation (A.4):

$$ 
\begin{aligned}
A'_1(\mathbf{p}) &= \left[ \prod_{\ell \in H} (1 - p_{\ell}) \right] A'_1(\mathbf{p}) \quad (A.5) \\
&= \prod_{\ell \in H} (1 - p_{\ell}) + \sum_{\ell \in H} p_{\ell} \left[ \prod_{s \in H / \{\ell\}} (1 - p_s) \right] + \sum_{S \subset H, |S| > 1} \left[ \exp \left( \sum_{S' \subset S, |S'| > 1} q_{S'} \right) \left( \prod_{\ell \in S} p_{\ell} \right) \left( \prod_{\ell \in H/S} (1 - p_{\ell}) \right) \right].
\end{aligned}
$$
Using the formulas for $B_1$, $A_2$, $B_2$ in Equation (A.2) and the {$q_S$} parametriza-
tion,

$$
\begin{align*}
B_1''(\mathbf{p}) &= \prod_{\ell \in H} (1 - p_\ell) + \sum_{\ell \in H} \left[ \exp(q_{\ell j}) p_\ell \left( \prod_{s \in H/\{\ell\}} (1 - p_s) \right) \right] \\
&\quad + \sum_{S \subset H, |S| > 1} \left[ \exp\left( \sum_{\ell \in H} q_{\ell j} + \sum_{S' \subset S, |S'| > 1} (q_{S'} + q_{S' \cup \{j\}}) \right) \left( \prod_{\ell \in S} p_\ell \right) \left( \prod_{\ell \in H/S} (1 - p_\ell) \right) \right] \\
A_2''(\mathbf{p}) &= \prod_{\ell \in H} (1 - p_\ell) + \sum_{\ell \in H} \left[ \exp(q_{\ell k}) p_\ell \left( \prod_{s \in H/\{\ell\}} (1 - p_s) \right) \right] \\
&\quad + \sum_{S \subset H, |S| > 1} \left[ \exp\left( \sum_{\ell \in H} q_{\ell k} + \sum_{S' \subset S, |S'| > 1} (q_{S'} + q_{S' \cup \{k\}}) \right) \left( \prod_{\ell \in S} p_\ell \right) \left( \prod_{\ell \in H/S} (1 - p_\ell) \right) \right] \\
B_2''(\mathbf{p}) &= \exp(q_{jk}) \prod_{\ell \in H} (1 - p_\ell) + \sum_{\ell \in H} \left[ \exp(q_{jk} + q_{\ell j} + q_{\ell k} + q_{\ell jk}) p_\ell \left( \prod_{s \in H/\{\ell\}} (1 - p_s) \right) \right] \\
&\quad + \sum_{S \subset H, |S| > 1} \left[ \exp\left( q_{jk} + \sum_{\ell \in H} (q_{\ell j} + q_{\ell k} + q_{\ell jk}) + \sum_{S' \subset S, |S'| > 1} (q_{S'} + q_{S' \cup \{j\}} + q_{S' \cup \{k\}} + q_{S' \cup \{jk\}}) \right) \right. \\
&\qquad \left. \cdot \left( \prod_{\ell \in S} p_\ell \right) \left( \prod_{\ell \in H/S} (1 - p_\ell) \right) \right]
\end{align*}
$$
For marginal distributions: Here are the formulas for $A_1^{[T]}$, $B_1^{[T]}$, $A_2^{[T]}$, and $B_2^{[T]}$:

$$A_1^{[T]}(\mathbf{p}, \mathbf{x}) = 1 + \sum_{\ell \in T} \exp\left(\frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) + \sum_{\ell \in H/T} \frac{p_{\ell}}{1 - p_{\ell}} + \sum_{S \in H, |S| > 1} \left[ \exp\left(\sum_{S' \subset S, |S'| > 1} q_{S'}\right) \exp\left(\sum_{\ell \in S \cap T} \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) \prod_{\ell \in S \cap H/T} \frac{p_{\ell}}{1 - p_{\ell}} \right]$$

$$B_1^{[T]}(\mathbf{p}, \mathbf{x}) = 1 + \sum_{\ell \in T} \exp\left(q_{\ell j} + \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) + \sum_{\ell \in H/T} \exp(q_{\ell j}) \frac{p_{\ell}}{1 - p_{\ell}} + \sum_{S \in H, |S| > 1} \left[ \exp\left(\sum_{S' \subset S, |S'| > 1} (q_{S'} + q_{S' \cup \{j\}})\right) \exp\left(\sum_{\ell \in S \cap T} \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) \prod_{\ell \in S \cap H/T} \frac{p_{\ell}}{1 - p_{\ell}} \right]$$

$$A_2^{[T]}(\mathbf{p}, \mathbf{x}) = 1 + \sum_{\ell \in T} \exp\left(q_{\ell k} + \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) + \sum_{\ell \in H/T} \exp(q_{\ell k}) \frac{p_{\ell}}{1 - p_{\ell}}$$

$$B_2^{[T]}(\mathbf{p}, \mathbf{x}) = \exp(q_{jk}) + \sum_{\ell \in T} \exp\left(q_{jk} + q_{\ell j} + q_{\ell k} + q_{\ell jk} + \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) + \sum_{\ell \in H/T} \exp(q_{jk} + q_{\ell j} + q_{\ell k} + q_{\ell jk}) \frac{p_{\ell}}{1 - p_{\ell}} + \sum_{S \in H, |S| > 1} \left[ \exp\left(\sum_{S' \subset S, |S'| > 1} (q_{S'} + q_{S' \cup \{j\}} + q_{S' \cup \{k\}} + q_{S' \cup \{j,k\}})\right) \cdot \exp\left(\sum_{\ell \in S \cap T} \frac{x_{\ell} - \mu_{\ell}}{\theta_{\ell}}\right) \prod_{\ell \in S \cap H/T} \frac{p_{\ell}}{1 - p_{\ell}} \right] \quad (A.6)$$

# B Proofs of results in Section 3

## B.1 Theorem 3.1

*Proof of Theorem 3.1.* Before proving i)-iv), we consider what happens if we reflect in one dimension. Suppose for now that $f$ has location-interaction parameters $\mathbf{q}^*$ (not necessarily the given $\mathbf{q}$). Choose $j \in \{1, \dots, m\}$. Reflect $x_j$ around $p_j$; replace $(x_j/\theta_j) - q_j^* = (x_j - \mu_j^*)/\theta_j$ by $(2p_j - x_j)/\theta_j - q_j^* = (2p_j - \mu_j^* - x_j)/\theta_j$.

In the formula of Equation 2.14 for $f(\mathbf{x}|\mathbf{q}^*, \theta)$, the numerator can be written as

$$\exp\left(\frac{x_j}{\theta_j} - q_j^* + \sum_{\ell \neq j} \left(\frac{x_\ell}{\theta_\ell} - q_\ell^*\right)\right) \quad (B.1)$$
and the denominator as

$$ \left[ \sum_{S \subset \{1, \dots, m\} / \{j\}} \exp \left( \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} q_{S'}^* \right) + \exp \left( \frac{x_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} (q_{S'}^* + q_{S' \cup \{j\}}^*) \right) \right]^2 . \quad (B.2) $$

Notice that the summation over $\{S' \subset S\}$ includes the case $S' = \emptyset$. After reflecting $x_j$ around $p_j$, (B.1) and (B.2) become

$$ \exp \left( \frac{2p_j - x_j}{\theta_j} - q_j^* + \sum_{\ell \neq j} \left( \frac{x_\ell}{\theta_\ell} - q_\ell^* \right) \right) \quad (B.3) $$

and

$$ \left[ \sum_{S \subset \{1, \dots, m\} / \{j\}} \exp \left( \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} q_{S'}^* \right) + \exp \left( \frac{2p_j - x_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} (q_{S'}^* + q_{S' \cup \{j\}}^*) \right) \right]^2 \quad (B.4) $$

Multiply (B.3) and (B.4) by $\exp(2(x_j - p_j)/\theta_j)$ to obtain

$$ \exp \left( \frac{x_j}{\theta_j} - q_j^* + \sum_{\ell \neq j} \left( \frac{x_\ell}{\theta_\ell} - q_\ell^* \right) \right) = \exp \left( \sum_{\ell=1}^{m} \left( \frac{x_\ell}{\theta_j} - q_\ell^* \right) \right) \quad (B.5) $$

and

$$ \left[ \sum_{S \subset \{1, \dots, m\} / \{j\}} \exp \left( \frac{x_j - p_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} q_{S'}^* \right) + \exp \left( \frac{p_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} (q_{S'}^* + q_{S' \cup \{j\}}^*) \right) \right]^2 \\ = \left[ \sum_{S \subset \{1, \dots, m\} / \{j\}} \exp \left( \frac{p_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} (q_{S'}^* + q_{S' \cup \{j\}}^*) \right) + \exp \left( -\frac{p_j}{\theta_j} + \frac{x_j}{\theta_j} + \sum_{\ell \in S} \frac{x_\ell}{\theta_\ell} - \sum_{S' \subset S} ((q_{S'}^* + q_{S' \cup \{j\}}^*) - q_{S' \cup \{j\}}^*) \right) \right]^2 \quad (B.6) $$

Notice that the last expression in Equation (B.6) is exactly like (B.2), but with

i) $q_{S'}^*$ replaced by $q_{S'}^* + q_{S' \cup \{j\}}^*$
ii) $q_{S'}^* + q_{S' \cup \{j\}}^* \text{ replaced by } q_{S'}^* + q_{S' \cup \{j\}}^* - q_{S' \cup \{j\}}^*$
for $S' \subset S$, $|S'| > 0$, and for $S' = \emptyset$,

iii) $q_{\emptyset}^*$ replaced by $-\frac{p_j}{\theta_j} + q_{\emptyset}^* + q_j^*$

iv) $q_{\emptyset}^* + q_j^*$ replaced by $\frac{p_j}{\theta_j} + q_{\emptyset}^*$

Now define a new set of constants, $\mathbf{q}^{**}$. Define

$$ 
\begin{align*}
q_S^{**} &= q_S^* + q_{S \cup \{j\}}^* \\
q_{S \cup \{j\}}^{**} &= -q_{S \cup \{j\}}^* \quad \text{if } S \subset \{1, \dots, m\} / \{j\} \\
q_j^{**} &= \frac{2p_j}{\theta_j} - q_j^*
\end{align*}
$$

and

$$ q_{\emptyset}^{\dagger} = -\frac{p_j}{\theta_j} + q_{\emptyset}^{*} + q_j^{*} \quad (B.7) $$

$q_{\emptyset}^{\dagger}$ will not be the final choice for the $\emptyset$-element of $\mathbf{q}^{**}$, as we shall see. Then after reflection of $x_j$ around $p_j$, the probability density is

$$ \frac{\exp\left(\sum_{\ell=1}^{m} \left(\frac{x_{\ell}}{\theta_{\ell}} - q_{\ell}^{*}\right)\right)}{\left[\sum_{S \subset \{1, \dots, m\}} \exp\left(-q_{\ell}^{\dagger} + \sum_{\ell \in S} \frac{x_{\ell}}{\theta_{\ell}} - \sum_{S' \subset S, |S'| > 0} q_{S'}^{**}\right)\right]^2} \quad (B.8) $$

What we want is to have

$$ f(\mathbf{x}|\mathbf{q}^{**}, \theta) = \frac{\exp\left(\sum_{\ell=1}^{m} \left(\frac{x_{\ell}}{\theta_{\ell}} - q_{\ell}^{**}\right)\right)}{\left[\sum_{S \subset \{1, \dots, m\}} \exp\left(\sum_{\ell \in S} \frac{x_{\ell}}{\theta_{\ell}} - \sum_{S' \subset S} q_{S'}^{**}\right)\right]^2} \quad (B.9) $$

Comparing (B.8) and (B.9), we see that

$$ \frac{\exp\left(\sum_{\ell=1}^{m} q_{\ell}^{*}\right)}{\exp(2q_{\emptyset}^{\dagger})} = \frac{\exp\left(\sum_{\ell=1}^{m} q_{\ell}^{**}\right)}{\exp(2q_{\emptyset}^{**})} \quad (B.10) $$
By Equations (B.7) and (B.10),

$$
\begin{align*}
q_{\emptyset}^{**} &= q_{\emptyset}^{\dagger} + \frac{1}{2} \sum_{\ell=1}^{m} (q_{\ell}^{**} - q_{\ell}^{*}) \\
&= -\frac{p_j}{\theta_j} + q_{\emptyset}^{*} + q_{j}^{*} + \frac{1}{2} \left( q_{j}^{**} - q_{j}^{*} + \sum_{\ell \neq j} (q_{\ell}^{**} - q_{\ell}^{*}) \right) \\
&= -\frac{p_j}{\theta_j} + q_{\emptyset}^{*} + q_{j}^{*} + \frac{1}{2} \left( 2\frac{p_j}{\theta_j} - q_{j}^{*} - q_{j}^{*} \right) + \frac{1}{2} \sum_{\ell \neq j} ((q_{\ell}^{*} + q_{\ell j}^{*}) - q_{\ell}^{*}) \\
&= q_{\emptyset}^{*} + \frac{1}{2} \sum_{\ell \neq j} q_{\ell j}^{*}.
\end{align*}
$$

So we have established the following

**Claim.** Suppose an MCL has location-interaction parameters **q**\* and variance parameters $\theta$. Then $f(R(\mathbf{x}, \mathbf{p}|\{j\})|\mathbf{q}^*, \theta) = f(\mathbf{x}|\mathbf{q}^{**}, \theta)$, where

a) $q_S^{**} = q_S^* + q_{S \cup \{j\}}^*$ if $S \subset \{1, \dots, m\}$, $|S| > 0$, and $j \notin S$

b) $q_S^{**} = -q_S^*$ if $S \subset \{1, \dots, m\}$, $|S| > 1$, and $j \in S$

c) $q_j^{**} = \frac{2p_j}{\theta_j} - q_j^*$

d) $q_{\emptyset}^{**} = q_{\emptyset}^{*} + \frac{1}{2} \sum_{\ell \neq j} q_{\ell j}^{*}$.

To prove the Theorem, we will use induction. We will show that

Initiation: i)-iv) hold if $|T| = 1$

Induction: if i)-iv) hold for all $T$ such that $|T| < n$, they hold if $|T| = n$, for $2 \le n \le m$.

(A) Initiation: Use the Claim, letting $\mathbf{q}^* = \mathbf{q}$, $T = \{j\}$.

i): If $S \subset \{1, \dots, m\}$ and $|S| > 1$, then either $j \in S$ or $j \notin S$.

If $j \in S$, then by b) of Claim, after reflection, $q_S^{**} = -q_S$. That is, $T/S = \emptyset$, so

$$
q_S^{**} = -1 * q_S = (-1)^1 \sum_{S' \subset \emptyset} q_{S \cup S'} = (-1)^{|T \cap S|} \sum_{S' \subset T/S} q_{S \cap S'}.
$$

If $j \notin S$, then by a) of Claim, after reflection $q_S^{**} = q_S + q_{S \cup \{j\}}$ and $T/S = \{j\}$.
So

$$
q_S^{**} = (-1)^0 (q_{S \cup \emptyset} + q_{S \cup \{j\}}) = (-1)^{|T \cap S|} \sum_{S' \subset \{j\}} q_{S \cup S'}.
$$

ii): We assume $|T| = 1$, so $j \in T$ means $T = \{j\}$, and the class $\{S \subset T/\{j\}, |S| > 0\}$ is empty, and so $\sum_{S \subset T/\{j\}, |S| > 0} q_{S \cup \{j\}} = 0$. Hence by c) of Claim, after reflection

$$
q_j^{**} = \frac{2p_j}{\theta_j} - q_j = \frac{2p_j}{\theta_j} - q_j - \sum_{S \subset T/\{j\}, |S|>0} q_{S \cup \{j\}}.
$$
This initiates induction for ii).

iii): Assume $\ell \notin T$. $T = \{j\}$, so this means $\ell \neq j$. Set $S = \{\ell\}$; by part a) of Claim, the $q$-parameter associated with $\{\ell\}$ after reflection of $x_j$ around $p_j$ is $q_{\ell j}^{**} = q_{\{\ell\}} + q_{\{\ell\}\cup\{j\}}$, or $q_{\ell}^{**} = q_{\ell} + q_{\ell j}$. The class $\{S \subset T: |S| > 0\}$ contains one set, $\{j\}$. So $q_{\ell j} = \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}}$. Hence $q_{\ell}^{**} = q_{\ell} + \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}}$.

iv): $T = \{j\}$, so $\ell \in \{1, \dots, m\}/T$ if and only if $\ell \neq j$. The class $\{S \subset T: |S| > 0\}$ contains one set, $\{j\}$. The class $\{S \subset T: |S| > 2\}$ is empty. So

$$ 
\begin{aligned}
& \sum_{\ell \in \{1, \dots, m\}/T} \left( \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}} \right) + \sum_{S \subset T, |S| > 2} (2 - |S|)q_S \\
&= \sum_{\ell \neq j} (q_{\{j\} \cup \{\ell\}}) + 0 = \sum_{\ell \neq j} q_{\ell j}. \qquad (B.11)
\end{aligned}
$$

By part d) of Claim and (B.11), after reflection of $x_j$ around $p_j$, the $q$-constant associated with $\emptyset$ is

$$ 
\begin{aligned}
q_{\emptyset}^{**} &= q_{\emptyset} + \frac{1}{2} \sum_{\ell \neq j} q_{\ell j} \\
&= q_{\emptyset} + \frac{1}{2} \left( \sum_{\ell \in \{1, \dots, m\}/T} \left( \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}} \right) + \sum_{S \subset T, |S| > 2} (2 - |S|)q_S \right).
\end{aligned}
$$

This initiates induction for iv).

(B) Induction. We assume i)-iv) hold if $|T| < n$. Now suppose we are reflecting $x_j$ around $p_j$ for $j \in T$, and $|T| = n$. Set $T' = T/\{j\}$. $|T'| = n - 1$, so i)-iv) hold for reflection of $x\ell$ around $p_\ell$ for $\ell \in T'$.

i): Choose $S \subset \{1, \dots, m\}$, $|S| > 1$. By the induction assumption for $T'$, the $q$-parameter associated with $S$ is currently

$$ q_S^* = (-1)^{|S \cap T'|} \sum_{S' \subset T' / S} q_{S \cup S'}. \qquad (B.12) $$

Either $j \notin S$ or $j \in S$.

**Case 1.** $j \notin S$. Reflect $x_j$ around $p_j$. By a) of the Claim, the $q$-parameter associated with $S$ is now

$$ q_S^{**} = q_S^* q_{S \cup \{j\}}^*. $$

By the induction assumption on $T'$,

$$ q_{S \cup \{j\}}^* = (-1)^{|(S \cup \{j\}) \cap T'|} \sum_{S' \subset T' / (S \cup \{j\})} q_{(S \cup \{j\}) \cup S'}. \qquad (B.13) $$

Since $j \notin T$ and $j \notin S$, $(S \cup \{j\}) \cap T' = S \cap T' = S \cap T$. Also $T' / (S \cup \{j\}) = T' / S = T / (S \cup \{j\})$. The class $\{S' \subset T' / S\}$ is exactly the class $S' \subset T / S: j \notin S'$ and is in 1-1 correspondence with the class $\{S' \subset T / S: j \in S\}$. So

$$ \sum_{S' \subset T' / S} q_{S \cup S'} = \sum_{S' \subset T / S: j \notin S'} q_{S \cup S'} $$
and

$$ \sum_{S' \subset T' / (S \cup \{j\})} q_{(S \cup \{j\}) \cup S'} = \sum_{S' \subset T' / S} q_{S \cup (\{j\} \cup S')} = \sum_{S' \subset T / S : j \in S} q_{S \cup S'}. $$

Hence by Equations (B.12) and (B.13), the q-parameter associated with S after reflecting $x_j$ around $p_j$ is

$$ \begin{aligned} q_S^{**} &= (-1)^{|S \cap T'|} \sum_{S' \subset T' / S} q_{S \cup S'} + (-1)^{|(S \cup \{j\}) \cap T'|} \sum_{S' \subset T' / (S \cup \{j\})} q_{(S \cup \{j\}) \cup S'} \\ &= (-1)^{|S \cap T'|} \left( \sum_{S' \subset T / S : j \notin S} q_{S \cup S'} + \sum_{S' \subset T / S : j \in S} q_{S \cup S'} \right) \\ &= (-1)^{|S \cap T'|} \sum_{S' \subset T / S} q_{S \cup S'}. \end{aligned} $$

**Case 2.** $j \in S$. Then $T'/S = T/S$. Also $S \cap T = S \cap (T' \cup \{j\}) = (S \cap T') \cup \{j\}$, and $|S \cap T| = |S \cap T'| + 1$. Now reflect $x_j$ around $p_j$. Since $j \in S$, part b) of the Claim applies; the q-parameter associated with S is now $q_S^{**} = -q_S^*$. By this and Equation (B.12),

$$ \begin{aligned} q_S^{**} &= -(-1)^{|S \cap T'|} \sum_{S' \subset T' / S} q_{S \cup S'} \\ &= (-1)^{|S \cap T'|} \sum_{S' \subset T / S} q_{S \cup S'}. \end{aligned} $$

So whether $j \in S$ or $j \notin S$, we have shown that after reflecting $x_j$ around $p_j$,

$$ q_S^{**} = (-1)^{|S \cap T|} \sum_{S' \subset T / S} q_{S \cup S'}. $$

This establishes the induction for i).

ii): $j \in T$. By the induction assumption for $T'$, after reflecting $x_\ell$ around $p_\ell$ for $\ell \in T'$, the q-parameter associated with $\{j\}$ is

$$ q_j^* = q_j + \sum_{S \subset T', |S| > 0} q_{S \cup \{j\}}. \qquad (B.14) $$

Now reflect $x_j$ around $p_j$. By part c) of the Claim, the q-parameter associated with $\{j\}$ is now

$$ q_j^{**} = \frac{2p_j}{\theta_j} - q_j^*. \qquad (B.15) $$

By Equations (B.14) and (B.15),

$$ q_j^{**} = \frac{2p_j}{\theta_j} - q_j^* - \sum_{S \subset T', |S| > 0} q_{S \cup \{j\}} $$
which is the statement of ii) for $T$. [Remember that $T' = T/\{j\}$.]

iii): Choose $\ell \notin T$. By the induction assumption, iii) holds for $T'$. After reflecting $x_\ell$ around $p_\ell$ for $\ell \in T'$, the $q$-parameter associated with $\{\ell\}$ is

$$q_{\ell}^{*} = q_{\ell} + \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}}. \qquad (B.16)$$

Now reflect $x_j$ around $p_j$. $j \neq \ell$ and $|\{\ell\}| = 1 > 0$, so part a) of the Claim applies. The $q$-parameter for $\{\ell\}$ is now

$$q_{\ell}^{**} = q_{\ell}^{*} + q_{\{\ell\} \cup \{j\}}^{*} = q_{\ell}^{*} + q_{\ell j}^{*}. \qquad (B.17)$$

i) holds for $T'$; the $q$-parameter for $\{\ell, j\}$ before reflection of $x_j$ around $p_j$ is

$$\begin{aligned} q_{\ell j}^{*} &= (-1)^{|\{\ell, j\} \cap T'|} \sum_{S' \subset T' / \{\ell, j\}} q_{\{\ell, j\} \cup S'} \\ &= (-1)^0 \sum_{S' \subset T'} q_{S' \cup \{\ell\} \cup \{j\}} \\ &= \sum_{S \subset T, j \in S} q_{S \cup \{\ell\}}. \end{aligned} \qquad (B.18)$$

By Equations (B.16) and (B.18),

$$\begin{aligned} q_{\ell}^{**} &= q_{\ell} + \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} + \sum_{S \subset T, j \in S} q_{S \cup \{\ell\}} \\ &= q_{\ell} + \sum_{S \subset T, |S| > 0, j \notin S} q_{S \cup \{\ell\}} + \sum_{S \subset T, j \in S} q_{S \cup \{\ell\}} \\ &= q_{\ell} + \sum_{S \subset T, |S| > 0} q_{S \cup \{\ell\}}. \end{aligned}$$

So we have shown that iii) holds for $T$.

iv): After reflection $x_\ell$ around $p_\ell$ for $\ell \in T'$, by the induction assumption,

iv) holds for $T'$. The $q$-constant associated with $\emptyset$ is

$$q_{\emptyset}^{*} = q_{\emptyset} + \frac{1}{2} \sum_{\ell \notin T'} \left( \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} \right) + \frac{1}{2} \sum_{S \subset T', |S| > 2} (2 - |S|)q_S \quad (B.19)$$

and i) holds; so for any $\ell \in \{1, \dots, m\} / \{j\}$,

$$q_{\ell j}^{*} = (-1)^{|\{\ell, j\} \cap T'|} \sum_{S' \subset T' / \{\ell, j\}} q_{\{\ell, j\} \cup S'}. \qquad (B.20)$$

Now reflect $x_j$ around $p_j$. By part d) of the Claim, the $q$-constant associated with $\emptyset$ is now

$$q_{\emptyset}^{**} = q_{\emptyset}^{*} + \frac{1}{2} \sum_{\ell \neq j} q_{\ell j}^{*}. \qquad (B.21)$$
The set {$\ell: \ell \neq j$} is a disjoint union of $T'$ and {$\ell: \ell \notin T$}. If $\ell \in T'$, the $|\{\ell, j\} \cap T'| = |\{\ell\}| = 1$ and $T'/\{\ell, j\} = T/\{\ell\}$. Also, $S' \subset T/\{\ell\}$ if and only if $S' \cup \{\ell\}$ is a subset of $T'$ that contains $\ell$. So if $\ell \in T$, (B.20) becomes

$$q_{\ell j}^* = - \sum_{S' \subset T', \ell \in S'} q_{S' \cup \{\ell\}}. \qquad (B.22)$$

If $\ell \notin T'$, then $\{\ell, j\} \cap T' = \emptyset$ and $t'/\{\ell, j\} = T'$. So (B.20) becomes

$$q_{\ell j}^* = (-1)^0 \sum_{S' \subset T'} q_{S' \cup \{\ell, j\}}. \qquad (B.23)$$

Put Equations (B.19)-(B.23) together, to express the new q-constant associated with $\emptyset$ as the sum of several components:

$$\begin{aligned} q_{\emptyset}^{**} &= q_{\emptyset}^* + \frac{1}{2} \left( \sum_{\ell \in T'} q_{\ell j}^* + \sum_{\ell \notin T} q_{\ell j}^* \right) \\ &= q_{\emptyset} + \frac{1}{2} \left( \sum_{\ell \notin T'} \left( \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} \right) (C1) + \sum_{S \subset T', |S| > 2} (2 - |S|) q_S \right) (C2) \\ &\quad - \sum_{\ell \in T'} \left( \sum_{S' \subset T', \ell \in S'} q_{S' \cup \{j\}} \right) (C3) + \sum_{\ell \notin T} \left( \sum_{S' \subset T'} q_{S' \cup \{\ell, j\}} \right) (C4). \end{aligned} \qquad (B.24)$$

W will now consider components C1-C4, break them up, and recombine. First, $C4$. $S \subset T'$ if and only if $S \cup \{j\}$ is a subset of $T$ that contains $j$. So for $\ell \notin T$,

$$\sum_{S \subset T'} q_{S \cup \{\ell, j\}} = \sum_{S \subset T, j \in S} q_{S \cup \{\ell\}} \quad (C4A) \qquad (B.25)$$

Also, {$\ell: \ell \notin T'$} is the disjoint union of {$\ell: \ell \notin T$} and {$j$}. Apply this to component C1:

$$\begin{aligned} \sum_{|ell \notin T'|} \left( \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} \right) &= \sum_{|ell \notin T|} \left( \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} \right) + \sum_{S \subset T', |S| > 0} q_{S \cup \{j\}} \\ &= \sum_{|ell \notin T|} \left( \sum_{S \subset T', |S| > 0} q_{S \cup \{\ell\}} \right) (C11) + \sum_{S \subset T, j \in S, |S| > 1} q_S (C12) \end{aligned} \qquad (B.26)$$

Also, in the double sum for component C3, any $S' \subset T'$ appears several times - one for each $\ell \in S'$ - that is, $|S'|$ times. This is zero unless $|S'| > 0$. So rewrite
C3:

$$
\begin{align}
\sum_{\ell \notin T'} \left( \sum_{S' \subset T', \ell \in S'} q_{S \cup \{j\}} \right) &= \sum_{S' \subset T', |S'| > 0} |S'| * q_{S' \cup \{j\}} \nonumber \\
&= \sum_{S \subset T, j \in S, |S'| > 1} (|S| - 1) q_S \tag{B.27}
\end{align}
$$

Now combine C12 and C3A:

$$
\begin{align}
(C12) - (C3A) &= \left( \sum_{S \subset T, j \in S, |S|>1} q_S \right) - \left( \sum_{S \subset T, j \in S, |S|>1} (|S|-1)q_S \right) \nonumber \\
&= \sum_{S \subset T, j \in S, |S|>1} (2-|S|)q_S = \sum_{S \subset T, j \in S, |S|>2} (2-|S|)q_S. \tag{B.28}
\end{align}
$$

$S \subset T$ means that either $j \in S$ or $S \subset T'$. By this and (B.28),

$$
(C2)+(C12)-(C3A) = \sum_{S \subset T', |S|>2} (2-|S|)q_S + \sum_{S \subset T, j \in S, |S|>2} q_S = \sum_{S \subset T, |S|>2} (2-|S|)q_S. \tag{B.29}
$$

Similarly, {$S \subset T : |S| > 0$} is a disjoint union of {$S \subset T' : |S| > 0$} and
{$S \subset T : j \in S$}. So $\sum_{S \subset T' : |S| > 0} + \sum_{S \subset T : j \in S} = \sum_{S \subset T : |S| > 0}$. Apply this to
C11 and C4A:

$$
\sum_{\ell \notin T} \left( \sum_{S \subset T: j \in S} q_{S \cup \{\ell\}} \right) + \sum_{\ell \notin T} \left( \sum_{S' \subset T': |S'|>0} q_{S \cup \{\ell\}} \right) = \sum_{\ell \notin T} \left( \sum_{S \subset T: |S'|>0} q_{S \cup \{\ell\}} \right). \tag{B.30}
$$

By Equations (B.28), (B.29), and (B.30), The *q*-constant associated with $\emptyset$ after
reflection $x_j$ around $p_j$ is

$$
\begin{align*}
q_{\emptyset}^{**} &= q_{\emptyset} + \frac{1}{2}(C1 + C2 - C3 + C4) \\
&= q_{\emptyset} + \frac{1}{2}(C11 + C12 + C2 - C3A + C4A) \\
&= q_{\emptyset} + \frac{1}{2}(C11 + C4A) + \frac{1}{2}(C12 + C2 - C3A) \\
&= q_{\emptyset} + \frac{1}{2}\sum_{\ell \notin T} \left( \sum_{S \subset T: |S|>0} q_{S \cup \{\ell\}} \right) + \frac{1}{2}\sum_{S \subset T: |S|>2} (2-|S|)q_S.
\end{align*}
$$

So iv) holds for $T$, and $T \subset \{1, \dots, m\}$ such that $|T| = n$. This establishes the
induction for iv) and concludes the Proof. $\square$
## B.2 Proof of Theorem 3.2

We first define some notation that will make the proofs presented easier to read.

**Definition B.1.** Let $m$ be a positive integer. Let $\{q_S : S \subset \{1, \dots, m\}\}$ be a set of constants, one for each subset of $\{1, \dots, m\}$. Let $S$ be a subset of $\{1, \dots, m\}$. Then

$$H_T(S) := \sum \{q_{S \cup S'} : S' \subset T/S, |S'| > 0\}$$

$$H_T^+(S) := \sum \{q_{S \cup S'} : S' \subset T/S\}$$

$$\text{So } H_T^+(S) = H_T(S) + q_S.$$

For the proof of this theorem, we will need the following technical lemma.

**Lemma B.1.** Let $m$ be a positive integer, and $q$ a set of constants assigned to subsets of $\{1, \dots, m\}$. Let $T$ be a nonempty subset of $\{1, \dots, m\}$.

Suppose for some integer $k$, $0 \le k \le |T|$, we have that

$$q_S = -\frac{1}{2} H_T(S) \text{ for all } S \subset \{1, \dots, m\} \text{ such that } |S \cap T| > k \text{ and } |S \cap T| \text{ is odd} \tag{B.31}$$

Then

$$H_T(S) = 0 \text{ for all } S \subset \{1, \dots, m\} \text{ such that } |S \cap T| > k \text{ and } |S \cap T| \text{ is even} \tag{B.32}$$

The proof is given after the proof of the Theorem.

*Proof of Theorem 3.2.* Necessity (only if): We assume that $f(\cdot|\mathbf{q}, \bar{\theta})$ is symmetric with respect to $R(\cdot, \mathbf{p}|T)$. So the Corollary to Theorem 3.1 holds. Let $S \subset \{1, \dots, m\}$ such that $|S| > 1$ and $|S \cap T|$ is odd. By part 2) of the Corollary,

$$q_S = -\frac{1}{2} \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'} = -\frac{1}{2} H_T(S).$$

But this is condition I) of Theorem 3.2. If in addition $\mathbf{p} = \{p_\ell : \ell \in T\}$ is specified, then part 3) of the Corollary holds: for $j \in T$, $q_j = -p_j/\theta_j - (1/2) \sum_{S \subset T/\{j\}} |S| > 0$ $q_{S \cup \{j\}}$. But this is condition 2) of Theorem 3.2.

Sufficiency (if): Condition I) is the condition of the Lemma, for $k=0$. So the Lemma holds: If $S \subset \{1, \dots, m\}$ and $|S \cap T|$ is even,

$$H_T(S) = \sum_{S' \subset T/S : |S'|>0} q_{S \cup S'} = 0.$$

This is part 1) of the Corollary. Condition I) itself is part 2) of the Corollary.

Suppose the center $\mathbf{p}$ of the hyperplane of reflection is not specified. This is the case for part A) of Theorem 3.2. Then part 3) of the Corollary is satisfied
provided that we set $p_j = -\theta_j(q_j + (1/2)H_T(\{j\}))$ for all $j \in T$. Now suppose that $\mathbf{p}$ is specified. This is part B) of Theorem 3.2. Then Condition II) is assumed. But Condition II) is equivalent to part 3) of the Corollary.

We could try to prove part 4) of the Corollary by direct calculation, as was done for part iv) of Theorem 3.1. However, a long proof is not necessary. Notice that part 4) of the Corollary is equivalent to the condition that $q_\emptyset^* = q_\emptyset$, where $q_\emptyset^*$ is the $q$-constant associated with $\emptyset$ after reflection of $x_j$ around $p_j$, all $j \in T$. But the constant of normalization - $q_\emptyset$ - is determined by the higher order interactions $\{q_S : S \subset \{1, \dots, m\}, |S| > 1\}$. Parts 1) and 2) of the Corollary have already been shown; they mean that $q_S^* = q_S$, all $S \subset \{1, \dots, m\}$ such that $|S| > 1$. All $q$-parameters for higher order interactions are unchanged by any reflection defined by $T$ and some $\mathbf{p}$, so $q_\emptyset$ is also unchanged. So part 4) of the Corollary holds, and all parts of the Corollary of Theorem 3.1 are true: $f(\cdot|\mathbf{q}, \bar{\theta})$ is symmetric with respect to $R(\cdot, \mathbf{p}|T)$. $\square$

*Proof of Lemma B.1.* Use induction. First, prove that Condition (B.32) holds if $|S \cap T|$ is odd and equal to $|T|$ or $|T| - 1$. Then we will show: If Condition (B.32) holds for all $S$ such that $|S \cap T| \ge 2s + 2$, then Condition (B.32) also holds if $|S \cap T| = 2s$, so long as $2s \ge k$.

**Initiation.** There are two cases: $|T|$ is odd or $|T|$ is even.

*Case 1.* $|T|$ is odd. Consider any set $S^* \subset \{1, \dots, m\}$ such that $S^* = T \cup S'$, for some $S' \subset \{1, \dots, m\}/T$. Then $T \subset S^*$, so $|T \cap S^*| = |T|$ and is odd. Also $|T \cap S^*| \ge k$, so Condition (B.32) holds: $q_{S^*} = -\frac{1}{2}H_T(S^*)$. But $H_T(S^*)$ is the sum of $q_{S'}$ for all $S'$ such that $S^* \subset S' \subset S^* \cup T$ and $S' \neq S^*$. But $S^* \cup T = S^*$, so there are no such $S'$. $H_T(S^*)$ is then a sum with zero summands and is zero. So $H_T(S^*) = 0$ for all $S^*$ such that $T \subset S^* \subset \{1, \dots, m\}$, and $q_{S^*} = 0$ for all such $S^*$.

Now let $S \subset \{1, \dots, m\}$ be such that $|S \cap T| = |T| - 1$, but $|S \cap T| \ge k$. This means that $T/S = \{j\}$, some $j \in \{1, \dots, m\}$, and $T \subset S \cup \{j\}$. $H_T(S)$ is the sum of $q_{S'}$ for all $S'$ such that $S \subset S' \subset S \cup T$ and $S \neq S'$. But $S \cup T = S \cup \{j\}$. So the conditions on $S'$ reduce to $S \subset S' \subset S \cup \{j\}$, $S \neq S'$. The only $S'$ that satisfies these conditions is $S \cup \{j\}$ itself, so $H_T(S) = q_{S \cup \{j\}}$. $T \subset S \cup \{j\}$, so by the preceding paragraph, $q_{S \cup \{j\}} = 0$. So if $|T|$ is odd and $|S \cap T| = |T| - 1$, $H_T(S) = 0$.

*Case 2.* $|T|$ is even. The largest subset of $T$ of even size is $T$ itself. Let $S \subset \{1, \dots, m\}$ be such that $|S \cap T| = |T|$. Then $T \subset S$, $S \cup T = S$. $H_T(S)$ is the sum of $q_{S'}$ for all $S'$ such that $S \subset S' \subset S \cup T$ and $S \neq S'$. For $S$ as chosen, the conditions reduce to $S \subset S' \subset S$, $S \neq S'$. No such $S'$ exist, so $H_T(S)$ has zero summands and is zero.

We have shown that $H_T(S) = 0$ if $|S \cap T|$ is even and $|T| - |S \cap T| \le 1$, whether $|T|$ is odd or even.

**Induction.** Now suppose that Condition (B.32) holds for any $S \subset \{1, \dots, m\}$ such that $|S \cap T|$ is even and $|S \cap T| \ge 2s + 2$. Choose $S \subset \{1, \dots, m\}$ so that $|S \cap T| = 2s$. If $q_{S'}$ is a summand of $H_T(S)$, then $S \subset S' \subset S \cup T$ and $S' \neq S$; so $S'$ contains at least one element $j \in T/S$, and $S \cup \{j\} \subset S' \subset S \cup T$, and $q_{S'}$
is in the sum $H_T^+(S \cup \{j\})$. So all terms in $H_T(S)$ are in the sum of sums

$$ \sum \{H_T^+(S \cup \{j\}) : j \in T/S\}. \qquad (B.33) $$

However, if $S' \subset S \cup T$ and contains two distinct elements $j_1, j_2$ of $T/S$, the $q_{S'}$ is a summand of both $H_T^+(S \cup \{j_1\})$ and $H_T^+(S \cup \{j_2\})$. So $q_{S'}$ would appear twice in the sum (B.33). To correct this, subtract the intersection of $H_T^+(S \cup \{j_1\})$ and $H_T^+(S \cup \{j_2\})$. It is not hard to see that

$$ H_T^+(S \cup \{j_1\}) \cap H_T^+(S \cup \{j_2\}) = H_T^+(S \cup \{j_1, j_2\}) \qquad (B.34) $$

So with this correction, (B.33) becomes

$$ \sum \{H_T^+(S \cup \{j\}) : j \in T/S\} - \sum \{H_T^+(S \cup \{j_1, j_2\}) : j_1, j_2 \in T/S \text{ and } j_1 \neq j_2\} \qquad (B.35) $$

This can also be expressed

$$ \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + 1\} - \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + 2\} \qquad (B.36) $$

However, suppose that $S' \subset S \cup T$ is such that $j_\ell \in S'$ for three distinct $j_\ell \in T/S$. Then $q_{S'}$ appears in $H_T^+(S \cup \{j_\ell\})$ for $\ell \in \{1, 2, 3\}$, but also in $H_T^+(S \cup \{j_1, j_2, j_3\})$ for $(\ell_1, \ell_2) \subset \{1, 2, 3\}$, $\ell_1 \neq \ell_2$. So $q_{S'}$ appears in expression (B.35) with coefficient zero. To correct this, add intersection of $\{H_T^+(S \cup \{j_{\ell_1}, j_{\ell_2}\}) : \ell_1 \neq \ell_2, \ell_1, \ell_2 \in T/S\}$, for all distinct triplets $\{j_1, j_2, j_3\} \subset T/S$. But (for a given $\{j_1, j_2, j_3\}$), this intersection is just $H_T^+(S \cup \{j_1, j_2, j_3\})$. So (B.36) with the correction becomes

$$ \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + 1\} - \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + 2\} + \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + 3\} \qquad (B.37) $$

But if $S \subset S' \subset S \cup T$ and $S'$ contains exactly four elements of $T/S$, $q_{S'}$ will not appear in (B.37) with coefficient one. A simple calculation will show that the coefficient of $q_{S'}$ in (B.37) will be two. To correct, subtract $H_T^+(S \cup \{j_1, j_2, j_3, j_4\})$. Do this for all foursomes $\{j_1, j_2, j_3, j_4\} \subset T/S$. So we subtract $\sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S'| = 2s + 4\}$. Continuing in this way, we arrive at the identity

$$ H_T(S) = \sum_{\ell=1}^{|T|-|T \cap S|} (-1)^{\ell-1} \left( \sum \{H_T^+(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + \ell \} \right). \qquad (B.38) $$
For any $S \subset \{1, \dots, m\}$, $H_T^+(S) = q_S + H_T(S)$. So Equation (B.38) can be rewritten

$$H_T(S) = \sum_{\ell=1}^{|T|-|S \cap T|} (-1)^{\ell-1} \left( \sum_{S'} \{q_{S'} + H_T(S') : S \subset S' \subset S \cup T, |S' \cap T| = 2s + \ell\} \right). \qquad (B.39)$$

Now consider $H_T(S')$ for $S \subset S' \subset S \cup T$. $|S \cap T| = 2s + \ell$ for some $\ell > 0$. If $\ell$ is odd, then $|S' \cap T|$ is odd; by Condition (B.32) of the Lemma, $q_{S'} = -(1/2)H_T(S')$, so

$$q_{S'} + H_T(S') = q_{S'} - 2q_{S'} = -q_{S'} = (-1)^{\ell} q_{S'}. \qquad (B.40)$$

If $\ell$ is even, then $|S' \cap T|$ is even, and $|S' \cap T| > 2s$. By the assumption for induction, $H_T(S') = 0$. So

$$q_{S'} + H_T(S') = q_{S'} = (-1)^{\ell} q_{S'}. \qquad (B.41)$$

By Equations (B.39), (B.40), and (B.41),

$$\begin{aligned} H_T(S) &= \sum_{\ell=1}^{|T|-|S \cap T|} (-1)^{\ell-1} \left( \sum_{S'} \{(-1)^{\ell} q_{S'} : S \subset S' \subset S \cup T, |S' \cap T| = 2s + \ell\} \right) \\ &= - \sum_{\ell=1}^{|T|-|S \cap T|} \left( \sum_{S'} \{q_{S'} : S \subset S' \subset S \cup T, |S' \cap T| = 2s + \ell\} \right) \\ &= - \sum_{S'} \{q_{S'} : S \subset S' \subset S \cap T, S \neq S'\} \\ &= -H_T(S). \end{aligned}$$

So $H_T(S) = 0$. This is true for any subset $S$ of $\{1, \dots, m\}$ such that $|S \cap T| = 2s$. This establishes the induction and proves the Lemma. $\square$

## B.3 Proof of Theorem 3.3

For the main proof, we will use a simple identity several times. That identity is proved here.

**Lemma B.2.** If $S_1$ and $S_2$ are two finite sets and $S_1 \subset S_2$, $S_1 \neq S_2$, then

$$\sum \{(-1)^{|S/S_1|} : S_1 \subset S \subset S_2\} = 0.$$

*Proof of Lemma B.2.* $S_1 \subset S \subset S_2$ if and only if $S = S_1 \cup S'$, where $S' \subset S_2/S_1$, and $|S| = |S_1| + |S'|$. So

$$\begin{aligned} & \sum \{(-1)^{|S/S_1|} : S_1 \subset S \subset S_2\} \\ &= \sum \{(-1)^{|S'|} : S' \subset S_2/S_1\} \\ &= (1 + (-1))^{|S_2/S_1|} = 0. \end{aligned}$$

$\square$
*Proof of Theorem 3.3.* The probability density function at $\mathbf{x} \in \mathbb{R}^m$ is

$$f(\mathbf{x}|\mathbf{q}, \bar{\theta}) = \frac{\exp\left(\sum_{j=1}^{m} \left(\frac{x_j}{\theta_j} - q_j\right)\right)}{\left[\sum_{S \subset \{1, \dots, m\}} \left(\exp\left(\sum_{j \in S} \frac{x_j}{\theta_j} - \sum_{S' \subset S} q_{S'}\right)\right)\right]^2}$$

For $T \subset \{1, \dots, m\}$, the partial reflection is

$$R(\mathbf{x}, \mathbf{p}|T)_s = \begin{cases} x_j & \text{if } j \notin T \\ 2p_j - x_j & \text{if } j \in T \end{cases}$$

so

$$f(R(\mathbf{x}, \mathbf{p}|T)|\mathbf{q}, \bar{\theta}) = \frac{\exp\left(\sum_{j \in T} \frac{2p_j - x_j}{\theta_j} + \sum_{j \notin T} \frac{x_j}{\theta_j} - \sum_{j=1}^{m} q_j\right)}{\left[\sum_{S \subset \{1, \dots, m\}} \exp\left(\sum_{j \in S \cap T} \frac{2p_j - x_j}{\theta_j}\right) + \sum_{j \in S \cap \{1, \dots, m\} \setminus T} \frac{x_j}{\theta_j} - \sum_{S' \subset S} q_{S'}\right]^2} \quad (B.42)$$

Multiply the numerator and denominator by $\exp(2 \sum_{j \in T} (x_j - p_j)/\theta_j)$. The numerator becomes $\exp(\sum_{j=1}^m ((x_j/\theta_j) - q_j))$. For a given $S \subset \{1, \dots, m\}$, the argument of the exponential in the denominator of Equation (B.42) becomes

$$\sum_{j \in S \cap T} \left( \frac{2p_j - x_j}{\theta_j} \right) + \sum_{j \in S \cap \{1, \dots, m\} \setminus T} \left( \frac{x_j}{\theta_j} \right) + \sum_{j \in S} \left( \frac{x_j - p_j}{\theta_j} \right) - \sum_{S' \subset S} q_{S'} \\ = \sum \left\{ \frac{x_j}{\theta_j} : j \in (S \cap \{1, \dots, m\} \setminus T) \cup (\{1, \dots, m\} \cap T) \right\} \\ + \sum_{j \in S \cap T} \frac{p_j}{\theta_j} - \sum_{j \in \{1, \dots, m\} \setminus T} \frac{p_j}{\theta_j} - \sum_{S' \subset S} q_{S'} \quad (B.43)$$

If $x_j$ and $x_j - p_j$ are positive and large for all $j$, and also large compared to all location parameters $\{q_j\}_{j=1}^m$, then the expression in Equation (B.43) is largest when it includes all $x_j$'s. Now,

$$(\{1, \dots, m\} \setminus T) \cup (S \cap \{1, \dots, m\} \setminus T) = (S \cup T) / (S \cap T).$$

and this is largest when $(S \cup T)/(S \cap T) = \{1, \dots, m\}$. This is true if and only if $S \cup T = \{1, \dots, m\}$ and $S \cap T = \emptyset$, that is, if $S = \{1, \dots, m\}/T$. If $S = \{1, \dots, m\}/T$, the expression in Equation (B.43) becomes

$$\sum_{j=1}^{m} \frac{x_j}{\theta_j} - \sum_{j \in T} \frac{p_j}{\theta_j} - \sum_{S' \subset \{1, \dots, m\} \setminus T} q_{S'}. \quad (B.44)$$
If $S \neq \{1, \dots, m\}/T$, the expression in Equation (B.43) is less than that in Equation (B.44) by $O\left(-\min_j \frac{x_j - p_j}{\theta_j}\right)$. (Remember that $x_j - p_j$ is assumed to dominate the constants $\{q_S\}$). So if $S \neq \{1, \dots, m\}/T$,

$$ \sum_{j \in S \cap T} \left( \frac{2p_j - x_j}{\theta_j} \right) + \sum_{j \in S \cap \{1, \dots, m\}/T} \left( \frac{x_j}{\theta_j} \right) + \sum_{j \in S} \left( \frac{x_j - p_j}{\theta_j} \right) - \sum_{S' \subset S} q_{S'} \\ = O\left(-\min_j \frac{x_j - p_j}{\theta_j}\right) \cdot \left( \sum_{j=1}^{m} \frac{x_j}{\theta_j} - \sum_{j \in T} \frac{p_j}{\theta_j} - \sum_{S' \subset \{1, \dots, m\}/T} q_{S'} \right). $$

So the denominator of Equation (B.42) becomes

$$ \left[ \sum_{S \subset \{1, \dots, m\}} \exp\left(\sum \left\{ \frac{x_j}{\theta_j} : j \in \{1, \dots, m\} / S \cap T \right\} \cup \left(S \cap \{1, \dots, m\} / T\right)\right) + \sum_{j \in S \cap T} \frac{p_j}{\theta_j} \right. \\ \left. - \sum_{j \in S \cap \{1, \dots, m\} / T} \frac{p_j}{\theta_j} - \sum_{S' \subset S} q_{S'} \right]^2 \\ = \left[ \exp\left(\sum_{j=1}^{m} \frac{x_j}{\theta_j} - \sum_{j \in T} \frac{p_j}{\theta_j} - \sum_{S' \subset \{1, \dots, m\} / T} q_{S'} \right) \cdot \left(1 + O\left(\exp\left(-\min_j \frac{x_j - p_j}{\theta_j}\right)\right)\right) \right]^2 $$

So

$$ \log f(R(\mathbf{x}, \mathbf{p}|T)|\mathbf{q}, \bar{\theta}) = \log(\text{numerator}) - \log(\text{denominator}) \\ = \sum_{j=1}^{m} \left(\frac{x_j}{\theta_j} - q_j\right) - 2 \left[ \sum_{j=1}^{m} \frac{x_j}{\theta_j} - \sum_{j \in T} \frac{p_j}{\theta_j} - \sum_{S' \subset \{1, \dots, m\}/T} q_{S'} + O\left(-\min_j \frac{x_j - p_j}{\theta_j}\right) \right]. \quad (B.45) $$

Now consider

$$ \sum \{(-1)^{|T|} \log f(R(\mathbf{x}, \mathbf{p}|T \cup \{1, \dots, m\}/U)|\mathbf{q}, \bar{\theta}) : T \subset U\}. $$

Apply $\sum \{(-1)^{|T|} : T \subset U\}$ to components of (B.45), replacing $T$ by $T \cup \{1, \dots, m\}/U$. First,

$$ \begin{align} & \sum \{(-1)^{|T|} \left( \sum_{j=1}^{m} \left( \frac{x_j}{\theta_j} - q_j \right) \right) : T \subset U \} \nonumber \\ &= \left( \sum_{j=1}^{m} \left( \frac{x_j}{\theta_j} - q_j \right) \right) \sum \{(-1)^{|T|} : T \subset U \} \nonumber \\ &= \left( \sum_{j=1}^{m} \left( \frac{x_j}{\theta_j} - q_j \right) \right) \cdot 0 = 0 \tag{B.46} \end{align} $$
Using Lemma B.2 with $S_1 = \{1, \dots, m\}/U$, $S_2 = \{1, \dots, m\}$. Similarly

$$ \sum \{(-1)^{|T|} \left( \sum_{j=1}^{m} \left( \frac{x_j}{\theta_j} \right) \right) : T \subset U \} = 0. $$

Now consider

$$ \sum \{(-1)^{|T|} \left( \sum_{j \in T \cup \{1, \dots, m\} / U} \frac{p_j}{\theta_j} \right) : T \subset U \}. $$

There is a subset $T \subset U$ and element $j \in T \cup \{1, \dots, m\}/U$ if and only if: There exists $j \in \{1, \dots, m\}$ and subset $T \subset U$ such that $j \in T$ or $j \in \{1, \dots, m\}/U$. That is, either $j \in T$ or $j \notin U$.

For any $j \in \{1, \dots, m\}$, either $j \in U$ or $j \notin U$. If $j \notin U$, then it is true that $j \in T \cup \{1, \dots, m\}/U$ for any $T \subset U$. But if $j \in U$, then $j \in T \cup \{1, \dots, m\}/U$ only if $j \in T$; so $T = T' \cup \{j\}$, where $T' \subset U/\{j\}$, and $|T| = |T' \cup \{j\}| = |T'| + 1$. So we can reverse the order of summation:

$$ \begin{aligned} & \sum \{(-1)^{|T|} \left( \sum_{j \in T \cup \{1, \dots, m\} / U} \frac{p_j}{\theta_j} \right) : T \subset U \} \\ &= \sum_{j=1}^{m} \left( \sum \{(-1)^{|T|} : T \subset U \text{ and } j \in T \cup \{1, \dots, m\} / U \} \right) \frac{p_j}{\theta_j} \\ &= \sum_{j \notin U} \left( \sum \{(-1)^{|T|} : T \subset U \} \right) \frac{p_j}{\theta_j} \\ & \quad + \sum_{j \in U} \left( \sum \{(-1)^{|T|} : j \in T \text{ and } T \subset U \} \right) \frac{p_j}{\theta_j} \end{aligned} \quad (B.47) $$

$\sum \{(-1)^{|T|} : T \subset U \} = 0$ by Lemma B.2, with $S_1 = \{1, \dots, m\}/U$, $S_2 = \{1, \dots, m\}$. Here $S_1 \neq S_2$, since $|U| \ge 1$. The sum at the end of Equation (B.47) can be rewritten:

$$ \begin{aligned} & \sum_{j \in U} \left( \sum \{(-1)^{|T|} : j \in T \text{ and } T \subset U \} \right) \frac{p_j}{\theta_j} \\ &= \sum_{j \in U} \left( \sum \{(-1)^{|T'|+1} : T' \subset U/\{j\} \} \right) \frac{p_j}{\theta_j} \\ &= \sum_{j \in U} \left[ -(1+(-1))^{|U/\{j\}|} \right] \frac{p_j}{\theta_j} \\ &= \sum_{j \in U} \left[ -(1+(-1))^{|U|-1} \right] \frac{p_j}{\theta_j}. \end{aligned} \quad (B.48) $$

The last expression is zero unless $|U| = 1$. But if $|U| = 1$, then $U = \{j\}$ for some $j$, and the last expression becomes $-p_j/\theta_j$.
Now consider $\sum_{S' \subset \{1, \dots, m\} / T} q_{S'}$, but with $T$ replaced by $T \cup \{1, \dots, m\} / U$. Now $T \cup \{1, \dots, m\} / U$ consists of elements of $\{1, \dots, m\}$ that are either in $T$ or not in $U$, so its complement includes all $j$ that are in $U$ but not in $T$:

$$ \{1, \dots, m\} / (T \cup \{1, \dots, m\} / U) = U/T. $$

So the sum to consider is

$$ \begin{aligned} & \sum \{(-1)^{|T|} \left( \sum_{S' \subset U/T} q_{S'} \right) : T \subset U \} \\ &= \sum_{S' \subset U} \left( \sum \{(-1)^{|T|} : T \subset U/S' \} \right) q_{S'} \\ &= \sum_{S' \subset U} (1 + (-1))^{|U/S'|} q_{S'} \\ &= \left( \sum \{0 \cdot q_{S'} : S' \subset U, S' \neq U\} \right) + q_U = q_U. \end{aligned} \quad (B.49) $$

Now put Equations (B.45)–(B.49) together:

$$ \begin{aligned} & \sum \{(-1)^{|T|} \log f(R(\mathbf{x}, \mathbf{p}|T \cup \{1, \dots, m\}/U) | \mathbf{q}, \bar{\theta}) : T \subset U \} \\ &= \sum \{(-1)^{|T|} \left[ \left( \sum_{j=1}^{m} \frac{x_j}{\theta_j} - q_j \right) \right. \\ & \qquad \left. - 2 \left( \sum_{j=1}^{m} \frac{x_j}{\theta_j} - \sum_{j \in T \cup \{1, \dots, m\}/U} \frac{p_j}{\theta_j} - \sum_{S' \subset U/T} q_{S'} + O \left( \exp(-\min_j \left( (x_j - p_j)/\theta_j \right)) \right) \right) \right] : \\ &= 0 + 2 \left[ \sum_{j \in U} \left( \sum \{(-1)^{|T'|} : T' \subset U/\{j\} \right) \frac{p_j}{\theta_j} - \sum_{S' \subset U} \left( \sum (-1)^{|T|} : T \subset U/S' \right) q_{S'} \right] \\ & \qquad + O \left( \exp(-\min_j \left( (x_j - p_j)/\theta_j \right)) \right) \\ &= 0 + 2q_U + O \left( \exp(-\min_j \left( (x_j - p_j)/\theta_j \right)) \right) \quad \text{if } |U| > 1 \\ &= -\frac{2p_{j_0}}{\theta_{j_0}} + 2q_{j_0} + O \left( \exp(-\min_j \left( (x_j - p_j)/\theta_j \right)) \right) \\ &= 2 \left( \frac{\mu_{j_0} - p_{j_0}}{\theta_{j_0}} \right) + O \left( \exp(-\min_j \left( (x_j - p_j)/\theta_j \right)) \right) \quad \text{if } U = \{j_0\}. \end{aligned} \quad (B.50) $$

Apply equation (B.50) to sequences $\{\mathbf{x}_s^*, \mathbf{p}, \mathbf{p}_s\}_{s=1}^\infty$. Remember that by the conditions (3.2) in the statement of the Theorem, $x_{j,s}^*, \mathbf{p} - p_{j,s}$ and $x_{j,s}^*, \mathbf{p}$ go to $\infty$ as $s \to \infty$. Parts (I) and (II) of the Theorem follow. □
C Description of random walks used in example in Section 4

The random walk model described below is related to random walks designed to simulate motion in a confined space; more specifically, motion of animals. The proportions we used as outcomes are measures of tortuosity of a path in two dimensions. See [1] for an explanation.

The data were generated from a series of second order correlated random walks. That is, the directions of successive steps form an AR(2) process. Each random walk has n points and n − 1 steps. The i-th point of the random walk is p$_{i}$ , i = 1 , . . . , n . p$_{i}$ = (0 , 0). The i-th step is s$_{i}$ = p$_{i}$$_{+1}$ − p$_{i}$ . s$_{i}$ = l$_{i}$ ∗ (cos θ$_{i}$ , sin θ$_{i}$ ), where l$_{i}$ is the length of step i , and θ$_{i}$ is the direction of step i . The steps are derived as follows:

$$
\begin{align*}
\theta_i &= \theta_{i-1} + \psi_i \\
\psi_i &= \phi_i + \epsilon_i \\
\phi_i &= \phi_{i-1} + \eta_i
\end{align*}
\text{the } \{\epsilon_j\} \text{ are i.i.d. } N(\mu, \kappa_2) \\
\text{the } \{\eta_i\} \text{ are i.i.d. } N(0, \kappa_1)
$$

The step lengths were generated separately, by a moving average process.

$$
l_i = \exp \left( \sum_{j=0}^{K-1} \rho_{i+j} \right)
$$

where the { ρ$_{j}$ } are i.i.d. N (0 , 1).

1000 random walks were generated, each with n = 250 points. For each, κ$_{1}$ , κ$_{2}$ , µ , K were generated randomly:

$$
\kappa_1 \sim N(0.01, 0.09); \kappa_2 \sim (N(0.01, 0.09); \mu \sim N(0, 0.02)
$$

and K was drawn from { 3 , . . . , 15 } , with uniform weights.

The outcomes:

Divide whole real surface into unit squares with integer coordinates. Pro- portion 1 is defined to be the number of unit squares crossed by path divided by area of smallest rectangle that contains the path and whose corners have coordinates that are integers. This is related to the minimum convex polygon (MCP) described in [1], but is not quite the same (and is easier to compute).

Proportion 2: proportion of steps of path that cross at least one other step. This is very similar to relative number of crossings (RNC), also defined in [1]. However, the measure here does not take into account a step that may cross more than one other step.

Proportion 3 is relative loop complexity (RLC). This is defined as the number of loops in the path that intersect other loops, divided by the potential maximum number of such loop intersections. Again, this is defined more exactly in [1].
References

[1] Miller, C. and Christman, M. (2011). Movement in a confined space: Estimating path tortuosity. *Applied Animal Behaviour Science*, vol. 135, pp. 13-23.