BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models

## Abstract

Abstract Predictive models often need to work with incomplete information in real-world tasks. Consequently, they must provide reliable probability or confidence estimation, especially in large-scale decision making and planning tasks. Current large language models (LLM) are insufficient for such accurate estimations, but they can generate relevant factors that may affect the probabilities, produce coarse-grained probabilities when the information is more complete, and help determine which factors are relevant to specific downstream contexts. In this paper, we make use of these capabilities of LLMs to provide a significantly more accurate probabilistic estimation. We propose Bird , a novel probabilistic inference framework that aligns a Bayesian network with LLM abductions and then estimates more accurate probabilities in a deduction step. We show Bird provides reliable probability estimations that are 30% better than those provided directly by LLM baselines. These estimates can further contribute to better and more trustworthy decision-making.

## 1 Introduction

Accurate, interpretable, and unbiased probabilistic estimation is crucial to artificial intelligence (AI) systems in mission-critical tasks that require trustworthy decision-making. This is because, in many real-world applications, AI systems have to work with partial and incomplete observations that are insufficient for deductive and deterministic decisions (McCarthy & Hayes, ). Consider the charging station planning scenario in Fig. 1, where we want to use large language models (LLM) to decide where to build a certain number of charging stations over a region, subject to some constraints.
Because we cannot always observe all the necessary information regarding a potential location, we can only provide confidence or probability estimates on whether we should use the candidate location. If the number of planned stations is significant, we need to perform an optimization process such as ILP (Roth & Yih, ) that maximizes the estimated probabilities of each individual station. A simple ranking of the candidates would not suffice in this scenario because it may yield sub-optimal solutions, as Fig. 1 shows.
However, current LLMs are not ideal for such probability estimation tasks since 1) LLMs tend to be inaccurate and overconfident when estimating numerical confidences and probabilities (Xiong et al., ) and 2) they cannot provide an out-of-the-box interpretable and controllable process on how their estimates are derived (Li et al., ). This prevents us from using LLMs’s decision-making capabilities in large-scale automatic decision-making tasks. Consequently, there is a need to find ways to infer better probabilities from these models.

In this work, we propose Bird (Bayesian Inference from Abduction and Deduction).
Bird builds on three observations. The first observation is that although LLMs cannot produce convincing numerical probabilities directly, they can generate neutral and comprehensive real-world factors relevant to these probabilistic estimates. Moreover, we observe that they can make relatively correct coarse probability estimations when using these factors as contexts that provide more complete information.
The last observation is that LLMs can decide how specific contexts and conditions are related to these factors via entailment classfications.
Together, these observations motivate Bird, where we approximate a Bayesian network (Friedman et al., ) using LLM-generated factors, optimize its internal conditional probabilities by sampling LLM coarse predictions on combinations of values of all the factors, and infer more accurate probabilities for specific downstream conditions using LLMs to decide how the conditions are mapped to the factors.

Figure: Figure 1: An example of mission-critical tasks. We first ask OpenAI o1 to separately predict the probability of building a charging station at each specific location. It estimates the same probabilities for two different conditions twice, while Bird (ours) successfully distinguishes the difference and can thus help the user make a more informed decision. This further demonstrates that while LLMs are capable of coarse estimations, they struggle to generate accurate probabilities. We further show through OpenAI o1 Ranking that although it can successfully rank all four locations based on how likely it should be used to build a new charging station, it is still insufficient to complete the task because of ties (e.g., 1/4 and 2/3 are the same from ranking perspectives.)
Refer to caption: extracted/5930199/Example.png

We formulate the probability estimation problem as deciding $P(O_{i}|C)$, where $O_{i}$ is a potential outcome and $C$ is the downstream condition or context (e.g., the location is on a busy highway) under a decision-making scenario $S$ (e.g., deciding whether to build a charging station at the location).
Given $S$, Bird first generates a comprehensive set of real-world factors and values $f_{1},f_{2},...,f_{n}$ that may contribute to the probabilistic estimations. Then, it approximates a Bayesian network structure using the generated factors and the outcomes $\{O\}$.
We propose a novel method for estimating the conditional probability tables (CPT) for the outcome, i.e., $P(O_{i}|f),f\in\mathcal{F}$ where $\mathcal{F}$ is the product space of value sets for all factors.(^1^11$\mathcal{F}$ is the set of all possible combinations of different assignments of values to all factors (e.g., selecting a value from each of the factors a/b/c/d/e/f in Fig. 2). Refer to §3.) Specifically, we assume a set of learnable parameters $P(O_{i}|f_{j})$ representing the conditional probability of each possible outcome, given a single variable $f_{j}$.
We then sample $f\in\mathcal{F}$ and acquire LLM’s coarse probability estimations $P_{LLM}(O_{i}|f)$.
Next, we approximate the conditional probability $P_{estimated}(O_{i}|f)$ in the CPT using the learnable parameters $P(O_{i}|f_{j})$ based on a derived approximation formula and optimize these parameters by minimizing the distributional distance between $P_{estimated}(O_{i}|f)$ and the LLM’s coarse predicted probability $P_{LLM}(O_{i}|f)$. From a high level, this abduction step aligns language models’ internal probability estimation with Bayesian constraints and has better error tolerance. The resulting Bayesian network is generic to all possible conditions in the scenario $S$.
In the following deduction step, we use the trained Bayesian parameters to estimate the probability of any possible conditions and observations $P(O_{i}|C)$. To do this, we employ an entailment process to determine $P(f_{i}|C)$, as shown by “LLM Entailment” in Fig. 2, and the outcome we are interested in, $P(O_{i}|C)=\sum_{f\in\mathcal{F}}P(O_{i}|f)P(f|C)$.

As we show in §4, the inferred probabilities from Bird are more reliable, consistent with human judgments on 30% more instances than those directly predicted by LLMs. We further show that Bird probability predictions are good enough to be directly used for decision-making (i.e., by computing an argmax on the predicted probabilities of each label for inference), outperforming standard chain-of-thought (CoT) (Wei et al., ) on several reasoning benchmarks. More importantly, Bird achieves better probability estimations by optimizing an interpretable Bayesian network with natural-language-based variables. There are two additional benefits of Bird. First, we show that Bird aligned probabilities can serve as more accurate and fine-grained supervision signals for training smaller models. In addition, we demonstrate that Bird’s factor generation process is neutral and unbiased to the specific conditions, so it can be used to generate better follow-up questions to improve decision making confidence. Combining these findings, we show that Bird improves trustworthiness: a crucial ability for mission-critical applications such as medical chatbots, where we need reliable decisions supported with enough evidence for high-confidence predictions.

## 2 Related Work

Direct Inference in LLMs. Our paper is first related to LLM direct inference methods (Wei et al., ; Wang et al., ), and decomposition-based reasoning (Wolfson et al., ; Tafjord et al., ; Zhou et al., ; Kassner et al., ). Compared to these works, Bird considers using abduction and deduction to facilitate probabilistic induction, which aligns with the slow-thinking systems (McGlynn, ; Lin et al., ). Our work is also related to chain-of-thought via latent-variable inference. Given a question-answer pair $(X,Y)$, existing methods  (Hoffman et al., ; Hu et al., ) aim to find latent chains of thought – token sequences $Z$ that contribute the most to the conditional likelihood. While Bird has a more structured $Z$ and focuses more on the proper probability estimation instead of merely finding the optimal $Z$.

Uncertainty Estimations and Decision making under Uncertainty in LLMs.
Most current works focus more on sampling to access the model’s uncertainty of factual knowledge (Kuhn et al., ; Yadkori et al., ; Lin et al., ). Some works verbalize the uncertainty of LLMs (Tian et al., ; Xiong et al., ). However, Xiong et al. () shows that LLMs are often overconfident when directly verbalizing their confidence. Hou et al. () is most related to us, as it also incorporates the intuition of decomposition, but it focuses more on addressing aleatoric uncertainty due to input ambiguity without rigorous probability calculation. This work is also related to decision making with uncertainty (Ren et al., ; Han et al., ; Liu et al., ) while all methods use LLM’s direct uncertainty estimation such as token logits or verbalization. Bird introduces external reliable Bayesian modeling to more accurately calibrate uncertainty in LLMs for decision making.

Probabilistic Inference with LMs.
Chen et al. () is one of the first papers to estimate a probability out of an NLI problem while with no explanation.  Ozturkler et al. () follows a similar intuition to ours, where the model first thinks by retrieving relevant associations, and then conducts probabilistic reasoning. Lew et al. (); Wong et al. () discuss translating from natural language to the probabilistic language of thought. However, none of the papers tackle the real-world complex settings under incomplete information as comprehensively as we do with the level of interpretability to users of any level.

## 3 Modeling

Figure: Figure 2: Overview of Bird. Given a scenario $S$, LLMs generate the factors $F$ (a/b/…) with potential values ($f_{1}$={a.1,a.2}). $\mathcal{F}$ is the set of all value combinations of $F$ (e.g., a.1/b.2/c.1/…). Bird approximates a Bayesian network parameterized by $\mathbb{P}(O_{i}|f_{j})$, and optimizes by sampling LLM coarse predictions on $\mathbb{P}_{\mathrm{LLM}}(O_{i}|f),f\in\mathcal{F}$, and minimizes the distributional distance between an approximated $\mathbb{P}_{\mathrm{estimated}}(O_{i}|f)$ and $\mathbb{P}_{\mathrm{LLM}}(O_{i}|f)$. At inference time, each context $C$ (S+U1/U2/…) is mapped to some $f_{j}$ via entailment, and a probability is derived using $\mathbb{P}_{\mathrm{estimated}}$. Bird can further be used to generate follow-up questions.
Refer to caption: extracted/5930199/Framework.png

### 3.1 Bayesian Inference from Abduction and Deduction

On a high level, given a decision $Y$ and partial information $X$, instead of direct inference using LLMs $({\rm induction}:X\rightarrow Y)$,
Bird conceptualizes the general scenario behind $X$ into intermediate factors $({\rm abduction}:X\rightarrow Z)$, and then fits a Bayesian model with $Z$. The model then estimates probabilities for $Y$ $({\rm deduction}:X,Z\rightarrow Y)$.
Our problem setting is a generic contextual inference problem of the type inspired by Feng et al. ().
Given the context $C$, which includes a general scenario $S$ and a corresponding downstream condition $U$, our task is to estimate $\mathbb{P}(O_{i}|C),i=1,2,C=(S,U)$, where $\{O\}$ is denoted as $O_{1}$ and $O_{2}$. We decompose the overall context $C$ into the general scenario $S$ and downstream conditions $U$ because we want to build a neutral intermediate space $Z$ based on only $S$, so that the probability estimation is unbiased towards any specific conditions. This would mitigate models relying on incorrect signals, such as charging stations are normally built in higher-income neighborhoods, but more generic factors like accessibility for trustworthy decision making.

Instead of directly making a prediction, we approximate a controllable and interpretable inference process based on explicit Bayesian modeling (Gelman et al., ) containing natural-language-based variables.
For a scenario $S$, we conceptualize $N$ factors $\{F_{j}\}_{j=1}^{N}$ based on $S$ that are relevant to the outcome $O$, which can take on two possible values, $O_{i},i=1,2$, similar to the structure found in Bayesian networks (Koller & Friedman, ). Here the real-world factors and the outcome are equivalent to the variables in the network. Therefore, the choice between outcomes $O_{1}$ and $O_{2}$ depends on the factors, with each discrete factor $F_{j}$ having a set of possible values as $\mathcal{F}_{j}$, containing all possible details of the factor required for deciding between $O_{1}$ and $O_{2}$.
We use $\mathcal{F}=\prod_{j=1}^{N}\mathcal{F}_{j}$ to denote the product space of value sets for all factors. For an instance in the space, each factor $F_{j}$ should be attributed a specific value $f_{j}$. We assume such derived factors are complete at this stage. That is, they include all the possible individual factors/angles that may affect the distribution of the outcomes.(^2^22Note that our framework can later accommodate any additional factors by selectively recalculating specific parts as needed.) As a result, $\mathcal{F}$ is a complete information space for decisions. As in Fig. 2, (factor a value 1 $a_{1}$, factor b value 1 $b_{1}$, factor c value 1 $c_{1}$, factor d value 1 $d_{1}$, factor e value 1 $e_{1}$, factor f value 1 $f_{1}$) is an element $f$ in the product space $\mathcal{F}$ where there are $2^{6}$ elements in total. We then calculate the conditional probability table (CPT) as $P(O_{i}|f),\forall f\in\mathcal{F},f=(f_{1},f_{2},...,f_{N})$.

We assume 1) $\{F_{j}\}_{j=1}^{N}$ is complete, so $O$ does not depend on any other intermediate factors, and 2) only one value can be assigned for each factor for a given $C$.
The predictive probability for the outcome is thus obtained by marginalizing over the product space $\mathcal{F}$ for a context $C$ (details in §A.1):

$$ $\mathbb{P}(O_{i}|C)=\sum_{f\in\mathcal{F}}\mathbb{P}(O_{i}|f)\mathbb{P}(f|C)$ (1) $$

where $i=1,2,C=(S,U)$, $\forall f\in\mathcal{F},f=(f_{1},f_{2},...,f_{N}),f_{j}\in\mathcal{F}_{j},j=1,
...,N$. $\mathcal{F}$ is the product space of value sets for all factors. $f$ is a possible instance of complete information in the space.

Assumption on Conditional Independence of Variables. To provide relaxation for real-world applications, and since the factors should cover as many different aspects as possible, we assume factors are conditionally independent given the context. Therefore,

$$ $\mathbb{P}(O_{i}|C)=\sum_{f\in\mathcal{F}}\mathbb{P}(O_{i}|f)\prod_{j=1}^{N} \mathbb{P}(f_{j}|C)$ (2) $$

In the following sections, we describe the missing implementation details of the framework: (1) the generation of the factors $\{F_{j}\}_{j=1}^{N}$ (§3.2); (2) the assignment of the conditional probability tables $\mathbb{P}(O_{i}|f)$ (§3.3), and (3) the computation of observations $\mathbb{P}(f_{j}|C),j=1,...,N$ (§3.4).

### 3.2 Abductive Factor Generation

Large language models, although they cannot solve some complicated cases, are shown to possess parametric knowledge that enables them to solve more common cases that can be directly found in their memory (Kambhampati et al., ; Li et al., ; Zhou et al., ). Our method for generating the factors follows this similar intuition.
For a scenario $S$, we first do abstractions with LLMs to derive $N$
factors $\{F_{j}\}_{j=1}^{N}$. The most straightforward approach involves utilizing LLMs to directly obtain a list of factors and their potential values. However, direct generation is effective but not sufficiently comprehensive, and we want to fully utilize past similar occurrences that the model memorizes in their parametric knowledge.

Therefore, to encourage models to fully explore the range of alternatives, we divide the task into two stages: 1) generating sentences describing familiar situations that would increase the likelihood of an outcome, and 2) summarizing these sentences into factors with corresponding values. We show the prompts in Appendix Fig. 5 and Fig. 6.
We further utilize LLMs to perform classification to assess how the value of each generated factor affect the decision, so that we can filter out factors that are unrelated to the outcomes.
The prompt is listed in Appendix Fig. 7.
In the example depicted in Fig. 2, a factor like “The color of the vehicle” will be eliminated because all potential values of this factor consistently support the same outcome, i.e., neutral.

### 3.3 Assignment of ℙ ⁢ ( O i | f ) ℙ conditional subscript 𝑂 𝑖 𝑓 \mathbb{P}(O_{i}|f) blackboard_P ( italic_O start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | italic_f ) as a constrained optimization problem

We now calculate each $\mathbb{P}(O_{i}|f),i=1,2,\forall f\in\mathcal{F}$ in the conditional probability table of the Bayesian network. Since we do not have any observed data for approximation and LLMs alone cannot directly give an accurate assignment on $\mathbb{P}(O_{i}|f)$, we adopt the formula proposed in Bordley (), which derived a version of the logarithmic opinion pool using axioms from the theory of additive conjoint measurement. This formula is proved to be the only continuous (over the open interval (0, 1)) group probability formula under the assumption detailed in §A.2 that the decision maker has an intuitive weak ordering on the set of factor odds ratios and a “noninteraction” property of factors. Our setting fits the assumption as the weak ordering of decisions is the most fundamental condition of consistency in decision making and we assume the conditional independence of factors. Therefore,

$$ $\mathbb{P}(O_{i}|f)=\frac{\prod_{j=1}^{N}(\frac{\mathbb{P}(O_{i}|f_{j})}{ \mathbb{P}(O_{i})})^{w_{j}}\mathbb{P}(O_{i})}{\prod_{j=1}^{N}(\frac{\mathbb{P} (O_{i}|f_{j})}{\mathbb{P}(O_{i})})^{w_{j}}\mathbb{P}(O_{i})+\prod_{j=1}^{N}( \frac{1-\mathbb{P}(O_{i}|f_{j})}{1-\mathbb{P}(O_{i})})^{w_{j}}(1-\mathbb{P}(O_ {i}))}$ (3) $$

To determine the weights $w_{j}$,(^3^33Weights here refer to a set of “hyperparameters” that decides how highly the final decision regards each factor and how correlated the factors are considered, not the learnable parameters in our Bayesian network.)
, we adopt the second case scenario mentioned in Bordley (), i.e., $\sum_{j=1}^{N}w_{j}>1$. In this case, the decision maker with complete information is more certain than the decision maker with only one piece of information through acquiring all different pieces of information from different factors. Since there isn’t a priori information that suggests another choice exists, we use equal weight $w_{j}=1,j=1,...,N$ and assume $\mathbb{P}(O_{i})=50\%,i=1,2$. The final estimation formula can be rewritten as:

$$ $\mathbb{P}_{\rm estimated}(O_{i}|f)=\mathbb{P}_{\rm estimated}(O_{i}|f_{1},f_{ 2},...f_{N})\\ \approx\frac{\prod_{j=1}^{N}\mathbb{P}(O_{i}|f_{j})}{\prod_{j=1}^{N}\mathbb{P} (O_{i}|f_{j})+\prod_{j=1}^{N}(1-\mathbb{P}(O_{i}|f_{j}))}\\$ (4) $$

Constrained optimization for estimating $\mathbb{P}(O_{i}|f)$.
From Eq. 4, it is not hard to see that each $\mathbb{P}(O_{i}|f),\forall f\in\mathcal{F}$ can now be approximated using only $\mathbb{P}(O_{i}|f_{j}),\forall f_{j}\in\mathcal{F}_{j},j=1,...,N$, which we view as learnable parameters.
As discussed in §1, LLMs can perform relatively correct coarse estimation under complete information, which refers to contexts with all factors’ values present (i.e. $f\in\mathcal{F})$. Consequently, we prompt the LLMs to output verbalized probability estimations for outcome $O_{i}$ given $f$.
This approximates LLM’s predictions of $\mathbb{P}_{LLM}(O_{i}|f)$.
With $\mathbb{P}_{\rm estimated}$ and $\mathbb{P}_{LLM}$, we can now form a constraint optimization problem that finds the values of parameters $\mathbb{P}(O_{i}|f_{j})$
that minimize the distributional distance between the Bayesian estimated probabilities $\mathbb{P}_{\rm estimated}(O_{i}|f)$ and LLM estimated probabilities $\mathbb{P}_{LLM}(O_{i}|f),\forall f\in\mathcal{F}$, while conforming to the structure in Eq. 4 and preliminary assessments about each $\mathbb{P}(O_{i}|f_{j}),\forall f_{j}\in\mathcal{F}_{j},j=1,...,N$ provided by the LLM classification as mentioned in §3.2.

Learning algorithm for constrained optimization to estimate $\mathbb{P}(O_{i}|f)$.
We propose a gradient-descent algorithm for this optimization problem.
The detailed algorithm can be found in §A.3, and we give a general description here.
As the parameter initialization, we utilize the LLM classification as mentioned in §3.2, where we direct the LLMs to ascertain which outcome is more likely supported by the value of each factor:

$$ $\mathbb{P}_{\rm init}(O_{i}|f_{j})=\left\{\begin{array}[]{lc}75\%&f_{j}\ {\rm supports }\ {\rm outcome}\ i\\ 50\%&f_{j}\ {\rm is}\ {\rm neutral}\\ 25\%&f_{j}\ {\rm supports}\ {\rm opposite}\ {\rm outcome}\ \neg i\\ \end{array}\right.$ (5) $$

For the training data,
given an instance of complete information $f^{{}^{\prime}}=(f_{1}^{{}^{\prime}},f_{2}^{{}^{\prime}},...,f_{N}^{{}^{\prime
}})$,
we prompt LLM to output a verbalized probability score for each outcome $\in$ {very unlikely, unlikely, somewhat unlikely, neutral, somewhat likely, likely, very likely} which we use as $\mathbb{P}_{LLM}(O_{i}|f^{{}^{\prime}})$. The detailed prompt can be found in Appendix Fig. 10. We then assign a specific probability value for each coarse estimation, i.e., very unlikely: 0%, unlikely: 20%, somewhat unlikely: 40%, neutral: 50%, somewhat likely: 60%, likely: 80%, very likely: 100%, as supervision signals to learn the correct value of $\mathbb{P}(O_{i}|f_{j}^{{}^{\prime}}),j=1,...,N$ for each $f_{j}^{{}^{\prime}}$ in $f^{{}^{\prime}}$ using Eq.4. We randomly sample 128 instances in the space of complete information as the training data for each scenario. Note that we manually assign probability values for LLM coarse decisions, but they can be changed as hyperparameters.

We adopt two loss functions: 1) we use the regular mean squared error loss for optimization and denote the loss as $\ell_{\rm MSE}(\mathbb{P}_{\rm estimated}(O_{i}|f^{{}^{\prime}}),\mathbb{P}_{
\rm LLM}(O_{i}|f^{{}^{\prime}}))$; 2) we use the margin ranking loss to ensure the final trained individual probability preserve the original direction of its supported outcome and denote the loss as $\ell_{MR}$:

$$ $\begin{split}\ell_{\rm MR}&={\rm max}(0,-y_{\rm target}(O_{i}|f_{j})(\mathbb{P }_{\rm trained}(O_{i}|f_{j})-0.5)+\epsilon)\\ y_{\rm target}(O_{i}|f_{j})&=\mathrm{sgn}(\mathbb{P}_{\rm init}(O_{i}|f_{j})-5 0\%)\\ \mathbb{P}_{\rm trained}(O_{i}|f_{j})&=\sum_{f^{*}\in\mathcal{F}^{*}}\mathbb{P }(O_{i}|f^{*}\cup f_{j})\mathbb{P}(f^{*}|f_{j})=\frac{1}{{\rm card}(\mathcal{F }^{*})}\sum_{f^{*}\in\mathcal{F}^{*}}\mathbb{P}(O_{i}|f^{*}\cup f_{j})\end{split}$ (6) $$

$\forall f_{j}\in\mathcal{F}_{j},j=1,...,N$. We apply probability marginalization to get the trained individual probability and approximate it with the mean values since there are no explicit priors. We use $\mathcal{F^{*}}$ to denote the product space of the remaining factors except $\mathcal{F}_{j}$, which is a subspace of $\mathcal{F}$ and $f^{*}=(f_{1},f_{2},...,f_{j-1},f_{j+1},...,f_{N}),f^{*}\in\mathcal{F^{*}}$. Therefore, the final loss function we use for training is defined as follows:

$$ $\mathcal{L}=\ell_{\rm MSE}+\alpha\ell_{\rm MR}$ (7) $$

where $\alpha$ balances the two losses. We use SGD as the optimization function. As a result, we can estimate any $\mathbb{P}(O_{i}|f),\forall f\in\mathcal{F}$ with the learned $\mathbb{P}(O_{i}|f_{j}),\forall f_{j}\in\mathcal{F}_{j}$ with Eq.4.

Customization to human preference. A benefit of our design is that we can explicitly ask for human preferences on how a particular factor value will affect the outcome, i.e., manually setting $\mathbb{P}(O_{i}|f_{j})$. Such preferences are desirable in certain applications with known constraints, and they can be easily integrated in Eq.4 by replacing $\mathbb{P}(O_{i}|f_{j})$ with the human preferred value.

### 3.4 Computation of ℙ ⁢ ( f j | C ) ℙ conditional subscript 𝑓 𝑗 𝐶 \mathbb{P}(f_{j}|C) blackboard_P ( italic_f start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | italic_C ) with LLM Entailment

This section discusses how we compute $\mathbb{P}(f_{j}|C)$. As noted in §3.1, $C$ is the context containing the general scenario $S$, and a specific additional condition $U$. Given $C$, we employ an entailment task formulation to find which factors and their corresponding values are implied or mapped by $C$.
Specifically, we adopt the prompt in Appendix Fig. 8 where we directly ask if the context entails a value from a factor. We conduct another round of self-correction using the prompt in Appendix Fig. 9 to check if the model believes all the previously identified values are actually entailed.

We define the notion of observed factors, which refers to a sub-set of factors among all factors that are being implied by the context. For example, as shown in Fig. 2, context $S+U_{1}$ implies $a_{1}$, $c_{1}$, and $d_{1}$, $S+U_{2}$ implies $c_{1}$ and $d_{1}$, while the additional condition $S+U_{3}$ implies $d_{1}$ and $f_{2}$. We assume only one value can be chosen per factor. We write the product value space of the implied factors as $\mathcal{F^{\prime}}=\prod_{m=1}^{M}{\mathcal{F}_{j_{m}}}$, which is a subspace of $\mathcal{F}$. We define the conditional probabilities of the values of these observed factors to be 1 if the value is implied by $C$, a 0 if not:

$$ $\begin{split}\mathbb{P}(f_{j_{m}}|C)=\left\{\begin{array}[]{lc}1&f_{j_{m}}=f_{ j_{m}}^{\ast},f_{j_{m}}\in\mathcal{F}_{j_{m}}\\ 0&f_{j_{m}}\neq f_{j_{m}}^{\ast},f_{j_{m}}\in\mathcal{F}_{j_{m}}\\ \end{array}\right.\end{split}$ (8) $$

Where the implied $f_{j_{m}}^{\ast}\in\mathcal{F}_{j_{m}},\forall m=1,...,M$.
For the unobserved factors, we have no information regarding which value to be selected. In order to be neutral and unbiased, we assume that, in an unobserved factor, each value has an equal probability of being selected.
We use ${F^{\prime}}^{\perp}=\prod_{m=M+1}^{N}\mathcal{F}_{j_{m}}$ to denote the space of unobserved factors, which is the orthogonal complement space of $\mathcal{F^{\prime}}$ in $\mathcal{F}$,

$$ $\mathbb{P}(f_{j_{m}}|C)=\frac{1}{{\rm card}(\mathcal{F}_{j_{m}})},f_{j_{m}}\in \mathcal{F}_{j_{m}}$ (9) $$

Where $\forall m=M+1,...,N$. We can therefore calculate the final probability as presented in Eq.1, utilizing the formulations provided in Eqs.2 through 9.

## 4 Experiments

### 4.1 Datasets and Settings

We convert three datasets focusing on reasoning and planning to our problem setting including $O$, $S$, and $U$, as described in §3.1. We detail this process in §A.4.
Common2sense (Singh et al., ) focuses on commonsense reasoning, and we gather 213 scenarios and 3822 instances, where 9 additional conditions on average support each outcome. Plasma (Brahman et al., ) focuses on planning, and we gather 279 scenarios and 1395 instances, where 5 additional conditions support the less common outcome. Today (Feng et al., ) focuses on temporal reasoning, where there are 1000 instances. Among the three datasets, the most challenging task is Today and the easiest is Common2sense, judged by baseline model performances. The complexity of the scenario and conditions is highest in Today. The conditions support the less common outcome in Plasma and Common2sense presents general decision making with common sense.

We conduct our experiments with Llama-2-70b-instruct and Llama-3.1-70b-Instruct.(^4^44[https://huggingface.co/upstage/Llama-2-70b-instruct](https://huggingface.co/upstage/Llama-2-70b-instruct) and [https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct)) We set the temperature = 0.7. For the factor generation task in §3.2, we generate 10 sentences for each outcome before summarization.
For the LLM classification task for factor-outcome mapping, we generate 3 times and conduct a majority vote.
For the LLM entailment task in §3.4, we sample 3 times and conduct a majority vote with 1 round of self-reflection.
We set learning rate = 1e-02, epoch = 20, batch size = 4, $\epsilon$ = 0, $\alpha$ = 10 for the learning algorithm mentioned in §3.3.

### 4.2 Intrinsic Evaluations of the Bird Framework

**Table 1: Preference-based pairwise evaluation using F1 score for predicted probabilities across various model settings. Different1 indicates the F1 score for a category where $C_{1}$ supports the gold outcome better than $C_{2}$, and vice versa for Different2. Same refers to the F1 score of the category where $C_{1}$ and $C_{2}$ equally support the outcome. Average refers to calculating F1 metrics globally by counting the total true positives, false negatives, and false positives. EC* settings see both $C_{1}$ and $C_{2}$ at the same time instead of predicting individual probabilities, hence not directly comparable.**
| Model | Different1 | Different2 | Same | Average |
| --- | --- | --- | --- | --- |
| Random Guessing | 0.333 | 0.333 | 0.333 | 0.333 |
| GPT3.5 CoT | 0.306 | 0.306 | 0.242 | 0.283 |
| GPT4 CoT | 0.312 | 0.357 | 0.216 | 0.289 |
| Llama2-70b Instruct Logit | 0.263 | 0.228 | 0.205 | 0.228 |
| Llama2-70b Instruct Vanilla | 0.375 | 0.333 | 0.243 | 0.311 |
| Llama2-70b Instruct CoT | 0.315 | 0.323 | 0.254 | 0.294 |
| Llama3.1-70b Instruct Logit | 0.300 | 0.282 | 0.242 | 0.269 |
| Llama3.1-70b Instruct Vanilla | 0.365 | 0.301 | 0.251 | 0.303 |
| Llama3.1-70b Instruct COT | 0.426 | 0.370 | 0.227 | 0.346 |
| Llama2-70b Instruct EC* | 0.530 | 0.529 | 0.207 | 0.503 |
| Llama3.1-70b Instruct EC* | 0.535 | 0.538 | 0.286 | 0.511 |
| GPT4 EC* | 0.588 | 0.533 | 0.3 | 0.54 |
| Llama3.1 Bird (ablation w 1/2 assumption) | 0.527 | 0.532 | 0.196 | 0.480 |
| Llama3.1 Bird (ablation w 1/n assumption) | 0.572 | 0.584 | 0.272 | 0.532 |
| Llama3.1 Bird (ablation w fixed initial prob) | 0.616 | 0.597 | 0.337 | 0.568 |
| Llama2 Bird (ours w optimized prob) | 0.614 | 0.624 | 0.450 | 0.592 |
| Llama3.1 Bird (ours w optimized prob) | 0.612 | 0.625 | 0.382 | 0.588 |

Reliability of Bird’s Estimated Probabilities.
We first propose an evaluation method that directly benchmarks the quality of probability estimations. We adopt a similar setting as mentioned in §3.1. Given a scenario $S$ and two potential outcomes $O_{1}$ and $O_{2}$, we find two specific additional conditions $U_{1}$ and $U_{2}$, both supporting $O_{1}$ over $O_{2}$.(^5^55Here $O_{1}$ will be the gold outcome. During evaluation, the two conditions can also support $O_{2}$ over $O_{1}$.) $S+U_{1}$ and $S+U2$ will form two contexts $C_{1}$ and $C_{2}$, respectively. The goal of this task is to evaluate if models can assign fine-grained probability estimations, that is, sufficient to distinguish which context between $C_{1}$ and $C_{2}$ better supports $O_{1}$ over $O_{2}$ or if both $C_{1}$ and $C_{2}$ equally support $O_{1}$ over $O_{2}$, by predicting probability values of $P(O_{i}|C1)$ for $C_{1}$ and $P(O_{i}|C2)$ for $C_{2}$ individually.(^6^66For proper evaluation, systems predict probabilities one at a time, without seeing other contexts.) That is, assuming $C_{1}$ better support $O_{1}$ over $O_{2}$, we expect $P(O_{1}|C_{1})>P(O_{1}|C_{2})>P(O_{2}|C_{2})>P(O_{2}|C_{1})$, and vice versa. If $C_{1}$ and $C_{2}$ equally support $O_{1}$ over $O_{2}$, we should see $P(O_{1}|C_{1})=P(O_{1}|C_{2})>P(O_{2}|C_{1})=P(O_{2}|C_{2})$. This task is significantly more difficult than providing two contexts that support different outcomes, as systems will have to understand the nuances in probability estimation.

We collect human gold labels for which context better supports the gold outcome with rigorous quality control using MTurk. We explain our choice of this relative evaluation over asking human annotators to provide absolute probabilities from a psychological perspective, along with the details of human annotation in §A.5. We evaluate on 350 randomly sampled instances from Common2sense when the conditions can be mapped to at least one factor and therefore Bird does not produce “unknown”(^7^77Although the setting may seem to favor our proposed method, it is a fair comparison, because “unknown” predictions are label-agnostic, and we can use the same direct inference approach, such as CoT, for such cases.) and all methods to be tested below have accurate binary decisions on all instances, i.e., both conditions support the gold outcome over the opposite outcome.
We report the F1 score as the evaluation metric. We further explain how the F1 score is computed in §A.5.

We compare the following methods that directly acquire probability estimations from LLMs as baselines. 1) Vanilla (Wang et al., ): models directly verbalize the estimated probabilities. 2) Logit: we select the normalized token probability of the decision. 3) CoT (Wei et al., ): models output a reasoning process followed by verbalized estimated probabilities. 4) EC: models select which additional condition better supports the outcome without outputting any probabilities. Note that this formulation is unfair to Bird and other baselines that individually estimate the probabilities. To demonstrate the effectiveness of our learning algorithm in §3.3, we propose two ablation baselines that use other methods to find the conditional probabilities in the Bayesian network: 1) 1/2 and 1/N Assumptions: We adopt two neutral assumptions without LLM classifications for CPT calculation. 1/2: $\mathbb{P}(O_{i}|f)$ is 1 only if all factors’ values, except for any neutral ones, support $O_{i}$; if no, 0. 1/N: $\mathbb{P}(O_{i}|f)$ is calculated as the percentage of factors whose values support outcome
$O_{i}$. 2) Fixed Initial Prob: We utilize the initial probability of Eq.5 to directly estimate $\mathbb{P}(O_{i}|f)$ with Eq.4. The details of the baselines are presented also in §A.6.

Table 1 lists the results. Our framework exhibits superior alignment with human preference, with both models achieving an F1 of 59%, more than 30% higher than GPT-4. Such results demonstrate that Bird produces reliable probability estimation without additional human supervision. Notably, such probability estimation is done by judging each condition separately (i.e., Bird assigns one probability to condition 1, and later another probability to condition 2). We find that vanilla language models do much better when both conditions are provided at the same time (i.e., the EC setting) where the model can perform argmax operations. However, even under this setting that is unfair to our framework, Bird still outperforms GPT-4 by 5%.
All results suggest that vanilla LLMs cannot properly estimate probabilities, and Bird effectively improves them. In addition, by outperforming all the ablation settings, we demonstrate that our constrained optimization algorithm proposed in §3.3 effectively approximates better Bayesian network’s conditional probabilities.

Applying Bird’s Probabilities in Decision-Making.
In this section, we evaluate the performance of applying the predicted probabilities from Bird in decision-making tasks. In the general decision-making setting, we similarly assume a context $C=S+U$, two possible outcomes $O_{1}$ and $O_{2}$, and the task is to decide which outcome is more likely given $C$, similar to Fig. 2. The inference process we use for Bird is simply taking the argmax between $\mathbb{P}(O_{1}|C)$ and $\mathbb{P}(O_{2}|C)$. We remove all instances where they cannot be mapped to any factor values(i.e., Bird’s decision is “unknown”). This does not undermine our experiment setting because such removal is label-agnostic, and we can always fall back to other methods such as CoT when Bird predicts “unknown”.
We compare our framework with chain-of-thought inference with self-consistency on the same instances where Bird predictions are not “unknown”. We present the prompt for COT in Appendix Fig.14.

The results are shown in Table 2. Our first observation is that Bird’s probability estimations are accurate enough for direct decision-making. Our proposed method outperforms the chain-of-thought baseline with Llama-3.1 on all benchmarks and performs comparably using Llama-2 as the base model. This is partially because Llama-3.1 generates more diverse and accurate factors, and we analyze this in §A.7. At the same time, we see that Bird performs better on the most challenging Today dataset on both models. This demonstrates that Bird is a desirable solution for more complex scenarios where CoT tends to fail more.

We also observe the inferior performance of our method compared to CoT may be attributed to the fundamental differences between probability estimation and hard label classification. Table 6 in the appendix shows an example that CoT sometimes does correctly due to the spurious signals and biases in the conditions, which highlights the importance of our controllable inference approach. We can conclude that Bird outperforms direct inference methods like CoT in most cases and is at least comparable in decision-making while offering significantly better interpretability and controllability throughout the process.

**Table 2: Performance comparisons of Bird and baselines on decision-making benchmarks. Bird_optimized is our final model; chain-of-thought with self-consistency (CoT) is our main baseline; others are ablation baselines.**
| Dataset | Bird_1/2 | Bird_1/n | Bird_fixed_prob | Bird_optimized (ours) | CoT |
| --- | --- | --- | --- | --- | --- |
| Llama-2-70b-instruct |  |  |  |  |  |
| Today | 73.7 | 72.8 | 73.4 | 73.9 | 71.5 |
| Plasma | 72.8 | 72.3 | 72.7 | 74.0 | 76.8 |
| Common2sense | 86.9 | 86.8 | 87.5 | 89.0 | 93.8 |
| Llama-3.1-70b-Instruct |  |  |  |  |  |
| Today | 65.5 | 68.9 | 65.5 | 74.3 | 72.6 |
| Plasma | 71.3 | 66.5 | 65.7 | 73.0 | 71.5 |
| Common2sense | 78.1 | 85.4 | 86.7 | 92.3 | 90.8 |

Ablation Studies on Bird’s Components. We conduct an ablation study regarding factor generation. We compare the final decision-making accuracy based on our proposed abductive sampling method and the direct factor generation method mentioned in §3.2. Appendix Table 7 shows the results: we see that our proposed method, on average, surpasses the direct method by 4.4%/3.9% in accuracy for llama2/llama3.1, indicating the effectiveness of our proposed factor generation that collects more comprehensive signals from models’ parametric knowledge and memorization.
We also conduct ablation studies on different ways of estimating the conditional probabilities of our Bayesian network (i.e., 1/2, 1/n, and fixed_prob). Results in Tables 1 and 2 show that our proposed constrained optimization learning algorithm effectively finds more accurate estimations by outperforming all ablation baselines.

### 4.3 Extrinsic Applications of the Bird Framework

The Usage of the Reliably Estimated Probability.
As shown in §4.2, Bird can produce reliable probabilities that contribute to better decision-making. Such findings motivate us to investigate whether these probabilities can be used as supervision signals, as models may benefit from seeing more than the discrete classification labels (Zhou et al., ).
Specifically, we use Bird to estimate probabilities of some decision-making training instances and finetune a T5-large model using these generated probabilities. We then evaluate its performance on cross-domain datasets. We fully replicate the experimental settings described in Feng et al. (). We use instances from Common2sense, where the argmax of the estimated probabilities from our framework using Llama2 supports the correct outcome, as additional training data, besides the originally proposed training data in that paper. We test on two general temporal reasoning datasets, Matres (Ning et al., ) and Tracie (Zhou et al., ), as well as two commonsense reasoning and planning datasets, Today, and Plasma. The results are shown in Table 3. We see that models achieve an average performance increase of 1.3% compared with Bird-generated probabilities as additional signals. We use cross-entropy loss with class probabilities as the target for our soft probability setting and cross-entropy loss with class labels as the target for the hard-label setting.
Such results suggest another effective application of the Bird framework, where smaller models can benefit from soft probability alignment and make better overall predictions.

**Table 3: System performances under different supervision data across three binary temporal benchmarks and one binary planning benchmark. For simplicity, we use “hard label” representing that we use Common2sense supervision data with explicit binary labels, and “Bird prob” representing that we use Common2sense supervision data with estimated probabilities. Today (exp) uses gold explanations during evaluation.**
| Model (Train Data) | Today (exp) | Today | Tracie | Matres | Plasma | Average |
| --- | --- | --- | --- | --- | --- | --- |
| T5_large | 77.8 | 57.9 | 73.0 | 83.5 | 48.1 | 65.6 |
| + hard label | 83.6 | 62.9 | 68.2 | 75.2 | 65.2 | 67.9 |
| + Bird prob (ours) | 84.3 | 63.4 | 71.4 | 77.6 | 63.3 | 68.9 |
| PatternTime | 82.2 | 60.9 | 79.8 | 85.8 | 50.5 | 69.3 |
| + hard label | 83.9 | 61.8 | 76.2 | 83.7 | 56.9 | 69.7 |
| + Bird prob (ours) | 85.1 | 62.6 | 75.7 | 85.1 | 61.5 | 71.2 |

Using Bird to Generate Better Follow-up Questions.
Generating accurate and informative follow-up questions is an important goal for LLMs to be applied in mission-critical tasks that require unambiguous decision-making. A natural benefit of our proposed Bird framework is that it analyzes what needs to be known through the comprehensive and unbiased abduction step (i.e., factor generation) and interprets how new information will change the probability or confidence of a desired decision during the deduction step (i.e., probability generation). We conduct a follow-up question generation experiment to demonstrate this benefit.
For a given decision-making problem, we ask both Bird and LLMs to generate binary (yes/no) follow-up questions. For Bird,
we identify implied factors, choose the most important unmentioned one, adopt the most important value aligning with the outcome, and rephrase it as a question. We directly ask LLM to generate questions for better decision making. The details can be referred to in §A.9. We then recruit crowd-workers on MTurk to provide pairwise preference between the Bird generated question and the question generated directly by the LLM. We do this on 250 examples from Common2sense.
An example can be seen in Appendix Fig. 4 and the details can also be seen in §A.9. Appendix Table 8 shows that the follow-up questions generated by Bird are more often preferred: questions generated by Bird are preferred 52.8% of the time, while those directly generated by LLM are preferred 32.8% of the time; in the remaining 14.4%, both are considered equally useful. This result motivates future work on applying Bird in an interactive agent system for more efficient and controlled decision making.

## 5 Conclusion

In this paper, we propose Bird, a Bayesian inference framework for large language models. Bird utilizes LLMs’ existing reliable capabilities such as abductive thinking and coarse-grained classification to optimize an explicit Bayesian network for more reliable probability generation. We show that Bird can output accurate probabilities, outperforming LLM vanilla probabilties by 30% on direct estimation tasks, and outperform chain-of-thought on decision-making tasks. At the same time, Bird’s outputs are explainable, because of its natural-language-based variables in the fully-interpretable Bayesian network. We also show that Bird can be used in several other settings, such as generating fine-grained supervision signals and asking informative follow-up questions, to further improve the trustworthiness and controllability of LLMs when applied to real-world applications.

## Appendix A Appendix

### A.1 Derivation of the equation for uncertainty estimation

We derive the outcome probability for the given context $C$ of partial information, i.e., any additional condition $U$ under the scenario $S$, based on the complete information space $\mathcal{F}$ of scenario $S$:

$$ $\displaystyle\begin{split}\mathbb{P}(O_{i}|C)&=\sum_{f\in\mathcal{F}}\mathbb{P }(O_{i}|f,C)\mathbb{P}(f|C)\\ &=\sum_{f\in\mathcal{F}}\mathbb{P}(O_{i}|f,C)\mathbb{P}(f|C)(\mathbbm{1}\{f \cap C=f\}+\mathbbm{1}\{f\cap C\neq f\})\\ &=\sum_{f\in\mathcal{F}\cap\{f\cap C=f\}}\mathbb{P}(O_{i}|f,C)\mathbb{P}(f|C)+ \sum_{f\in\mathcal{F}\cap\{f\cap C\neq f\}}\mathbb{P}(O_{i}|f,C)\mathbb{P}(f|C )\\ &=\sum_{f\in\mathcal{F}\cap\{f\cap C=f\}}\mathbb{P}(O_{i}|f)\mathbb{P}(f|C)+ \sum_{f\in\mathcal{F}\cap\{f\cap C\neq f\}}\mathbb{P}(O_{i}|f,C)\mathbb{P}(f|C )\\ &=\sum_{f\in\mathcal{F}\cap\{f\cap C=f\}}\mathbb{P}(O_{i}|f)\mathbb{P}(f|C)+ \sum_{f\in\mathcal{F}\cap\{f\cap C\neq f\}}\mathbb{P}(O_{i}|f)\mathbb{P}(f|C) \\ &=\sum_{f\in\mathcal{F}}\mathbb{P}(O_{i}|f)\mathbb{P}(f|C)\end{split}$ (10) $$

Where the fourth line follows from that $\mathbb{P}(O_{i}|f,C)=\mathbb{P}(O_{i}|f)$ since $f\cap C=f$. And the second last line is derived from that $\mathbb{P}(f|C)=0$ if $f\cap C\neq f$ as for a given $C$, only one value can be chosen for each factor as in Fig. 2.

### A.2 Assumptions for estimating ℙ ⁢ ( O i | f ) ℙ conditional subscript 𝑂 𝑖 𝑓 \mathbb{P}(O_{i}|f) blackboard_P ( italic_O start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | italic_f )

Week ordering of the set of factor odds ratios. Suppose given factors $f_{1},f_{2},...,f_{N}$, we assess odds of $o_{1},o_{2},...,o_{N}$, respectively in favor of the outcome $O_{i},i=1,2$ occurring. $o_{j}=\frac{\mathbb{P}(O_{i}|f_{j})}{1-\mathbb{P}(O_{i}|f_{j})},j=1,...,N$, the intuitive ordering $\geq_{E}$ is defined as:

$$ $(o_{1},o_{2},...,o_{N})\geq_{E}(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$ (11) $$

if and only if the decision maker feels at least as sure about the outcome $O_{i}$, occurring
when given factors $f_{1},f_{2},...,f_{N}$ we assess odds $o_{1},o_{2},...,o_{N}$ -as when given factors $f_{1},f_{2},...,f_{N}$ we assess odds $o_{1}^{*},o_{2}^{*},...,o_{N}^{*}$. The ordering $\geq_{E}$ is a Weak Ordering if and only if for any
$(o_{1},o_{2},...,o_{N})$,$(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$,$(o_{1}^{**},o_{2}^{**},...,o_{N}^{**})$, we have:

- •
(a) either $(o_{1},o_{2},...,o_{N})$ $\geq_{E}$ $(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$ or $(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$ $\geq_{E}$ $(o_{1},o_{2},...,o_{N})$,
- •
(b) if $(o_{1},o_{2},...,o_{N})$ $\geq_{E}$ $(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$ and if $(o_{1}^{*},o_{2}^{*},...,o_{N}^{*})$ $\geq_{E}$ $(o_{1}^{**},o_{2}^{**},...,o_{N}^{**})$ then $(o_{1},o_{2},...,o_{N})$ $\geq_{E}$ $(o_{1}^{**},o_{2}^{**},...,o_{N}^{**})$.

“Noninteraction” property of factors. The non-interaction condition says that the decision maker’s interpretation of how one factor affects the outcome does not depend on how other factors affect the outcome.

### A.3 Learning algorithm for estimating ℙ ⁢ ( O i | f ) ℙ conditional subscript 𝑂 𝑖 𝑓 \mathbb{P}(O_{i}|f) blackboard_P ( italic_O start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | italic_f )

We present the complete algorithm in Appendix Algorithm 1. Since $\mathbb{P}(O_{1})$ and $\mathbb{P}(O_{2})$ are complementary, we only need to learn $\mathbb{P}(O_{1}|f_{ja}),f_{ja}\in\mathcal{F}_{j},j=1,...,N,a=1,...,{\rm card}
(\mathcal{F}_{j})$, and $\mathbb{P}(O_{2}|f_{ja})=1-\mathbb{P}(O_{1}|f_{ja})$. We randomly sample 128 instances in the space of complete information as the training data for each scenario. For a scenario with fewer than 128 instances in the space of complete information, we employ a resampling technique, selecting instances repeatedly until we reach a total of 128 samples.

Figure: Algorithm 1 Training single probabilities to estimate $\mathbb{P}(O_{i}|f)$ .

### A.4 Experiment Dataset Description

We convert three datasets focusing on reasoning and planning to our problem setting.

- •
Common2sense (Singh et al., ): focuses on commonsense reasoning and planning. The original task constitutes a multi-domain complementary commonsense reasoning benchmark and comprises natural language true/false statements. We exclusively focus on instances of comparative reasoning in both train and test sets where a pre-trained smaller model exhibits low confidence in its prediction logits. Using GPT-4(^8^88We use GPT-4-0613 when utilizing GPT-4 for annotations.), we rewrite each instance statement into its opposite, creating two distinct outcomes. We then outline a general scenario based on the two statements and generate 10 additional sentences for each statement, with detailed conditions that would support the statement. Additionally, we implement a reverse verification step to eliminate any incorrect additional sentences to ensure dataset quality. We gather 213 scenarios and 3822 instances. 9 additional conditions on average support each outcome.
- •
Today (Feng et al., ): focuses on temporal reasoning. The dataset is designed to study how introducing an extra sentence to a specific scenario affects the temporal relation, aligning well with the context of our task. There are 1000 instances.
- •
Plasma (Brahman et al., ): focuses on planning. The original task is to generate or revise a plan based on a given goal while adhering to an additional condition. We’ve observed that the key variations between the initial and the revised plans given the additional condition typically focus on a specific step within the plan. We then utilize GPT-4 to identify the step and adapt the description of the step in the original and revised plans into two potential outcomes. We manually identify the less common outcome in the 279 scenarios and use GPT-4 to generate 5 additional sentences for the outcome, with detailed conditions that would support this outcome instead of the other. We gather 279 scenarios and 1395 instances with 5 additional conditions for each scenario.

For the two GPT-4 generated datasets, we recruited the authors and their colleagues to conduct a human quality check on 100 randomly selected instances from each dataset, with three people reviewing each instance. All three reviewers agreed on the label for 91% of the generated data. For 94% of the data, at least two reviewers agreed on the label.

Figure: Figure 3: The interface for human evaluation on preference-based pairwise evaluation of the estimated probabilities.
Refer to caption: extracted/5930199/human_annotation.jpg

### A.5 Human evaluation on Bird estimated probabilities

We first explain our choice of relative evaluation over asking human annotators to provide absolute probabilities from a psychological perspective. TVERSKY & KAHNEMAN () shows that humans use heuristic principles to make probability judgments, which often suffer from systematic and predictable errors due to cognitive biases. Kahneman () further discusses how human judgment in probabilistic situations is generally better in relative than absolute terms due to cognitive biases and limitations in processing complex information. We conducted a pilot study with human annotators on MTurk, asking them to provide absolute probabilities for each additional condition separately. However, we were unable to reach a consensus among the 7 annotators.

We then introduce the human annotation process for pairwise comparison. We collect human gold labels with rigorous quality control for the preference-based pairwise evaluation of the estimated probabilities. We restrict each annotator to take 5 samples at most and there are 386 distinct annotators. We only allow participants who have 98% or more HITs acceptance rate and are located in the US. We also require annotators to spend at least 1 minute for each instance (the hourly
salary is  $15). We show the experiment interfaces in Appendix Fig. 3. We first carry out the human annotation process five times on 450 randomly sampled instances from Common2sense when the condition can be mapped to at least one factor and therefore Bird does not produce “unknown”. If the 5 annotations have a less than 80% agreement, i.e., less than 4 out of the 5 annotators agree on the same outcome label, we invite 2 additional annotators to reassess this instance. If the agreement of the 7 annotators is more than 70%, i.e., at least 5 out of the 7 annotators agree on the same outcome label, we accept the instance. Other than the previously accepted instances, we also accept an instance as a neutral label if all three labels have been chosen during the annotations and the agreement for the neutral label is more than or equal 40% and the agreement for any of the remaining two labels is less than or equal 40%. This leaves us with 366 instances. We then test the instances across all our methods, retaining only those where all methods can produce an accurate binary decision, i.e., both conditions support the gold outcome over the opposite outcome. We get 350 instances in total, where 12.3% of the instances have a neutral label where condition 1 and condition 2 equally support the outcome, 44.0% of the instances have a label towards condition 1 where condition 1 supports the corresponding outcome better than condition 2, 43.7% of the instances have a label towards condition 2 where condition 2 supports the corresponding outcome better than condition 1.

We now introduce how precision and recall are calculated. Given a testing instance, assuming the gold outcome is $O^{*}$ and the opposite outcome is $O^{\neg}$, there will be 3 model output labels: 1) $P(O^{*}|C1)>P(O^{*}|C2)$, i.e., the model thinks C1 better support the gold outcome; 2) $P(O^{*}|C2)>P(O^{*}|C1)$, i.e., the model thinks C2 better support the gold outcome; 3) $P(O^{*}|C2)=P(O^{*}|C1)$, i.e., the model thinks both $C_{1}$ and $C_{2}$ equally support the gold outcome. We want to check if the model’s label is the same as the human annotator’s label. Therefore, this can further be seen as a 3-label classification problem.

### A.6 Baseline Description

We adopt the following baselines to evaluate our Bird framework.

- •
Vanilla Verbalization with Self-Consistency (Vanilla). (Wang et al., ) Given the scenario, an additional condition, and two corresponding outcomes, the model is asked to directly verbalize an estimated probability for each outcome. We show the Prompt in Appendix Fig. 11.
The above process is carried out independently for each of the two additional conditions. The additional conditions with higher output probabilities for the gold outcome are the predicted answers. We use temperature = 0.7 to generate 3 responses and take the majority vote answer as the final answer.
- •
Logit-based Probability. We directly use the output token probability for the decision. We use greedy decoding here to set the temperature = 0.
- •
Chain-of-Thought Verbalization with Self-consistency (CoT). (Wei et al., ) We ask for a reasoning process to think step by step before verbalizing an estimated probability. The remaining process is the same as vanilla verbalization with SC. We show the Prompt in Appendix Fig. 12.
- •
Explicit Comparison (EC). Instead of evaluating each additional condition separately, both conditions are presented simultaneously to the model. Given the scenario and the gold outcome, the model is asked to select which additional condition better supports the outcome without outputting any probabilities. Note that the task is less challenging for this particular baseline than the others due to the explicit formatting of inference in this baseline. As a result, performances under the EC setting are not directly comparable with our model and other baselines’ performances since they only see one condition at a time and cannot perform argmax operations. We show the Prompt in Appendix Fig. 13.
- •
1/2 Assumption. Instead of utilizing the learning algorithm mentioned in §3.3 to estimate $\mathbb{P}(O_{i}|f)$. We adopt a neutral assumption. An outcome is determined based on the factors only if, except the values of the factors that are neutral, the values of all remaining factors uniformly support that particular outcome. Conversely, if the values of the factors indicate different outcomes and there is no prior information, we cannot conclusively determine the outcome.
$\mathbb{P}(O_{i}|f)=\left\{\begin{array}[]{lc}1&\mathbb{P}(O_{i}|f_{j})\geq
\frac{1}{2},\forall j\ {\rm except}\ \mathbb{P}(O_{i}|f_{j})=\frac{1}{2},j=1,.
..,N\\
\frac{1}{2}&{\rm otherwise}\\
\end{array}\right.$
(12)
The intuition behind this assumption is that as we focus solely on the present context, if the current information supports both outcomes and no additional information is available to finalize the decision, it becomes impossible to predict which outcome is more likely to occur.
- •
1/N Assumption. Instead of utilizing the learning algorithm mentioned in §3.3 to estimate $\mathbb{P}(O_{i}|f)$, we adopt another commonly used neutral assumption. The outcome $O_{i}$’s probability is determined by the aggregate count of factors whose values support outcome $O_{i}$.
$\mathbb{P}(O_{i}|f)=\frac{\sum_{j=1}^{N}\mathbbm{1}\{\mathbb{P}(O_{i}|f_{j})>
\frac{1}{2}\}}{N}$
(13)
- •
Fixed Initial Prob. Instead of training with the algorithm to get an estimated prob for each $\mathbb{P}(O_{i}|f_{j})$ in §3.3, we utilize the initial probability of Eq. 5 to directly estimate $\mathbb{P}(O_{i}|f)$ with Eq. 4.

### A.7 Analysis of Direct Decision Making

We first conduct error analysis for llama-2-70b-instruct on direct decision making where Bird gets the wrong outcomes while COT with SC makes the correct decisions. We detail the type of errors with its frequency percentage in Appendix Table 4. We observe that our proposed method outperforms the chain-of-thought baseline with Llama-3.1 rather than using llama-2. This is partially because Llama-3.1 generates more diverse and accurate factors, and we present an example in Appendix Table 5.

**Table 4: Error analysis for Llama-2-70b-instruct for direct decision making.**
| Error Type | Frequency Percentage |
| --- | --- |
| The generated factors and their values are not comprehensive enough to cover all elements of the conditions. The selected values only imply partial information in the additional condition. | 56.7% |
| The additional condition is mapped to incorrect values of the factors that it does not necessarily imply. | 30% |
| The calculation of the final probability is inaccurate. | 13.3% |

**Table 5: An example where LLama 3.1 generates better factors than Llama 2.**
| Scenario: the ease of carrying a cup is being compared between one person and six people. |
| --- |
| Outcome 1: It would be easier for one person to carry the cup, than it would be for six people to carry the cup. |
| Outcome 2: It would be easier for six people to carry the cup, than it would be for one person to carry the cup. |
| Llama-2-70b-Instruct Generated Factors |
| Weight and Size of the Cup: |
| The cup is light and small |
| The cup is heavy and large |
| The cup is of medium weight and size |
| Material of the Cup: |
| The cup is fragile |
| The cup is sturdy |
| The cup is made of moderately durable material |
| Strength of the People: |
| The person is able-bodied and strong enough to carry the cup |
| The combined strength of six people is greater than that of one person |
| Experience of the Person or People Involved: |
| The person has more experience in carrying objects |
| The six people have more experience in lifting objects |
| Formation of the People: |
| The six people carry the cup in a specific formation designed to distribute the weight more evenly |
| The six people do not carry the cup in a specific formation |
| Llama-3.1-70b-Instruct Generated Factors |
| Cup Size and Weight: |
| The cup is small and light |
| The cup is large and heavy |
| The cup is small and heavy |
| The cup is large and light |
| Cup Material: |
| The cup is fragile |
| The cup is sturdy |
| The cup is constructed from a material of moderate durability |
| Cup Design and Features: |
| The cup has a comfortable grip and balanced ergonomics |
| The cup has an uncomfortable grip and unbalanced ergonomics |
| The cup has a neutral grip and average ergonomics |
| Carrier’s Physical Abilities: |
| The one carrier has more strength and endurance than the six carries |
| The combined strength of the six carries is stronger than the one carrier |
| Group Dynamics and Coordination: |
| The group is well-coordinated and works together seamlessly |
| The group is disorganized and has difficulty working together |
| The group has average coordination and works together moderately well |
| Physical Demands of the Task: |
| The task requires exceptional strength and endurance |
| The task requires minimal strength and endurance |
| The task requires moderate strength and endurance |
| The Size and Shape of the Carrying Area: |
| The carrying area is narrow and confined |
| The carrying area is wide and open |

We then give insights into LLM direct performance. We present an example from the Common2sense dataset in Appendix Table 6. where the LLM accurately infers the correct outcome based predominantly on inductive biases, ignoring the other primary information provided in the condition, i.e., the peppers are used in large quantities, which will be captured in our framework.

**Table 6: An example where the LLM correctly makes the decision but with inadequate emphasis on the context.**
| Example |
| --- |
| Scenario: The spiciness of a hot sauce is being discussed in relation to the choice between using habanero |
| peppers or jalapeno peppers. |
| Condition: The hot sauce recipe uses a significant amount of habanero peppers relative to the quantity |
| of other ingredients. |
| Outcome 1: A hot sauce is going to be hotter if it uses habanero peppers rather than jalapeno. |
| Outcome 2: A hot sauce is going to be hotter if it uses jalapeno peppers rather than habanero. |
| Llama-2 Generated Deicision Making Process |
| Habanero peppers are generally much hotter than jalapeno peppers. This is measured on the Scoville |
| scale, which is a measure of the pungency (spiciness or heat) of chili peppers and other spicy foods. So |
| the condition better supports Outcome 1: A hot sauce is going to be hotter if it uses habanero peppers |
| rather than jalapeno. |

### A.8 Ablation Study towards Abductive Factor Generation

We compare the final decision-making accuracy based on two different factor generation methods and present the results in Appendix Table 7. We see that our proposed method performs much better than the direct generation method, indicating the effectiveness of utilizing the model’s inductive bias inherited from training data.

**Table 7: An ablation study towards different factor generation methods. Direct refers to the direct factor generation method. Abduction refers to our abductive factor generation.**
| Dataset | Method | $\mbox{{Bird}}{}_{llama2}$ | %Unknown | $\mbox{{Bird}}{}_{llama3.1}$ | %Unknown |
| --- | --- | --- | --- | --- | --- |
| Today | Direct | 70.9 | 74.1 | 68.9 | 46.3 |
|  | Abduction | 73.9 | 54.0 | 74.3 | 34.0 |
| Plasma | Direct | 68.6 | 40.7 | 69.0 | 24.1 |
|  | Abduction | 74.0 | 31.4 | 73.0 | 19.0 |
| C2S | Direct | 84.2 | 54.9 | 89.9 | 29.0 |
|  | Abduction | 89.0 | 34.6 | 92.3 | 27.1 |

### A.9 Yes-or-no followup Question Generation

We experiment to see if Bird can further facilitate decision making in a controllable way, where we use Bird to generate a yes-or-no follow-up question about an unobserved factor in the context. Specifically, given a scenario and an additional condition, we first identify which factors and their corresponding values are implied by the condition, we then select a factor that is not mentioned and is most important for the decision in the remaining unmentioned factors, as the target question area. We adopt the factor value that better aligns with the outcome direction suggested by the current implied factor values and has the largest condition probability. We then rewrite the statement of the factor value into its corresponding question. Simultaneously, we ask llama2-70b-instruct to directly generate a yes-or-no follow-up question. The prompt is in Appendix Fig. 15.

We use MTurk and ask annotators to provide pairwise preference between the Bird generated question and LLM directly generated question. We adopt the same Mturk setting as mentioned in §A.5. we carry out the human annotation process three times, using the majority vote to find the preferred generated question. We show the experiment interfaces and an example in Appendix Fig. 4. Appendix Table 8 shows that the follow-up questions generated by Bird are more often preferred.

Figure: Figure 4: The interface for human evaluation on preference-based pairwise evaluation of the generated questions. Question 1 is generated by Bird and question 2 is generated by LLM directly. The three annotators all prefer question 1.
Refer to caption: extracted/5930199/human_annotation_1.jpg

**Table 8: Preference judgments for yes-or-no follow-up questions generated by Bird versus questions directly generated by LLMs. We randomly sample 250 instances from Common2sense.**
| Dataset | Bird Win Direct | Tie | Bird Lose Direct |
| --- | --- | --- | --- |
| Common2sense | 132 | 36 | 82 |

### A.10 Prompt

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: In Section 4.3, "The Usage of the Reliably Estimated Probability", it seems that the original T5 large model is better on some datasets compared to the fine-tuned models. What would be the reasons for this? Besides, what is "PatternTime" in Table 3? It is not mentioned anywhere else in the manuscript.

Please see "Weaknesses" for a few additional questions.
soundness: 2
strengths: 1. The proposed Bayesian inference framework is conceptually interesting. Factoring the overall decision-making into multiple fine-grained factors is intuitive and useful for mitigating LLMs' limitations in providing accurate fine-grained probability/confidence estimation.

2. The experiments performed are carefully designed, and the overall writing is clear and with enough details.

3. The experiment results demonstrate the effectiveness of BIRD, showing its value in multiple application scenarios.
summary: This work proposes an inference framework that aims to produce more reliable and accurate probability/confidence estimations using LLMs for large-scale decision-making and planning tasks. The proposed method, BIRD, uses abductive factor generation to factorize the inference task, and estimates the outcome by accumulating the predictions made on the specific factors. The experimental results on three datasets show that BIRD is able to produce more accurate probability estimations in decision-making and planning, which can be used to (1) improve the accuracy of decision-making; (2) supervise the training of small models; (3) propose follow-up questions for decision-making. For all these applications, BIRD shows superior performance than the baseline methods.
weaknesses: 1. The training setting for "Learning algorithm for constrained optimization to estimate $P(O_i|f)$" (Line 266) should be more clearly described. The current description does not clarify (1) the number of learnable parameters ($P(O_i|f_j)$); (2) whether different instances share the same factors $f_j$; (3) whether there are $f_j$ in the test data that are unseen in the training data.

2. In the experiment for "Applying BIRD’s Probabilities in Decision-Making" (Line 430), the test instances on which BIRD's decision is unknown are filtered out. While it is claimed that "this does not undermine our experiment setting because such removal is label-agnostic" (Line 450), it may still introduce biases since this filtering process depends on the output of BIRD. It might be the case that BIRD predicts "unknown" when it is uncertain internally.

---

confidence: 3
contribution: 4
initial_rating: 8
presentation: 4
questions: There several magic numbers in 3.3 and 3.4 (mappings of rankings to probabilities), as well as key design decisions in model formulation. Can you comment on sensitivity of the method to these values/decisions?

Will the evaluation data - and the code - be released?

372, 449: What instances lead to 'unknown' predictions for all factors? Is this a shortcoming with the factor generation method, or is it that the problem itself is riddled with uncertainty.

452: Can you elaborate on the spurious biases that affect CoT, giving it such strong performance? The same criticism could be applied to the LLM prompting as part of Bird, and the probabilities produced.
soundness: 3
strengths: This paper presented a very novel idea, a detailed method for achieving this, and a convincing evaluation of the utility of the technique. Specific strengths include:
 - mechanism for prompting a LLM to extract factors & potential outcomes of a complex decision problem
 - means to convert the factors into a binary Bayes network
 - incorporation of model uncertainty estimates into the parameterisation of the Bayes network
 - evaluation against human judgements of probability
 - strong baselines
summary: This paper presents a method allowing LLMs to reason more soundly about complex problems with uncertainty, by combining LLM introspection and verbal estimates of uncertainty into Bayesian reasoning about outcomes. In many ways it resembles chain-of-thought, in that the LLM is being asked to provide high level reasoning about an uncertain scenario, and is required to brainstorm to solve the problem. The incorporation of a Bayes net sets the method apart from prior work, and experiments show that this leads to much better estimates in pairwise evaluation of propositions, but less compelling evidence for the work when evaluated on decision making benchmarks.
weaknesses: The paper was quite dense to start, and took several pages to make the application scenario clear. On my first pass it wasn't apparent to me what elements of the data were part of the problem definition, what parts were computed by the LLM, and what was computed by inference in a Bayes net. Fig 1 is reasonably easy to follow, but Fig 2 is less clear - why have the conditions changed between Fig 1 and Fig2, and what parts of the central box are given as part of the data versus inferred from LLM interactions? As I understand it, the whole of the inner box is auto-generated, as well as the edges linking the given conditions with coarse probability estimates.  

The paper also takes a long and detailed dive into Bayes net formality, which is more than needed given the setting is relatively simple and could be communicated effectively with more of a focus on the running example (Fig 2 could be unpacked). As it stands, the paper takes several pages to get going, but ends up deferring many critical details to the Appendix. The method for prompting the LLM is of fundamental importance to the technique, for instance, but none of this is in the main paper.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: 1) Page 1: LLMs’s → LLMs’
2) The LaTeX expression for P in the introduction differs from its expression in the later sections.
3) Where is the example of the factor, "The color of the vehicle," mentioned by the authors in Line 228, depicted in Figure 2? I couldn't find it in the figure as described in the text.
4) Appendix A.2: Week →Weak
5) This may be a minor issue, but there could be confusion between Line 318 'S + U₃ implies d₁ and f₂' and f, which is a possible instance of complete information due to the notation.
6) An additional question is, have the authors considered any experiments involving more than two outcomes (i.e., beyond binary classification)?
soundness: 4
strengths: 1) The proposed framework and optimization settings, which account for weak ordering and non-interaction in the set of factors, are valid.
2) The authors' assumptions about enhancing the validity of the proposed Bayesian Inference Framework are reasonable.
3) Including unobserved factors in the calculations strengthens the effectiveness and utility of the authors' proposed method.
4) Besides precise probability estimation, the presence of follow-up generation and cross-domain experiments demonstrates the proposed method's reliability and practicality.
summary: This paper proposes a BIRD framework that enables more reliable probability estimation based on incomplete instances from LLMs. It leverages factors generated by LLMs to make a Bayesian network and estimates more precise probabilities. BIRD with optimized prob presents a better performance in several experiments than existing methods. BIRD also demonstrates improved reasoning performance across various datasets, particularly excelling in more complex scenarios.
weaknesses: 1) In Line 147 and Line 155, the explanation of the outcomes is separated. It seems possible to consolidate these explanations into one part. Additionally, the explanation of 𝐹 in the same paragraph could be made more concise. The space saved from these revisions could then be used to more clearly elaborate on the connection between Equation (5) and Equation (6) and the content in Algorithm 1.

2) Ablation settings could be expanded to cover more aspects. For example, experiments could be conducted by excluding the MR loss from the unified loss function or by assuming that in an unobserved factor, each value does not have an equal probability of being selected.

Meta-review: The paper introduces BIRD, a Bayesian inference framework that combines LLMs with structured Bayesian networks to produce more reliable probability estimations. The method shows significant improvements in decision-making accuracy compared to LLM baselines and demonstrates applications in model supervision and follow-up question generation.

*Strengths*  
Novel framework: BIRD uniquely combines LLM-based factor generation with Bayesian networks, addressing limitations in probability estimation.  
Methodology: The framework effectively decomposes complex tasks into interpretable components while incorporating model uncertainty.
Empirical Validation: Thorough evaluation against strong baselines demonstrates consistent improvements across multiple datasets.

*Weaknesses*  
-Some critical implementation details are deferred to appendices, such as LLM prompting methods.  
-The framework produces "unknown" predictions for some instances (~2%) due to limitations in factor generation.  
-While preliminary ablations show robustness, a more comprehensive analysis of parameter choices would strengthen the work.  

All reviewers recommend acceptance, and recognize the paper's novel approach and strong empirical results. The authors have addressed concerns about technical details and unknown predictions in their responses.

Decision: Accept (Oral)