The human judge, unable to review full financial records, can evaluate this specific claim. Through repeated debates on many applications, agents learn that deceptive arguments (exploiting data cherry-picking) get caught by adversaries, incentivizing honest, comprehensive analysis. This scales human oversight beyond direct review capacity.

# Exercises

## 1. Gridworld Safety:

(a) Design a Gridworld environment with “lava” states that terminate the episode with a large negative reward.

(b) Train a standard Q-learning agent. Observe how many times it falls into the lava during training.

(c) Implement a “Safety Layer” that masks out actions leading to lava (assuming the dynamics are known). Compare the training safety.

## 2. Lagrangian Relaxation:

(a) Implement a PPO agent for the CartPole environment.

(b) Add a constraint: The cart position must not exceed $x = 1.0$ (standard limit is 2.4).

(c) Use the Lagrangian method to enforce this constraint. Plot the reward and the constraint violation over time.

## 3. RLHF Toy Example:

(a) Define a ground truth reward function (e.g., $R(x) = -|x - 5|$).

(b) Generate “human preferences” based on this function (with some noise).

(c) Train a reward model (neural network) to predict these preferences.

(d) Optimize an agent using the learned reward model and compare the result to the ground truth optimum.

## 4. Control Barrier Functions:

(a) Design a simple 2D navigation environment with a circular obstacle.

(b) Define a CBF $h(s) = \|s - s_{obs}\|^2 - r^2$ where $s_{obs}$ is the obstacle center and $r$ is the safety radius.

(c) Implement a safety layer that solves the QP at each step to ensure $\dot{h}(s) \ge -\alpha h(s)$.

(d) Compare trajectories with and without the safety layer. Visualize the invariant set.

## 5. Constrained Policy Optimization:

(a) Implement CPO for the Safety Gym benchmark (e.g., PointGoal with hazards).

(b) Compare convergence and constraint satisfaction against vanilla PPO and Lagrangian PPO.

(c) Plot the Pareto frontier of reward vs. cost for different constraint thresholds.

(d) Analyze how the trust region size affects both reward and constraint violation.

## 6. Reward Modeling:

(a) Collect synthetic preference data by comparing trajectories based on multiple criteria: (1) distance to goal, (2) smoothness of path, (3) energy consumption.
(b) Train reward models with different architectures: MLP, LSTM, Transformer.

(c) Evaluate reward model accuracy on held-out preferences.

(d) Investigate the impact of label noise (disagreeing annotators) on reward model quality.

## 7. Safe Exploration Algorithms:

(a) Implement Safe Model-Based Policy Optimization (SMBPO) using an ensemble of dynamics models.

(b) Compare exploration efficiency and safety violations against TRPO-Lagrangian.

(c) Analyze how pessimism in the uncertainty estimation affects exploration.

(d) Test on continuous control tasks with safety constraints (e.g., joint limits, velocity bounds).

## 8. DPO Implementation:

(a) Implement Direct Preference Optimization for a small language model.

(b) Compare training dynamics with RLHF using PPO.

(c) Measure the alignment tax: how much task performance degrades after alignment.

(d) Investigate the effect of the KL coefficient $\beta$ on alignment strength and diversity.
# 11 Out-of-Distribution Detection and Uncertainty Quantification

A trustworthy AI system must recognize when it is operating outside its domain of competence. Out-of-Distribution (OOD) detection and uncertainty quantification provide mechanisms for AI systems to express confidence calibrated to their actual reliability, enabling appropriate human intervention when the system encounters unfamiliar inputs.

## 11.1 The Problem of Distribution Shift

Machine learning models are trained on data from a distribution $P_{train}(X, Y)$. At deployment, the model encounters data from a potentially different distribution $P_{test}(X, Y)$. When these distributions differ, model performance can degrade significantly, often without any indication to the user.

**Definition 11.1** (Types of Distribution Shift).

* **Covariate Shift:** $P_{test}(X) \neq P_{train}(X)$ but $P_{test}(Y|X) = P_{train}(Y|X)$. The input distribution changes, but the input-output relationship is preserved.

**Detailed Explanation:** In covariate shift, the fundamental relationship between features and labels remains unchanged—what changes is the relative frequency of different types of inputs. Mathematically, the conditional probability $P(Y|X)$ is invariant, meaning if we observe the same feature vector $x$ in both training and test sets, the probability distribution over labels remains identical.

**Concrete Example:** Consider a spam detection model trained on emails from 2020, where 80% were legitimate work emails and 20% were spam. At deployment in 2023, promotional emails became more common, changing the distribution to 60% work emails, 30% promotional, and 10% spam. The features that make an email spam (suspicious links, urgency language, misspellings) haven't changed—$P(Y = spam|X)$ remains the same—but the prevalence of different email types has shifted. Importance weighting can correct for this: we reweight training examples by $w(x) = P_{test}(X)/P_{train}(X)$, giving more weight to examples similar to test distribution. This is solvable through density ratio estimation or by training a classifier to distinguish training from test data and using predicted probabilities as weights.

* **Label Shift:** $P_{test}(Y) \neq P_{train}(Y)$ but $P_{test}(X|Y) = P_{train}(X|Y)$. The class proportions change, but the class-conditional distributions are preserved.

**Detailed Explanation:** Label shift occurs when the prevalence of different classes changes between training and deployment, but the characteristics of each class remain consistent. The class-conditional densities $P(X|Y = y)$ are identical in both domains, meaning examples labeled $y$ “look the same” statistically.

**Concrete Example:** A medical diagnostic system trained on hospital data where 5% of patients had a rare disease is deployed in a specialized clinic where 30% of patients have the disease (due to referral bias—the clinic specializes in that condition). The symptoms and test results associated with the disease ($P(X|Y = \text{disease})$) haven't changed, but the base rate has increased dramatically. The confusion matrix method can estimate the new class proportions: using the classifier's predicted labels on test data and assuming the confusion matrix (calibrated on validation set) remains valid, we solve for $P_{test}(Y)$ via $P_{test}(\hat{Y}) = C \cdot P_{test}(Y)$ where $C$ is the confusion matrix. The solution $P_{test}(Y) = C^{-1}P_{test}(\hat{Y})$ allows recalibration of predictions.

* **Concept Drift:** $P_{test}(Y|X) \neq P_{train}(Y|X)$. The relationship between inputs and outputs changes.
*Detailed Explanation:* Concept drift represents a fundamental change in the underlying data-generating process—the same input features now map to different labels, or the decision boundary has shifted. This is the most challenging type of shift because the training data becomes misleading: past patterns no longer apply.

*Concrete Example:* A stock trading model learned in 2019 that high volatility ($X_1 > \theta$) predicts market decline ($Y = \text{down}$). During the COVID-19 pandemic in 2020, the relationship reversed: high volatility often preceded rapid recovery rallies as governments announced stimulus packages. The feature $X_1$ (volatility) remains measurable, but $P(Y = \text{up}|X_1 > \theta)$ increased from 0.2 to 0.7. Another example: a spam filter trained pre-2020 might have learned that COVID-related emails are spam, but during the pandemic, legitimate organizational emails frequently discussed COVID protocols. No reweighting can fix this—the model must be retrained with new labeled data. Concept drift detection monitors statistics like prediction accuracy over sliding time windows; when performance drops significantly, it triggers model retraining or human review.

* **Domain Shift:** Both the marginal and conditional distributions change simultaneously: $P_{test}(X) \neq P_{train}(X)$ and $P_{test}(Y|X) \neq P_{train}(Y|X)$.

*Detailed Explanation:* Domain shift combines multiple types of distribution changes, making it the most severe and general form of shift. Neither the input statistics nor the predictive relationships are preserved.

*Concrete Example:* An autonomous vehicle vision system trained on sunny California highways (clear weather, well-marked lanes, specific road signs) is deployed in rural India during monsoon season (heavy rain reducing visibility, faded lane markings, different sign conventions, pedestrians and livestock on roads). Both $P(X)$ changes (different visual statistics: rain, fog, different infrastructure) and $P(Y|X)$ changes (a particular visual pattern might indicate “safe to proceed” in California but “hazard” in India). Domain adaptation techniques become necessary: unsupervised domain adaptation aligns source and target feature distributions using adversarial training ($\min_f \max_D E_{\text{source}}[\log D(f(x))] + E_{\text{target}}[\log(1 - D(f(x)))]$), self-training uses high-confidence predictions on target domain as pseudo-labels, or multi-task learning jointly trains on both domains with shared representations.

## 11.2 Approaches to OOD Detection

### 11.2.1 Softmax-Based Methods

The simplest approach uses the maximum softmax probability (MSP) as a confidence score [17]. For a classifier $f$ with softmax output, the OOD score is:

$$S_{MSP}(x) = 1 - \max_{c} P(Y = c | X = x) = 1 - \max_{c} \text{softmax}(f(x))_{c} \quad (205)$$

where $f(x)$ are the logits and $c$ indexes the classes.

However, deep networks are known to produce overconfident predictions even on OOD data. The softmax function normalizes logits to sum to 1, which can produce high confidence regardless of input proximity to training data.

**Temperature Scaling** adjusts the softmax temperature to improve calibration:

$$P(Y = c | X = x; T) = \frac{\exp(z_c/T)}{\sum_{c'} \exp(z_{c'}/T)} \quad (206)$$

where $z_c$ are the logits and $T$ is the temperature parameter.
### 11.2.2 ODIN: Out-of-Distribution Detector for Neural Networks

ODIN [22] combines temperature scaling with input preprocessing:

$$ \tilde{x} = x - \epsilon \cdot \mathrm{sign}(\nabla_x \log \max_c P(Y = c | X = x; T)) \quad (207) $$

where $\epsilon$ is the perturbation magnitude and the gradient maximizes the softmax probability.

The perturbation $\tilde{x}$ is designed to increase the softmax score for in-distribution data while having less effect on OOD data. The gradient points toward the decision boundary, pushing in-distribution samples further into high-confidence regions.

### 11.2.3 Energy-Based OOD Detection

The energy function [23] provides a theoretically grounded alternative to softmax probability:

$$ E(x) = -T \cdot \log \sum_c \exp(f_c(x)/T) \quad (208) $$

where $f_c(x)$ is the logit for class c.

In-distribution samples have lower energy (higher density under the model), while OOD samples have higher energy. This approach is connected to the log-partition function in energy-based models.

**Theorem 11.1 (Energy and Density).** For a classifier trained with cross-entropy loss, the energy function approximates the negative log-likelihood under a joint energy-based model:

$$ E(x) \approx -\log p(x) + \mathrm{const} \quad (209) $$

where $p(x)$ is the data density.

*Proof Sketch.* Consider the energy-based formulation where the joint distribution is:

$$ p(x, y) = \frac{\exp(f_y(x)/T)}{Z} \quad (210) $$

where $Z$ is the partition function. The marginal density of $x$ is:

$$ p(x) = \sum_y p(x, y) = \frac{1}{Z} \sum_y \exp(f_y(x)/T) = \frac{1}{Z} \exp\left(\frac{-E(x)}{T}\right) \quad (211) $$

where the summation marginalizes over all classes $y$. where $E(x) = -T \log \sum_y \exp(f_y(x)/T)$ is the free energy. Taking the log:

$$ \log p(x) = -\frac{E(x)}{T} - \log Z \implies E(x) = -T \log p(x) + \mathrm{const} \quad (212) $$

where the relationship links energy to negative log-likelihood. Thus, the energy function is proportional to the negative log-density. During training with cross-entropy, the model implicitly learns to assign lower energy to in-distribution samples, making energy a natural OOD score. ☐
### 11.2.4 Deep Ensembles

Training multiple models with different random initializations provides a measure of model uncertainty:

$$ \mathrm{Var}[\hat{y}] = \frac{1}{M} \sum_{m=1}^{M} (f_m(x) - \bar{f}(x))^2 \qquad (213) $$

where M is the number of models and $f_m$ is the prediction of the m-th model.

High disagreement among ensemble members indicates uncertainty, which is often elevated for OOD inputs. Deep ensembles capture *epistemic uncertainty* arising from model uncertainty, not just *aleatoric uncertainty* from inherent data noise.

### 11.2.5 Mahalanobis Distance-Based Detection

Fit a class-conditional Gaussian to the penultimate layer features:

$$ \mu_c = \frac{1}{|D_c|} \sum_{x \in D_c} h(x), \quad \Sigma = \frac{1}{N} \sum_c \sum_{x \in D_c} (h(x) - \mu_c)(h(x) - \mu_c)^T \qquad (214) $$

where $h(x)$ are the features, $D_c$ is the set of samples in class c, and $\Sigma$ is the tied covariance matrix.

The Mahalanobis distance to the nearest class centroid serves as OOD score:

$$ M(x) = \min_{c} (h(x) - \mu_{c})^{T} \Sigma^{-1} (h(x) - \mu_{c}) \qquad (215) $$

where $M(x)$ is the minimum distance to any class centroid.

## 11.3 Bayesian Approaches to Uncertainty

### 11.3.1 Bayesian Neural Networks

Instead of point estimates $\theta^*$, Bayesian neural networks maintain a posterior distribution $P(\theta|D)$. Predictions are made by marginalizing over the posterior:

$$ P(y|x, D) = \int P(y|x, \theta)P(\theta|D)d\theta \qquad (216) $$

where $P(\theta|D)$ is the posterior distribution over parameters. Since exact inference is intractable, various approximations are used. **Variational Inference** approximates $P(\theta|D)$ with a tractable distribution $q_{\phi}(\theta)$ by minimizing KL divergence. **Monte Carlo Dropout** uses dropout at test time to sample from an approximate posterior. **Laplace Approximation** approximates the posterior with a Gaussian centered at the MAP estimate.

### 11.3.2 Monte Carlo Dropout

Dropout at test time provides an approximation to Bayesian inference. For T forward passes with dropout:

$$ \hat{y}(x) = \frac{1}{T} \sum_{t=1}^{T} f(x; \theta_t) \qquad (217) $$
where $T$ is the number of forward passes and $\theta_t$ are the dropout-masked parameters.

$$ \mathrm{Var}[\hat{y}] \approx \frac{1}{T} \sum_{t=1}^{T} f(x; \theta_t)^2 - \left( \frac{1}{T} \sum_{t=1}^{T} f(x; \theta_t) \right)^2 \qquad (218) $$

where the variance estimates the predictive uncertainty.

**Theorem 11.2 (Gal & Ghahramani).** Training with dropout and $L_2$ regularization is mathematically equivalent to variational inference in a Bayesian neural network with specific prior and variational family.

*Proof Sketch.* Consider a neural network with dropout applied to each layer with dropout rate $p$. Let $\mathbf{z}_i \in \{0, 1\}^{d_i}$ be binary dropout masks for layer $i$. The variational distribution is:

$$ q(\mathbf{w}) = \prod_i q(\mathbf{w}_i | \mathbf{z}_i) = \prod_i p^{d_i} \delta_{\mathbf{w}_i \mathrm{diag}(\mathbf{z}_i)}(\mathbf{w}_i) \qquad (219) $$

where $\mathbf{z}_i$ are the dropout masks and $\delta$ is the Dirac delta function.

$$ \mathcal{L} = - \sum_{(x,y)} \mathbb{E}_{q(\mathbf{w})}[\log p(y|x, \mathbf{w})] + D_{KL}(\|q)(\mathbf{w})\|p(\mathbf{w})) \qquad (220) $$

where the first term is the expected log-likelihood and the second is the KL divergence to the prior. With a Gaussian prior $p(\mathbf{w}) \propto \exp(-\|\mathbf{w}\|^2/(2\tau^2))$ and approximating the expectation with dropout sampling, the objective becomes:

$$ \mathcal{L} \approx \sum_{(x,y)} \ell(y, f(x; \mathbf{W} \odot \mathbf{z})) + \frac{\lambda}{2} \|\mathbf{W}\|^2 \qquad (221) $$

where $\ell$ is the loss function and $\lambda$ is the weight decay coefficient. $\square$

### 11.3.3 Decomposing Uncertainty

Total predictive uncertainty can be decomposed into:

$$ \underbrace{\mathrm{Var}[y|x]}_{\text{Total}} = \underbrace{\mathbb{E}[\mathrm{Var}[y|x, \theta]]}_{\text{Aleatoric}} + \underbrace{\mathrm{Var}[\mathbb{E}[y|x, \theta]]}_{\text{Epistemic}} \qquad (222) $$

where the total variance is split into data uncertainty (aleatoric) and model uncertainty (epistemic). **Aleatoric uncertainty** refers to inherent noise in the data that cannot be reduced with more data, while **Epistemic uncertainty** represents model uncertainty that can be reduced with more data or better models. For OOD detection, epistemic uncertainty is most relevant since OOD inputs should have high model uncertainty.

## 11.4 Calibration

**Definition 11.2 (Calibration).** A probabilistic classifier is **calibrated** if among all predictions with confidence $p$, the fraction of correct predictions is $p$:

$$ P(Y = \hat{Y} | P(\hat{Y}) = p) = p \quad \forall p \in [0, 1] \qquad (223) $$

where $\hat{Y}$ is the predicted class and $p$ is the confidence.
### 11.4.1 Expected Calibration Error (ECE)

ECE measures the average gap between confidence and accuracy:

$$ \text{ECE} = \sum_{b=1}^{B} \frac{|B_b|}{N} |\text{acc}(B_b) - \text{conf}(B_b)| \qquad (224) $$

where B is the number of bins, $B_b$ is the set of samples in bin b, and N is the total number of samples.

### 11.4.2 Reliability Diagrams

A reliability diagram plots accuracy vs. confidence for each bin. A perfectly calibrated model lies on the diagonal $y = x$.

### 11.4.3 Post-hoc Calibration Methods

**Temperature Scaling:** Learn a single temperature T to minimize negative log-likelihood on a validation set:

$$ T^* = \arg\min_{T} - \sum_{(x,y) \in D_{\text{val}}} \log P(Y = y | X = x; T) \qquad (225) $$

where $D_{\text{val}}$ is the validation set.

**Platt Scaling:** Fit a logistic regression on the logits:

$$ P(Y = 1 | z) = \sigma(az + b) \qquad (226) $$

where z is the logit and a, b are learned scaling parameters.

**Isotonic Regression:** Fit a non-parametric isotonic (monotonically increasing) function mapping predicted probabilities to calibrated probabilities.

## 11.5 Conformal Prediction

Conformal prediction provides distribution-free, finite-sample valid prediction sets. Given a desired coverage level $1 - \alpha$, conformal prediction outputs a set $C(x)$ that contains the true label with probability at least $1 - \alpha$.

**Definition 11.3 (Conformal Prediction Set).** Given calibration data $\{(x_i, y_i)\}_{i=1}^n$ and a conformity score function $s(x, y)$:

1. Compute scores $s_i = s(x_i, y_i)$ for calibration data

2. Find threshold $\hat{q} = \text{Quantile}_{(n+1)(1-\alpha)/n}(\{s_1, \dots, s_n\})$

3. For new input $x$, output $C(x) = \{y : s(x, y) \le \hat{q}\}$

**Theorem 11.3 (Coverage Guarantee).** If $(x_1, y_1), \dots, (x_n, y_n), (x_{n+1}, y_{n+1})$ are exchangeable, then:

$$ P(y_{n+1} \in C(x_{n+1})) \ge 1 - \alpha \qquad (227) $$

where $C(x_{n+1})$ is the prediction set and $1 - \alpha$ is the target coverage.
Proof. Define the conformity scores $s_i = s(x_i, y_i)$ for $i = 1, \dots, n + 1$. By exchangeability, all permutations of $(s_1, \dots, s_{n+1})$ are equally likely. The quantile threshold $\hat{q}$ is defined as the $\lceil (n+1)(1-\alpha) \rceil$-th smallest value in the augmented set $\{s_1, \dots, s_n, \infty\}$.

$$ \hat{q} = \text{Smallest}_{\lceil (n+1)(1-\alpha) \rceil} (\{s_1, \dots, s_n\} \cup \{\infty\}) \qquad (228) $$

where $\hat{q}$ is the conformal threshold. The prediction set is $C(x_{n+1}) = \{y : s(x_{n+1}, y) \le \hat{q}\}$. We include $y_{n+1}$ if and only if $s_{n+1} \le \hat{q}$. Consider the rank $R$ of $s_{n+1}$ among $\{s_1, \dots, s_{n+1}\}$ (ties broken randomly). By exchangeability:

$$ P(R = k) = \frac{1}{n+1} \quad \forall k \in \{1, \dots, n+1\} \qquad (229) $$

where $R$ is the rank of the test score among calibration scores. We have $s_{n+1} \le \hat{q}$ if and only if $R \le \lceil (n+1)(1-\alpha) \rceil$. Therefore:

$$ P(y_{n+1} \in C(x_{n+1})) = P(R \le \lceil (n+1)(1-\alpha) \rceil) = \frac{\lceil (n+1)(1-\alpha) \rceil}{n+1} \ge 1-\alpha \qquad (230) $$

where the probability is over the randomness of the calibration and test data. This proves the coverage guarantee holds for any exchangeable sequence, without any assumptions on the model or data distribution. □

The key insight is that conformal prediction makes no distributional assumptions beyond exchangeability, providing valid coverage even when the model is misspecified.

## 11.6 Selective Prediction (Rejection)

In selective prediction, the model can abstain from making a prediction when uncertain:

$$ g(x) = \begin{cases} f(x) & \text{if } \kappa(x) \ge \tau \\ \perp (\text{abstain}) & \text{otherwise} \end{cases} \qquad (231) $$

where $\kappa(x)$ is the confidence function and $\tau$ is the rejection threshold.

**Risk-Coverage Trade-off:** Let the coverage be $\phi(\tau) = P(\kappa(x) \ge \tau)$ and the selective risk be:

$$ R(\tau) = \frac{\mathbb{E}[\ell(f(x), y) \cdot \mathbb{I}(\kappa(x) \ge \tau)]}{\phi(\tau)} \qquad (232) $$

where $\phi(\tau)$ is the coverage and $\ell$ is the loss function.

The risk-coverage curve shows how accuracy improves as coverage decreases (the model abstains on more inputs).

## Exercises

### 1. OOD Detection Benchmark:

(a) Train a classifier on CIFAR-10 (in-distribution).

(b) Evaluate OOD detection using SVHN and CIFAR-100 as OOD datasets.

(c) Compare MSP, ODIN, Energy, and Mahalanobis methods using AUROC.

### 2. Calibration:
(a) Train a ResNet on CIFAR-10 and plot its reliability diagram.

(b) Apply temperature scaling and re-plot. Calculate ECE before and after.

(c) Investigate how calibration changes under distribution shift.

### 3. Conformal Prediction:

(a) Implement conformal prediction for a multi-class classifier.

(b) Verify the coverage guarantee empirically by measuring coverage on held-out data.

(c) Plot the average prediction set size vs. coverage level $1 - \alpha$.

### 4. Bayesian Neural Networks:

(a) Implement Monte Carlo dropout and compare uncertainty estimates with a deep ensemble.

(b) Visualize the predictive uncertainty on a 2D toy classification problem.

(c) Show that epistemic uncertainty is high in regions far from training data.

### 5. Selective Prediction:

(a) Implement a selective classifier that abstains when confidence is below a threshold.

(b) Plot the risk-coverage curve for different threshold values.

(c) Compare the area under the risk-coverage curve (AURC) for different confidence functions: MSP, entropy, MC dropout variance.

(d) Implement the SelectiveNet architecture that jointly learns prediction and rejection functions.

### 6. Energy-Based OOD Detection:

(a) Implement the energy score $E(x) = -T \log \sum_c \exp(f_c(x)/T)$.

(b) Compare with MSP on various OOD benchmarks (CIFAR-10 vs SVHN, CIFAR-100, Textures).

(c) Visualize the energy distribution for in-distribution vs OOD samples.

(d) Implement energy-based fine-tuning to improve OOD detection.

### 7. Mahalanobis Distance Detection:

(a) Extract features from multiple layers of a pre-trained network.

(b) Fit class-conditional Gaussians to the training features.

(c) Implement the Mahalanobis distance score with input preprocessing.

(d) Compare single-layer vs multi-layer ensemble for OOD detection.

### 8. Advanced Conformal Prediction:

(a) Implement Adaptive Conformal Inference (ACI) for handling distribution shift.

(b) Compare split conformal, cross-conformal, and full conformal prediction.

(c) Implement conformalized quantile regression for uncertainty estimation.

(d) Analyze the trade-off between average set size and coverage under covariate shift.

### 9. Uncertainty Decomposition:

(a) Implement both aleatoric and epistemic uncertainty estimation using a heteroscedastic neural network.
(b) Train a model to output both mean and variance predictions.

(c) Visualize the two uncertainty types on a regression problem with varying noise levels.

(d) Show that epistemic uncertainty decreases with more training data while aleatoric remains constant.
# 12 Engineering and Governance

## 12.1 Formal Verification of Neural Networks

As neural networks are deployed in safety-critical applications (autonomous vehicles, medical devices, aircraft control), there is a growing need for mathematical proofs of their properties. Formal verification aims to provide such proofs, guaranteeing that a neural network satisfies a specification for all inputs in some domain.

### 12.1.1 The Verification Problem

**Definition 12.1** (Neural Network Verification). Given a neural network $f: \mathbb{R}^d \to \mathbb{R}^k$, an input region $C \subseteq \mathbb{R}^d$, and a property specification $\phi: \mathbb{R}^k \to \{\text{True, False}\}$, the verification problem asks:

$$ \forall x \in C : \phi(f(x)) = \text{True} \qquad (233) $$

where $C$ is the input region and $\phi$ is the property specification.

Common verification properties include **Local Robustness** ($\forall x' : \|x' - x\|_p \le \epsilon \implies f(x') = f(x)$), **Output Bounds** ($\forall x \in C : f(x) \in [l, u]$), and **Safety Constraints** ($\forall x \in C : f(x) \in S$ for safe output set $S$).

### 12.1.2 Complexity of Verification

**Theorem 12.1** ([19]). *The problem of verifying properties of neural networks with ReLU activations is NP-complete.*

This means that exact verification is computationally intractable for large networks in the worst case. Practical approaches either provide sound but incomplete verification (may fail to prove true properties) or use approximations that trade tightness for scalability.

### 12.1.3 Bound Propagation Methods

The key idea is to propagate bounds through the network layer by layer. Given input bounds $x \in [l^{(0)}, u^{(0)}]$, we compute bounds for each layer's pre-activations and post-activations.

**Interval Bound Propagation (IBP):**

*Detailed Explanation:* IBP propagates interval bounds through each layer by computing the tightest possible output interval given input intervals and layer operations. For affine transformations, this is exact and tractable. For a linear layer $z = Wx + b$, we split the weight matrix $W$ into positive and negative components: $W^+ = \max(W, 0)$ contains all positive weights (negative entries zeroed), and $W^- = \min(W, 0)$ contains all negative weights (positive entries zeroed). To compute lower bounds on the output, we use lower bounds of inputs multiplied by positive weights (both small $\to$ small product) and upper bounds of inputs multiplied by negative weights (large magnitude $\times$ negative $\to$ small result). The formulas are:

$$ l^{(i+1)} = W^{+}l^{(i)} + W^{-}u^{(i)} + b \qquad (234) $$

$$ u^{(i+1)} = W^{+}u^{(i)} + W^{-}l^{(i)} + b \qquad (235) $$
where $l^{(i)}$, $u^{(i)}$ are the lower and upper bounds at layer $i$, and $W^+$, $W^-$ are the positive and negative weight matrices.

For ReLU activations: $l' = \max(l, 0)$, $u' = \max(u, 0)$. If the input interval $[l, u]$ is entirely positive, ReLU is identity; if entirely negative, output is $[0, 0]$; if $l < 0 < u$, output is $[0, u]$.

IBP is computationally efficient (O(network size), linear in parameters) but produces loose bounds because it treats each neuron independently, ignoring correlations. For example, if $z_1 = x$ and $z_2 = -x$ with $x \in [-1, 1]$, IBP computes $z_1 + z_2 \in [-2, 2]$, while the true range is exactly $\{0\}$.

**Concrete Example:** Consider a 2-layer network with input $x \in [0, 1]$, first layer $z_1 = 2x - 1$, ReLU, then $z_2 = -3 \cdot \text{ReLU}(z_1) + 1$.

* Layer 1: $z_1 = 2x - 1$ with $x \in [0, 1]$ gives $z_1 \in [2 \cdot 0 - 1, 2 \cdot 1 - 1] = [-1, 1]$.

* ReLU: $\text{ReLU}(z_1) \in [\max(-1, 0), \max(1, 0)] = [0, 1]$.

* Layer 2: $z_2 = -3 \cdot \text{ReLU}(z_1) + 1$. Here $W = -3$ (negative), so $W^+ = 0$, $W^- = -3$. Lower bound: $l^{(2)} = 0 \cdot 0 + (-3) \cdot 1 + 1 = -2$. Upper bound: $u^{(2)} = 0 \cdot 1 + (-3) \cdot 0 + 1 = 1$. Thus $z_2 \in [-2, 1]$.

IBP proves the output range is $[-2, 1]$, useful for checking if $z_2 > 0$ always holds (answer: no, verification fails).

## Linear Relaxation:

**Detailed Explanation:** Tighter bounds can be obtained by relaxing non-linear activations with linear upper and lower bounds, preserving more relational information between neurons. For ReLU on interval $[l, u]$:

* If $u \le 0$: $\text{ReLU}(z) = 0$ (neuron always inactive)

* If $l \ge 0$: $\text{ReLU}(z) = z$ (neuron always active)

* If $l < 0 < u$: neuron may be active or inactive, requiring approximation

The tightest convex relaxation of ReLU in the unstable case ($l < 0 < u$) is the “triangle relaxation”:

$$ \text{ReLU}(z) \ge 0, \quad \text{ReLU}(z) \ge z, \quad \text{ReLU}(z) \le \frac{u}{u-l}(z-l) \quad (236) $$

where $l, u$ are the pre-activation bounds and the inequalities define the convex hull of the ReLU function.

The upper bound is a line connecting $(l, 0)$ and $(u, u)$, which is the tightest linear upper bound containing the ReLU curve. The lower bounds express that ReLU is non-negative and greater than its input. This convex relaxation allows propagating linear constraints through the network using linear programming, tracking dependencies between neurons rather than just intervals.

**Concrete Example:** For ReLU with $z \in [-2, 3]$ (unstable neuron):

* Lower bounds: $\text{ReLU}(z) \ge 0$ and $\text{ReLU}(z) \ge z$ (second is inactive in this range).

* Upper bound: $\text{ReLU}(z) \le \frac{3}{3-(-2)}(z - (-2)) = \frac{3}{5}(z + 2) = 0.6z + 1.2$.

For input $z = 1$, the true value is $\text{ReLU}(1) = 1$, while the upper bound gives $0.6(1) + 1.2 = 1.8$, so $\text{ReLU}(1) \le 1.8$ (relaxation introduces slack but maintains soundness). For $z = -1$, true value is 0, upper bound is $0.6(-1) + 1.2 = 0.6$. This linear relaxation can be back-propagated through the network to compute tighter output bounds than IBP.
### 12.1.4 Symbolic Bound Propagation and Abstract Interpretation

CROWN (Convex Relaxation with Optimal Weights for Neural Networks) computes bounds by back-propagating linear constraints to the input layer:

$$f(x) \geq A_L x + b_L \quad \forall x \in \mathcal{C} \qquad (237)$$

where $A_L, b_L$ are the linear relaxation coefficients. where $A_L, b_L$ are derived from the linear relaxations of activations. The lower bound is then $\min_{x \in \mathcal{C}} (A_L x + b_L)$.

Abstract Interpretation methods use geometric domains to track correlations. For example, the **Zonotope** domain (used in DeepZ) represents values as:

$$z_j^{(i)} = c_j + \sum_{k=1}^{m} \alpha_{jk} \epsilon_k, \quad \epsilon_k \in [-1, 1] \qquad (238)$$

where $c_j$ is the center, $\alpha_{jk}$ are the generators, and $\epsilon_k$ are the noise terms. where $c_j$ is the center and $\epsilon_k$ are noise terms shared across neurons. **DeepPoly** uses a more flexible domain with upper and lower linear bounds for each neuron.

### 12.1.5 Mixed Integer Linear Programming (MILP)

For exact verification, the ReLU network can be encoded as a MILP:

$$z = Wx + b \qquad (239)$$

$$a_j \geq z_j \quad (\text{lower bound}) \qquad (240)$$

$$a_j \geq 0 \quad (\text{non-negative}) \qquad (241)$$

$$a_j \leq u_j \cdot \delta_j \quad (\text{upper bound if active}) \qquad (242)$$

$$a_j \leq z_j - l_j(1 - \delta_j) \quad (\text{linear if active}) \qquad (243)$$

$$\delta_j \in \{0, 1\} \quad (\text{binary indicator}) \qquad (244)$$

where $a_j$ is the activation, $\delta_j$ is the binary indicator for neuron activity, and $l_j, u_j$ are the bounds. MILP provides exact solutions but scales exponentially with the number of ReLU units.

### 12.1.6 Certified Training

Standard adversarial training provides no formal guarantees. Certified training optimizes a provable bound on the worst-case loss:

$$\min_{\theta} \mathbb{E}_{(x,y) \sim D} [\ell(f_{\theta}(x), y; \epsilon)] \qquad (245)$$

where $\bar{\ell}$ is the upper bound on the worst-case loss within the perturbation radius $\epsilon$. where $\bar{\ell}$ is an upper bound on $\max_{\|x' - x\|_p \leq \epsilon} \ell(f_{\theta}(x'), y)$ computed via bound propagation.

**IBP Training**: Use interval bounds for certified training. Fast but may produce loose bounds.

**CROWN-IBP**: Interpolate between IBP and CROWN bounds during training for tighter certificates.

## 12.2 From Theory to Practice: MLOps for Trustworthy AI

Building trustworthy AI is not just about mathematical proofs; it requires rigorous engineering practices throughout the entire lifecycle. This integration of Machine Learning and DevOps is known as MLOps.
### 12.2.1 The Trustworthy AI Lifecycle

The lifecycle of a trustworthy AI system can be divided into three phases. In the **Design Phase**, stakeholders are consulted to define fairness and safety criteria (Value Alignment), and potential adversaries and failure modes are identified (Threat Modeling). The **Development Phase** involves rigorous testing, including Unit Testing for individual components, Integration Testing for system compatibility, and specific Fairness/Robustness Testing on “challenge sets” (e.g., adversarial examples, specific subgroups). Finally, the **Deployment Phase** requires continuous Monitoring of performance metrics and Drift Detection to identify Covariate Shift or Concept Drift using statistical tests like Kolmogorov-Smirnov (KS) or Maximum Mean Discrepancy (MMD).

### 12.2.2 Uncertainty Estimation

A trustworthy model should know when it doesn't know.

* **Aleatoric Uncertainty**: Inherent noise in the data (e.g., sensor noise). Irreducible.

* **Epistemic Uncertainty**: Lack of knowledge due to limited data. Reducible with more data.

**Calibration**: A model is calibrated if its predicted probabilities match the empirical frequencies.

$$P(Y = y | \hat{P} = p) = p \qquad (246)$$

where $\hat{P}$ is the predicted probability. The Expected Calibration Error (ECE) measures the deviation from perfect calibration.

## 12.3 Documentation and Transparency

Transparency is a prerequisite for trust. Standardized documentation helps users understand the capabilities and limitations of AI systems.

### 12.3.1 Datasheets for Datasets

Proposed by [15], Datasheets are like nutrition labels for data. Key sections include **Motivation** (purpose and funding), **Composition** (what instances represent, subpopulations, sensitivity), **Collection Process** (acquisition and consent), **Preprocessing** (cleaning and augmentation), and **Uses** (intended and unintended uses).

### 12.3.2 Model Cards

Proposed by [25], Model Cards provide critical information about trained models. They detail **Model Details** (architecture, version, date), **Intended Use** (primary uses and out-of-scope cases), **Factors** (demographic or environmental factors affecting performance), **Metrics** (performance disaggregated by subgroups), and **Ethical Considerations** (risks and mitigations).

## 12.4 AI Governance and Regulation

Governance refers to the policies and processes that ensure AI systems are developed and deployed responsibly.
**12.4.1 Ethical Principles**

Most AI ethics guidelines converge on four bioethics principles: **Beneficence** (AI should benefit humanity), **Non-maleficence** (AI should not harm), **Autonomy** (humans should retain control), and **Justice** (benefits and burdens should be distributed fairly).

**12.4.2 Risk-Based Governance Approaches**

Standardized governance frameworks increasingly adopt a **risk-based approach**. **Critical Risk** applications, such as systems controlling autonomous safety components or medical diagnostics, require strict verification, high-quality data provenance, and continuous human oversight [31]. **Operational Risk** systems face transparency obligations (e.g., ensuring users understand system limitations). This tiered approach ensures that safety measures are proportional to the potential impact of the AI application [8].

**Exercises**

1. Drift Detection:

(a) Train a model on the MNIST dataset.

(b) Create a “drifted” test set by rotating the images or adding noise.

(c) Implement a drift detector using the Kolmogorov-Smirnov test on the model's confidence scores.

(d) Plot the p-value of the test as the intensity of the drift increases.

2. Calibration:

(a) Train a ResNet on CIFAR-10.

(b) Plot the Reliability Diagram (Confidence vs. Accuracy).

(c) Calculate the Expected Calibration Error (ECE).

(d) Apply Temperature Scaling to calibrate the model and re-evaluate ECE.

3. Documentation:

(a) Select a public dataset (e.g., CelebA).

(b) Write a Datasheet for it, researching its origin and potential biases.

(c) Train a simple model on it and write a Model Card, explicitly reporting performance on different demographic subgroups.

4. Formal Verification:

(a) Implement Interval Bound Propagation (IBP) for a 2-layer MLP.

(b) Verify local robustness of the network for a specific input and $\epsilon$ radius.

(c) Compare IBP bounds with exact MILP verification (using a small network).

(d) Measure the "gap" between IBP upper bound and true worst-case loss.

5. Certified Training:

(a) Train a small network on MNIST using standard training vs. IBP training.

(b) Measure both clean accuracy and certified robust accuracy at $\epsilon = 0.1$.
(c) Plot the certified accuracy vs. perturbation size curve.

(d) Implement CROWN-IBP training and compare with pure IBP.

6. MLOps Pipeline:

(a) Set up a model monitoring pipeline using MLflow or similar tools.

(b) Implement automatic retraining triggers based on drift detection.

(c) Create dashboards for tracking fairness metrics over time.

(d) Implement model versioning and rollback capabilities.

7. Threat Modeling:

(a) For a loan approval system, enumerate potential threat actors and attack vectors.

(b) Create a threat model using STRIDE (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege).

(c) Propose mitigations for each identified threat.

(d) Implement adversarial testing to validate the mitigations.

8. Red Teaming for AI Safety:

(a) Conduct red team testing on a content moderation classifier.

(b) Attempt to find adversarial examples that bypass moderation.

(c) Document failure modes and propose improvements.

(d) Create a challenge set for continuous testing.
# 13 Advanced Topics in Trustworthy AI

## 13.1 Neural Architecture Design for Trustworthiness

The choice of neural network architecture significantly impacts trustworthiness properties. This section explores architectural considerations for building inherently more robust, fair, and interpretable models.

### 13.1.1 Lipschitz-Constrained Networks

Networks with bounded Lipschitz constants provide natural robustness guarantees.

**Definition 13.1** (Lipschitz Constant). A function $f: \mathbb{R}^d \to \mathbb{R}^k$ has Lipschitz constant $L$ if:

$$ \|f(x) - f(x')\| \le L\|x - x'\| \quad \forall x, x' \in \mathbb{R}^d \qquad (247) $$

where $f$ is the function, $L$ is the Lipschitz constant, and $\| \cdot \|$ denotes the norm.

For a feedforward network $f = f_L \circ f_{L-1} \circ \dots \circ f_1$, the Lipschitz constant is bounded by the product of layer-wise Lipschitz constants:

$$ \mathrm{Lip}(f) \le \prod_{i=1}^{L} \mathrm{Lip}(f_i) \qquad (248) $$

where $\mathrm{Lip}(f)$ is the Lipschitz constant of the network, and $\mathrm{Lip}(f_i)$ is the Lipschitz constant of the $i$-th layer.

**Proposition 13.1** (Spectral Normalization). For a linear layer with weight matrix $W$, the Lipschitz constant is the spectral norm $\sigma_1(W)$. Spectral normalization constrains this by:

$$ \tilde{W} = \frac{W}{\sigma_1(W)} \qquad (249) $$

where $\tilde{W}$ is the normalized weight matrix, $W$ is the original weight matrix, and $\sigma_1(W)$ is the largest singular value (spectral norm) of $W$. ensuring $\mathrm{Lip}(\tilde{W}) = 1$.

**Orthogonal Networks:** Constraining weight matrices to be orthogonal ($W^T W = I$) gives $\mathrm{Lip}(W) = 1$ exactly, enabling tight Lipschitz bounds.

### 13.1.2 Monotonic Networks

For certain applications, monotonicity constraints provide interpretability and fairness guarantees.

**Definition 13.2** (Monotonic Neural Network). A network $f: \mathbb{R}^d \to \mathbb{R}$ is monotonically increasing in feature $x_j$ if:

$$ x_j > x'_j \implies f(x) \ge f(x') \qquad (250) $$

where $x_j, x'_j$ are the values of the $j$-th feature, and $f(x)$ is the network output. for all $x, x'$ differing only in component $j$.

Monotonicity can be enforced by:

* **Non-negative weights:** All weights connecting to monotonic inputs are constrained to be non-negative
* **Lattice networks:** Use calibrated interpolation on a lattice structure

* **Certification:** Post-hoc verification that monotonicity holds on a test set

**Theorem 13.2 (Monotonicity and Fairness).** If a feature $x_j$ represents qualification and the model is monotonically increasing in $x_j$, then higher-qualified individuals always receive higher scores, regardless of protected attributes.

*Proof.* Suppose individuals $i$ and $i'$ differ only in qualification: $x_j^i > x_j^{i'}$ and $x_k^i = x_k^{i'}$ for all $k \neq j$. By monotonicity:

$$f(x^i) = f(x_1^i, \dots, x_j^i, \dots, x_d^i) \ge f(x_1^i, \dots, x_j^{i'}, \dots, x_d^i) = f(x^{i'}) \quad (251)$$

where $f(x^i)$ is the score for individual $i$, and $x_j^i$ is the qualification feature for individual $i$. This holds regardless of the values of other features, including protected attributes. Therefore, if individual $i$ has strictly higher qualification ($x_j^i > x_j^{i'}$) but the same values for all other features, $i$ must receive a weakly higher score than $i'$. This prevents the unfairness where less qualified individuals receive higher scores than more qualified ones, which could occur if the model were non-monotonic in the qualification feature. □

### 13.1.3 Attention Mechanisms and Interpretability

Attention mechanisms provide a form of built-in interpretability by producing attention weights that indicate which input elements influence the output.

**Definition 13.3 (Self-Attention).** For input sequence $X = [x_1, \dots, x_n]$, self-attention computes:

$$Q = XW_Q, \quad K = XW_K, \quad V = XW_V \quad (252)$$

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V \quad (253)$$

where $Q, K, V$ are query, key, and value matrices, $W_Q, W_K, W_V$ are learnable weight matrices, $d_k$ is the dimension of the keys, and softmax is applied row-wise.

**Remark 13.1 (Attention as Explanation).** While attention weights provide some interpretability, they should not be equated with feature importance. Research shows attention can be manipulated to produce arbitrary weights while maintaining predictions, and attention may not reflect the true causal influence of inputs.

### 13.1.4 Evidential Deep Learning

Evidential neural networks output parameters of a prior distribution over class probabilities, enabling principled uncertainty quantification.

**Definition 13.4 (Dirichlet Prior).** For K-class classification, the network outputs concentration parameters $\alpha = (\alpha_1, \dots, \alpha_K)$ of a Dirichlet distribution:

$$f(\mathbf{p}; \alpha) = \frac{1}{B(\alpha)} \prod_{k=1}^{K} p_k^{\alpha_k - 1} \quad (254)$$

where $p$ is the vector of class probabilities, $Dir(\alpha)$ is the Dirichlet distribution parameterized by $\alpha$, and $p_k$ is the probability of class $k$. The predictive distribution is:

$$P(Y = k | \alpha) = \frac{\alpha_k}{\sum_j \alpha_j} \quad (255)$$

where $\alpha_k$ is the concentration parameter for class $k$.
The total evidence $S = \sum_k \alpha_k$ indicates model confidence.

$$ \text{Uncertainty} = \frac{K}{S} \qquad (256) $$

where $K$ is the number of classes and $S$ is the total evidence (sum of Dirichlet parameters). Low evidence (high uncertainty) indicates the model has seen few similar examples.

## 13.2 Robust Statistics and Heavy-Tailed Data

Classical machine learning assumes light-tailed (sub-Gaussian) distributions. Real-world data often exhibits heavy tails, outliers, and contamination that can severely bias standard estimators.

### 13.2.1 Huber's Contamination Model

**Definition 13.5 (Contamination Model).** The observed data comes from a mixture:

$$ P = (1 - \epsilon)P_0 + \epsilon Q \qquad (257) $$

where $P_0$ is the true distribution, $Q$ is an arbitrary contamination distribution, and $\epsilon$ is the contamination fraction.

**Definition 13.6 (Breakdown Point).** The breakdown point of an estimator is the largest fraction of contamination $\epsilon^*$ it can tolerate while remaining bounded:

$$ \epsilon^* = \sup\{\epsilon : \sup_Q \|\hat{\theta}(P) - \theta_0\| < \infty\} \qquad (258) $$

where $\hat{\theta}(P)$ is the estimator applied to the contaminated distribution $P$, and $\theta_0$ is the true parameter.

The sample mean has breakdown point 0 (a single outlier can make it arbitrarily bad), while the median has breakdown point 0.5 (optimal).

### 13.2.2 Robust Mean Estimation

**Theorem 13.3 (Median of Means).** Divide $n$ samples into $k$ groups, compute the mean of each group, then take the median of these means. This estimator achieves:

$$ |\hat{\mu}_{MoM} - \mu| \le O\left(\sigma\sqrt{\frac{k}{n}}\right) \qquad (259) $$

where $\hat{\mu}_{MoM}$ is the median-of-means estimator, $\mu$ is the true mean, $\sigma$ is the standard deviation, $n$ is the total number of samples, and $k$ is the number of groups. with probability $1 - e^{-\Omega(k)}$, assuming only bounded variance.

*Proof.* Let $X_1, \dots, X_n$ be i.i.d. samples with mean $\mu$ and variance $\sigma^2$. Partition into $k$ groups of size $m = n/k$. For group $i$, the mean is:

$$ \hat{\mu}_i = \frac{1}{m} \sum_{j \in G_i} X_j \qquad (260) $$
where $\hat{\mu}_i$ is the sample mean of the i-th group $G_i$. By Chebyshev's inequality, for each group:

$$P(|\hat{\mu}_i - \mu| > t) \le \frac{\sigma^2/m}{t^2} = \frac{\sigma^2 k}{nt^2} \qquad (261)$$

where $t$ is the deviation threshold. Set $t = C\sigma\sqrt{k/n}$ for some constant $C$. Then:

$$P(|\hat{\mu}_i - \mu| > C\sigma\sqrt{k/n}) \le \frac{1}{C^2} \qquad (262)$$

Choosing $C = \sqrt{3}$, at most $k/3$ of the group means deviate by more than $\sqrt{3}\sigma\sqrt{k/n}$ from $\mu$ with high probability. The median of $k$ values, at least half of which lie within distance $\sqrt{3}\sigma\sqrt{k/n}$ of $\mu$, must itself lie within this distance. By a union bound over the $k$ groups and using concentration of the median, we get:

$$P(|\hat{\mu}_{MoM} - \mu| > C'\sigma\sqrt{k/n}) \le e^{-\Omega(k)} \qquad (263)$$

for an appropriate constant $C'$. The exponential tail bound comes from the fact that the median requires at least $k/2$ groups to deviate, which occurs with exponentially small probability. $\square$

**Theorem 13.4 (Trimmed Mean).** *Sort samples, discard the smallest and largest $\lfloor \epsilon n \rfloor$ samples, and compute the mean of the remaining samples. This achieves breakdown point $\epsilon$ and optimal rate under bounded variance.*

### 13.2.3 Robust Covariance Estimation

The sample covariance matrix is highly sensitive to outliers. Robust alternatives include:

**Definition 13.7 (Minimum Covariance Determinant (MCD)).** Find the subset $H$ of size $h$ that minimizes the determinant of its sample covariance:

$$\hat{H} = \arg\min_{H \subset \{1, \dots, n\}, |H|=h} \det(\mathrm{Cov}(X_H)) \qquad (264)$$

where $H$ is a subset of samples of size $h$, and $\mathrm{Cov}(X_H)$ is the sample covariance of subset $H$. Typically $h = \lfloor n/2 \rfloor + \lfloor (d+1)/2 \rfloor$.

**Theorem 13.5 (Robust PCA).** *Under the Huber contamination model:*

$$X_i \sim (1 - \epsilon)\mathcal{D}(\mu, \Sigma) + \epsilon Q \qquad (265)$$

where $\mathcal{D}$ is a distribution with structure (e.g., spiked covariance), and $Q$ is an arbitrary outlier distribution. Robust PCA algorithms can recover the principal components even when a constant fraction of points are arbitrarily corrupted.

## 13.3 Causal Machine Learning

Integrating causal reasoning into machine learning enables models to generalize under distribution shift and support counterfactual queries.

### 13.3.1 Invariant Causal Prediction

The key insight is that causal relationships are invariant across environments, while spurious correlations change.
**Definition 13.8 (Invariant Prediction).** A set of features $S \subseteq \{1, \dots, d\}$ satisfies the invariant prediction property if:

$$Y = f(X_S) + \epsilon, \quad \epsilon \perp X_S \qquad (266)$$

where $Y$ is the target variable, $X_S$ is the subset of invariant features, $f$ is the causal mechanism, and $\epsilon$ is independent noise. This property must hold simultaneously across all environments $e \in \mathcal{E}$, meaning the conditional distribution $P^e(Y|X_S)$ is invariant.

**Theorem 13.6 ([28]).** Under faithfulness assumptions, the set of invariant features contains only direct causes of $Y$ and is a subset of the Markov blanket.

### 13.3.2 Invariant Risk Minimization (IRM)

IRM [6] learns representations $\Phi(x)$ that enable the same optimal classifier across all training environments:

**Definition 13.9 (IRM Objective).**

$$\min_{\Phi} \sum_{e \in \mathcal{E}_{train}} R^e(\Phi) + \lambda \|\nabla_w R^e(w \circ \Phi)|_{w=1.0}\|^2 \qquad (267)$$

where $\Phi$ is the data representation, $w = 1.0$ is a fixed dummy linear classifier, $R^e$ is the risk in environment $e$, and $\lambda$ is the regularization strength. The regularization term penalizes representations where the optimal classifier locally deviates from the fixed identity.

**Theorem 13.7 (IRM Optimality).** If the true causal parents of $Y$ are recoverable and IRM finds a representation achieving zero training loss across all environments, then $\Phi$ captures only causal features.

### 13.3.3 Domain Adaptation with Causal Models

Causal models enable principled domain adaptation by identifying which distribution shifts are problematic.

**Definition 13.10 (Types of Shift).**

* **Covariate shift:** $P^{source}(X) \neq P^{target}(X)$ but $P(Y|X)$ unchanged

* **Target shift:** $P^{source}(Y) \neq P^{target}(Y)$ but $P(X|Y)$ unchanged

* **Mechanism shift:** $P(Y|X)$ changes (requires causal knowledge to adapt)

**Theorem 13.8 (Adaptation with Causal Knowledge).** If we know the causal graph $G$ relating $X$, $Y$, and domain variable $D$:

1. Shifts in causes of $Y$ ($X \to Y$): Standard importance weighting works

2. Shifts in effects of $Y$ ($Y \to X$): Target shift correction works

3. Mechanism shifts: Requires intervention or structural assumptions
## 13.4 Privacy-Preserving Machine Learning: Advanced Topics

### 13.4.1 Local Differential Privacy

In local DP, data is privatized before leaving the user's device, removing the need to trust a central curator.

**Definition 13.11 (Local Differential Privacy).** A randomized mechanism $\mathcal{R}: \mathcal{X} \to \mathcal{Y}$ satisfies $\epsilon$-local DP if for all $x, x' \in \mathcal{X}$ and $y \in \mathcal{Y}$:

$$P(\mathcal{R}(x) = y) \le e^{\epsilon} P(\mathcal{R}(x') = y) \qquad (268)$$

where $\mathcal{R}$ is the randomized mechanism, $x, x'$ are any two input values, $y$ is the output, and $\epsilon$ is the privacy budget.

**Proposition 13.9 (Randomized Response).** For binary data, with probability $\frac{e^{\epsilon}}{1+e^{\epsilon}}$ report truthfully; otherwise, report the opposite. This satisfies $\epsilon$-local DP.

Local DP is stronger than central DP but requires more data for the same utility.

### 13.4.2 Shuffle Model

The shuffle model provides intermediate privacy between local and central DP.

**Definition 13.12 (Shuffle Model).** Each user applies local randomizer $\mathcal{R}$, messages are randomly shuffled (breaking user-message association), then analyzed.

**Theorem 13.10 (Privacy Amplification by Shuffling).** If $\mathcal{R}$ is $\epsilon_0$-local DP and there are $n$ users, the shuffle mechanism satisfies $(O(\epsilon_0 \sqrt{\ln(1/\delta)/n}), \delta)$-central DP for small $\epsilon_0$.

### 13.4.3 Machine Unlearning

The right to data deletion (a core tenet of data sovereignty) creates challenges for ML: how to remove the influence of specific training data?

**Definition 13.13 (Exact Unlearning).** Given model $\theta$ trained on $D$, unlearning produces $\theta'$ such that $\theta' \sim \theta$ trained on $D \setminus \{z\}$ (statistically indistinguishable).

Approaches include:

* **Retraining:** Retrain from scratch without the data point (exact but expensive)

* **SISA Training:** Partition data into shards, train separate models, aggregate. Unlearning requires retraining only one shard.

* **Influence-based:** Use influence functions to approximate the effect of removing a point:

$$\theta_{-z} \approx \theta - H_{\theta}^{-1} \nabla_{\theta} \ell(\theta; z) \qquad (269)$$

where $\theta_{-z}$ is the parameter vector after removing point $z$, $\theta$ is the original parameter vector, $H_{\theta}$ is the Hessian matrix, and $\nabla_{\theta} \ell(\theta; z)$ is the gradient of the loss at $z$.

## 13.5 Trustworthy AI for Large Language Models

The emergence of large language models (LLMs) presents unique trustworthiness challenges due to their scale, emergent capabilities, and deployment in open-ended settings.
### 13.5.1 Hallucination and Factuality

LLMs can generate fluent but factually incorrect text ("hallucinations").

**Definition 13.14** (Types of Hallucination).

* **Intrinsic**: Contradicts the source document in summarization/QA

* **Extrinsic**: Cannot be verified from the source (fabricated facts)

Mitigation strategies include:

* **Retrieval Augmentation**: Ground generation in retrieved documents

* **Self-Consistency**: Sample multiple responses and check consistency

* **Calibration**: Train models to express uncertainty when unsure

### 13.5.2 Jailbreaking and Prompt Injection

Despite safety training, LLMs can be manipulated to produce harmful outputs.

**Definition 13.15** (Jailbreak Attack). An input prompt designed to circumvent safety guardrails and elicit prohibited outputs (harmful content, private information, etc.).

Attack categories include:

* **Roleplay**: "Pretend you are DAN who has no restrictions..."

* **Obfuscation**: Encode requests in base64, pig latin, etc.

* **Gradient-based**: Optimize adversarial suffixes using white-box access

**Definition 13.16** (Prompt Injection). Malicious instructions embedded in user-provided content that hijack the LLM's behavior:

$$ \text{Input: "Summarize: \(\underbrace{\text{Ignore previous instructions and...}}_{\text{injection}}"} \qquad (270) $$

where the injection overrides the original task instruction.

### 13.5.3 Constitutional AI

Constitutional AI (CAI) trains models to follow a set of principles ("constitution") through self-critique and revision.

1. **Red Teaming**: Generate harmful queries

2. **Self-Critique**: Model critiques its own response using constitution

3. **Revision**: Model produces improved response

4. **RLHF on Revisions**: Fine-tune on (query, revised response) pairs
### 13.5.4 Watermarking and Detection

Detecting AI-generated text is crucial for preventing misuse.

**Definition 13.17 (Statistical Watermarking).** During generation, partition vocabulary into "green" and "red" lists based on hash of previous tokens. Bias generation toward green tokens.

**Theorem 13.11 ([20]).** For watermarked text of length T with green list fraction $\gamma$ and bias $\delta$:

$$z = \frac{|G| - \gamma T}{\sqrt{T\gamma(1-\gamma)}} \xrightarrow{T \to \infty} \mathcal{N}(0, 1) \qquad (271)$$

where $z$ is the z-score, $|G|$ is the number of green tokens, $T$ is the text length, and $\gamma$ is the green list fraction. Under watermarking, $\mathbb{E}[|G|] = (\gamma + \delta(1 - \gamma))T$.

The watermark is detectable with high probability for sufficiently long text but robust to paraphrasing remains challenging.
# 14 Mathematical Background

## 14.1 Probability Theory Foundations

### 14.1.1 Random Variables and Distributions

**Definition 14.1** (Random Variable). A random variable $X$ is a measurable function from a probability space $(\Omega, \mathcal{F}, P)$ to a measurable space $(E, \mathcal{E})$:

$$X: \Omega \to E \qquad (272)$$

where $\Omega$ is the sample space, and $E$ is the measurable space (e.g., $\mathbb{R}$). For discrete random variables, $E = \{x_1, x_2, \dots\}$ and the distribution is characterized by the probability mass function $P(X = x_i) = p_i$.

**Definition 14.2** (Probability Density Function). For continuous random variables, the probability density function $f_X(x)$ satisfies:

$$P(a \le X \le b) = \int_a^b f_X(x)dx \qquad (273)$$

where $f_X(x)$ is the PDF, and the integral represents the probability of $X$ falling in the interval $[a, b]$. with $f_X(x) \ge 0$ and $\int_{-\infty}^{\infty} f_X(x)dx = 1$.

### 14.1.2 Common Distributions

* **Gaussian (Normal):** $X \sim \mathcal{N}(\mu, \sigma^2)$ with $f(x) = \frac{1}{\sqrt{2\pi}\sigma^2} \exp(-\frac{(x-\mu)^2}{2\sigma^2})$

* **Laplace:** $X \sim \text{Lap}(b)$ with $f(x) = \frac{1}{2b} \exp(-\frac{|x|}{b})$

* **Exponential:** $X \sim \text{Exp}(\lambda)$ with $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$

* **Bernoulli:** $X \sim \text{Bern}(p)$ with $P(X = 1) = p, P(X = 0) = 1 - p$

* **Categorical:** $X \sim \text{Cat}(\mathbf{p})$ with $P(X = k) = p_k$ for $k \in \{1, \dots, K\}$

* **Dirichlet:** $\mathbf{p} \sim \text{Dir}(\alpha)$ with $f(\mathbf{p}) \propto \prod_k p_k^{\alpha_k - 1}$

### 14.1.3 Expectation and Moments

**Definition 14.3** (Expectation). For a random variable $X$ with distribution $P$:

$$\mathbb{E}[X] = \int x dP(x) = \begin{cases} \sum_x x \cdot P(X = x) & \text{discrete} \\ \int_{-\infty}^{\infty} x \cdot f_X(x) dx & \text{continuous} \end{cases} \qquad (274)$$

where $\mathbb{E}[X]$ is the expected value (mean) of $X$.

**Definition 14.4** (Variance and Standard Deviation).

$$\text{Var}[X] = \mathbb{E}[(X - \mathbb{E}[X])^2] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2, \quad \sigma_X = \sqrt{\text{Var}[X]} \qquad (275)$$

where $\text{Var}[X]$ is the variance, and $\sigma_X$ is the standard deviation.

**Definition 14.5** (Covariance and Correlation).

$$\text{Cov}[X, Y] = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])], \quad \rho_{XY} = \frac{\text{Cov}[X, Y]}{\sigma_X \sigma_Y} \qquad (276)$$

where $\text{Cov}[X, Y]$ is the covariance, and $\rho_{XY}$ is the correlation coefficient.
### 14.1.4 Conditional Probability and Independence

**Definition 14.6 (Conditional Probability).** where $P(A|B)$ is the conditional probability of event A given event B.

**Theorem 14.1 (Bayes' Theorem).**

$$P(A|B) = \frac{P(B|A)P(A)}{P(B)} = \frac{P(B|A)P(A)}{\sum_a P(B|A=a)P(A=a)} \quad (277)$$

where $P(B|A)$ is the likelihood, $P(A)$ is the prior, and $P(B)$ is the marginal likelihood (evidence).

**Definition 14.7 (Independence).** Events A and B are independent ($A \perp B$) if $P(A \cap B) = P(A)P(B)$, equivalently $P(A|B) = P(A)$. Random variables $X \perp Y$ if $P(X, Y) = P(X)P(Y)$.

**Definition 14.8 (Conditional Independence).** $X \perp Y|Z$ if $P(X, Y|Z) = P(X|Z)P(Y|Z)$.

## 14.2 Concentration Inequalities

Concentration inequalities bound how far random quantities deviate from their expectations.

**Theorem 14.2 (Chebyshev's Inequality).** For random variable X with mean $\mu$ and variance $\sigma^2$:

$$P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2} \quad (278)$$

where $k > 0$ is the number of standard deviations.

**Theorem 14.3 (Hoeffding's Inequality).** Let $X_1, \dots, X_n$ be independent random variables such that $a_i \leq X_i \leq b_i$. Let $S_n = \sum X_i$. Then:

$$P(|S_n - \mathbb{E}[S_n]| \geq t) \leq 2 \exp \left( - \frac{2t^2}{\sum (b_i - a_i)^2} \right) \quad (279)$$

where $t > 0$ is the deviation, and $[a_i, b_i]$ is the range of $X_i$.

**Theorem 14.4 (McDiarmid's Inequality).** Let $f: \mathcal{X}^n \to \mathbb{R}$ satisfy the bounded difference property with constants $c_1, \dots, c_n$:

$$\sup_{x_1, \dots, x_n, x'_i} |f(x_1, \dots, x_i, \dots, x_n) - f(x_1, \dots, x'_i, \dots, x_n)| \leq c_i \quad (280)$$

Then for any $t > 0$:

$$P(|f(X) - \mathbb{E}[f(X)]| \geq t) \leq 2 \exp \left( - \frac{2t^2}{\sum c_i^2} \right) \quad (281)$$

where $c_i$ bounds the change in $f$ when the $i$-th input changes.

**Theorem 14.5 (Bernstein's Inequality).** Let $X_1, \dots, X_n$ be independent with $\mathbb{E}[X_i] = 0$, $|X_i| \leq M$, and $\sum \mathrm{Var}[X_i] \leq V$. Then:

$$P\left(\left|\sum_{i=1}^{n} X_i\right| \geq t\right) \leq 2 \exp \left(-\frac{t^2/2}{V+M t / 3}\right) \quad (282)$$

where $M$ is the bound on $|X_i|$, and $V$ is the total variance.

**Theorem 14.6 (Sub-Gaussian Concentration).** A random variable $X$ is $\sigma$-sub-Gaussian if for all $\lambda \in \mathbb{R}$:

$$\mathbb{E}[e^{\lambda(X - \mathbb{E}[X])}] \leq e^{\lambda^2 \sigma^2 / 2} \quad (283)$$

where $\sigma$ is the sub-Gaussian parameter.

For $\sigma$-sub-Gaussian $X$: $P(|X - \mathbb{E}[X]| \geq t) \leq 2e^{-t^2/(2\sigma^2)}$.
## 14.3 Information Theory

**Definition 14.9 (Entropy).** For discrete random variable $X$ with distribution $P$:

$$H(X) = -\sum_{x} P(x) \log P(x) = \mathbb{E}[-\log P(X)] \qquad (284)$$

where $H(X)$ is the entropy, and $P(x)$ is the probability mass function. Entropy measures the average information content or uncertainty.

**Definition 14.10 (Conditional Entropy).**

$$H(Y|X) = \mathbb{E}_X[H(Y|X=x)] = -\sum_{x,y} P(x,y) \log P(y|x) \qquad (285)$$

where $H(Y|X)$ is the conditional entropy of $Y$ given $X$.

**Definition 14.11 (Mutual Information).**

$$I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = \sum_{x,y} P(x,y) \log \frac{P(x,y)}{P(x)P(y)} \qquad (286)$$

where $I(X;Y)$ is the mutual information. Mutual information measures the information shared between $X$ and $Y$.

**Definition 14.12 (KL Divergence).** For discrete distributions $P$ and $Q$:

$$D_{KL}(P||Q) = \sum_{x} P(x) \ln \frac{P(x)}{Q(x)} = \mathbb{E}_{x \sim P} \left[ \log \frac{P(x)}{Q(x)} \right] \qquad (287)$$

where $D_{KL}(P||Q)$ is the Kullback-Leibler divergence from $Q$ to $P$. KL divergence is non-negative with $D_{KL}(P||Q) = 0$ iff $P = Q$.

**Theorem 14.7 (Pinsker's Inequality).**

$$\delta_{TV}(P,Q) = \frac{1}{2} \sum_{x} |P(x) - Q(x)| \le \sqrt{\frac{1}{2} D_{KL}(P||Q)} \qquad (288)$$

where $\delta_{TV}(P,Q)$ is the total variation distance.

**Definition 14.13 (Cross-Entropy).**

$$H(P,Q) = -\sum_{x} P(x) \log Q(x) = H(P) + D_{KL}(P||Q) \qquad (289)$$

where $H(P,Q)$ is the cross-entropy. Cross-entropy is minimized when $Q = P$.

**Definition 14.14 (Renyi Divergence).** For $\alpha > 0$, $\alpha \neq 1$:

$$D_{\alpha}(P||Q) = \frac{1}{\alpha - 1} \log \sum_{x} P(x)^{\alpha} Q(x)^{1-\alpha} \qquad (290)$$

where $D_{\alpha}(P||Q)$ is the Rényi divergence of order $\alpha$. $D_{1}(P||Q) = \lim_{\alpha \to 1} D_{\alpha}(P||Q) = D_{KL}(P||Q)$.

**Theorem 14.8 (Data Processing Inequality).** For Markov chain $X \to Y \to Z$:

$$I(X;Z) \le I(X;Y) \qquad (291)$$

where $I(X;Z)$ is the mutual information between $X$ and $Z$. Post-processing cannot increase information.
## 14.4 Convex Optimization

**Definition 14.15 (Convex Set).** A set $C$ is convex if for all $x, y \in C$ and $\theta \in [0, 1]$:

$$ \theta x + (1 - \theta)y \in C \qquad (292) $$

where $C$ is the convex set, and $\theta$ is the interpolation parameter.

**Definition 14.16 (Convex Function).** $f: \mathbb{R}^d \to \mathbb{R}$ is convex if for all $x, y \in \text{dom}(f)$ and $\theta \in [0, 1]$:

$$ f(\theta x + (1 - \theta)y) \le \theta f(x) + (1 - \theta)f(y) \qquad (293) $$

where $f$ is the convex function. Equivalently, $f$ is convex iff its epigraph $\{(x, t) : f(x) \le t\}$ is convex.

**Theorem 14.9 (First-Order Characterization).** For differentiable $f$, convexity is equivalent to:

$$ f(y) \ge f(x) + \nabla f(x)^T (y - x) \quad \forall x, y \qquad (294) $$

where $\nabla f(x)$ is the gradient of $f$ at $x$.

**Theorem 14.10 (Second-Order Characterization).** For twice-differentiable $f$, convexity is equivalent to:

$$ \nabla^2 f(x) \succeq 0 \quad \forall x \qquad (295) $$

where $\nabla^2 f(x)$ is the Hessian matrix (positive semidefinite). (Hessian is positive semidefinite).

**Theorem 14.11 (Jensen's Inequality).** If $f$ is convex and $X$ is a random variable, then:

$$ f(\mathbb{E}[X]) \le \mathbb{E}[f(X)] \qquad (296) $$

where $\mathbb{E}[X]$ is the expectation of $X$.

**Definition 14.17 (Strong Convexity).** $f$ is $\mu$-strongly convex if $f(x) - \frac{\mu}{2}\|x\|^2$ is convex, equivalently:

$$ f(y) \ge f(x) + \nabla f(x)^T (y - x) + \frac{\mu}{2} \|y - x\|^2 \qquad (297) $$

where $\mu > 0$ is the strong convexity parameter.

**Definition 14.18 (Smoothness).** $f$ is $L$-smooth if $\nabla f$ is $L$-Lipschitz:

$$ \|\nabla f(x) - \nabla f(y)\| \le L \|x - y\| \qquad (298) $$

where $L$ is the smoothness constant (Lipschitz constant of the gradient). Equivalently: $f(y) \le f(x) + \nabla f(x)^T (y - x) + \frac{L}{2}\|y - x\|^2$.

**Theorem 14.12 (Gradient Descent Convergence).** For $L$-smooth, $\mu$-strongly convex $f$, gradient descent with step size $\eta = 1/L$ satisfies:

$$ f(x_t) - f(x^*) \le \left(1 - \frac{\mu}{L}\right)^t (f(x_0) - f(x^*)) \qquad (299) $$

where $x_t$ is the iterate at step $t$, $x^*$ is the optimum, and $\kappa = L/\mu$ is the condition number. The condition number $\kappa = L/\mu$ determines convergence rate.

**Theorem 14.13 (Singular Value Decomposition).** Any matrix $A \in \mathbb{R}^{m \times n}$ has decomposition:

$$ A = U \Sigma V^T = \sum_{i=1}^{r} \sigma_i u_i v_i^T \qquad (300) $$

where $U \in \mathbb{R}^{m \times r}$, $V \in \mathbb{R}^{n \times r}$ have orthonormal columns, and $\Sigma = \text{diag}(\sigma_1, \dots, \sigma_r)$ with $\sigma_1 \ge \dots \ge \sigma_r > 0$.
**Theorem 14.14** (Eckart-Young Theorem). The best rank-k approximation in Frobenius norm is:

$$A_k = \arg\min_{\mathrm{rank}(B) \le k} \|A - B\|_F = \sum_{i=1}^k \sigma_i u_i v_i^T \quad (301)$$

with error $\|A - A_k\|_F = \sqrt{\sum_{i>k} \sigma_i^2}$.

**Definition 14.19** (Positive Semidefinite Matrices). A symmetric matrix A is positive semidefinite ($A \succeq 0$) if all eigenvalues are non-negative, or equivalently $x^T A x \ge 0$ for all x.
# 15 Advanced Proofs and Derivations

## 15.1 Proof of the VC Dimension Lower Bound

**Theorem 15.1** (Fundamental Theorem of Statistical Learning). Let $\mathcal{H}$ be a hypothesis class with VC dimension $d$. For any $\delta \in (0, 1)$ and any $\epsilon > 0$:

1. (Upper Bound) $m \ge \frac{8}{\epsilon^2}(d \log \frac{16}{\epsilon} + \log \frac{2}{\delta})$ samples suffice for PAC learning

2. (Lower Bound) $m \ge \frac{d-1}{32\epsilon}$ samples are necessary

where $m$ is the sample size, $d$ is the VC dimension, $\epsilon$ is the error tolerance, and $\delta$ is the failure probability.

*Proof of Lower Bound Sketch.* We use the probabilistic method. Consider a set of $d$ points that can be shattered by $\mathcal{H}$. For each labeling $\sigma \in \{0, 1\}^d$, let $h_\sigma$ be the hypothesis achieving this labeling.

The adversary chooses $\sigma$ uniformly at random. Any algorithm seeing $m$ samples has probability at most $2^m/2^d = 2^{m-d}$ of guessing $\sigma$ correctly.

For the algorithm to achieve error $\le \epsilon$ with probability $\ge 1 - \delta$, it must correctly classify at least $(1 - \epsilon)d$ of the $d$ shattered points. This requires learning approximately $(1 - \epsilon)d$ bits of information about $\sigma$.

By information-theoretic arguments, $m \ge \Omega((1 - \epsilon)d) = \Omega(d)$ samples are necessary. A more careful analysis gives $m \ge \frac{d-1}{32\epsilon}$. $\square$

## 15.2 Derivation of the Evidence Lower Bound (ELBO)

In Variational Autoencoders (VAEs) and Bayesian inference, we often wish to approximate an intractable posterior $P(Z|X)$. The ELBO provides a tractable lower bound on the marginal log-likelihood that can be optimized via gradient descent.

**Theorem 15.2** (Evidence Lower Bound). For any distribution $Q(Z|X)$ over latent variables $Z$:

$$ \log P(X) \ge \underbrace{\mathbb{E}_{z \sim Q}[\log P(X|z)]}_{\text{Reconstruction}} - \underbrace{D_{KL}(Q(Z|X)\|P(Z))}_{\text{Regularization}} =: \text{ELBO} \quad (302) $$

where $P(X)$ is the marginal likelihood, $Q(Z|X)$ is the approximate posterior, $P(X|z)$ is the likelihood, and $P(Z)$ is the prior. with equality if and only if $Q(Z|X) = P(Z|X)$.

*Proof.* We start with the KL divergence between the approximate and true posterior (which is always non-negative):

$$ 0 \le D_{KL}(Q(Z|X)\|P(Z|X)) = \mathbb{E}_{z \sim Q} \left[ \log \frac{Q(z|X)}{P(z|X)} \right] \quad (303) $$

where $D_{KL}(Q(Z|X)\|P(Z|X))$ is the KL divergence between the approximate and true posteriors.

Applying Bayes' rule, $P(z|X) = \frac{P(X|z)P(z)}{P(X)}$:

$$ D_{KL}(Q(Z|X)\|P(Z|X)) = \mathbb{E}_{z \sim Q} \left[ \log Q(z|X) - \log P(X|z) - \log P(z) + \log P(X) \right] \quad (304) $$
Since $\log P(X)$ is constant with respect to $z$:

$$D_{KL}(Q(Z|X)\|P(Z|X)) = \log P(X) - \mathbb{E}_{z\sim Q}[\log P(X|z)] + \mathbb{E}_{z\sim Q}\left[\log \frac{Q(z|X)}{P(z)}\right] \quad (305)$$

$$= \log P(X) - \mathbb{E}_{z\sim Q}[\log P(X|z)] + D_{KL}(Q(Z|X)\|P(Z)) \quad (306)$$

Rearranging and using non-negativity of KL:

$$\log P(X) = \underbrace{\mathbb{E}_{z\sim Q}[\log P(X|z)] - D_{KL}(Q(Z|X)\|P(Z))}_{\text{ELBO}} + \underbrace{D_{KL}(Q(Z|X)\|P(Z|X))}_{\ge 0} \quad (307)$$

Therefore:

$$\log P(X) \ge \text{ELBO} = \mathbb{E}_{z\sim Q}[\log P(X|z)] - D_{KL}(Q(Z|X)\|P(Z)) \quad (308)$$

□

**Intuition:** The ELBO balances two objectives:

* **Reconstruction Term** $\mathbb{E}[\log P(X|Z)]$: Encourages the model to explain the observed data well by maximizing the likelihood of $X$ given the latent representation $Z$.

* **KL Regularization** $D_{KL}(Q(Z|X)\|P(Z))$: Prevents $Q(Z|X)$ from diverging too far from the prior $P(Z)$ (often a standard Gaussian), acting as a regularizer that encourages smooth, generalizable latent representations.

## 15.3 Proof of Differential Privacy Composition

**Theorem 15.3 (Basic Composition).** If $M_1$ is $(\epsilon_1, \delta_1)$-DP and $M_2$ is $(\epsilon_2, \delta_2)$-DP, then the composition $(M_1(D), M_2(D))$ is $(\epsilon_1 + \epsilon_2, \delta_1 + \delta_2)$-DP.

*Proof.* For any measurable sets $S_1, S_2$ and adjacent datasets $D, D'$: where $M_1, M_2$ are the mechanisms, and $S_1, S_2$ are subsets of the output space.

By $(\epsilon_1, \delta_1)$-DP of $M_1$:

$$P(M_1(D) \in S_1) \le e^{\epsilon_1} P(M_1(D') \in S_1) + \delta_1 \quad (309)$$

Let $p = P(M_1(D') \in S_1)$ and $q = P(M_2(D') \in S_2)$. Then:

$$P(M_1(D) \in S_1) \cdot P(M_2(D) \in S_2) \le (e^{\epsilon_1} p + \delta_1)(e^{\epsilon_2} q + \delta_2) \quad (310)$$

$$= e^{\epsilon_1+\epsilon_2} pq + e^{\epsilon_1} p\delta_2 + e^{\epsilon_2} q\delta_1 + \delta_1\delta_2 \quad (311)$$

Since $p, q \le 1$:

$$\le e^{\epsilon_1+\epsilon_2} pq + \delta_1 + \delta_2 \quad (312)$$

□

**Theorem 15.4 (Advanced Composition).** For $k$ applications of $(\epsilon, \delta)$-DP mechanisms, the composition is $(\epsilon', k\delta + \delta')$-DP where:

$$\epsilon' = \sqrt{2k \ln(1/\delta')} \cdot \epsilon + k\epsilon(e - 1) \quad (313)$$

where $\epsilon'$ is the composed privacy budget, $k$ is the number of compositions, and $\delta'$ is the slack parameter. For small $\epsilon$, this is approximately $O(\sqrt{k\epsilon})$ rather than $O(k\epsilon)$.
## 15.4 Convergence of SGD for Convex Objectives

We analyze the convergence of Stochastic Gradient Descent (SGD) for a convex, L-Lipschitz function $f$.

**Theorem 15.5.** Let $f$ be convex and L-Lipschitz. Let $x^* \in \arg\min f(x)$ with $\|x_1 - x^*\| \le R$. If we run SGD with step size $\eta_t = \frac{R}{L\sqrt{t}}$, then: where $T$ is the number of iterations, $R$ is the radius of the domain, and $L$ is the Lipschitz constant.

*Proof.* Let $g_t$ be the stochastic gradient at step $t$, such that $\mathbb{E}[g_t] = \nabla f(x_t)$. The update rule is $x_{t+1} = x_t - \eta g_t$. Consider the distance to the optimum:

$$ \|x_{t+1} - x^*\|^2 = \|x_t - \eta g_t - x^*\|^2 \quad (314) $$

$$ = \|x_t - x^*\|^2 - 2\eta \langle g_t, x_t - x^* \rangle + \eta^2 \|g_t\|^2 \quad (315) $$

where $\eta$ is the step size. Taking expectations:

$$ \mathbb{E}[\|x_{t+1} - x^*\|^2] = \mathbb{E}[\|x_t - x^*\|^2] - 2\eta \mathbb{E}[\langle \nabla f(x_t), x_t - x^* \rangle] + \eta^2 \mathbb{E}[\|g_t\|^2] \quad (316) $$

By convexity, $f(x^*) \ge f(x_t) + \langle \nabla f(x_t), x^* - x_t \rangle$, so $\langle \nabla f(x_t), x_t - x^* \rangle \ge f(x_t) - f(x^*)$. Also, assume bounded gradients $\mathbb{E}[\|g_t\|^2] \le L^2$.

$$ \mathbb{E}[\|x_{t+1} - x^*\|^2] \le \mathbb{E}[\|x_t - x^*\|^2] - 2\eta \mathbb{E}[f(x_t) - f(x^*)] + \eta^2 L^2 \quad (317) $$

Rearranging and summing over $t = 1...T$:

$$ 2\eta \sum_{t=1}^{T} \mathbb{E}[f(x_t) - f(x^*)] \le \|x_1 - x^*\|^2 - \mathbb{E}[\|x_{T+1} - x^*\|^2] + T\eta^2 L^2 \quad (318) $$

Since $\|x_{T+1} - x^*\|^2 \ge 0$ and $\|x_1 - x^*\|^2 \le R^2$:

$$ \sum_{t=1}^{T} \mathbb{E}[f(x_t) - f(x^*)] \le \frac{R^2}{2\eta} + \frac{T\eta L^2}{2} \quad (319) $$

Using Jensen's inequality on the LHS (convexity of $f$) and setting $\eta = \frac{R}{L\sqrt{T}}$ yields the result. $\square$

## 15.5 Proof of Rademacher Complexity Bound

**Theorem 15.6 (Rademacher Complexity Generalization Bound).** where $R(h)$ is the true risk, $\hat{R}(h)$ is the empirical risk, $\hat{R}_S$ is the empirical Rademacher complexity, and $n$ is the sample size.

*Proof Sketch.* The proof proceeds in three steps:

**Step 1: Symmetrization.** Using a ghost sample $S'$, we show:

$$ \mathbb{E}_S \left[ \sup_h (R(h) - \hat{R}_S(h)) \right] \le 2 \mathbb{E}_{S,\sigma} \left[ \sup_h \frac{1}{n} \sum_{i=1}^n \sigma_i \ell(h, z_i) \right] \quad (320) $$

where $\sigma_i$ are Rademacher variables. **Step 1: Symmetrization.** Using a ghost sample $S'$, we show:

$$ \mathbb{E}_S \left[ \sup_h (R(h) - \hat{R}_S(h)) \right] \le 2 \mathbb{E}_{S,\sigma} \left[ \sup_h \frac{1}{n} \sum_{i=1}^n \sigma_i \ell(h, z_i) \right] \quad (321) $$

**Step 2: Concentration.** Using McDiarmid's inequality with bounded differences $c_i = 2/n$:

$$ P \left( \sup_h |R(h) - \hat{R}(h)| \ge \mathbb{E}[\sup_h |R(h) - \hat{R}(h)|] + t \right) \le e^{-nt^2/2} \quad (322) $$

**Step 3: Combine.** Setting $t = \sqrt{\log(2/\delta)/(2n)}$ gives the result. $\square$
## 15.6 Shapley Value Axioms and Uniqueness

The Shapley value is the unique solution satisfying four axioms.

**Definition 15.1 (Shapley Axioms).** 1. **Efficiency:** $\sum_{i \in N} \phi_i(v) = v(N) - v(\emptyset)$ where $\phi_i(v)$ is the Shapley value of player $i$, $v$ is the value function, and $N$ is the set of players.

**Theorem 15.7 (Shapley Uniqueness).** The unique allocation satisfying all four axioms is:

$$ \phi_i(v) = \sum_{S \subseteq N \setminus \{i\}} \frac{|S|!(|N| - |S| - 1)!}{|N|!} [v(S \cup \{i\}) - v(S)] \quad (323) $$

where the sum is over all subsets $S$ not containing $i$.

*Proof Sketch.* The formula can be derived by considering all possible orderings of players joining the coalition. Player $i$ arrives at position $k$ with probability $1/|N|!$ for each of the $(|S|)! \times (|N| - |S| - 1)!$ orderings where $|S|$ players come before $i$. The marginal contribution is $v(S \cup \{i\}) - v(S)$, and averaging gives the formula.

Uniqueness follows from showing that any function satisfying the axioms must equal this formula. The key insight is that by linearity (additivity), it suffices to verify for "unanimity games" where $v(S) = 1$ iff $T \subseteq S$ for some coalition $T$. $\square$
# 16 Python Implementation Details

This appendix provides concrete implementation details for key algorithms discussed in the text, using PyTorch.

## 16.1 Fast Gradient Sign Method (FGSM)

```python
import torch
import torch.nn(functional as F)

def fgsmattack(image, epsilon, dataGrad):
    """Generate adversarial example using FGSM.

   Args:
        image: Original input image tensor
        epsilon: Maximum perturbation magnitude
        dataGrad: Gradient of loss w.r.t. input

   Returns:
        Perturbed adversarial image
    """
    # Collect the element-wise sign of the data gradient
    sign_dataGrad = dataGrad.sign()
    # Create perturbed image
    perturbed_image = image + epsilon * sign_dataGrad
    # Clip to maintain valid image range [0,1]
    perturbed_image = torch.clamp(perturbed_image, 0, 1)
    return perturbed_image

# Example usage:
# data requiresGrad = True
# output = model(data)
# loss = F-crossentropy(output, target)
# model.zeroGrad()
# loss backward()
# perturbed_data = fgsmattack(data, epsilon=0.03, dataGrad=data.grad)
```

<caption>Listing 4: FGSM Attack Implementation</caption>

## 16.2 Differential Privacy: Laplace Mechanism

```python
import numpy as np

def laplace_mechanism(true_value, sensitivity, epsilon):
    """Add Laplace noise to achieve epsilon-differential privacy.

   Args:
        true_value: True query result (scalar or array)
        sensitivity: L1 sensitivity of the query
        epsilon: Privacy parameter (smaller = more private)

   Returns:
        Noisy query result satisfying epsilon-DP
    """
    scale = sensitivity / epsilon
```
```python
noise = np.random.laplace(loc=0, scale=scale,
                         size(np.shape(true_value))
)
return true_value + noise

# Example: Private mean computation
def private_mean(data, lower_bound, upper_bound, epsilon):
    """Compute differentially private mean."""
    n = len(data)
    true_mean = np.mean(data)
    # Sensitivity of mean is (upper - lower) / n
    sensitivity = (upper_bound - lower_bound) / n
    return laplace_mechanism(true_mean, sensitivity, epsilon)
```

<caption>Listing 5: Laplace Mechanism for Differential Privacy</caption>

## 16.3 Fairness Regularization

```python
import torch

def demographic_parityakuarizer(y_pred, sensitive_attribute):
    """Differentiable proxy for Demographic Parity.

    Minimizes covariance between predictions and sensitive attribute,
    encouraging independence P(Y_hat | A) = P(Y_hat).

   Args:
        y_pred: Predicted probabilities, shape (batch_size,)
        sensitive_attribute: Binary attribute, shape (batch_size,)

    Returns:
        Squared covariance as regularization term
    """
    mean_pred = torch.mean(y_pred)
    mean_sens = torch.mean(sensitive_attribute.float())

    # Covariance measures linear dependence
covariance = torch.mean(
    (y_pred - mean_pred) * (sensitive_attribute.float() -
                          mean_sens)
)

return covariance ** 2

# Usage in training:
# total_loss = cross-proximity_loss + lambda_fair *
demographic_parity_Regularizer(y_pred, A)
```

<caption>Listing 6: Differentiable Fairness Regularization</caption>

## 16.4 Projected Gradient Descent Attack

```python
import torch
import torch.nn(functional as F

def pgd attack(model, images, labels, epsilon, alpha, num_iter):
    """Projected Gradient Descent adversarial attack.
    """
```
```python
train_loader: DataLoader with training data
epochs: Number of training epochs
epsilon, alpha, num_iter: PGD attack parameters
device: torch device (cuda or cpu)

model.to device()
optimization = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
criterion = nn.CrossEntropyLoss()

for epoch in range(epochs):
    model train()
    total_loss = 0
    correct = 0
    total = 0

    for batch_idx, (images, labels) in enumerate(train_loader):
        images, labels = images.to device(), labels.to device())

        # Generate adversarial examples
        model.eval() # Use eval mode for attack
        adv_images = pgdorsk model, images, labels, epsilon,
        alpha, num_iter)
        model train() # Back to train mode

        # Forward pass on adversarial examples
        optimizer.zeroGrad()
        outputs = model(adv_images)
        loss = criterion outputs, labels)

        # Backward pass
        loss backward()
        optimizer.step()

        # Statistics
        total_loss += loss.item()
        predicted = outputs.max(1)
        total += labels.size(0)
        correct += predicted.eq labels).sum().item()

    acc = 100.0 * correct / total
    print(f'Epoch {epoch}:
        Loss={total_loss/len(train_loader):.4f}, Adv
        Acc={acc:.2f}%')
```

<caption>Listing 8: Complete Adversarial Training</caption>

## 16.6 Conformal Prediction

```python
import numpy as np
from scipy.special import softmax

class ConformalClassifier:
    """Conformal prediction wrapper for any classifier."""

def __init__(self, base_model, alpha=0.1):
    """
   Args:
```
```python
base_model: Classifier with predict_proba method
alpha: Desired error rate (1 - coverage)

#"""
self.base_model = base_model
self.alpha = alpha
self.quantile = None

# def calibrate(self, X Cal, y Cal):
# """Calibrate conformal predictor on held-out calibration
# set."""
# Get softmax scores
probs = self.base_model Predict proba(X Cal)

# Compute non-conformity scores (1 - probability of true class)
n = len(y Cal)
scores = 1 - proba np Arange(n), y Cal]

# Compute quantile for coverage guarantee
# We want the k-th smallest score where k =
ceil((n+1) (1 - alpha))
k = int(np.ceil((n + 1) * (1 - self.alpha)))

if k > n:
    self.quantile = np.inf
else:
    # Get k-th smallest value (index k-1 in sorted array)
    self.quantile = np.sort(scores) [k-1]

return self

# def predict_set(self, X):
# """Return prediction sets with coverage guarantee."""
probs = self.base_model Predict proba(X)

# Include all classes with score > threshold
prediction_sets = []
for prob in proba:
    included = np where (1 - prob) <= self.quantile) [0]
    prediction_sets.append(set(included))

return prediction_sets

# Example usage:
# from sklearn.ensemble import RandomForestClassifier
# rf = RandomForestClassifier() .fit(X train, y train)
# conformal = ConformalClassifier(rf, alpha=0.1)
# conformal.calibrate(X Cal, y Cal)
# pred_sets = conformal Predict_set(X test)
# coverage = np mean([y test[i] in pred_sets[i] for i in
range(len(y test))])
```

<caption>Listing 9: Conformal Prediction for Classification</caption>

16.7 Integrated Gradients

```python
import torch
import numpy as np
```
def integrated Gradients(model, input par tson, baseline, target_class, steps=50):

"""Compute Integrated Gradients attribution.

Args:

model: Neural network

input tensor: Input to explain, shape (1, channels, height, width)

baseline: Baseline input (typically zeros)

target_class: Class to explain

steps: Number of interpolation steps

Returns:

Attribution map with same shape as input

###

# Generate interpolated inputs

scaled inputs = [baseline + (float(i) / steps) * (input tensor - baseline)

for i in range(steps + 1)]

scaled inputs = torch.cat(scaled inputs, dim=0)

scaled inputs requires grad = True

# Forward pass

outputs = model(scaled inputs)

# Backward pass for target class

target outputs = outputs [:, target class]

grads = torch .autograd .grad(target outputs .sum() , scaled inputs )[0]

# Average gradients

avg grads = grads .mean(dim=0, keepdim =True)

# Scale by (input - baseline)

integrated grad = (input tensor - baseline) * avg grads

return integrated grad

def visualize Attribution (attribution, original image):

"""Visualize attribution as heatmap overlay."""

import matplotlib .pyplot as plt

# Sum over channels for visualization

attr_sum =attribution .squeeze() .sum(dim=0) .cpu() .numpy()

# Normalize

attr_sum = (attr_sum - attr_sum .min()) / (attr_sum .max() -
attr_sum .min() + 1e-8)

fig, axes = plt .subplots(1, 3, figsize=(12, 4))

axes [0].imshow (original_image .permute(1, 2, 0) .cpu() .numpy())

axes [0].set_title('Original')

axes [1].imshow (attr_sum, cmap='hot')

axes [1].set_title('Attribution')

axes [2].imshow (original_image .permute(1, 2, 0) .cpu() .numpy())

axes [2].imshow (attr_sum, cmap='hot', alpha=0.5)

axes [2].set_title('Overlay')

for ax in axes:
```python
58 ax.axis('off')
59 plt.tight_layout()
60 return fig
```

<caption>Listing 10: Integrated Gradients Attribution</caption>
# 17 Comprehensive Evaluation Metrics

This appendix provides a reference for evaluation metrics used throughout the document.

## 17.1 Robustness Metrics

**Definition 17.1** (Clean Accuracy). The standard classification accuracy on unperturbed test data:

$$ \mathrm{Acc}_{\mathrm{clean}} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(f(x_i) = y_i) \qquad (324) $$

where $N$ is the number of samples, $f(x_i)$ is the predicted label, $y_i$ is the true label, and $\mathbb{I}$ is the indicator function.

**Definition 17.2** (Robust Accuracy). Accuracy under adversarial attack with perturbation budget $\epsilon$:

$$ \mathrm{Acc}_{\mathrm{robust}}(\epsilon) = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}\left(\min_{\|\delta\| \le \epsilon} f(x_i + \delta) = y_i\right) \qquad (325) $$

where $\delta$ is the perturbation vector. In practice, this is approximated by the accuracy under a specific attack (e.g., PGD).

**Definition 17.3** (Certified Robust Accuracy). The fraction of samples with a certified radius $\ge \epsilon$:

$$ \mathrm{Acc}_{\mathrm{certified}}(\epsilon) = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(r(x_i) \ge \epsilon) \qquad (326) $$

where $r(x_i)$ is the certified radius from formal verification or randomized smoothing.

**Definition 17.4** (Attack Success Rate (ASR)). For targeted attacks, the fraction where the adversarial example achieves the target class:

$$ \mathrm{ASR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(f(x_i + \delta_i) = y_{\mathrm{target}}) \qquad (327) $$

where $y_{\mathrm{target}}$ is the target class.

## 17.2 Fairness Metrics

**Definition 17.5** (Demographic Parity Difference (DPD)).

$$ \mathrm{DPD} = |P(\hat{Y} = 1 | A = 0) - P(\hat{Y} = 1 | A = 1)| \qquad (328) $$

where $\hat{Y}$ is the predicted label, and $A$ is the sensitive attribute. Values closer to 0 indicate fairer predictions with respect to demographic parity.

**Definition 17.6** (Equalized Odds Difference (EOD)).

$$ \mathrm{EOD} = \frac{1}{2} (|\mathrm{TPR}_{A=0} - \mathrm{TPR}_{A=1}| + |\mathrm{FPR}_{A=0} - \mathrm{FPR}_{A=1}|) \qquad (329) $$

where TPR is the true positive rate, and FPR is the false positive rate.
**Definition 17.7** (Equal Opportunity Difference).

$$ EO = |P(\hat{Y} = 1|Y = 1, A = 0) - P(\hat{Y} = 1|Y = 1, A = 1)| \qquad (330) $$

where Y is the true label. Focuses only on the true positive rate difference.

**Definition 17.8** (Predictive Parity). Requires equal positive predictive values:

$$ P(Y = 1|\hat{Y} = 1, A = 0) = P(Y = 1|\hat{Y} = 1, A = 1) \qquad (331) $$

where $P(Y = 1|\hat{Y} = 1)$ is the precision.

**Definition 17.9** (Disparate Impact Ratio).

$$ DIR = \frac{P(\hat{Y} = 1|A = \text{disadvantaged})}{P(\hat{Y} = 1|A = \text{advantaged})} \qquad (332) $$

where values less than 1 indicate disadvantage. The "80% rule" considers $DIR < 0.8$ as evidence of disparate impact.

## 17.3 Privacy Metrics

**Definition 17.10** (Privacy Budget $(\epsilon, \delta)$). The parameters of $(\epsilon, \delta)$-differential privacy. Smaller $\epsilon$ implies stronger privacy. The parameter $\delta$ represents the probability of a privacy breach.

**Definition 17.11** (Reconstruction Error). For model inversion attacks, the mean squared error between reconstructed and original data:

$$ RE = \frac{1}{N} \sum_{i=1}^{N} \|x_i - \hat{x}_i\|^2 \qquad (333) $$

where $\hat{x}_i$ is the reconstructed input.

**Definition 17.12** (Maximum Calibration Error (MCE)).

$$ MCE = \max_{b \in \{1, \dots, B\}} |\text{acc}(B_b) - \text{conf}(B_b)| \qquad (334) $$

where MCE measures the worst-case calibration error.

**Definition 17.13** (Brier Score).

$$ BS = \frac{1}{N} \sum_{i=1}^{N} \sum_{c=1}^{C} (p_{ic} - y_{ic})^2 \qquad (335) $$

where $p_{ic}$ is the predicted probability for class c and $y_{ic}$ is the one-hot indicator.

**Definition 17.14** (Negative Log-Likelihood (NLL)).

$$ NLL = -\frac{1}{N} \sum_{i=1}^{N} \log p_{iy_i} \qquad (336) $$

where $p_{iy_i}$ is the predicted probability for the true class $y_i$. where $p_{ic}$ is the predicted probability for class c and $y_{ic}$ is the one-hot indicator.

**Definition 17.15** (Negative Log-Likelihood (NLL)).

$$ NLL = -\frac{1}{N} \sum_{i=1}^{N} \log p_{iy_i} \qquad (337) $$

Also known as cross-entropy loss on test data.
## 17.4 OOD Detection Metrics

**Definition 17.16 (AUROC).** Area Under the ROC Curve; measures the probability that an in-distribution sample scores higher than an OOD sample.

**Definition 17.17 (FPR at 95% TPR (FPR95)).** False positive rate when true positive rate is 95%; measures the fraction of OOD samples incorrectly classified as in-distribution when 95% of in-distribution samples are correctly identified.

**Definition 17.18 (AUPR).** Area Under the Precision-Recall Curve; more informative when classes are imbalanced.

## 17.5 Explainability Metrics

**Definition 17.19 (Faithfulness (Deletion)).** Measure how prediction changes when important features are removed:

$$ \text{Faithfulness}_{\text{del}} = \text{AUC} \left( \{f(x_{-S_k})\}_{k=1}^K \right) \qquad (338) $$

where $x_{-S_k}$ removes the top-k most important features.

**Definition 17.20 (Faithfulness (Insertion)).**

$$ \text{Faithfulness}_{\text{ins}} = \text{AUC} \left( \{f(x_{+S_k})\}_{k=1}^K \right) \qquad (339) $$

where $x_{+S_k}$ starts from baseline and adds top-k features.

**Definition 17.21 (Stability).** For input $x$ and small perturbation $\delta$:

$$ \text{Stability} = \frac{1}{M} \sum_{j=1}^{M} \text{sim}(\phi(x), \phi(x + \delta_j)) \qquad (340) $$

where $\phi$ is the explanation function and $\text{sim}$ is a similarity metric.
# 18 Glossary of Key Terms

## Adversarial Example

An input crafted by applying small, often imperceptible perturbations to cause a model to make incorrect predictions with high confidence.

## Aleatoric Uncertainty

Inherent randomness in the data that cannot be reduced by collecting more data (e.g., sensor noise, labeling ambiguity).

## Calibration
A property where predicted probabilities match actual outcome frequencies: among predictions with confidence *p*, the fraction correct should be *p*.

## Certified Robustness

A mathematical guarantee that a model's prediction is invariant to all perturbations within a specified radius.

## Conformal Prediction

A distribution-free method for constructing prediction sets with finite-sample coverage guarantees.

## Counterfactual Explanation

An explanation of the form "the prediction would have been $Y'$ if the input had been $X'$," providing actionable recourse.

## Demographic Parity

A group fairness criterion requiring equal positive prediction rates across protected groups: $P(\hat{Y} = 1 | A = 0) = P(\hat{Y} = 1 | A = 1)$.

## Differential Privacy

A mathematical framework quantifying privacy loss, ensuring that the output of an algorithm is approximately independent of any single individual's data.

## Distribution Shift

A change between the training and deployment data distributions, including covariate shift, label shift, and concept drift.

## ELBO
Evidence Lower Bound; a tractable lower bound on log-likelihood used in variational inference: $\log p(x) \ge E_q[\log p(x|z)] - D_{KL}(q(z|x)||p(z))$.

## Epistemic Uncertainty

Uncertainty arising from limited data or model capacity that can be reduced with more data or better models.

## Equalized Odds

A fairness criterion requiring equal true positive and false positive rates across groups.

## Feature Attribution

Assignment of importance scores to input features indicating their contribution to a model's prediction.

## Federated Learning

A distributed learning paradigm where multiple parties collaboratively train a model without sharing raw data.
**FGSM** Fast Gradient Sign Method; a one-step adversarial attack using the sign of the loss gradient: $x_{adv} = x + \epsilon \cdot \text{sign}(\nabla_x \ell)$.

**Hallucination** Generation of plausible-sounding but factually incorrect content by language models.

**Individual Fairness**

A fairness criterion requiring similar individuals to receive similar predictions: $d_Y(f(x), f(x')) \le L \cdot d_X(x, x')$.

**Integrated Gradients**

A feature attribution method satisfying completeness by integrating gradients along a path from baseline to input.

**Jailbreak** An adversarial prompt designed to circumvent safety guardrails and elicit prohibited outputs from language models.

**KL Divergence**

Kullback-Leibler divergence; a measure of difference between probability distributions: $D_{KL}(P||Q) = E_P[\log(P/Q)]$.

**Laplace Mechanism**

A differential privacy mechanism adding Laplace noise scaled to query sensitivity: $M(x) = f(x) + \text{Lap}(\Delta f/\epsilon)$.

**Lipschitz Continuity**

A function $f$ is $L$-Lipschitz if $|f(x) - f(x')| \le L\|x - x'\|$ for all $x, x'$.

**LIME** Local Interpretable Model-agnostic Explanations; explains predictions by fitting a local linear model.

**Model Card** Standardized documentation describing a model's capabilities, limitations, and performance across subgroups.

**OOD Detection**

Out-of-Distribution detection; identifying inputs that differ significantly from the training distribution.

**PAC Learning** Probably Approximately Correct learning; a theoretical framework for computational learning theory.

**PGD** Projected Gradient Descent; an iterative adversarial attack projecting perturbations onto an $\epsilon$-ball.

**Privacy Budget**

The cumulative privacy loss parameter $\epsilon$ consumed by multiple differentially private queries.

**Rademacher Complexity**

A measure of a hypothesis class's ability to fit random noise, used in generalization bounds.

**Randomized Smoothing**

A technique for certified robustness by classifying based on majority vote under Gaussian noise.

**RLHF** Reinforcement Learning from Human Feedback; training language models to align with human preferences.
**Robustness** The property that a model's predictions remain correct under input perturbations or distribution shift.

**Sensitivity** In differential privacy, the maximum change in a query's output when a single data point changes.

**SHAP Values** SHapley Additive exPlanations; feature attributions based on Shapley values from cooperative game theory.

**Structural Causal Model (SCM)** A formal model specifying causal relationships via structural equations and a directed acyclic graph.

**VC Dimension** Vapnik-Chervonenkis dimension; the maximum number of points that can be shattered by a hypothesis class.

**Watermarking** Embedding detectable patterns in AI-generated content to enable attribution and detection.

**Excess Risk Decomposition** Breakdown of generalization error into approximation error (bias from hypothesis class) and estimation error (variance from finite samples).

**Approximation Error** Error due to the hypothesis class not containing the true function.

**Estimation Error** Error due to estimating parameters from finite data.

**Bias-Variance Decomposition** Decomposition of expected error into irreducible noise, bias (systematic error), and variance (sensitivity to data).

**Irreducible Error** Inherent noise in the data-generating process that no model can eliminate.

**Uniform Convergence** Property that empirical risk converges to true risk uniformly over the hypothesis class.

**Sauer's Lemma** Bounds the growth function of a hypothesis class in terms of VC dimension.

**Rademacher Complexity** Measure of a class's ability to fit random noise, used for generalization bounds.

**Double Descent Phenomenon** Test error decreases again after interpolation threshold in over-parameterized regimes.

**Under-parameterized Regime** Model capacity insufficient to fit training data, leading to high bias.

**Critical Regime** Capacity just enough for interpolation, causing high variance.
**Over-parameterized Regime**

Capacity exceeds data needs, allowing implicit regularization.

**PAC Learning** Framework ensuring algorithms learn with high probability and low error.

**Agnostic PAC Learning**

PAC learning without assuming the true function is in the hypothesis class.

**Margin-Based Generalization Bounds**

Bounds using classifier margins for better generalization analysis.

**Algorithmic Stability**

Property that small data changes lead to small output changes, enabling generalization.

**Linearity Hypothesis**

Adversarial vulnerability arises from local linearity in high-dimensional spaces.

**Concentration of Measure**

Probability mass concentrates near low-dimensional structures in high dimensions.

**Isoperimetric Inequality**

Bounds on set expansion in high dimensions.

**White-box Attacks**

Attacks with full model access (weights, gradients).

**Black-box Attacks**

Attacks with only input-output access.

**Gray-box Attacks**

Attacks with partial model knowledge.

**Targeted Attacks**

Misclassify to a specific wrong class.

**Untargeted Attacks**

Misclassify to any wrong class.

**$L_\infty$ Constraint** Perturbations bounded by maximum per-dimension change.

**$L_2$ Constraint** Perturbations bounded by Euclidean norm.

**$L_0$ Constraint** Perturbations bounded by number of modified dimensions.

**Physical Attacks**

Perturbations realizable in the physical world (e.g., stickers).

**First-Order Adversaries**

Attacks using gradient information.

**Zeroth-Order Gradient Estimation**

Estimating gradients via function queries.

**Random Direction Gradient Estimation**

Efficient gradient estimation using random projections.
**NES (Natural Evolution Strategies) Attack**

Black-box attack using evolutionary optimization.

**Square Attack** Query-efficient black-box attack using random square perturbations.

**Adversarial Training**

Training on adversarial examples to improve robustness.

**Robust Optimization**

Min-max optimization for worst-case perturbations.

**Zero-Sum Game**

Formulation of adversarial training as a game between trainer and attacker.

**Minimax Theorem**

Order of min-max does not matter under convexity.

**Danskin's Theorem**

Gradient of max function equals gradient at optimum.

**Certified Robustness**

Mathematical guarantee of invariance to perturbations.

**Randomized Smoothing**

Converts classifiers to certifiably robust via noise addition.

**Smoothed Classifier**

Classifier based on majority vote under Gaussian noise.

**Natural Distribution Shifts**

Legitimate changes in data distribution (e.g., covariate shift).

**Domain Adaptation**

Adapting models to new domains without labels.

**Importance Weighting**

Reweighting samples to correct for distribution shift.

**Domain Adversarial Training (DANN)**

Adversarial training to learn domain-invariant features.

**Gradient Reversal Layer (GRL)**

Layer that negates gradients for adversarial training.

**Invariant Risk Minimization (IRM)**

Learning features invariant across environments.

**Distributionally Robust Optimization (DRO)**

Optimizing for worst-case distribution in an uncertainty set.

**f-divergence ball**

Uncertainty set defined by f-divergence from empirical distribution.

**Wasserstein ball**

Uncertainty set defined by Wasserstein distance.

**Group DRO**

Uncertainty set as convex hull of group distributions.
**Philosophical Foundations**

Ethical theories underlying fairness (e.g., Rawlsian justice).

**Rawlsian Justice**

Inequalities should benefit the least advantaged.

**Luck Egalitarianism**

Distinguishes circumstantial vs. chosen inequalities.

**Anti-subordination Theories**

Focus on reducing group-based disadvantages.

**Group Fairness**

Fairness across demographic groups.

**Individual Fairness**

Similar individuals receive similar predictions.

**Impossibility Results**

Theorems showing incompatibility of fairness definitions.

**Intersectionality**

Fairness across multiple intersecting identities.

**Multi-Group Fairness**

Fairness in settings with many groups.

**Disparity Metrics**

Measures of fairness violations (e.g., DPD, DIR).

**Calibration Metrics**

Measures of probability accuracy across groups.

**Pre-processing Methods**

Fairness interventions before training.

**Reweighing**

Adjusting sample weights for fairness.

**Fair Representation Learning**

Learning unbiased latent representations.

**In-processing Methods**

Fairness constraints during training.

**Adversarial Debiasing**

Adversarial training to remove bias.

**Lagrangian Relaxation**

Converting constraints to penalties.

**Post-processing Methods**

Adjusting predictions after training.

**Threshold Optimization**

Group-specific decision thresholds.

**Calibration Adjustment**

Correcting probability estimates.
**Causal Fairness**

Fairness defined via causal graphs.

**Counterfactual Fairness**

Predictions unchanged under sensitive attribute interventions.

**Path-Specific Fairness**

Fairness along specific causal pathways.

**Proxy Variable**

Feature correlated with sensitive attribute, enabling indirect discrimination.

**Kleinberg-Chouldechova Impossibility Theorem**

Incompatibility of calibration and separation under different base rates.

**Mechanisms for Fair Learning**

Techniques for achieving fairness in practice.

**Differential Privacy (DP)**

Framework quantifying privacy loss via output indistinguishability.

**Privacy Loss Random Variable**

Random variable measuring per-sample privacy cost.

**Laplace Mechanism**

Adding Laplace noise for DP.

**Gaussian Mechanism**

Adding Gaussian noise for DP.

**Exponential Mechanism**

Sampling from utility-scored outputs.

**Composition**

Combining multiple DP mechanisms.

**Renyi Differential Privacy (RDP)**

Composition-friendly DP variant.

**DP-SGD**

Differentially private stochastic gradient descent.

**Privacy Amplification by Subsampling**

Privacy increases with subsampling.

**Moments Accountant**

Tight composition bounds via moment tracking.

**PATE**

Private aggregation via teacher-student framework.

**Secure Multi-Party Computation (MPC)**

Joint computation without revealing inputs.

**Secret Sharing**

Splitting secrets for MPC.

**Shamir's Secret Sharing**

Threshold secret sharing scheme.

**Additive Secret Sharing**

Simple sharing for linear operations.
**Garbled Circuits**

Secure two-party computation via encrypted circuits.

**Homomorphic Encryption (HE)**

Computation on encrypted data.

**Local Differential Privacy**

Privacy where users randomize their own data.

**Randomized Response**

Mechanism for local DP on binary data.

**Reconstruction Attacks**

Reconstructing data from DP outputs.

**Dinur-Nissim Theorem**

Limits on answering many queries privately.

**Membership Inference Attacks (MIA)**

Detecting if a sample was in training data.

**Model Inversion Attacks**

Reconstructing inputs from model outputs.

**Attribute Inference Attacks**

Inferring sensitive attributes from predictions.

**Explainable AI (XAI)**

Methods for making AI decisions understandable.

**Interpretability**

Inherent understandability of a model.

**Intrinsic vs. Post-hoc**

Interpretable by design vs. analyzed after training.

**Global vs. Local**

Explanations for the whole model vs. single predictions.

**Model-Agnostic vs. Model-Specific**

Works on any model vs. exploits internals.

**Feature Attribution Methods**

Assigning importance scores to features.

**LIME**

Local explanations via interpretable approximations.

**Gradient-Based Methods**

Using gradients for attribution.

**Saliency Maps** Visualizing gradient magnitudes.

**Integrated Gradients (IG)**

Path-integral attribution method.

**Concept-Based Explanations**

Explanations in terms of human concepts.
**Concept Bottleneck Models (CBM)**

Models predicting via interpretable concepts.

**TCAV**

Testing sensitivity to high-level concepts.

**Evaluating Explanations**

Metrics for explanation quality.

**Fidelity**

How well explanation matches model behavior.

**Robustness**

Stability of explanations to perturbations.

**Sanity Checks**

Verifying explanations depend on model parameters.

**Algorithmic Recourse**

Methods for users to change predictions.

**Counterfactual Explanations**

"What if" scenarios for different outcomes.

**Wachter's Method**

Optimizing for minimal counterfactual changes.

**Actionability Constraints**

Restrictions on changeable features.

**DICE**

Generating diverse counterfactuals.

**Causal Counterfactuals**

Counterfactuals respecting causal structure.

**Recourse Robustness**

Counterfactuals valid under model changes.

**Algorithmic Recourse via Integer Programming**

MIP for tree-based recourse.

**Ladder of Causation**

Levels of causal reasoning (association, intervention, counterfactuals).

**Association**

Observing correlations.

**Intervention**

Acting to change variables.

**Counterfactuals**

Reasoning about hypothetical worlds.

**d-Separation**

Graphical criterion for conditional independence.

**Do-Operator**

Notation for interventions.

**Back-Door Criterion**

Conditions for identifying causal effects.

**Adjustment Formula**

Formula for causal effect estimation.

**Do-Calculus**

Rules for transforming causal queries.
**Counterfactuals**

Queries about alternative outcomes.

**Federated Learning (FL)**

Distributed training without data sharing.

**FedAvg**

Standard FL algorithm via model averaging.

**Statistical Heterogeneity**

Non-IID data across clients.

**FedProx**

FL with proximal regularization for heterogeneity.

**Personalized Federated Learning**

Client-specific model tailoring.

**Local Fine-Tuning**

Global model fine-tuned per client.

**Personalized Layers**

Shared base with client-specific heads.

**MAML-based Personalization**

Meta-learning for personalization.

**Mixture of Experts**

Client-specific expert combinations.

**Gradient Compression**

Reducing communication via quantization/sparsification.

**Quantization**

Reducing gradient precision.

**Sparsification**

Transmitting only top gradients.

**Low-Rank Approximation**

Compressing gradients via SVD.

**Secure Aggregation**

MPC for aggregating without revealing updates.

**Byzantine Attacks**

Malicious clients poisoning FL.

**Krum**

Robust aggregation via nearest neighbors.

**Coordinate-wise Median**

Median per gradient coordinate.

**Trimmed Mean**

Averaging after removing outliers.

**Alignment Problem**

Ensuring RL agents pursue intended goals.

**Reward Hacking**

Exploiting reward function loopholes.
**Specification Gaming**

Achieving reward via unintended means.

**Coast Runners Effect**

Optimizing for speed by not finishing the race.

**Negative Side Effects**

Unintended consequences of actions.

**Reward Tampering**

Modifying the reward signal.

**Constrained Markov Decision Processes (CMDP)**

MDPs with safety constraints.

**Lagrangian Relaxation**

Penalty method for constraints.

**Constrained Policy Optimization (CPO)**

Trust-region method for CMDPs.

**Safe Exploration**

Exploration without violating safety.

**Safety Layers**

Real-time safety enforcement.

**Control Barrier Functions (CBF)**

Functions ensuring forward invariance.

**Lyapunov-Based Safe Learning**

Stability via Lyapunov functions.

**Shielding**

Verified backup policies for safety.

**Value Alignment**

Learning aligned reward functions.

**Inverse Reinforcement Learning (IRL)**

Inferring rewards from behavior.

**Maximum Entropy IRL**

Probabilistic IRL variant.

**Feature Matching IRL**

Matching feature expectations.

**Preference Learning**

Learning from human preferences.

**Bradley-Terry Model**

Probabilistic preference model.

**Reinforcement Learning from Human Feedback (RLHF)**

Aligning via human feedback.

**Direct Preference Optimization (DPO)**

Preference-based policy optimization.
**Reward Shaping**

Adding auxiliary rewards.

**Iterated Amplification**

Recursively improving alignment.

**AI Safety via Debate**

Using debates for better decisions.

**Out-of-Distribution (OOD) Detection**

Identifying inputs outside training distribution.

**Uncertainty Quantification**

Measuring model confidence.

**Distribution Shift**

Changes between train and test distributions.

**Covariate Shift**

Input distribution changes.

**Label Shift**

Class proportion changes.

**Concept Drift**

Relationship between inputs/outputs changes.

**Domain Shift**

Simultaneous marginal and conditional changes.

**Softmax-Based Methods**

OOD detection via softmax probabilities.

**Maximum Softmax Probability (MSP)**

Confidence via max softmax.

**Temperature Scaling**

Adjusting softmax for better calibration.

**ODIN**

OOD detection with input preprocessing.

**Energy-Based OOD Detection**

Using free energy for detection.

**Deep Ensembles**

Uncertainty via model ensembles.

**Mahalanobis Distance-Based Detection**

Distance to class centroids.

**Bayesian Neural Networks**

Uncertainty via posterior distributions.

**Variational Inference**

Approximating posteriors.

**Monte Carlo Dropout**

Dropout for uncertainty estimation.

**Laplace Approximation**

Gaussian posterior approximation.
**Aleatoric Uncertainty**

Data-inherent noise.

**Epistemic Uncertainty**

Model uncertainty.

**Calibration**

Probabilities match frequencies.

**Expected Calibration Error (ECE)**

Average calibration gap.

**Reliability Diagrams**

Plotting accuracy vs. confidence.

**Post-hoc Calibration Methods**

Adjusting probabilities after training.

**Platt Scaling**

Logistic calibration.

**Isotonic Regression**

Non-parametric calibration.

**Conformal Prediction**

Distribution-free prediction sets.

**Selective Prediction (Rejection)**

Abstaining on uncertain inputs.

**Risk-Coverage Trade-off**

Balancing accuracy and coverage.
19 List of Notation

<table><thead><tr><th>Symbol</th><th>Description</th></tr></thead><tbody><tr><td>X</td><td>Input space</td></tr><tr><td>Y</td><td>Output/label space</td></tr><tr><td>D</td><td>Data distribution over X × Y</td></tr><tr><td>S = {(x<sub>i</sub>, y<sub>i</sub>)<sub>i=1</sub><sup>n</sup></td><td>Training dataset of n samples</td></tr><tr><td>f<sub>θ</sub>: X → Y</td><td>Model parameterized by θ</td></tr><tr><td>ℓ(f(x), y)</td><td>Loss function measuring prediction error</td></tr><tr><td>R(f)</td><td>True/expected risk: E<sub>(x,y)~D</sub>[ℓ(f(x), y)]</td></tr><tr><td>R̂(f)</td><td>Empirical risk: 1/n Σ<sub>i=1</sub><sup>n</sup> ℓ(f(x<sub>i</sub>), y<sub>i</sub>)</td></tr><tr><td>H</td><td>Hypothesis class</td></tr><tr><td>d<sub>VC</sub></td><td>VC dimension</td></tr><tr><td>R<sub>n</sub>(H)</td><td>Rademacher complexity</td></tr><tr><td>ε</td><td>Perturbation budget (adversarial) or privacy parameter (DP)</td></tr><tr><td>δ</td><td>Probability of failure (in PAC learning or DP)</td></tr><tr><td>∇<sub>x</sub></td><td>Gradient with respect to x</td></tr><tr><td>||·||<sub>p</sub></td><td>L<sub>p</sub> norm</td></tr><tr><td>B<sub>ε</sub>(x)</td><td>ε-ball around x: {x': ||x' - x|| ≤ ε}</td></tr><tr><td>Π<sub>C</sub></td><td>Projection onto set C</td></tr><tr><td>A</td><td>Sensitive/protected attribute</td></tr><tr><td>ŷ</td><td>Predicted label</td></tr><tr><td>TPR</td><td>True positive rate: P(ŷ = 1|Y = 1)</td></tr><tr><td>FPR</td><td>False positive rate: P(ŷ = 1|Y = 0)</td></tr><tr><td>DPD</td><td>Demographic parity difference</td></tr><tr><td>EOD</td><td>Equalized odds difference</td></tr><tr><td>M</td><td>Randomized mechanism (DP)</td></tr><tr><td>Δf</td><td>Sensitivity of function f</td></tr><tr><td>Lap(b)</td><td>Laplace distribution with scale b</td></tr><tr><td>D<sub>α</sub>(P||Q)</td><td>Rényi divergence of order α</td></tr><tr><td>G = (V, E)</td><td>Causal graph with vertices V and edges E</td></tr><tr><td>do(X = x)</td><td>Intervention setting X to value x</td></tr><tr><td>Y<sub>x</sub></td><td>Counterfactual: value of Y under intervention do(X = x)</td></tr><tr><td>Pa(X)</td><td>Parent nodes of X in causal graph</td></tr><tr><td>K</td><td>Number of clients (federated learning)</td></tr><tr><td>E</td><td>Number of local epochs</td></tr><tr><td>η</td><td>Learning rate</td></tr><tr><td>σ</td><td>Standard deviation of Gaussian noise</td></tr><tr><td>C</td><td>Gradient clipping threshold</td></tr><tr><td>E[·]</td><td>Expectation</td></tr><tr><td>Var[·]</td><td>Variance</td></tr><tr><td>Cov[·, ·]</td><td>Covariance</td></tr><tr><td>N(μ, σ<sup>2</sup>)</td><td>Gaussian distribution</td></tr><tr><td>Dir(α)</td><td>Dirichlet distribution</td></tr><tr><td>D<sub>KL</sub>(P||Q)</td><td>KL divergence from Q to P</td></tr><tr><td>H(X)</td><td>Entropy of X</td></tr></tbody></table>
<table><thead><tr><th>Symbol</th><th>Description</th></tr></thead><tbody><tr><td>I(X; Y)</td><td>Mutual information between X and Y</td></tr><tr><td>α</td><td>Class K function parameter (CBF); significance level (Conformal); interpolation parameter (IG); inner loop learning rate (MAML); FPR (Fairness); order for Renyi divergence (RDP)</td></tr><tr><td>β</td><td>TPR (Fairness); trade-off parameter (TRADES); KL coefficient (RLHF/DPO); regularization strength (IRM)</td></tr><tr><td>γ</td><td>Discount factor (CMDP); interpolation parameter (IG)</td></tr><tr><td>δ</td><td>Trust region size (CPO); failure probability (DP); perturbation (Adversarial); dual variable (CPO)</td></tr><tr><td>η</td><td>Perturbation (Linearity Hypothesis); learning rate (FedAvg)</td></tr><tr><td>θ</td><td>Model parameters; policy parameters (RL)</td></tr><tr><td>κ</td><td>Confidence parameter (C&W); confidence function (Selective Prediction)</td></tr><tr><td>λ</td><td>Lagrange multiplier; regularization parameter; moment order (Moments Accountant); hyperparameters (DICE)</td></tr><tr><td>μ</td><td>Proximal term (FedProx); step size (Zeroth-Order); class mean (Mahalanobis)</td></tr><tr><td>ν</td><td>Dual variable (CPO)</td></tr><tr><td>ξ</td><td>Random binary vector (Quantization); LIME objective</td></tr><tr><td>π</td><td>Policy (RL); mixing coefficients (Mixture of Experts); proximity kernel (LIME)</td></tr><tr><td>ρ</td><td>Radius (DRO)</td></tr><tr><td>σ</td><td>Noise scale (Gaussian Mechanism); standard deviation (Gaussian noise); shield (Shielding)</td></tr><tr><td>τ</td><td>Threshold (Selective Prediction); temperature (Temperature Scaling)</td></tr><tr><td>φ</td><td>Coverage (Selective Prediction); shared layers (Personalized Layers); Shapley value</td></tr><tr><td>Φ</td><td>Potential function (Reward Shaping); inverse CDF (Randomized Smoothing)</td></tr><tr><td>χ</td><td>Chi-squared divergence (DRO)</td></tr><tr><td>ψ</td><td>Personal layers (Personalized Layers)</td></tr><tr><td>A</td><td>Adversary (Security); actions (MDP)</td></tr><tr><td>B</td><td>Uncertainty set (Robust Recourse); bins (ECE)</td></tr><tr><td>C</td><td>Safe set (CBF); prediction set (Conformal Prediction)</td></tr><tr><td>E</td><td>Training environments (IRM)</td></tr><tr><td>F</td><td>Structural equations (SCM)</td></tr><tr><td>G</td><td>Graph (manipulated, e.g., G<sub>X</sub>)</td></tr><tr><td>I</td><td>Immutable/increasing features (Actionability)</td></tr><tr><td>M</td><td>Randomized mechanism (DP); SCM</td></tr><tr><td>P</td><td>Probability distribution; protocol (MPC)</td></tr><tr><td>R</td><td>Risk; output space (DP)</td></tr><tr><td>S</td><td>States (MDP); perturbation set; unsafe states</td></tr><tr><td>U</td><td>Uncertainty set (DRO); unsafe states</td></tr><tr><td>Z</td><td>Latent space; exogenous variables (SCM)</td></tr><tr><td>w</td><td>Model weights (Bayesian NN)</td></tr><tr><td>z</td><td>Dropout masks (Monte Carlo Dropout)</td></tr></tbody></table>
<table><thead><tr><th>Symbol</th><th>Description</th></tr></thead><tbody><tr><td>W<sub>p</sub>(Q, P)</td><td>Wasserstein distance</td></tr><tr><td>D<sub>α</sub>(P||Q)</td><td>Renyi divergence</td></tr><tr><td>D<sub>f</sub>(Q||P)</td><td>f-divergence</td></tr><tr><td>sgn(·)</td><td>Sign function</td></tr><tr><td>⊙</td><td>Element-wise multiplication</td></tr><tr><td>diag(·)</td><td>Diagonal matrix</td></tr><tr><td>Top<sub>k</sub></td><td>Top-k sparsification</td></tr><tr><td>Quantile<sub>p</sub></td><td>p-quantile</td></tr><tr><td>acc(B<sub>b</sub>)</td><td>Accuracy in bin b</td></tr><tr><td>conf(B<sub>b</sub>)</td><td>Confidence in bin b</td></tr><tr><td>yloss</td><td>Loss for target class</td></tr><tr><td>dpp_diversity</td><td>Diversity metric</td></tr><tr><td>f<sub>x</sub>(S)</td><td>Prediction with feature subset S</td></tr><tr><td>S<sub>MSP</sub></td><td>Maximum softmax probability score</td></tr><tr><td>S<sub>C,k</sub></td><td>Conceptual sensitivity</td></tr><tr><td>J<sub>R</sub>/J<sub>C</sub></td><td>Expected return/cost</td></tr><tr><td>L<sub>π<sub>k</sub></sub></td><td>Surrogate objective</td></tr><tr><td>A<sup>π<sub>k</sub></sup></td><td>Advantage function</td></tr><tr><td>D&#773;<sub>KL</sub></td><td>Average KL divergence</td></tr><tr><td>q&#770;</td><td>Conformal quantile</td></tr><tr><td>y&#770;</td><td>Predicted label (ensemble or MC dropout)</td></tr><tr><td>R(τ)</td><td>Selective risk</td></tr><tr><td>Z</td><td>Privacy loss random variable</td></tr><tr><td>Δ<sub>1</sub>f/Δ<sub>2</sub>f</td><td>L1/L2 sensitivity</td></tr><tr><td>α<sub>M</sub>(λ)</td><td>Moment function</td></tr><tr><td>n<sub>c</sub></td><td>Vote count (PATE)</td></tr><tr><td>k<sub>w</sub><sup>v</sup></td><td>Key for wire value</td></tr><tr><td>Enc</td><td>Encryption function</td></tr><tr><td>G&#773;<sub>X</sub></td><td>Graph with arrows into X removed</td></tr><tr><td>G&#773;<sub>X&#773;</sub></td><td>Graph with arrows out of X removed</td></tr><tr><td>G&#773;<sub>X&#773;Z</sub></td><td>Graph with arrows into X and out of Z removed</td></tr><tr><td>G&#773;<sub>X&#773;Z&#773;(W)</sub></td><td>Graph with arrows into X and into Z(W) removed</td></tr><tr><td>V<sub>i</sub></td><td>Endogenous variable i</td></tr><tr><td>f<sub>i</sub></td><td>Structural equation for variable i</td></tr><tr><td>Pa<sub>i</sub></td><td>Parents of variable i</td></tr><tr><td>U<sub>i</sub></td><td>Exogenous variable i</td></tr><tr><td>x'<sub>downstream</sub></td><td>Downstream features under intervention</td></tr><tr><td>x'<sub>actionable</sub></td><td>Actionable features</td></tr><tr><td>L</td><td>Leaves (Decision Trees)</td></tr><tr><td>P<sub>ℓ</sub></td><td>Paths for leaf ℓ</td></tr><tr><td>z<sub>ℓ</sub></td><td>Binary variable for leaf ℓ</td></tr><tr><td>L<sup>+</sup></td><td>Positive leaves</td></tr><tr><td>u<sub>j,ℓ</sub>/l<sub>j,ℓ</sub></td><td>Bounds for feature j in leaf ℓ</td></tr><tr><td>p<sub>k</sub></td><td>Client weight (FL)</td></tr><tr><td>F<sub>k</sub></td><td>Local loss for client k</td></tr><tr><td>S<sub>t</sub></td><td>Selected clients at round t</td></tr><tr><td>σ<sub>k</sub></td><td>Variance for client k</td></tr><tr><td>σ<sub>g</sub></td><td>Heterogeneity bound</td></tr><tr><td>ϕ</td><td>Shared layers</td></tr></tbody></table>
<table><thead><tr><td>Symbol</td><td>Description</td></tr></thead><tbody><tr><td>ψ<sub>k</sub></td><td>Personal layers for client k</td></tr><tr><td>Q(g)</td><td>Quantized gradient</td></tr><tr><td>U/V</td><td>Low-rank matrices</td></tr><tr><td>r</td><td>Rank</td></tr><tr><td>x*/y*</td><td>Reconstructed input/label</td></tr><tr><td>h</td><td>Barrier function</td></tr><tr><td>L<sub>RM</sub></td><td>Reward model loss</td></tr><tr><td>L<sub>DPO</sub></td><td>DPO loss</td></tr><tr><td>π<sub>ref</sub></td><td>Reference policy</td></tr><tr><td>F</td><td>Shaping function</td></tr><tr><td>P<sub>train</sub>/P<sub>test</sub></td><td>Training/test distributions</td></tr><tr><td>x̃</td><td>Perturbed input</td></tr><tr><td>E(x)</td><td>Energy function</td></tr><tr><td>q<sub>φ</sub></td><td>Variational distribution</td></tr><tr><td>ECE</td><td>Expected calibration error</td></tr><tr><td>T*</td><td>Optimal temperature</td></tr><tr><td>s</td><td>Conformity score</td></tr><tr><td>C(x)</td><td>Prediction set</td></tr><tr><td>κ(x)</td><td>Confidence function</td></tr></tbody></table>

## 20 Acknowledgments

Below, I gratefully acknowledge the open learning community online and the instructors behind these excellent video courses. Their willingness to share high-quality educational resources openly has made this material possible.

* Stanford CS230: Deep Learning I (Autumn 2025)

* Theoretical Aspects of Trustworthy AI

* Trustworthy ML Initiative Seminar Series

* Stanford CS229: Machine Learning I (Spring 2022)

* Causal Inference

* Google DeepMind AGI Safety Course

* Alignment Workshop

* Stanford CS329H: Machine Learning from Human Preferences (Autumn 2024)

* Stanford EE364A: Convex Optimization
References

[1] Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In *International conference on machine learning*, pages 22–31. PMLR, 2017.

[2] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. *Advances in neural information processing systems*, 31, 2018.

[3] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In *International conference on machine learning*, pages 60–69. PMLR, 2018.

[4] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. *arXiv preprint arXiv:1606.06565*, 2016.

[5] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In *European conference on computer vision*, pages 484–501. Springer, 2020.

[6] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. *arXiv preprint arXiv:1907.02893*, 2019.

[7] Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. *Journal of machine learning research*, 3(Nov):463–482, 2002.

[8] Igor Calzada, Géza Németh, and Mohammed Salah Al-Radhi. Trustworthy ai for whom? genai detection techniques of trust through decentralized web3 ecosystems. *Big Data and Cognitive Computing*, 9(3):62, 2025.

[9] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In *2017 ieee symposium on security and privacy (sp)*, pages 39–57. Ieee, 2017.

[10] Arunraju Chinnaraju. Explainable ai (xai) for trustworthy and transparent decision-making: A theoretical framework for ai interpretability. *World Journal of Advanced Engineering Technology and Sciences*, 14(3):170–207, 2025.

[11] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. *Big data*, 5(2):153–163, 2017.

[12] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In *international conference on machine learning*, pages 1310–1320. PMLR, 2019.

[13] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In *International conference on machine learning*, pages 2206–2216. PMLR, 2020.

[14] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In *Proceedings of the 3rd innovations in theoretical computer science conference*, pages 214–226, 2012.

[15] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. Datasheets for datasets. *Communications of the ACM*, 64(12):86–92, 2021.
[16] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. *arXiv preprint arXiv:1412.6572*, 2014.

[17] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. *arXiv preprint arXiv:1610.02136*, 2016.

[18] Andrés Herrera-Poyatos, Javier Del Ser, Marcos López de Prado, Fei-Yue Wang, Enrique Herrera-Viedma, and Francisco Herrera. Responsible artificial intelligence systems: A roadmap to society's trust through trustworthy ai, auditability, accountability, and governance. *arXiv preprint arXiv:2503.04739*, 2025.

[19] Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In *International conference on computer aided verification*, pages 97–117. Springer, 2017.

[20] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In *International Conference on Machine Learning*, pages 17061–17084. PMLR, 2023.

[21] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. *Advances in neural information processing systems*, 30, 2017.

[22] Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. *arXiv preprint arXiv:1706.02690*, 2017.

[23] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. *Advances in neural information processing systems*, 33:21464–21475, 2020.

[24] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. *arXiv preprint arXiv:1706.06083*, 2017.

[25] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In *Proceedings of the conference on fairness, accountability, and transparency*, pages 220–229, 2019.

[26] Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In *Proceedings of the 2020 conference on fairness, accountability, and transparency*, pages 607–617, 2020.

[27] Judea Pearl. *Causality*. Cambridge university press, 2009.

[28] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. *Journal of the Royal Statistical Society Series B: Statistical Methodology*, 78(5):947–1012, 2016.

[29] Rajesh Ranjan, Shailja Gupta, and Surya Narayan Singh. Loka protocol: A decentralized framework for trustworthy and ethical ai agent ecosystems. *arXiv preprint arXiv:2504.10915*, 2025.

[30] Deevashwer Rathee, Mayank Rathee, Nishant Kumar, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. Cryptflow2: Practical 2-party secure inference. In *Proceedings of the 2020 ACM SIGSAC conference on computer and communications security*, pages 325–342, 2020.
[31] Eugenia Stamboliev and Tim Christiaens. How empty is trustworthy ai? a discourse analysis of the ethics guidelines of trustworthy ai. *Critical Policy Studies*, 19(1):39–56, 2025.

[32] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In *International conference on machine learning*, pages 3319–3328. PMLR, 2017.

[33] Vladimir N Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. In *Measures of complexity: festschrift for alexey chervonenkis*, pages 11–30. Springer, 2015.

[34] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. *Harv. JL & Tech.*, 31:841, 2017.

[35] Wenqi Wei and Ling Liu. Trustworthy distributed ai systems: Robustness, privacy, and governance. *ACM Computing Surveys*, 57(6):1–42, 2025.

[36] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. *arXiv preprint arXiv:1611.03530*, 2016.

[37] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In *International conference on machine learning*, pages 7472–7482. PMLR, 2019.

[38] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. *Advances in neural information processing systems*, 32, 2019.