Aligning Model Properties via Conformal Risk Control

Authors: william  overman
jacqueline jil vallon
mohsen  bayati
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 19:52:09

Abstract: AI model alignment is crucial due to inadvertent biases in training data and the underspecified machine learning pipeline, where models with excellent test metrics may not meet end-user requirements. While post-training alignment via human feedback shows promise, these methods are often limited to generative AI settings where humans can interpret and provide feedback on model outputs. In traditional non-generative settings with numerical or categorical outputs, detecting misalignment through single-sample outputs remains challenging, and enforcing alignment during training requires repeating costly training processes.
In this paper we consider an alternative strategy. We propose interpreting model alignment through property testing, defining an aligned model $f$ as one belonging to a subset $\mathcal{P}$ of functions that exhibit specific desired behaviors. We focus on post-processing a pre-trained model $f$ to better align with $\mathcal{P}$ using conformal risk control. Specifically, we develop a general procedure for converting queries for testing a given property $\mathcal{P}$ to a collection of loss functions suitable for use in a conformal risk control algorithm. We prove a probabilistic guarantee that the resulting conformal interval around $f$ contains a function approximately satisfying $\mathcal{P}$. We exhibit applications of our methodology on a collection of supervised learning datasets for (shape-constrained) properties such as monotonicity and concavity. The general procedure is flexible and can be applied to a wide range of desired properties. Finally, we prove that pre-trained models will always require alignment techniques even as model sizes or training data increase, as long as the training data contains even small biases.

Main paper: 
 introduction: 
 The emergence of large foundation models has increased the attention to the problem of alignment. Aligned models are artificial intelligences designed to pursue goals that align with human values, principles, and intentions (Leike et al., 2018; Ouyang et al., 2022; Hendrycks et al., 2023; Ngo et al., 2024) . Although the alignment problem is predominantly examined in the context of potential artificial general intelligence (AGI), large language models (LLMs), and reinforcement learning (RL) agents, it also has roots in the modern machine learning pipeline (D'Amour et al., 2022) . Motivated by this, we introduce a broader notion of alignment in this paper, extending beyond the aforementioned generative models to include even tabular regression models.As an example, consider a regression task where property P represents models that are monotonically decreasing in a given feature (covariate). For example, predicting cancer patient survival should be monotonically decreasing in cancer stage (Vallon et al., 2022 (Vallon et al., , 2024)) . Constraining a prediction model during training to maintain monotonicity in this feature can be viewed as a form of alignment. For a pre-trained model f that was trained without such constraints, ensuring monotonically decreasing predictions in this feature can be more complex. This complexity arises particularly in non-generative settings where the user cannot update f or obtain any outputs other than point predictions f (X) for a given input X.In this work, we propose an approach to aligning a pre-trained model f that is motivated by property testing (Ron, 2008; Goldreich, 2017) and conformal risk control (Angelopoulos et al., 2024) . Property testing aims to design efficient algorithms for determining membership to the set P of functions with a given property, that require fewer resources than learning algorithms for P (Ron, 2008) . This is particularly relevant for modern deep learning, where a user may need to determine if a pre-trained model f belongs to P without the resources to train a model of comparable size.Property testing algorithms use local queries to determine, with high probability, whether a function has a given global property or is far from having it. We map such queries for a property P to a set of loss functions, which we then use in a conformal risk control procedure (Angelopoulos et al., 2024) to establish a notion of alignment for P. We prove that this procedure yields a conformal interval around f containing a function close to P.We demonstrate our methodology on real-world datasets for the properties of monotonicity and concavity. Motivated by the potential for systematic under-or over-estimation bias in f , we provide a straightforward extension of Angelopoulos et al. (2024) to obtain asymmetric conformal intervals with multi-dimensional parameters. While we examine both monotonicity and concavity constraints, the majority of our focus is on monotonicity, as these constraints have been shown to promote crucial aspects of alignment to human values, such as fairness and adherence to social norms (Wang and Gupta, 2020) .While our methodology provides a way to align pre-trained models, one may question whether such techniques will remain necessary as AI capabilities advance. Given the outstanding capabilities of modern AI models with substantially large numbers of parameters and training data, one may argue that the alignment problem may naturally disappear as such advances continue (Kaplan et al., 2020) . However, another contribution of this paper is to refute this argument in a stylized setting, building on recent advances in the theory of linearized neural networks (Mei and Montanari; Misiakiewicz and Montanari, 2023) . Specifically, we show that increasing the size of the training data or the number of parameters in a random feature model (a theoretically tractable neural network proxy where hidden layer weights are randomly initialized and fixed (Rahimi and Recht, 2007 )) cannot help it satisfy a property P, if the pre-training data has biased labels. Our simulations show that the result holds even if only a small fraction of the training labels are impacted by the bias.Summarizing our main contributions, we: (1) introduce an alignment perspective based on property testing, (2) use conformal risk control to post-process predictions of pre-trained models for better alignment, and (3) demonstrate that increasing training data and parameters in a random feature model does not eliminate the need for alignment. We discuss related work in Section 6, particularly our connections to Yadkori et al. (2024) , who use conformal risk control to address large language model hallucinations (Ji et al., 2023) .Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly state the contributions of proposing a model alignment approach via conformal risk control and property testing, with applications demonstrated on real-world datasets. The claims are supported by theoretical results and experimental validation. See Section 3 and 4. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
preliminaries: 
 In this section we provide key definitions drawn from property testing as well as a condensed overview to conformal prediction and conformal risk control. We provide a short introduction to propery testing in Appendix A and an extensive introduction to the field can be found in Goldreich (2017) .Our perspective on alignment in this work is motivated by the field of property testing (Goldreich, 2017; Ron, 2008) . Property testing studies algorithms that, by making a small number of local queries to a large object (such as a function or a graph), can determine whether the object has a certain property or is significantly far from having it.Classic examples include linearity testing of Boolean functions (Blum et al., 1993) , testing whether a function is a low-degree polynomial (Kaufman and Ron, 2006; Bhattacharyya et al., 2009) , and testing k-juntas (Blais, 2009) . These algorithms generally operate by randomly sampling and querying the object, leveraging local information to infer global properties.In this work, we focus on set-valued functions, which are functions that map elements of a domain X to subsets of a codomain Y, i.e., F : X → 2 Y . While the standard definitions of property testing are technically sufficient for our purposes-since we can consider set-valued functions as functions with range 2 Y -we introduce specialized definitions to maintain clarity and to facilitate the transition between discussing Y and 2 Y . Definition 1 (Satisfying and Accommodating a Property). Let property P denote a specific subset of all functions that map X to Y. A function f : X → Y satisfies the property P if f ∈ P.A set-valued function F : X → 2 Y accommodates a property P if there exists a function g ∈ P such that g(x) ∈ F (x) for all x ∈ X .Intuitively, F accommodates P if it contains at least one function g satisfying P within its possible outputs.We extend the notion of ε-farness from a property (as defined in Appendix A) to set-valued functions. For set-valued functions, we measure the distance based on how often the outputs of any function g ∈ P fall within the sets provided by F . Definition 2 (ε-Faraway). For a set-valued function F : X → 2 Y , a distribution D over X , ε > 0, and a property P, we say F is ε-Faraway from P with respect to D if δ P,D (F ) > ε, whereδ P,D (F ) def = inf g∈P δ D (F, g) and δ D (F, g) def = Pr X∼D [g(X) ̸ ∈ F (X)].Note. Throughout this work, we assume that D is the empirical distribution of a fixed and finite calibration dataset, and thus has finite support. While this assumption is not strictly necessary, most property testing results are over finite domains. Property testing over functions with Euclidean domains is a in general a difficult problem, though there have been notable recent successes (Fleming and Yoshida, 2020; Arora et al., 2023) .. With these definitions in place, we can define testers for set-valued functions. We focus on one-sided error testers, which are algorithms that take in a set-valued function F , a distribution D, and a distance parameter ε and output either Accept or Reject. These algorithms never reject a function that accommodates the property. The standard definition of one-sided error testers (provided in Appendix A) extends naturally to set-valued functions by replacing the notion of satisfying a property with accommodating it. Definition 3 (One-Sided Error Tester for Set-Valued Functions). A one-sided error tester for a property P in the context of set-valued functions is a probabilistic oracle machine M that, given a distance parameter ε > 0, oracle access to a set-valued function F : X → 2 Y , and oracle access to samples from a fixed but unknown distribution D over X , satisfies:1. If F accommodates P, then Pr[M F,D (ε) = Accept] = 1. 2. If F is ε-Faraway from P with respect to D, then Pr[M F,D (ε) = Accept] ≤ 1 3 .Here, M F,D (ε) denotes the execution of the tester M when given oracle access to the function F , the distribution D, and the parameter ε.Note that M itself is an abstract algorithm; M F,D is the instantiation of this algorithm with specific oracle access to F and D.In many property testing algorithms, the parameter ε is used only to determine the number of iterations or samples required, not the core logic of the tester. This leads to the concept of proximity-oblivious testers (POTs), where the basic testing procedure is independent of ε. The general definition of POTs (given in Appendix A) also extends naturally to set-valued functions. Definition 4 (Proximity-Oblivious Tester for Set-Valued Functions). A proximity-oblivious tester for a property P in the context of set-valued functions is a probabilistic oracle machine T that satisfies:1. If F accommodates P, then Pr[T F,D = Accept] = 12. There exists a non-decreasing function ρ : (0, 1] → (0, 1] (called the detection probability) such that if F is ε-Faraway from P,Pr[T F,D = Reject] ≥ ρ(ε).Here, T F,D denotes the execution of the tester T when given oracle access to the function F and the distribution D.To obtain a one-sided error tester with parameter ε, we can make Θ 1 ρ(ε) independent calls to the POT T and accept if and only if all the calls accept (Goldreich and Ron, 2008) . We denote by T F,D (X) the output when applied to a specific sample X ∼ D, and note that with abuse of notation we will late consider D to be the empirical distribution of calibration dataset {(X i , Y i )} n i=1 in which case we write T F,D (X i , Y i ) for the output on this specific sample from D.Example. Consider functions f : R d → R, and let P denote the property that f is constant in the k-th dimension. This property has has connections to fairness among other applications Caton and Haas (2024) . Assume D is the empirical distribution of the inputs X ∈ R for some fixed dataset.Restrict to set-valued functions F that output compact and connected intervals of the form [a, b] ⊆ R for a, b ∈ R. The candidate POT T F,D for whether such a set-valued function F accommodates P is then as follows: sample X, X ′ ∼ D, If F (X) ∩ F (X ′ ) ̸ = ∅, then Accept; otherwise, Reject. We prove that this satisfies Definition 4 in Appendix B.1.Our main tool for achieving alignment from this property perspective is built on conformal prediction and conformal risk control (Vovk et al., 2005; Bates et al., 2021; Angelopoulos et al., 2024) . Conformal prediction post-processes the outputs of any model f to create prediction intervals C(•) that ensure certain statistical coverage guarantees. Using a calibration dataset {(X i , Y i )} n i=1 consisting of ground truth input-output pairs, conformal prediction constructs intervals around the predictions of fsuch that Pr[Y n+1 / ∈ C(X n+1 )] ≤ α for a user-specified error rate α on a test point (X n+1 , Y n+1 ).This guarantee is notably distribution-free and holds for any function f . The probability is over the randomness in all n + 1 points; both the calibration set and the test point. The construction of C(•) depends on both the model f and the draw of the calibration data.The conformal risk control framework extends conformal prediction to notions of error beyond miscoverage (Angelopoulos et al., 2024) . Consider a paramater set Λ ⊂ R ≥0 that is a bounded subset of the nonnegative reals. Given an exchangeable collection of non-increasing, random loss functions L i : Λ → (-∞, B], i = 1, . . . , n + 1, conformal risk control uses the first n loss functions and calibration data {(X i , Y i )} n i=1 to determine λ such thatE[L n+1 ( λ)] ≤ α.Consider loss functions of the formL i (λ) = ℓ(C λ (X i ), Y i ), where C λ (X i) is a set of outputs constructed by f and the calibration data. Larger values of λ generate more conservative prediction sets C λ (•). Let the risk on the calibration data for a given λ be Rn (λ) = 1 n n i=1 L i (λ). For a user-specified risk rate α, we letλ = inf λ : n n + 1 Rn (λ) + B n + 1 ≤ α .This choice of λ guarantees the desired risk control E[L n+1 ( λ)] ≤ α (Angelopoulos et al., 2024) .λ, λ ′ ∈ R k , if λ ≤ λ ′ (i.e., λ i ≤ λ ′ i ∀i = 1, 2, . . . , k), then F λ (X) ⊆ F λ ′ (X).This definition ensures that increasing the parameters λ leads to larger (more conservative) prediction sets, and that when all parameters are zero, the prediction set reduces to the point prediction given by f (X).Following the original scalar λ setting, we assess F λ using non-increasing random loss functionsL i = ℓ(F λ (X i ), Y i ) ∈ (-∞, B] for B < ∞.In particular, we consider an exchangeable collection of non-increasing random functionsL i : Λ → (-∞, B], i = 1, ..., n + 1, where Λ ⊂ R k ≥0 is a bounded subset of R k ≥0 , with bound λ max j in each dimension j ∈ [k].As in Angelopoulos et al. (2024) , we use the first n functions to determine λ so that the risk on the (n + 1)-th function is controlled, specifically so that E[L n+1 ( λ)] ≤ α.We apply a similar algorithm. Given α ∈ (∞, B) and lettingRn (λ) = L1(λ)+•••+Ln(λ) n , define Λ min = min λ : n n + 1 Rn (λ) + B n + 1 ≤ αto be the set of minimal elements (Boyd and Vandenberghe, 2004) of Λ that satisfy the condition n n+1 Rn (λ) + B n+1 ≤ α. Let g : Λ → R be a strictly increasing function such that L i (λ) is nonincreasing with respect to the level sets defined by g(λ). Then select λ ∈ Λ min to be a minimizer of g over Λ min .We then deploy the resulting set-valued function F λ on the test point X n+1 . For this choice of λ, we have a risk control guarantee that mimics the result of Angelopoulos et al. (2024) , specifically: Proposition 1. Assume that L i (λ) is non-increasing with respect to the partial ordering of Λ inherited from R k . Additionally, assume that L i (λ) is non-increasing with respect to g(λ) for some strictly increasing function g : Λ → R. Also assume L i is right-continuous in each dimension, L i (λ max ) ≤ α, andsup λ L i (λ) ≤ B < ∞ almost surely. Then E[L n+1 ( λ)] ≤ α.The proof is similar to the proof of the guarantee for the conformal risk control algorithm in Angelopoulos et al. (2024) and is deferred to Appendix C.To provide intuition on g(λ), we note that for our primary use case we will take g(λ) = k i=1 λ i . Clearly this function is strictly increasing in λ and intuitively it is reasonable to consider loss functions L i that are non-increasing as the sum of the components of λ increases.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The paper mentions the assumptions made for the theoretical results, such as the nature of the data distribution and the properties of the models. The discussion section also reflects on the limitations regarding the types of properties and models to which the methodology can be applied. See Section 3. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conformal property alignment: 
 Our main methodology is to use conformal risk control to create prediction intervals that align with specific properties P. Our approach allows us to post-process the outputs of a pre-trained model f to ensure that within the resulting conformal band, with a given probability, there exists predictions that adhere to desired properties such as monotonicity.We make particular use of the conformal risk control algorithm to allow for a k-dimensional vector of tuning parameters λ = (λ 1 , λ 2 , . . . , λ k ), where larger values of λ ∈ Λ ⊂ R k yield more conservative outputs, where Λ ⊂ R k ≥0 is a bounded subset of R k ≥0 . This works by mapping λ to a scalar and then applying standard conformal risk control. We emphasize that this result is not new and follows essentially directly from Angelopoulos et al. (2024) . The construction of the output set F λ (X) ⊆ Y depends on the specific application and provides flexibility in how the function f (X) and the parameters λ are utilized. Definition 5 (Construction of F λ (X)). Let f : X → Y be a given function. For each λ ∈ R k , define the set-valued function F λ : X → 2 Y such that, for each X ∈ X , F λ (X) is a set of predictions for X constructed from f and λ. The specific construction of F λ (X) should satisfy the following properties:1. When λ = 0, we have F 0 (X) = {f (X)}.We now demonstrate how to construct a conformal risk control problem using proximity-oblivious testers (POTs) for a given property P. Suppose we are given a pre-trained model f : X → Y. We aim to extend the point predictions of f to prediction sets, where the size or conservativeness of the set is parameterized by a parameter λ. Let F λ : X → 2 Y denote the set-valued function that outputs, for each X ∈ X , the set F λ (X) ⊆ Y determined by f , X, and λ.Let T F,D be a proximity-oblivious tester for whether a set-valued function F accommodates the property P as given y Definition 4. We denote the random output of T F,D evaluated at (X, Y ) ∼ D by T F,D (X, Y ).We now define a loss function, generated from T F,D , which will be crucial in formulating our conformal risk control problem. Definition 6 (Loss Function Generated from a POT). Let T F,D be a proximity-oblivious tester for a property P. We define the loss function L i as:L i = 0, if T F,D (X i , Y i ) = Accept, 1, otherwise,where (X i , Y i ) are samples from the distribution D.Example. Consider the POT for the property P of a function f : R → R being constant, as mentioned in Section 2.1. Assume we have access to a calibration set {(X i , Y i )} n i=1 of size n. We use a two-dimensional parameter λ = (λ -, λ + ), and define the set-valued function:F λ (X) = [f (X) -λ -, f (X) + λ + ],for each X ∈ R. This creates prediction intervals around the point prediction f (X), with widths controlled by λ -and λ + .We then apply the loss function generated by T F λ ,D as given in Definition 6, and use conformal risk control to tune λ such that the expected loss on the (n + 1)th point falls below a given target level α.Note that in this case the tester and loss function does not depend on the Y i . This is because the property of f being constant does not depend on the Y i from the calibration set and here D is only used to obtain samples of the X i . This is not the case in general, however, and properties can be defined with respect to the whole sample (X i , Y i ) ∼ D. For example, we could consider the property P that f does not over-predict, that is, for (X, Y ) ∼ D we have f (X) ≤ Y . Now we state our main theorem. Theorem 1. Let T be a proximity-oblivious tester for a property P with detection probability function ρ(•). Assume access to a calibration dataset {(X i , Y i )} n i=1 sampled independently from a distribution D. Suppose we run conformal risk control on this calibration dataset using risk parameter α and loss functions L i for property P generated from T (as in Definition 6). Then, for any ε such that ρ(ε) > α, the probability that F λ is ε-Faraway from P satisfies: Pr (X1,Y1),...,(Xn,Yn)F λ is ε-Faraway from P ≤ α ρ(ε) .Proof. Let E denote the event that F λ is ε-Faraway from the property P. Our goal is to bound the probability Pr (X1,Y1),...,(Xn,Yn) [E] .The conformal risk control procedure ensures that the expected loss on a new sample (X n+1 , Y n+1 ) satisfies: E (X1,Y1),...,(Xn,Yn),(Xn+1,Yn+1) [L n+1 ] ≤ α. Now, we can writeE[L n+1 ] = Pr(E) • E[L n+1 | E] + Pr(E c ) • E[L n+1 | E c ].When E occurs, F λ is ε-Faraway from P. By the properties of the proximity-oblivious tester T , we have:Pr (X,Y )∼D T F λ,D (X, Y ) = Reject | E ≥ ρ(ε).Thus, the conditional expected loss satisfies:E[L n+1 | E] = Pr T F λ ,D (X n+1 , Y n+1 ) = Reject | E ≥ ρ(ε).And when sincePr(E c ) • E[L n+1 | E c ] is non-negative because L n+1 is non-negative, we obtain E[L n+1 ] ≥ Pr(E) • ρ(ε).Combining this result with our guarantee from the conformal risk control procedure,α ≥ E[L n+1 ] ≥ Pr(E) • ρ(ε).This implies:Pr(E) ≤ α ρ(ε).Therefore, the probability that F λ is ε-Faraway from P satisfies:Pr (X1,Y1),...,(Xn,Yn)F λ is ε-Faraway from P ≤ α ρ(ε).Amplifying Detection Probability via Independent Calls. When the detection probability ρ(ε) of the proximity-oblivious tester T is less than or close to the risk parameter α, the bound provided by Theorem 1 may not be tight or meaningful (since α/ρ(ε) could be greater than or equal to 1).To address this issue, we can amplify the detection probability by performing multiple independent executions of T and combining their results appropriately.To increase the detection probability beyond α, we execute the proximity-oblivious tester T independently k times on independent samples and define a new tester T ′ that rejects if any of the k executions reject (i.e., by applying a logical OR to the outcomes). This amplification technique yields an adjusted detection probabilityρ ′ (ε) = 1 -(1 -ρ(ε)) k ,representing the probability that at least one of the k independent executions rejects when the function is ε-Faraway from P.In this approach, the calibration dataset needs to be partitioned into n ′ = n k disjoint batches, each containing k samples. Each batch provides the independent samples required for the k executions of T per calibration point. As a result, the effective sample size available for calibrartion becomes n ′ due to this batching of samples. 
monotonicity: 
 Monotonic behavior is important in various applications. We focus on monotonicity in a single feature, where we expect that f (X) should have monotonically increasing or decreasing behavior with respect to a certain feature x k when other features x -k are held fixed. While there is a longstanding literature on using monotonic constraints for regularization (Brunk et al., 1973; Sill and Abu-Mostafa, 1996; You et al., 2017; Bonakdarpour et al., 2018) and on integrating such monotonic shape constraints into prediction models (Groeneboom and Jongbloed, 2014; Cano et al., 2018; Runje and Shankaranarayana, 2023) , our aim is not to view monotonicity as a possible means to improve test accuracy, but rather as a user-desired property for safe or fair deployment of a given model. For example, Wang and Gupta (2020) highlight the importance of monotonicity in models for criminal sentencing, wages, and medical triage.Consider a user given a pre-trained model f that was not trained with monotonic constraints. The user, however, wishes for the sake of safe or fair deployment to make predictions in a way that is as monotonic as possible. In particular, let P be the property that f is monotonically decreasing in dimension k. To apply our methodology we consider the proximity oblivious tester T for P as given in Algorithm 1.We prove in Appendix B.2 that Algorithm 1 is indeed a POT for the property P of being monotonically decreasing in a given dimension. Then let M be the one-sided error tester for P resulting from Θ(1/ρ(ε)) calls to T . Now assume we have access to a calibration dataset {(X i , Y i )} n i=1 sampled from D of size n ∈ Ω(1/ρ(ε)). We will use this calibration dataset to determine the setting of λ = (λ + , λ -) via conformal risk control where the loss function is generated as in Definition 6. Here the set-valued function will be constructed asF λ (X) = [f (X) -λ -, f (X) + λ + ].Then by Theorem 1 if the tester has sufficient detection probability ρ(ε) > α we expect to obtain a set-valued function F λ at most ε from P. We now investigate this empirically.Algorithm 1 POT T for property P of monotonically decreasing in dimension k1: Sample X 1 ∼ D. Let X 1 = (x 1 , x -k ) 2: Sample x 2 from the marginal distribution of D in dimension k. Set X 2 = (x 2 , x -k ) 3: if x 1 < x 2 and max F (X 1 ) < min F (X 2 ) then 4: return Reject 5: else if x 2 < x 1 and max F (X 2 ) < min F (X 1 ) then 6:return Reject 7: end if 8: return Accept Setup. We align for monotonicity on various UCI ML repository datasets (Dua and Graff, 2023) with a 70-15-15 train-calibrate-test split, averaged over 30 random splits. We use XGBoost regression models (Chen and Guestrin, 2016) . For each dataset, we select a feature for which we desire the model to be monotonic, not with the intention of improving test-set accuracy, but from the perspective of a user who desires this property.We train two models per dataset: one unconstrained, trained on the training set, and another constrained to be monotonic, trained on both the training and calibration sets. The conformal risk control procedure is applied to the unconstrained model using the calibration data. The constrained model can be considered best possible from the user's perspective, using all available pre-test data and satisfying the monotonicity property P during training.To compare performance with respect to the training metric of accuracy, we convert conformal intervals into point predictions by taking k-quantiles of the constrained feature, linearly interpolating between adding λ + at the lowest quantile to subtracting λ -at the highest quantile for monotonically decreasing, or vice versa for monotonically increasing.Results. Table 4 .1 presents results on the test set for the Combined Cycle Power Plant dataset (Tfekci and Kaya, 2014) . In practice, Exhaust-vacuum is known to negatively influence turbine efficiency (Tfekci and Kaya, 2014) . The conformal procedure outperforms the constrained model in terms of MSE for all α, which is a fortuitous but unexpected outcome. The constrained model should be seen as an oracle benchmark in the sense that the model was given to the user already trained to satisfy the desired property. The results on this dataset and The risk metric closely matches the theoretical guarantee from conformal risk control and achieves optimal performance of 0 for the constrained model. Additional datasets and results are detailed in the appendix. Concavity and convexity are crucial behaviors in many applications. In this context, we focus on concavity in a single feature. A common example where users might expect concave behavior is in recommendation or preference prediction models. According to economic theory, the utility function with respect to the quantity of an item is often quasi-concave, reflecting the principle of diminishing marginal utility (Mas-Colell et al., 1995) . Jenkins et al. (2021) propose a novel loss function to account for this expected concavity, which aligns the model with the concavity property P during training. Here we again consider aligning a pre-trained model, not trained to satisfy P, using a proximity oblivious tester T for P as described in Algorithm 2. Algorithm 2 POT T for property P of concavity in dimension k 1: Sample X mid ∼ D 2: Sample δ left , δ right from empirical differences in feature k 3: Set X left by decreasing feature k of X mid by δ left 4: Set X right by increasing feature k of X mid by δ right 5: Query F (X mid ), F (X left ), and F (X right )6: Compute α = X right [k] -X mid [k] X right [k] -X left [k] 7: if min F (X mid ) > α max F (X left ) + (1 -α) max F (X right ) then 8:return Reject 9: end if 10: return Accept Again we can use a calibration dataset to determine the setting of λ = (λ + , λ -) via conformal risk control where the loss function is generated as in Definition 6. Here again the set-valued function will be constructed asF λ (X) = [f (X) -λ -, f (X) + λ + ].We demonstrate running conformal risk control with this loss function on a real-world dataset in Appendix D.2. 
a stylized examination of alignment persistence in ai models: 
 Consider data generated as: y = g(X) + h(X) + ε , where ε is mean-zero noise with variance τ 2 independent of X. Here, h(X) is biased noise we want to ignore, aiming to learn only g(X). Consider the case in which experts expect data to follow g(X) + unbiased noise, but biased noise h(X) can obscure this.One potential reason for the presence of biased noise in data could be due to a measurement error of the outcome that is correlated with select features, leading to an incorrectly calculated outcome. A biased measurement error could occur if there is incomplete data and the presence of the incomplete data is correlated with select features in an unexpected, systematic way. Our goal is to understand how this bias affects model behavior when trying to learn g(X) alone.Given n i.i.d. samples {(X i , Y i )} n i=1 from the above model, we denote this dataset by D n . We use a random feature model:f RF (X; a, {w j } j∈[N ] ) = 1 √ N j∈[N ] a j σ(⟨X, w j ⟩) ,where a ∈ R N are learned weights, and {w j } j∈[N ] are fixed random weights. The squared loss is minimized by ridge regression:âλ = arg min a∈R N i∈[n] (Y i -f RF (X i )) 2 + λ∥a∥ 2 2 .Users expect a model to exhibit a property P, satisfied by g(X) but not necessarily by g(X) + h(X).We can constrain training to ensure P.Let C P = {a | a ∈ R N and f RF (X; a) satisfies P} , yielding a constrained model: âλ,P = arg min a∈C P i∈[n] (Y i -f RF (X i )) 2 + λ∥a∥ 2 2 .Assuming g and h are polynomials with deg g < deg h , and given specific conditions on data size and model parameters, we consider two settings: (i) Classic:d deg g +δ < N < d deg h -δ , and (ii) Underspecified: N > d deg h +δ for a small δ > 0.In Appendix E, we utilize results from Misiakiewicz and Montanari (2023) to derive insights into the impact of model complexity and data size on adherence to P. In particular, we show that under certain assumptions, including small noise bias and robustness of property P, the constrained and unconstrained models have zero distance in the classic setting: âλ,P = âλ . However, in the underspecified setting, the constrained and unconstrained models will differ, resulting in a non-zero distance: âλ,P ̸ = âλ . This result implies that in the presence of noise bias, the overparameterized models (i.e., underspecified setting) fail to satisfy the property P, and this cannot be remedied as the data size increases.Question: Does paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The paper provides links to the UCI ML repository for the datasets used, and the code used for experiments will be made available upon publication. Sufficient instructions are included for reproducing the experiments. See Section 4 and Appendix B.Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
related work: 
 Our paper draws from a broad range of areas, hence we refer the reader to textbooks and surveys in alignment (Everitt et al., 2018; Hendrycks et al., 2022; Ji et al., 2024; Hendrycks, 2024) , conformal prediction (Angelopoulos and Bates, 2022) , property testing (Ron, 2008; Goldreich, 2017) , and linearized neural networks (Misiakiewicz and Montanari, 2023) .RLHF (Christiano et al., 2017) has been notably effective in aligning LLMs with human values and intentions, as demonstrated by (Ouyang et al., 2022) . Our work considers attempts at alignment that generalzies to models without human-interpretable outputs, which has connections to the scalable oversight problem (Irving et al., 2018; Christiano et al., 2018; Wu et al., 2021) . Goal misgeneralization (Langosco et al., 2022; Shah et al., 2022) has potential connections to the underspecified pipeline (D'Amour et al., 2022) considered in this paper in the sense that models with equivalent performance according to the training metric may differ in some other user-desired property during deployment. One of the main methods of assurance (Batarseh et al., 2021) , which is concerned with assessing the alignment of pre-trained AI systems, is safety evaluations (Perez et al., 2022; Shevlane et al., 2023) meant to assess risk during deployment, which also has connections to our approach.The work of Yadkori et al. ( 2024) closely aligns with ours in both methodology and theme, utilizing conformal risk control to reduce LLM hallucinations (Ji et al., 2023) . We discuss connections to this work in Appendix F. 
discussion: 
 We introduce a method to align pre-trained models with desired user properties using conformal risk control. By post-processing outputs using property dependent loss functions, we provide probabilistic guarantees that conformal intervals contain functions close to the desired set. This allows for alignment without retraining, effective in both generative and non-generative contexts. Future work should extend these techniques to more properties, explore sample complexity and adaptive querying, and potentially apply them to policy functions in MDP settings for RL agent safety guarantees.In this section, we provide a brief introduction to property testing, drawing upon standard definitions and concepts from the field. Property testing is a framework for designing algorithms that quickly decide whether a large, complex object (such as a function or a graph) possesses a certain property or is far from having that property. The goal is to make this decision by inspecting only a small, random portion of the object, thereby significantly reducing the computational effort required compared to examining the entire object.Our definitions and explanations are based on Goldreich (2017) , which offers a comprehensive introduction to property testing. We focus on the basic notions essential for understanding property testing as used in this paper, including the concepts of distance between objects, testers, and proximityoblivious testers.A central concept in property testing is the notion of distance between objects, which allows us to formalize what it means for an object to be far from satisfying a property. For functions, this distance is typically measured with respect to a distribution over the input domain. Definition 7 (ε-far). Let f : X → Y be a function, D a distribution over X , ε > 0, and P a property (a set of functions). We say that f is ε-far from P with respect to D if the distance δ P,D (f ) > ε, whereδ P,D (f ) = inf g∈P δ D (f, g), and δ D (f, g) = Pr X∼D [f (X) ̸ = g(X)].In other words, a function f is ε-far from P if any function g ∈ P agrees with f on at most a 1 -ε fraction of inputs sampled from D. This notion of distance allows us to quantify how much f differs from satisfying the property P.While many property testing algorithms assume D to be the uniform distribution over X , in some settings, particularly when dealing with arbitrary data distributions, it is natural to consider general distributions. This leads to the distribution-free property testing model Goldreich (2017) ; Halevy and Kushilevitz (2007) , where algorithms are designed to work with any underlying distribution D.Property testing algorithms, or testers, aim to distinguish between objects that have a certain property and those that are ε-far from it, by querying the object at a small number of locations. Testers can be categorized based on their error probabilities:• One-sided error testers: These testers always accept objects that have the property (zero false negatives), but only reject objects that do not satisfy the property with some probability that may be below 1. • Two-sided error testers: These testers may err in both directions, accepting objects that are ε-far from the property or rejecting objects that have the property, but with bounded error probabilities.Technically such testers are referred to as oracle machines. An oracle machine is a theoretical computational model, specifically a Turing machine that has access to an oracle. The oracle is a black box that can compute certain functions or provide certain information that the machine cannot compute on its own. In our context, the oracle machine M can query the function F at any point x ∈ X to receive F (x) and can obtain samples from the distribution D. It does not have explicit knowledge of the internal structure of F or D but can interact with them through these oracle queries.For our purposes in this work we will only need to discuss one-sided error testers, which we can formally define as follows: Definition 8 (One-Sided Error Tester). A one-sided error tester for a property P is a probabilistic oracle machine M that, given a distance parameter ε > 0, oracle access to a function f : X → Y, and oracle access to samples from a distribution D over X , satisfies:1. If f ∈ P, then Pr[M f,D (ε) = Accept] = 1. 2. If f is ε-far from P with respect to D, then Pr[M f,D (ε) = Accept] ≤ 1 3 .The choice of error probability 1 3 is arbitrary and can be reduced to any δ ∈ (0, 1 2 ) by repeating the tester multiple times and taking the majority outcome.In many property testing algorithms, the proximity parameter ε is used only to determine the number of iterations or samples needed, rather than affecting the core logic of the tester. This observation leads to the concept of proximity-oblivious testers (POTs), where the basic testing procedure is independent of ε. Definition 9 (Proximity-Oblivious Tester). A proximity-oblivious tester for a property P is a probabilistic oracle machine T that operates without knowledge of the proximity parameter ε and satisfies:1. If f ∈ P, then Pr[T f,D = Accept] = 1 2. If f / ∈ P, then Pr[T f,D = Reject] ≥ ρ(δ P,D (f )), where ρ : (0, 1] → (0, 1] is nondecreasing function called the detection probability.Note that the probability of rejecting a function increases with its distance from P, as quantified by the detection probability function ρ.To convert a POT into a standard tester with error probability 1 3 , we can repeat the basic testing procedure a sufficient number of times, determined by ρ(ε), and accept or reject based on the aggregate outcomes. Making O 1 ρ(ε) independent calls to T and accepting if all calls accept yields a tester that satisfies the standard definition with the desired error probability Goldreich and Ron (2008) .Testers can also be classified based on whether they are adaptive or non-adaptive in their querying strategy:• Non-adaptive testers: The queries made by the tester are determined in advance, based solely on the random coins and explicit inputs, and do not depend on the answers to previous queries.• Adaptive testers: The tester's queries may depend on the answers to previous queries, allowing for potentially more powerful testing strategies.Non-adaptive testers are often simpler and easier to analyze, and many property testing algorithms are designed to be non-adaptive. In the context of POTs, non-adaptivity is particularly advantageous because the tester's behavior remains consistent regardless of the function being tested. In this work we focus only on non-adaptive testers, but exploring the use of adaptive testers is an exciting future direction.We provide proofs that each of the testing algorithms we use are indeed Proximity Oblivious Testers according to Definition 4. However we note that these properties are well studied in the f : X → Y case and thus we provide these proofs in the set-valued f : X → 2 Y case mainly for completeness.For the sake of convenience we repeat the setting from the Example in Section 2.1.Consider functions f : R d → R, and let P denote the property that f is constant in the k-th dimension. Assume D is the empirical distribution of the inputs X ∈ R d for some fixed dataset.Restrict to set-valued functions F that output compact and connected intervals of the form [a, b] ⊆ R for a, b ∈ R. The candidate POT T F,D for whether such a set-valued function F accommodates P is then as follows: sample X, X ′ ∼ D, If F (X) ∩ F (X ′ ) ̸ = ∅, then Accept; otherwise, Reject. We claim that this is indeed a POT.Proof. Assume that F accommodates P. This means there exists a function g ∈ P such that for all X ∈ X , g(X) ∈ F (X). Since g is constant in the k-th dimension, there exists a function h : R d-1 → R such that for all x k ∈ R andx -k ∈ R d-1 , g(x k , x -k ) = h(x -k ).And in the algorithm we sample X = (x k , x -k ) from D and we independently sample x ′ k from the marginal distribution of x k and set X ′ = (x ′ k , x -k ). Then since x -k is the same for both X and X ′ , and g is constant in x k , we have:g(X) = h(x -k ) = g(X ′ ).Because g(X) ∈ F (X) and g(X ′ ) ∈ F (X ′ ), it follows that:h(x -k ) ∈ F (X) ∩ F (X ′ ).Therefore, F (X) ∩ F (X ′ ) ̸ = ∅, and the algorithm returns Accept.Now suppose that F is ε-far from P. This means that for any function g ∈ P,δ D (F, g) = Pr X∼D [g(X) / ∈ F (X)] > ε.For each g ∈ P, define the set of points where g does not belong to F (X):S g = {X ∈ X : g(X) / ∈ F (X)}.Although the sets S g may vary with g, for each g ∈ P, we have D(S g ) > ε. Fix an arbitrary function g ∈ P. For a fixed x -k ∈ R d-1 , define:p g (x -k ) = Pr x k ∼D x k |x -k [g(x k , x -k ) / ∈ F (x k , x -k )],whereD x k |x -k is the conditional distribution of x k given x -k . Since δ D (F, g) = E x -k [p g (x -k )]> ε, it follows that the average error probability over x -k is greater than ε.When we sample X = (x k , x -k ) and X ′ = (x ′ k , x -k ) with the same x -k and independent x k and x ′ k , the probability that both g(X) / ∈ F (X) and g(X ′ ) / ∈ F (X ′ ) is:Pr[g(X) / ∈ F (X) and g(X ′ ) / ∈ F (X ′ ) | x -k ] = p g (x -k ) 2 .Taking expectation over x -k , we get:Pr[g(X) / ∈ F (X) and g(X ′ ) / ∈ F (X ′ )] = E x -k [p g (x -k ) 2 ] ≥ E x -k [p g (x -k )] 2 > ε 2 .Under the event where both g(X) / ∈ F (X) and g(X′ ) / ∈ F (X ′ ), we have g(X) = h(x -k ) / ∈ F (X) and g(X ′ ) = h(x -k ) / ∈ F (X ′ ). Therefore, h(x -k ) / ∈ F (X) ∩ F (X ′ ).Since g is constant in x k , it assigns the same value h(x -k ) to both X and X ′ . No other function g ′ ∈ P can assign a different value at x -k , because all functions in P must be constant in the k-th dimension. Thus, there is no value y such that y ∈ F (X) ∩ F (X ′ ) that could correspond to g ′ (X) = y and g ′ ∈ P. Hence, F (X) ∩ F (X ′ ) = ∅, and the algorithm returns Reject.Thus the algorithm Rejects with probability at least ε 2 and therefore the algorithm has detection probability ρ(ε) = ε 2 and satisfies Definition 4 for being a Proximity Oblivious Tester.We prove that Algorithm 1 is indeed a POT for the property P of a function being monotonically decreasing in a given dimension k.Proof. Assume that F accommodates P. Then there exists a function g ∈ P such that for all X ∈ X , g(X) ∈ F (X).Since g is monotonically decreasing in x k , for any fixed x -k ∈ R d-1 and anyx k , x ′ k ∈ R with x k ≤ x ′ k , g(x k , x -k ) ≥ g(x ′ k , x -k ).In the algorithm, we sample X 1 = (x k 1 , x -k ) and X 2 = (x k 2 , x -k ), where x -k is fixed and x k 1 , x k 2 are independent samples from the marginal distribution of x k . First consider the case that x k 1 < x k 2 . Since g is monotonically decreasing, we have g(X 1 ) ≥ g(X 2 ). Because g(X i ) ∈ F (X i ), it follows that max F (X 1 ) ≥ g(X 1 ) ≥ g(X 2 ) ≥ min F (X 2 ). Therefore, max F (X 1 ) ≥ min F (X 2 ), and the condition max F (X 1 ) < min F (X 2 ) in step 3 is not satisfied. Now in the second case we havex k 2 < x k 1 . Similarly, g(X 2 ) ≥ g(X 1 ). Thus, max F (X 2 ) ≥ g(X 2 ) ≥ g(X 1 ) ≥ min F (X 1 ), implying max F (X 2 ) ≥ min F (X 1), so the condition in step 4 is not satisfied.In both cases, the algorithm does not Reject and therefore Accepts. Thus, if F accommodates P, the algorithm always Accepts. Now suppose F is ε-far from P. This means that for any g ∈ P,δ D (F, g) = Pr X∼D [g(X) / ∈ F (X)] > ε.For each g ∈ P, define the setS g = {X ∈ X : g(X) / ∈ F (X)}.While S g may vary with g, we have D(S g ) > ε for each g ∈ P.For a fixed x -k ∈ R d-1 , definep g (x -k ) = Pr x k ∼D x k |x -k [g(x k , x -k ) / ∈ F (x k , x -k )],where D x k |x -k is the conditional distribution of x k given x -k . Then,E x -k [p g (x -k )] = Pr X∼D [g(X) / ∈ F (X)] > ε.When we sample X 1 and X 2 with the same x -k and independent x k values, the probability that both g(X 1 ) / ∈ F (X 1 ) and g(X2 ) / ∈ F (X 2 ) is Pr[g(X 1 ) / ∈ F (X 1 ) and g(X 2 ) / ∈ F (X 2 ) | x -k ] = p g (x -k ) 2 .Taking expectation over x -k , we getPr[g(X i ) / ∈ F (X i ) for i = 1, 2] = E x -k [p g (x -k ) 2 ] ≥ E x -k [p g (x -k )] 2 > ε 2 ,Under the event where both g(X 1 ) / ∈ F (X 1 ) and g(X 2 ) / ∈ F (X 2 ), consider the following cases. In the first case, x k 1 < x k 2 .Since g is monotonically decreasing, g(X 1 ) ≥ g(X 2 ). However, g(X 1 ) / ∈ F (X 1 ) and g(X 2 ) / ∈ F (X 2 ). To prevent any g ′ ∈ P from satisfying g ′ (X i ) ∈ F (X i ), it must be that max F (X 1 ) < min F (X 2 ). If max F (X 1 ) ≥ min F (X 2 ), a function g ′ could exist with g ′ (X i ) ∈ F (X i ) and satisfying monotonicity. In the second case, x k 2 < x k 1 , similar reasoning leads to max F (X 2 ) < min F (X 1 ). Therefore, when both g(X i ) / ∈ F (X i ) and the appropriate ordering of x k holds, the algorithm Rejects.Since the probability of both g(X i ) / ∈ F (X i ) occurring is at least ε 2 , and the conditions for rejection are met under these events, the algorithm Rejects with probability at least ε 2 . Thus, Algorithm 1 satisfies the conditions of a Proximity-Oblivious Tester for the property P of monotonically decreasing in a given dimension k.Proof: Assume that F accommodates P; that is, there exists a function g ∈ P such that for all X ∈ X , g(X) ∈ F (X). Since g is concave in the k-th dimension, for any X left , X mid , X right constructed as in the algorithm, the following inequality holds:g(X mid ) ≤ αg(X left ) + (1 -α)g(X right ), where α = X right [k] -X mid [k] X right [k] -X left [k] .Because g(X i ) ∈ F (X i ) for i ∈ {left, mid, right}, we have:min F (X mid ) ≤ g(X mid ), g(X left ) ≤ max F (X left ), g(X right ) ≤ max F (X right ).Substituting these into the concavity inequality, we obtain:min F (X mid ) ≤ α max F (X left ) + (1 -α) max F (X right ).Therefore, the condition in line 7 of the algorithm is not satisfied (i.e., the inequality for rejection does not hold), and the algorithm returns Accept.Now suppose that F is ε-far from P. This means that for any g ∈ P,δ D (F, g) = Pr X∼D [g(X) / ∈ F (X)] > ε.For each g ∈ P, define the set: S g = {X ∈ X : g(X) / ∈ F (X)}. While S g may vary with g, we have D(S g ) > ε for each g ∈ P. The probability that g(X mid ) / ∈ F (X mid ) is greater than ε, and similarly, the probabilities that g(X left ) / ∈ F (X left ) and g(X right ) / ∈ F (X right ) are each greater than ε. Thus, by independence of the samples, the joint probability that all three events occur is at least ε 3 :Pr g(X i ) / ∈ F (X i ) for all i ∈ {left, mid, right} ≥ ε 3 .Under the event where g(X i ) / ∈ F (X i ) for all i ∈ {left, mid, right}, we will show that no concave function g ′ ∈ P can satisfy g ′ (X i ) ∈ F (X i ) for all i.Assume for contradiction that there exists a concave function g ′ ∈ P such that g ′ (X i ) ∈ F (X i ) for all i ∈ {left, mid, right}.Since g ′ is concave in the k-th dimension, it must satisfy:g ′ (X mid ) ≤ αg ′ (X left ) + (1 -α)g ′ (X right ).Using the bounds from F (X i ):g ′ (X mid ) ≥ min F (X mid ), g ′ (X left ) ≤ max F (X left ), g ′ (X right ) ≤ max F (X right ).Substituting these into the concavity inequality:min F (X mid ) ≤ α max F (X left ) + (1 -α) max F (X right ).However, the algorithm's condition in line 7 specifies that:min F (X mid ) > α max F (X left ) + (1 -α) max F (X right ).This is a contradiction. Therefore, our assumption is false; no concave function g ′ ∈ P exists that satisfies g ′ (X i ) ∈ F (X i ) for all i. Since under this event (which occurs with probability at least ε 3 ) no concave function g ′ ∈ P can fit within F (X i ) at the points X left , X mid , X right , the algorithm correctly Rejects.Therefore, the algorithm Rejects with probability at least ε 3 , so the algorithm satisfies the conditions of a Proximity-Oblivious Tester for the property P of concavity in the k-th dimension.We emphasize that this claim follows directly from the original proof and result of conformal risk control (Angelopoulos et al., 2024) and we present it here mainly for completeness given our use of it in this multi-dimensional case.Proof of Proposition 1. Our proof closely mimics the original proof of boudned risk found in Angelopoulos et al. (2024) .EQUATIONΛ min = Min λ ∈ Λ : Rn+1 (λ) ≤ α λ′ ∈ arg min λ∈Λmin g(λ)The fact that inf λ L i (λ) = L i (λ max ) ≤ α and the fact that g is striclty increasing implies λ′ is well-defined almost surely. Since L n+1 (λ) ≤ B, we haveRn+1 (λ) = n n + 1 Rn (λ) + L n+1 (λ) n + 1 ≤ n n + 1 Rn (λ) + B n + 1 . So if λ is such that n n + 1 Rn (λ) + B n + 1 ≤ α,Then we know Rn+1 (λ) ≤ α. So when there exists λ ∈ Λ such that the the above inequality holds, this implies that either λ′ ≤ λ or λ ∈ Λ min , which in either case implies g( λ′ ) ≤ g( λ). And if n n+1 Rn (λ) + B n+1 > α for all λ ∈ Λ, then we must have λ = λ max ≥ λ′ . So we have g( λ′ ) ≤ g( λ) almost surely. And since L i (λ) is non-increasing with respect to g(λ) we haveE L n+1 ( λ) ≤ E L n+1 ( λ′ ).Let E = {L 1 , . . . , L n+1 }, where L n+1 (λ) | E ∼ Uniform(E) by exchangeability, and λ′ is constant conditional on E. Then with the right-continuity of L i in each dimension we haveE L n+1 ( λ′ ) | E = 1 n + 1 n+1 i=1 L i ( λ′ i ) ≤ α.Then applying the law of total expectation and our inequality from above we haveE L n+1 ( λ) ≤ α,completing the proof.We present more examples of the conformal alignment procedure for monotonicity from Section 4.1.We keep everything the same in terms of 70-15-15 train-calibrate-test split, averaging over 10 runs, and using XGBoost regression models. All experiments were run on an Apple M1 MacBook Pro.We train one constrained model on the train and calibration sets and one unconstrained model on the training set. We use default parameters without hyperparameter tuning. We use the same POT T to obtain loss functions for applying conformal risk control on the unconstrained model using the calibration set. We report the results of applying the conformal alignment procedure for the property of concacity using Algorithm 2 as our POT. Implementation details are the same as for the monotonicity examples in Section 4.1 besides this change of POT and our procedure for choosing a specific point in order to measure performance on the accuracy metric, which here we take to be MAE. In this case our choice of the point prediction uses a scaling adjustment factor based on the position relative to the feature value at the maximum prediction, scaling from adding 0 to the predictions at the minimum feature value to adding λ to the prediction at the feature value of the maximum prediction, then adding 0 again at the maximum feature value. We re-emphasize, though, that the actual point prediction is not our main concern but rather the risk, which we can see matches the anticipated values well. We begin by describing the stylized model setting that can help us better understand the impact of overparameterization and data quality on a model's ability to satisfy a property.Notation. We consider feature vectors X to be samples from the uniform distribution on the sphere of radius√ d in R d , that is denoted by S d-1 ( √ d).For a function f , defined on a domain of the random variable X, we define itsL 2 norm by ∥f ∥ 2 L 2 = E X [f (X) 2 ]where the expectation is with respect to the randomness in X. The notation o n,P (1) for a positive integer means a quantity that goes to zero as n goes to infinity.Data model. Consider a model where the data is generated according to the following equation,y = g(X) + h(X) + ε ,where ε has mean 0 and variance τ 2 and is independent of X. Assume that h(X) represents 'bad' data that is undesired and we do not want to learn. That is, we only want to learn g(X), but there is a presence of biased noise, h(X), in the data generating model.Relating this to an alignment setting, domain experts expect the data to be generated according to g(X) + unbiased noise, such that g(X) exhibits some desired property P/ However, the data is generated according to g(X) + biased noise, potentially obscuring the desired behavior. One potential reason for the presence of biased noise in data could be due to a measurement error of the outcome that is correlated with select features, leading to an incorrectly calculated outcome. A biased measurement error could occur if there is incomplete data and the presence of the incomplete data is correlated with select features in an unexpected, systematic way (e.g., it could hypothetically be correlated to the type of hospital or location at which a patient is treated).The goal of these analyses is then to examine the impact of the noise bias term on a model's ability to describe a behavior that is upheld by g(X), but not necessarily g(X) + h(X), across model complexity. For example, g(X) might be monotonically increasing in the first coordinate of x, but the summation of g(X) + h(X) is not always. We will then try to draw conclusions about the distance between two kinds of models: one that is trained regularly without additional constraints, and another where we add a constraint to the training that forces the model to replicate the behavior of g(X), despite h(X). In the stylized model, when referring to the 'distance' between two models, we consider it to be zero if their learned parameters are the same, and non-zero if their learned parameters are different, when trained on the same data.Learning method. We will assume we have access to a data set of n iid samples {(X i , Y i )} n i=1 obtained from the above data generating model, i.e., Y i = g(X i ) + h(X i ) + ε i . We denote this data set by D n .The function class we use to fit the data is the random feature model, which can be thought of as a two-layer neural network, with N hidden nodes, with the weights of the first layer set randomly, while the second layer weights are learned. This means the function class can be written as:EQUATIONwhere a ∈ R N denotes the second layer weights and σ(•) : R → R is a non-linear activation function.The random weights {w j } j∈[N ] are such that each w j is an iid sample from a fixed distribution. For brevity, we drop the reference to a and w j 's and use f RF (X) or f RF (X; a).For the stylized model analysis, we consider a random feature model, because this function class has been studied in recent theory of deep learning models, and has shown some of the specific properties of modern overparameterized deep learning models such as concepts of benign overfitting and double descent Mei and Montanari; Misiakiewicz and Montanari (2023) . We can thereby use this recent literature as a theoretical foundation to explain the potential impact of model complexity on model's ability to satisfy a property in large train set sizes.Interestingly, the squared loss function for this problem is convex and we can minimize it by ridge regression. Mathematically, this means for any regularization parameter, λ, we will search for a vector of real numbers âλ ∈ R N such that, âλ = arg mina∈R N i∈[n] (Y i -f RF (X i )) 2 + λ∥a∥ 2 2 .(2)Assume a certain property, denoted by P such that the function g(X) satisfies P. However, assume that h(X) is such that g(X) + h(X) does not satisfy P.Assume there is a way of constraining the training of f RF (X) to satisfy the P. More formally, assume C P ⊂ R N is the set of all coefficients a such that f RF (X; a) satisfies P, i.e.,C P = {a | a ∈ R N and f RF (X; a) satisfies P} .Then we can consider the following constrained version of the model:EQUATIONNow assume that the functions g and h are polynomials such that deg g < deg h . Further assume that n > d deg h +δ , where δ is some small positive constant (e.g., δ = 0.1) and that d is sufficiently large. Also assume that the activation function σ upholds the following genericity conditions (as stated and assumed in Theorem 3 in Section 4.2 of Misiakiewicz and Montanari ( 2023)): 1) |σ(x)| ≤ c 0 exp(c 1 |x|) for constants c 0 , c 1 > 0; 2) for any k ≥ 0, σ has a non-zero Hermite coefficient µ k (σ) := E G {σ(G)He k (G)} ̸ = 0 (where G ∼ N (0, 1) and He k is the k-th Hermite polynomial, with the standard normalization ⟨He j ,He k ⟩ L 2 (N (0,1)) = k!1 {j=k} ).To analyze the impact of model complexity on a model's adherence to P, we consider two settings for the stylized model:• Classic: d deg g +δ < N < d deg h -δ . • Underspecified: N > d deg h +δ .To formalize our results, we will in part rely on Theorem 3 in Section 4.2 of Misiakiewicz and Montanari (2023) . This theorem illustrates the roles that the number of hidden nodes, N , and the number of data points, n, play in determining the test error that can be achieved by a random feature model when approximating a function. Given the assumptions made in their work, in the setting where N ≪ n, the number of hidden nodes, N , will limit the test error, while if n ≪ N , the number of data points, n, will limit the test error Misiakiewicz and Montanari (2023) . Building on this intuition, we first state an informal implication of this theorem Misiakiewicz and Montanari (2023) applied to our stylized model: under the same assumptions made in their work, for a sufficiently large d, in the classic setting, there exists a range of regularization parameters [0, λ * ] such that for all λ in that range, f RF (X; âλ ) will learn a polynomial of at least deg g , but strictly less than deg h . But in the underspecified setting, f RF (X; âλ ) learns both parts g(X) and h(X), for all λ in a different range of regularization parameters [0, λ * ], meaning it learns the noise bias. This means, in a large data domain, the behavior of the classic and underspecified setting is different.In order to formalize this intuition for the underspecified setting, we need to state an assumption on the degree of violation of property B around the function g + h. Assumption 1 (Property Unsatisfied in a Neighborhood). There exists a positive constant R such that, for all a ′ with ∥(g + h) -f RF (•; a ′ )∥ L 2 ≤ R, the function f RF (X, a ′ ) does not satisfy the property P.Theorem 2 (Non-vanishing Distance in Underspecified Setting). Assume N > d deg h +δ , n > d deg h +δ , max(N/n, n/N ) ≥ d δ for a constant δ > 0, Assumption 1 holds, σ upholds the genericity conditions stated above, and d is sufficiently large. Then, in the underspecified setting, there exists a constant λ * , such that for any λ ∈ [0, λ * ] the following holds:EQUATIONProof. Using Theorem 3 of Misiakiewicz and Montanari (2023), we have that∥(g + h) -f RF (•; âλ )∥ 2 L 2 = o d,P (1) • (∥g + h∥ 2 L 2+η + τ 2). for all η > 0. With d sufficiently large, the square root of the quantity on the right hand side will be less than R, which combined with Assumption 1, means f RF (•; âλ ) does not satisfy P, and therefore, âλ / ∈ C P .Under two additional assumptions, we can further state a result showing that in the classic setting, for a given range of regularization parameters, we will be able to learn the property P. At a high level, these assumptions are that 1) the overall impact of h is small, for example, when it only impacts a small portion of the data; and 2) the property P is preserved in a small neighborhood of g among the function class learned in the classic setting. Formalizing the assumptions and stating the result: Assumption 2 (Small Noise Bias). Assume there is a small constant ϵ h such that∥h∥ L 2 ≤ ϵ h .Assumption 3 (Property Robustness in Function Class). For any fixed N , there exists a positive constant R N such that, for all a ′ with ∥g - âλ,P = âλ .f RF (•; a ′ )∥ L 2 ≤ R N ,(5)Proof. Using the triangle inequality and Theorem 3 of Misiakiewicz and Montanari ( 2023) again, we have that∥f RF (•; âλ ) -g∥ 2 L 2 ≤ 2(∥f RF (•; âλ ) -g -h∥ 2 L 2 + ∥h∥ 2 L 2 ) ≤ 4∥h∥ 2 L 2 ≤ 4ϵ 2h . Taking the square root, the right hand side of the last term is less than R N which combined with Assumption 3 means f RF (•; âλ ) satisfies property P.Remark 1. Note that the constant R N will converge to zero as N grows, meaning the assumption R N > 2ϵ h will break down when N is large. For example, in the underspecified setting, this assumption breaks down and hence the distance does not vanish.In this secton, we first numerically validate the above theoretical results, in a setting that noise bias only impacts a small fraction of the pre-training data. Next, we apply our proposed procedure of alignment through conformal risk control. We take the desired property P to be monotonicity and use the same procedure as on the real-world datasets in Section 4.1.We consider g(X) = ⟨X, β⟩ with β a constant in R d with β 1 > 0 and ∥β∥ 2 = 1. Then let h(X) = min(M, αZ p x 4 1 ) where Z p is a Bernoulli random variable with a success probability p, that is sampled independently for each data point, and α and M are positive constants. This function is aimed to model a biased noise that is capped by magnitude M and only applies to a fraction p of the population. This is to mimic characteristics of a real dataset where only a portion of the data might have the bias (i.e., 'bad' data) . This is therefore capturing a more realistic case of the stylized model setting, where the measurement error may only occur for a small subset of the data. Using a large train set size, we fit the data generated according to the outcome model above using the random feature model with λ tuned via cross-validation, assuming the ReLU activation function, i.e., σ(x) = max(0, x), and w j is iid N (0, 1). We then assess how model complexity affects whether the predicted function is monotone in x 1 by looking at the univariate partial dependence plot of the predicted function versus the (counterfactual) unbiased outcome, g(X), for the first coordinate x 1 . We vary model complexity by N , the number of hidden nodes in the model.Applying this synthetic simulation for specific parameters, Figure 2 shows that when training a relatively simple model (N = 5) on the generated data, the model describes a monotonically increasing relationship. From this we conclude that if we had constrained the model to uphold monotonicity in x 1 , it would not have changed the final model's parameters. Hence, the distance between the constrained and unconstrained model would be zero and we require no conformal correction. In contrast, as seen in Figure 3 , a more complex model (N = 5000) predicts a nonmonotone relationship, and therefore, if we had constrained the model to uphold monotonicity in x 1 , it would have changed the final model's parameters, such that the distance between the constrained and unconstrained model would be non-zero and we do require a conformal correction to post-process and align this model.The synthetic simulation seems to validate that with a seemingly large training dataset, increasing model complexity, on the one hand, can enable high performance, but can also make ML models more sensitive to small noise bias in the data, which can subsequently lead to model behavior inconsistent with requirements by the end-user. Although it is not the main focus of this paper, we show in this section how our general methodology connects to the work of Yadkori et al. (2024) . They examine a potentially random LLM f : X → Y and assume access to a confidence score function g : X → R. They use an abstention function a : Λ × X → {0, 1} that decides whether the model should abstain from answering where a λ (X) = 1 if g(X) < λ and a λ (X) = 0 if g(X) ≥ λ . The match function m : X × Y × Y → {0, 1} is used to determine if a response Y ′ is semantically equivalent to Y , indicating hallucination if not. Yadkori et al. (2024) use conformal risk control to find an optimal λ such that the pair (a λ, f ) hallucinates at a rate less than α. The loss function they use is ℓ : X × Y × Λ → R such that ℓ punishes failure to abstain when the response does not match the label, that is EQUATIONIn this case we do not change the outputs of the function f itself, instead we use conformal risk control to change the outputs of a. Consider the property P to be that (a, f ) does not hallucinate. A tester for P is then simply to query (a, f ) on some input X and Reject if f hallucinates or Accept otherwise. Now when g(X) < λ, we have C λ (X) = {f (X), Abstain} meaning that (a, f ) can avoid hallucinating, so the tester will Accept.Yadkori et al. ( 2024) use a separate conformal procedures for setting λ and a parameter β with which they define m(X; Y ′ , Y ) = I(s(X; Y ′ , Y ) ≥ β), where the similarity function s : X × Y × Y → R is assumed to be given. We can use our multi-lambda conformal risk control with λ = (λ a , λ m ) to set these parameters simultaneously using a single calibration set {(X i , Y i )} n i=1 of ground truth query-response pairs sampled from the true data distribution D. Letℓ(X, Y ; λ) = (1 -a λa (X))(1 -I(s(X; f (X), Y ) ≥ 1 -λ m ))Then we can take g(λ) = c 1 λ a + c 2 λ m for c 1 , c 2 ≥ 0 and choose λ by our multi-lambda conformal risk control preocedure. As required, this loss function is non-increasing with respect to Λ and g(λ).Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The paper reports mean squared errors and risks, and the experiments are averaged over multiple random splits, providing sufficient information on variability and significance suitable for our context. See Section 4 and Appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: The paper specifies the use of XGBoost models and mentions typical execution times and resources used for the experiments. See Appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research adheres to ethical guidelines by ensuring transparency, reproducibility, and careful consideration of societal impacts, especially in the context of model alignment and fairness.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The discussion includes potential positive impacts of improved model alignment on fairness and transparency, as well as the challenges and limitations that need to be addressed. See Section 1. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does not involve high-risk data or models. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [NA] Justification: The paper does not involve crowdsourcing or research with human subjects. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.38th Conference on Neural Information Processing Systems (NeurIPS 2024). 
 
 Appendix:  


Decision: This work discusses model alignment, and suggests a new frameworks towards doing so by combining ideas from property-testing and conformal risk prediction. And this idea seems novel and interesting as mentioned by some of the reviewers.
Nevertheless, the writing and clarity of the manuscript need to be improved as per the comments of the reviewers; and the technical novelty is quite mild. Nevertheless, conceptually the paper does bring novelty, and may very well spark new research directions.
From these reasons I view the paper as borderline, with tendency to accept.

Reviews: review: paper_summary: paper_summary: This paper proposes a method to post-process a pre-trained model to align with a subset of hypotheses on which the specific desired behaviors can be attained. The proposed method relies on proximity oblivious testers to give detection for the misalignment, based on which a conformal risk control process is used to calibrate the prediction interval to guarantee the farness to the desired subset of hypotheses.
strength_weakness: strength_weakness: 1. The proposed aligning method is largely built on conformal risk control to calibrate the prediction interval, so the original technical contribution needs to be highlighted.
2. The generalization to multi-dimension conformal risk control is a bit straightforward and an immediate result from the original conformal risk control. The technical challenges need to be highlighted also.
3. The proposed method depends on the reliability of the POT, as mentioned in Line 181. However, to guarantee a rigorous prediction, it is important to consider the failure and errors from POT and how to guarantee the valid prediction in this case in details.
questions: questions: N.A.
limitations: limitations: N.A.
paper_summary: paper_summary: The authors propose a way to solve the problem of alignment in Machine Learning using Conformal Risk Control. They first expand the previous work of conformal risk control to multidimensional parameters $\boldsymbol{\lambda}$, then used this extension to propose a way to test if a function belongs to a certain class of functions $\mathcal{P}$ that we assumed are aligned with user interests. Finally, they demonstrate how this method can be used to test monotonicity of functions.
strength_weakness: strength_weakness: * There is only a single example with results. Presenting a method for concavity is interesting but it would be nice to see it applied to a real world examples
* There is still a significant gap between monotonicy/concavity and alignment as we understand it in AI.
questions: questions: see weakness in concavity.
limitations: limitations: * The authors advertise this paper as being useful in GenAI. What are examples of potential class of functions that would satisfy alignment with a generated text / image for example. It is indeed a good first step to test the belonging to a class, but defining that class can be very hard in the most important application
paper_summary: paper_summary: This article offers an alternative approach to alignment by testing whether trained models follow specific properties. This is done, for instance, on monotone or concave functions. A small modification for the use case is made to the Conformal Risk Control setting to allow vectorial parameters. Using CRC and property testers (POTs), a guarantee is obtained that the predictor approximately satisfies the property.
strength_weakness: strength_weakness: However, several parts could be improved. Section 2.1 is unclear, and the introduction of lambda without details is bound to confuse readers not familiar with CRC. 
Moreover, I found section 3.2 to be overly verbose and lacking clear, explicit mathematical formulations of the loss, particularly but not exclusively.
In case it is a misunderstanding, I believe a rewriting and clear formalism would help in understanding.
Moreover, I want to emphasize the lack of links between different sections of the papers. Notations, like the function "g" are reused with different meanings, and the threshold "epsilon" disappears from the method.

Additional typo: line 147 "setof".
questions: questions: Although early in the article, it's specified that " we can imagine a user who wishes to determine whether a pre-trained model f belongs to P but is unable to train a model of f ’s size" (42), line 178 proposes to "modify" the model. How is that done?
Moreover, in the appendix (table 4), unconstrained models have varying performance when alpha varies, how is that explained? (moreover, no notion of uncertainty is provided on the metrics)
limitations: limitations: I believe it's worth mentioning that this multidimensional  CRC is essentially one-dimensional, as it replaces the non-increasingness in lambda by one in a mapping of lambda to R. 
Moreover, the monotonicity in the l1 norm of lambda is often not verified (think of different slopes for the different dimensions), and several experiments do not use this result.
paper_summary: paper_summary: The paper connects conformal risk control to define prediction sets containing results that satisfy a property. In short, assume that function $f$ (a trained model) does not satisfy some property $\mathcal{P}$, we can define a prediction interval around results of $f$ s.t. those intervals satisfy $\mathcal{P}$ with $1 - \alpha$ probability. The authors first generalize conformal risk control to multidimensional conservative parameter, through a proxy from $\mathbb{R} ^d \mapsto \mathbb{R}$, and use that in addressing the property testing. They define the risk as if there is no output in the prediction interval that satisfies the property.
strength_weakness: strength_weakness: 1. Notations in the paper can be introduced in a better way. For instance a mathematic notation of $\mathcal{P}$, which can be a subset of the function space.

2. There are minor typos in the mathematical notation e.g. in line 88, $x \sim \mathcal{D}$ but $f(X) \neq g(X)$. Also I do not understand why the authors used $\mathrm{Min}$ instead of $\min$; I believe that might have to relate to the fact that they are minimizing over vectors but still it could be indicated by $\min_{\boldsymbol{\lambda}\in\boldsymbol{\Lambda}}$. Also shouldn't we find the minimal $\boldsymbol{\lambda} _\mathrm{min}$ instead of $\boldsymbol{\Lambda} _\mathrm{min}$?
questions: questions: 1. Why did the authors choose monotonicity and concavity as the properties to test. Is there any real-life use-case of those properties or does more justifiable properties remain as a possible future work? Ofcourse there is a dicussion on applications of monotonicity to causal inference, and fairness but is there any chance that the authors could provide experiments leading to those areas?

2. Definition 2 has some confusing points:
- Should there be any nested property over the prediction sets for various lambdas? Like if $\lambda_a \le \lambda_b$ then $\mathcal{C}(x_i; \lambda_a) \subseteq \mathcal{C}(x_i; \lambda_b)$? If yes then how the order is defined in this vector space? Does the nested property hold for a linear combination of $\lambda_i$ values or the nested property should be present for any $\lambda_i$. 
- How the interval is defined over function? Is it just the function value $\pm$ some radius around it or is it a something apply over function parameters?

3. In line 156, shouldn't we find the minimal $\boldsymbol{\lambda}_\mathrm{min}$ instead of $\boldsymbol{\Lambda}_\mathrm{min}$?
limitations: limitations: To the best of my understanding, the paper defines the guarantee over a property, and a joint guarantee of property + accuracy is not defined in its framework. I see that their extension of risk control supports multi-dimensional set constant not a multi-dimensional risk. However, I believe this is not a shortcoming of the method but a possible open problem to solve.

Comments: comment: We would like to thank the reviewer again for their initial review and feedback on our paper. We hope that the reviewer may find our provided discussions and clarifications within our rebuttal convincing, but we would also be happy to address any further questions the reviewer may have regarding our paper or our rebuttal. Thank you again for your time and effort.
We appreciate the reviewer's increased score. Thank you for the continued engagement with our paper and for appreciation of the improved clarity and corrected typos. Thank you again.
Thank you for your continued engagement with our paper and maintaining your positive review of the work. We are happy to hear that our answer to reviewer 9EgD with regards to the multidimensional $\lambda$ and our discussion regarding applications to LLMs was convincing. Thank you again.
I appreciate the author's rebuttal, the improved clarity, and the fact that some inconsistencies were typos. I will increase my score accordingly.
Thank you for your taking the time to provide this detailed answer.

I was also questioning the improvement from the multidimensional $\lambda$ but have been convinced by your answer to reviewer 9EgD. 

The applications to LLMs with the non hallucination is also pretty convincing. Running an experiment against Yakdori et al. (2024) method would have made it clearer to the reader on how well it improves from that baseline to use the multidimensional $\lambda$ instead of separate ones, but obviously there is not enough time for it.

I still  think 7 is the appropriate rating for this paper.