Latent Feature Mining with Large Language Models

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 18:48:45

Abstract: Predictive modeling often faces challenges due to limited data availability and quality, especially in domains where collected features are weakly correlated with outcomes and where additional data collection is constrained by ethical or practical difficulties. Traditional machine learning (ML) models struggle to incorporate unobserved yet critical factors.  We propose a framework that leverages large language models (LLMs) to augment observed features with latent features, enhancing the predictive power of ML models in downstream tasks. Our novel approach transforms the latent feature mining task to a text-to-text propositional reasoning task. We validate our framework with a case study in the criminal justice system, a domain characterized by limited and ethically challenging data collection. Our results show that inferred latent features align well with ground truth labels and significantly enhance the downstream classifier. Our framework is generalizable across various domains with minimal domain-specific customization, ensuring easy transfer to other areas facing similar challenges in data availability.

Main paper: 
 introduction: 
 In numerous application domains, predicting individual outcomes and optimizing resource planning are critical but often limited by gaps in data availability and quality. Despite the popular belief that we operate in a "large data regime," many decisions, especially those impacting human lives, have to be made based on small amounts of data with limited features, such as in criminal justice, healthcare, and social services (Lu et al., 2021; Yuan et al., 2023) . This poses both technical limitations and ethical concerns. Traditional ML models, while powerful, are limited by the availability of collected (observed) data features. This limitation is especially prominent when it comes to incorporating unstructured data or inferring nuanced relationships between observed features and the outcomes. In this paper, we explore how domain-informed language models can help identify latent (unobserved) features and improve prediction accuracy for downstream tasks.We illustrate our motivation with an example from the criminal justice setting. Predicting an individual's in-program revocation probability (chance of committing a new crime during probation) is critical for determining their eligibility for incarceration-diversion programs and for planning resources like staffing ratios (Rotter and Barber-Rioja, 2015; Li et al., 2024) . Typically, the data collected includes only a limited set of features, e.g., basic demographic and criminal history information. Crucial factors such as socio-economic status, community support availability, or psychological profiles, which significantly influence outcomes, are often missing from these datasets.Collecting such sensitive information can be invasive and raises ethical concerns. Additionally, the process of gathering these data can be logistically challenging and resource-intensive. Human case managers in these settings often have the advantage of drawing on their professional experience and human intuition to infer these critical but unrecorded details from observed data. In contrast, traditional ML models cannot reason beyond the explicit data provided, leading to predictions based on incomplete information. This limitation not only undermines the accuracy of the models but also Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.poses concerns regarding the fairness of decisions derived from such data. Moreover, ML models are not designed to handle unstructured data like case notes, which may contain contextual insights to improve prediction accuracy.Recent advancements in large language models (LLMs) offer a promising avenue to bridge these data gaps (Brown et al., 2020; Ouyang et al., 2022; Achiam et al., 2023) . LLMs are capable of processing and generating information in a way that mimics human reasoning, allowing for the inference of latent features that are not directly observable but are critical for accurate predictions and decision-making.They can also analyze both structured and unstructured data to offer a holistic view of the underlying factors influencing individual outcomes.Our proposed framework leverages LLMs to augment observed features collected in given datasets with latent features, enhancing the predictive power of ML models for downstream tasks such as classifications. Unlike conventional data augmentation approaches to increase the sample size, we train LLMs to infer underlying socio-economic conditions, treatment needs, and other critical but often unrecorded characteristics from collected features. This augments the feature space X to improve predictions. Additionally, our framework enables generating more complete and realistic synthetic data points via learned correlations between observed and unobserved features for simulation and counterfactual policy analysis. We summarize our main contributions as follows.1. We introduce a novel approach to formulate latent feature mining as text-to-text propositional logical reasoning. This approach effectively infers latent features from observed features, offering significantly improved accuracy and interpretability compared to alternative approaches.2. We develop a four-step framework to implement our approach, which is generalizable with minimal domain-specific customization and has remarkably low human-annotated training data requirements. This framework expands data utility by enhancing downstream predictions without additional invasive or forbidden data collection.3. We empirically validate our framework in the criminal justice setting to address weak observed features and unbalanced datasets. Designed as a plug-and-play solution, we demonstrate our framework's adaptability through two different prediction tasks, making it valuable for various applications with similar challenges. 
background and related works: 
 Data Augmentation and Latent Feature Extraction. Data augmentation is a technique commonly used in AI (Van Dyk and Meng, 2001) . Generative models, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), learn data patterns and generate synthetic data to augment training sample size (Goodfellow et al., 2014; Kingma and Welling, 2013) . Unlike these approaches, our framework leverages LLMs to augment the features of different individuals. Trained on crowd-sourced data rich in human behavior and societal context, LLMs have the potential to enhance feature spaces for social computing and operations improvement.Latent features are hidden characteristics in a dataset that are not directly observed but can be inferred from available data. Incorporating meaningful latent features can enhance the performance of downstream applications (Zhai and Peng, 2016; Jiang et al., 2023) . Two common approaches to infer latent features are human annotation and machine learning models. Human annotation, while reliable, is often expensive and time-consuming. It requires significant effort and resources, making it impractical for large-scale tasks. Machine learning methods like Expectation-Maximization (EM) and VAEs offer alternative techniques to infer latent features from observed data. EM algorithms estimate latent variable assignments and update model parameters to maximize data likelihood, but their results can be hard to interpret and require strong parametric assumptions. Similarly, VAEs use probabilistic approaches to describe data distribution with latent variables, but the learned mappings can also be difficult to interpret.Synthetic Data for Training. Fine-tuning is a promising approach for LLMs to reduce hallucinations and align outputs with real-world data and human preferences (Tonmoy et al., 2024; Qiao et al., 2022; Hu et al., 2021) . Synthetic data has proven to be an effective, low-cost alternative to real data to improve the LLMs' reasoning performance across various domains (Liu et al., 2024) . Studies by (Zelikman et al., 2022) , (Wang et al., 2022) demonstrate that synthetic data improves model generalization and robustness. Our approach also uses synthetic data to augment training during fine-tuning. Unlike existing work that directly mimics observed features, we are one of the first to formulate the generation of synthetic latent features as a reasoning task. Our approach employs few-shot prompting to create synthetic data that infers these latent features, followed by fine-tuning to enhance model accuracy and reduce hallucinations. This technique falls under the self-instruction paradigm, where models iteratively learn from augmented data.Note that we distinguish between augmenting the feature space and augmenting training data. Our primary goal is to augment the feature space by inferring and adding latent features to the observed data to improve downstream predictions. As part of the steps in our framework to achieve this goal, we augment training data for LLM fine-tuning with synthetic samples to improve the model's reasoning capabilities. We obtained de-identified data from our community partner for a state-wide incarceration-diversion program in Illinois. The consolidated dataset includes records of adult participants admitted to the program. The collected data features include timestamps such as the arrival and termination dates to the program, program outcomes, and individual features such as the race, gender, education, county, marriage status, housing, risk assessment scores, prior crime history, and sources of referral (e.g., from probation officer or from the court). See Appendix F for summary statistics.Question: Does the paper discuss the limitations of the work performed by the authors?Answer:[Yes] Justification: Yes. We discuss the limitations in section 7.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
the problem setting: 
 In this section we formally describe our problem setting that leverages latent features to enhance downstream tasks. The downstream task we focus on is a multi-class classification problem, but the framework can easily extend to other downstream prediction tasks such as regression problems.In a standard multi-class classification problem setting, suppose we have a dataset D = (x 1 , y 1 ), (x 2 , y 2 ), . . . , (x n , y n ), where x i is a d-dimensional vector representing the input features X ∈ X and y i ∈ Y = {1, 2, . . . , C} denotes the corresponding class label Y for individual i = 1, . . . , n. The goal is to learn a classifier f : X → Y that accurately predicts the class labels.Consider the following scenarios in which f struggles to capture the relationship between X and Y :1. The size of the training dataset is small relative to the complexity of the classification task or the dimensionality of the feature space; 2. When the input features X are weakly correlated with class labels Y , the input features may not provide discriminating information to accurately predict the corresponding class labels.To address these challenges, we could use additional informative features to enhance the classifier's ability to capture the relationship between X and Y . Latent features can serve such a purpose.Latent features, denoted as Z, represent underlying attributes that are not directly observed within the dataset but are correlated with both the observed features X and the class labels Y . We use a function g with Z = g(X) to denote the correlations between the latent features and the observed features X. As shown in figure 3 , latent features Z are correlated with X and Y . One can learn the latent features from the original features X and augment the features f (X, Z) to learn the classifier Y .In typical ML settings, latent features primarily reduce the dimensionality of the feature space.Beyond this, latent features can capture discriminative information not explicitly present in the original features. Our approach focuses on this latter benefit, extracting informative latent representations to help classifiers better differentiate between classes. Essentially, Z acts as ensemble features derived from the original features X, capturing complex patterns that individual features might miss, especially when X is weakly correlated with the outcome Y .While this approach seems beneficial intuitively, it is important to note that adding more features is not always helpful if the extracted features are not meaningful and introduce noise. In the following lemma, we show in a simple logistic regression setting that while adding features can reduce in-sample loss, it does not always reduce out-of-sample loss if the added features are not informative. We use the log-loss (the cross-entropy loss) of the logistics regression for binary outcome Y ∈ {0, 1}. We denote the optimal coefficients that minimize the in-sample log-loss function as β * for the original features and β * for the augmented features.Lemma 1. The in-sample log-loss always follows L in ( D, β * ) ≤ L in (D, β * ). When the added features are non-informative, there exist instances such that the out-of-sample log-loss L out ( D, β * ) >L out (D, β * ).The results in the lemma can be generalized to multi-class labels. Since augmenting the feature space is not necessarily beneficial unless the added features are meaningful, a major part of our case study is to empirically test whether the extracted features from our framework indeed improve downstream prediction. If the added features significantly enhance downstream prediction accuracy, this provides strong evidence that the inferred latent features are meaningful.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: Access to the data and code is restricted under the terms of the non-disclosure agreement signed with our data-providing partner. The code includes sensitive details pertinent to the data, such as specific information embedded within the prompts. Consequently, we are unable to share the code at this time.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
latent feature mining with llms: 
 To overcome the limitations of existing approaches, we propose a new approach to efficiently and accurately extract latent features and augment observed features to enhance the prediction accuracy.At a high level, our approach transform the latent feature mining as a text-to-text propositional reasoning task, i.e., infer the relationship Z = g(X) through logical reasoning with natural language.Following the framework established in previous work (Zhang et al., 2022) , we denote the predicates related to the observed features as P 1 , P 2 , . . . , P m . Consider a propositional theory S that contains rules that connect P 's to the latent feature Z. We say Z can be deduced from S if the logic implication(P 1 ∧ P 2 ∧ . . . ∧ P m ) → Z is covered in S.For potentially complicated logical connections between P 's and Z, we also introduce intermediate predicates O's and formulate a logical chain (a sequence of logical implications) that connects X to the latent features Z as follows:X → (P 1 ∧ P 2 ∧ . . . ∧ P m ) → (O 1 ∧ O 2 ∧ . . . ∧ O ℓ ) → Z.(1)Our approach formulates this logical chain as a multi-stage Chain of Thoughts (CoT) prompt template, and then guide LLMs to infer Z from X using the prompt template. Specifically, we first extract predicates P 's from X. for the predicates P 's formulated from X could be:P 1 :"the client has part-time job", P 2 : " the client hasn't complete high school", P 3 :"the client is single", P 4 : "the client has drug issue", P 5 :" the client lives in high crime area", P 6 : " the client is assessed with high risk" ...To infer the latent feature Z -in this example, the required programs to attend during probation -we go through a multi-stage reasoning to infer the intermediate predicates O's; see the white (middle) boxes in Figure 1 . One example logic that connects P 's to O's could be:P 1 = "The client has unstable employment"P 2 = "The highest education level of client is less than 10th grade"O 1 = "The client has low socioeconomic status"If (P 1 ∧ P 2 → O 1 ) ∈ S, then O 1 is True.Finally, with P 's and O's, we can connect X with Z though the logic chains. One example of the logical chain is as follows:"The client is grappling with unstable employment and a relatively low educational level, factors that likely contribute to a low socioeconomic status. Additionally, being single, struggling with drug issues, and residing in a high-crime area further exacerbate the lack of positive social support. Given these circumstances, education could serve as a valuable intervention. Community service can be particularly beneficial for someone who is single and may lack a broad support network.Substance abuse treatment is crucial for individuals from lower socioeconomic backgrounds to aid in recovery from substance abuse. Hence we can choose education, substance abuse treatment, community service for this client."Here, "unstable employment and a relatively low educational level" and "being single, struggling with drug issues, and residing in a high-crime area" are P 's extracted from the features X, while "a low socioeconomic status" and "lack of positive social support" are O's. Finally, the rationales "education could serve as a valuable intervention . . . recovery from substance abuse. Hence we can choose education, substance abuse treatment, community service for this client connect the intermediate predicates to the latent variables Z (program requirements) we want to infer, i.e., Z 1 ='education', Z 2 ='substance abuse treatment', Z 3 ='community service'.Step 1: Formulate Correlation (1) Formulate baseline rationales: The first step is to formulate baseline rationales, whic serve as guidelines for LLMs to infer latent features from observed ones. This involves two sub-steps:-The first sub-step is to develop some baseline rationales, i.e., identify observed features potentially correlated with latent features and formulate their relationships -the logic chain that connects X to Z.Sources to help formulate these baseline rationales include established correlations (e.g., risk score formulas), human input, and external information like socio-economic status in the neighborhood.-In the second sub-step, we craft prompts with interactive alignment. This is a critical component to establish correct reasoning steps for prompts used in Step 2 to generate synthetic rationales. We involve human who are experienced in the domain to provide a prompt template for LLMs to generate rationales aligned with the baseline rationales, then test the prompt template on a few examples using zero-shot. If the LLM fails to certain example, we provide the ground truth back to the LLM, allowing it to revise the prompt template (Miao et al., 2023) . This process iteratively refines the template until LLMs consistently generate the desired output for all selected examples.(2) Enlarge data with synthetic rationales for fine-tuning: We generate synthetic training data in self-instruct fashion (Wang et al., 2022) . With a handful of examples of the baseline rationales as a reference, we then guide the LLMs via in-context learning to generate similar rationales to enlarge the training data samples. To ensure the quality and diversity of the generated dataset, we introduce human-in-the-loop interventions to filter out low-quality or invalid data based on heuristics. We also leverage automatic evaluation metrics for quality control, e.g., removing data that lack essential keywords.(3) Fine-tuning LLMs: To enhance the reasoning capabilities of the LLMs and better align their outputs in specific domains, we employ a fine-tuning process which utilizes the processed dataset from the previous step (Qiao et al., 2022) . Fine-tuning not only boosts the accuracy and reliability of the LLMs, but also significantly improves their ability to reason with complex inputs, and reducing hallucination (Tonmoy et al., 2024) .(4) Latent feature inference: The fine-tuned model is able to mirror the nuanced decision-making process of human experts. We use the fine-tuned model to identify latent features and feed them into downstream prediction tasks.Regarding the generalizability of our framework, Steps 2-4 rely primarily on the mechanics of LLMs, which naturally have a high degree of adaptability across different domains.Step 1, which involves the identification and formulation of baseline domain-specific rationales, requires more expert knowledge. To assist with Step 1, our interactive-alignment strategy can help craft effective prompts by allowing iterative refinement based on feedback, reducing the burden on domain experts. 
experiments setup: 
 In this section, we demonstrate the efficacy of our proposed framework on a unique dataset from a state-wide incarceration diversion program as described in Section 2. We design two sets of experiments to empirically investigate: (1) Can our approach accurately imitate the human thinking process to infer latent features? (2) Is our approach more effective than alternative techniques to infer latent features? (3) Does our approach enhance the performance of downstream prediction tasks?In the first experiment, we treat the risk level of individuals as a latent feature, despite it being collected in the dataset. This experiment examines whether the latent features Ẑ inferred by LLMs match well with the actual features Z. In the second experiment, we assume that the program requirements are latent features, which lack ground truth labels for most individuals (only a few dozen individuals have the program requirements recorded in the data). We first have LLMs deduce these requirements, then add them to the downstream prediction task of program outcomes Y ∼ f (X, Ẑ) and evaluate whether the prediction accuracy is improved, i.e., the inferred features are indeed beneficial and not detrimental (recall the results in Lemma 1).Task Description. In this task we treat an observed feature-Risk Level-as the latent feature to infer. The task is a multi-classification problem to learn Z ∼ g(X) among four labels for the latent variable Z ∈ {moderate, high, very_high} based on each client's profile X.Implementation Details. We implement our proposed framework as follows. All prompt templates are attached to Appendix section C.-Step 0. Profile writing: In this pre-processing step, we translate structured profile data X into text that can be better handled by LLMs, i.e., formulating predicates P 's from the features X. To enrich the profile with important in formations that could potential benefits the following steps, we formulate the intermediate predicates O's, where we prompt LLMs to extract and summarize underlying information such as background, socio-economic status, and challenges in two or three sentences. We then merge these sentences into the client's profile. We use zero-shot prompting with GPT-4 for this step.-Step 1. Formulating rationales: Using human input, established risk score calculations (Corrections), and the code book with risk calculation details provided by our community partner, we summarize a general rule for inferring risk levels from the predicates, i.e., establishing the logic chains from P 's and O's to Z. We then sample 40 client features from the dataset and manually formulate 40 baseline rationales that logically connect features to corresponding risk levels and that are aligned with the high-level general rule. To avoid the primacy effect of LLMs, we rate risk scores from 0 to 10 to add variability in the labels, categorized as follows: 0-4 (moderate risk), 4-7.5 (high risk), and 7.5-10 (very high risk).-Step 2. Enlarge fine-tuning data: With the 40 baseline rationales, we generate additional synthetic rationales. We sample client features and corresponding ground truth risk scores from the dataset, using one of the 40 rationales as an example, to prompt LLMs to produce similar narratives with CoT prompts. In total we got 3000 rationales for the training data.-Step 3. Fine-tune LLMs: Our framework is designed to be plug-and-play, allowing the synthetic data generated in the previous step to be used across different language models. We fine-tune two pre-trained language models for cross-validation purposes: GPT-3.5 and Llama2-13b(OpenAI, 2021) .We use OpenAI API to fine-tune GPT-3.5-turbo-0125 (Touvron et al., 2023; OpenAI) . We fine-tune Llama2-13b-chat using LoRA (Hu et al., 2021) .-Step 4. Inference with LLMs: We prompt fine-tuned LLMs to infer risk level Ẑi from features X i for each client i in the test data and evaluate the out-of-sample accuracy by comparing the inferred latent variable (risk level) Ẑi with the ground truth label Z i .Evaluation. We choose ML classifiers (e.g., Neural Networks or Gradient Boosting Trees) as the baseline to infer Ẑi from features X i . We compare the prediction performance of Ẑi inferred from our approach with that from ML models using out-of-sample accuracy and F1 score. Additionally, we evaluate the quality of generated text with an automatic evaluation metric. In the pre-processing step, we assess the keyword coverage rate in the generated profile assuming each feature value is a keyword. For synthetic rationales, we use YAKE, a pretrained keyword extractor (Campos et al., 2020) , to identify keywords. We then evaluate the keyword coverage rate with a rule-based detector to determine how many logical information points are covered.Task Description. In this task, we treat the program requirements (e.g., substance treatment, counseling) for each client as the latent features Z and use them to augment the original feature X for outcome prediction, which is a multi-classification problem to learn Y ∼ f (X, Z) among four labels for the outcome Y ∈ {Completed, Revoked, N otCompleted, Other}. The raw dataset does not record the program requirements except for a very few clients; thus, the latent feature Z in this task is truly unobservable (in contrast to the one used in the first task). Available program requirement options for this task are attached to the appendix section D.Implementation Details. Steps 0 and 2-4 remain almost the same as in the risk-level prediction task.Step 1 requires a slight adjustment (as discussed in Section 4, this step is the main part in our framework that requires customization). Here, we formulate 40 baseline rationales in step 1 to deduce clients' program requirements from their features. We leverage multi-stage prompting strategy (Qiao et al., 2022) to break down the task into three sub-tasks: (1) identify the main challenges from the client's profile, (2) rank these challenges by priority, (3) match the challenges with suitable requirements. Particularly, the third task is our main goal, with the first two serving as steps to streamline the process and simplified the task.Evaluation. We train an ML classifier to predict outcomes with and without the inferred latent features, i.e., Ŷi ∼ f (X i , Ẑi ) versus Ŷi ∼ f (X i ). We evaluate the out-of-sample accuracy by comparing the predicted outcome Ŷi with the true label Y i in the test data. This comparison allows us to assess whether incorporating the latent features enhances the classifier's performance. 
results: 
 In this section, we demonstrate experiments results for two case studies we designed and additional results for sensitivity analyses.As mentioned in Section 5.1, we infer risk level on the client's profile. We compare our approach's performance to baseline ML model's performance using the accuracy score and F1 score. Before showing this performance comparison, we first show results on the generated text quality. Generated Text Quality. For profile writing in Step 0, we treat each individual feature in X i as a keyword to cover, and measure the keyword coverage rate. The generated profiles demonstrated an average keyword coverage rate of 98%, indicating that they effectively capture the most important information from the original data. For the generated synthetic rationales in Step 2, we treat terms such as age, gender, employment, and education as critical keywords and assess their coverage rate.The fine-tuned GPT-3.5 and Llama2-13b-chat both achieved a keyword coverage rate of 100%. This indicates that the generated content adheres strictly to the guidelines established in the training data, ensuring that all necessary information is accurately represented.Latent Variable Inference Performance. As shown in Figure 3 (a), our approach achieves the highest overall accuracy. In particular, the fine-tuned GPT-3.5 achieves an accuracy that is 20% higher than other baseline ML approaches. The reason that ML models struggle to predict well is due to the fact that there is no strong correlation between the observed features and the targets (risk level); see the correlation plot in Appendix F. In contrast, our approach demonstrates superior performance, since it more effectively handles datasets with subtle or non-obvious relationships between the observed and target variables. This result shows that our approach is able to make accurate inference of latent features and outperforms traditional ML approaches. Table 3(b) details the prediction performance by class, showing F1 scores for each class using ML models and our approach. Notably, all ML models struggle with the 'Very High Risk' category -this category is often misclassified as 'High Risk' due to similar feature distributions of these two categories and unbalanced data (only 371 training points for 'Very High Risk'). In contrast, our approach significantly improves the prediction performance for this category, highlighting its effectiveness for unbalanced datasets. This improvement is likely because our LLM-based approach has intermediate steps (profile writing to obtain the socio-economic status and other contextual factors in step 0 and connecting these factors with the latent variables in step 1), which help capturing the subtle distinctions between 'High Risk' and 'Very High Risk' that are not explicitly recorded.As mentioned in Section 5.2, we infer program requirements as additional latent features and use them for the downstream outcome prediction task. We compare the performance of the downstream classifiers that trained with and without the latent features. Note that in the first task (risk-level inferrence), GPT3.5 demonstrated better performance than llama2-13b. Thus, we focused on finetuning GPT-3.5 when using our approach for this task.As illustrated in Table 4 (a), incorporating latent features significantly improves the performance of the downstream classifiers. Specifically, the addition of latent features increases the ROC AUC score of Logistic Regression from 0.70 to 0.89 and from 0.84 to 0.92 for the Gradient Boosting Tree.Furthermore, the feature importance in Figure 4 (b) shows that the inferred features -'requirement_1', 'requirement_2', and 'requirement_3' -are among the top-ranked features. This implies the significant relevance of these features on the downstream classification task. Hence, we can conclude that our approach has the capability of enhancing the downstream classifier's accuracy with inferred latent features. In our sensitivity analysis, we further investigate the following three questions: (1) How sensitive is our approach to the quality of human guidelines? (2) How important is fine-tuning in our framework?For the first question, perhaps not surprisingly, our approach is sensitive to human guidelines, specifically the baseline rationales and prompt templates formulated in Step 1. We have conducted an ablation study to determine the optimal level of details required in the prompts. As shown in Figure 9 in Appendix D, the best performance was achieved with the most reasoning steps and a sentence length of two per step. In other words, increasing the number of reasoning steps allows us to decompose the task into simpler components and enhances the performance of LLMs. More importantly, while human guidelines are important, the interactive self-revise alignment strategy can significantly help during the sub-step of Step 1 (prompt crafting). By providing ground truth and encouraging self-reflection, GPT-4 can revise the prompt template to include crucial details, ensuring a more accurate evaluation.The answer to the second question is that fine-tuning is necessary. We have conducted another ablation study, where we repeated the risk-level prediction task with zero-shot, one-shot, and threeshot prompting to compare with our fine-tuned model. In zero-shot, we provided only the task description. In one-shot and three-shot, we included randomly selected human-verified examples.Accuracy rankings from lowest to highest were: three-shot (40%), zero-shot (55%), one-shot (60%), and the fine-tuned model (75%); see Table 9 in Appendix D. The three-shot's poor performance may be due to information loss from long inputs. Zero-shot responses are highly variable and not well-suited for downstream tasks. Although one-shot showed improvement, the fine-tuned model significantly outperformed all others.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We clarify all experiment setting in section 5. We also provide more training details on Appendix section C.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
discussion: 
 This study presents a framework that leverages the capabilities of LLMs to enhance the prediction accuracy in downstream tasks without necessitating invasive data collection methods. Our approach reduces the need for collecting extensive personal data, thus mitigating privacy concerns. This aligns with ethical data usage standards, especially in sensitive domains. Note that we do not explicitly address bias in the data or LLM reasoning processes in this paper. We excluded the 'race' feature in our case study and found alignment in risk level distributions across genders, implying no additional bias introduced by our approach. However, existing biases in LLMs could be perpetuated if not monitored and adjusted. Addressing these biases is beyond this paper's scope and is left for future research as a critical area. This framework has vast potential applications, particularly in areas with limited data and ethical constraints. For example, in healthcare, our framework can help predict readmission or post-discharge mortality by inferring unrecorded social determinants of health. For low-volume niche product recommendations, our framework can synthesize customer preference data to enhance recommendation systems without extensive user tracking. which completes the first part of the lemma.For the second part of the lemma, we first assume that for the given data D, L in ( D, β * ) = L in (D, β * ) -ϵ/n where ϵ ≥ 0 from the first part of the lemma. We now construct an instance with an out-of-sample dataset D ′ that contains n + 1 samples, where D ′ consists of (i) the n data points that exactly match with D (or D) for the first n samples, and (ii) one additional sample (x i+1 , y i+1 ) (or ((x i+1 , z i+1 ), y i+1 ) when using the augmented features). Without loss of generality, assume that y i+1 = 1. Then we haveL out (D ′ , β * ) = 1 n + 1 nL in (D, β * ) -log(p i+1 )andL out ( D′ , β * ) = 1 n + 1 nL in ( D, β * ) -log(p i+1 ) .When the added features Z's are non-informative, we consider the scenarios that they are noise and the additional term β * 2 Z also contributes noise to the predictions. In other words, the coefficients β *do not generalize well to the test data. Therefore, there exists an instance where the realization of Z, z i+1 deviates from the predicted probability significantly, such thatpi+1 < p i+1 / exp(ϵ) ≤ p i+1 .Note that this instance exists since the noise terms do not correspond to any actual pattern in the test data, causing incorrect predictions, and in our construction, a smaller predicted probability would be less accurate as the label y i+1 = 1. Therefore,EQUATIONL out ( D′ , β * ) = 1 n + 1 nL in (D, β * ) -ϵ -log(p i+1 ) > 1 n + 1 nL in (D, β * ) -log(p i+1 ) = L out (D ′ , β * ).For all experiments, we split data into training and testing dataset with ratio of 8:2.For experiment 1 (risk level prediction), we finetune LLaMA2-13b-chat on 2 X NVIDIA RTX A6000 for 4 hours with LoRA. And we finetuned three times for different subtasks. We use OpenAI offical API to finetune GPT3.5 model, which requires no GPUs. Each finetune job takes about 2 hours. We repeat 3 times for different sub tasks. Additionally, we also run Machine Learning baseline model on CPU (Intel i7). We run grid search for each classifier.For experiment 2 (outcome prediction), we use OpenAI offical API to finetune GPT3.5 model, which requires no GPUs. Each finetune job takes about 2 hours. We repeat 6 times for different sub tasks.Additionally, we also run Machine Learning baseline model on CPU (Intel i7). We run grid search for each classifier.All other experiments (e.g. sensitive experiment) are conducted on ChatGPT, which requires no GPU.1. Write sentences to cover all basic information provided. 2. Provide information about the area of this client live in, as much more details as you can.3. Infer social economic status of this client 4. Infer the challenges that this client might facing.Here are the basic information of the client: <features>.Here is the reference of living area context: <additional info>Here is the profile of a client: <profile> Given the client's information, please infer a risk score out of 10. Given the identified challenges for the client, infer the priority of each challenge in terms of immediate action and long-term impact on his reintegration into society. Please response in the ranking order. Here are the challenges: Here are the challenges <challenges>:Here is the available list of programs <program list>: Given the profile and challenges of the client, select the top 3 program requirements that would be most beneficial for the client.Here is the profile of client: <profile + top 3 ordered challenges> Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We change the random seed during the train/test splitting, and repeat the experiment 5 times with different seed. The standard deviation of results are within 0.02.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We have a brief introduction of experiments compute resources in section 5.We have more detailed information in Appendix section C.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the potential societal impacts in the section 1 Introduction, and section7 Discussion.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes] Justification: We promote human-in-the loop verification and emphasized on domain expertise. Moreover,we are not gonna make the dataset public -the framework is genralizable but we caution users to be aware of bias and use human-in the loop verification.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA] Justification: Our paper does not use existing assets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: The paper proposes a framework of using LLMs for reasoning in a chain-of-thought fashion. The framework is applied to a real world problem on criminal justice data. The following weaknesses were raised:
- The approach is tested only for a specific application (criminal justice). Although in the rebuttal, the authors came up with a second application, it seems that significant manual effort is required to generalize the approach to each new application.
- The use of LLM to generate synthetic data for such applications raises the problem of biases in the LLMs. This is especially important in the application that is considered in this paper. Although manual examples are provided to guide the LLM, there is insufficient discussion on how the LLM could be checked against biases.

Reviews: review: paper_summary: paper_summary: This paper used large language models to infer latent variables that are important for downstream prediction tasks to augment the existing models. In particular, the author demonstrated the use of the proposal on a criminal justice system use case, in which the LLM-mined-latent features significantly boost the prediction performance.

Overall the paper presents an interesting question and how LLM could help mining for latent features, but I have several questions regarding 1. the generalizability of the proposed method; 2. Appropriate combinations/baseline methods; and 3. The potential ethical implications of this method. I will detail these points in the strength/weakness sections below.
strength_weakness: strength_weakness: 1. Despite the general initial framing in Section 3, it was not very clear how generalizable results from Sections 4-6 are — this includes not only the COT and the prompts used, but more critically the selection of what kind of latent features we are including. 
2. The current work does not seem to go into depth about what kind of latent features are LLM particularly good at constructing and which ones are particularly “bad” (eg subject to the most bias and systematic over-or-under-prediction). I think this is particularly relevant for social science applications where many of the categories and features are more of a “construct” and often qualitative in nature.
3. Have the authors compared the results by using text embeddings of the descriptions as an input feature? It’s interesting that the fine-tuning strategy is necessary for good performance, which seems to suggest that learning the intermediate classification rule is important since direct manipulation of natural language yields less impressive  results.
questions: questions: 1. In addition to the questions listed in the weakness section, I would encourage the authors to engage much more critically with the limitations of these approaches, esp. when high-risk predictions in CJS are very high-stake. 
2. I think this paper would also be much stronger if the proposed framework of mining for latent features (which requires quite extensive human rationale collection) works for another prediction problem (either different data and/or different tasks).
limitations: limitations: Mentioned above.
paper_summary: paper_summary: The paper presents a framework that uses LLMs to improve predictive modeling by augmenting observed features with inferred latent features. This approach transforms the latent feature mining task into a text-to-text propositional reasoning task, enabling LLMs to infer unobserved yet crucial factors from available data. The framework is tested through a case study in the criminal justice system, demonstrating improved accuracy in scenarios where collected features are weakly correlated with outcomes.
strength_weakness: strength_weakness: 1. The paper does not adequately address how to measure the impact of errors introduced by the LLM-based solution on predicted outcomes, nor does it provide uncertainty estimates. This process surely introduces errors, as with any ML-based solution. How do we measure its effect on predicted outcomes, including uncertainty estimates? I suspect more labeled data would be needed to assess this properly (see Egami et al. @ NeurIPS 2023).

2. It is unclear whether the approach should be viewed as a form of dimensionality reduction (based on existing features) or if it extrapolates information not present in the original data. This ambiguity raises concerns about potential bias amplification. For instance, Figure 1 shows deductions made by the model that are not clearly supported by the evidence, suggesting that the method might be amplifying existing biases rather than mitigating them.

3. Connecting to the previous point, the method's ability to learn latent information that is causally predictive of the outcome, as opposed to relying on spurious correlations, remains uncertain. Conducting an out-of-distribution test, where the characteristics of individuals differ from the training data, would be crucial in evaluating the model's generalizability and causal inference capabilities. 

4. The rationale behind not using all available data directly in the LLM for prediction is not well-justified. Directly prompting the LLM with the full data might provide more accurate predictions without the need for dimensionality reduction.
questions: questions: 1. How do you measure the impact of errors introduced by the LLM-based solution on predicted outcomes, including uncertainty estimates? 

2. Should the proposed method be viewed as a form of dimensionality reduction, or does it extrapolate information not present in the original data? How do you ensure that this process does not amplify existing biases? 

3. How do you ensure that the method learns latent information that is causally predictive of the outcome and does not simply rely on spurious correlations?
limitations: limitations: The authors have acknowledged limitations of their work, particularly in addressing the ethical concerns associated with data collection and the need for privacy.
paper_summary: paper_summary: The authors propose a unique form of LLM data-augmentation that attempts to generate informative latent variables to improve downstream tasks. They do this by transforming the latent feature mining task into a text-to-text propositional reasoning task.
Validation is performed with a case study in the criminal justice system and latent features align well with ground truth labels + significantly enhance downstream classifier performance.
strength_weakness: strength_weakness: - Type on line 203: "whic serve"
- Potential for LLM biases in the latent variable finding. E.g. Marijuana usage does not necessarily require Substance Abuse Treatment.
- Lack of evaluation on multiple datasets/domains and no publically released code
- Lack of ablations exploring generalizability with x\% features removed
questions: questions: See weakness

- How does this compare with other forms of data augmentation?
- How does this compare with the traditional methods of obtaining latent space representations -> VAE, etc?
limitations: limitations: Yes
paper_summary: paper_summary: This paper proposes a framework to augment latent features from observed features, with the help of LLM. They frame the problem as a text-to-text reasoning problem.  The method can be adapted to different domains easily.  The method is also validated with a real world dataset.
strength_weakness: strength_weakness: The key concern for me is that when using the LLM for inference and text generation, I worry about the social bias and fairness of the problem. Some research has shown that LLM is still biased in some sense, can the author conduct some evaluation on whether the latent feature is biased towards some sensitive attributes like race, gender, etc?

The other thing is that I wonder how much human labor effort and expert labor effort will be needed to have the latent features. 

Typo in line 203
questions: questions: In line 166, how do you determine the number l ?
limitations: limitations: N/A

Comments: comment: Thank you for the thoughtful rebuttal.  I will be maintaining my current score.
thank you for thoughtfully addressing my questions, and especially for adding an experiment.

i still think that a more careful design w.r.t to causality and potential errors would make this paper substantially more useful.
**Thank you for taking the time to review our work and for replying our responses.** We appreciate the opportunity to clarify and expand on our findings. Here are our response to your concerns **point-by-point**:

---

***Concern on generalizability:***

> "It is quite difficult to validate the quality of the model even with 
> the MIMIC dataset. Even if human-in-the-loop evaluations are used on
> the generated examples, there is no guarantee of the trend holding for
> different datasets.”

As discussed in Sections 1 and 4 of our paper, our framework is designed to **enhances Machine Learning model by externalizing expert knowledge to generate new features**. Our experiments with the MIMIC dataset demonstrate that, with sufficient expert input, **our framework generates strong features that significantly enhance predictive accuracy**. The success observed in the two tested domains indicates that our framework is **highly flexible and can be effectively transferred to other domains**.It is important to note that adapting it to new areas requires careful integration of expert knowledge as a crucial first step to ensure that the framework maintains its robustness and accuracy across different applications

  ---

***Concern on the robustness of CoT :***

> "From personal experience, 'logical reasoning leads to correct 
> predictions, and incorrect reasoning does not produce correct results
> seems to slightly contradict the results found in. COT
> explanations can be plausible yet misleading.“

**Thank you for highlighting this concern.** We carefully reviewed the paper you cited. Their experiments are conducted in **zero-shot and few-shot settings**. We agree that CoT can be plausible generate misleading or biased output in few shot setting. It’s important to note that our framework trains LLMs to better align with human knowledge. Fine-tuning, as an important component of our approach, significantly increase the quality of generated CoT, and enhances the reasoning capabilities of LLMs. As indicated in Section 6 (line 370) of our paper, our ablation study on the **fine-tuning process shows that fine-tuned LLMs substantially outperform those in zero-shot and few-shot scenarios**.

Additionally, **CoT has been validated as an effective technique across various domains**. For instance, the CoT strategy has been shown to significantly improve LLMs' performance in document understanding and citation generation[1]. It also helps VLMs mimic multi-hop reasoning in answering SCIENCEQA questions [2]. Additionally, LLMs fine-tuned with CoT exhibit marked improvements in reasoning ability across different datasets [3].

 ---
 
 ***Concern on the robustness of Hallucination :***
 
> "Any LLM that extrapolates information not explicitly present in the
> original data could also suffer from hallucinations, which is an open
> problem.”

**We address hallucinations through fine-tuning and post-generation validation, aiming to filter out such inaccuracies**. Moreover, our error analysis of the generated reasoning text revealed that the fine-tuned LLMs consistently used accurate information from the profile without fabricating details  (For more details, please refer to our response to Reviewer U9i4). **As you mentioned,  hallucination remains an open problem in the field, it falls beyond the immediate scope of our work**. We will leave this issue as a direction for future research.

Additionally, figure 3 in the paper presents the fine-tuned LLMs' inference results on risk level prediction task, where **our framework was applied to infer features with known ground truth**. The results show that LLMs is able to high-quality feature generation even when extrapolating to unseen information, and outperform traditional Machine Learning approaches in accurately inferring latent features. This demonstrates the accuracy and effectiveness of our framework, giving us confidence in **its ability to generate high-quality features**.

---

**Thanks for engaging the discussion period. We are looking forward any further feedback. If this response address your concerns, we kindly request a reconsideration of our merits. Thank you again for your valuable input!**  

---

***References:***

[1] Ji, B., Liu, H., Du, M., & Ng, S.-K. (2024). Chain-of-Thought Improves Text Generation with Citations in Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16), 18345-18353. https://doi.org/10.1609/aaai.v38i16.29794

[2] Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K. W., Zhu, S. C., ... & Kalyan, A. (2022). Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. Advances in Neural Information Processing Systems, 35, 2507-2521.

[3] Ho, N., Schmid, L., & Yun, S. Y. (2023, July). Large Language Models Are Reasoning Teachers. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 14852-14882.
The additional evaluations are more encouraging, especially the ablations with Removing Features Mentioned in the Provided Guidelines as well as the Removing Features Not Included in the Provided Guidelines. However, I still think it is quite difficult to validate the quality of the model even with the MIMIC dataset, as the interpretability results are still quite difficult to evaluate in its current state. Even if human-in-the-loop evaluations are used on the generated examples, there is no guarantee of the trend holding for different datasets, especially those out of distribution (e.g. LLM's reasoning on highly specific domains such as chemistry remains quite nonsensical). 

I am not entirely convinced by the following responses to Reviewer U9i4's first or second points. From personal experience, "logical reasoning leads to correct predictions, and incorrect reasoning does not produce correct results" seems to slightly contradict the results found in [1], where COT explanations can be plausible yet misleading. Also, any LLM that "extrapolates information not explicitly present in the original data" could also suffer from hallucinations, which is an open problem. 

I'd like to thank the authors for putting in the time to answer my responses, but I still think it's difficult for me personally to accept the paper.

[1] Turpin, M., Michael, J., Perez, E., & Bowman, S. (2024). Language models don't always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36.
**Thank you for your  comments and valuable suggestions, here are our response to your suggestions:**

---

- **Suggestion1:  Add ablations exploring generalizability with x% features removed.**
- **Response:** Thank you for your valuable suggestion regarding the inclusion of ablation studies. We recognize this as an important aspect of framework evaluation and have thus conducted additional experiments to gauge the robustness and dependence of our model on specific features. We systematically removed different proportions of features and reran experiments on the risk level prediction task.
	- **Experiment 1: Removing Features Mentioned in the Provided Guidelines**
  As discussed in the paper, we use domain knowledge and expert input to provide guiding principles for the LLMs' inference process. For this experiment, we removed two features explicitly mentioned in the guiding principles—age and marriage status—and observed the following impacts on model performance:

    | Model | ROC_AUC score |
    |:------------------:|:-------------:|
    | MLP | 50% |
    | GBT | 43% |
    | GPT3.5 Baseline | 56% |
    | GPT3.5 Full-feature | 75% |

  The results demonstrate that excluding features specified in the guiding principles (e.g., GPT3.5 Baseline) reduces the final accuracy but still outperforms traditional machine learning models such as MLP and Gradient Boosting Trees (GBT).
  
	- **Experiment 2: Removing Features Not Included in the Provided Guidelines**
  In this second experiment, we removed the feature “referral source”, which was not explicitly recommended to be used in the guiding principles:

  | Model | ROC_AUC score |
  |:------------------:|:-------------:|
  | MLP | 54% |
  | GBT | 58% |
  | GPT3.5 Baseline | 71% |
  | GPT3.5 Full-feature | 75% |
  
These results indicate that the exclusion of features not included in the guiding principles (e.g., GPT3.5 Baseline) only slightly reduces final accuracy, yet continues to outperform the machine learning baselines (MLP and GBT).

  ---
  
 - **Suggestion 2:**  Release code for more comprehensive evaluation
- **Response:** We planed to release the code after the review. Following your suggestion, we are providing access to the code for the MIMIC experiments as a preview. Following the rebuttal instruction, we have submitted an anonymous link to the code to the AC. This link will become available for preview upon approval by the AC.

----

**Thank you again for your valuable input! We are looking forward to further discussion.**
**Thank you for  thoughtful comment! Following your suggestion, we plan to consider adding following discussions into the main paper.**

---

- **Discussion 1: What are limitations of the framework when high-risk predictions in CJS are very high-stakes?**
-   **Response:** -   Thank you for this insightful question. Below we list more detailed discussion on the limitations of the framework for high-risk predictions in the criminal justice system (CJS). We will incorporate this discussion into the main paper during our revision process.
    
	- **Data Quality and Dependency:** The performance and reliability of the framework depend on the quality of both the training data and the input data used during operation. Inconsistencies, errors, or gaps in data can lead to inaccurate predictions, which is critical when these predictions aim to influence decisions about individuals’ freedoms in the CJS. We are cautious of such errors and this is why we closely involve human experts during the whole research process. While acknowledging that bias sometimes is inherent in the input data, we emphasize that our framework provides the transparency of how the inference/prediction is made, which can significantly help interpret results and correct for such bias by introducing human-in-the-loop. In other words, our framework allows an iterative fine-tuning process, where we can incorporate feedback loops with domain experts to adjust and correct the model’s reasoning pathways. This iterative method ensures that the latent features inferred, such as the need for substance abuse treatment, are aligned with nuanced real-world outcomes rather than broad statistical correlations.

	- **Impact on Public Trust:** The use of AI in areas impacting fundamental rights can affect public trust in the justice system. If the public perceives these tools as opaque or biased, it could undermine confidence in judicial processes, which is critical for the effective functioning of any democratic legal system. LLMs provide the interpretability that is useful for gaining public trust and allows an iterative process to refine the prediction results.

---

-   **Discussion 2: What kind of latent features are your framework particularly good at constructing ?**
-   **Response:** Thank you for such an insightful question. Our framework excels at constructing latent features from observable and structured data with clear relationships, such as socio-economic status indicators and risk levels. These features benefit from well-defined guidelines and domain knowledge, allowing our framework to infer them with higher accuracy and reliability by mimicking the human reasoning process.

	The framework can be less effective with subjective latent features or those with subtle relationships, like personal behavioral tendencies. These complexities can introduce bias and lead to overfitting and systematic errors. To mitigate these issues, we use domain expertise and a human-in-the-loop approach to refine latent feature construction. Despite these measures, some biases may persist, and addressing them remains a focus for future research. We will continue to explore improvements.
----

**Thank you again for your valuable input! We are looking forward to further discussion.**