# Predicting Missing Values in Survey Data Using Prompt Engineering for Addressing Item Non-Response
## 3 Methods
### Our Methods
#### 3.3.1 Row Selection  
The Row Selection process is designed to strategically select the most relevant few-shot examples that closely resemble the current respondent, thereby enhancing the model's performance in predicting missing data. The selection is based on the cosine similarity between the response vector of the target respondent \(\mathbf{r}_{t_{i}}\) and the vectors of other respondents\(\mathbf{r}_{k}\) in the dataset \(D\). The detailed procedure for Row Selection, illustrated in Figure 2, is as follows:  
1. [label=0.,leftmargin=*]
2. Cosine Similarity Calculation: the cosine similarity \(\text{sim}(\mathbf{r}_{t_{r}},\mathbf{r}_{k})\) is calculated as follows: \[\text{sim}(\mathbf{r}_{t_{r}},\mathbf{r}_{k})=\frac{\sum_{j\in Q^{\prime}}r_{ t_{r}j}\cdot\tau_{kj}}{\sqrt{\sum_{j\in Q^{\prime}}r_{t_{r}j}^{2}}\cdot\sqrt{ \sum_{j\in Q^{\prime}}r_{kj}^{2}}},\] (3) where \(Q^{\prime}\subseteq Q\) is the set of columns where both the \(\mathbf{r}_{t_{r}}\) vector and the \(\mathbf{r}_{k}\) vector have non-NaN values. By using Equation (3) as a scoring function, this ensures that the similarity is computed based on the commonly answered questions, effectively addressing the issue of missing data.
3. Ranking Based on Similarity: After calculating the cosine similarity scores for all respondents \(\mathbf{r}_{k}\), the respondents are ranked in descending order of their similarity scores \(\text{sim}(\mathbf{r}_{t_{r}},\mathbf{r}_{k})\). This ranking allows us to identify the respondents most similar to the target respondents.
4. Option-Wise Selection: From the top-ranked respondents, \(p\) respondents are selected for each answer category of the target question \(q_{t_{q}}\). This step ensures that the few-shot examples include a balanced representation of all possible answer options for the target question.
5. Balanced Few-Shot Example Selection: Finally, the selected respondents from each answer category are combined to form a balanced few-shot dataset with \(n=p\times q\) examples, where \(q\) is the number of answer categories (options) for the target question \(q_{t_{q}}\). This balanced dataset is then used to provide context for imputing the missing responses of the target respondent.  
Through this process, we can select respondents who are most similar to the target respondent as few-shot examples. Additionally, by selecting \(p\) respondents from each answer category of the target question, we ensure that the few-shot examples are not biased and provide a balanced representation of all possible answers.  
Figure 2: A detailed illustration of the Row Selection. The Row Selection process involves four key steps to select the most relevant few-shot examples for predicting missing survey responses: (1) Calculate Cosine Similarity: measure the similarity between the target respondent and others based on their responses. (2) Rank Similarity: rank all respondents by their similarity scores to the target respondent. (3) Divide by Option: group the top-ranked respondents according to the different answer options for the target question. (4) Choose Balanced Options: select a balanced number of respondents from each answer option to form a few-shot dataset, thereby enhancing diversity and reducing bias in the predictions. The Row Selection process is crucial for constructing a few-shot dataset that is both diverse and unbiased, resulting in enhanced predictive performance of the model.