# A Study of a Gain Based Approach for Query Aspects in Recall Oriented Tasks
## 5 Conclusions and Future Work  
In this paper, we have presented a study that evaluates different definitions of the GAIN of a reformulation for an information need. We adapted the definition of gain proposed by [PERSON] et al. (2018) to the context of Consumer Health Search, and we used a standard test collection to evaluate our hypotheses: can we use the gain metric to predict the performance of each reformulation? Is there a better formulation that can produce an order of the importance of each reformulation in terms of recall? We found that for recall based tasks where the number of documents to retrieve may be large, \(N>100\), the original definition of GAIN saturates quickly to 1. We proposed an alternative definition that mitigates this problem, and we also presented a similarity based approach that tries to capture the 'optimal' query reformulation among all the available formulations of an information need. The analysis of the results confirms that our approach significantly improves the prediction of the order of importance of each reformulation in terms of recall. In conclusion, the proposed technique is meant to help practitioners in tackling the systematic reviewing task. In this sense, our technique is not meant for a general-purpose query suggestion strategy (more in line with the model proposed by [PERSON] et al. (2018)). The described approach is meant to boost queries with higher recall, in a context where the practitioner is, in a sense, forced to explore a large number of documents. Assuming that a practitioner needs to review _all_ the documents about a specific topic, it is vital to reduce as possible time spent reviewing documents. Therefore, our technique can help in determining, among a series of queries for the very same information need, which one is more likely to return the most relevant documents. Being able to explore first more promising queries, can greatly speed up the systematic reviewing process. Among our future work, we plan to further investigate the robustness and generalizability of the approach. In particular, we plan to include new collections, topics and formulations. We plan \begin{table}
\begin{tabular}{l l l l l l} \hline \hline  & & \multicolumn{4}{c}{**[PERSON]’s \(\tau\)**} \\ \hline
**Predictor** & **Cutoff** & **Q1** & **Mean (Std)** & **Q3** & **Sign.** \\ \hline \multirow{4}{*}{Original Gain (Kendall, 2018)} & 100 & 0.0000 & 0.3422 (0.4696) & 0.6831 & 15 \\  & 1000 & 0.0000 & 0.3783 (0.3527) & 0.6609 & 13 \\  & 10,000 & 0.0000 & 0.1600 (0.2606) & 0.4765 & 2 \\ \cline{2-6}  & 100 & 0.1456 & 0.3822 (0.4936) & 0.7320 & 16 \\ \cline{2-6}  & 1000 & 0.2417 & 0.4042 (0.4796) & 0.7320 & 17 \\  & 10,000 & 0.2709 & 0.5111 (0.3984) & 0.8000 & 19 \\ \cline{2-6}  & 100 & 0.1429 & 0.3768 (0.4636) & 0.6581 & 13 \\ \cline{2-6}  & 1000 & 0.3083 & 0.5069 (0.3505) & 0.7143 & 17 \\ \cline{2-6}  & 10,000 & 0.2521 & 0.5443 (0.3930) & 0.8876 & 24 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: [PERSON]’s \(\tau\) correlation observed for the task of predicting the query formulation recall, using the similarity based approaches. In bolt, best mean score for each cutoff. Note that all the methods considered perform better than traditional predictors(cfr Table 3). We also have a higher number of significant rankings compared to the one observed before. to investigate the news domain through multiple formulations available in the UQV100 collection [1]. We are currently investigating the possibility to smooth the contribution of each reformulation in the similarity matrix \(S\) with a _locality_ parameter \(w\). This parameter can be used as an exponent for each element of \(S\) and decide whether to get reformulations closer, \(w<1\), or push them far away, \(w>1\), to create sub-clusters of reformulations and obtain a better prediction. Conceptualization, G.M.D.N. and G.F.; methodology, G.M.D.N. and G.F.; software, G.M.D.N. and G.F.; validation, G.M.D.N. and G.F.; formal analysis, G.M.D.N. and G.F.; investigation, G.M.D.N. and G.F.; resources, G.M.D.N.