RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Figure 2: The overall architecture of the proposed Parameter-efficient Conversation Recommendation System (PECRS), where the PEFT denotes the parameter-efficient fine-tuning. Instead of fine-tuning backbone model, we inject PEFT plugins into backbone model and fine-tune the PEFT weights (see the figure in the right).; The overall architecture of the proposed Parameter-efficient Conversation Recommendation System (PECRS), where the PEFT denotes the parameter-efficient fine-tuning. Instead of fine-tuning backbone model, we inject PEFT plugins into backbone model and fine-tune the PEFT weights (see the figure in the right).
Figure 3: The R@50 results of PECRS-small using the different M_{\text{train}} and M_{\text{inference}} pairs on ReDial dataset.; The R@50 results of PECRS-small using the different M_{\text{train}} and M_{\text{inference}} pairs on ReDial dataset.
Figure 4: R@50 of PECRS on ReDial per number of conversation turns prior to the CRS response.; R@50 of PECRS on ReDial per number of conversation turns prior to the CRS response.
Figure 5: An example of dialogue from INSPIRED Hayati et al. ( 2020 ) , where blue color denotes the movie items.; An example of dialogue from INSPIRED Hayati et al. ( 2020 ) , where blue color denotes the movie items.
Statistics on ReDial and INSPIRED datasets, combined over train, dev and test sets.
Results of the recommendation task compared with the state-of-the-art on ReDial and INSPIRED. Results are taken from respective papers. Best numbers are in bold , second best underlined .
Results of conversation task compared with the state-of-the-art on ReDial.
Human evaluation on 100 random ReDial test data points. We show the average scores for three human raters, with standard deviation in parenthesis.
Models comparison with different modules and optimization strategies on ReDial with PECRS-small.
Effect of pruning fields of items metadata at inference on INSPIRED with PECRS-small.
Comparison between PECRS-small and two popular LLMs in zero-shot on INSPIRED test set.
The conversation performance of PECRS-small with different decoding strategies on ReDial. Except Greedy decoding , all other techniques use a beam width of 4.
Accuracy w.r.t genre prediction on ReDial test set broken down by movie genre.
Accuracy w.r.t genre prediction on INSPIRED test set broken down by movie genre.