# Improving User Experience with Recommender Systems by
## 3 Methodology
### Experimental Design  
A multi-method experiment was conducted employing a counterbalanced mixed (between-within) subjects design involving five (5) factors (i.e., counterbalanced 2 \(\times\) 2 \(\times\) 2 \(\times\) 2 \(\times\) 2 for a total of 32 conditions), tested using three (3) stimuli per condition (i.e., 96 stimuli). The factors involved (i) information sequence (problem-to-solution vs. solution-to-problem); information specificity, comprising (ii) problem specificity (vague vs. specific) and (iii) solution specificity (vague vs. specific); (iv) text styling (plain vs. bold); and (v) situation complexity (low vs. high). For the latter, complexity was varied by manipulating the product type involved in the situation--soft drinks vs. meat patties--given the following 'complicating' considerations: cost (low-cost soft drinks vs. high-cost meat patties), durability (non-perishable soft drinks vs. perishable meat patties), handling (soft drink bottles require no special handling vs. meat patties require special--refrigerated--handling). Hence, the situations with soft drinks were overall of lower complexity than those pertaining to meat patties. Additionally, we manipulated the product's availability (available vs. unavailable with available substitute product). The factor manipulations are illustrated in the sample recommendation messages in Table 1. Moreover, in order to reduce the time needed to complete a session, the study was divided into 8 groups, each comprising 4 of the 32 conditions (corresponding to 12 stimuli per participant), which required approximately 15 min to complete.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Factor** & **Respective Levels: Vague/Vague/Plain/Solution-to-Problem/High Complexity** & **Respective Levels: Specific/Specific/Bold/Problem-to-Solution/Low Complexity** \\ \hline \multirow{3}{*}{Problem Specificity} & I noticed that you are running low on soft drinks. I & I noticed that you only have 10 bottles of Coca-Cola in \\  & recommend ordering 20 6-packs of Coca-Cola bottles & \\  & today. Shall I proceed with the order? & \\ \hline \multirow{3}{*}{Solution Specificity} & I noticed that you ordered 10 6-packs of Coca-Cola & \multirow{3}{*}{I noticed that you ordered 10 6-packs of Coca-Cola bottles, but the product is no longer available from your \\  & supplier. I recommend substituting the missing product. & \\  & Shall I proceed with the substitution? & \\ \hline \multirow{3}{*}{Text Styling} & I recommend ordering 20 6-packs of Coca-Cola bottles & \multirow{3}{*}{I recommend ordering 20 6-packs of Coca-Cola bottles today because I noticed that you only have 10 bottles of **Coca-Cola left** in stock. Shall I proceed with the order?} \\  & I recommend ordering 100 ground beef patties today because I noticed that you only have 10 ground beef patties left in stock. Shall I proceed with the order? \\ \hline \multirow{3}{*}{Information Sequence} & I noticed that you only have 10 ground beef patties left & \multirow{3}{*}{I noticed that you only have 10 ground beef patties stock. I recommend ordering 100 ground beef patties today. Shall I proceed with the order?} \\  & I noticed that you ordered 100 ground beef patties, but the product is no longer available from your supplier. I recommend substituting ground beef patties with ground \\ \hline \multirow{3}{*}{Situation Complexity} & I noticed that you only have 10 bottles of Coca-Cola in \\  & the product is no longer available from your supplier. I recommend substituting ground beef patties with ground \\ \cline{1-1}  & real patties. Shall I proceed with the recommendation? & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental manipulation of independent variables.