# Predicting Online Item-Choice Behavior: A Shape-Restricted Regression Approach
## 6 Experiments
### Prediction Performance of Our PV Sequence Model  
Figure 3 shows F1 scores of the two-dimensional probability table and our PV sequence model using the sampled training sets, where the number of selected items is \(N\in\{3,5,10\}\), and the setting of the PV sequence is \((n,m)\in\{(7,3),(5,6)\}\).  
When the full-sample training set was used, SeqUM and SeqUS always delivered a better prediction performance than the other methods did. When the 1%- and 10%-sampled training sets were used, the prediction performance of SeqUS decreased slightly, whereas  
Figure 3: Comparison of prediction performance with the two-dimensional probability table.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & & & \multicolumn{2}{c}{**\#Cons in Equation (8)**} & \multicolumn{2}{c}{**Time [s]**} & \multicolumn{2}{c}{**F1 Score [\%], \(N=3\)**} \\ \cline{3-10} \(n\) & \(m\) & _\#Vars_ & **SeqUM** & **SeqUS** & **SeqUM** & **SeqUS** & **SeqEmp** & **SeqUM** & **SeqUS** \\ \hline
3 & 30 & 29,791 & 84,630 & 118,850 & 86.72 & 241.46 & 12.25 & 12.40 & 12.40 \\
4 & 12 & 28,561 & 99,372 & 142,800 & 198.82 & 539.76 & 12.68 & 12.93 & 12.95 \\
5 & 6 & 16,807 & 63,798 & 85,722 & 86.02 & 286.30 & 12.90 & 13.18 & 13.18 \\
6 & 4 & 15,625 & 62,500 & 76,506 & 62.92 & 209.67 & 13.14 & 13.49 & 13.48 \\
7 & 3 & 16,384 & 67,584 & 76,818 & 96.08 & 254.31 & 13.23 & 13.52 & 13.53 \\
8 & 2 & 6561 & 24,786 & 25,879 & 19.35 & 17.22 & 13.11 & 13.37 & 13.35 \\
9 & 2 & 19,683 & 83,106 & 86,386 & 244.15 & 256.42 & 13.07 & 13.40 & 13.37 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Computational performance of our PV sequence model (7)–(9).  
SeqUM still performed best among all the methods. When the 0.1%-sampled training set was used, 2 dimMono always performed better than SeqUS did, and 2 dimMono also had the best prediction performance in the case of \((n,m)=(5,6)\). These results suggest that our PV sequence model performs very well, especially when the sample size is sufficiently large. The prediction performance of SeqEmp deteriorated rapidly as the sampling rate decreased, and this performance was always much worse than that of 2 dimEmp. Meanwhile, SeqUM and SeqUS maintained high prediction performance even when the 0.1%-sampled training set was used. This suggests that the monotonicity constraint (8) in our PV sequence model is more effective than the monotonicity constraints (3) and (4) in the two-dimensional monotonicity model.  
Figure 4 shows F1 scores for the machine learning methods (LR, ANN, and RF) and our PV sequence model (SeqUM) using the full-sample training set, where the number of selected items is \(N\in\{3,5,10\}\), and the PV sequence setting is \((n,m)\in\{(7,3),(5,6)\}\). Note that in this figure, SeqUM\((\,*\,)\) represents the optimization model (7)-(9), where the item-choice probabilities computed by each machine learning method were substituted into \((\hat{\pi}_{v})_{v\in\Gamma}\) (see Section 4.4).  
Prediction performance was better for SeqUM than for all the machine learning methods, except in the case of Figure 4f, where LR showed better prediction performance.  
Figure 4: Comparison of prediction performance with machine learning methods.  
Moreover, SeqUM(\(\ast\) ) improved the prediction performance of the machine leaning methods, particularly for ANN and RF. This suggests that our monotonicity constraint (8) is also very helpful in correcting prediction values from other machine learning methods.