# Recommendations for Different Tasks Based on the Uniform Multimodal Joint Representation
## 5 Experiments and Results
### Pin Recommendation  
We invited 10 users to engage in the evaluation of pin recommendation. Each user was given 200 randomly selected target images and corresponding recommendation results of different methods. They were required to decide whether to pin some images of three candidates, but not if they were the owner of the target pin. Table 2 shows the precision of recommendations. A simple content-based filtering, which randomly selects an image with the same category as the target image, was implemented as a reference. All other methods achieved higher accuracies than the category-based method, simply because they utilized more information to reduce the affect of related categories. Object-based and interest-based methods used the probability layer from the original AlexNet and multilabel AlexNet, respectively. The results of those two methods were comparable, while interest distributions were more compact than object distributions. This indicates that even coarse-grained interests of an image were a little more important than what this image was on CCSNs. The other methods computed cosine similarity between representations. We note that using only dominant categories as the label to fine-tune AlexNet led to a decline, which may have been caused by confusion of similar images with different categories. Notice that the performance of multimodal  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Dimension** & **Dominant Category Accuracy** & **Mean Nonzero Error** & **Mean Error** \\ \hline Multiclass & 4096 & 45.85\% & — & — \\ Multilabel & 4096 & 82.71\% & 0.1320 & 0.0141 \\ Word2 Vec & 300 & 42.88\% & 0.3249 & 0.0415 \\ MAE & 1778 & 83.02\% & 0.1307 & 0.0136 \\ ICMAE & 1898 & 83.96\% & 0.1223 & 0.0128 \\ _Ours_ & **2048** & **84.13**\% & **0.1181** & **0.0119** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of Pin Category Prediction(MAE: Multimodal Autoencoder).  
features was worse than that of image features. We believe that the descriptions could not completely describe all the interests and characteristics that images have. Our text representations were clearly not as effective as our image representations, therefore, image representations were more suitable for image recommendation.  
Figure 7 illustrates 10 images and their recommendation results. Obviously, intrinsic characteristics such as background, scene, pattern, texture, color, object, material, and so forth are maintained in the image representation and usually had an effect on the recommendation, especially for images in the left panel. Images in the right panel show that some abstract notions, for example, style and user interest, influenced the results. All these high-level image features learned from CNNs could significantly improve the accuracy and diversity of recommendations.  
From the recommendation results, we can clearly see that our model recommended similar styles and types of images. This means that our model could achieve a good recommendation effect in terms of content-based recommendations. This further illustrates that the features extracted by our multimodal joint representation model were effective. The recommendation data were different from the training data, which proves that our model had a good generalization ability.