Venue: NeurIPS 2022 Workshop SVRHM

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper discusses various techniques to factorize a 3D tensor of first
impressions, where the different modalities are face stimuli, traits, and
participants.

The objective of the paper is interesting, but I have some methodical concerns.

Mainly, the authors don't seem to take into account the number of parameters
that each of the tested methods has. Wouldn't it be sensible to include some
form of Occam's razor here, and not purely rely on cross validation to ensure
the models aren't overfitting?

In addition, the selection of models that are compared is somewhat biased
towards attention. Model number 2 (direct concatenation) doesn't perform that
badly, so perhaps giving the final MLP a few more layers would have been 
sufficient to reach a similar performance?

Overall, I would also liked to have seen whether there are any qualitative conclusions that could be drawn from the resulting factorizations. I.e. what do we learn from this experiment about first impressions?

Other than this, some notes regarding clarity:

- It wasn't immediately clear to me what the loss function for training the
  "deep" tensor factorization methods is, and that the authors used cross
  validation with a held-out dataset to arrive at the results. This could be
  presented earlier on in the paper.

- In Figure 1, "Option 2" does not connect to the prediction score. This seems
  misleading, given that from the rest of the paper, it seems that that method
  still relies on the final MLP for making a prediction.

- It is misleading to talk about an "optimal" model for representing the
  dataset. Optimal means that there provably can't be a better model. I don't think that
  claim can be substantiated here.

review_title: Interesting, but some methodical concerns

final_score: rating: 3: Clear rejection
review_content: This papers presents a set of machine learning models to predict people's first impressions of others, merging images of faces, linguistic information from a language model and demographics, via deep tensor factorisation.

My overall impression of this submission is that there are multiple aspects in which the paper could be improved, hence my rating below acceptance level.

* Motivation: While I understand the goal of the paper, I miss a solid motivation of why it is important to develop machine models of people's first impressions of others and what is the research gap that this paper is aiming to fill. 
* Ethics: Related to motivation, I would also expect a discussion about the ethical implications of this work, given that it machine models of people's first impressions could potentially acquire biases harmful to some groups or individuals, as well as being misused.
* Experimental setup: The paper studies the performance of five models as well as a baseline. First, I missed a discussion of the motivation for the choice of methods. Why is deep tensor factorisation a method of interest for this problem? Why are simpler methods not studied? Second, I would argue that the baseline used (linear regression) is significantly weaker than the deep tensor factorisation methods. In between linear regression and deep tensor factorisation there exists a multitude of methods that could be more appropriate baselines to judge the performance of the proposed models.
* Conclusions: In my opinion, the paper falls short of providing an in-depth analysis and conclusion from the work presented in the paper. The main conclusion by the authors seems to be that "deep tensor factorization methods outperformed the baseline regression and vanilla tensor factorization". What are the implications of this conclusion and how does it advance our knowledge about what? I believe the discussion remains at a quite superficial level. Later, one paragraph reads that the paper shows that "deep tensor factorization provides the key to creating computational models of biased first impressions", but I disagree that the paper shows such a strong statement. Nevertheless, the discussion is also shallow in this regard.
* Writing: The writing of the paper could also be polished. For instance the article uses informal expressions ("and so on"), contains non rigorous statements ("various machine learning models (e.g., CNN and Eigenfaces [3, 14]) or deep learning algorithms"; "Under a learning rate..."), and some paragraphs that are not easy to read, such as the last paragraph of Section 4. As a more minor comment, I would not use a "." after "Figure" or "Table" since the whole word is used.
review_title: Limited motivation, experimental setup and conclusions, among other aspects

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The paper was fairly clear to follow. Please see the couple of suggestions below. Overall this work would be very interesting for the audience of SVRHM. Multi-modality is intrinsic to the real world. This work presents some fascinating comparisons of various ways to combine this plethora of information towards people first impressions. What would be super cool as future work on this direction is also combining audio data as yet another source of features. Aside from visual information and language used, I could see a lot of bias towards first impressions coming from audio input (aka how people sound). 

nit: Please consider changing Model 1, 2,...5 to Option 1, 2,..., 5 in the text or vice versa (change Option 1, 2,..., 5 to Model 1, 2,..., 5 in the figure) to match the two. It took a bit of starring to make the right connections between text and figure.

"Vanilla tensor factorization was selected to conduct the experiments, for Table. 1": Vanilla tensor factorization is mentioned as Model 1 from Models 1-5 the paper compares. Table 1 shows results from all 5 models including the linear regression baseline. Could you please clarify what this sentence means?
review_title: Very interesting work combining various modalities towards predicting first impressions

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This is an interesting paper using deep tensor factorization models to understand human judgements of first impressions. The authors compare different tensor factorization techniques to combine 1) visual information about faces, 2) trait (semantic) information, and 3) participant demographic information.

While I liked the paper and approach, I was confused about some of the motivation and several technical details.

It wasn't clear to me why the authors decided to include linguistic/semantic information about traits. Face trait judgements are made within a few milliseconds. Do the authors think humans engage similar semantic reasoning on this timescale? (In fact, this seems to have very low prediction in experiment 2.)

I understand space was tight but the methods text was somewhat confusing. The term ‘3-way side feature vectors’ is introduced without explanation in the methods. I believe that refers to the 3 modalities described above, but that should be clarified, and it would help to define ‘side features’ for those unfamiliar. It would also help the reader to add Model #’s to the appendix.

A smaller point is that the names of the 3 modalities should be consistent across the paper. I think 'visual' and 'semantic' would be more clear than 'face' and 'trait' which could both be visual or semantic.

The text just says the baseline model is a linear regression. A regression between what? All 3 types of features and impressions? This seems like an unfair comparison given the number of computational layers in models 1-5. 

A more interesting question would be to separate out the 3 types of information to understand the contribution of each. I think this is what Experiment 2 aimed to do, but I did not understand these. Didn't all models predict participant trait judgements? How were the different entities held out and then predicted?
review_title: Novel approach on an important topic, but motivation and technical details could be clarified

Decision: Accept(Poster)