Interpretable (meta)factorization of clinical questionnaires to identify general dimensions of psychopathology
Authors: Ka Chun Lam
Bridget W Mahony
Armin Raznahan
Francisco Pereira
Venue: Submitted to ICLR 2023 (2023)
Keywords: Factor analysis
matrix factorization
meta-factors
latent constructs
Healthy Brain Network Study

Abstract: Psychiatry research aims at understanding manifestations of psychopathology in behavior, in terms of a small number of latent constructs. These are usually inferred from questionnaire data using factor analysis. The resulting factors and relationship to the original questions are not necessarily interpretable. Furthermore, this approach does not provide a way to separate the effect of confounds from those of constructs, and requires explicit imputation for missing data. Finally, there is no clear way to integrate multiple sets of constructs estimated from different questionnaires. An important question is whether there is a universal, compact set of constructs that would span all the psychopathology issues listed across those questionnaires.  We propose a new matrix factorization method designed for questionnaires aimed at promoting interpretability, through bound and sparsity constraints. We provide an optimization procedure with theoretical convergence guarantees, and validate automated methods to detect latent dimensionality on synthetic data. We first demonstrate the method on a commonly used general-purpose questionnaire. We then show it can be used to extract a broad set of 15 psychopathology factors spanning 21 questionnaires from the Healthy Brain Network study. We show that our method preserves diagnostic information against competing methods, even as it imposes more constraints. Finally, we demonstrate that it can be used for defining a short, general questionnaire that allows recovery of those 15 meta-factors, using data more efficiently than other methods.

Under review as a conference paper at ICLR 2023
INTERPRETABLE (META)EACTORRIZAIOON OF CLINI- CAL QUESTIONNAIRES TO IDENTIFY GENERAL DIMNN- SIONS OF PSYCHOPATHOLOGY
Anonymous authors Paper under double-bling brind review
ABSTRACT
Psychiatry research aims at understanding manifestations of psychopathology behavior, in terms of small number of latent constructs. These are usually in- ferred from questionnaire data using factor analysis. The resulting factors and relationship the original questions more, this approach does not provide a way to separate the effect of confounds from those of constructs, and requires explicit imputation for missing data. Fi- nally, there is no clear way to integrate multiple sets of constructs estimated from different questionnaires. An important question is whether there universal, compact set of constructs that would span all the psychopathology issues listed across those questionnaires. We propose new matrix factorization method de- signed for questionnaires aimed at promoting interpretability, through bound and sparsity constraints. We provide an optimization procedure with theoretical con- vergence guarantees, and validate automated methods to detect latent dimension- ality on synthetic data. We first demonstrate the method on a commonly used general-purpose questionnaire. We then show it can be used to extract broad set 15 psychopathology factors spanning 21 questionnaires from the Healthy Brain Network study. We show that method preserves diagnostic information against competing methods, even as it imposes more constraints. Finally, we demonstrate that can be used for defining short, general questionnaire that allows recovery of those 15 meta-factors, using data more efficiently than other methods.
not necessarily interpretable. Further-
INTRODUCTION
Standardized questionnaires are common tool in psychiatric practice and research, for purposes ranging from screening to diagnosis or quantification of severity. A typical questionnaire comprises questions usually referred to items reflecting the degree to which particular symptoms or behavioural issues are present in study participants. Items are chosen as evidence for the presence of latent constructs giving rise to the psychiatric problems observed. For many common disorders, there is practical consensus on constructs. If so, a questionnaire may be organized so that subsets of the items can be added up yield subscale score quantifying the presence of their respective construct. Otherwise, the goal may be to discover constructs through factor analysis.
The factor analysis of a questionnaire matrix partiiiisssants x #items) expresses it as the product of a factor matrix (#participants x #factors) and a loading matrix (#factors #items). The method assumes that answers items may be correlated, and can therefore be explained in terms of a smaller number of factors. The method yields two real-valued matrices, with uncorrelated columns the factor matrix. The number of factors needs to specified priori, or estimated from data. This solution is often subjected to rotation so that, after transformation, each factor has non-zero loadings on few variables, and each variable has a high-loading on single factor, if possible. The values of the factors for each participant can then be viewed as succinct representation of them.
Interpreting what construct factor may represent is done by considering its loadings over all the items. Ideally, if very few items have non-zero loading, it will be easy to associate the factor with them. However, in practice, the loadings could an arbitrary linear combination of items, with positive and negative weights. Factors real-valued, and neither their magnitude nor their sign

---

Under review as a conference paper at ICLR 2023
intrinsically meaningful. Beyond this, any missing data will have to be imputed, or the respective items ommitted, before factor analysis can be used. Finally, patterns in answers that are driven by other characteristics of participants (e.g. age or sex) are absorbed into factors themselves, acting as confounders, instead of being represented separately or controlled for.
Over time, many different questionnaires have been developed. Some focus on constructs relevant to particular disorders or behavioral issues; others aim at screening for a wide range of problems. One important question for psychiatry researchers is how many constructs would suffice to explain most manifestations of psychopathology. In addition to its scientific interest, an answer to this question would also be of clinical use, informing design of light-weight questionnaires designed estimate all key constructs from a minimal number of items. The availabiiity of datasets such as Healthy Brain Network (Alexander et al. 2017), where tens of questionnaires are collected for thousands of children and adolescent participants, makes it possible to address this question in data-driven way. However, a joint factor analysis of many questionnaires faces additional obstacles, e.g. their having different response scales, very disparate numbers of items, or patterns of missing entries.
In this paper, we propose to address all the issues above with a novel matrix factorization method specifically designed for use with questionnaire data, through the following contributions.
Contribution #1: We introduce (ICQF), a new matrix factorization method for questionnaire data. Our method was designed to incorporate characteristics that increased interpretability of the resulting factors, based on several desiderata from active clinical researchers in psychiatry. First, factor values constrained to be in the range [0, 1]. So as to represent a degree of presence of the factor. Second, the loadings across items for each factor have be in the same range answers in the original questionnaire (lypically, [0, m. This makes it possible to examine them as a pattern of answers associated with the factor. Third, the reconstructed matrix obtained by multiplying factors by factor loadings is constrained, so that no entry exceeds the range- or observed maximum value- - of the original questionnaire. Fourth, the method handles missing data directly, so no imputation required. Finally, the method supports pre-specifying some factors to model known variables, such as age sex, to capture the answer patterns correlated with them (e.g. drinking problems appearing as age increases). We demonstrate ICQF in the Childhood Behavior Checklist (CBCL), widely used questionnaire, and show that it preserves all diagnostic information in various questionnaires, even with additional regularization.
cerppretabllty-Conttrainnn
Questionnaire Factorization
Contribution #2: We provide theoretical guarantees on the convergence and performance of the optimization procedure. We introduce an optimization procedure for ICQF] using alternating minimization with ADMM. We demonstrate that this procedure converges to local minimum of the optimization problem. We implement blockwise-cross weeedwss--coos--viidaatee -ross-validation (BCV) to determine the number of factors. If this number of factors is close to that underlying the data, the solution will be close to global minimum. Finally, we show that our procedure detects the number of factors more precisely than competing methods, as evaluated in synthetic data with different noise density.
Contribution #3: We use two-level meta-factorization of 21 questionnaires to identify 15 gen- eral factors of psychopathology in children and adolescents. We apply [CQF individually to 21 Healthy Brain Network questionnaires (first-level), and then again to a concatenation of the resulting 21 factor matrices (sccond-level), yielding meta- veta-factorization with 15 interpretable meta-factors. We show that these meta- fectora can outperform individual questionnaires in diagnostic prediction. We also show that the meta-factorizae can be used to produce a short, general questionnaire, with little loss of diagnostic information, using data much more efficiently than competing methods.
2 RELATED WORK
The extraction of latent variables (a.k.a. factors) from matrix data is often done through low rank matrix factorizations, such as singular value decomposition (SVD), principal component analysis (PCA) and exploratory Factor Analysis (hereafter, just FA) (Golub Van Loan 2013. Bishop &
Nasrabadi 2006). While SVD and PCA aim reconstructing the data, I'A aims at explaining cor-
relations between (questions) items through latent factors (Bandalos & Boehh- Kautmmn 2010).
Factor rotation (Browne) 2001; Sass & Schmitt 2010; Schmitt & Sass| (011) is then performed to obtain a sparser solution which is easier to interpret and analyze. For a comprehensive review of FA, see Thompson (2004); Gaskin & Happell (2014); Gorsuch] (2014); Goretzko et al. (021)
2

---

Under review as a conference paper at ICLR 2023
Non-negative matrix factorization (NMF) was proposed as way of identifying sparser, more in- terpretable latent variables, which can be added to reconstruct the data matrix. It was introduced
in Paatero & Tappee (1994) and was further developed in Lee & Scung (2000). Different varieties
of NMF-based models have been proposed for various applications, such as the sparsity-controlled
(Eggert & Korner 2004 Qian et al.] 201T), manifold-regularizzd (Lu et al. 2012), orthogonal Ding
ctt (2006):]Chh))) 22008). convex/semi-convex (Ding et al., 2008), or archetypal regularized NMF
(Javadi & Montanari 2020). Recently, the Deep-NMF (Trigeorgis al. 2016 Zhao et al. 2017)
and Deep-MF (Xue et 2017 Fan & Cheng 2018 Arora et al. 2019) have been introduced that can model non-linearities on top of (non-negative) factors, when the sample is large (Fan 2021)
These methods do not directly model either the interpretability characteristics or the constraints that we view as desirable. If the goal is to identify latent variables relevant for multiple matrices, the standard approach is multi-view learning (Sun et al., 2019), or variants that can handle only partial overlap in participants across matrices (Ding et al. 2014; Gunasckar et al., 2015 Gaynanova &
Li 2019). Finally, non-negative matrix tn-laacorrizttion (NMTF) (Li et al. 2009; Pei et al. 2015),
supports an additional matrix mapping between latent representations for different matrices.
Obtaining factorization with these methods requires both specifying the number of latent variables, and solving an optimization problem. In SVD/PCA, the number of variables is often selected based on the percentage of variance explained, determined via techniques such spectral analysis, the
Laplace-PCA method, or Velicer's s MAP test [Velicer 1976 Velicer et al. 2000 Minka (000)) For FA, several methods have been proposed: Bartlett's 's test (Bartlett 1950 parallel analysis (Horn
1965 Hayton et al. 2004),, MAP test and comparison data (Ruscio & Roche 2012). For NMF,
iterative detection algorithms are recommended, c.g. the Bayesian information enteriin (BIC) (Sto-
ica & Selen 2004), cophenetic correlation coefficient (CCC) (Fogel et al. (007) and the dispersion (BiCV) (Owen & Perry] (209) and its generalization. the blookwise-roos--alidatii (BCV) (Kanagal & Sindhwani] 2010 which we use in this paper. The optimization problem for NMF non-convex, and different
(Brunet et al.j [004). More recent proposals for NMF are El-cross-validatinn
algorithms for solving it have been proposed. Multiplicative update (MU) (Lee & Seung, 2000) is
the simplest and mostly used, Projected gradient algorithms such as the block coordinate descent (Cichocki & Phan| 2009 Xu & Yin 2013 Kim et al.) 2014) and the alternating optimization (Kim & Park] 2008) Mairal et al. (2110) aim at scalability and efficiency in larger matrices. Given that our optimization problem has various constraints, we use combination of alternative optimization and
Alternating Direction Method of Multipliers (ADMM) (Boyd et al. 2011 Huang et al. (0166).
3 METHODS
3.1 INTERPRETABLE CONSTRAINED QUESTIONNAIRE FACTORIZATION (ICQF)
Inputs Our method operates on a questionnaire data matrix M E m with n participants 20
and m questions, where entry () is the answer given by participant i to question j. Given that questionnaires often have missing data, we also have a mask matrix M E {0. 1 of the same di- mensionality as M, indicating whether each entry is available (=1) or not (=0). Optionally, we may have a confounder matrix C E R20 encoding e known variables for each participant that could account for correlations across questions (e.g. age or sex). If the jth confound Cl.,) is categorical, we convert it to indicator columns for each value. it is continuous, we first rescale it into [o, 1] (range in the dataset), and replace it with two new columns, Cj.Jj and 1 - Ch.s This mirroring procedure ensures that both directions of the confounding variables are under consideration (e.g. answer patterns more common the younger or the older the participants are).
Optimization problem We seek to factorize the questionnaire matrix M as the product of a n x factor matrix W E [0, 1], with the confound matrix C E [0, 1] as optional additional columns, and a m x (k + c) loading matrix Q == [&Q with a loading pattern RQ over m questions for each of the k factors (and CQ for optional confounds). Denoting the Hadamard product as O our optimization problem minimizes the squared error of this factorization
minimize WEW.QEQ.Zer ZEZ
1/2 o (M - -) + B- R(W.Q, Q)
" = Z, -= [2 Min < Zij S ma)) (ICQF)
such that
º= == 0 < Qτ) and W = (W) 0 < Wy 1
3

---

Under review as a conference paper at ICLR 2023
subject to entries of Q being in the same value range question answers, so loadings inter- pretable, and bounding the reconstruction by the range of values the questionnaire matrix M. We further regularize W and Q through R(W Q) == [ 1 + > =/llQlp.. Y
I max (M). where |))//////))// Here, we use p = q = for sparsity control. y is
p..
|| = 1
heuristic to balance the sparsity control between W and Q. With a slight abuse of notation, y is absorbed into 3 of Q if no ambiguity results.
Choice of number of factors For each B, we choose the number of factors using blockwise- cross-validation (BCV). Given matrix M, for each k, we shuffle the rows and columns of M and subdivide it into b, x be blocks. These blocks are split into 10 folds and we repeatedly omit blocks in fold, factorize the remainder, impute the omitted blocks via matrix completion and compute the error" of that imputation. We choose with the lowest average error. This procedure can adapt to the distribution of confounds C' by stratified splitting. We compared this with other approaches for choosing k for ICQF and other methods, over synthetic data, and report the results in Appendix F
3.2 SOLVING THE OPTIMIZATION PROBLEM
The ICQF problem is non-convex and requires satisfying multiple con- straints. We solve it through an ADMM optimization procedure. The Lagrangian Lp is:
Optimization procedure
(W Z.zz -1/2| @ (M- Z) + Iww() + W||,,,, (Q( + QlIi,
(1)
<oz Z [W. [WC/)=²) + p/2 z [W, WW Iz (Z)
where p the penalty parameter, az is the vector of Lagrangian multipliers and (x = if X X' and otherwise. We alternatingly update primal variables W, Q and the auxiliary variable Z by solving the following sub-problems:
(++1) arg min p/211Z [W, (.Q +p +p- 2 W 11,1
Wew
((+1)= arg min (/2 Q
[W++1), CIQ" x a a +311 11Q11111
( arg min || O (M 2) pl (1+11) c (++1+ + --- Zez
(4)
for some penalty parameter p. Lastly, az is updated via
z - az + p (++1) ( ((Q((t))))
(i+1)
(5)
Equations 2 and 3 can be further split into row-wise constrained Lasso problems and there is a closed form solution for equation 4 The optimization details are further discussed in Appendix [ Given the flexibility of ADMM, similar procedure can also be used with other regularizations.
Convergence of the optimization procedure In Appendix we provide a proof that the con-
straint p 2 /2 the penalty parameter / guarantees monotonicity of the optimization procedure, and that it will converge to local minimum. Integrating this constraint with the adaptive selection of p (Xu et al., 2017). we obtain an efficient optimization for ICQF Furthermore, Bjorek et al.) (20TT) showed that, if k = k= of ground-truth solution (W*, "Q") in non-negative matrix factorizations, the error JM- WQ I is star-convex towards (W", Q"). and the solution is close to global minimum. In Appendix
C we show that, if k / k*, the relative error between W* and W increases with Wk/k"* 1|. Inaccurate estimation of k* thus affects both the interpretability of (W, Q) and the convergence to global minima. As reported in Appendix (F] BCV is more robust to noise when estimating k than other alternatives, and this is why we use it.
3.3
META-PACTORRIZATION
ICQF
produces interpretable factors for individual questionnaires. As discussed earlier, our second goal is to obtain interpretable factors that explain psychopathology across a range of questionnaires. can also be used to obtain these meta-factors, through two leve--factorizzatoo:: factorize each
ICQF
Appropriate weighting is multiplied to the error number of blocks in the last fold less than others.
4

---

Under review as a conference paper at ICLR 2023
individual questionnaire, concatenate their respective factor matrices, and then factorize this matrix. The main obstacle is that each participant may only have answered a subset of the questionnaires available. This is the second reason for including mask matrix M in our problem formulation.
In describing our factoracctiinitiion procedure, we suppress C' and the regularization terms R to simplify the discussion. Let {M, j= to be the data matrices of S questionnaires with dimensions m,)) Note that {M. j= can be fully, partially non-overlapped with each other. For di- mension consistency, we extend M, (so as Wi, M,)) into n rows by padding rows of zeros for miss- ing participants, where 2 n, Vi = I..... S denotes the number of unique participants from the S questionnaires. We then introduce mask matrices & E (0. 1) ] == D, . MM, which is composed of the extended M, and diagonal mask matrix D; {0, j indicating the availability of partic- ipants. Performing matrix factorization for each questionnaire, we obtain & @ M, R &, @ (W,Q,)
for = -,...,,.. S. We then concatenate {W;), and perform a second level factorization:
[Di W..W..... Ds Ws] ~ [Di II,..., Ds 1s] @ WQ",
(6)
where 1, is a I-matrix of dimension x K2 with K denoting the number of factors in W,, for i S. The columns of W are the meta-factors.
There are alternative approaches for factorizing multiple questionnaires. The most obvious would be factorize the concatenation of all { as [&1 M Eg o Ms] WQ" It requires a wider detection range for the best l (f1,.... Σ. m²)). This is used for competing methods in our experiments. Moreover, any low-rank matrix completion algorithms could be used for meta- factorization. However, constraining W and Z from each questionnaire is crucial; as discussed in Maisog et al. (202T), and witnessed by in practice, simple normalization before estimating k for meta-facter W may induce unpredictable effects. We could also optimize meta-objective function: 1/4
[ a E& @ (M; - (Q )1()) This has a smaller range of practice, but extra hyper-parameters og (relative importance of data matrix M,) are introduced. Finally, could use tri-factorization: [&] @ MI,..., @ Ms] x WGQ". This did not work well in our case.
4 DATA
The Healthy Brain Network (HBN) (Alexander et al. 2017 is an ongoing project to create a biobank from New York City area children and adolescents. Data are publicly available, and include psychi- atric, behavioral, cognitive, multimodal brain imaging, and genetics. In this work, we use a subset of 21 psychiatric questionnaires about behavioral and emotional problems. They were selected by domain experts by their focus on psychopathology. frequency of use in clinical research, and com- pleteness in the HBN dataset. This subset contains peneral-purpose questionnaires covering different domains of psychopathology (e.g. CBCL, SDQ and SympChck) and others focusing on specific dis- orders (c.g. ASSQ for autism screening, SWAN for ADHD, and SCARED for anxiety). The full list of 21 questionnaires is reported in Table 4 in Appendix G. Across all questionnaires, we have 978 questions and 3572 unique participants. Finally, we have the age and sex at birth of each participant, which will be used as confounds, and diagnostic labels for 11 conditions, if applicable.
5 EXPERIMENTS AND RESULTS
5.1 ICQF FACTORIZATION OF THE CHILD BEHAVIOR CHECKLIST
We begin with a qualitative assessment of ICQF applied to the 2001 Child Behavior Checklist (CBCL), which is designed to detect behavioral issues. The checklist includes 113 questions, grouped into 8 syndrome subscales: Aggressive, Anxiety/Depressed, Attention, Rule Break, Social, Somatic, Thought, Withdrawn problems. Answers are scored on a three-point Likert scale (0=absent, ==occurs sometimes, 2=occurs often) and the time frame for the responses is the past 6 months.
We estimated the latent dimensionality k- = 8 using BCV to compute test error for ICQF at each possible l. The regularization parameter B = 0.5 was set the same way (See bottom- leatom--eft-pee panel of Figure D. The top-panel shows the heat map of the latent factors W, and the loadings CQ for the confounds C. Questions are grouped by syndrome subscale. While there were factors that loaded primarily in questions from one subscale, expected,
Q CQ), the loadings over questions "Q for
5

---

Under review as a conference paper at ICLR 2023
Aggressive
AnxDep
Attention
Other
RuleBreak
Social
Somatic
Thought
WithDep
F:3
F-4
F8
Yuung
Famaic
Factol
(CBCL,56)5 55, Temoer Conouu ar terr per-
Agg essive
Regularization parameter 8
3300
0.01
0.05
0.2
(CBCL Argues M--
Agg rosive
3200
Validation
[CSCL 60) Screara a is-
Agg vosive
(CRC) (6] Sussbar, caller inteare
1og worke
3000
(CBC) 271 Eadyy jesluss
Socli
(050 Unusualy
10
Dimension k
0.2
0.4 0.6
loadings
Figure 1: Top: Heat map of factor and confound loadings Q := ==" QQ]. Note that questions
grouped by syndrome subscale; some factors are syndrome specific, while others bridge syndromes. Bottom left: Detection of (ke, B) by BCV. Bottom right: Top 10 questions Q (t..) loading mostly on the Aggressive subscale. Top 10 questions for all factors are shown in Appendix
we were surprised to find others that grouped questions from multiple subscales. These were deemed sensible co-occurrences by our clinical collaborators. We show the top questions, ranked by magnitude of loading, for the first factor Q1aa1 from "Q, interpret the factor siittom-right-panne of Figure D]. As a further, sanity check, we inspected the loadings of confound Old (increasing age) and observe that they covered issues such "Argues", "Act Young", "Swears" and "Alcohol". The loadings of Q also reveal the relative importance among questions in each estimated factor; subscales deem all questions equally important.
demonstration of how one might
5.2 META-ICQF FOR META-FAACTORIZATIIO OF THE HBN QUESTIONNAIRES
This section provides a qualitative evaluation of meta meta ICQF, analogous to that of ICQF in Sec- tion 5.1 As described earlier, the meta-factorization requires first-level ICQF of each question- naire in HBN, yielding factor matrix W. and loading matrix Q, On the second level factorization, concatenate { and use ICQF to get meta-factors W and respective loadings Q over first level factors as described in equation 6 (note the change in font for these). Figure 2 (left) shows the lower triangular part of the correlation matrix of Q The first level factors from all question- naires are grouped through agglomerative clustering as many clusters as meta-factors. k on their meta-factor loadings. The sparse. block-diagonal pattern and the diversified factor-origins within each block demonstrate how meta-factorization can combine related latent factors from multiple questionnaires. Figure 2 (bottom--rghh) shows the trend of validation errors with different (k: 8)) using the BCV detection scheme. The optimal inflection point is & = 15 and / = 0.1. Finally, we can paok-propegate Q from factor- to question-leve by multiplying diag [Qi..... .l.....Qs.. Qs] Q = Q. These loadings retrieve the question's 's latent representation in the meta-factor space; the magnitude of each question's 's entry in each column of Q reveals influence of the corresponding meta-factor: Figure 2 (top-right) shows the top 10 questions of the first column of Q ranked by their magnitude,
For multi-questionnaire setting, we abuse the notation Q to denote the latent representation of questions, either by direct factorization, or the factofaactoizattion followed back-propag

---

Under review as a conference paper at ICLR 2023
meta-ICQF Factor
34. Has trouble paying attention, and affects school work or performance
SympChck_34 34C
difficulty remaining
SympChck_366
Restless, overactive, cannot stay still for long
$DQ_02
easily
SympChck_
Constantly fidgeting squiriing
5D0_10
can't pay long
CBCL_08
78. Inattentive easily distracted
CBCL_78
37. Often acts before thinking
SympChck_370
Easily distracted, concentration wanders
(Impairmen)) 10. Sits still (controls movement of hands or feet or controls squirming)
SWAN_10
0.0
100
Regularization parameter
0.0
0.02
0.1
Validation error
480
0.01
0.05
420
MF-1
MF-22
MF-3
Mff-4
MF
MF-66
MP-1
MF-
MF-9
MF 10
MF-11
MP-12
MF.
MP-14
MF-15
400
25
Dimension k
Correlation matrix of Q, showing clusters of similar ICQF factors for all question- naires on the left (format questionnaire- #), and the fecta-factos 15) those clusters correspond to at the bottom. Bottom-right: Detection of (k, B) by BCV. Top-right: The top 10 questions as- sociated with meta-factor their loading value, and their questionnaire of origin. This meta-factor reflects attention issues. A similar plot for every meta-factor is reported in Appendix
Figure 2: Left:
with their questionnaire of origin. This meta meta-factor reflects attention issues. Similar plots for all 15 meta-factors are reported in Appendix There is strong topic coherence among the top ranked ques- tions in each meta-factor. The meta-facton have been deemed interpretable and clinically plausible presentations by psychiatry collaborators.
5.3 DIAGNOSTIC CLASSIFICATION
Given the absence of ground-truth factorizations for participants in the HBN study. it is challeng- ing to carry out a quantitative evaluation of ICF versus other factorization methods, or subscales. In this section, we report on two different experiments based on predicting diagnostic labels for each participant, from factor scores. The first tests whether factor matrices W preserve the nec- essary information for this, when applied to general-purposed questionnaires or combination of every HBN questionnaire. The second tests whether the question ranking induced by Q, across all questionnaires in HBN, selects the most informative questions for each factor.
5.3.1 EXPERIMENTAL SETUP
Baseline methods Our first baseline method is 4 re-ullaarized NMF (6 NMFF) (Cichocki & Phan 2009, as it also imposes negatiiiiyy and sparsity constraints. As constructs (or questions) can be correlated, we rule out other NMF methods with orthogonality constraints. FA with promax rotation (FA-promax) [Hendrickson cluded because it is commonly used in analyzing questionnaires. Syndrome subscales included available for a questionnaire, since they are often used for diagnoses. Finally, we include raw questionnaire answers, they have all the information available. To estimate the number of factors k. we use BCV for é NMF and ICQF] and parallel analysis for FA. The choice was driven by the experiments synthetic questionnaire data reported in Appendix
White 1964) using minimum residual as estimation method in-

---

Under review as a conference paper at ICLR 2023
Table 1: Averaged ROC-AUC scores of the diagnostic prediction under different factorizations.
Factorization
Questionnaire
meta meta-ICQF
ICQF
4NNMF
FA-promax
subscales
raw
CBCL SDQ
0.768
0.762
0.763
0.740
0.763
0.762
0.755
0.757
0.752
0.739
SympChck
0.753
0.761
0.756
NaN
0.747
HBN
0.792
0.782
0.777
0.778
0.766
0.788
Questionnaires The two experiments are motivated by the routine use of general-purpose ques- tionnaires in our dataset CBCL, SDQ (Symptoms and Difficulties), and SympChck (Symptom check) screen and refer patients to pediatric psychiatry clinics, for variety of diagnoses (Heflinger et al. 2000 Biedeerman et al. 2005 2020). The referral is based either on raw answers on the questionnaire or syndrome-speciffi subscales derived from them. Beyond this, and given that have 21 questionnaires from HBN, carried out experiments
W derived from them. The factors are obtained using the meta- factoorattiization described in Section 333 (meta-ICCF). or by con- catenating the questionnaires and factorizing the result (&, NMF, FA-promax), simply using the concatenation (raw), possibly aggregated (by subscales, if defined, or all added otherwise).
Dataset splits We use similar evaluation procedure in both experiments. We group the 21 HBN questionnaires, and split participants into train, validation, and test sets with ratio 70/15/15 based on participant availability across questionnaires and the distribution of confounds and diagnostic labels. This ensures a similar data distribution in the three sets, as shown in Figure in Appendix E. where more details are provided. We resample 50 dataset splits using different seeds, and carry out both experiments in each split. The results reported are the average across results in all splits.
Model training and inference Let W.cd denote the participant factor matrix in ICQF or NMF, or the factor score in FA. The subscript is the questionnaire index, and is dropped if considering only one. The superseript denotes the set. Similarly, let Q, denote the question loadings for each method. Model training will yield a (Wtrainn Q) for participants in the training set. Inference with the model will produce Wealidate and Waeatt in validation and test sets, using the trained Q and confounds Cvalidate Ceest (if applicable). See Figure 4 in Appendix D for diagram. A sans serif font (e.g. W) indicates a second-level factorization result. In meta- factoriactii,,,,nn, both model training and inference are performed on individual questionnaires to obtain (Wtrain W. Waaaiddtee Witast Q:) at the first level. At the second level, we concatenate (Wfrain Ji=1 and do model training to get meta-factor Wirain and Q, followed by model inference on the concatenated (Wiraaiidtee fi=1 and W Ji=1- This then gives Woulidate and Wiest, While factoriatti is possible for NMF or FA, we not use it, results were same or worse than factorizing concatenated questionnaires.
5.3.2 DIAGNOSTIC PREDICTION FROM FACTORS
For each one of diagnostic labels, we train a logistic regression model with & regularization, and balanced class weights, on Witrain (Wtrein). ). The regularization strength is tuned using Wealsdate (W)alidate). ). Prediction assessment conducted on Wiesst (Wisst) using the ROC-AUC metric (Krzanowski & Hand 2009). The ROC-AUC of each setting is then averaged across all random dataset splits. As results were obtained over the same datasets, for 11 different classification prob- lems, we use Friedman test with significance level a 0.05 (followed by a posthoc Nemenyi test if the null hypothesis of the Friedman test is rejected), following Demšar (2006).
Table D shows the summary of AUCs obtained on the various questionnaires using different fac- torizations, averaged across all 11 diagnostic labels. HBN corresponds to the use of all 21 ques- tionnaires, as described earlier. Results for each problem are provided in Table [ in Appendix J In CBCL, the null hypothesis is rejected and the post-hoc Nemenyi test indicates that subscales is significantly worse than all other factorizations. The null not rejected for SDQ or SympChck. In the IIN setting, the null hypothesis is rejected and the Nemenyi test indicates significant differ- ence between the group (meta-ICQ) raw) and the group (f1-N -MF, subscales). Overall, we conclude that ICQF and meta ICQF preserve diagnostic information, in spite of additional regularization and constraints versus other methods. -omaan-defined subscales are slightly but significantly worse. A parallel experiment on another CBCI. dataset from different population is reported in Appendix K
8

---

Under review as a conference paper at ICLR 2023
Depression
0.75
AUC
11-NMF-15
0.05
1001
0.50
FA promex
0.55
melafactor
100
200
500
Number of questions
Figure 3: ROC-AUC trend of prediction performance for Depression using the l-questionnaire, for increasing The methods are [CQF (estimated i: 22), meta ICQF (estimated k 15), 4MMM and Frommxx (with k = 15 or estimated),, and metafactor (meta ICQF with all questions in HBN)
5.3.3 DIAGNOSTIC PREDICTION FROM SYNTHESIZED QUESTIONNAIRES
Our goal is to assess the degree to which the process of extracting factors from all of the HBN questionnaires yields chopathology. If so, we should be able to parlay those loadings into general-purpose questionnaires defined in purely data-driven way, using the most informative question subset. We operational- ize this as follows. We first perform factorization in the training set, as described for the previous prediction problem. For each column Qi:a) rank questions according the absolute magnitude of their loadings. By grouping top & questions from each of the k (meta-) factors, we can derive new questionnaire, which we call the t-questionnaiie. The t-questionnatre inherits the ranking of questions and ideally, preserves the key information for diagnostic prediction.
where loadings identify informative questions for every domain of psy-
The experimental setup parallels Section 5.3.2 but using either the 21 components of the t- questionnaire, for ICQF or their concatenation, otherwise. For the latter we trained NNMMF and FA- FA-promax, either selecting the number of factors from data as before, or setting meta-ICQF (k= 15). We also trained ICQF directly on the concatenation (estimated k 22). We then trained a & regularized logistic regression, with regularization parameter set over validation data, and evaluated its performance on the test set. This procedure was carried out for / up 40 (or #questions if < t). Figure shows the trend of prediction performance for Depression for increasing The red dotted line is the average performance without eliminating any questions (same HBN in Table5). The trends for the other 10 diagnostic predictions are reported in Appendix [ They are broadly similar in relative terms across the methods (except for suspected ASD). Across the range of t, meta-ICQF and ICQF had substantially higher AUCs than other methods, especially when t was small. This suggests that meta- meQa-ICCQ] effecttve at determining the relative importance of 1000 questions from 21 questionnaires, as well as grouping them into interpretable meta-factors.
that of
DISCUSSION
In this paper, we have introduced ICQF a method for non-negative matrix factorization with ad- ditional constraints further enhance the interpretability of factors, and the capability directly handle confounds. We have demonstrated ICQF a widely used questionnaire, and showed that in- terpretability does affect our ability to make diagnostic predictions from factors. We also showed that ICQF can be used for two-level meta-factoree ceeta-fcctoizztionns of sets of questionnaires. This allowed us to identify 15 meta-factors of psychopathology that coherently group questions from many ques- tionnaires, and correspond to clinical presentations of patients. Furthermore, we showed that the resulting meta-factorization induces ranking of the most informative questions for each question- naire. This makes it possible to generate minimal general-purposs questionnaire for estimating the 15 meta-fiat neta-ffacoors while maintaining a specified level of diagnostic prediction performance. In the fu- ture, we plan to the 15 meta-fate meta-factors brain imaging data available in HBN. We also plan on releasing the [CQFI code for community use, and carrying clinical validations of the l-questionnaires generated from the meta-factorizatton.
latent variables for studying the structural and functional

---

Under review as a conference paper at ICLR 2023
REFERENCES
Lindsay M Alexander, Jasmine Escalera, Lei Ai, Charissa Andreotti, Karina Febre, Alexander Man- gone, Natan Vega-Potler, Nicolas Langer, Alexis Alexander, Meagan Kovaes, et al. An open resource for transdiagnostic research in pediatric mental health and learning disorders. Scientific data, 4(1):1-26, 2017.
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo. Implicit regularization in deep matrix factorization. Advances in Neural Information Processing Systems, 32, 2019.
Deborah I. Bandalos and Meggen R Boehm-Kaufman Four common misconceptions in exploratory factor analysis. In Statistical and methodological myths and urban legends, pp. 81-108. Rout-
ledge, 2010.
Maurice S Bartlett. Tests of significance in factor analysis. British journal of psychology, 1950.
Amir Beck and Mare Teboulle. fast iterative problems. SIAM journal on imaging sciences, 2(1):183-200 2009.
algorithm for linear inverse
J Biederman, MC Monuteaux, Kendrick, KL Klein, and SV Faraone. The cbel screen for psychiatric comorbidity in paediatric patients with adhd. Archives of Disease in Childhood, 90 (10):1010-1015, 2005.
Joseph Biederman, Maura DiSalvo, Carrie Vaudreuil, Janet Wozniak, Mai Uchida, K Yvonne Wood- worth, Allison Green, and Stephen V Faraone. Can the child behavior checklist (chel) help charac- terize the types psychopathologic conditions driving child psychiatry referrals? Scandinavian journal of child and adolescent psychiatry and psychology, 2020.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, vol- ume 4. Springer, 2006.
Johan Bjorck, Anmol Kabra, Kilian Q Weinberger, and Carla Gomes. Characterizing the loss land- scape in non-negative matrix factorization. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 6768-6776, 2021.
Christos Boutsidis and Efstratios Gallopoulos. Svd based initialization: A head start for nonnegative matrix factorization. Pattern recognition, 41(4):1350-136 2008
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimiza- tion and statistical learning via the alternating direction method of multipliers. Foundations and Trends@ in Machine learning, 3(1):1-122, 2011.
Michael W Browne. An overview of analytic rotation in exploratory factor analysis. Multivariate behavioral research, 36(1):111-15) 2001.
Jean-Philippe Brunet, Pablo Tamayo, Todd R Golub, and Jill P Mesirov. Metagenes and molecular pattern discovery using matrix factorization. Proceedings of the national academy of sciences, 101(12)4164-4166 2004.
Seungjin Choi. Algorithms for orthogonal nonnegative matrix factorization. In 2008 ieee interna- lional join! conference neural networks (ieee world congress compulational intelligence), pp. 1828-1832. IEEE, 2008
Andrzej Cichocki and Anh-Huy Phan. Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE transactions on fundamentals of electronics, communications and computer sciences, 92(3):708-721, 2009.
Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7:1-30, 2006.
Chris Ding, Tao Li, Wei Peng, and Haesun Park. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 126-135, 2006.
10

---

Under review as a conference paper at ICLR 2023
Chris HQ Ding, Tao Li, and Michael I Jordan. Convex and semi nonnegattnee matrix factorizations. IEEE transactions on pattern analysis and machine intelligence, 32(1):45-55) 55, 2008.
Guiguang Ding, Yuchen Guo, and Jile Zhou. Collective matrix factorization hashing for multimodal data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 2075-2088 2014.
Julian Eggert and Edgar Korner. Sparse coding and nmf. In 2004 IEEE International Joint Confer- ence on Neural Networks (IEEE Cat. No. 04CH37541), volume 4, pp. 2529-2533. IEEE, 2004.
Jicong Fan. Multi-mode deep matrix and tensor factorization. In International Conference on Learning Representations, 2021.
Jicong Fan and Jieyu Cheng. Matrix completion by deep matrix factorization. Neural Networks, 98: 34 41, 2018.
Paul Fogel, S Stanley Young, Douglas M Hawkins, and Nathalie Ledirac. Inferential, robust non- negative matrix factorization analysis of microarray data. Bioinformatics, 23(1):44-49, 2007.
Brian R Gaines, Juhyun Kim, and Hua Zhou. Algorithms for fitting the constrained lasso. Journal of Computational and Graphical Statistics, 861-888 2018.
Cadeyrn Gaskin and Brenda Happell. On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use. International journal of nursing studies, 51(3):511-522 2014.
Irina Gaynanova and Gen Li. Structural learning and integrative decomposition of multi-view data. Biometrics, 75(4): 121-1132, 2019.
Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
David Goretzko, Trang Thien Huong Pham, and Markus Bühhnr. Exploratory factor analysis: Cur- rent use, methodological developments and recommendations for good practice. Current Psy- chology, 40(7):3510-3555 3521, 2021.
Richard Gorsuch. Factor analysis: Classic edition. Routledge, 2014.
Suriya Gunasekar, Makoto Yamada, Dawei Yin, and Yi Chang. Consistent collective matrix comple- tion under joint low rank structure. In Artificial Intelligence and Statistics, pp. 306-314. PMLR, 2015.
James C Hayton, David G Allen, and Vida Scarpello. Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational research methods, 7(2):191-205, 2004.
Craig Anne Heflinger, Celeste G Simpkins, and Terri Combs-Orna Using the ebel to determine the clinical status children in state custody. Children and youth services review, 22(1):55-73, 2000.
Alan Hendrickson and Paul Owen White. Promax: quick method for rotation to oblique simple structure. British journal of statistical psychology, 17(1):65-70, 1964.
John Horn A rationale and test for the number of factors in factor analysis. Psychometrika, 30 (2):179-185, 1965.
Kejun Huang, Nicholas D Sidiropoulos, and Athanasios P Liavas. A flexible and efficient algo- rithmic framework for constrained matrix and tensor factorization. IEEE Transactions on Signal Processing, 64(19):5052-5065, 2016.
Hamid Javadi and Andrea Montanari. Nonnegative matrix factorization via archetypal analysis. Journal of the American Statistical Association, -156 2020.
Bhargav Kanagal and Vikas Sindhwani. Rank selection in low-rank matrix approximations: study of cross-validation for nmfs. In Proc Conf Adv Neural Inf Process, volume 1, pp. 10-15, 2010.

---

Under review as a conference paper at ICLR 2023
Hyunsoo Kim and Haesun Park. Nonnegative matrix factorization based alternating nonneg- ativity constrained least squares and active set method. SIAM journal on matrix analysis and applications, 30(2):713-730, 2008
Jingu Kim, Yunlong He, and Haesun Park. Algorithms for nonnegative matrix and tensor fac- torizations: A unified view based on block coordinate descent framework. Journal of Global Optimization. 58(2):255-11, 2014.
Wojtek J Krzanowski and David J Hand. ROC curves for continuous data. Chapman and Hall/CRC, 2009.
Daniel Lee and H Sebastian Scung. Algorithms for non-negative matrix factorization. Advances in neural information processing systems, 13, 2000.
Tao Li, Yi Zhang, and Vikas Sindhwani. A non-negative matrix tri-factorization approach to senti- ment classification with lexical prior knowledge. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACI. and the 4th International Joins Conference on Natural Language Processing of the AFNLP, pp. 244-252, 2009.
Xiaoqiang Lu, Hao Wu, Yuan Yuan, Pingkun Yan, and Xuelong Li. Manifold regularized sparse nmf for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 51(5): 2815-2826, 2012.
Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online learning for matrix factor- ization and sparse coding. Journal of Machine Learning Research, 11(1), 2010.
José M Maisog, Andrew T DeMarco, Karthik Devarajan, Stanley Young, Paul Fogel, and George Luta. Assessing methods for evaluating the number of components in non-negative matrix factor- ization. Mathematics, 9(2):2884, 2021.
Mark Meckes and Stanislaw Szarek. Concentration for noncommutative polynomials in random matrices. Proceedings of the American Mathematical Society, 140(5):1803-188) 1813, 2012.
Thomas Minka. Automatic choice of dimensionality for pea. Advances in neural information pro-
cessing systems, 13, 2000.
Art Owen and Patrick O Perry. Bi-cross-validation of the svd and the nonnegative matrix factor- ization. The annals of applied statistics, 3(2):564-5944 2009.
Pentti Paatero and Unto Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics. 5(2):111-124 1994.
Yulong Pei, Nilanjan Chakraborty, and Katia Sycara. Nonnegative matrix tri-factorization with graph regularization for community detection in social networks. Twenty-fourth international joint conference on artificial intelligence, 2015.
Kristopher Preacher, Guangjian Zhang, Cheongtag Kim, and Gerhard Mels. Choosing the optimal number of factors exploratory factor analysis: model selection perspective. Multivariate Behavioral Research, 48(1):28 56, 2013.
Yuntao Qian, Sen Jia, Jun Zhou, and Antonio Robles-Kelly. Hyperspectral unmixing via (11122 oparsity-constrainet nonnegative matrix factorization. JEEE Transactions on Geoscience and Remote Sensing, 40(11):422-44297, 2011.
John Ruscio and Brendan Roche. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological assessment, 24 (2):282, 2012.
Daniel A Sass and Thomas A Schmitt. A comparative investigation of rotation criteria within ex- ploratory factor analysis. Multivariate behavioral research, 45(1):73-103, 2010.
Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. Finding a" kneedle" in haystack: Detecting knee points in system behavior. In 201 31st international conference on distributed computing systems workshops, pp. 166-171. IEFF, 2011.
12

---

Under review as a conference paper at ICLR 2023
Thomas A Schmitt and Daniel A Sass. Rotation criteria and hypothesis testing for exploratory factor analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, ((()):95113,, 2011. 2011.
Petre Stoica and Yngve Selen. Model-order selection: review of information criterion rules. IEEE Signal Processing Magazine, 21(4):36-47, 2004.
Shiliang Sun, Liang Mao, Ziang Dong, and Lidan Wu. Multiview machine learning. Springer, 2019.
Bruce Thompson. Exploratory and confirmatory factor analysis: Understanding concepts and appli- cations. Washington, DC, 10694(000), 2004.
George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Björ W Schuller. A deep matrix factorization method for learning attribute representations. IEEE transactions pattern analysis and machine intelligence, 39(3):417-429, 2016.
Wayne F Velicer. Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3):321-327, 1976.
Wayne F Velicer, Cheryl A Eaton, and Joseph L Fava. Construct explication through factor or com- ponent analysis: A review and evaluation of alternative procedures for determining the number of factors or components. Problems and solutions human assessment, pp. 41-71, 2000.
Marley W Watkins, Exploratory factor analysis: A guide to best practice. Journal of Black Psychol- ogy, 44(3):219-246, 2018.
Yangyang Xu and Wotao Yin. block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM Journal on imaging sciences, 1758 2013.
Zheng Xu, Mario Figueiredo, and Tom Goldstein. Adaptive admm with spectral penalty parameter selection. In Artificial Intelligence and Statistics, pp- 718-727. PMLR, 2017.
Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. Deep matrix fac- torization models for recommender systems. In IJCAI, volume 17, pp. 3203-3209. Melbourne, Australia, 2017.
Handong Zhao, Zhengming Ding, and Yun Multi-view clustering deep matrix factorization. In This AAAI conference on artificial intelligence, 2017.
13

---

Under review as a conference paper at ICLR 2023
APPENDIX
CONTENT
Appendix A
Detailed description of the optimization algorithm.
Appendix
Demonstrations for theoretical guarantees algorithm convergence.
Appendix
Error analysis with respect to the choice of latent dimension
Appendix
Visualization the experimental setup for diagnostic prediction evaluation.
Appendix Visualization train, validation and test sets in the multi-questionnaire set-
ting.
Appendix Details of the synthetic questionnaire experiment (both data generation and the experimental setup).
Appendix G of the 21 questionnaires.
Table summarizing optimal (k, (3) detected using ICQF with BCV for each
Full list of top 10 questions for each factor the loading matrix Q. obtained by factorizing the CBCL questionnaire.
Appendix H
Appendix : Full list of top 10 questions for each factor in the loading matrix Q. obtain by performing meta-ICQQoo on the 21 questionnaires.
Appendix M: ROC-AUC scores of each diagnostic prediction obtained using different fac- torizations (Expansion of Table D
Appendix Evaluation and model inference in CBCL-ABCD dataset
Appendix ROC-AUC trend of t-questionnaire for all diagnoses.
A OPTIMIZATION PROCEDURE OF ICQF
Recall that the Lagrangian 4, of ICQF is:
1 IM (M 2)ll (w(w) 3|| + I2(Q) 3| Q11,1 (W ZZ
+
+ - -
1 +I)
Following the ADMM approach, we alternatingly update primal variables W, Q and the auxiliary variable Z, instead of updating them jointly. In particular, we iteratively solve the following sub- problems:
w(++1) arg min
(1)
0.T
+ 4-07
(2)
+ +
(Sub-problem 1)
WEW
2
Q (i+1) ((+1) arg min
0
7 (w(+1)
CIQTT
+
a
(1)
+
+
(Sub-problem 2)
QEQ
2
p
F
Z (++1) arg min
| M 2)
P
2
w(++1)
CIQ
(++
(
Zez
2
(Sub-problem 3)
for some penalty parameter p. We denote the Hadamard product as O. The vector Lagrangian multipliers az is updated via
(1+1) oz + az + (p(z(i+1)
- [W (1((()))
SUB- PROBLERSS 1 AND (EQUATIONS 2 AND
Note that equation 2 (and similarly equation 3 by taking the transpose) can be split into row-wise constrained Lasso problem. Specifically, the n row problem can be simplified into:
14

---

Under review as a conference paper at ICLR 2023
x" arg min L1bb Axll2 + B| 0S
A= b z cQ
(0) (nd
Here the Matlab matrix notation
[n]] represent row extraction operation. As suggested
in Gaines et al. (2018) one can also use ADMM to solve equation
x (1+1) = arg min
L
1b
Axl
1
1 11 B|| -11e y
(9)
2
y (i+1) P x
(1-1)
=
(10)
1 (++1 t ( 7 (i+1) (++1))
(11)
Similarly, μ is the vector of Lagrangian multipliers and 7 the penalty parameter. Projjo.1 refers the orthogonal projection into [0. 1] (inherited from the box-constraints of W). Equation can be solved via the well-established FISTA algorithm (Beck & Teboulle, 2009). Consider the following optimization problem
arg min Al a|| + fx)
(12)
The FISTA algorithm for solving 12 is summarized as follows:
Algorithm 1: FISTA for equation 12
Initialize: 1c-6; ø = 0, Zo = to
Input: L. Lipschitz constant of f
Result: Solution æ of equation] 12
while |a. 2; 1 > 5 do
11+1 arg ming 1/11=]1= 1|1
(x:; [x;)) [|:
4+1 1+v1++1
11+1
t-11 441
01+1
xz);
end
To solve equation 9 with FISTA algorithm, using the notation introduced in equation have
f(x) = pl| - Ax 112221033 + + ( + 00
-
(13)
To compute L, the Lipschitz constant of f., we have
Vf(x) = 2p T A a b) + 7 -c)
A A 71) - 2(pA Ab + TC)
(14)
where c= ( (0) Thus, is just equal the largest eigenvalue of 2(pA + +I).
As recommended in Huang et al. (2016), ADMM provides flexibility to use various types of loss functions and regularizations without changing the procedure. For example, we can simply change to L2,1 norm and equation 8 becomes a constrained ridge-regression problem, which can be ef- ficiently solved by non-negative quadratic programming algorithms. For most clinical usage, the size of questionnaire data is manageable on single machine. However, if optimal computational and memory efficiency is required, various stochastic optimization approaches such as Mairal et al. (010)) replace the ADMM procedure. Yet, unbiased sampling scheme for generating dom batches that handles missing responses is also needed. Such a scheme is non con-trivial to obtain, especially under the multi-questionnairss scenario.
15

---

Under review as a conference paper at ICLR 2023
BUB-PROBLEM 3 (EQUATION
Since both terms in equation are in Frobenius-norm, Z can be optimized entry-wise. In particular, we have the following closed-fore solution for 1
Proj (Mo +
[McM+plW(H)) p(W( C)(Q(t)))"- -a o (pl+
(15)
[min (Min(MM).uuu mmx.
where 1 is 1-matrix with appropriate dimension and @ is the Hadamard division.
B NON-INCREASING PROPERTY OF THE OPTIMIZATION ALGORITHM
In the following, we provide self-contained convergence proof and show that, under appropriate choice of the penalty parameter p. the ADMM optimization scheme discussed in Section 3.2 con- verges to a local minimum. To simplify notation, we denote W(...k)) = (W()), Q0) Z (9) to be the tuple of variables W, Q and Z during iteration (i), (j) and (k) respectively. If
k,
abbreviate it as V(,,, We also denote R [W( ()(()) and for any matrices A, with appropriate dimensions, (A, B) = Trace(A's B). In the following, we are going to show that the Lagrangian is decreasing across iterations. Particularly, we consider the difference of Lagrangian between consecutive iterations:
Cp(W(tt)) az ( Cp(11, )a
= -L/(W(+1), .z 11t1)) Cp ,0z (1)
+ (D((tt))) (0) (2(W0)) (
(16)
(1)
(1T)
Expanding term (I), we have
(V((t)) ,0z (+11) (Z((it)
(i+1) (1)
((+1) R((t))
(1 -zz
(17)
laz 0z 11?
(i+1)
Expanding term (II), we have
CD((1))) az (1)
(A)
(8)
(1)) - ((((6ttii))))) (1) (v((+ti1)))))) azz) L, C,((()) 2222
(V(H)))))) a
+ (Z(((())) azz L(S()), 2z)
(1)
(18)
Expanding (A) by the definition, have
1 I (M (M-Z(41)) 2(++1) 11
2 1IMo
1
SIMOOM- (M 2(0) (a z(41) R(++))
-a0 ( z00 R² R(+t1)
+ マ/1 z(1t1) -R(tD)) F - z R
0
(MO (Z (41) M), M o (z(++)) - 2)0) - MIM o (2 - )1
+ +a22 ((4) Z(9) + o (2(+)) R(41)) 2((11) 2(0) - (+|2((+1)= - 10
= -(MO (2(+1) M) + p. Z (41) +0z
(6) pR (+(++) z(++1) - z
-IMO (z z ( (z(())))) - -z
-(MO (MO(Z( (z()-)) M) (1 (M @ (i(11) -z(1))
16

---

Under review as a conference paper at ICLR 2023
Since Z is the minimizer of equation 4 we have
MO (M (M--2 -
+ + 2 (+11) - -R(+4)
1
(1)
+
=-az
Z
F.
p
P
2
$ MO
(M (M
-200| + 1 =1 R(it1)
+ az
(8)
P
which gives
2 MM (2(+11) -M) (1(t)) (Z(t)) - -IIM (Z 11
5-2 z(+1) + +az pR(it), z(+11) (0(1t))(+++11)) - + +pllZ(+11 - 1
(1)
It further implies
p 2 (it1) + +az - pPR z(i+1) - =z(i)
S - <-(MO (Z(it)) -M), M @ (z(tt))zz())
- 2(0) + 1 1l (2 -Z
111 -z //
+ 2
By direct substitution, we have
(A) s (Mo O (Z (z -2)
M), 2() ((+1) -2(1)
- (MO M (z(++)) - )(0))
+ + (MO(((1)) (2(+1) 1 (1((-1)) 10
|M O (2(1+)) (
2
#l((Z(+1) 110 (M 3 (2((1)
(1 - M) @ (Z(++)) - 2(01)
2 wIM - -
1
(1(2(+1) - 20) S - (l((tt - 11 (19)
2
2
For the second term (B), by definition, we have,
(B)
-
Z R (++1) +
00 2
(8)
z - [W(++]) ,qQ(1.T
+ -a z
1 (1)
2
0
2
p
31QQ 1 11111 31QQ 11,11
R ((+)) - -z(0 1
az [W
(1)
(((()))) -
Q(1:T)
=p
p
L w (i+1), (I(Q))))) ( - + 3(1 (1 11..)
We recall that Q is updated via solving constrained Lasso problems for every row Q (
y = arg min B||z=|11 + Libb lb Arl where A = (w(+)), C),b= 2 + 2z
1 (3)
(20)
p
[rc]
One obtains y if and only if there exists g E d1ly/11 the sub-differential of || 1/1 such that
pA" (Ay - b) + 8+ = 0.
(21)
As || || is convex, we have
| 2 ||//|1 + (xc v.
(22)
which gives
11lll1 - ||a/||1 < (y--a - x. P PAT(Ay (Ay -b) = (Aly (a)x)) 2(y (A) b)
-(Aly
(23)
17

---

Under review as a conference paper at ICLR 2023
Re-substituting x = Q [r: y = Q (1+1)2
A = [w(+t), C). b z + 1 (0)
and sum
[n
p Z
[nd
over r, we have
3 31Q 1 I1.11 311Q S -p R(itt) - -z(0) -0 (1) ( C C -Q0)))
(24)
Therefore, we have
B < - [ .C((Q - -
(25)
With similar argument, we can bound (C) by
(C) - - P
(26)
2
To get an upper bound of ||az (++1) (1)
0z 11,,
we have
|1, (i+1) ( 11?
- S Z(( - 1 + - 12
z(+1) 1 | (1-1) (+Q(+)).T
(+)))T (i(t))) (T (
[((11)) c0Q 1T (w C 10.T
Z (++1) - ||[
(+1),7 Q )| + [[( w cio
(27)
Combining equation
17
27
18
19
25
and 26
with equation 16. have
(1++1) Cp
(6+1))
CV((),
2
((+1)
az F ((+1))
[1]
z
p
[ CI(Q -0007))
a 2
2
[W w 0l0(0.71
2
p (--3) (126t)) - 2 + [[ C(Q -Q0H))))-QQ-)III
C)Q
(28)
This is summarized into the following theorem: Theorem B.1 WNon-increasing property). Assume p 2 22, for all i, we have
(p((1)) ((++1), (1+1), az ((+1) C, (w(), (W())(() z 2
(29)
We set p 3 in all experiments for sufficiency.
C ERROR ANALYSIS WHEN k + k*
Assume that there is ground-truth factorization (W*, Q*) of the given M = ("QQ""" with latent dimension k-,,, where W and Q' are matuix-valued random variables with entries sampled from some bounded distributions. With high probability, the error [M- WQ ||? are minimiz- ing is star-convex towards (W* whenever = k. (Bjorck et al. 2021). To demonstrate the importance of the choice of k. we consider the scenario when l 7 A. below.
First, a more precise assumption for ICQF is to model W as row-independent bounded random matrices. Recall that W is generated by arranging participants' latent representation rows of n x matrix, where the n participants are assumed to be independent from each other and their corresponding latent representations follow high-dimensional bounded distribution.
18

---

Under review as a conference paper at ICLR 2023
Second, let (W1, Q1) and (W2, Q2) be two factorizations with dimensions ln and 22 respectively. Assume both factorizations achieve (a): equivalent mismatching loss in expectation, and (b): equiv- alent expectation approximation to data matrix M:
(a) E |M
W WiQ Q IE
[M - W2 Q NMM-W.00
and (b) B| Q [[
We also assume (c): E
o. and E Σ E
og for all ki
i = 2,
Expanding (a), have
E [Traee (IM- (M 100M-WW00 (oM--())) IE [Traee (IM W2Q W2Q2))
This gives
E (race (W7 W W Q Q 2M WiQ Q() TF (Trace (W2 1 2 W2Q2Q. 2M w2Q
Denote E W, = w.,, [[Q..] lqq for i = 2, we have W, W; H'w, and Q, = Q, HQ. where W, and Q; denote the corresponding centered variables. Note that by the independence of W, and Q and linearity of trace and expectation operator,
E Trace (M' W]QF)]
=F Trace (M w, Qf + M W'W + M ,w,, Of + M jw,, - }
= Trace(MM [ [ = (Tace(M" Q = E Trace (M (2)
(30)
which yields
E Trace N/TWQQ/ Q:) IE [Trcee (W WWW.QQ W2200 Qa)
(31)
Consider E Trace [WT,,Q w, Qf Q(Q)) via definition, we have
IE Trace (Wf W of Q1)
= -Trace (IF
[W] W TE [Q[
(W1)),
=Trace
F
E (W 11)))
Σ m
プー= Q1)
0
x E
0
E
W )
E
(32)
x=1
Incorporating assumption (c), we have
= IE Trace (W7 w, Q(Q: = klow, o..
(33)
Consider equation 3 with kn > kg. For Wi, Q1 W.L.O.G. we pad K2 k columns of zeros. Moreover, let P be an optimal l2 l2 permutation matrix, we also have
E Trace ((w,)) ")
[Trae (W2 W Q2Q o..
(34)
Combining with equation 31 it is equivalent to
klow, oā kno oz
(35)
which gives
a E W
a
E
W [
[|W
(36)
19

---

Under review as a conference paper at ICLR 2023
To evaluate the impact of interpretability of latent representation under different latent dimension,
we consider IE [ WWPllE
E
W W E Trace (W, -2P) (W, W2P))]
a W
L
1| - 2 Trace W W2P)
(37)
As Trace (W] W2) | || W2P F. we also have
E [Traee(W/ WP))
< IB W [|]
E (I
|||
EI
V
oQ.
IE
W1ll
W
[1
(38)
which implies
2
Q Q
E [ W
P1
1 - 2
2/
E |
E
1 1
Q
(39)
Since W; is generated from row-wise independent bounded distribution, if we add a mild assump- tion that 2 ow == ow for all i through re-scaling, Equation 35 implies k102 k2 and therefore
2
2
E W W2 2 1
2
12
12
E
W
V
(
1
E
||:1]
(40)
ki
ki
Ki
If substitute k-" Q1) (W", Q²), we have
E - -
K2
1 IE [[]]]]]]
(41)
which means the relative expected difference between W and W2 is bounded below by
(Vie V1)-
To prove that equation 41 holds in general, we consider the matrix concentration incqualities and show that large deviations from their means exponentially unlikely. Benefitting from the model constraints, we can further assume that W is generated from some high dimensional bounded dis- tribution. In the following, we make use of the main theorem proposed in Meckes & Szarek (2012) on concentration of non-commutative random matrices polynomials. As W; are generated from bounded distributions, I|W ElWilll is uniformly bounded. Therefore, satisfies the convex concentration properties. The theorem achieves the following results:
P [I] - E [[ > tkn?} < C1 exp (-Ca min (2/2)))
(42)
2 E W [1:] 2F (Tace WTW,P)) By % padding W, and W₂ with zeros columns, we assume that W, are all n x n matrices. Then the probability that the any of the terms is deviating from their mean by relative factor is less than C exp( ( for some small c. By the union bound, the probability that the either of them does is less than equal C3 exp((C4ee1)
Recall that IE ||W W2P1 = [|
20

---

Under review as a conference paper at ICLR 2023
D VISUALIZATION OF THE EXPERIMENTAL SETUP FOR DIAGNOSTIC PREDICTION EVALUATION
train/ validate/ test split
matrix factorization
building testing prediction model
train labels
train split
M
Model
Data Matrix M
validate split
walidote
validate labell
proti modictions
inference
sest labels
prodictions
test split
W
evaluation
Figure 4: Setup for diagnostic prediction experiments.
21

---

Under review as a conference paper at ICLR 2023
E VISUALIZATION OF TRAIN, VALIDATION AND TEST SETS IN MULTIPLE QUESTIONNAIRES SETTING
Data avaliability (train) Original Value ranges
Subjects
Age Sex
Data avaliability (validation)
subjects
Age
Data avaliability (test)
Subjects
22
RILL ASSO
con.
wro on mois
SOOED
SCD
S5.
TE
Yok-
Age Sex
souRT
Questionnaires
Figure 5: Train, validation and test split of one data re-sampling.
Table 2: Number of positive diagnostic labels HBN dataset
number of positive cases
Diagnosis
Depression
211
GenAnxiety
262
ADHI
506
Suspected ASD
222
Panic, Agoraphobia, Separation Anxiety, Social Anxiety
327
BPD
299
Specific phobia
234
OCD
64
Fating Disorder
97
PTSD Trauma
27
Sleep problems
118
F DETAILS OF THE SYNTHETIC QUESTIONNAIRE EXPERIMENT
This experiment aimed at comparing the effectiveness of BCV and other procedures for estimat- ing the number of latent factors in synthetic example, for ICQF and two baseline methods. We generated the synthetic guestionnaire with &- 10 factors (0, 1 bounded), where each factor was present in isolation for 200 participants and in tandem with another factor for others. Each factor had an associated loading over 100 questions. The answers to questions are bounded between 0 and 100.
22

---

Under review as a conference paper at ICLR 2023
To generate the answer matrix M 2000 100 we design the underlying matrices W and Q di- rectly. We first create matrix D of size 200 x 10, shown in are correlated with a step-like pattern, where each "step" is of length 20 and entries on the "step" have weight 1. Every consecutive pair of steps is overlapped by 10 units to synthesize correlation between latent factors. An entry Wli, is then defined to be
The 10 columns of D
W == D(i.j] *a*b, a~ a~ (0.5,0 1), b ~~ p 0.9)
(43)
where (0.55 1) is the uniform distribution between [0.5, 1] and B(1.p 0.9) is the Bernoulli distribution with probability p = 0.9. The matrix Q of size 100 x 10, shown in Figure 6 (center) is defined to be
Qii.1] = c+d, d, c~ (O, 100), d d~ dd~Bli 0.,)
(44)
Having W and Q, we obtain a noiseless data matrix Melea = min(0, (0, max (WQ", 100)). To introduce additive noise, we modify Maleon by
max(Maaeam Melean +e* * f, 100)), f ~ (-100, 100) M = min (0, max
(45)
where e follows a discrete probability distribution with P(e 1) 6, P(e 0) 1 - 6. This
yields a data matrix M, shown in Figure 6 (right) for å 0.3. If 8 is high, more entries in the M will be contaminated by the additive noise f.
W. range [0.00, 1.00]
Q, range = [0.00, 99.70]
M. 6 = 0.3
18
104
120
136
168
194
SMRKAM
Latent dimension
Latent dimension
Figure 6: Synthetic example of W, Q and M with noise density 6 = 0.3.
We compared ICQF against f--regularized NMF (G)-NMF) (Cichocki & Phan 2009) and fac- tor analysis with promax rotation (FA-promax) (Hendrickson & White 1964). as factors can be correlated. Both ICQF and NNNMFF were initialized with NNDSVD (Boutsidis & Gallopoulos) 2008), and the sparsity (3 = le-1) and stopping criterion (relative iteration convergence tolerance le- 3) for fairness. The estimation method for FA was minimum residual.
Table 3 shows the mean error E and the standard error sg of the detected k versus ground-truth
k. I0, across different generated datasets. Statistics are marked with asterisk they contain runs
where inflection point was detected using the kneed algorithm (Satopaa et al.) (001 We tested
five popular detection algorithms: BCV (Kanagal & Sindhwani] (0100) [BC,, BTC² (Stoica & Selen 2004 CCC (Fogel et al. 2007) and Dispersion (Brunet et al. 2204). For ICQF BCV is the
best overall detection scheme at all noise levels; BIC, performs well for low noise only. Within the
three common schemes for FA, Horn's PA (Hom) 1965) and MAP (Velicer) 1976) are superior to
BIC₂ (Preacher et al.] 2013). PA achieves the best performance when the noise density is high. This
result aligns with the conclusions Velicer al. (2000); Watkins (2018); Goretzko et (202I).
and (K) := log (IIM WQ (P:) km log (min( /m, vn)?). There other similar versions of ma BIC but their performances are indistinguishable, thus we only include two representatives in the manuscript.
'The two BIC versions respectively BTC)(k) = log (IIM WQT
+ km log m++
mn
23

---

Under review as a conference paper at ICLR 2023
Table 3: Experiment & estimation for synthetic questionnaires. The average error (?) and the standard error of estimated (sg) are reported. Statistics with asterisk (*) ignore undetectable inflection points.
Noise density ò
Detection schemes for NMF
Methods
BCV
BlCi
BJC₂
CCC
Dispersion
0.1
(0.10, 0.06)
(0.13, 0.08)
(0.10, 0.06)
(1.33, 0.24)
(0.23, 0.09)
ICQF
0.2
(0.11, 0.06)
(1.23, 0.34)
(0.67, 0.23)
(1.14, 0.21)
(0.93, 0.16)
0.3
(0.77,0.13 0.15)
NaN
(2.40, 0.48)
(0.96, 0.18)
(2.60, 0.26)
0.1
(0.17,0.07)
(1.20, 0.21)
(0.90, 0.23)
NaN
NaN
NMMMF
0.2
(2.37, 0.33)
NaN"
(1.10, 0.30)
NaN
NaN"
0.3
(2.40,
NaN
(2.77, 0.54)"
NaN
NaN
Detection schemes for FA
PA
MAP
BICs
0.1
[0.17, 0.07)
(0.11, 0.06)
(0.30, 0.03)
FA-promax
0.2
(0.53, 0.10)
(0.13, 0.06)
(0.93, 0.11)
0.3
(0.87, 0.14)
(1.27, 0.20)
NaN
G TABLE SUMMARIZING THE OPTIMAL (k, B3 OF THE 21 QUESTIONNAIRES
Table Optimal 3) of all 21 questionnaires.
Questionnaire
Abbreviation
y questions
Subscales
B
Affective Reactivity Index (Parent- Report)
ARLP
nan
0.01
Affective Reactivity Index (Self-Report
ARLS
nan
0.01
Autism Spectrum Screening Questionnaire
ASSQ
27
nan
0.01
Conners (Self-Report)
C3SR
9
4
0.05
Child Behavior Checklist
CBCL
119
9
0.5
Extended Strengths
Weaknesses Assess-
ESWAN
65
nan
0.2
ment of Normal Behavior
Inventory of Callous-Unemotion Unemotional Traits
ICU_P
24
3
4
0.1
(Purent-Report)
Inventory of Antemtioonn Traits
ICU SR
24
3
0.1
(Self-Report)
Mood and Feelings Questionnaire
MFQ P
34
nan
0.1
(Parent-Reed Report)
Mood and Feelings Questionnaire (Self-Re Repoot)
MFQ_SR
33
0.1
The Positive and Negative Affect Schedule
PANAS
20
2
0.05
Repetitive Behaviors Scale
RBS
43
5
0.1
Screen for Child Anxicty Related Disorders Rerent-Report)
SCARED
41
5
0.1
Screen for Child Anxiety Related Disorders
SCARED_SR
41
5
0.3
(Self-Report)
Social Communication Questionnaire
SCQ
40
nan
0.02
Strength and Difficulties Questionnaire
SDQ
33
9
6
0.05
Social Responsiveness Scale (School Age)
SRS
65
7
0.5
The Strengths and Weaknesses Assessment of Normal Behavior Rating Scale ADHD
SWAN
2
0.02
Symptom Checklist Report)
SympChck
63
nan
3
0.1
Teacher Report Form (School Age)
TRF
116
19
8
0.5
Youth Self Report
YSR
119
11
0.2
24

---

Under review as a conference paper at ICLR 2023
H FULL LIST OF ToP 10 QUESTIONS FROM FACTORIZING CBCL QUESTIONNAIRE
CBCL Factor 1
[CBCL_95] 93. Temper tantrums hot temper
Aggressive
[CBCL_33] 19. Demands a lot attention
Aggrassive
[CBCI_03] 3. Argues
Aggressive
[CBOL
Aggressive
[COCL_68] 68) 60. Screams
Aggressive
[CBCL 41] 41. Impulsive acts without thinking
Attention
[CBCL_80 [6] a6, Stubborn, sullen, inritble
Aggressive
(EBCL_8))22 28.
school,
[CBCL_27]27
Social
[CBCL_164] 104, Unusually loud
Aggressive
0.8
1.0
CBCL Factor
[CBCL_5G] S0. The teartul or anxious
AnxDep
[CBCL 112] 112. Worries
AnxDep
[CDCL_71] 71] 71, Self-corscious or easll emberressed
AnxDep
[CBCL [5] highstrung. or
AnxDep
[CBCL_32]] Feels
perfect
AnxDep
(CRCI_29) 29, Fears certain animals, stuations, places, other than school
AnxDep
[CBCL 75] 75, Too shy timid
WithDep
[CBCL, 103) 103. Unhepuy, sed, or depressed
WitDepp
[CBCL_35]
AnxDep
100) Thouble sleeping
Thought
CBCL Factor
[CBCL 08] Can't concentraee, can attention for long
Attention
(CBCL 78) 78. Inattentive or easily distracted
Attention
[CSCL_10| 10) 10, Can't sit still, restiess or hyperactive
Attention
[CBCL_G4 [0C
things haishe starts
1C Daydreams
hisshar thoughts
Attantion
[CBCL_61] 61. Poor school work
Attention
(CBCL_43) 41] 41. Impubsive or ects withou: thinking
Atention
[CBCL_93] 93, Telks too much
Other
[CBCLO1]] tao young for age
[CBCI_an) Stares
Attention
Figure 7: Top 10 questions ranked by Q in CBCL using Q obtained from ICQF (Factor 1-3).
25

---

Under review as a conference paper at ICLR 2023
CBCL Factor 4
[CDCL 61] 61, Pour school work
Aitention
[CBCL_0G [o]
keeps things
Withepp
[S] There very
enjoys
[CBCL_26] 26, Doesn't seem to feel guilty after misbehavinn
RuleBreak
[CaCI_04] 4. Fails to finish things he/she starts
Attention
[CBCL 26] 28. Breaks rules at home, school, or elsewhere
Rulebreak
[CBCL 43] Lying cheating
RuleBreak
[CBCL 42) Would rether be along than with others
WithDep
[CBCL_DDE) [03] 103, Unhappy. sad, cepressed
WithDap
[CRCI 102] [ERCI_102]
102, Underactive, slew mowing, or locks energy
WtDDep
0.0
0.2
0.4
0.6
0.8
CBCL Factor
[CBCL_79] 79. Speech prablem
Social
ICBCI_011 too young for hishher age
Attention
being younger
Social
[CBCL 66] 66. Repeats certain
compulsions
Thought
(CBCL 62) 62. Poorly coortinated clumsy
Social
[CBCL [8] 38, Gets teased a lot
Social
[CBCL_ 84] Strange bohavlor
Thought
[CRC] 48. by other
Social
[CBOL 42] 42. Would rather along than with others
WithDep
[cstt_ yc] 80. Stares blankly
Attention
0.00
0.25
0.50
0.75
1.00
1.25
CBCL Factor 6
[CRCL_C7] Bragging, boasting
Other
[C3CL_M4] 74, Showing or clowning
Other
[COCL
fingernails
[CBCL_93] 93) Taiks
[CBCL_56R [3C]]
565: Stomachaches
Somatic
[CBC1 56R1
556. Headaches
Somatic
[CBOL 431 lying or cheating
RuleBreak
[COCL
being
RuleBreak
[CBCL_50) 58. Picks skin, other bady
Thought
[CBCL_38] 338 38, Gets teased lor
Social
0.0
0.4
0.6
Figure 8: Top 10 questions ranked by Q CBCL using Q obtained from ICQF (Factor 4-6).
26

---

Under review as a conference paper at ICLR 2023
CBCL Factor 7
[CDCL_55] 55, Overweight
Other
[CBCL_3] Overeating
[CBCI_ 102] Underactive, moving, lacks energy
WimDep
[cacL] 568] 568. Headaches
Somatic
|CACL_S6A [66A]
56A, Aches or pains (not stomacc headachasi
Somatic
(CBCL 54] 54, Overtired without good reason
Somatic
[CBCL, 62. Poorly
clumsy
[CBOL SBF]
58F. Stomachhaches
Somatic
[CBCL] S6C]
SBC. Nausea, feels sick
Somatic
[CRCI_aE] 36, Gets hurt lst, accident prone
Social
0.0
0.2
0.4
0.6
0.8
CBCL Factor B
[CBCL_ 44] 44, Bites fingeralls
Other
ICBCI_581 Picks nose, skin, other parts body
Thought
highstrung,
[CaCL] 46] Nervous movements twitching
Thought
[CBCL 109) 109. Whining
Other
[CBCL [6] 86, Stutborn, su liln,, imitza
Aggressive
(CBCL_95]99 95, Tempor cantrums temper
Aggressive
[CRCI_01]
yaung hisher age
Attention
[CaCI 03) 3, Argues
Aggressive
(CHCL_ 68) b8, Screams lot
Aggressive
0.0
0.5
1.0
1.5
2.0
Figure 9: Top 10 questions ranked by Q CBCL. using Q obtained from ICQF (Factor 7-8).
27

---

Under review as a conference paper at ICLR 2023
1 FULL LIST OF TOP 10 QUESTIONS FROM META-FACTORIIATTION
paying attention,
Sympphhc__ee
diffiiulty remaining sested schaol
SympChck_36c
Restless, overactive, cannot still long
SDQ_02
35. is often easily distracted
SympChck_SS0 35C
Constantyy y fidgeting or squirming
SOQ 10
Cen't concentrate, can't pay attention long
75. Inattentive distracted
37. Ofton acts before thinking
SympChok_37C
Easily distracted, concentration wanders
SD0_15
(mppeimmen) Sits still (controls movement of hends or feet or controls squirming)
SWAN_10
0.0
0.5
1.0
meta-ICQF ICQF
(Impainment) Helpful it someone hurt, upset feeling
SDQ_08
Considerate people's feelings
(impeimentt Often offers others (perents, teachers, chiddren)
$D0_20
(mpaireent) Shares readily with other children, for examale toys. treats, pencill Ifar 11-17 year dids: Shares readily with other youth, for exampla CD's, games, food)
$D0_04
11. like it when people say good things about me.
C35R
(Impairment) younger
SDQ_17
happy cheerful.
32. feel shy with people don't know well
SCARED 5R
10. feel nervous with people don't know well
SCARED SR_10
26, It is hard me to talk with people don't know well
SCARED SR
0.0
0.2
0.4
meta-ICOF Factor
(Impainmentt Helpffl if somneone is hurt, upset or feeling
$0Q_09
easy
feeling.
ICU_SR_14
(Impaimentt) Cons derate other people's teelings
I expressive and emotional.
ICU_SR_
[mpairmmnt) Often offers help others (parents, teachers, children:
$D0,20
11. it when people say good things about me.
C35R
(Impaiment) Stares readily with other childrrnn examale toys, treats, pencils (for 11-17 year dids; Shares readily with youth, example CD's, games, food)
25.
cheerful.
C35R
10. Proud
PANAS,10 10
(1 (Ipaimment) 19, Active
PANAS_19
0.0
0.1
0.2
Figure 10: Top 10 questions ranked by Q in meta ICQF (Factor 1-3),
28

---

Under review as a conference paper at ICLR 2023
meta-ICC)
Factor
5. lis scrong and explosive feelings anger
Sympphck_co OOC
becomes really upset
temper
SympChck_a8C
Loses nasily
ARI_P_00
Often loses hisher temper
ARI
Umpaimennt) 4a. Control the intensity of anger at home
DMDD_44 _AA
(ingaimmen)] Be. Avoid or limit temper Cantrums home
DMDD_8A
angry frequently
Expresses hishner feelings
ICU P
19. expressive and emotional.
ICU
14. s oasy to tell how halthe is foeling.
ICU_P_14
0.0
0.2
0.4
0.6
meta ICQF Factor
Talks out turn
TRF 53
Imgulsive or acts without thinking
TRF_41
TRF
67. Disnupts class discipline
TRC
19. Demands lot of attention
TRF _19
91. Talks too much
TRT 93
Can't still, restess hyperactive
TRF
Argues let
TRE
23. Disobedient school
TRF 23
Defiant, talks back to staff
TRF 06
0.0
0.2
0.4
0.6
0.8
1.0
1.2
meta-ICQF Factor
5. Has strong and explosive feelings of angar
SympChck_osc
36, Often becomes really upset and loses hisher temper
Sympphcck38C
Argues back
age
SympChck_
2. Often feeks annayed imrittte
SympChck_0ee
Lases temper casily
ARI ARI___06
Often loses his/her temper
ARI 02
40. Actively aisobeys or doesn't listen to edult rules
SympChck 40C
inrisbiilty iit causes
Gets angry frequently
ARI 05
(Im (mmairment) 4a. Control the intensity of home
DMCD, 4A
0.0
0.2
0.4
0.8
Figure 1I: Top questions ranked by Q in meta- ICQF (Factor 4-6).
29

---

Under review as a conference paper at ICLR 2023
meta-ICOF Factor
17, Deydresms or gets loss, his/herr thoughts
TRT_17
things
78. Inattentive distracted
8. Can't concentrane, can't attention for long
TRF 08
61. Poor school work
TRF_61
190, Fails to carry out assigned tasks
TRF_100
relationships peers
lacks best friend
can be with other children but only on hishhr terms
ASSQ_16
makes nalve and embarransing remariss
ASSO
0.0
0.1
0.2
0.3
0.4
0.5
0.6
meta ICQF Factor 8
19, very expressive and emetional.
ICU_P_15
is easy tel nelshe feeling.
ICU_P_14
Responds anpropriately changes others le.g.
playmate's moad
changes happy
Expresses histher feelings openly.
ICU P_01
7. aware of what others are thinking feeling.
SRS 07
21. able imitate others actions.
21
Focuses his attention others are looking listening- use gestures, other pointing pulling your
you shethe
43. Separates easily from vers.
SRS,43
22. Does she/he ever spontanecusly point at things around herthim just show things (not
because sheihe viants them)?
SCD 22
0.0
0.1
0.2
0.3
meta-ICQF Factor 9
23. My child is wornee
SCARED_P_23
35. My child worries about how well hershe does things
SCARED_P_35
CBCL,112
worries
S00_09
5. My child worries about other people llnng himyher
SCARED_P_00
14, My child warries about being as good as other kids
SCARED_P_14
My chil worries about things working out himuher
SCARED_P.21.
geing hepper
SCARED,
Nervous situations, lases confidence
SOQ_18
50. Too feorful anxious
CBCL,50
0.0
0.4
0.6
Figure 12: Top 10 questions ranked by Q in meta- ICQF (Factor 7-9).
30

---

Under review as a conference paper at ICLR 2023
meta-ICOF Factor 10
(mpairment] 5, Orgenices Lasks and vitiveties
SWAN 005
(Ipaiiment) Keeps thingo necessary
07
(Irpairment) Enngages
require sustained
SWAN
limparment 4, Follows through on instructions and finisnes school work and chores
SWAN 04
Impairmentt 1. Gives close attention to detail and avoids careless mistakes
SIWAN 00
(Impaimeent) 8A, Concentrate
MDD,
concentrate, pay
long
(mpeirment) Sustains attention tasks activities
SWAN
Umpairent) B. linnores extrancous stimul
SWAN 08
78. Inattentive or caslly distracted
CBCL_78
0.00
0.25
0.50
0.75
1.00
meta ICQF Factor
often lase my temper
ARI___02
my temper easily
ARI_5_06
easily annoyed by
AR|_S_01
Overall iritability problems
AR] 5,00
get angry frequenttyy
ARI
112. I worry lor
YSR, YSR_132 132
45. am nervous tense
stay angry long
ARI
33, about what is happen the future
SCARED SR
71. I am sell conscious or easily embarrassed
YSR 71
0.0
0.2
0.4
0.6
0.8
1.0
meta-ICQF Factor 12
(mpaiment) 6c. Handle angar when doas not get what helshe wants at school
DMDO_6C
(Impairennt 7c, Hendle frustration on day-to-day basis at school
DMDO,7C
(impaiment) quickly
at
DMD0,5C
(Impaiment) anger does
halshe friends
DMD0_60
[mmpaimmntt) 1c. Avcid control 'arguing or yallingg at school
DMDO_IC
(Impair menti 4c. Control the intensity of anger at school
DMDQ_4C
(Impairment) 7o, Handle frustration on day-to-day basis with friends
DMD0_78
Iimairment Compared (iriemmment)
tempger
school
child maintainnd appropriate and bohavior school
age, past well this
DMCO_10C
(mpairment) 1b. Avaid control 'arguing yelling" with friends
DMDD_10
0.00 0.25 0.50
0.75
Figure 13: Top 10 questions ranked by Q meta ICQF (Factor 10-12).
31

---

Under review as a conference paper at ICLR 2023
meta-ICOF Factor 13
23, Does shethe ever use gestures, other than pointing ar pulling your hand, let you know what she/he wonts?
500,23
21. Does shefhe ever spontaneeuslyy copy you other people! or what you are doing (such vacuuming, gardening. or mending things]?
$00_21
spontanecusly point things
show you them)?
because
8. am concerned about the feelings of others.
ICU [2__5R08
5, reel bad or guilty when do something wrong.
KCU_SR_0S
work herd everything do.
ISU_5R_28
ICU (R,5RR
(mpairment) Retuin quickly baseline mued behavier when frustrated schoel
DMDD,
(Imaaiment) Ic. Awaid cantrol 'arguing yelling" at school
DMDD_IC
penpalmmett 4c. Control the intersity of anger at school
DMDD_4C
0.0
0.2
0.4
0.6
meta ICQF Factor 14
Restless, overactive, cannot stay still for long
SDQ_02
Constantly fidgeting squrriing
SD0_10
use phrases or say thing over and over exactly way (elther parsses
other people
shethe up?
34. Has trouble paying strention, affects school performance
SympChck__ee
4, Does shefhe ever use socially inappropriate questions statements? For example, does she/he ever regularly ask personal questions make personal comments awkward times?
SCQ,04
(mpairment) 11. Stays seated (when required by class rales or social convent ons)
SWAN
Hos fftcaity remaining seated at home school
SympChok_36c
(Impaiment) still
controls se.imming
SWAN 10
Docs she/e special appropriate her/his
thoir dinosaurs)?
atherwist.
SCO,13
group
7. Does sheihe ever say the same thing ever and ever again?
SCQ 07
0.0
0.5
meta-ICQF Factor 15
32, My child teels shy with people heishe doeen't know well
SCARED_
My child shy
SCARED P.41
My
people
P_03
hard for my child to talk with people hetshe doesn't well
SCARED P_26
child feels nervous when helstee
dances, place where there will
people thet heishe doesn't well
SCARED P_40
10. My child feels nervous with people hefshe doesn't know well
SCARED P_10
something while they watch himiher (for example: read aloud, speak, play agame, play sport)
39, My child fents nervous when he/shr with other children ar adults and nalshe has da
SCARED P.39
shy with people
SCARED
10. nervous with people don't know well
SCARED SR__RR_1
26, hard for me to talk with people don't know well
SCARED_SR.2 5R,
0.0
0.5
1.0
Figure 14: Top 10 questions ranked by Q meta ICQF (Factor 13-15).
32

---

Under review as a conference paper at ICLR 2023
J ROC-AUC SCORES FOR EACH DIAGNOSTIC PREDICTION ACHIEVED BY DIFFERENT FACTORIZATIONS (EXPANSION TABLE
Table 5: ROC-AUC scores of the diagnostic prediction under different data inputs and factorizations.
Factorization
Diagnosis
Questionnaire
meta ICQF
ICQF
NMM
FA-promax
subscales
raw
CBCL
0.812
0.802
0,809
0.782
0.803
SDQ
0.814
0.810
0.808
0.809
0.799
Depression
SympChck HBN
0.806
0.805
0.803
NaN
0.784
0.829
0.823
0.806
0.813
0.813
0.818
0.823
0.825
0.825
0.798
0.816
0.825
0.817
0,830
0.815
0.817
GenAnxiety
"
0.759
0.798
0.794
NaN
0.778
0.840
0.833
0.832
0.836
0.832
0.840
0,803
0,806
0,809
0.791
0.806
0,806
0.804
0.805
0.799
0.798
ADHD
0,809
0,812
0.811
NaN
0,805
0.831
0.828
0.827
0.833
0.798
0.826
0,834
0.837
0.842
0.783
0.849
Suspected ASD
0.800
0.801
0,814
0,807
0.808
"
0.793
0.836
0.831
NaN
0.849
0.891
0.898
0.891
0.903
0.873
0.903
Panic, Agoraphobia, Separation Anxicty,
0.766
0.755
0.751
0.742
0.767
0.742
0.737
0.738
0.746
0.739
"
0.750
0.754
0.760
NaN
0.753
Social Anxiety
0.802
0.786
0.785
0.790
0.770
0.789
0.776
0.769
0.785
0.780
0.780
0.767
0.769
0.767
0.759
0.777
BPD
0.750
0.749
0.756
NaN
0.754
0.786
0.777
0.775
0.775
0.777
0.784
0.623
0.614
0.632
0.621
0.647
Specific
0.663
0.654
0.663
0.657
0.737
Phobia
0.658
0.662
0.648
NaN
0.656
0.654
0.669
0.628
0.643
0.790
0.767
0.775
0.778
0.755
0.740
0.744
0.742
0.728
0.657
OCD
0.765
0.786
0.801
NaN
0.818
0.782
0.722
0.764
0.738
0.722
0.783
0,635
0.626
0.599
0.586
0.602
Eating Disorder
0,651
0.631
0.620
0.610
0.737
0.633
0.637
0.606
NaN
0.610
0.665
0.637
0.665
0.697
0.645
0.799
0.767
0.749
0.710
0.767
PTSD Trauma
0.770
0.735
0.741
0.745
0.633
"
0.768
0.746
0.727
NaN
0.645
0.778
0.784
0.715
0.717
0.732
0.734
0.787
0.787
0.825
0.770
0.803
Sleep problems
0.805
0.800
0.802
0.796
0.782
0.797
0.789
0.787
NaN
0.761
0.850
0.841
0.832
0.828
0.826
0.839
33

---

Under review as a conference paper at ICLR 2023
K
EVALUATION AND MODEL INFERENCE IN CBCL-ABCD DATASET
K.1 DATA
Data from the Adolescent Brain Cognitive Development (ABCD) study (hutps:/abbcdstudy.orgg held the NIMH Data Archive (NDA), consists of over 11,000 youth age 9-10. They are re- cruited from 21 sites across the US, aiming to provide socio-demographiiclly--ieee Psychopathology was examined in the children using the parent-reported Child Behavior Checklist (CBCL). Similarly,
sample.
have the age and sex at birth of each participants confounds and diagnostic labels (Table 66 for 11 conditions used for the model evaluation. By climinating subjects with in- complete age and information, obtain 11,681 unique subjects in CBCL-ABCD questionnaire.
Table 6: Number of positive diagnostic labels in ABCD dataset
number positive cases
Diagnosis
Depression
743
GenAnxiety
715
ADHD
2550
Suspected ASD
3244
Panic, Agoraphobia, Separation Anxiety, Social Anxiety
2031
BPD
802
Specific phobia
3131
OCD
1258
Eating Disorder
1205
PTSD Trauma
550
Sleep problems
1129
K.2 DIAGNOSTIC PREDICTION (PARALLEL TO SECTION 5.3.2
We perform the same diagnostic prediction evaluation on CBCL-ABCD dataset as reported in Sec- tion 5.3.2 We split participants into train, validation and test sets with ratio 70/15/12 based on the distribution of confounds and diagnostic labels. We resample 50 dataset splits using different seeds and carry out prediction experiment each split. Table reports the summary of averaged AUCs achieved by different factorization methods.
Table 7: Averaged ROC-AUC scores of the diagnostic prediction in CBCL-ABCD dataset
Factorization FA-promax
Diagnosis
ICQF
fMNMF
subscales
raw
ADHD
0.914
0.913
0.912
0.907
0.915
BPD
0.781
0.776
0.781
0.775
0.783
Depression
0.787
0.785
0.786
0.794
0.791
Eating Disorder
0.546
0.551
0.543
0.555
0.567
GenAnxiety
0,867
0.830
0.872
0.868
0.884
OCD
0.789
0.748
0.788
0.781
0,805
PTSD Trauma
0.804
0.803
0.812
0.798
0.814
Panic, Agoraphobia, Separation Anxicty,
Social Anxiety
0.767
0.727
0.769
0.765
0.777
Sleep problems
0.825
0.726
0.762
0.743
0.833
Specific Phobia
0.684
0.654
0.680
0.681
0.695
Suspected ASD
0.777
0.755
0.774
0.769
0.782
Average
0.776
0.752
0.771
0.767
0.786
The null hypothesis is rejected and the post-hoc Nemenyi test indicates that subscales and 4MMMFF slightly but significantly worse than the raw setting, while other differences inconclusive. Similarly, we conclude that ICQF diagnostic information even though we introduce addi- tional regularization and constraints versus other methods.
34

---

Under review as a conference paper at ICLR 2023
K.3 STABILITY OF MODEL
In Section 5.3.3 we demonstrated that the additional regularization and constraints in improve the interpretability of the factorization. Coherently, we should expect to obtain consistent factorization result from [CQFF if is applied on dataset from another population. Using CBCL- ABCD dataset, we extend our evaluation via studying the stability of factor loadings (matrix Q) in cross-population scenario.
Pearson Correlation best-matched pair factors Q from ABCD and HBN dataset
1.0
FA-promax
f1-NMF
0.8
ICQF
Correlation
0.4
0.2
0.0
-0.2
2
3
Factor
Figure 15: Sorted Pearson correlation of best-matched pair of factor loadings Q obtained from CBCL-ABCD and CBCI.-HBN questionnaire.
For each method [CQF L NNMF and FA-promax), we factorize CBCL-IIBN (care-seeking pop- ulation) and CBCL-ABCD dataset (general-diversified population) separately obtain the factor loadings Q HBN and Q ABCD respectively. In this experiment, for fairness sake, we set the number of factors & = 8 for all methods. Considering QHBN as the reference, we perform the greedy-matching algorithm to re-order factors in QBCD and compute the Pearson correlation coefficient for every best-matched pair of factors from (QIBN, QABCD).
Figure 15 shows the trend of the 8 Pearson correlation coefficients achieved by ICQF NMFM and FA-prrmmx factorizations. Since there sign ambiguity FA-promax, we take an absolute to correlation coefficients achieved from FA-promax. The first 5 correlation coefficients are close to for all three methods, suggesting that methods capture similar factors in two different populations. However, starting the 6th pair of factors, correlation coefficient achieved by ICQF is superior to those from & NMF and FA-promas This indicates that the extra constraints help stabilize the factorization outcomes and are potentially advantageous transfer learning model interpretation.
35

---

Under review as a conference paper at ICLR 2023
L ROC-AUC TREND OF OTHER DIAGNOSES IN L-META-QUESTIONNAARE
GenAnxiety
0.90
0.80
AJC
11-NMI
0.70
100F
0.00
0.50
FA: promax
PA-promax-12
0.55
Number of questions
ADHD
0.90
0.25
AUC
11-NMF
ICOF
0.65
FAppromax
0.00
FA propromaa 15
Number of questions
Suspected ASD
0.80
AUG
0.90
11-NMF
ICQF
M-promex
0.05
FA promax 15
Number of questions
Ponic Agoraphobia SeparozionAnx SocialAnx
0.9
ADC
I1-NMF
ICOF
FA-pro
W-promax-11
metsfacter
500
Number questions
Figure 16: ROC-AUC trend of GenAnxiety, ADHD, Suspected ASD and (Panic, Agoraphobia, SeparationAnx, AnxialAnx).
36

---

Under review as a conference paper at ICLR 2023
0.90
0.85
0.80
AUC
0.75
11-NMF-15
1001
TA promixx
melafactor
100
200
500
Number of questions
Specific Phobia
0.70
0.65
AUS
I1-NMF
11-NMF- -1
ICQF
0.50
0.45
FA-pro nax
Number of questions
OCD
AUC
11-NMF
11 NMP 15
ICQF
FA. pro max
Pl-promex-15
0.3
300 Number of questions
Fating Disorder
0.9
0.8
AUC
0.5
NMF
11-NMF-15
ICQF
0.4
FA-pro nax
FA-promex-15
320
Number of questions
Figure 17: ROC-AUC trend of BPD, Specific Phobia, OCD and Eating Disorder.
37

---

Under review as a conference paper at ICLR 2023
PISD_ Trauma
AUC
11-NMF-15
0.4
I00F
0.2
FA promex
metafactor
0.0
100
200
30C
400
500
Number of questions
Sleep Probs
AUC
I1-NM
11 [1-NMF-U 15
ICQF
0.5
FA-pro nax
max-15
0.5
Number of questions
Figure 18: ROC-AUC trend of PTSD Trauma and Sleep Problems
38


Reviews:
[{"review_id": "wrrdsaLNf5", "reviewer": "Reviewer_53sj", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors present a novel factorization method for questionnaires that emphasizes preserving interpretability (ICQF). Authors provide theoretical convergence guarantees for their algorithm. The approach is validated using synthetic data with known latent dimensionality. Furthermore, authors apply their approach to real data and show ICQF can uncover a reduced set of common factors spanning multiple questionnaires. This work shows that ICQF algorithm can preserve relevant information for diagnosis and performs well compared with similar approaches.", "summary_of_the_review": "Overall a solid contribution to the field of factor analysis and a very interesting tool for analysis of questionnaire data. In my opinion this work deserves a spot in the conference.\n", "strength_and_weaknesses": "Strengths \n\nA sound method with some theoretical guarantees that performs well in practice and has a wide range of impactful applications.\n\nWeaknesses", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "sM_nWCaTWJ", "reviewer": "Reviewer_988h", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "A new matrix factorization method is presented in this paper in order to improve the interpretability of questionnaires through bounds and sparsity constraints. The proposed method utilizes an optimization procedure with theoretical convergence guarantees to detect latent dimensionality in synthetic data. The empirical studies only applied to two datasets, including, a commonly used general-purpose questionnaire, and the Healthy Brain Network study. Overall, I found this paper difficult to follow, and I am not sure from the standpoint of machine learning how the proposed method is a novel one. Additionally, the empirical studies did not provide sufficient evidence to support the proposition that the proposed method is superior to existing approaches. Major concerns and minor comments are presented in the review section.", "summary_of_the_review": "1) The performance of the proposed method is only benchmarked on two datasets, i.e., the general-purpose questionnaire, and the Healthy Brain Network study. There are several reasons why empirical studies did not provide sufficient evidence for evaluating a novel machine learning approach - such as batch effects, data collection error, missed label handling, etc. Several more datasets should be benchmarked in order to be able to trust the current results.\n\n2) According to the data section, the Healthy Brain Network has 11 class labels. Therefore, reporting ROC-AUC (which is mainly used for binary classification) in Table 1 and Figure 3 may not be an appropriate method of evaluating the proposed method. What was the reason behind not using classification accuracy in the process?\n\n3) The proposed method should be compared with the state-of-the-art factor analysis techniques and method techniques that are widely used for questionnaire/tabular data.\n\n4) According to the author(s), the proposed method is more interpretable than existing approaches. The proposed method could be compared with other techniques in terms of explainable artificial intelligence (XAI).\n\n5) In Figures 1 and 2, why validation errors are reported rather than test errors?\n\n6) In Figure 1 Bottom Right, what is X axis metric? What are the upper and lower bounds for this axis? \n\n7) The current format of Figure 1 Top is not informative. Furthermore, this figure illustrates that the majority of the factors in Q are close to zero - we assume that these characteristics cannot significantly affect the prediction procedure. If we apply thresholding or regularization to push them to zero, what will happen?\n", "strength_and_weaknesses": "Please refer to the review section.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "kYozuNs7IPc", "reviewer": "Reviewer_R2ij", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an algorithm to identify \"meta-factor\" from subjects' answers to multiple questionnaires with different lengths. Specific constraints are introduced and the problem is formulated a constrained optimization problem. An ADMM algorithm is derived to compute the factorization solution. Performance comparison with some existing methods have been presented and the way of using the proposed method to interpret the meta-factors extracted from the questionnaires' results was illustrated.", "summary_of_the_review": "This paper proposed a matrix factorization algorithm which can handle multiple questionnaire data matrices together with the objective to extract some meta-factor to support psychiatry research. While the paper is clearly motivated and presented from the perspective of application, the novelty from the perspective of machine learning algorithms is considered incremental.", "strength_and_weaknesses": "Strengths:\n+ The application scenario is clearly presented.\n+ The problem is clearly formulated in general.\n\nWeaknesses:\n- The key contribution of the paper is not clear. As mentioned in the paper, there are different alternatives to organize the multiple questionnaire data matrices for the factorization. It seems that performance gained in terms of ROC-AUC together with the interpretability is due to the application of constraints to factor values and reconstructed matrices. Is it not clear what happen if similar tricks are applied to the setting with the questionnaire data matrices concatenated. Also, the way to handle missing values and the use of ADMM for constrained optimisation is not new. Overall speaking, the novelty in term of algorithm is considered incremental.\n- It seems that the interpretability could be related to the control of the sparsity as discussed in Section 3.1. I noticed only that the values of p and q for controlling the sparsity are set to be 1. There seems no more investigation on sparsity afterwards.\n\nQuestion:\nIn Section 3.1, regarding continuous confound, if one wants to consider different ranges of value, will the mirroring procedure be sufficient?\n\nSome typos\n* The variables p and q are used twice to denote different concepts, sparsity control and row and column shuffling in BCV (Section 3.1). \n* The mask defined in ICQF is not the same as the mask defined in 3.3, but the same notation is used.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YFLwM9xH2_X", "reviewer": "Reviewer_V6jc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "On the context of interpreting questionnaires in psychology, this submission contributes a method for non-negative matrix factorization with additional constraints to further enhance the interpretability of factors.\n\nThe approach uses penalties on both matrices of the factorization and solves the minimization problem by alternated minimization with inner problems solved by ADMM.\n\nThe method is demonstrated on a psychology cohort dataset, looking at diagnostic performance with prediction from latent factors. ", "summary_of_the_review": "An interesting idea but that does not stand out as very different from prior efforts. ", "strength_and_weaknesses": "The latent factors seem indeed to reveal some interesting structure in the questionnaires.\n\nHowever, the method is very classic and many variants of such matrix factorization have been published. \n\nAlso, in terms of solver, the alternated minimization are very slow. Online methods are much faster, such as the original work of Mairal and later work adapting it to me complex settings (including brain imaging).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NT2JmXES-V", "reviewer": "Reviewer_dWDF", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a factorization method for clinical questionnaires by by idea of matrix completion. Given an original data matrix which stacks answers of a questionnaire from all respondents. The method searches by optimizing a regularized loss function under interpretability constraints for a pair of factor and loading matrices so that the product of the factor and loading matrix recover the original matrix. In the situation that there are multiple questionnaires, a meta-factorization is proposed in this paper by first factorizing each questionnaire, and then perform a second factorization on the concatenation of all first-step factor matrices. Numerical experiment has been performed on real questionnaires.", "summary_of_the_review": "Overall I think it is a high-quality paper. It is well written and its methodology has been carefully tested with numerical experiments and theoretical study. Of course, its technical quality builds on the abundance of literature on matrix factorization and related optimization techniques like ADMM and cross validation, so the methodological novelty is perhaps not its strongest part. However, I feel the method has been well demonstrated with interesting datasets and application, which deserves recognition", "strength_and_weaknesses": "- This is a solid paper. Merits of the proposed methodology have been well substantiated by comparing with alternative methods on data as well as simulation. The choice of cross validation seems appropriate. Theoretical guarantees for the convergence of algorithm have been provided.\n- Numerical experiments show significant advantage over existing methods.\n\nQuestions: \n- For meta-factorization, is it necessary to have equal number of participants $n$ for all questionnaires? Otherwise, it seems one can’t concatenate all factor matrices like in Eq. (6). If so, I wonder how one can relax this constraint? Because in practice, the number of participants may not be the same for each questionnaire due to non-compliance, so requiring the number of participants to be the same could be unrealistic.\n- As a follow-up to the last question about meta-factorization, I wonder if the sets of respondents have to be the same for all questionnaires? In practice, they may be different due to non-compliance or missing data. Some participants who answered questionnaire A may not answer questionnaire B. \n- About the optimization problem (ICQF), Contribution 2 mentioned: “if this number of factors is close to that underlying the data, the solution will be close to a global minimum”. Could “the number of factor of the underlying data” be more explicitly defined? In practice, I wonder which optimization problem is more non-convex, harder to reach a global minimum and more numerical unstable: the one with number of factors over-estimated or the one with number of factors under-estimated.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject