Idiographic Personality Gaussian Process for Psychological Assessment
Authors: Yehu Chen
Muchen Xi
Joshua J. Jackson
Jacob Montgomery
Roman Garnett
Venue: NeurIPS 2024 poster (2024)
Keywords: Applications -- Cognitive science
Gaussian process
Latent variable model
Primary area: machine_learning_for_social_sciences
TL;DR: We introduce an idiographic personality Gaussian process (IPGP) framework of time-series survey data for individualized psychological assessment.

Abstract: We develop a novel measurement framework based on Gaussian process coregionalization model to address a long-lasting debate in psychometrics: whether psychological features like personality share a common structure across the population or vary uniquely for individuals. We propose idiographic personality Gaussian process (IPGP), an intermediate model that accommodates both shared trait structure across individuals and "idiographic" deviations. IPGP leverages the Gaussian process coregionalization model to conceptualize responses of grouped survey batteries but adjusted to non-Gaussian ordinal data, and exploits stochastic variational inference for latent factor estimation. Using both synthetic data and a novel survey, we show that IPGP improves both prediction of actual responses and estimation of intrapersonal response patterns compared to existing benchmarks. In the survey study, IPGP also identifies unique clusters of personality taxonomies, displaying great potential in advancing individualized approaches to psychological diagnosis.

Idiographic Personality Gaussian Process for Psychological Assessment
Yehu Chen, Muchen Xi, Jacob Montgomery Joshua Jackson, Roman Garnett Washington University in St Louis
chenyehu, mm.xi, fackson,jacat ceatt//mmrryggrretttaaee
Abstract
We develop a novel measurement framework based on Gaussian process core- gionalization model to address a long-lasting debate in psychometrics: whether psychological features like personality share common structure across the pop- ulation, vary uniquely for individuals, or some combination. We propose the idiographic personality Gaussian process (IPGP) framework, an intermediate model that accommodates both shared trait structure across a population and "idiographic" deviations for individuals. TPGP leverages the Gaussian process coregionaliza- tion model to handle the grouped nature of battery responses, but adjusted to non-Gaussian ordinal data. We further exploit stochastic variational inference for efficient latent factor estimation required for idiographic modeling at scale. Using synthetic and real data, we show that IPGP improves both prediction of actual responses and estimation of individualized factor structures relative to existing benchmarks. In third study, we show that IPGP also identifies unique clusters of personality taxonomies in real-world data, displaying great potential to advance individualized approaches to psychological diagnosis and treatment.
Introduction
Building models for the evaluation of latent traits from observed responses is crucial to understand long-term behaviors through repeated quantitative assessments. These are used, for example, to study emotional stability after medical treatment or the development of academic ability during secondary education 1 However, existing frameworks face several interrelated limitations. First, there are strong reasons believe that standard taxonomies may over-generalize failing to distinguish between related psychological phenomena that often differ in etiology, symptoms, and biological processes between individuals le.g., 1 4 and this may lead to inaccuracy when making predictions [5). A related issue is that measurement models are rarely individualized, instead assuming that (1) the correlation between latent traits of interest and survey responses are invariant across individuals and (2) the relationship between the latent traits themselves are the same for everyone. Lastly, current models are almost always developed for cross-sectional data that are collected only once from each respondent, which overlooks any potential dynamics of psychological processes.
To address these limitations, previous research has adopted three different approaches, each inadequate in its own way. First, recent work has proposed an idiographic approach that builds completely distinct taxonomy for each individual [6-8) However, complete personalization can sacrifice generalizability and interpretability for clinicians, as any possible population commonality is ignored. second line of research focuses on building dynamic psychometric models of time-series data some variant of item response theory 19, [0 3), longitudinal structural equation modeling (ННН3) vectorized autoregression [14] [5] and/or Gaussian process (GP) latent trajectories [16]18]. However, all these models adopt the nomothetic approach, assuming that the responses of all individuals
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

share an identical latent structure. Finally, there is a smaller body of work that adopts intermediate approaches to create individualization while maintaining group commonality [e.g.. 19]. However, prior research models quantitative responses directly, ignoring the latent structures that are often the actual focus of domain researchers.
In this work, we propose an idiographic personality Gaussian process (IPGP) framework for assess- ing dynamic psychological taxonomies from time-series survey data. This framework combines nomothetic and idiographic approaches by employing a common structure to explain typical circum- stances, while allowing individual structures accommodate deviations into distinct forms. We leverage the Gaussian process coregionalization model based on multi task kernels to conceptualize responses of grouped survey batteries, adjusted non- non-Gaussian ordinal data, and utilize IPGP for hypothesis testing of domain theories. Methodologically, our approach combines Gaussian process latent variable models (GPLVM) 20], Gaussian process dynamic systems (GPDM) [7 18 and GP ordinal regression for Likert-type survey data [21 22). Despite involving latent variables and GPS, IPGP differs from GPLVM in two ways: (1) it optimizes the factor loading matrix while marginalizing the latent variables, and (2) it accommodates categorical data using non-Gaussian ordered probit likelihood. Computationally, our framework exploits stochastic variational inference for latent factor estimation, contrasting with other GP measurement models relying on Gibbs sampling that may not scale efficiently to intensive longitudinal setups [18 23
To our knowledge, our work presents the first multi-task Gaussian process latent variable model for dynamic idiographic assessment. While multi-task GPs have found recent applications areas such as causal inference (6), environmental science 22] and biomedical research 30,, their potential remains largely unexplored in psychology. Current psychometric models typically focus on cross-sectional settings without dynamics 6 31 g or single task settings that ignore inter-battery
correlations [32 33). We conducted an extensive simulation study comparing IPGP against benchmark methods and analyzed an existing cross-sectional personality dataset. Our results demonstrate that IPGP simultaneously enhances the estimation of idiographic taxonomies and improves the prediction of responses. Additionally, we collected novel IRp-appe longitudinal dataset. When applied to this data, IPGP not only shows superior performance in response prediction, but also suggests unique personality taxonomies. These findings highlight IPGP's significant potential for advancing individualized approaches to psychological diagnosis and treatment.
2
Background
We start by laying out the ordinal factor model for building standard taxonomy from survey data We then briefly discuss several existing idiographic longitudinal models in psychological assessment and review the Gaussian process model.
34 35
Ordinal factor analysis. Consider the scenario where some set of units, E (1...., N). repeatedly answer the same set of j E fi..... survey items over t E {I...., periods with ordinal observations Vigt E {l,..., C'} up to C' levels. For example, the responses could be Likert-typed, ranging from "strongly disagree" to "strongly agree". The latent factor model posits that the jth
underlying latent variable 14 (1) for unit at time / can be recovered as wf x(),, where x;(() E R
are unit-level latent factors and W, E R. are factor loadings. The 1,S (()s are then mapped to ordinal responses via an ordered probit model: p(ligt c 1 (1) (1)=11 f) (bb. f) -/b- (b. f) with threshold parameters by < <... < bc. Usually bo and be are fixed to -oo and +oo so that the resulting categorical probability vector sums to 1, while by....,bcs bb- are allowed to move freely. Stacking (t)s, w,s and Yijt's 's into matrices x, w and tensor y, the joint likelihood can be written as C(y w) = II, 11, II. Pl (Vaj (t),), W Identification is guaranteed with standard orthogonality and normalization constraints (36). This factor model is also known as an item response model [37 38) and can be estimated via maximum likelihood, weighted least squares, or an EM
algorithm 33 41
Idiographic longitudinal assessment. In psychological assessment, the idiographic approach emphasizes intrupersonal variation by requiring distinct loadings, while the nomothetic approach identifies general interpersonal variation assuming shared factor loadings [42]. terms of data
collection, the idiographic approach usually surveys each individual multiple times (N = 1 and large

---

K pop
W pop
f()
D
Factors (1)
J
J
U
Factor Loadings
Cov Matrix
T
1
Ordinal Model
K
Z
(8)
K time
RBF
D
J
T
Figure 1: Proposed IPGP model for inferring latent factors and factor loadings from dynamic ordinal data. Input ordinal observations across indicators are modeled as ordinal transformations of latent dynamic Gaussian processes with individualized RBF kernels and loading matrices.
T) for learning personalized taxonomy rather than many individuals at single shot (large N and T= To extract individualized dynamics from time-serics data, recent psychometric models have utilized longitudinal structural equations by explicitly specifying any intrapersonal and temporal dynamics. However, these typically require strong model-based assumptions from domain-theory about this dynamic process, and may be sensitive model misspecification [I] [3]. Meanwhile, hierarchical vector autoregression models may automatically learn individual trajectories over time, but are designed to model observed responses directly rather than latent traits of interest domain scholars [14 15).
Gaussian process. A Gaussian process (GP) can be used to define distribution over such that the values of at arbitrary finite subset of X have a joint multivariate Gaussian 43 A GP(u, ) is specified with mean function μ:: X' R and a positive-defintt kernel function K: X' x X' + R; evaluating these functions pointwise determines the mean and covariance of these dimensoammnsional distributions. The most common kernel is the squared exponential (RBF) kernel K (x1.X2) = exp (- (-}x1 Px2) with precision matrix P = diag(1/ff... 1/23) and d card(X'). The posterior of GP can be derived analytically for Gaussian likelihood but must be approximated in modeling latent variables with categorical indicators. We discuss the variational approximation method we use for inference in Sec.
Methodology
We propose an idiographic personality Gaussian process (IPGP) framework for assessing individual- ized dynamic psychological taxonomies from time-series survey data. Instead of joint estimation of latent factors and their loadings that cannot guarantee rotational and scaling invariance, we marginal- ize out the latent variables and focus on learning taxonomies of loadings. The overall architecture of IPGP illustrated in Figure (I), where the ordinal input observations across the indicators are modeled as ordinal transformations of latent dynamic GP with individualized RBF kernels and loading matrices.
3.1 Multi-task learning
Typically in psychological assessment, survey questions are meticulously grouped such that each group gauges particular latent trait (e.g. dimension personality). Hence, we conceptualize assessment of psychological traits multi-task learning problem, where each question represents distinct task can be correlated with other tasks. A multi-task is an extension of the single-task GP but for valued functions [31. To motivate the multi-task framework, first consider the

---

two-task scenario with two T x 1 vector f1 (4) and f2" ( denoting the latent temporal processes of unit i for question j = 2. To fix the scale of latent factors, time-level Gaussian process prior is placed on (1) ~ GP(O, K (4)
im ). By exploiting affine property of Gaussians, the induced joint distribution
of vectorized [f (0) f2 " can be written
f1 (+)
w W
K (t) Wi
w| W. K (i) w2 W2
GP
f2 (
K (P) W
(4) time
time
whose covariance of shape 27' x 2T contains four block matrices K Kine scaled by different w" wy
G.'' E (1.2)). Specifically, w/ Wa controls the inter-task covariance between these two tasks and Wj wys (j E (1,2}) control their intra-task variance. This multi-task structure is also known as the linear model of coregionalization (LMC) [4)), which models output functions as linear combi- nations of several independent latent processes. In our case, each dimension in (t( represents one latent process, which jointly defines the functions as f2()) w ((). To extend this, let
p(0) = (6) ( fy ( represents the flattened JT x 1 vector consisting of all J tasks. We write f(0) in a formal multi-tase task GP notation using Kronecker product 8:
......,f
GP K
K
(i)
(2)
task
time
W pop
W
pop + Z ( Z
where K usk denotes the unit- task covariance matrix, consisting of the self inner products of x shared interpersonal loading Weep and D unit-specific low-rank ( (D < D) for intrapersonal deviations that serves to be the additional idiographic component, independent of W pop In our experiments, we found degraded performance for D' but extra computational costs so we focused on D 1. The Kronecker product @ then multiplies each entry in the J x J task covariance with K!# lime. and returns the stacked JT x JT covariance for p(() Through the use of time kernel (i) time properties of the latent trait trends such periodicity autocorrelation could be incorporated. Here we use the common RBF kernel K time (1) = exp ( - [L - (())/(+) to account for dynamic changes in the latent attributes, whose bandwidth is determined by the unit-specific length scale L. fit. Finally, the latent variables f()ss are further projected to response space by the ordered probit model.
but any other kernel can substitute RBF practitioners
3.2 Variational inference
Due to the non-Gaussian ordinal likelihood, we adopt the variational inference technique (VI) with inducing points introduced in 33 Dropping superscript for demonstration, VT utilizes a variational distribution g(u) = - N( ),) on inducing variables u to approximate p(f yy) using the conditional p(f u). Hence, the conditional log- lig-liillhhod log p(y u) can be lower bounded by the expected logllikeldhood w.r.t. p(f u), after exploiting the non-negativity of Kullback-Leblerr (KL) divergence between pf u) and p(f y):
log u) 2 Eptrjun log p(y f
4
Furthermore, a lower bound on model evidence (ELBO) can be obtained by combining Eg. 44 and an
inequality derived by another KL divergence KL(g(u) || p(u y)] 2 0 (see Appendix B for details):
logply)
2 Ey(w) ply u)] KL(g(u) || p[u)]
2 E,(r) logp(y f)] KL(g(u) || p(u)]
(6)
where the KL divergence KL(g(u) p(u) between the variational quu) and prior p(u) can be computed in closed form as both distributions are Gaussians. The expectation of log-likelihood log ply | f) under the marginal distribution (f) J' u)q(u)du is intractable but can be numerically approximated using Gauss-Hermite quadrature method. The variational parameters yn and En, individualized loadings w, and diag(v) as well as likelihood parameters (be]s are then optimized to maximize this lower bound. Finally, the predictive likelihood of new p(y*) = p(y" f"'))((' u)q" (u)du is obtained by marginalizing out the optimized q" (u). Throughout experiments, adopt stochastic inference for computational scalability.

---

Table 1: Comparison averaged accuracy. log-likelihood and correlation matrix distance between IPGP. baselines, and ablated models in the simulation study. The full IPGP model (indicated in bold) significandly outperforms ablated and baseline methods. Results from ablations imply that IPGP succeeds in predicting the correct labels due its idiographic components and proper likelihood, and well-informed population kernel is crucial in recovering the factor loadings. " "-" indicates baseline software that cannot handle missing values.
TRAIN ACC 1
TRAIN LL f
TEST ACC 1
TEST LL 1
CMD +
GRM
0.261 + 0.005
-3.556 + 0.092
0.266 + 0.006
-3.578 + 0.098
0.657 + 0.021
GPCM
0.562 + 0.017
-2.067 + 0.182
0.495 + 0.012
-2.409 - 0.143
0.545 0.016
SRM
0.286 + 0.006
-7.408 + 0.063
0.289 + 0.008
-7.341 + 0.084
0.300 0.024
GPDM
0.687 1 0.010
-4.358 1 0.028
0.667 1 0.010
-4.377 + 0.029
0.262 1 0.016
LSM
0.539 1 0.021
-0.961 + 0.015
1.266100011
TVAR
0.554 1 0.018
1..68 1 0.014
0.987 1 0.011
IPGP-NOM
0.807 0.007
-0.535 + 0.015
0.790
0.008
-0.555
0.017
0.257 + 0.009
IPGP-IND
0.932 + 0.003
-02443 = 0.008
0,916 - 0.004
-02667
+ 0.009
0.530 0.005
IPGP-LOW
0.897 + 0.004
-0.313 # 0.010
0.884 + 0.005
0.331
+ 0.011
0.397 0.007
IPGP-NP
0.898 + 0.003
0..18 + 0.009
0.883 + 0.005
0.312
+ 0.011
0.467 0.010
IPGP
0.957 + 0.002
0.159 + 0.005
0.942 + 0.002
-0.184 + 0.006
0.128 = 0.006
3.3
Theory testing
Our IPGP framework also naturally facilitates downstream tasks such as domain theory testing between models with and without shared or idiographic components. We adopt posterior odds ratio test, using posterior p(M; y) = plyl M.),(M...
plylyyMMM))) P.Y.)) over a pool of models [M:}} conditioning on observations y with prior weights p(.M,), as the hypothesis test determine whether the latent structures for each individual are indeed distinct or are simply explainable by interpersonal commonality. Specifically, refer the multi-task model in Eq. (3) as the idiographic model, and compare it with an nomothetic model without unit-specific components:
task = W
pop WT pop
Note that compared to this baseline nomothetic model, our proposed idiographic model in Eq. introduces additional unit-level JN loading parameters that enlarge the hyperparameter optimization space. Hence, we propose to first learn the interpersonal loading matrix W pop using the standard cross-scctional data from a nomothetic model that focuses on learning of population taxonomy, and then use the estimated W pop as informative prior in the full model. We will show empirically in Sec. (4) that with this stronger prior IPGP achieves more precise estimation of individual taxonomics.
Experiments
We now evaluate TPGP in learning idiographic latent taxonomies and predicting actual responses against baseline methods from both psychometrics and Gaussian process literature in three experi- ments: simulation study, a re-analysis of a large cross-sectional dataset, and a pilot study of repeated measures of the Big Five [55 personality traits.
4.1 Simulation and ablation
Setup. Our simulation considers longitudinal data of N 10 units over 30 periods. We assume that the latent traits of each unit i have dimension D = 5. and each latent vector is generated independently from a x; (d) (1) GP K t with a unit-specific length scale uniformly ran-
(() E [10, 20, 30). We split J 20 items into D subsets of size J/D 4, such domly picked from time
that each subset dominates one dimensional in the latent traits. Specifically, we set high value of in the population factor loading matrix W pop for entries corresponding to the kth subset for dimension k, and low values drawn from Unif otherwise. We also set each unit-specific loading w, from 1, To introduce sparsity and reverse coding, we randomly set half of the loadings to zero and invert the signs of the remaining half. Finally, we generate the VijtS according to the ordered probit model with C = 5 levels, and apply 80% 50%% splitting for training and testing.
5

---

Table 2: Model comparison where the model rank varies from S 8 while the true rank is 5. The best models are indicated in bold, and models that are not significantly worse than the best model are indicated in italics.
RANK
TRAIN ACC f
TRAIN LL f
TEST ACC f
TEST LL
CMD +
(LOWER)
0.897 0.004
-0.313 = 0.010
0.884 0.005
-0.334 + 0.011
0.397 + 0.007
5 (TRUE)
0.957 = 0.002
-0.159 = 0.005
0.942 = 0.002
-0.184 = 0.006
0.128 + 0.006
8 (HIGHER)
0.957 = 0.002
-0.161 = 0.004
0.945 + 0.002
-0.183 = 0.005
0.124 = 0.006
Metrics and baselines. We consider two sets of metries for evaluation: (1) the in-sample and out-of-sample predictive accuracy (ACC) and log-likelihood (LL) of the actual responses, (2) the correlation matrix distance (CMD) between the estimated factor loading matrix and the true ones,
which is defined for two covariance matrices R1, R₂ as d(R) R2) = = [R. | [46] where f
is the Frobenius norm. Note that CMD becomes zero if R1, Rg are equal up to a scaling factor, and one if they are orthogonal after flattening. We compare IPGP (1) various latent variable models for ordinal responses, including the graded response model (GRM) [37. the generalized partial credit model (GPCM) [47] and the sequential response model (SRM)
48 (2) Gaussian process dynamic model (GPDM) 17 18 where the continuous predictions are rounded to the nearest ordinal level, with trait-dependent latent variables and (4) time-varying vector autoregression (TVAR) with regularized kernel smoothing [15]. We also compare IPGP with several ablated models: (1) IPGP-NOM without the idiographic kernel, (2) IPGP-IND without the population kernel, (3) IPGP-LOW with lower-rank factors of than actual rank of 5 in the synthetic setup and (4) IPGP-NP where the population kernel is learned from scratch rather than fixed to the informative prior. Note that W pop in the full IPGP model is fixed as learned from IPGP-NOM.
(3) latent structural model (LSM) 13 49]
Results. We use 100 inducing points and the ADAM optimizer with learning rate 0.05 to optimize ELBO for 10 epoches with batch size of 256. We repeat our simulation with 25 different random seeds using 300 cores on Intel Xeon 2680 CPUs. Table shows the comparison of the average predictive accuracy, log-likelihood, and correlation matrix distance between IPGP and baselines and ablated models. Our TPGP model (indicated in bold) significantly outperforms all ablated models and baseline methods in estimated correlation matrix, predictive accuracy, and log-likelihood of both training and testing sets in paired-t tests. We found that IPGP is able to predict the correct labels due to its idiographic components and proper likelihood, since IPGP-NOM and IPGP-GL are two of the worst ablations for all prediction metrics. In addition, IPGP-IND and IPGP-NP have the worst correlation matrix estimation, implying that a well-informer population kernel is crucial in recovering the underlying factor structures.
Robustness of IPGP. To assess IPGP's robustness to rank misspecification, we conducted additional
exploratory factor analysis using our simulated data. We tested model performance with ranks of 5, and 8, where 5 represents the true rank of the data. As shown in Table both the true-rank (5) and higher-rank (8) models significantly outperform the low-rank (2) model. However, increasing the rank beyond the true rank of 5 yields no additional performance benefits: the high-rank model is not significantly better than the truc-rank model in a paired t-test. These results demonstrate two key points: first, low-rank approximations inherently lack the capacity to fully capture the underlying structure, and second, increasing the rank beyond the true rank provides no additional benefit. This underscores the importance of careful exploratory analysis in practical applications to determine the appropriate rank.
4.2 Cross-sectional factor analysis
We next validate the popular Big Five personality theory using standard cross-sectional data via factor analysis, where a range of factors are tested and then compared according to model evidence. This serves to show that the model works appropriately to detect known latent traits even in non- dynamic settings, and to validate the informative prior for the W pop matrix in our next experiment. We utilize an existing dataset called life outcomes of personality replication (LOOPR) (5), which collected from 5,347 unique participants on the Big Five Inventory (51) consisting of 60 questions.
Our validation considers a range of latent trait dimension counts from D = 1,...,5. 5. For each
dimension count, we first apply principal component analysis (PCA) directly on the correlation matrix

---

Figure 2: Illustration of raw correlation matrix (left) our estimated Big loading matrix (right). Both correlation matrices display block pattern, where estimated interpersonal variation show strong correlation between questions within the same factor the Big Five personalities and weak correlation across different factors. Besides, questions corresponding negative emotionality show minor negative correlation with those corresponding to extraversion and conscientiousness, suggesting trait- sait-by-tre by-trait interaction effects.
Table 3: In-sample accuracy and averaged log lik of method and haselines for various ranks 1) in L.OPR. Best model for each D is indicated in bold and the best model across different Ds is further indicated in italic.
ACC f
LL/Nt
MODEL
D
3
4
D=5
D=1
D=2
D=4
5
PCA
0.106
0.099
0.123
0.217
0.192
-1955
1.990
-2000
-2033
-2.551
0.238
0.107
0.178
0.113
0.146
-1.838
-1.832
-1.814
-1.838
-1.841
0.213
0.156
0.186
0.159
0.163
-1.754
-1.761
-1.764
-1.750
-1.756
SRM
0.243
0.134
0.179
0.125
0.155
-1784
-1784
-1788
-17880
-1.667
GPDM
0.268
0.272
0.266
0.268
0.263
-2.155
-21558
-21558
-2159
-21558
LSM
0.188
0.114
0.110
0.105
0.104
-1997
-1960
-1900
-.845
-1.775
IPGP
0.322
0.319
0.323
0.318
0.318
-1478
-1477
-1477
11477
-1.766
of the cross-sectional observations to learn vanilla population factor loading matrix. We then initialize W Wpep our model with this vanilla loading matrix, and optimize the loading matrix jointly with the variational parameters. Note that T = 1 in LOOPR, so we drop the idiographic components.
Validation of Big Five. TableBpresents the predictive accuracy and averaged log-likelinne ligllikel for our method and various baselines (excluding TVAR due to lack of low-rank assumption) across different values of D in LOOPR. Bold numbers indicate the best model for each D, while italic numbers highlight the best model across all D values. Although IPGP with = 5 shows slightly lower in-sample predictive accuracy compared to the D = model, it demonstrates significantly stronger model evidence than all alternatives. Posterior odds ratio test reveals that the second-best X 10 times less likely than the five-factor model.
model is exp(-79)
We further evaluated IPGP's performance through exploratory analysis, testing model ranks from 1 to 10. Results shown in Table provide strong support for rank-5 model, which achieves both higher model evidence and lower BIC, strengthening the evidence for the Big Five theory. The BIC follows V-shaped pattern, decreasing as the rank approaches 5 and increasing thereafter, indicating that rank-5 represents an optimal balance point: ranks below provide insufficient model capacity, while higher ranks lead to overfitting. These findings demonstrate that when analyzing psychological measurements from standard cross-sectional data, IPGP successfully identifies the correct underlying factor structure, making it valuable for downstream applications.

---

Table Model performance IPGP with model ranks from 10 in LOOPR data.
RANK
2
3
4
LL/N T
-1478
1.477
1.477
1.477
1.476
1.477
-1447
-1.477
-1448
1.477
BIC (x10")
1.2736
1.2726
1.2728
1,2726
1.2722
1,2725
1.2726
1.2732
1.2732
1.2724
2.000
0.58
0.27
2.09
6015
2.000
-0015
40331
-2062
D006
2.000
5:224
0.311
0.23
-0.03
0.000
3.543
-0.597
-0.057
-0:20
5.106
2.077
0/50
0.000
-3.085
5,50
0.200
2.005
6.115
0.200
-3.397
0.000
0.77
8.089
0.82
2.000
0.94
22
-2.002
5.00
0.500
3.000
4.066
2.009
+0150
0.22
3.000
0/06
-0.057
+4205
0.177
0.000
6.181
8.055
4000
0.42
2.094
0,00
2.066
4.355
4.144
-1.66
0,00
-2.85
4.884
40181
3.066
6/000
-0.256
0.00
0.18L
6.00
E
A
A
8
A
8
A
Figure 3: Four residual correlations identified by k-mean clustering. Each heatmap displays the trait-level residual correlation averaged across corresponding batteries for one cluster, with darker red and blue indicating larger positive and negative deviations. For instance, agrceableness (A) is more correlated to extraversion (E) than the population profile in the first profile, hut less correlated to openness (O) in the second profile. Moreover, these directions of deviations are even exacerbated in the third and fourth profiles.
Estimated interpersonal variation. Figure 2 compares the raw correlation matrix with our esti- mated Big Five correlation matrix. Both matrices exhibit distinctive block pattern, characterized by strong correlations between questions within each Big Five factor and weak correlations across different factors. We also observe that questions related to negative emotionality demonstrate slight negative correlations with questions measuring extraversion and conscientiousness, suggesting the presence of meaningful trait-by-trait interaction effects.
4.3 Longnitudinal Pilot Study
To further demonstrate IPGP in longitudinal setting for learning idiographic psychological taxonomies, we collected intensive longitudinal data using experience sampling measures (ESM). We highlight the predictive ability of IPGP through a prediction and a leave-one-trait-ou cross-validation task. We also illustrate how IPGP identifies unique personality taxonomies that might advance individualized approaches to psychological diagnosis and inspire new theory.
Data collection. We employed an experience sampling method (ESM) design where participants completed personality assessments six times daily over three-week period, allowing for a maximum of 126 assessments per person. Our study included 93 valid student participants, yielding 8,770 total assessments with an average of 94 assessments per participant. We based our assessment the BFI-2 [52], which provides comprehensive coverage of the trait space and ensures proper identification of latent factors. While the original BFI-2 contains 60 items (four items for each of the three sub-factors within each Big-Five domain), we modified it for our ESM design by removing items unsuitable for contextualized assessment. To reduce participant fatigue and minimize learning effects from repeated measures, we implemented planned missing design: participants randomly responded to two out of three items for each sub-factor, resulting in a streamlined 30-item assessment. Note that our data is collected from student sub-population non-representative samples, and future studies may explore the model's applicability across diverse populations.
Comparison between nomothetic and idiographic models. We run the full IPGP model with idiographic component and unit-specific time kernel on the collected longitudinal data. Again we set the ranks of the population and individual loading matrices to and respectively, and incorporate prior knowledge of the cross-sectional data by fixing the population loadings as the Big Five loadings estimated Sec. (4.2) and optimizing the individual loadings. We contrast our proposed idiographic model (TPGP) and baselines in Table which shows the in-sample prediction, averaged log- likeeiidddodd and posterior odds ratio. We found that IPGP outperforms IPGP-NOM with higher predictive accuracy and log- regllikllihood, and is decisively favored by a posterior odds ratio of exp(1.06 x 10*).
8

---

Table 5: In-sample prediction and averaged log- likelihood of our proposed model (IPGP) and baselines for the longitudinal data, well as log posterior odds ratios to JPGP. indicates self comparison.
prediclive ace
0.25
0.15
ACC
LL/N
log(OR)
INGI-NON
predictivn log
IPGP
GRM
0.210
-.2666
2.32 X 106
-14
-1.7
GPCM
0.288
-1.516
-3.80 X 104
4.8
SRM
0.260
-1.927
-1.44 x 10
GPDM
0.382
-3.865
-7.80 x 109
4.9
LSM
0.226
-1.999
-7.72 103
forecast dayls)
leave
TVAR
0,382
-1.466
-4.47 X 104
Figure 4: Predictive accuracy and log lik of IPGP and IPGP-NOM for the forecasting task and leave- one-trait-out cross-validation task.
IPGP-NOM
0,403
-1.100
-1.06 X 104
IPGP
0.417
-13669
Predictive performance of IPGP. We also evaluate the out-of-sample performance of the id- iographic and nomothetic models using two prediction tasks: forecasting future responses and sevv--one-trit-out cross validation. For the forecasting task, we train both models with data from the first 40 days and predict future responses for the last 5 days. For the cross-vale coss-validation task, predict responses of each trait by training on data belonging to the other four traits, where 20% of responses for one trait was held out (randomly choosing the trait and items remove). Figure (4) shows the predictive accuracy and log-likelihood varying horizons and for the leave-one- ien-trrit-out cross
IPGP and IPGP-NOM for the forecasting task cross-validates validation task. IPGP has consistently better performance than IPGP-NOM in both tasks, except for being slightly less accurate in predicting extraversion. Overall, IPGP is favored than IPGP-NOM by posterior odds ratios of exp(43) and exp(716) in these two tasks.
Discovery of taxonomies. Despite our small cohort size (93 respondents), we also explore how the profiles of personality that substantially differ from the interpersonal commonality cluster into informative groups. Specifically, we first perform a k-mean clustering using 93 estimated individ- ual correlation matrix with CMD as the distance metric, and then compute the residual correlation between each estimated clustering centroid and the population correlation. Figure (3) illustrates four residual correlations identified by our k-mean clustering. Each heatmap displays the residual correlation at the trait level averaged between the corresponding batteries for one cluster, with darker red and blue indicating larger positive and negative deviations. For instance, agrecableness (A) is more correlated to extraversion (E) than population profile the first profile, but less correlated to openness (O) in the second profile. Moreover, these two directions of deviations are even exacerbated in the third and fourth profiles.
The discovery of unique personality taxonomies suggests a potential resolution to the longstanding idiographic versus nomothetic debate personality and psychometric sciences. Our findings indicate that the optimal approach lies between these two extremes, rather than fully embracing either perspective. The four distinct profiles that we identified, while derived from the Big Five framework, demonstrate how individuals can meaningfully deviate from a common taxonomy. These deviations may provide valuable insights into individuals' motivations, behavioral patterns, and self-concepts. For example, individuals matching Profile 4 show strong correlation between Extraversion and Agreeableness, possibly reflecting a tendency toward warm and socially engaging behavior (such someone who naturally connects with and shows kindness everyone social gatherings). Furthermore, these distinct profiles enhance the predictive power of individual- (N 1) models by allowing them to learn from people with similar characteristic patterns.
5 Related work
Idiographic assessment captures critical individual characteristics that are often lost in simplified tax- onomies psychological behaviors [53]. Evidence from multiple psychometric fields demonstrates that nomothetic models, which focus solely interpersonal variation, often lack generalizability

---

[]. Researchers have proposed various solutions to address this limitation. Song and Ferrer [54 enhanced dynamic factor models with random effects to analyze psychological processes. Jongerling et al. (55] developed multilevel first-order autoregressive model incorporating random intercepts to track daily positive effects across weeks. Beltz. al. [19] bridged nomothetic and idiographic approaches by introducing individual components to the group iterative multiple model (GIMEE) for clinical data analysis. However, these methods share a common limitation: they model in response space rather than latent space when handling ordinal survey data.
Gaussian process latent variable model (GPLVM) is a dimensional reduction method for Gaussian data, where the latent variables are optimized after integrating out the function mappings [20 [6]. Our proposed framework differs from GPLVM as we optimize the factor loading matrix while marginalizing the latent variables. In addition, our model contrasts GPLVM and (variational) Gaussian process dynamical model (GPDM) [6 17 in our non-Gaussian ordered logistic observation model and the use of multi-task kernel for latent structure. Finally, our longitudinal framework with stochastic variational inference learning differs from the static GP item response theory (GPIRT) 23
with more
computationally demanding Gibbs sampling.
Longitudinal measurement models have gained prominence as researchers increasingly incorporate temporal dynamics into psychological theories through longitudinal survey designs [57] [88. This development has spawned various methodological approaches. One family of methods includes longi- tudinal structural equation models (SEM), such as multiple-group longitudinal SEM and longitudinal growth curve models, designed for repeated measurement studies [T. The Mplus software later extended these capabilities by implementing dynamic SEM with Bayesian Gibbs sampling [3] 49). Another stream of research produced dynamic item response models (9]
I0 3
and time-varying
vector autoregressive models [14 [:3] to track latent trait trajectories. While multi-task Gaussian process time series have been successfully applied to Gaussian observations in behavioral research 18] and found applications in causal inference (24] 26), environmental science (28), and biomedical research [30], they remain unexplored for survey batteries with non-Gaussian likelihood where exact inference is not possible.
6 Conclusion
We introduce the idiographic personality Gaussian process (IPGP) model, novel approach for personalized psychological assessment that learns intrapersonal taxonomies from longitudinal ordinal survey data, configuration that remains underexplored in Gaussian process dynamic systems literature. Our model leverages Gaussian process coregionalization to capture the between-battery structure and employs stochastic variational inference to ensure scalable computation. Looking ahead, we envision extending TPGP to other psychological domains, such as emotion research, and enhancing it by incorporating contextual information about behaviors and activities.
Our proposed IPGP framework also provides insight for domain theory testing, contributing to the substantive debate in psychometrics surrounding the shared versus unique structures of psychological features. Our experimental results show that IPGP decisively favored over the nomothetic baseline, and substantive deviations from the common trend persist in for many individuals.
Acknowledgments and Disclosure of Funding
This work supported by the 2023 Seed Grant of Transdisciplinary Institute in Applied Data Sciences at Washington University in St Louis. YC and RG were supported by the National Science Foundation (NSF) under award number IIS 1845444.
References
[1] Peter CM Molenaar. A Manifesto on Psychology as Idiographic Science: Bringing the Person Back Into Scientific Psychology, This Time Forever. Measurement, 2(4):201-218, 2004.
[2] Xiaojing Wang, James Berger, and Donald Burdick. Bayesian Analysis of Dynamic Item Response Models in Educational Testing. The Annals of Applied Statistics, 7(1):126-153, 2013,
10

---

[3] Denis Dumas, Daniel McNeish, and Jeffrey A Greene. Dynamic Measurement: A Theoreti- cal Pel-Psyccometicc Paradigm for Modern Educational Psychology. Educational Psychologist, 55 (2):88-105, 2020.
[4] Denny Borsboom, Gideon Mellenbergh, and Jaap Van Heerden. The Theoretical Status of Latent Variables. Psychological Review, 1 110(2):003-111 2003.
[5] Pengda Wang, Hwayeon Myeong, and Frederick L Oswald. On Putting the Horse (Raters and Criteria) Before the Cart (Variance Components in Ratings). Industrial and Organizational Psychology, 17(3):309-33 2024.
[6] Peter Borkenau and Fritz Ostendorf. The Big Five as States: How Useful Is the Five-Factor Model to Describe Intraindividual Variations over Time? Journal of Research in Personality, 32(2):202-221, 1998.
[7] Emorie D Beck and Joshua Jackson. Consistency and Change in Idiographic Personality: A Longitudinal ESM Network Study, Journal of Personality and Social Psychology, 118(5): 1080-1 100, 2020.
[8] Emorie Beck and Joshua Jackson. Within-person Variability. In The Handbook of Personality Dynamics and Processes, pages 75-100. Elsevier, 2021.
[9] Frank Rijmen, Francis Tuerlinckx, Paul De Boeck, and Peter Kuppens. A Nonlinear Mixed Model Framework for Item Response Theory. Psychological Methods, 8(2):185-205, 2003.
[10] Steven P Reise and Niels G Waller. Item Response Theory and Clinical Measurement. Anmual Review of Clinical Psychology, 5:27-48, 2009.
[11] Todd D Little. Longitudinal Structural Equation Modeling. Guilford Press, 2013.
[12] Eun Sook Kim and Victor L. Willson. Testing Measurement Invariance Across Groups in Longitudinal Data: Multigroup Second-Order Latent Growth Model. Structural Equation Modeling: A Multidisciplinary Journal, 21 21(4):566-5) -7656666-766666 2014.
[13] Tihomir Asparouhov, Ellen L Hamaker, and Bengt Muthén. Dynamic Structural Equation Models. Structural Equation Modeling: Multidisciplinary Journal, 25(3):359-388 -88, 2018
[14] Feihan Lu, Yao Zheng, Harrington Cleveland, Chris Burton, and David Madigan. Bayesian Hierarchical Vector Autoregressive Models for Patient-level Predictive Modeling. PloS One, 13 (12):c0208082, 2018.
[15] Jonas MB Haslbeck, Laura F Bringmann, and Lourens Waldorp. A Tutorial on Estimating Time-Varying Vector Autoregressive Models. Multivariate Behavioral Research, 56(1):120-149, 2021.
[16] Jack Wang, Aaron Hertzmann, and David J Fleet. Gaussian Process Dynamical Models. In Advances in Neurul Information Processing Systems, volume 18, pages 1441-1448, 2005.
[17] Andreas Damianou, Michalis Titsias, and Neil Lawrence. Variational Gaussian Process Dynam- ical Systems. Advances in Neural Information Processing Systems, 24:25 10-2588,, 2011.
[18] Robert Dürichen, Marco AF Pimentel, Lei Clifton, Achim Schweikard, and David A Clifton. Multitask Gaussian Processes for Multivariate Physiological Time-Series Analysis. IEEE Transactions on Biomedical Engineering, 62(1):314 322, 2014.
[19] Adriene M. Beltz, Aidan G. C. Wright, Briana N. Sprague, and Peter C. M. Molenaar. Bridging the Nomothetic and Idiographic Approaches to the Analysis of Clinical Data. Assessment, 23 (4):447-458, 2016.
[20] Neil Lawrence. Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data. Advances in Neural Information Processing Systems, 16:329-336, 2003.
[21] James T Croasmun and Lee Ostrom, Using Likert-Type Scales in the Social Sciences. Journal of Adult Education, 40(1):19-22, 2011.

---

[22] Wei Chu and Zoubin Ghahramani. Gaussian Processes for Ordinal Regression. Journal of Machine Learning Research, 6(35):1019-104) 2005.
[23] JBrandon Mayk-Mayr, Roman Garnett, and Jacob Montgomery, GPIRT: A Gaussian Process Model for Item Response Theory. Conference on Uncertainty in Artificial Intelligence, pages 520-529. PMLR, 2020.
[24] Ahmed Alaa and Mihaela van der Schaar. Bayesian Inference of Individualized Treatment Effects Using Multi-task Gaussian Processes, In Advances in Neural Information Processing Systems, volume 30, pages 3424-3432, 2017.
[25] Virginia Aglietti, Theodoros Damoulas, Mauricio Álvarez, and Javier González. Multi-task Causal Learning with Gaussian Processes. Advances in Neural Information Processing Systems, 33:6293 6304, 2020.
[26] Yehu Chen, Annamaria Prati, Jacob Montgomery, and Roman Garnett. A Multi- Task Gaussian Process Model for Inferring Time- Varein Treatment Effects Panel Data. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, pages 4068-4088. PMLR, 2023.
[27] Astrid Dahl and Edwin V Bonilla. Grouped Gaussian Processes for Solar Power Prediction. Machine Learning, 108(8):1277-1306 2019.
[28] Yong Zhou, Yanfeng Liu, Dengjia Wang, Gejirifu De, Yong Li, Xiaojun Liu, and Yingying Wang. A Novel Combined Multi-task Learning and Gaussian Process Regression Model for the Prediction of Production, 284:124710, 2021.
and culti-component of Solar Radiation. Journal of Cleaner
[29] Yangtao Li, Tengfei Bao, Zexun Chen, Zhixin Gao, Xiaosong Shu, and Kang Zhang. A Missing Sensor Measurement Data Reconstruction Framework Powered by Multi-task Gaussian Process Regression for Dam Structural Health Monitoring Systems. Measurement, 1 186: 10085, 2021.
[30] Kai Zhang, Siddharth Karanth, Bela Patel, Robert Murphy, and Xiaoqian Jiang. A Multi-task Gaussian Process Self-attention Neural Network for Real-time Prediction of the Need for Mechanical Ventilators in COVID-19 Patients. Journal of Biomedical Informatics, 130:104079, 2022.
[31] Edwin V Bonilla, Kian Chai, and Christopher Williams, Multi-task Gaussian Process Prediction. Advances in Neural Information Processing Systems, 20:153-160 2008.
[32] Edward Snelson and Zoubin Ghahramani. Sparse Gaussian Processes using Pscudo-inputs. In Advances in Neural Information Processing Systems, volume 18, pages 1257-1264, 2005,
[33] James Hensman, Alexander Matthews, and Zoubin Ghahramani. Scalable Variational Gaussian Process Classification. In Artificial Intelligence and Statistics, pages 351-360. PMLR, 2015.
[34] John M Digman. Higher-order Factors of the Big Five. Journal of Personality and Social Psychology, 73(6): 1997.
[35] James Baglin. Improving Your Exploratory Factor Analysis for Ordinal Data: A Demonstration Using FACTOR. Practical Assessment, Research, and Evaluation, 19(1):1-15, 2014.
[36] Kenneth A Bollen. Structural Equations with Latent Variables. John Wiley & Sons, 1989.
[37] Fumiko Samejima. Estimation of Latent Ability Using Response Pattern of Graded Scores. Psychometrika Monograph Supplement, 34:1-97, 1969.
[38] Wim Van der Linden and RK Hambleton. Handbook of Item Response Theory. Taylor & Francis Group, 1997.
[39] Darrell Bock and Murray Aitkin. Marginal Maximum Likelihood Estimation of Item Parame- ters: Application of an EM Algorithm. Psychometrika, 46(4):443-45 -599 1981.
12

---

[40] Carlos G Forero, Alberto Maydeu-Olivares, and David Gallardo-Pujor Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation. Structural Equation Modeling: A Multidisciplinary Journal, 16(4):625-66) 2009.
[41] Cheng-Hsien Li. Confirmatory Factor Analysis with Ordinal Data: Comparing Robust Max- imum Likelihood and Diagonally Weighted Least Squares. Behavior Research Methods, 48: 936-949, 2016.
[42] Sergio Salvatore and Jaan Valsiner. Between the General and the Unique: Overcoming the Nomothetic versus Idiographic Opposition. Theory & Psychology, 20(6):817-83) 2010.
[43] Carl Edward Rasmussen and Christopher K. Williams, Gaussian Processes for Machine Learning. The MIT Press, 2006.
[44] Mauricio A Alvarez, Lorenzo Rosasco, Neil Lawrence, al. Kernels for Vector- Valued Functions: A Review. Foundations and Trends@ in Machine Learning, 4(3):195-266)) 266, 2012.
[45] Robert R McCrae and Oliver John. An Introduction to the Five-Factor Model and Its Applications. Journal of Personality, 60(2):175-2155 1992.
[46] Markus Herdin, Nicolai Czink, Hüseyin Ozcelik, and Ernst Bonek. Correlation Matrix Distance, Meaningful Measure for Evaluation of Non Non-stationary MIMO Channels. In 2005 IEEE 61st Vehicular Technology Conference, volume 1, pages 136-140. IEEE, 2005.
[47] Eiji Muraki. A Generalized Partial Credit Model: Application of EM Algorithm. Applied Psychological Measurement, 16(2):159-176, 1992.
[48] Gerhard Tutz. Sequential Item Response Models with an Ordered Response. British Journal of Mathematical and Statistical Psychology, 43(1):39-55, 1990.
[49] Daniel McNeish, Jennifer A Somers, and Andrea Savord. Dynamic Structural Equation Models with Binary and Ordinal Outcomes in Mplus. Behavior Research Methods, pages 1-27, 2023.
[50] Christopher J Soto. How Replicable Are Links Between Personality Traits and Consequential Life Outcomes? The Life Outcomes of Personality Replication Project. Psychological Science, 30(5):711-727, 2019.
[51] Oliver P John, Sanjay Srivastava, et al. The Big-Five Trait Taxonomy: and Theoretical Perspectives. Handbook of Personality: Theory and Research, 2:102-138, 1999.
History, Mcasurement,
[52] Christopher J Soto and Oliver John. The Next Big Five Inventory (BF1-2):
Developing and Assessing Hierarchical Model with 15 Facets Enhance Bandwidth, Fidelity, and Predictive power. Journal of Personality and Social Psychology, 113(1):117-143, 2017."
[53] Ellen L Hamaker and Conor V Dolan. Idiographic Data Analysis: Quantitative Methods-Frro Simple to Advanced. In Dynamic Process Methodology in the Social and Developmental Sciences, pages 191-216. Springer. 2009.
[54] Hairong Song and Emilio Ferrer. Bayesian Estimation of Random Coefficient Dynamic Factor Models. Multivariate Behavioral Reseurch, 47(1):26-60, 2012.
[55] Joran Jongerling. Jean- Jean-Philippe Philippe Laurenceau, and Ellen L Hamaker. A Multilevel AR(I) Model: Allowing for Inter-Individual Differences in Trait-Scores, Inertia, and Innovation Variance. Multivariate Behavioral Research, 50(3):334-349 2015.
[56) Vidhi Lalchand, Aditya Ravuri, and Neil D. Lawrence. Generalised GPLVM with Stochastic Variational Inference. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151, pages 7841-7864. PMI.R, 2022.
[57] Andrew T Jebb, Louis Tay, Wei Wang, and Qiming Huang. Time Series Analysis for Psy- chological Research: Examining and Forecasting Change. Frontiers in Psychology, 6:727, 2015.
[58] Sigert Ariens, Eva Ceulemans, and Janne K Adolf. Time Series Analysis of Intensive Longitu- dinal Data Psychosomatic Research: A Methodological Overview. Journal of Psychosomatic Research, 137:11019, 2020.
13

---

A
Notations of IPGP
Throughout our notation, superscript (i) indicates unit and underseript indicates task.
y vy
YJ 17 represents the flattened JT X 1 response vector consisting of all J tasks and periods for unit
f()) = [f, (8) fu 1T represents the flattened J7 latent vector consisting of all J tasks and T periods for unit mapped response space via ordered- proebit likelihood.
W pop is the D x J (D latent dimensions, J tasks) shared interpersonal loading matrix.
( is the D by J unit-specific low-rank loading matrix that serves to be the additional idiographic component, independent of W pop-
KL the unit-specific task covariance matrix with shared component W W pop
pop and low-rank approximation Zti) z of D D for unit-specific deviations.
K(O time
is the time covariance for dynamic changes in the latent attributes. We used an RBF
kernel K (1) time
(t, t') exp (t / with unit-specific bandwidth l/s.
W pop is global parameter estimated for the entire population while z+) is unit-level parameter. These are combined in Eq. (3) to create a unique kernel for each unit that combines both components.
B Mathematical Details of Evidence Lower Bound
We provide the full mathematical details of the evidence lower bound defined in Eq. 6 As KL divergence is always non-negative, first consider the KI. divergence between p(f u) and p(f y):
p(f u)
KL[p(f u) p(f y)] (y(elu) p(f y)
log
(7)
p(f
u)p(y)
(F(F(u) log
(8)
ply ff)(())
p(f u)p(y u)p(u)
(F|())) log
(9)
p(y [f)x(()
p(y u) p(y f)
Ep(f(m) log
(10)
log u) Ep(pj) logp(y ff 2
(11)
Moving (f(() log f) to the R.H.S the above inequality will lead to Eq. the inequality given by [L(((u)) p(u y)] 0:
4 We then exploit
KL(g(u) || p(u y)] Eg(u) log quu) p(u y)
(12)
(u)p(())
Bq(n) log ply u)p(u)
(13)
-Ey(u) Jog((y u) + (L(q()) p(u) + logp(y) 2
(14)
Rearranging the above inequality, applying Eq. and exploiting notation q(f) pf((f |))(u)du leads to the ELBO:
logp(y) 2 Ey(u)) log u) KL[[g(u) || p())]
(15)
Eg(u) [Ep(tju) log f)] KL[g(u) p())
(16)
Egtr) loy (L) p())
(17)
14

---

C Data Collection and Demographics
The LOOPR dataset from Soto [50] in our first case study used Qualtries and quota sampling to ensure representative samples of the U.S. population, and hence was very diverse:
age: 11% ages 18-24, 18% ages 25-34, 17% ages 35-44, 19% ages 45-54, 17% ages 55-64,
18% ages 65 and older
sex: 52% female, 48% male
race/cthnicity: 74% non-Hispanic white/Caucasian, 11% black/Afrrcan American, 10% Hispanic/Latinoo, 3% Asian/Asian American, 2% American Indian/Native American
educational attainment: 10% did not complete high school, 33% high school graduate, 28% some college, 19% college graduate, 10% graduate or professional degree
annual household income (S): 14% <20,000, 12% 20,000-29,999, 11% 30,000-39,997 15% 40,000-49,999, 26% 50,000-79,999, 999, 22% 80,000+
As social and personality psychology often faces challenges with model building due to reliance on non-representative samples, past researches rely heavily on student populations. We follow such practice but aim to broaden our sample within budget and available diversity. Participants in our longitudinal study (the third experiment) are mostly college students with average age of 20.23 (SD 1.94);; 70% are female, 26% are male and are self-identified as other; ethnicities self-reported as 42% Caucasian, 39% Asian, 12% African American, and 7% Other. We used the BF1-2 (52), widely used personality measure across cultures and ages. We acknowledge the typical biases of convenient sampling in higher education, including socio-economic and ethnic diversity limitations.
D Runtime in Simulation Study
Table ( shows the average runtime of TPGP and its competitors the simulation study. TPGP does require time train due to the enlarged model space.
Table 6: Average runtime of IPGP and its competitors in the simulation study.
MODEL
AVG RUNTIME (SEC)
GRM
343
GPCM
367
SRM
1398
GPDM
17359
LSM
311
TVAR
468
IPGP-NOM
17594
IPGP-IND
21562
IPGP-LOW
10839
IPGP-NP
31150
IPGP
30141
15

---

E Estimated Correlations of Selective Individuals
Figure (5) shows the estimated correlations of selective individuals for the identified four profiles in the longitudinal study.
Figure 5: Estimated correlations of selective individuals for the identified four profiles in the longitudinal study.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims accurately reflect this paper's contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in Sec 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the information needed to reproduce the main experimental results of this paper is reported in Sec
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification: The data and code to reproduce the main experimental results of this paper are uploaded as supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https /nipss.cc/pulli//guu guwde//ddee/eee Cot//ooodeeesiseeeeeeeeee hwwpbbmmsssioooooooooooi for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the training and test details to understand the results of this paper are reported in Sec (4)
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All error bars about the statistical significance of experiments are reported in Sec
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
19

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: All computer resources needed to reproduce the experiments are reported in See
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttl::://euurpp.cccpbbbbbbbb ctw.ettteeiiccccceeeeneeiie
Answer: [Yes]
Justification:
The research conducted in this paper conforms the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the societal impacts of our work in Sec 6
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets used in this paper are properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Documentation of new assets introduced in the paper is uploaded as supple- mentary materials.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Questions given participants is uploaded as supplementary materials.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: Research with human subjects in this paper is approved by IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "yTCWBiAsOH", "reviewer": "Reviewer_5m3j", "summary": "UPDATE: I am updating my scores in light of the excellent authors' response.\n\nThis paper considers psychometric data composed of ordinal responses $y_{ijt}$, each of which represent how unit _i_ answered survey item _j_ during time period _t_. The key characteristic of this item-response data is that the same $N$ units are longitudinally surveyed on the same $J$ items over $T$ periods. \n\nThe paper takes an ordered logit factor modeling approach to analyzing such data, wherein $f_j^{(i)}(t) = \\mathbf{w}_j^\\top \\mathbf{x}_i(t)$ is the latent ideal point of unit _i_ on on item _i_ at time _t_, and $\\mathbf{w}_j$ and $ \\mathbf{x}_i(t)$ are the $K$-dimensional loadings and factor vectors, respectively.\n\nThe paper moreover advocates an \"idiographic approach [which] emphasizes _intrapersonal_ variation by requiring distinct loadings $\\mathbf{w}_j^{(i)}$\" which are different for each unit $i$. It is able to effectively achieve this by exploiting the repeated measurements of each unit-item $(i,j)$ pair over different time periods $t$.\n\nThe proposed model is an instance of a multi-task Gaussian process (MTGP). Intrapersonal variation is modeled using a unit-specific kernel $K_{time}^{i}$. The full covariance is then $JT \\times JT$, resulting from a Kronecker product of $\\mathbf{K}_{\\textrm{time}}^{(i)}$ (applied to input $T$ periods) with a low-rank matrix $J \\times J$ matrix that represents covariance between survey items (or \"tasks\"). This all extends the previously-presented linear model of coregionalization (LMC), which was originally developed for the simpler case where observations of $(i,j)$ pairs are not repeated over time.\n\nThe paper derives a variational inference inference algorithm for the model, which follows closely from previous work.\n\nThe paper then reports three sets of experiments: 1) a synthetic study that reports excellent results on parameter recovery (when the true parameters are known), 2) a re-analysis study that purports to show that a non-dynamic (i.e., non-idiographic) version of proposed model is able to identify the correct factor structure (K=5) from survey data designed to assess the \"Big Five\" psychometric traits, and 3) an illustrative case study involving a novel longitudinal data set.", "questions": "As this is an emergency review, after the reviewer discussion period, I am not sure if the authors will be able to reply. But if they are able, I would just ask that they respond to the various points made in previous sub-sections.", "limitations": "This paper did involve a fairly intense human subjects experiment for data collection, but the paper reports it was IRB-approved. Psychometrics of this kind are fairly common; I do not think the paper introduces any new potential negative societal impacts.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is very well-written. It brings an interesting application to life and convinces the reader that the proposed modeling approach is well-tailored to the problem at hand. The paper gives a clear review of the prerequisite concepts and presents its modeling approach clearly. \n\nThe paper covers related work in psychometrics well is convincing that the modeling approach is novel within that applied community. Experiments bear out this claim, as the proposed model performs much better than models which are currently used in psychometrics.\n\nThe paper presents a novel longitudinal survey dataset composed of $N=93$ subjects who were given personality assessment surveys over the course of three weeks. If I am reading the paper correctly, each subject was asked to complete a survey _six times per day_. This data sounds highly non-trivial to collect, and should be considered a main contribution in itself.", "weaknesses": "The paper is vague about its technical contributions. It makes the following hedged novelty statement: \"the first multi-task GP latent variables model _for dynamic idiographic assessment_\". I took this to mean that the proposed approach is not very new technically, but it has never yet been applied to dynamic idiographic assessments. However, the paper also makes the general claim that it \"advances the literatures on Gaussian process latent variable models...\". I am of the opinion that tailoring existing modeling frameworks to new applications does provide a technical contribution, as it contributes a new view and set of interpretations/metaphors that can help to better understand the abstract model. I think the paper does contribute in this way. But I am unsure if the paper further contributes more substantially to the area of Gaussian process latent variable models, as the paper is vague about that.\n\nThere is sloppiness in some of the main equations. For instance, equation 2 confuses a distribution with a random variable stating $p(\\mathbf{f}^{(i}) \\sim \\cdots$, and I believe equation 3 confuses an inner with an outer product (shouldn't it be $w_i w_i^\\top$?). There's also some confusing overloading of symbols between the background and the model sections. The background sets up the idea that an ideographic approach has distinct loadings $w_j^{(i)}$ different across $i$. But the proposed model itself does not seem to sport this as each $w_j$ is global for each task. I believe effectively the model does achieve an idiographic interpretation through the unit-specific kernel, but the connection between the background and model section is not clear.\n\nI think the second experiment has some major flaws. The paper claims this experiment validates the \"Big Five\" because model performance peaks at $K=5$. However, the model is only fit for $K=1...5$. We can clearly see from the log likelihood numbers that model performance for IPGP is monotonically increasing in $K$ for the values considered, suggesting that it will likely be even better at larger values of $K$; if that were true, it would not validate the \"Big Five\" theory. As this is an applied paper, I would expect a much higher level of rigor on one of the two applied case studies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hgFXbgeu1F", "reviewer": "Reviewer_YTHZ", "summary": "1. This paper introduces an innovative measurement framework utilizing the Gaussian process coregionalization model to resolve the question of whether psychological attributes such as personality exhibit a universal structure among the populace or are uniquely individualized. \n2. An Idiographic Personality Gaussian Process (IPGP), a hybrid model that accounts for both the commonality of traits across people and individual-specific \"idiographic\" variations. \n3. Gaussian process coregionalization model to interpret the responses from grouped survey batteries, adapted for non-Gaussian ordinal data, and employs stochastic variational inference for estimating latent factors. \n4. The application of IPGP on both synthetic data and an original survey demonstrates its performance.", "questions": "Please see the weakness above", "limitations": "Please see the weakness above", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper’s main innovation lies in its unique combination of multitask Gaussian process, which results in a conceptualization of the subject matter.\n2. The innovative use of multitask Gaussian process in this paper offers a promising solution to a long-standing challenge of psychological assessment.\n3. The quantitative and qualitative methods in this paper provide a potential in advancing psychological diagnosis and treatment.\n4. The interdisciplinary nature of the research is interesting. \n5. The authors provide a holistic view of the issue at hand.", "weaknesses": "1. there are many MTGPs, the author ignored comparison with them.\n2. incorrect description, such as: \"multi-task structure is also known as the linear model of coregionalization (LMC)\".\n3. This represents a particular implementation of MTGP, but it doesn't introduce any novel methodological advancements.\n4. The paper presents a valuable contribution, but its innovative aspects are limited, as it largely builds upon existing theories without introducing significant new insights.\n5. The literature review appears somewhat limited. Expanding it to include more recent or relevant MTGPs could provide a more comprehensive context for the research.\n6. The statistical analysis used in the study seems inadequate given the complexity of the data.\n7. The research seems to be more of an incremental advancement rather than a novel contribution, which may limit its impact on the field.\n8. The experimental evidence provided in the paper is not as comprehensive as it should be to support the claims made, suggesting a need for more extensive testing.\n9. Only one experiment is related to psychological assessment.\n10. No idiographic personality is discovered and presented in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer YTHZ,\n\nThank you for your valuable feedback. Please see our responses below.\n\n> there are many MTGPs, the author ignored comparison with them\n\t\nTo our knowledge, Duerichen et al. [17] is the only existing MTGP model in the behavioral literature for multivariate physiological time-series analysis, but focuses on modeling responses directly rather than uncovering the latent structure, as we do here. So the construction of additional MTGPs for this setting is a novel contribution of its own. Note that we do include several ablated models in Tables 1-3 that are variants of MTGPs. \n\n> incorrect description, such as: \"multi-task structure is also known as the linear model of coregionalization (LMC)\".\n\nIndeed, the linear model of coregionalizaiton is only one (quite common) realization of the multi-task GP framework. We will reword accordingly.\n\n> This represents a particular implementation of MTGP, but it doesn't introduce any novel methodological advancements.\n\nPlease see the shared rebuttal regarding the scope  and nature of our contributions.\n\n> The paper presents a valuable contribution, but its innovative aspects are limited, as it largely builds upon existing theories without introducing significant new insights.\n\nPlease see the shared rebuttal regarding the scope  and nature of our contributions.\n\n\n> The literature review appears somewhat limited. Expanding it to include more recent or relevant MTGPs could provide a more comprehensive context for the research.\n\nMTGPs are underexplored in the behavioral literature. We did include a discussion on existing MTGP models for multivariate physiological time-series analysis [Duerichen, et al]. Outside the field of psychology, MTGPs have been recently applied in causal inference [1,2,3], environmental science [4, 5, 6] and biomedical research [7]. We add these related works to the literature review.\n\n[1] Aglietti et al. NeurIPS 2020\n\n[2] Alaa and van der Schaar. NeurIPS 2017\n\n[3] Chen, et al. AISTATS 2023\n\n[4] Zhou, et al. Journal of Cleaner Production doi:10.1016/j.jclepro.2020.124710\n\n[5] Li, et al. Measurement doi:10.1016/j.measurement.2021.110085\n\n[6] Dahl and Bonilla. Machine Learning doi:0.1007/s10994-019-05808-z\n\n[7] Zhang, et al. Journal of Biomedical Informatics doi:10.1016/j.jbi.2022.104079\n\n> The statistical analysis used in the study seems inadequate given the complexity of the data.\n\nWe used paired t-tests for model evaluation in Table 1 and likelihood-ratio tests in Tables 2-3, both of which are standard tools for statistical hypothesis testing. We are happy to continue this conversation. Could you please provide more details regarding your concern (for example, an alternative approach or a specific deficiency in ours) so that we can offer meaningful additional comments during the author-reviewer discussion phase?\n\n> The experimental evidence provided in the paper is not as comprehensive as it should be to support the claims made, suggesting a need for more extensive testing.\n\nWe summarize our claims and empirical evidence that supports the claims here:\n\n1. IPGP is a novel psychological assessment model that improves both prediction of actual\nresponses and estimation of individualized factor structures relative to existing benchmarks (see Table 1-2 for evidence).\n\n2. Substantive deviations from the common psychological structures persist in considerable individuals, as IPGP is decisively favored than the nomothetic baseline (see Table 3 for model comparison and Figure 4 for our learned idiographic profiles).\n\nWe are happy to continue this conversation. Could you please provide more details regarding your concern (for example, an alternative approach or a specific deficiency in ours) so that we can offer meaningful additional comments during the author-reviewer discussion phase?\n\n> Only one experiment is related to psychological assessment.\n\nOur paper includes three experiments: a simulation study, a re-analysis of a large cross-sectional dataset (itself a psychological assessment), and a pilot study of repeated measures of the Big Five personality traits (a second, separate assessment). We note that collecting this pilot study was itself an intensive investment requiring multiple months and over $20k in subject payments. Repeated assessment datasets of this type are not widely shared to protect human subjects, hence why we had to collect our own. \n\n> No idiographic personality is discovered and presented in the paper.\n\nFigure 4 presents four residual correlation matrices w.r.t to the main population, each highlighting one distinct profile of idiographic personality. Selected full correlation matrices of idiographic personality types can also be found in Appendix B. Further, the results in Table 3 show clearly that the model that allows for idiographic personality structure is strongly favored.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hHFw0UZiCv", "reviewer": "Reviewer_mvMV", "summary": "Gives a multi-task/output GP formulation for multiple time-series (or intrinsic co-regionalisation model) for pyschological assessments. Design of the factor loadings informs the task-correlations and reflects the knowledge of individual's correlation between his responses and inter-person correlations. More of an application paper rather than a paper advancing machine learning; but a good application nonetheless.", "questions": "I do not have any critical questions.", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The application is a natural fit for the model chosen, and results are good and convincing. \n2. The use of Bayes factor for model testing make the paper better as an reference for this application.\n3. Sections 4.2 and 4.3 evaluates the model from different aspects.", "weaknesses": "1. The paper is weak from the perspective of advancing state-of-the-art machine learning algorithms.\n2. The paper is totally unrelated to GPLVM and tasks that GPLVM are designed for. Mentions of GPLVM only confuses the reader.\n3. Equation 3 needs fixing.\n4. In section 3.2, the method is variational inference, and not stochastic variational inference.\n5. Description of the setup in section 4.1 needs to be clearer to explain also in terms of $K_{task}$.\n6. Line 192 mentions \"informative prior\". What is this \"informative prior\"?\n\n*Minor*\n7. Line 319: \"addressing\" is too strong. Suggest to change to \"contributing\".\n8. Line 321: \"than\" -> \"over\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer mvMV,\n\nThank you for your valuable feedback. Please see our responses below.\n\n> The paper is weak from the perspective of advancing state-of-the-art machine learning algorithms.\n\nPlease see the shared rebuttal regarding the scope  and nature of our contributions.\n\n> The paper is totally unrelated to GPLVM and tasks that GPLVM are designed for. Mentions of GPLVM only confuses the reader.\n\nPlease allow us to elaborate on this claim. We believe that IPGP is spiritually related to GPLVM in that both involve the estimation of low-dimensional latent variables in order to gain more insight into high-dimensional observations. We agree that there is considerable departure in how that estimation proceeds. Further, reviewer A4W7 commented on the necessity of including GPLVM in the related work: “I think its similarities and differences with GPLVM and GPDM are well presented, and the overall application is far from trivial regarding the nature of data”.\n\n> Equation 3 needs fixing.\n\nIndeed, there is a misplaced transpose; it should be $W'W + ww'$, where $W$ is a $K\\times J$ matrix and $w$ is a $J\\times 1$ column vector. We will update accordingly. (Please indicate if you had something else in mind.)\n\n> In section 3.2, the method is variational inference, and not stochastic variational inference.\n\nThank you for pointing this out. We will revise accordingly.\n\n> Description of the setup in section 4.1 needs to be clearer to explain also in terms of $W_\\text{task}$.\n\nThank you for pointing this out. In Section 4.1 we describe the setup of the shared interpersonal loading matrix $W_\\text{pop}$ and unit-specific loading $w_i$, which are then used to construct the $W_\\text{task}$ matrix according to Equation 3. We will make this more explicit in revision.\n\n> Line 192 mentions \"informative prior\". What is this \"informative prior\"?\n\nThe informative prior refers to the interpersonal loading matrix $W_\\text{pop}$ estimated from the standard cross-sectional data (the LOOPR data), which is used to reduce the number of hyperparameters of the full idiographic kernel from K*J (J items, K dimensions) + n*J (J items, n individuals) to nJ. Practically, the model training procedure includes two steps: (1) learning $W_\\text{pop}$ using cross-sectional data, and then (2) learning the rest of the hyperparameters while holding $W_\\text{pop}$ fixed. We show in the simulation that IPGP achieves more precise estimation of individual taxonomies with this stronger prior (see Table 1).\n\n> (Minor) Line 319: \"addressing\" is too strong. Suggest to change to \"contributing\". Line 321: \"than\" -> \"over\"\n\nThank you for the suggestions; we will adopt them in revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZeqP8uP8yV", "reviewer": "Reviewer_A4W7", "summary": "This paper presents a multi-output Gaussian process model for classification in the context of psychological studies. It's based on a linear model of co-regionalisation leveraging unit-level latent factors and RBF kernels to jointly model population and individual personality traits to tackle an old debate in psychometrics regarding their original nature. The authors leverage a variational formulation to derive a lower bound for the model evidence and optimise the individual-specific, population, and GP-related parameters. An extensive simulation and real-data study is proposed, presenting longitudinal and multi-subject survey analyses, personality correlations and predictions.", "questions": "I only have a few questions regarding computation times, scaling and robustness of the inter-output covariance matrices, which are well-known limitations of co-regionalisation GP models in practice. \n\n- In this sense, could you provide numerical evidence and comparison regarding training and prediction times for IPGP and competitors?  \n- Even though the number of tasks/topics remains probably limited in psychological applications, could you discuss a bit more, in Section 4.1 **Result**, the decrease in performance between full IPGP and the low-rank version. What could we expect for higher dimensions? Would the low-rank approximation remain robust for a more significant gap with the actual rank? \n- In my experience, several multi-output GP models (or maybe their implementations) are somewhat unstable when it comes to estimating $\\textbf{K}_{task}$, and pathological cases can often arise. Have you experienced such problems? And if so, how did you manage to tackle this issue? \n\n**Typos:**\n\nPage 5: The indices of the CMD definition have a slight error. All terms are written as $R_1$, and the norm is 'f' instead of '$l_2$'  \nPage 6: The title of 4.2 \"Cross-secitonal\" ==> Cross-sectional  \nPage 7: \"Both correlation matrices **displace** a block pattern\" ==> display?  \nPage 7: \"questions corresponding negative emotionality\" ==> corresponding to", "limitations": "In my opinion, the limitations are adequately discussed. The application spectrum and position in the related literature are well documented.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper is remarkably well-written and presented. The flow of derivations is easy to follow, and the illustrations are helpful and of excellent quality. The treated problem is far from trivial, with data that present several degrees of correlations (time, individual, population) and technical constraints (missing and categorical data). The experiments are impressive, with extensive comparison against many competitors and excellent results overall. The method shows great promise in providing practical insight into the domain, and even though I'm not a specialist in psychology, the ability to reconcile idiographic and nomothetic approaches seems particularly valuable.", "weaknesses": "One could argue that the methodological novelty is limited as the presented multi-output GP model is fairly well-known in the GP community. However, I think its similarities and differences with GPLVM and GPDM are well presented, and the overall application is far from trivial regarding the nature of data (categorical outputs, latent variables, missing data, ...). This relative weakness is more than compensated by the strength of results and the potential such a methodology brings in a field like psychology, where the signal-on-noise ratio is generally low, and the nature of measurements is often challenging.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer A4W7,\n\nThank you for your valuable feedback. Please see our responses below\n\n> In this sense, could you provide numerical evidence and comparison regarding training and prediction times for IPGP and competitors?\n\nSure, please see below. We’d however first like to comment that runtime is typically not a primary concern in such analysis. The data we collected here required several months of time and over $20k in subject payments, so a few hours (or even a couple days) of one-time work is quite reasonable in light of that initial outlay if it provides additional insight.\n\nThe table below shows the average runtimes of IPGP and competitors on the simulation study data from the paper. IPGP does require more time to train due to the larger model space, but the runtime is still quite manageable even with a non-optimized reference implementation. We will add this table in the appendix and include some explicit discussion on runtime to the manuscript.\n\n| Model | GRM  | GPCM | SRM  | GPDM | DSEM | TVAR | IPGP-NOM | IPGP-IND | IPGP-LOW | IPGP-NP | IPGP  |\n|---------------|------|------|------|------|------|------|----------|----------|----------|---------|-------|\n| Avg runtime (sec)   | 343  | 367  | 1398 | 17359| 311  | 468  | 17594    | 21562    | 10839    | 31150   | 30141 |\n\n> Even though the number of tasks/topics remains probably limited in psychological applications, could you discuss a bit more, in Section 4.1 Result, the decrease in performance between full IPGP and the low-rank version. What could we expect for higher dimensions? Would the low-rank approximation remain robust for a more significant gap with the actual rank?\n\nIn general, a low-rank approximation inevitably lacks the model capacity of its full-rank counterpart, but this deficiency only matters up to the intrinsic rank of the data. Blindly increasing the rank beyond that point will not improve performance. In practice, when the rank is not known, a data-driven search could be performed to identify a suitable rank.\n\nWe ran an additional experiment during the rebuttal period using the simulation data from Section 4.1 in the manuscript where we varied the model rank in {2, 5, 8} (recall the true rank of the data was 5), repeating this simulation using 25 different random seeds. The results are summarized in the below table. Both the true rank (5) and high rank (8) models outperform the low rank (2) model, but there is no performance gain by increasing the rank more than necessary. \n\nWe will add this table and discussion to the appendix.\n\n| Rank | Train Acc          | Train LL            | Test Acc           | Test LL            | CMD               |\n|------|--------|------------|-----------|-------|-------|\n| 2    | 0.897 $\\pm$ 0.004  | −0.313 $\\pm$ 0.010  | 0.884 $\\pm$ 0.005  | −0.334 $\\pm$ 0.011 | 0.397 $\\pm$ 0.007 |\n| 5    | 0.957 $\\pm$ 0.002  | −0.159 $\\pm$ 0.005  | 0.942 $\\pm$ 0.002  | −0.184 $\\pm$ 0.006 | 0.128 $\\pm$ 0.006 |\n| 8    | 0.957 $\\pm$ 0.002  | −0.161 $\\pm$ 0.004  | 0.945 $\\pm$ 0.002  | −0.183 $\\pm$ 0.005 | 0.124 $\\pm$ 0.006 |\n\n\n> In my experience, several multi-output GP models (or maybe their implementations) are somewhat unstable when it comes to estimating, and pathological cases can often arise. Have you experienced such problems? And if so, how did you manage to tackle this issue?\n\nWe have not experienced such issues. The GPyTorch software that our multi-output GP implementation is based on is quite stable.\n\n> Typos:\n\nThank you for pointing these out. We will fix them in revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Two reviewers commented on the nature of our contributions. We post a shared comment on the scope of our contributions here.

We would like to first stress that our contributions are not merely theoretical, but that our work also represents applied machine learning for science. Both applications and ML for science are listed as topics of interest in the call for papers https://neurips.cc/Conferences/2024/CallForPapers. We would also like to highlight the following language in the both the call and reviewer instructions:

> There are many examples of contributions that warrant publication at NeurIPS. These contributions may be theoretical, methodological, algorithmic, empirical, connecting ideas in disparate fields (‘bridge papers’), or providing a critical analysis.

A primary contribution of our work is in “bridging” between advances in (MT)GP modeling and the disparate field of psychological assessment and diagnosis. Building this bridge required significant engagement from another community and significant care in modeling to address the nuances and challenges presented by this setting (to quote reviewer A4W7: “categorical outputs, latent variables, missing data, ...”). Indeed, reviewer A4W7 commented further on the nature -- and relative strength -- of our contributions in this light:

> One could argue that the methodological novelty is limited as the presented multi-output GP model is fairly well-known in the GP community. However, I think its similarities and differences with GPLVM and GPDM are well presented, and the overall application is far from trivial regarding the nature of data (categorical outputs, latent variables, missing data, ...). This relative weakness is more than compensated by the strength of results and the potential such a methodology brings in a field like psychology, where the signal-on-noise ratio is generally low, and the nature of measurements is often challenging.

To provide our contributions here a bit more context, and underscore the importance of the bridge we build here, there is an underlying debate in psychology that we are addressing with this work: whether (i) all individuals have a shared personality structure, (ii) all individuals have a (unique) idiosyncratic structure, or (iii) something in between. The IPGP framework offers a powerful new tool to engage in this debate as each of these scenarios can be modeled with a special case of our model. Bayes factors (and predictive capacity) then allow us to measure the relative merit of these claims in a principled and data-driven manner.  

Our experimental results reaffirm the common Big Five personality model through a factor analysis study in Table 2, and show that the idiographic model is decisively favored to the nomothetic model in Table 3. Moreover, we identify distinct personality profiles that substantially differ from the interpersonal commonality (see Figure 4). Our results therefore provide a significant challenge to dominant paradigms of personality by showing evidence for an intermediate outcome where there is neither a perfectly shared structure nor a perfectly idiosyncratic structure, but rather structured idiosyncratic deviations from a common baseline. We believe that our methodological approach sets the stage for significant advancements in theorizing, evaluation, and (eventually) clinical care in the psychological domain.

Meta-review: NOTE: One reviewer provided vague and highly formulaic criticism that they were unable or unwilling to defend during the discussion period; I am ignoring their review in making my recommendation.

This is a very interesting applied paper, that connects recent methodology on multi-output Gaussian processes to a long-standing debate in psychometrics on the whether "features like personality share a common structure across the population, vary uniquely for individuals, or some combination"---i.e., the "nomothetic" versus "idiographic" approach.

The paper (and authors) should be credited for introducing a novel longitudinal data set composed of survey responses from N=93 subjects who were asked to fill out a personality assessment six times per day for 3 weeks. This data seems very labor-intensive to collect, and pairs well with the proposed model and theoretical question.

There was back-and-forth between the authors and reviewers about the proposed model, whether it was novel outside its applied domain, and whether it can be said to contribute broadly to the area of GPs and GP latent variable models. The paper hedges in several places about the novelty of the model, generally saying that it is novel for this (fairly narrow) applied domain, and the authors did not clarify this in their responses. The authors did however agree in the the discussion period to walk back some statements in the paper about whether the proposed model advances general methodology in GPLVMs. All-in-all, the discussion period clarified that the technical novelty of the model itself should not be overstated, but that the model nevertheless represented a non-trivial and correct adaptation of current methodology to an interesting applied problem, and was a contribution to the literature in the sense of providing a new illustration and use-case of multi-output GPs.

The authors cleared up several other problems raised by the reviewers during the discussion period. The reviewers detected some sloppiness in the equations; however, the discussion revealed that these were all minor typos which were easy to fix, and did not impede understanding of the technical content. One reviewer pointed out a major flaw in the main applied experiment which purported to validate the "Big Five" personality trait by running the model with K=1...5, and showing that K=5 performed best; the flaw was that the paper did not report values of K beyond 5. However, in their response, the authors provided supplemental experimental evidence that showed that the model's performance did in fact decline for K > 5.

I am in favor of acceptance. From the strict perspective of advancing general-purpose GP methodology, this paper is not very strong. However, the paper stands as a good example of translational applied work. It encodes an ongoing theoretical debate in psychology into a non-trivial multi-output GP model and applies the model to a novel data set that was difficult to obtain and bears directly on the applied question. I think NeurIPS should welcome work like this.

In the event of acceptance, the authors should make the following revisions to their camera-ready version:
- Any statements about the connection to GPLVMs should be made more precise and edited in accordance to the discussion with reviewers.
- Any novelty statements about the proposed model should be made more precise and/or omitted. If the model is not novel generally, but novel to this particular application, then this should be stated clearly, and citations to papers that employ the same (or very similar) models to other domains should be cited in the related work section. 
- The typos in equations that reviewers pointed out should be fixed.
- The supplementary experiments reported during the discussion phase should be incorporated into the paper.
Decision: Accept (poster)