Further Analysis of Goldberg-540-TDA Data
Creators: name: Saucier, Gerard
Publisher: name: Open Science Framework
Year: 2022

description: ANALYSIS PLAN: Potential High-Dimensionality Structure in the 540-tda (Self and Peer Ratings) OSF Project Name: Further Analysis of Goldberg-540-TDA Data RESEARCH QUESTIONS: Question 1: How much does this lexical-personality-study data (the 540 trait-descriptive adjectives, which although not the largest is the most frequently used source for the structure of English-language personality-trait adjectives) yield evidence supporting ‘high-dimensionality’ structures of 15, 21, or 28 factors? The high-dimensionality structures mentioned were derived from a more comprehensive selection of 1710 adjectives (although in a smaller set of self-ratings). The same question can be asked in two more specific forms. Question 1a: When one extracts 15, 21, or 28 factors in these data, how many of the expected factors appear? Question 1b: When one determines the most robust ‘emic maximum’ factor structure for this data, how much do the factors in that structure overlap with the high-dimensionality structures of 15, 21, and 28 factors? ‘Emic maximum’ refers to the structure having the largest number of interpretable factors that are sufficiently sized. ‘Robustness’ is here operationally defined as a tendency to appear regardless of data handling (ipsatized versus original data), use of orthogonal versus oblique rotation method, and use of self versus peer rating subsets of the data. Note: See the reference list representing prominent previous uses of the 540-tda data. Question 2: In these data, how do high-dimensionality structures compare with each other, and with conventional models of one to six factors, in internal consistency, in factor independence, and in predicting important outcomes or diverse sets of variables from other measurement platforms? The variable selections are all a priori rather than determined by the current investigators, and in fact are all derived from the 1970s, work by Norman and Goldberg 40-50 years ago. 1. A PRIORI–SPECIFIED PROCEDURE FOR EXPLORATORY FACTOR ANALYSES 1. For the 540-tda (trait descriptive adjective) set, with ipsatized data, apply two methods that recommend a specific number of factors (parallel analysis and the MAP-index based on PCA [based e.g. on Zwick and Velicer, 1986) to determine an initial range of ‘numbers of factors’ to consider – from the lowest recommended by any method to the highest recommended by any method/variable-set combination, as well as the best starting point (that number most redundantly recommended across these combinations, if any are redundant). Specifically, parallel analysis (PA) will be executed first, as it does not require initially setting a maximum number of factors; if PA recommends more than 33 factors, that number recommended by PA will be set of maximum number of factors, otherwise the default of 33 factors will be the initial set maximum for the MAP procedure (based on a previous study with the more comprehensive 1710-tda in which 33 factors was the maximum number of factors that was ever indicated by PA or MAP methods), although if no clear minimum has been obtained at 33 factors or less, a higher minimum will be sought. 2. Within that range, run varimax, equamax, and oblimin rotations (as in Goldberg, 1990) for various numbers of factors. Previous work with the 1710-tda indicated the latter two methods are best for identifying high-dimensionality structures, but varimax is also included because it has been the typical rotation method in studies of personality descriptors from the natural language. One prominent method of oblique rotation – promax – is intentionally omitted from the analyses because it is merely a variant of varimax and would tend to converge extremely highly (be redundant) with varimax. 3. Within the determined range, discard solutions that have any insufficiently sized factors, by a preset ‘sufficient size’ minimum of 3 salient terms with a loading of at least .3 in absolute magnitude, also including at least one salient term with a loading of at least .4 in absolute magnitude. These threshold numbers of salient terms are proportional in comparison to those used in a study with the much larger 1710-tda, in which the threshold was 8 salients of at least .3, including 3 with at least .4. 4. For the solutions that remain, determine the maximum number of interpretable factors. That is, any solutions that have any factors that are judged by both of the two investigators as impossible to interpret substantively are eliminated. 5. For each of the remaining solutions – the maximum number-of-factors (that are sufficiently sized and interpretable) within each data set are set out as a candidate structure, and then compared across data sets, so as to sort out which number-of-factors has relatively better and worse convergence across methods (i.e., use of ipsatized versus original/raw data, of orthogonal vs. oblique rotation, and of self- versus peer-rating subsets of the data). Presumably, just one of the three emic-maximum candidate structures (i.e., the one from varimax, from equamax, and from oblimin) will stand out as the most robust and the best emic-maximum model. Steps 1 through 5 are all oriented toward identifying the optimal ‘emic maximum’ model from these data. Steps 6 pertains to the various, more a priori, ‘etic’ models also used in this study’s comparisons. 6. To the extent that the above procedure did not examine solutions at the one- to six-factor level, and at the 15-, 21-, and 28-factor level, add these, with comparison on the same three robustness indices. The one-factor level will be indexed by the first unrotated principal component. Two- through six-factor levels will be indexed by varimax-rotated components, consistent with the previous literature. Consistent with the method associated with each in the previous study, the 15- and 28-factor levels will be indexed by an equamax rotation, and the 21-factor level by an oblimin rotation (delta=0). 7. Replication of the 15-, 21-, and 28-factor structures in the 540-tda data will be enabled by use of marker terms for each factor determined from previous study results. That study identified 8-item adjective scales for each factor, based mechanistically on the highest loadings on each pole of the factor. That subset of each set of 8 items that is represented in the 540-tda will be used as marker-indicators for each factor, provided at least 3 items are found in the 540-tda. If fewer than 3 are found, factor-loading tables from the previous study will be consulted to identify enough additional terms based on the next highest loading(s), and is found also in the 540-tda and can be used as a marker-indicator; in this manner, each factor in the 15-, 21-, and 28-factor models should have at least three marker-indicators. The sets of marker-indicators can be employed to gauge replication in two ways: (a) what percentage of the marker-indicators is associated with a unique factor in the 540-tda data, and (b) how much the scored composites of the marker-indicators correlate with the various 540-tda factors. By these indices a high degree of replication would involve (a) high percentages for each 540-tda factor, and for each set of marker-indicators, and each high-percentage for a 540-tda factor associated with one and only one of the marker-indicator sets, and (b) high convergent and low divergent correlations of the marker-indicator sets with the 540-tda factors. In either case, high degree of replication would involve one-to-one matches between marker-indicator sets and individual factors. Notes: (a) Orthogonal versus oblique comparisons will be varimax vs. oblimin for one-to six-factor solutions and for the emic-maximum varimax model. For all other comparisons, these will be equamax vs. oblimin comparisons, on the basis that as the number of factors becomes large these two methods have been observed to be able to produce more interpretable factors of sufficient size. (b) The methods for examining robustness in step 5 must vary according to the comparison: The ipsatized-original comparison can utilize canonical correlation analysis, the orthogonal-oblique comparison will need to utilize comparison of factor scores (the next most optimal method), and the self-peer comparison (because these are same variables but different cases) will need to rely on Tucker coefficients of factor congruence. (c) Because most previous studies, including all since 1992 cited below, combined self- and peer-rating data to simplify analyses, this study does the same; however unlike those studies it uses self- vs. peer-data replication as an index of robustness. (d) In this study, one-to six-factor models are examined only for comparison with high-dimensionality models with respect to robustness (and with respect to prediction, as in question 2). For comparison, we will also examine the parallel analysis and MAP results for original (non-ipsatized) data; these are expected to indicate slightly more factors, but previous study indicates that factors based on original data tend to be somewhat less robust than those based on ipsatized data. 2. COMPARING MODELS WITH RESPECT TO PREDICTIVE VALIDITY, INTERNAL CONSISTENCY, AND FACTOR INDEPENDENCE Factor scores (i.e., component scores from principal components analyses) will be retained for each factor model derived from the 540-tda, but the prime comparisons will involve aggregates of adjectives selected as best representatives of the various factors; this is because factors could be orthogonal even if the most salient indicators for them are not orthogonal upon aggregation. For each factor, up to eight adjectives will be selected by the ‘highest-loading items’ method. For each of the opposing poles of the factor, the four adjectives with the highest loading will be selected, so long as the loading in question exceeds .30; if fewer than four can be found for a pole of the factor, further adjectives will be added to the contrasting pole until the total number of items is eight; no adjectives with loading under .30 in magnitude will be selected. The models (the emic-maximum model from the 540-tda, and the imported models including 1-to-6 factor and 15, 21, and 28-factor models, plus any reduced version of those models including only those imported factors judged to be replicated in the 540-tda) will be compared on the following criteria by the procedures associated with each below. 1. Minimally acceptable internal consistency. Minimally acceptable is defined as coefficient Alpha of .60 (or, for scales with fewer than 8 items, a mean inter-item correlation of .20). Lower levels will be taken to indicate an entirely unsuitable degree of incoherence among the marker items. 2. A high degree of factor independence. The correlation matrix of the various marker scales in each structure will be computed, and the average (absolute value) intercorrelation will constitute the prime indicator of factor independence versus non-independence. A secondary indicator will be the maximum correlation within that matrix. 3. Predictive validity (a). Self-reported height and weight is available for nearly all participants in the self-report sample, and self-report BMI will be calculated from these values (BMI = [weight in pounds/squared height in inches]*703). The shrinkage-adjusted multiple correlation for the factor-scores and (separately) the marker scales will be computed for each structure, for the prediction of BMI, after removing the effects of gender as a first step (i.e., gender-residualized BMI will be the criterion). Because BMI has demonstrated effects on health outcomes, and is likely affected by behavior patterns, it is a useful criterion variable for personality measures. 4. Predictive validity (b). Scores on the various scales of either (a) the Wiggins Content Scales from a somewhat reduced version of the MMPI (Minnesota Multiphasic Personality Inventory; Wiggins, 1966) or (b) the Interpersonal Styles Index (Lorr &amp; Youniss, 1973) are available for most participants in the self-report sample, and indices of predictive validity will be (i) the mean shrinkage-adjusted multiple R for all the factors in the lexical structure model, in predicting alternatively the MMPI and the ISI scales, and (ii) the sum of percentage of variances (set 2 accounting for set 1) from canonical correlation analysis where the MMPI or ISI scales define set 1 and the scales or factors in the structural model define set 2. To the extent a structural model yields (1) a full set of scales with minimally acceptable internal consistency, (2) a high degree of factor independence, and (3) higher predictive validity coefficients, in comparison to all other models, it will be judged the best supported by empirical data in these analyses. References: Principal previous uses of 540-tda data… De Raad, B., Barelds, D. P. H., Levert, E., Ostendorf, F., Mlacic, B., Di Blas, L., Hrebickova, M., Szirmak, Z., Szarota, P., Perugini, M., Church, A. T., &amp; Katigbak, M. S. (2010). Only three factors of personality description are fully replicable across languages: A comparison of 14 trait taxonomies. Journal of Personality and Social Psychology, 98, 160-173. Goldberg, L. R. (1990). An alternative "Description of personality": The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216-1229. Hofstee, W. K. B., De Raad, B., &amp; Goldberg, L. R. (1992). Integration of the Big-Five and circumplex approaches to trait structure. Journal of Personality and Social Psychology, 63, 146-163. Hofstee, W. K. B., Kiers, H. A., De Raad, B., Goldberg, L. R., &amp; Ostendorf, F. (1997). A comparison of Big Five structures of personality traits in English, Dutch, and German. European Journal of Personality, 11, 15-31. Prediction criterion measures… Lorr, M., &amp; Youniss, R. P. (1973). An inventory of interpersonal style. Journal of Personality Assessment, 37, 165-173. Wiggins, J. S. (1966). Substantive dimensions of self-report in the MMPI item pool. Psychological Monographs, 80 (22, Whole No. 630).