Scale construction method It is possible that something similar to your scale will already exist, so including those scale(s) and possible dependent variables in your survey may increase validity of your scale. Begin by generating at least ten items to represent each of the scales. Administer the survey; the more representative and larger the sample, the more confidence one will have in the scales. Review the means and standard deviations for the items, dropping any items with skewed means or very low variance. Run a principal components analysis with oblique rotation on your items and the other items for scales - it will be important to differentiate from your own. Request components with eigenvalues (for calculating eigenvalue for each factor square the factor loading's and sum down the columns) greater than 1. It is easier to group the items by targeted scales. The more distinct the other items, the better the chances the items will load only on your own scale. “Cleanly loaded items” are those that load at least .40 on one component and more than .10 greater on that component than on any others. Identify those. “Cross loaded items” are those that do not meet the above criterion. These are candidates to drop. Identify components with only a few items that do not represent clear concepts, these are “uninterpretable scales.” Also identify any components with only one item. These components and their items are candidates to drop. Look at the candidates to drop and the components to be dropped. Is there anything that needs to be retained because it is critical to your construct ? For example, if a conceptually important item only cross loads on a component to be dropped, it is good to keep it for the next round. Drop the items, and rerun asking the program to give you only the number of components after dropping the uninterpretable and single-item ones. Go through the process again starting at Step 3. Keep running through the process until you get “clean factors” (all components have cleanly loaded items). Run the Alpha program (asking for the Alphas if each item is dropped). Any scales with insufficient Alphas should be dropped and the process repeated from Step 3. [Coefficient alpha=number of items2 x average correlation between different items/sum of all correlations in the correlation matrix (including 1s)] For better practices, keep the final components and all loadings of yours and similar scales selected to be used in the Appendix of your scale. With comparative scaling, the items are directly compared with each other (example: Do you prefer Pepsi or Coke?). In noncomparative scaling each item is scaled independently of the others. (Example: How do you feel about Coke?) Comparative scaling techniques Pairwise comparison scale – a respondent is presented with two items at a time and asked to select one (example : Do you prefer Pepsi or Coke?). This is an ordinal level technique when a measurement model is not applied. Krus and Kennedy (1977) elaborated the paired comparison scaling within their domain-referenced model. The Bradley–Terry–Luce (BTL) model (Bradley and Terry, 1952; Luce, 1959) can be applied in order to derive measurements provided the data derived from paired comparisons possess an appropriate structure. Thurstone's Law of comparative judgment can also be applied in such contexts. Rasch model scaling – respondents interact with items and comparisons are inferred between items from the responses to obtain scale values. Respondents are subsequently also scaled based on their responses to items given the item scale values. The Rasch model has a close relation to the BTL model. Rank-ordering – a respondent is presented with several items simultaneously and asked to rank them (example : Rate the following advertisements from 1 to 10.). This is an ordinal level technique. Bogardus social distance scale – measures the degree to which a person is willing to associate with a class or type of people. It asks how willing the respondent is to make various associations. The results are reduced to a single score on a scale. There are also non-comparative versions of this scale. Q-Sort – Up to 140 items are sorted into groups based on rank-order procedure. Guttman scale – This is a procedure to determine whether a set of items can be rank-ordered on a unidimensional scale. It utilizes the intensity structure among several indicators of a given variable. Statements are listed in order of importance. The rating is scaled by summing all responses until the first negative response in the list. The Guttman scale is related to Rasch measurement; specifically, Rasch models bring the Guttman approach within a probabilistic framework. Constant sum scale – a respondent is given a constant sum of money, script, credits, or points and asked to allocate these to various items (example : If you had 100 Yen to spend on food products, how much would you spend on product A, on product B, on product C, etc.). This is an ordinal level technique. Magnitude estimation scale – In a psychophysics procedure invented by S. S. Stevens people simply assign numbers to the dimension of judgment. The geometric mean of those numbers usually produces a power law with a characteristic exponent. In cross-modality matching instead of assigning numbers, people manipulate another dimension, such as loudness or brightness to match the items. Typically the exponent of the psychometric function can be predicted from the magnitude estimation exponents of each dimension. Scale evaluation Scales should be tested for reliability, generalizability, and validity. Generalizability is the ability to make inferences from a sample to the population, given the scale you have selected. Reliability is the extent to which a scale will produce consistent results. Test-retest reliability checks how similar the results are if the research is repeated under similar circumstances. Alternative forms reliability checks how similar the results are if the research is repeated using different forms of the scale. Internal consistency reliability checks how well the individual measures included in the scale are converted into a composite measure.
Scale construction method It is possible that something similar to one's scale will already exist, so including those scale(s) and possible dependent variables in one's survey may increase validity of one's scale. Begin by generating at least ten items to represent each of the scales. Administer the survey; the more representative and larger the sample, the more confidence one will have in the scales. Review the means and standard deviations for the items, dropping any items with skewed means or very low variance. Run a principal components analysis with oblique rotation on one's items and the other items for scales - it will be important to differentiate from one's own. Request components with eigenvalues (for calculating eigenvalue for each factor square the factor loading's and sum down the columns) greater than 1. It is easier to group the items by targeted scales. The more distinct the other items, the better the chances the items will load only on one's own scale. “Cleanly loaded items” are those that load at least .40 on one component and more than .10 greater on that component than on any others. Identify those. “Cross loaded items” are those that do not meet the above criterion. These are candidates to drop. Identify components with only a few items that do not represent clear concepts, these are “uninterpretable scales.” Also identify any components with only one item. These components and their items are candidates to drop. Look at the candidates to drop and the components to be dropped. Is there anything that needs to be retained because it is critical to one's construct ? For example, if a conceptually important item only cross loads on a component to be dropped, it is good to keep it for the next round. Drop the items, and rerun asking the program to give one only the number of components after dropping the uninterpretable and single-item ones. Go through the process again starting at Step 3. Keep running through the process until one get “clean factors” (all components have cleanly loaded items). Run the Alpha program (asking for the Alphas if each item is dropped). Any scales with insufficient Alphas should be dropped and the process repeated from Step 3. [Coefficient alpha=number of items2 x average correlation between different items/sum of all correlations in the correlation matrix (including 1s)] For better practices, keep the final components and all loadings of yours and similar scales selected to be used in the Appendix of one's scale. With comparative scaling, the items are directly compared with each other (example: Does one prefer Pepsi or Coke?). In noncomparative scaling each item is scaled independently of the others. (Example: How does one feel about Coke?) Comparative scaling techniques Pairwise comparison scale – a respondent is presented with two items at a time and asked to select one (example : does one prefer Pepsi or Coke?). This is an ordinal level technique when a measurement model is not applied. Krus and Kennedy (1977) elaborated the paired comparison scaling within their domain-referenced model. The Bradley–Terry–Luce (BTL) model (Bradley and Terry, 1952; Luce, 1959) can be applied in order to derive measurements provided the data derived from paired comparisons possess an appropriate structure. Thurstone's Law of comparative judgment can also be applied in such contexts. Rasch model scaling – respondents interact with items and comparisons are inferred between items from the responses to obtain scale values. Respondents are subsequently also scaled based on their responses to items given the item scale values. The Rasch model has a close relation to the BTL model. Rank-ordering – a respondent is presented with several items simultaneously and asked to rank them (example : Rate the following advertisements from 1 to 10.). This is an ordinal level technique. Bogardus social distance scale – measures the degree to which a person is willing to associate with a class or type of people. It asks how willing the respondent is to make various associations. The results are reduced to a single score on a scale. There are also non-comparative versions of this scale. Q-Sort – Up to 140 items are sorted into groups based on rank-order procedure. Guttman scale – This is a procedure to determine whether a set of items can be rank-ordered on a unidimensional scale. It utilizes the intensity structure among several indicators of a given variable. Statements are listed in order of importance. The rating is scaled by summing all responses until the first negative response in the list. The Guttman scale is related to Rasch measurement; specifically, Rasch models bring the Guttman approach within a probabilistic framework. Constant sum scale – a respondent is given a constant sum of money, script, credits, or points and asked to allocate these to various items (example : If one had 100 Yen to spend on food products, how much would one spend on product A, on product B, on product C, etc.). This is an ordinal level technique. Magnitude estimation scale – In a psychophysics procedure invented by S. S. Stevens people simply assign numbers to the dimension of judgment. The geometric mean of those numbers usually produces a power law with a characteristic exponent. In cross-modality matching instead of assigning numbers, people manipulate another dimension, such as loudness or brightness to match the items. Typically the exponent of the psychometric function can be predicted from the magnitude estimation exponents of each dimension. Scale evaluation Scales should be tested for reliability, generalizability, and validity. Generalizability is the ability to make inferences from a sample to the population, given the scale one have selected. Reliability is the extent to which a scale will produce consistent results. Test-retest reliability checks how similar the results are if the research is repeated under similar circumstances. Alternative forms reliability checks how similar the results are if the research is repeated using different forms of the scale. Internal consistency reliability checks how well the individual measures included in the scale are converted into a composite measure.
R
your
one's
R
your
one's
R
your
one's
R
your
one's
R
your
one's
R
your
one's
R
your
one's
R
you
one
R
you
one
R
your
one's
R
Do you
Does one
R
do you
does one
R
Do you
does one
R
you
one
R
you
one
R
you
one