Shared Flashcard Set

Details

Psychometrics Final Exam
Factor Analysis and Validity Flash Cards
95
Psychology
Graduate
12/02/2012

Additional Psychology Flashcards

 


 

Cards

Term
The goal of factor analysis
Definition
to summarize the interrelationships among variables (items) in a concise but accurate manner as an aid in conceptualization
Term
What does factor analysis do?
Definition
It analyzes the relationships among a set of variables across a set of individuals, and assumes there are more important, but hidden, variables (factors) causing these relationships
Term
A factor represents...
Definition
an area of generalization that is qualitatively distinct from an area represented by any other factor
Term
separate factors are...
Definition
areas that are qualitatively different where relatively little generalization can be made from one area to another
Term
What does factor analysis allow us to do?
Definition
Factor analysis allows finite humans to analyze numerous variables at a time, to unravel relationships among variables correlated in highly complex ways, to report graduated relationships of variables to factors, and to arrive at parsimonious solutions
Term
Constructs built on factor analysis are....
Definition
conceptually clearer than the a priori ideas, and these constructs are integrated through the development of theories; these theories are generally interested in summarizing major variations in the data, and minor variations are generally excluded
Term
Plasmodes
Definition
demonstration problems that check the validity and robustness of factor analysis; they are often from the physical world (Thurstone’s box problem) where the factor structure is universally known
Term
Uses of Factor Analysis
Definition
• Search data for possible qualitative and quantitative distinctions, this is particularly useful when the sheer amount of available data exceeds comprehensibility (exploratory factor analysis)

• Test hypotheses regarding certain qualitative and/or quantitative distinctions in the data (confirmatory factor analysis)

• Analyze items for scale/subscale development

• Minimize the number of variables for further research, while maximizing the amount of information in the analysis; reduces the set of variables to the most important ones

• Orthogonalize (force to be uncorrelated) variables, typically the independent variables in a multivariate analysis, making the results easier to interpret
Term
Paradigms in Factor Analysis
Definition
these operate as philosophies guiding a person's approach to factor analysis

Mathmatical
Scientific
Factors are inventions
Factors are real
Term
Mathematical paradigm
Definition
the stress is upon the derivation and exact computation of all the procedures; procedures which only give estimates are much less valued than procedures which can be exactly calculated
Term
Scientific paradigm
Definition
concerned with factor analyzing data in order to build and test scientific theories; more concerned about the congruence of the results with future factor analyses than whether or not the procedures can be derived or calculated directly, and estimates which are consistently given across several sets of data may be more highly valued than an exact computation which does not generalize as well
Term
Factors are Inventions
Definition
all scientific constructs, including factors, are human inventions to help us organize our thoughts about data

• We cannot know underlying realities, but only their surface manifestations.
• We integrate these in many ways; since all of these integrations are philosophically equal, we are free to pick the one that is the simplest for us to understand and use
Term
Factors are Real
Definition
factors are expressions of underlying realities with the factors waiting to be discovered

• Cattell notes that in our data it is not appropriate to say that a person’s response to one item influences their response to another item
• There must be a third, latent (but real), variable that is influencing them to answer two items the same way
Term
Moderate Position
Definition
data that has good content validity can be designed so that it determines the outcome rather than the investigator, and if the procedures are applied objectively, the results will be independent of the investigator and the particular variables
Term
Those who hold that factors are only human constructs tend to....
Definition
emphasize the mathematical approach
Term
those who emphasize that factors are real...
Definition
emphasize the scientific approach because these are the most replicable
Term
Criteria for Choosing in Factor Analytic Decisions
Definition
Generalizability
Simplicity
Term
Generalizability
Definition
o Robustness across random samplings of subjects or variables (replication), and invariance across theoretically relevant samples of both subjects (i.e., Gender) and variables (i.e., multi-method measures)

o The prime function of science is to find such generalizability and to develop theories that interrelate and explain it
Term
Simplicity (parsimony or elegance)
Definition
o Whenever there are two explanations for a given phenomenon, the simpler of the two is accepted
o UNLESS the more complex one has a clear advantage in terms of explaining more of the data
Term
DECISION POINTS IN EXPLORATORY FACTOR ANALYSIS
Definition
Number of Factors
Variables
Subjects
Analysis Matrix
Factor Extraction Methods
Factor loading
Communalities
Factor Rotation
Term
Number of Factors
Definition
Usually the first question in the factor analyst’s mind

The most subjective decision, but very important
• When too few factors are extracted, they are likely to include considerable error
• Too many factors is not as serious of a problem, but over extraction of more than a couple of factors also leads to error
• When in doubt, extract an extra factor and then see if it generalizes in new studies
Term
Variables
Definition
• Variables with good reliabilities and high interrelationships are best, but factor analysis will also work with items
• It is best to have at least 5 variables/factor
• You can often get away with as few as 3 variables/factor
Term
Subjects
Definition
• It is best to have at least 20 subjects/variable
• You can often get away with as few as 10 subjects/variable
• 100 total subjects is the minimum

However, IF the variables have good reliability AND there are low correlations between the factors, then even as few as 2 variables/factor and 5 subjects/variable are OK
Term
Analysis Matrix
Definition
Only product moment correlations (SPSS default) necessarily give a legitimate correlation matrix, and these are the only correlation coefficients recommended for exploratory factor analysis; do not use the covariance matrix
Term
Factor Extraction Methods
Definition
• (Mathematical/Invented) Principal Components (SPSS default): a linear transformation of the data that perfectly reproduces the variance of each variable across all components
o Assumes perfect measurement, no error term.
o Easier for pre-computer calculations.
o Only use with variables having strong reliability.
o Especially bad for items.
• (Scientific/Real) Common factors (principal axes): a linear combination of variables that reproduces the significant variance of the matrix and assumes leftover variance is trivial and can be ignored
o Assumes that the variables are fallible, has error term.
o Works well on items and practically all variables.
o Much preferred.

The standard practice in principal component analysis is to not keep all of the components for analysis, so Mathmatical/Inventions researchers also end up ignoring variance.
Term
Factor loading
Definition
the measure of the degree of generalizability between each variable and each factor; the farther the factor loading is from 0, the more one can generalize from that factor to the variable. Loadings can be negative or positive.
Term
Component Loadings
Definition
are inflated
Term
common factor loadings
Definition
are conservative
Term
The usual procedure in science (in relation to factor loading) is
Definition
to maximize fidelity with the data and, if any error might be made, to make that error in the conservative direction; only common factors fit the scientific paradigm
Term
Common factors are also best because
Definition
they are more accurate with fewer than 40 variables; however, once you have more than 40 variables, the extraction method does not matter.
Term
Communalities (the reliability of the variables)
Definition
• (M/I) Components use 1.0. This made pre-computer calculations easier. In modern software, usually mathematical iterations are used to increase fit to the data, again a concession to the real-world lack of perfect reliability (SPSS default for components).
• (S/R) Common factors (principal axes) typically use the variable's squared multiple correlation with all the other variables, and then use mathematical iterations to increase accuracy (SPSS default for principal axes).
• Gorsuch strongly recommends using no more than 3 iterations because otherwise the communality estimates can become artificially inflated (SPSS default is 25).
Term
Factor Rotation
Definition
Always rotate your factors if you have more than one; rotation appropriately spreads the variance among the extracted factors (SPSS default is no rotation).
• (M/I) Orthogonal: forces uncorrelated factors; varimax is best because it produces the clearest factor structure. (Boyd prefers this for building scales with items; the summed item scales will then revert to their natural correlations)
• (S/R) Oblique: factors allowed to be correlated; promax (kappa = 3) is best because it most closely reproduces the actual correlations between the variable scales (Boyd prefers this for exploratory factor analysis)
Term
Orthogonal factors
Definition
are the most conceptually clear and distinct
Term
Oblique factors
Definition
are the most realistic, generalize the best, and can be either correlated or uncorrelated depending upon the data

allow the possibility of a higher-order factor analysis
Term
Higher order factor analysis
Definition
make the primary factors into variables, and then factor these variables; always do a higher order factor analysis when there are substantial correlations among the oblique factors
Term
How to interpret the higher order factor
Definition
The correlation of the higher order factors with the original variables should then be computed to help interpret the higher order factor
Term
Rotated Factor Structure (NOT Factor Pattern) Matrix
Definition
the variables’ correlations with the rotated factors
• By examining which variables load high with the factor and which load low, it is possible to draw some conclusions as to the nature of the factor
• The factor can then be named
Term
Salient loading
Definition
one that is sufficiently high to assume that a relationship exists between the variable and the factor; .3 or higher has come to be seen as the absolute lower bound for a salient loading, and many factor analysts use .4
Term
Cross-loading
Definition
When a variable saliently loads on more than one factor
Term
Eigenvalues (roots)
Definition
a measure of the variance accounted for by all possible factors; in principle components the sum of the eigenvalues equals the number of the extracted components (found in the Total Variance Explained SPSS output table)
Term
Roots > 1 (SPSS default)
Definition
the most popular criteria; the principal components eigenvalues are examined, and the researcher counts the number of those greater than one, and that number is the number of factors; also called the Kaiser criterion, and eigenvalues > 1
• NEVER USE as it is only accurate by accident of the variables/factors ratio and the variable intercorrelations
• It is especially poor with items
Term
Rational Approach
Definition
various numbers of factors are extracted, with different rotations, and the researcher selects the one that makes the most theoretical sense
• Better than roots > 1, which is accurate by chance
• Subjective, susceptible to researcher bias
Term
Scree Test
Definition
the principal component eigenvalues are plotted in order of size:
• The correct number of factors coincides with the point on the eigenvalues plot where the downward descending curve straightens (or breaks) into a much gentler and even slope
• This gentle even slope (the scree) represents trivial/random variance factors
• Requires the use of a straight edge or ruler
Term
Cattell Scree Rules
Definition
1. The scree ≥ 3 points in a row
2. There can be a little variance of the scree points about the line, but not much (Boyd says all points should touch the line)
3. The slope of the scree must not approach the vertical
4. There is generally a relatively sharp, even if small, break in vertical level between the last point on the scree and the next point above
Term
Boyd/Gorsuch Salient Loadings criterion:
Definition
• More factors are extracted than will likely be kept (scree criterion + ~50% more will usually give a safe starting point)
Term
Boyd/Gorsuch Salient Loadings criterion: • A SIGNIFICANT factor has:
Definition
o At least 3 variables that load highest on it at .40 or greater
o Alternatively, at least 2 variables that load highest on it at .50 or greater
o Alternatively, at least 1 variable that loads highest on it at .60 or greater
o In addition, none of the above loadings may have cross-loadings nearer than .10
Term
Boyd/Gorsuch Salient Loadings criterion: A TRIVIAL factor
Definition
does NOT have at least 3 (≥ .40), 2 (≥ .50), or 1 (≥ .60) variable(s) that load highest on it without being negated by cross-loadings nearer than .10
Term
Scale-Builder’s Addendum
Definition
for a factor to be significant, its salient items must also form a scale with a reliability of ≥ .60, indicating it could likely be developed into a scale
Term
When a factor has a single salient loading:
Definition
it is called a singlet factor
• The item can be viewed as one that is reliable enough to be a single item measure of the construct, but generally needs additional items to be developed into good measure
• The scale-builder’s addendum for singlet factors:
o Check the item’s extracted (not initial) communality
o If it is ≥ .60, it meets the criterion
o Recall that the communalities are the estimates of the items’ reliabilities in the factor analysis
Term
Other criteria beyond the scope of Psychometric Foundations
Definition
parallel analysis, minimum average partial correlation, and factor replication/invariance
Term
Factor Scores (creating a factor variable or scale)
Definition
• (M/I) Components: regression scores are calculated (SPSS default for components)
• (S/R) Common factors: regression scores are estimated (SPSS default for principal axes); in terms of generalization to a new sample, factor score estimates clearly work as well or better than calculated factor scores
• (S/R) Unit weights are actually better than regression weights unless your N is about 500 or more
o They allow for less capitalization upon chance and are more stable, thus generalizing to new samples better
o 1’s for items that load saliently, 0’s for items that don’t
Term
Unit weight scales
Definition
will contain the natural correlations between the items, whether they are created from varimax or promax factor loading tables
Term
Promax factor regression
Definition
scores with kappa = 3 tend to intercorrelate at levels similar to the unit weight scales (SPSS default kappa = 4, which tends to create correlations higher than the unit weight scales)
Term
Varimax factor regression score
Definition
variables will not be correlated
Term
VALIDITY
Definition
does the measure assess the intended attribute well?
Term
Construct validity
Definition
measuring the psychological attribute adequately; the overall validity
Term
Predictive validity
Definition
the measure shares a statistical relationship with a particular criterion
Term
Content validity
Definition
the items are a representative sampling from the pool of required content
Term
Internal structure (factorial) validity
Definition
the factor structure matches the theory
Term
Response processes validity
Definition
the process the respondents use when filling out the measure matches what the psychologist thinks they are using
Term
Trait/state change validity
Definition
a trait measure is stable, a state measure is changeable
Term
Consequential validity
Definition
the measure is used in an ethically-appropriate manner
Term
Validation requires
Definition
intuition, theory, and data; it is a matter of degree rather than an all or nothing property, and is an unending process
Term
CONTENT VALIDITY
Definition
The adequacy with which a specified domain of content is sampled
Term
Content validity rests upon
Definition
a theoretical appeal to the correctness of content and the way it is presented
Term
Generalization
Definition
the extent to which one can generalize from a particular collection of items to all possible items in the construct
Term
Face Validity
Definition
the extent to which the test taker, or a casual observer, feels the instrument measures what it is intended to measure; this is after an instrument is constructed, and has only minor relevance to content validity (done before construction)

Often underestimated; probably the most important problematic aspect of IQ tests
Term
PREDICTIVE VALIDITY
Definition
Using an instrument to estimate some criterion behavior that is external to the measuring instrument itself, the size of the correlation directly indicates the predictive validity
Term
Predictive validity is determined by
Definition
the correlation between predictor and criterion
Term
connection between predictor and criterion
Definition
Courts of law are increasingly demanding some logical connection between predictor and criterion because of the issue raised of cultural bias; however, no amount of theory can substitute for a lack of correlation between predictor and criterion
Term
(range restriction)
Definition
something may occur to eliminate or minimize relevant differences on the predictor or criterion
Term
A test may have good ______ validity but not have good _____ validity
Definition
construct
predictive
Term
Predictive validity coefficient
Definition
the correlation between the predictor test and the criterion variable, which specifies the degree of validity of that generalization
Term
Predictive validity coefficients based on
Definition
a single predictor rarely exceed .3 to.4; people are too complex to permit a highly accurate estimation of their proficiency in most performance related situations, and it's actually remarkable that predictor tests correlate as highly with criteria as they do
Term
Things affecting predictive validity
Definition
• Reliability of measures creating attenuated correlations
• Restriction of range, often on criterion
• Proportion of dichotomous criterion variable, or having only a few target cases
• Time: correlations decrease as time elapses between the measures of the predictor and the criterion.
• Types of events: single events difficult to predict, aggregated events easier
Term
Predictive validity for diagnostic tests
Definition
Sensitivity
Specificity
• Measures are often better at one than the other
o A highly sensitive measure, when negative, rules OUT disorder
o A highly specific measure, when positive, rules IN a disorder
Term
concurrent vs predictive validity
Definition
Concurrent validity is not the same as predictive validity because there is no future prediction of a criterion; in concurrent validity the correlation is between the predictor test and another measure of the same construct in the same data collection
Term
A construct
Definition
something that psychologists theorize as existing that does not exist as an observable dimension of behavior; it is abstract and latent (hidden) rather than concrete and observable
Term
Three aspects of construct validation
Definition
• Specifying the domain of observables related to the construct (outlining the construct)
• Determining the extent to which observables tend to measure the same thing, several different things, or many different things from empirical research and statistical analyses
• Performing subsequent individual differences studies and/or experiments to determine the extent to which supposed measures of the construct are consistent with "best guesses" about the construct
Term
The adequacy of a construct's outline is tested by
Definition
determining how well the measures of observables "go together" empirically (intercorrelate); if you are investigating one thing, you want your items to vary in the same pattern
Term
Internal consistency
Definition
is necessary, but not sufficient, for construct validity; Cronbach's alpha is our best measure of internal consistency among items
Term
construct validity
Definition
The degree to which to measures are affected similarly by a variety of experimental treatments defines their similarity; when a variety of measures behave similarly over a variety of experimental treatments, it becomes meaningful to speak of them as measuring the same construct
involves both theory and correlations
Term
Factor analytic/reliability process of construct validity
Definition
• If all the proposed measures or items correlate highly with one another, it can be concluded that they all measure the same thing; keep investigating the construct
• If the measures or items tend to split up into clusters such that the members of a cluster correlate highly with one another and correlate much less with the members of other clusters, they measuring number of different things; split the construct
• A third possibility is that the correlations among the measures or items are all near zero, so that they measure different things and there is no meaningful construct unity; consider a new construct
Term
The danger of construct validity
Definition
is circular logic
1. Tests 1 and 2 correlate (a validity coefficient)
2. According to widely accepted theory, Constructs A and B correlate
3. Test 1 is a measure of Construct A
4. Test 2 is a measure of Construct B
Only Hypothesis 1 can be tested directly; it is necessary to infer the truth of the other hypotheses from this test; the paradigm for determining construct validity is invalid from an inductive standpoint, it cannot be completely proven with data
Term
concurrent validity
Definition
Constructs should correlate highly with similar constructs, but it is not automatic that the construct name that motivated the research is appropriate; these correlations are validity coefficients, sometimes called concurrent validity
What is done in practice is to assume that at least two of hypotheses 2 through 4 are correct; an empirical test of hypothesis 1 then allows for a valid inference about the remaining hypothesis

Alternatively, Tests 1 and 2 may have both been designed to measure Construct A
Term
The danger of circular logic can be lessened by
Definition
• Restricting investigations of construct validity to those situations in which some of the hypotheses are very probable
• Correlating supposed measures of constructs where the domain of one has previously been both well defined and highly restricted
Term
Campbell and Fiske's Construct Validation: Multitrait-Multimethod Matrix
Definition
Reliability and validity as points along a continuum rather than as sharply distinguished ideas, since each involves degrees of agreement between measures
• Validation is typically convergent because it is concerned with demonstrating that two independent methods of inferring an attribute lead to similar ends
• A measure should have divergent validity in the sense of measuring something different from existing measures, and should not correlate to an extremely high degree with other measures, especially of different constructs
• A measure is defined by both the content of its measured attribute (what it's measuring), and its method (how it's measuring it)
• At least two attributes, each measured by at least two methods, are required (a minimum of 4 measures)
Term
Reliability correlations
Definition
the extent to which a measure is internally consistent.
o F & B call these the monotrait-monomethod correlations
o In parentheses
Term
Trait correlations
Definition
the relationship between two measures of the same attribute using different methods.
o F & B call these the monotrait-heteromethod correlations
o In bold
Term
Method correlations
Definition
the relationship between two measures that share a common method but assess different attributes
o F & B call these the heterotrait-monomethod correlations
o In the solid-line triangles
Term
Neither correlation
Definition
the relationship between different attributes using different methods
o F & B call these the heterotrait-heteromethod correlations
o In the dashed-line triangles
Term
reliability vs trait vs method vs neither correlations
Definition
One expects the reliability correlations to have the highest values, and the “neither” correlations to have the lowest; construct validation demands that trait correlations be relatively high to reflect convergent validity, and that method correlations are relatively low to reflect divergent validity
Term
Reification
Definition
treating a term as if it denotes a real entity or process; this has caused many problems in science
Term
interpretation of constructs
Definition
It would be good if words denoting constructs were altered as evidence is obtained about the relevant sets of observables, but this is unfortunately not done as frequently as it should be
Term
Nunnally & Bernstein’s commonsense point of view of interpretation of constructs
Definition
• If a measuring instrument produces interesting findings over the course of numerous investigations, and fits the construct name applied to it:
o Continue using the instrument.
o Continue using the construct name referring to it.
• If the resulting evidence is poor, ask:
o Is instrument is worth fixing?
o Does the instrument really fit the name of the attribute used to describe it?
• If it is not possible to find sets of variables or items that relate to the construct, the construct itself should be questioned
Supporting users have an ad free experience!