Beware Ye Who Hunt Factor Analysis Monsters

In Psychology, most of our hypotheses are about relationships between variables, like happier people do less critical thinking and women are more trusting than men (gender predicting empathy). Our basic statistics like t-tests and correlations help us answer our questions. But what about our hypotheses about the underlying structure of things? Hypotheses like: people have a general level of intelligence inside them and we have personalities varying along five big dimensions. Charles Spearman (1904) developed factor analysis to help us test hypotheses like these. Let's intuitively explore how factor analysis works.

Let's sail about and try finding sea monsters. I'd rather not hunt them, because Friedrich Nietzsche (1886) cautions us we become the monsters we hunt. And because I'm a vegetarian. Besides, every sea monster I have ever met is so kind. Let's just count them. We sail along and stumble upon a scene.

How many sea monsters are we seeing?

If you're like most people, you see one sea monster. And you're correct. If we could see under the murky water, we'd see we're looking at one sea monster.

Let's keep sailing about and see if we can find more. We sail along and stumble upon a scene.

How many sea monsters are we seeing?

If you're like most people, you see two sea monsters. And you're correct. But before looking beneath the murky water:

which body parts we see are actually the same sea monster?

If we could see under the murky water, we'd see we're looking at two sea monsters and the middle body and head are really a single sea monster. But try reflecting to figure out how you knew that?

Let's keep sailing about and see if we can find more. We sail along and stumble upon a scene.

How many sea monsters are we seeing?

If we could see under the murky water, we'd see we're looking at three sea monsters. The tail, torso, and head are three different sea monsters.

Look back at our three observations, they each begin with the exact same scene above the water: a tail, torso, and head? But we concluded differently each time.

How did we know?

How could you tell the number of sea monsters when you could only see parts of them? You saw visible parts sometimes move together and others move independently; you did an intuitive correlation. For example, in our second observation, the torso and head moved with a perfect positive correlation but neither part had any correlation with the tail.

By looking at the correlations between all the observable and measurable parts (operational definition), we can infer something about their underlying nature (theoretical constructs)

Factor Analysis is a statistical method looking at how lots of different observations correlate. Through a lot of elaborate number crunching we typically have a computer do, we learn the number of theoretical constructs being observed, which we call factors. In our example, each sea monster is a factor because we could tell the underlying structure below the water even though we couldn't directly see.

Something powerful about factor analysis is we don't tell the computer how many factor we think there are. The computer tells us. Technically, we actually intuit the number of factors within a narrow range based on a scree plot and Eigenvalues our computer provides. We have a similar statistical technique called a structural equation model; essentially, we tell our computer how we believe items load and it tells us how closely our model fits. Sometimes it's called a confirmatory factor analysis and what we're discussing is called an exploratory factor analysis in contrast.

We still need to be cautious when evaluating a factor analysis because the quality of our result depends on the quality of the data we give the computer. It doesn't care if we're sloppy researchers, it just number crunches. Another caution is the computer can't tell us what each factor means. We put labels on each factor.

Here's an example of what a factor analysis looks like:

Orienting ourselves to a factor analysis, the computer gives us a table with each row representing an item we gave it and a column for each factor the computer identified. The computer provides factors from the one accounting for the most variance to those accounting for less. The numbers tell us how closely each item "loads" on the factor; it's just like a correlation between the item and the underlying factor.

In this particular example, I had 12 month old infants complete 9 little games: 4 elicited imitation (Bauer, 1995), 4 delayed match to sample (Diamond, 1991), and a new one I made up kinda' like both kinds of tasks. The tasks come from two different lines of research, but I wondered if they're really measuring the same thing. Notice how all the loadings (numbers) under factor one are positive correlations? I inferred all the tasks measure the same thing, what I suggest is working memory (Grobman, 2006). But the tasks aren't only loading on a single factor. Something else is underneath babies' performance. Notice how factor two has mostly near zero correlations except a few very positive and very negative loadings. Those tasks especially rely on different hands the baby uses for each task. So I infer factor two is measuring right- versus left-handedness. The third factor is hardest to interpret, but I believe it's telling us about inhibitory control. Notice the computer told me how many factors there are. I didn't think I was studying handedness! Notice too I decided what each factor means; maybe you disagree.

Factor analysis prominently features in psychological research, helping us understand underlying structures of very different phenomena.

Charles Spearman (1904) invented factor analysis to study intelligence. He found a single factor underlies every task we use our minds with, at least a little, and he calls this a general intelligence factor, or "g," Later, as computers became a tool, Raymond Cattell (1963) identified tasks with our minds load into two related factors he called crystallized intelligence (acquired knowledge) and fluid intelligence (quick pattern recognition in novel contexts). By the 1980's models of intelligence became much more varied and nuanced. Debate continues, both about if intelligence is a thing and if factor analysis is the best way to uncover it's structure; for example, Howard Gardner (1983) offers a totally different approach identifying multiple intelligences.

Another historical context factor analysis figures prominently is personality. Assuming any meaningful trait people differ along has a word for it (lexical hypothesis), Gordon Allport and Henry Odbert (1936) identified 4505 stable traits. Many researchers did factor analyses with the traits, most notably Warren Norman (1946) polished the lexical hypothesis into 2800 distinct traits words. Decades later, Lewis Goldberg (1990) identified a five factor model of personality traits we still use today, the Big Five. Our field came to a broad consensus the traits we've refined our items as much as possible, which is why it's become our standard model. Looking back, it's interesting to notice we put labels on what each factor means. For example, Goldberg called the fourth factor "emotional stability" but most studies today call it "neuroticism" and reverse score it. Also note debate never ends; a prominent growing competing model is the HEXICO, adding a sixth trait of humility and tweaking neuroticism into emotionality (Ashton & Lee, 2000).

We widely use factor analysis when designing measures. Practically any valid, reliable scale and sub-scale you find was created with factor analysis. I hope gaining an intuitive understanding of factors analysis helps you interpret psychology papers and maybe inspires you to explore creating your own measures.

Allport, G. W., & Odbert, H. S. (1936). Trait-names: A psycho-lexical study. Psychological Monographs, 47(1), i-171.

Ashton, M. C., & Lee, K. (2000). A theory of personality trait hierarchies and their structural bases. European Journal of Psychological Assessment, 16(3), 173-192.

Bauer, P. J. (1995). Development of explicit recall and the elicited imitation task in the second year of life: Implications for cognitive development. The Journal of Genetic Psychology, 156(2), 165-190.

Diamond, A. (1991). Neuropsychological insights into the meaning of object concept development. In S. Carey & R. Gelman (Eds.), The epigenesis of mind: Essays on biology and cognition (pp. 67-110). Lawrence Erlbaum Associates Publishers.

Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. Basic Books.

Gleitman, H. (1981). Psychology. W. W. Norton & Company.

Goldberg, L. R. (1990). An alternative “description of personality”: The Big Five factor structure. Journal of Personality and Social Psychology, 59(6), 1216-1229.

Grobman, K. H. (2003) How social Learning opportunities and individual differences in working memory capacity contribute to the development of domain general problem solving strategies during infancy [Dissertation, The Pennsylvania State University]

Nietzsche, F. (1886). Beyond good and evil: Prelude to a philosophy of the future. Penguin Classics.

Norman, W. T. (1946). Toward an adequate taxonomy of personality attributes: Replicated item-analysis factor-analytic comparison of four tests. Journal of Abnormal and Social Psychology, 41(3), 247-269.

Spearman, C. (1904). “General intelligence” objectively determined and measured. American Journal of Psychology, 15(2), 201-292.