Imagine a financial analyst asked to assess the health of companies in one industry. Based on prior research, there are 25 financial ratios worth watching: liquidity, solvency, profitability, efficiency, and so on. Analyzing 25 numbers for every company, then comparing them across companies, is exhausting work, and the result is often confusing because many of those ratios are related to each other.
What if those 25 ratios could be summarized into three to five simpler indices, without losing the important information?
That is what Principal Component Analysis (PCA) and Factor Analysis offer. They are among my favorite methods because they always feel like a magic trick that can be explained: what looks complicated turns out to have a simpler structure behind it.
The core idea: many variables actually measure the same thing
Variables in a dataset often overlap. For example: income, monthly spending, vehicle ownership, and house size. Technically they are different, but they very likely move together because they all reflect one big thing: “level of affluence”. If we can summarize the four into one affluence index, our analysis becomes more compact and often more stable.
Some practical uses of these methods:
- Summarizing many variables into a few indices, like the financial ratios example above.
- Dealing with multicollinearity in regression. If several explanatory variables are so strongly correlated that they make regression estimation difficult, but all of them should still be used, we can form new variables (components) that are uncorrelated with each other, and use them in place of the original variables.
- Testing the structure of a questionnaire. For an instrument with many items, we want to know whether the items cluster according to the theoretical dimensions. This is part of construct validity testing.
- Visualization. Data with dozens of dimensions can be projected into two or three dimensions so that group patterns are easy to see.
PCA and factor analysis: similar, but not the same
They are often discussed together and often run from the same menu in software, but their purposes differ.
PCA is a data reduction technique. It forms new components that are linear combinations of the original variables, arranged so that the first component captures the largest variance, the second captures the largest remaining variance uncorrelated with the first, and so on. Its focus: summarizing total variance.
(Exploratory) factor analysis is a modeling technique. It assumes there are latent factors (unobserved concepts, such as “satisfaction” or “affluence”) that cause the variables we measure to be related. Its focus: explaining the correlations among variables through a few common factors, and separating common variance from each variable’s unique variance.
Rule of thumb: if your goal is to summarize data or build an index, PCA is enough. If your goal is to understand the structure of the concepts underlying questionnaire items, factor analysis is conceptually more appropriate. In many theses the two are mixed up, and a sharp examiner will ask about the difference.
Practical steps
Here is the framework I use, in SPSS or other software:
Step 1: Check whether the data is suitable. Two common tests: KMO (Kaiser-Meyer-Olkin), which measures sampling adequacy and ideally is above 0.50 (the higher the better), and Bartlett’s test, which tests whether the correlation matrix differs from an identity matrix (we want a significant result, because it means there are correlations among variables that can be summarized). If your variables are barely correlated with each other, there is nothing to summarize.
Step 2: Decide the number of components or factors. There are several criteria, and they are best used together: (a) eigenvalues greater than 1 (Kaiser criterion), (b) the scree plot, looking for the “elbow” where the curve starts to flatten, (c) the cumulative percentage of variance explained (a target of around 60% or more is often used in the social sciences), and (d) ease of interpretation. Do not just follow the eigenvalue criterion mechanically, because it tends to produce too many components.
Step 3: Rotate (if relevant). Initial results are often hard to interpret because many variables load on several components at once. Rotation (for example varimax) turns the axes so that each variable tends to load high on only one factor. This does not change the total variance explained, it only makes the pattern easier to read.
Step 4: Read the loading matrix. Loadings show how strongly a variable relates to a factor. As a benchmark, a loading of about 0.50 or more is usually considered meaningful in medium-sized samples, and an item that loads high on two factors at once (cross-loading) needs to be reconsidered.
Step 5: Name and interpret. Look at the variables that gather on one factor, then ask what they have in common. In the financial ratio example, one group might load liquidity ratios (“liquidity factor”), another group loads profitability ratios, and a third loads debt ratios.
Step 6: Use the scores. Each respondent’s component or factor scores can be saved and used in further analysis, such as regression, clustering, or comparisons across groups.
Common mistakes
- A sample that is too small. These analyses need an adequate sample. A rough, often-quoted guideline is several times the number of variables, and no fewer than about 100 respondents. Small samples produce unstable solutions.
- Including unsuitable variables. Nominal categorical variables are not suitable for this analysis in its standard form.
- Ignoring theory. If the analysis produces factors you cannot explain substantively, you are looking at a statistical artifact, not a real concept.
- Using exploratory factor analysis to prove a theoretical model. To test a structure already specified by theory, use confirmatory factor analysis.
- Deleting as many items as possible until the result looks tidy. If you delete half the questionnaire for a good structure, your instrument no longer represents the concept.
A simple workplace example
An HR manager has an engagement survey with 30 statements. Instead of reporting 30 scores to the board, she runs a factor analysis and finds that the items cluster into five dimensions: leadership, compensation, workload, career development, and communication. The report becomes five easy-to-read indices, and comparisons across divisions become more meaningful. That is the practical value of this method: not just a technique, but a way to make information manageable.
Having a lot of data is not the problem. Data that cannot be read is the problem. And with the right tools, a lot of data can be summarized into a clear story.
An example of reading a loading table
Imagine the varimax-rotated result for six financial variables produces two factors. The variables current ratio, cash ratio, and working capital have loadings of 0.86, 0.81, and 0.77 on Factor 1, but below 0.20 on Factor 2. The variables profit margin, ROA, and ROE have loadings above 0.80 on Factor 2 and below 0.15 on Factor 1. The pattern is clean: Factor 1 can be named ‘liquidity’ and Factor 2 ‘profitability’. If one variable, for example the debt ratio, loads 0.45 on both factors, it is ambiguous, and you need to decide whether to drop it, keep it, or review it theoretically. That decision must be explainable in the methods chapter.
Need help with factor analysis, PCA, construct validity, or summarizing a questionnaire with many items? The datalytics.indo team can help with everything from research design and questionnaires, through analysis (factor analysis, clustering, regression, CB-SEM and PLS-SEM, path analysis, and more), to interpretation. Private tutoring in SPSS and SmartPLS is also available. Let’s discuss your needs first. We’ll provide a tailored quote based on your project.