Correlation and Regression

Correlation measures how strongly two continuous variables move together in a straight line, using the coefficient r from −1 to +1. Regression goes further: it fits an equation, such as ŷ = b₀ + b₁x, that predicts the output Y from one or more inputs X and estimates how much Y changes per unit of X.

In Six Sigma both are Analyze-phase tools for testing a suspected Y = f(X) relationship. Neither proves cause and effect on its own.

An original worked example with 12 illustrative orders, computed in scipy. The points lie close to the fitted line, and the residuals show no pattern, so a straight-line model is adequate.

Reading a scatter diagram

Always plot before you calculate. A scatter diagram puts the suspected cause (X) on the horizontal axis and the effect (Y) on the vertical axis. Look for direction (up or down), form (straight or curved), strength (tight band or loose cloud), and unusual points such as outliers, clusters, or a few influential values at the extremes. A strong curve can have a correlation near zero, and a single outlier can create or hide a correlation, which is why the plot comes first.

What does the correlation coefficient measure?

Pearson’s r = Σ(x − x̄)(y − ȳ) / √[Σ(x − x̄)² · Σ(y − ȳ)²]. It is unitless and runs from −1 (perfect negative line) through 0 (no linear relationship) to +1 (perfect positive line).

\|r\|	Common reading
0.9 to 1.0	Very strong linear relationship
0.7 to 0.9	Strong
0.4 to 0.7	Moderate
Below 0.4	Weak or none (check for a curve)

The coefficient of determination R² = r² is the share of the variation in Y explained by the linear relationship with X. Whether a correlation is statistically different from zero depends on n: test it with t = r√(n − 2)/√(1 − r²) on n − 2 degrees of freedom. For ranked or non-linear but monotonic data, use Spearman’s rank correlation.

Correlation is not causation. Two variables can move together because both are driven by a third (for example, ambient temperature), because of time trends, or by coincidence. In Six Sigma, confirm a cause with process knowledge, a controlled change, or a designed experiment.

How does simple linear regression work?

Least squares chooses the line that minimizes the sum of squared vertical distances (residuals) between the points and the line:

Slope b₁ = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)²
Intercept b₀ = ȳ − b₁x̄
Standard error of the regression s = √[Σ(y − ŷ)² / (n − 2)], the typical size of a prediction error

Test whether the slope is real with t = b₁ / SE(b₁) on n − 2 degrees of freedom, which is a hypothesis test of H₀: β₁ = 0. Report a confidence interval for the slope, and use a prediction interval, not just the fitted value, when you predict an individual outcome.

Worked example: pick time and order size

A warehouse team records order size (lines, X) and pick time (minutes, Y) for 12 orders. Regression gives:

ŷ = 4.57 + 0.782x: about 47 seconds per extra line, plus about 4.6 minutes of fixed time per order
r = 0.975, R² = 0.950: order size explains about 95% of the variation in pick time
Slope t = 13.8, p ≈ 7.7e-08; 95% confidence interval for the slope 0.66 to 0.91 minutes per line
s = 1.36 minutes. For a 15-line order, the prediction is 16.3 minutes, with a 95% prediction interval of about ±3.1 minutes

The fixed 4.6 minutes per order is an improvement target in itself, for example travel to the first pick face, and the slope tells you what batching or slotting changes could save per line.

Residual analysis

A regression is only as good as its assumptions, and residuals (e = y − ŷ) are how you check them:

Residuals vs fitted values or X: should be a random band around zero. A curve means the relationship is not linear (add a squared term or transform); a funnel means non-constant variance (transform Y).
Normal probability plot of residuals: should be roughly straight, which matters for intervals and tests (see normality). In the example, Shapiro-Wilk p = 0.90.
Residuals vs time order: runs or cycles mean the observations are not independent.
Unusual points: large standardized residuals (beyond about ±2) are outliers; points with extreme X values have high leverage and can pull the line. Check influence, for example with Cook’s distance.

Multiple regression

Multiple regression models Y from several X’s: ŷ = b₀ + b₁x₁ + b₂x₂ + … Each coefficient is the change in Y per unit of that X with the other X’s held constant. Use adjusted R², which penalizes extra terms, to compare models, and look at each coefficient’s p-value. Watch for multicollinearity: when X’s are correlated with each other, coefficients become unstable. A variance inflation factor (VIF) above about 5 to 10 is a warning. Categorical X’s enter as indicator (dummy) variables. For a pass/fail Y, use logistic regression instead.

Logistic regression for a pass/fail Y

When Y is binary (leak/no leak, defective/good, churned/stayed), a straight line can predict probabilities below 0 or above 1, and the errors are not normal. Binary logistic regression models the log-odds of the event as a linear function of the X’s:

ln[p / (1 − p)] = b₀ + b₁x₁ + b₂x₂ + … so p = 1 / (1 + e^{−(b₀ + b₁x₁ + …)})

The coefficients are estimated by maximum likelihood rather than least squares. Each coefficient is tested with a Wald z = b / SE(b), and the whole model with the likelihood-ratio statistic G = 2(LL_model − LL_null) on chi-square. X’s can be continuous or categorical (as indicator variables), exactly as in multiple regression.

Worked example: seal leaks and sealing temperature

Five pouches were sealed at each of eight temperatures from 150 °C to 185 °C and leak-tested, for 40 pouches and 11 leakers. Data are illustrative and were fitted by iteratively reweighted least squares, with temperature centered at 165 °C:

Original chart. Dots are the observed share of leaking pouches at each temperature, and the curve is the fitted model.

Term	Coefficient	SE	z	p
Constant (at 165 °C)	-1.081	0.446	-2.42	—
Temperature − 165	-0.1272	0.0460	-2.77	0.0056

Model test: G = 2(-17.86 − (-23.53)) = 11.33 on 1 df, p = 0.0008. Temperature clearly matters. McFadden’s pseudo-R² = 0.24, which is typical for individual pass/fail outcomes and should not be compared with linear-regression R².
Predicted leak probability: 70% at 150 °C, 39% at 160 °C, 15% at 170 °C, 5% at 180 °C.
Setting the leak rate to 5% and solving gives T ≈ 165 + [ln(0.05/0.95) − (-1.081)] / (-0.1272) ≈ 180 °C, inside the tested range. This makes it a candidate operating window to confirm, together with the upper limit at which the film is damaged.

Reading the coefficients: odds ratios

e^b is the odds ratio for a one-unit increase in X. Here e^-0.1272 = 0.881 per °C, or 0.53 per 5 °C (95% CI 0.34 to 0.83). Each 5 °C increase roughly halves the odds of a leak. An odds ratio of 1 means no effect, and a confidence interval that excludes 1 is significant. Odds are not probabilities: the same odds ratio moves the probability most in the middle of the curve and least near 0 or 1.

Check the fit as well. Compare observed and predicted counts in groups (Hosmer-Lemeshow), look for influential points, and keep at least about 10 events (here, leaks) per term in the model, or the estimates become unstable.

Ordinal and nominal (multinomial) logistic regression

Nominal (multinomial) logistic regression handles Y with three or more unordered categories, for example defect type: scratch, dent or stain. One category is the reference, and the model fits M − 1 logits, each comparing a category with the reference, with its own intercept and coefficients. This lets an X raise the odds of scratches but not dents. Fitting them together is more efficient than running separate binary models.
Ordinal logistic regression handles ordered categories, such as a 1–5 customer rating or minor/major/critical severity. The usual proportional-odds model models cumulative events (“category 1”, “category 2 or lower”, …), with a separate intercept (cut point) for each but a single coefficient per X. That shared coefficient is an assumption: the X must shift every cumulative odds by the same factor. Test it (for example with a test of parallel lines) and fall back to a nominal model if it fails. Software differs in the sign convention (some write b₀ − bx), so check which direction a positive coefficient means before interpreting it.

Logit versus probit

Both link a probability to a linear predictor. Logit uses the logistic function, and probit uses the standard normal cumulative distribution, p = Φ(b₀ + b₁x). The curves are almost the same shape, so they nearly always give the same conclusions. On the seal data, the log-likelihoods are -17.86 (logit) and -17.92 (probit), and the logit slope is 1.76 times the probit slope (about 1.6 to 1.8 is typical). Choose logit when you want odds ratios, which are the norm in quality and medical work. Choose probit when the event is naturally a threshold on a normally distributed latent variable, as in dose-response and reliability, where it is the tradition.

Frequently asked questions

What is the difference between correlation and regression?

Correlation gives one number for the strength and direction of a linear relationship and treats X and Y symmetrically. Regression fits an equation that predicts Y from X, gives the size of the effect in real units, and supports prediction intervals.

What is a good R-squared?

It depends on the use. For a physical process with good measurement, R² above about 0.8 is common. For human or market behavior, 0.3 may still be useful. A high R² does not guarantee a correct model, so always check the residuals.

Can I use regression to predict outside the range of my data?

Be careful. The straight-line relationship is only supported within the range of X you observed. Extrapolating beyond it assumes the same relationship continues, which often fails.

Does a significant correlation prove that X causes Y?

No. It shows association. Confounding variables, time trends, or reverse causation can produce the same pattern. Use process knowledge and controlled changes or a DOE to confirm causation.

When should I use logistic regression instead of linear regression?

Use it whenever Y is a category rather than a measurement: binary (pass/fail) for binary logistic regression, ordered categories for ordinal logistic regression, and unordered categories for nominal logistic regression. Linear regression on a 0/1 Y can predict impossible probabilities and gives invalid tests.

Last updated: November 11, 2026 · Published by Lean Six Sigma Training Guide (sixsigmatrainingfree.com)