Correlation & Covariance

Definition

Covariance measures how two variables move together; correlation scales it to −1…1.

Helps select features and spot multicollinearity.

Diagram

Flowchart

Example: Pearson correlation

x, y = [1, 2, 3, 4], [2, 4, 6, 8]
mx, my = sum(x) / 4, sum(y) / 4
cov = sum((a - mx) * (b - my) for a, b in zip(x, y))
sx = sum((a - mx) ** 2 for a in x) ** 0.5
sy = sum((b - my) ** 2 for b in y) ** 0.5
print(round(cov / (sx * sy), 3))

Output: 1.0

Key points

• Correlation is not causation
• Only captures linear relationships

Interview questions

More Machine Learning topics