Covariance Explained: Change of Origin vs. Scale Made Simple!
Welcome to PostNetwork Academy’s Data Science and AI Lecture Series! In this post, we’ll explore the mathematical concept of covariance and how it behaves under changes of origin and scale. Let’s break it down step by step.
Theorem: Covariance Independence
We aim to prove that:
- Covariance is independent of the change of origin.
- Covariance is dependent on the change of scale.
Covariance Formula
The covariance between two variables \(X\) and \(Y\) is given by:
$$
\text{Cov}(X, Y) = \frac{1}{n} \sum_{i=1}^n (x_i – \overline{X})(y_i – \overline{Y})
$$
Where:
- \(\overline{X}\) and \(\overline{Y}\) are the means of \(X\) and \(Y\), respectively.
- \(n\) is the number of data points.
1. Change of Origin
Let:
- \(u_i = \frac{x_i – a}{h} \quad \implies \quad x_i = a + h u_i\)
- \(v_i = \frac{y_i – b}{k} \quad \implies \quad y_i = b + k v_i\)
Here, \(a, b\) are constants for the origin shift, and \(h, k\) are scaling constants.
Substitution into the Mean:
For \(x_i\):
$$
\overline{X} = \frac{1}{n} \sum_{i=1}^n x_i = a + h \cdot \frac{1}{n} \sum_{i=1}^n u_i = a + h \overline{U}
$$
For \(y_i\):
$$
\overline{Y} = \frac{1}{n} \sum_{i=1}^n y_i = b + k \cdot \frac{1}{n} \sum_{i=1}^n v_i = b + k \overline{V}
$$
Substitution into the Covariance Formula:
The difference terms become:
$$
x_i – \overline{X} = h (u_i – \overline{U}), \quad y_i – \overline{Y} = k (v_i – \overline{V})
$$
2. Change of Scale
When substituting scaled terms into the covariance formula:
$$
\text{Cov}(X, Y) = \frac{1}{n} \sum_{i=1}^n h (u_i – \overline{U}) \cdot k (v_i – \overline{V})
$$
Simplification:
Factor out \(h\) and \(k\):
$$
\text{Cov}(X, Y) = hk \cdot \frac{1}{n} \sum_{i=1}^n (u_i – \overline{U})(v_i – \overline{V})
$$
Thus:
$$
\text{Cov}(X, Y) = hk \cdot \text{Cov}(U, V)
$$
PDF Presentation
CovOriginvsScalehttps://www.postnetwork.co/wp-content/uploads/2024/12/CovOriginvsScale.pdf
Video
Conclusion
- Covariance is independent of origin shifts (\(a, b\)).
- Covariance is dependent on scale factors (\(h, k\)).
Stay Connected!
Learn more at PostNetwork Academy or subscribe to our YouTube Channel.
Don’t forget to share this post if you found it helpful!