Covariance Explained: Change of Origin vs. Scale Made Simple!

Covariance Explained: Change of Origin vs. Scale Made Simple!

Welcome to PostNetwork Academy’s Data Science and AI Lecture Series! In this post, we’ll explore the mathematical concept of covariance and how it behaves under changes of origin and scale. Let’s break it down step by step.


Theorem: Covariance Independence

We aim to prove that:

  • Covariance is independent of the change of origin.
  • Covariance is dependent on the change of scale.

Covariance Formula

The covariance between two variables \(X\) and \(Y\) is given by:

$$
\text{Cov}(X, Y) = \frac{1}{n} \sum_{i=1}^n (x_i – \overline{X})(y_i – \overline{Y})
$$

Where:

  • \(\overline{X}\) and \(\overline{Y}\) are the means of \(X\) and \(Y\), respectively.
  • \(n\) is the number of data points.

1. Change of Origin

Let:

  • \(u_i = \frac{x_i – a}{h} \quad \implies \quad x_i = a + h u_i\)
  • \(v_i = \frac{y_i – b}{k} \quad \implies \quad y_i = b + k v_i\)

Here, \(a, b\) are constants for the origin shift, and \(h, k\) are scaling constants.

Substitution into the Mean:

For \(x_i\):

$$
\overline{X} = \frac{1}{n} \sum_{i=1}^n x_i = a + h \cdot \frac{1}{n} \sum_{i=1}^n u_i = a + h \overline{U}
$$

For \(y_i\):

$$
\overline{Y} = \frac{1}{n} \sum_{i=1}^n y_i = b + k \cdot \frac{1}{n} \sum_{i=1}^n v_i = b + k \overline{V}
$$

Substitution into the Covariance Formula:

The difference terms become:

$$
x_i – \overline{X} = h (u_i – \overline{U}), \quad y_i – \overline{Y} = k (v_i – \overline{V})
$$


2. Change of Scale

When substituting scaled terms into the covariance formula:

$$
\text{Cov}(X, Y) = \frac{1}{n} \sum_{i=1}^n h (u_i – \overline{U}) \cdot k (v_i – \overline{V})
$$

Simplification:

Factor out \(h\) and \(k\):

$$
\text{Cov}(X, Y) = hk \cdot \frac{1}{n} \sum_{i=1}^n (u_i – \overline{U})(v_i – \overline{V})
$$

Thus:

$$
\text{Cov}(X, Y) = hk \cdot \text{Cov}(U, V)
$$

PDF Presentation

CovOriginvsScale

https://www.postnetwork.co/wp-content/uploads/2024/12/CovOriginvsScale.pdf

Video

Conclusion

  • Covariance is independent of origin shifts (\(a, b\)).
  • Covariance is dependent on scale factors (\(h, k\)).

Stay Connected!

Learn more at PostNetwork Academy or subscribe to our YouTube Channel.

Don’t forget to share this post if you found it helpful!

©Postnetwork-All rights reserved.