Why is Covariance Bounded? The Power of Cauchy-Schwarz Inequality Data Science and A.I.

Why is Covariance Bounded? The Power of Cauchy-Schwarz Inequality

 

---

Covariance and Standard Deviation

Definitions:

1. Sample Covariance:
   \[
   \text{Cov}(X, Y) = \frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})(Y_i – \bar{Y})
   \]
2. Sample Standard Deviations:
   \[
   \sigma_X = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})^2}, \quad
   \sigma_Y = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (Y_i – \bar{Y})^2}
   \]

---

Cauchy-Schwarz Inequality

The Cauchy-Schwarz inequality states:

\[
\left( \sum_{i=1}^n A_i B_i \right)^2 \leq \left( \sum_{i=1}^n A_i^2 \right) \left( \sum_{i=1}^n B_i^2 \right),
\]

where \( A_i = X_i – \bar{X} \) and \( B_i = Y_i – \bar{Y} \).

This leads to:

\[
\left( \text{Cov}(X, Y) \right)^2 \leq \sigma_X^2 \sigma_Y^2
\]

or equivalently:

\[
|\text{Cov}(X, Y)| \leq \sigma_X \sigma_Y.
\]

---

Step-by-Step Example 1: Standard Deviations

Given Data:

\[
X = \{1, 2, 3, 4, 5\}, \quad Y = \{5, 4, 3, 2, 1\}
\]

1. Means:
   \[
   \bar{X} = 3, \quad \bar{Y} = 3
   \]
2. Standard Deviations:
   \[
   \sigma_X = \sigma_Y = \sqrt{\frac{1}{4} \sum_{i=1}^5 (X_i – 3)^2}
   \]
   Simplifying:
   \[
   \sigma_X = \sigma_Y = \sqrt{\frac{1}{4} \left[ (-2)^2 + (-1)^2 + 0^2 + 1^2 + 2^2 \right]} = \sqrt{\frac{10}{4}} \approx 1.58
   \]
3. Result:
   \[
   \sigma_X \cdot \sigma_Y = 1.58 \cdot 1.58 = 2.50
   \]

---

Step-by-Step Example 2: Covariance Calculation

Given Data:

\[
X = \{1, 2, 3, 4, 5\}, \quad Y = \{5, 4, 3, 2, 1\}
\]

1. Covariance Formula:
   \[
   \text{Cov}(X, Y) = \frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})(Y_i – \bar{Y})
   \]
2. Substitute Values:
   \[
   \text{Cov}(X, Y) = \frac{1}{4} \left[ (-2)(2) + (-1)(1) + (0)(0) + (1)(-1) + (2)(-2) \right]
   \]
3. Simplify:
   \[
   \text{Cov}(X, Y) = \frac{1}{4} \left[ -4 – 1 + 0 – 1 – 4 \right] = \frac{-10}{4} = -2.50
   \]
4. Result:
   \[
   |\text{Cov}(X, Y)| = 2.50, \quad \sigma_X \cdot \sigma_Y = 2.50
   \]
   Thus:
   \[
   |\text{Cov}(X, Y)| \leq \sigma_X \cdot \sigma_Y
   \]

---

PDF Presentation

 

upperboundcov

Video

Conclusion

The Cauchy-Schwarz inequality guarantees that the covariance is always bounded by the product of the standard deviations:

\[
|\text{Cov}(X, Y)| \leq \sigma_X \sigma_Y.
\]

This result is fundamental in statistics, ensuring the relationship between variance, standard deviation, and covariance is mathematically consistent.

---