Why is Covariance Bounded? The Power of Cauchy-Schwarz Inequality
Covariance and Standard Deviation
Definitions:
- Sample Covariance:
\[
\text{Cov}(X, Y) = \frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})(Y_i – \bar{Y})
\] - Sample Standard Deviations:
\[
\sigma_X = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})^2}, \quad
\sigma_Y = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (Y_i – \bar{Y})^2}
\]
Cauchy-Schwarz Inequality
The Cauchy-Schwarz inequality states:
\[
\left( \sum_{i=1}^n A_i B_i \right)^2 \leq \left( \sum_{i=1}^n A_i^2 \right) \left( \sum_{i=1}^n B_i^2 \right),
\]
where \( A_i = X_i – \bar{X} \) and \( B_i = Y_i – \bar{Y} \).
This leads to:
\[
\left( \text{Cov}(X, Y) \right)^2 \leq \sigma_X^2 \sigma_Y^2
\]
or equivalently:
\[
|\text{Cov}(X, Y)| \leq \sigma_X \sigma_Y.
\]
Step-by-Step Example 1: Standard Deviations
Given Data:
\[
X = \{1, 2, 3, 4, 5\}, \quad Y = \{5, 4, 3, 2, 1\}
\]
- Means:
\[
\bar{X} = 3, \quad \bar{Y} = 3
\] - Standard Deviations:
\[
\sigma_X = \sigma_Y = \sqrt{\frac{1}{4} \sum_{i=1}^5 (X_i – 3)^2}
\]
Simplifying:
\[
\sigma_X = \sigma_Y = \sqrt{\frac{1}{4} \left[ (-2)^2 + (-1)^2 + 0^2 + 1^2 + 2^2 \right]} = \sqrt{\frac{10}{4}} \approx 1.58
\] - Result:
\[
\sigma_X \cdot \sigma_Y = 1.58 \cdot 1.58 = 2.50
\]
Step-by-Step Example 2: Covariance Calculation
Given Data:
\[
X = \{1, 2, 3, 4, 5\}, \quad Y = \{5, 4, 3, 2, 1\}
\]
- Covariance Formula:
\[
\text{Cov}(X, Y) = \frac{1}{n-1} \sum_{i=1}^n (X_i – \bar{X})(Y_i – \bar{Y})
\] - Substitute Values:
\[
\text{Cov}(X, Y) = \frac{1}{4} \left[ (-2)(2) + (-1)(1) + (0)(0) + (1)(-1) + (2)(-2) \right]
\] - Simplify:
\[
\text{Cov}(X, Y) = \frac{1}{4} \left[ -4 – 1 + 0 – 1 – 4 \right] = \frac{-10}{4} = -2.50
\] - Result:
\[
|\text{Cov}(X, Y)| = 2.50, \quad \sigma_X \cdot \sigma_Y = 2.50
\]
Thus:
\[
|\text{Cov}(X, Y)| \leq \sigma_X \cdot \sigma_Y
\]
PDF Presentation
upperboundcov
Video
Conclusion
The Cauchy-Schwarz inequality guarantees that the covariance is always bounded by the product of the standard deviations:
\[
|\text{Cov}(X, Y)| \leq \sigma_X \sigma_Y.
\]
This result is fundamental in statistics, ensuring the relationship between variance, standard deviation, and covariance is mathematically consistent.