Introduction to Machine Learning

Definition and Types

Welcome to this detailed introduction to Machine Learning. This post explores the fundamental definitions, types of machine learning, and their mathematical representations.

What is Machine Learning?

• What is Machine Learning?
• What are the different types of Machine Learning?
• How can we mathematically define each type?

Definition of Machine Learning

Herbert Simon: Learning is any process by which a system improves performance from experience.

Tom Mitchell: A computer program learns from experience \(E\) with respect to a task \(T\) and a performance measure \(P\), if its performance at task \(T\), as measured by \(P\), improves with experience \(E\).

Mathematically, it is represented as:

\[ \text{Performance}(T) \propto f(E) \]

Supervised Learning

Given a dataset:

\[ D = \{(x_1, y_1), (x_2, y_2), …, (x_n, y_n)\} \]

The goal is to learn a function \(f(x)\) such that:

\[ f(x) \approx y \]

Example: Classification, Regression

Loss function:

\[ \mathcal{L} = \sum_{i=1}^{n} (y_i – f(x_i))^2 \]

Semi-Supervised Learning

Uses both labeled and unlabeled data. Given:

\[ D = D_{\text{labeled}} \cup D_{\text{unlabeled}} \]

Useful when labels are expensive to obtain.

Example: Speech recognition, Medical diagnosis.

Unsupervised Learning

No labeled data, goal is to find patterns.

Given dataset:

\[ D = \{x_1, x_2, …, x_n\} \]

Objective: Find a mapping function:

\[ g: X \to C \]

where \( C \) represents clusters.

Example: Clustering (K-Means, DBSCAN).

Reinforcement Learning

Based on agent-environment interaction.

At each time step \(t\), the agent:

• Observes state \(s_t\).
• Takes action \(a_t\).
• Receives reward \(r_t\).
• Moves to new state \(s_{t+1}\).

Goal: Maximize cumulative reward:

\[ R = \sum_{t=0}^{T} \gamma^t r_t \]

Example: Robotics, Game AI.

References

• Tom Mitchell (1997), Machine Learning.
• Sutton & Barto (2018), Reinforcement Learning: An Introduction.
• Andrew Ng, Coursera ML Course.
• Geoffrey Hinton, Deep Learning Lectures.

Thank You!

For more educational content, stay tuned!