Theory of Generalization - How an infinite model can learn from a finite sample. The most important theoretical result in machine learning.

Lectures:- Play ►
### The Learning Problem

01:21:33Yaser Abu-MostafaThe Learning Problem - Introduction; supervised, unsupervised, and reinforcement learning. Components of the learning problem.

- Play ►
### Error and Noise

01:18:22Yaser Abu-MostafaError and Noise - The principled choice of error measures. What happens when the target we want to learn is noisy.

- Play ►
### Training Versus Testing

01:16:58Yaser Abu-MostafaTraining versus Testing - The difference between training and testing in mathematical terms. What makes a learning model able to generalize?

- Play ►
### The VC Dimension

01:13:31Yaser Abu-MostafaThe VC Dimension - A measure of what it takes a model to learn. Relationship to the number of parameters and degrees of freedom.

- Play ►
### Bias-Variance Tradeoff

01:16:51Yaser Abu-MostafaBias-Variance Tradeoff - Breaking down the learning performance into competing quantities. The learning curves.

- Play ►
### The Linear Model II

01:27:14Yaser Abu-MostafaThe Linear Model II - More about linear models. Logistic regression, maximum likelihood, and gradient descent.

- Play ►
### Neural Networks

01:25:16Yaser Abu-MostafaNeural Networks - A biologically inspired model. The efficient backpropagation learning algorithm. Hidden layers.

- Play ►
### Overfitting

01:19:49Yaser Abu-MostafaOverfitting - Fitting the data too well; fitting the noise. Deterministic noise versus stochastic noise.

- Play ►
### Regularization

01:15:14Yaser Abu-MostafaRegularization - Putting the brakes on fitting the noise. Hard and soft constraints. Augmented error and weight decay.

- Play ►
### Validation

01:26:12Yaser Abu-MostafaValidation - Taking a peek out of sample. Model selection and data contamination. Cross validation.

- Play ►
### Support Vector Machines

01:14:16Yaser Abu-MostafaSupport Vector Machines - One of the most successful learning algorithms; getting a complex model at the price of a simple one.

- Play ►
### Kernel Methods

01:18:19Yaser Abu-MostafaKernel Methods - Extending SVM to infinite-dimensional spaces using the kernel trick, and to non-separable data using soft margins.

- Play ►
### Radial Basis Functions

01:22:08Yaser Abu-MostafaRadial Basis Functions - An important learning model that connects several machine learning models and techniques.

- Play ►
### Three Learning Principles

01:16:18Yaser Abu-MostafaThree Learning Principles - Major pitfalls for machine learning practitioners; Occam's razor, sampling bias, and data snooping.

- Play ►
### Epilogue

01:09:28Yaser Abu-MostafaEpilogue - The map of machine learning. Brief views of Bayesian learning and aggregation methods.