ex5.m
implement regularized linear regression and use it to study models with different bias-variance properties.
- Plot Data (in ex5data1.mat)
- Compute Regularized Linear Regression Cost
lambda: 1, theta: [1 ; 1]
- Compute Regularized linear regression gradient
lambda: 1, theta: [1 ; 1]
- Train linear regression and plot fit over the data
lambda: 0
- Comput train error and cross validation error for linear regression
lambda: 0
training error: evaluate the training error on the first i training examples (i.e., X(1:i, :) and y(1:i))
cross-validation error: evaluate on the entire cross validation set (Xval and yval).
- Plot learning curve for linear regression
Since the model is underfitting the data, we expect to see a graph with “high bias”
- Map X onto Polynomial Features and Normalize
X_poly(i, :) = [X(i) X(i).^2 X(i).^3 … X(i).^p]
- Train Polynomial regression and plot fit over the data
- Comput train error and cross validation error for polynomial regression
lambda: 0
training error: evaluate the training error on the first i training examples (i.e., X(1:i, :) and y(1:i))
cross-validation error: evaluate on the entire cross validation set (Xval and yval).
- Plot learning curve for polynomial regression
Since the model is overfitting the data, we expect to see a graph with “high variance”
- Test various values of lambda and compute error
- Plot validation curve
use validation curve to select the “best” lambda value
the best value of lambda is around 3
