Machine Learning

Types of Machine Learning

• Supervised Learning
  • Regression Problem: Continuous valued output.
  • Classification Problem: Discrete valued output.
• Unsupervised Learning
  • Clustering

Linear Regression

Advanced Optimization Algorithms

There are advanced algorithms (from numerical computing) to minimize the cost function other than the gradient descent. For all of the following algorithms all we need to supply to the algorithm is a code to compute the function $J(\theta)$ (the cost function) and the partial derivatives of the cost function $\frac{\partial}{\partial \theta_i} J(\theta)$.

1. Conjugate gradient
2. BFGS
3. L-BFGS

Advantages

• No need to manually pick $\alpha$ (the learning rate in gradient descent)
• Often faster than gradient descent

Disadvantages

• More complex

Classification Problem

Cocktail Party Problem

• Algorithm
  • [W, s, v] = svd((repmat(sum(x.*x, 1), size(x, 1), 1).*x)*x');

$\log yh_\theta(x) + (1-y) \log (1-h_\theta(x))$ comes from Maximum Likelihood Method in Statistics