Learning & Optimization: Gradient Descent & Back-Propagation

In the previous notes, we learned how data flows through a Feed-Forward Network to make a prediction. But when a network is first created, its weights are completely random. Its first prediction will be completely wrong.

So, how does the network adjust its weights to become accurate? It uses a combination of a Loss Function, Gradient Descent, and Back-Propagation.

1. The Loss Function (Measuring the Error)