Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.

Gradient Descent

• Orthogonal to lines of equal cost.
  • Look for direction of steepest descent and take a step.
  • Can use a positive semi-definite matrix to move more carefully as the learning rate for more advanced mechanisms.
  • [[Stochastic Gradient Descent]]

— Kunal