Momentum.md (522B)
1 # Momentum 2 3 ML P580 4 5 **Definition:** Momentum optimization is an optimization algorithm that uses the idea of momentum to reach an optimum faster. 6 7 As we continue to have a negative gradient the optimizer moves faster and faster until it inverts where it then begins to slow down the gradient steps and subsequently change directions. 8 9 The gradient is used as an acceleration factor and not as the speed. 10 11 This is always faster than gradient descent and can also have a friction factor to reduce overshooting the target.