Dropout.md (638B)
1 # Dropout 2 3 ML P604 4 5 **Definition:** Dropout is a regularization technique for deep neural networks where upon each pass every neuron has a constant probability of being 'dropped out' meaning the output is 0. 6 7 This works very well with a rate somewhere between 10%-50%. With RNNs we often do 20%-30% and with CNNs we use 40%-50%. 8 9 This form of regularization ensures the model has multiple neurons that perform similar functions instead of being dependent upon one or only a few neurons to do important things. 10 11 When using dropout we never drop output neurons. Additionally, it is common to only dropout neurons from the first few layers.