Which technique improves neural network training by normalizing layer inputs to have a mean of zero and a variance of one for each mini-batch?
Show answer
Batch Normalization
Introduced by Ioffe and Szegedy in 2015, Batch Norm allows for higher learning rates and makes the network less sensitive to initialization. It also has a slight regularization effect, sometimes reducing the need for Dropout.
Also asked as:
- What neural network training technique involves adjusting layer inputs to have zero mean and unit variance for each mini-batch?