What is the term for the value added to the denominator in Batch Normalization to prevent division by zero?
Show answer
Epsilon
Epsilon is a very small constant (e.g., 1e-5) used for numerical stability. In Batch Normalization, it is added to the variance before taking the square root to ensure the calculation remains defined even if the variance is zero.
Also asked as:
- In Batch Normalization, what is the name of the small constant added to the variance to ensure numerical stability?