What is the name of the theorem stating that a feed-forward network with a single hidden layer and a nonpolynomial activation function can approximate any continuous function on a compact set to arbitrary accuracy?

Universal Approximation Theorem

This theorem provides the mathematical justification for why neural networks are such versatile tools, proving they can approximate any continuous mapping given enough neurons. While it proves that a single hidden layer is theoretically 'universal,' in practice, deep architectures with many layers are usually more efficient at solving complex problems. It essentially confirms that neural networks are limited by data and training time rather than their inherent mathematical capacity.