What is the time complexity of a single forward pass through a fully connected layer with N input neurons and M output neurons?
Show answer
O(N * M)
A forward pass in a fully connected (dense) layer involves a matrix-vector multiplication. Since every input is connected to every output, the number of operations is proportional to the product of the input and output dimensions.
Also asked as:
- What is the Big O time complexity for a forward pass in a dense layer with N inputs and M outputs?