Abstruct-An artificial neural network (ANN) is commonly modeled by a threshold circuit, a network of interconnected processing units called linear threshold gates. The depth of a circuit represents the number of unit delays or the time for parallel computation. The size of a circuit is the number of gates and measures the amount of hardware. It was known that traditional logic circuits consisting of only unbounded fanin AND, OR, NOT gates would require at least R(log nllog log n) depth to compute common arithmetic functions such as the product or the quotient of two n-bit numbers, if the circuit size is polynomially bounded (in n). It is shown that ANN'S can be much more powerful than traditional logic circuits, assuming that each threshold gate can be built with a cost that is comparable to that of AND/OR logic gates. In particular, the main results show that powering and division can be computed by polynomial-size ANN'S of depth 4, and multiple product can be computed by polynomial-size ANN'S of depth 5. Moreover, using the techniques developed here, a previous result can be improved by showing that the sorting of n n-bit numbers can be carried out in a depth-3 polynomial size ANN. Furthermore, it is shown that the sorting network is optimal in depth.
On the positive side, several arithmetic functions such as division and multiplication were shown to admit efficient implementations using threshold circuits of constant depth REF .
Depth efficient neural networks for division and related problems
IEEE Trans. Inf. Theory
IEEE Trans. Inf. Theory
Computer Science