Neural networks in J

This paper presents an attempt to implement an Artificial Neural System (ANS) in J, a significantly simplified and enhanced version of APL [1,2]. Neural networks are fairly straightforward to program in a language such as J. A software tool (based on C and J) for the development and simulation of neural networks is presented in this paper. It provides a rudimentary platform for implementing ANS systems. Introduction The purpose of this investigation was to illustrate how artificial neural networks can be implemented in J. Artificial neural networks are massively parallel interconnected networks of nonlinear elements in an hierarchical organization. They are intended to interact with objects of the real world in the same way as biological nervous systems do [3]. Each element is characterized by an internal threshold or offset q and by the type of nonlinearity. The output of an element (node) can be expressed in the following way: Xj=f(xwjixi-ei), (1) where ~ is the response function, w are the respective weights of the node interconnections, and q is the node threshold. Neural network models are specified by the net topology, node characteristics, and training or learning rules. These rules specify an initial set of weights and indicate how weights should be adapted during use to improve performance [4]. At present neural net models have the greatest potential in areas such as pattern recognition, knowledge databases for Permission to copy wittmul fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinefy. To copy otherwise, or to republish, requires a fee andlor specific permission 01992 ACM 0-89791 -478-3 92/0007-0218 $1.50 Neural Networks in J 216 stochastic information, optimization computations, robot control, and decision making. This research illustrates how a back propagation network can be implemented in J. Matrix Representation of Neural Networks The simplest form of back propagation occurs in networks which have a layer of input units at the bottom, any number of intermediate layers, and a layer of output units at the top. An input vector is presented to the network by setting the states of the input units. Then the states of the units in each layer are determined by applying equations (1) and (2) to the connections coming from the lower layers. All units within a layer have their states set in parallel, but different layers have their states set sequentially, starting at the bottom and working upwards until the states of the output elements are determined f (x) = 1 (2) -----------------1 +e”x The threshold is simulated by adding to each layer (except the output layer) an additional element whose output is always 1, and which is connected to every element of the next layer by means of connections whose weight is equal to the thresholds of these elements [5,6]. Therefore, a neural network with (nl) input nodes, (ncl) output nodes, (nhl) intermediate (hidden) layers, and hidden node distribution vector (nh) can be modeled by a complex structure (noun). It consists of (nhl+ 1) boxed arrays created by the verb “<”. The shape ($) of these arrays is (nb + I)x(nt), where nb is the number of elements in the bottom layer and nt is the number of elements in the next (top) layer. For example, if nl= :4, ncl=:2, nhl= :2, nh= :(3,2), then the neural network can be fully represented by the noun in Figure 1, where the first row is the threshold element.