From: robert@cs.strath.ac.uk (Robert B Lambert)
Newsgroups: comp.ai.neural-nets
Subject: Correctness of NNs
Date: 24 Jul 92 15:47:11 GMT
Organization: Comp. Sci. Dept., Strathclyde Univ., Glasgow, Scotland.

This query is directed to Prof. Armstrong, although I would be interested in
opinions in general.

Coming from a vision background, I was puzzled after reading Prof. Armstrong's
posting on the reliability of neural networks. Surely to be able to state that
network X is 100% reliable, the entire input set must be known. From experience,
any pattern recognition task which has a fully defined input set with 
appropriate responses can most easily be solved with a look-up table. 

I had thought that the principal strength of neural networks (including the
brain) was the ability to form adaptive responses/rules based on a subset of 
all possible system inputs. If responses are wrong, a neural network has the
ability to correct itself, whilst improving its response rate on subsequent
new inputs.

With respect to safety in critical applications, how do you prove a system to
be correct? You first have to determine every possible input to that system.
A great deal of time and money has gone into developing tools for proving the 
correctness of software. The general opinion of the researchers in this field 
is that for practical real time applications, it is not feasible to prove 
software correct, no matter how critical the application. The same problem 
applies to computer hardware. Many companies will only use a particular 
component years after it has been introduced, assuming that any faults will 
have surfaced and been corrected in that time. The probability of an error is 
small, but exists.

It is never possible to eliminate errors from any real system no matter how
good it looks on paper. The current approach to this problem is redundancy.
Build a number of systems from different component running different software
and make sure they all produce the same response during use. Is this not one
of the strengths of NNs? If a cell fails or a connection is broken, the 
degradation of the response to each input is slight.

Neural networks, like fuzzy logic have the advantage of being able to produce
sensible responses to new inputs. In handwritten OCR for example, reading a 
character at a time be it by a human, ANN, or some other technique, must give a
recognition rate of less than 100%. Everyone writes differently and no system
could be trained on examples of every human beings writing. The error rate can 
be reduced if context is included, but not removed. If the NN is correctly 
designed, ambiguous characters can at least be highlighted and alternatives 
suggested with appropriate weightings.

When an ANN misclassified an input, how close is the generated response to the
correct response. From my experience the differences are small. Personally I
feel a lot happier with a pilot (with their NN which makes mistakes) flying an
air-craft rather than a computer. When the unforeseen event occurs, the pilot 
can make a choice based on experience which has a chance of being correct. If 
the computer is exposed to an unforeseen event, the air-craft will almost 
certainly crash.

----
Robert B Lambert
University of Strathclyde
Scotland, UK.

robert@cs.strath.ac.uk