They gave a lot of IAAI talks! WOW! I gave only one, from our joint work with GM.

It looks like they were able to make expert systems useful.

Our work on KnowledgeTool might be equally useful!

After all, it appears that their work was stuck with C, and our work on KnowledgeTool was based on, was a pre-processor to -- sorry to mention this -- PL/I. But, gee, they based on C, so mentioning PL/I should not be so bad!

Our use of KnowledgeTool was to monitor server farms and networks. There in particular we were trying to do problem detection. So, in expert systems,

When I see A and B and
5 or more cases of C
without a D, then it
looks really bad --
raise an alarm.

and, thus, have staff investigate.

And this statement of a symptom, test, for something wrong, was supposed to come from operator experience. So, broadly the idea of KnowledgeTool for that work was not to find better ways to monitor but just to encode what operators had already learned, mostly just from experience.

Hmm .... In such monitoring, we have two ways to be wrong, (1) a false alarm where we say that the system is sick when it is well and (2) a missed detection where we say that a system is well when it is sick.

So, we want the false alarm rate, to be called the probability of Type I error, and the missed detection rate, to be called the probability of Type II error.

Quite broadly there is a trade-off between these two rates: If we accept a higher false alarm rate, then we stand to get a lower missed detection rate.

Not all means of detection are equivalent: Some poor detectors have really high rates for both false alarms and missed detections, and some good detectors have really low rates on both of the errors.

A problem with expert systems was that we had no idea what rates we were getting or if our detectors were poor or good.

Commonly at least in principle we can adjust the rate of false alarms. That is good because (1) a false alarm sends the bridge staff on wild goose chasing and (2) false alarm rate too high is a standard complaint.

Looking at the problem in a little more detail, we easily had wide, deep rapidly flowing oceans of data. So, we could easily have data on 1000 variables with the data for each variable arriving at 100 times a second. So, write some expert system rules? No thanks!

So, I dreamed up a solution, basically, right, as guess from Type I/II, a statistical hypothesis test but one that is both distribution free and multi-dimensional, maybe the first such. False alarm rate can be set in advance of collecting data, and is in a useful sense for each selected false alarm rate has the detection rate the highest possible (for that false alarm rate).

I did this work to improve on expert systems for monitoring. Yes, I published the work, in Information Sciences.

Ah, beating expert systems! What's in the paper is some applied math based on some advanced pure math prerequisites, especially an idea from ergodic theory. What's in the paper really SHOULD be deployed. I suspect that more could be done.

Well, the OP starts with

> Developed at NASA’s Johnson Space Center from 1985 to 1996, the C Language Integrated Production System (CLIPS) is a rule-based programming language useful for creating expert systems and other programs where a heuristic solution is easier to implement and maintain than an algorithmic solution.

Gee, I thought that my "algorithmeic", really original mathematical statistics, was easier than doing as well with heuristics.