Research Proposal: Foundation of Multiple Classifier Systems and Active Learning

Computational learning theory (COLT) research has made significant progress in our understanding of concept learning from sampled labeled instances. Although many efficient algorithms were constructed for learning various classes of concepts, these concept classes are extremely simple. Further, numerous hardness results seem to suggest that learning more expressive concept classes seems to be extremely difficult. Two research directions have evolved in an effort to address the difficulty of learning more expressive concepts. One is to construct multiple classifier systems (MCS) and the other is to consider the possibility of actively interacting with the environment (a.k.a. active learning). Currently, due to the success of Adaboost, theoretical research on MCS (and in fact machine learning in general) tends to focus rather narrowly on ensemble methods. Similarly, active learning research has been concentrated on the use of membership queries, and to some extent the teaching model. Other types of MCS and active learning that may lead to more intelligent learning systems have not been well studied. A good understanding of how to build more sophisticated MCS and exploit other possibilities of extracting information from the environment will move us one step closer to achieving the original intent of machine learning, which is to automate the knowledge acquisition process. There are many research problems to consider. The following is a list of some of the problems that this project will examine. The list is not intended to be exhaustive, but rather gives some concrete starting points that are illustrative of the type of research problems to be pursued. Other problems may also be studied in this project. This list also serves to demonstrate some of the work that the PI has done in MCS and active learning. The PI has previously investigated these problems, but many important issues remain to be investigated. Although the main approach in this research is that of COLT, empirical studies will be performed to test the efficacy of the algorithms and heuristics that are derived from the research. MCS 1: Learning When to Trust Which Experts: If we have a number of experts (or learning algorithms) each having its own strengths and weaknesses, how can a learner learn the conditions in which each expert’s predictions can be trusted? Such an understanding will give rise to better ways of combining the experts’ advice than simply taking a weighted vote as done in ensemble methods. MCS 2: Learning Multiple Intermediate Concepts: The target concept may be dependent on or correlated to some other related intermediate concepts. How do we exploit such dependencies and learn these intermediate concepts so that we can use them to enhance our approximation of the target concept? Although such situations occur quite often in practice, most research work concentrates on learning a single target concept. Active Learning 1: Learning from Data with Unspecified Attribute Values: Almost all work in COLT assumes that the attribute values of the instances are totally specified. However, it is common in practice that the instances are often partially specified. How can a learner learn in the presence of unspecified attribute values where an instance can be classified as ‘undeterminable’ (based only on those attributes with specified values)? Active Learning 2: Learning by Observing an Expert Perform the Classification Task: Instead of having a teacher who actively teaches the learner, how can a learner learn by observing an expert perform the classification task? This provides a better alternative than the current teaching model which assumes that the student is unwilling to learn (so as to circumvent the “outright-encoding” problem).