Abstract:
A question generation means is developed for intelligent tutors wherein a question menu having dynamic questions is capable of being displayed after each step of a problem. The generated question set contains both elementary and advanced questions and changes after each step is performed, thereby enhancing the potential that a student using the tutorial understands every aspect of a problem.

Description:
SPECIFIC REFERENCE 
   The present invention claims priority of filing date established by provisional application Ser. No. 60/322,080, filed Sep. 14, 2001. 

   GOVERNMENT RIGHTS 
   This invention was made with the United States Government support under SBIR Grant No. DMI-0060379 from the National Science Foundation. The United States Government has certain rights in this invention. 

   BACKGROUND 
   1. Field of the Invention 
   The present invention relates generally to computer-implemented expert rules for artificial intelligence programs. In particular, a computerized method for generating questions applicable for intelligent tutorials is disclosed. In the present embodiment, a list of question-generating rules is added to an intelligent tutoring system, and the applicability of each rule in the list is tested upon every action that changes the state of the tutoring session. Students then using the tutorial are presented a menu of applicable questions that may be asked about each step taken, and accordingly ask at least one of the generated questions and get an answer. An intelligent tutoring system that is adapted to be enhanced by the present invention is that such computerized tutorial disclosed in U.S. patent application Ser. No. 09/769,910, now U.S. Pat. No. 6,540,520, the entire disclosure of which is herein incorporated by reference. 
   2. Description of the Related Art 
   Current tutoring programs routinely limit the problems and or questions available to the student to a fixed preset list, which rules out, for example, the possibility of providing tutoring on the teacher&#39;s own assignments. Typically, in non-intelligent tutorial software, the student only selects answers in multiple-choice form, and is told whether the selection is right or wrong. Several programs present a worked-out solution for study, but this does nothing to address the student&#39;s own mistake and the reasons behind them. 
   This has led to criticism of scripted “linear” instructional approaches that are too rigid and inflexible to support meaningful learning. There is a real need for a further dimension of interactivity. Fortunately, the approach used for constructing meaningful explanations, which is described by U.S. Pat. No. 6,540,520, is now taught. There now exists the opportunity for answering other useful questions about problems and their solutions. A way to leverage the benefit of the reasoning process of an artificial intelligence expert system is by exporting it in the form of various questions the system can answer. This can permit the student to ask questions about the situation even before they are ready to attempt the problems. Therefore, there is a need for a question-generating mechanism to be implemented into an expert system framework, wherein a student can ask dynamic, context-dependent questions about the problem or situation. 
   By interactively answering a variety of detailed questions for the student at each step, or at least providing that option, the particular pedagogical approach of the tutor is specifically oriented to help beginning and lower-performing students the most, who often cannot make any start on a problem, or do not feel comfortable attempting to do so. The ability of the tutor to answer questions makes it possible for students to conduct exploratory inquiry even before they can attempt the problems. At each step, questions are displayed in a menu the student can choose from as needed, selecting as many or as few questions as desired. Many different paths of inquiry are possible for the same problem, with the student directing the inquiry. The questions and answers are highly targeted and context-specific, changing at each step, and model good scientific thinking about the problem domain and underlying concepts. This is very important in fostering the development of the student&#39;s own self-explanation and question-asking abilities. 
   SUMMARY 
   Internally generated problem-solving information was used at each step of an externally supplied problem to construct a set of detailed questions the system can answer for the student, whereby one-on-one questions about the problem solving process may be address by the user. The problem can be accented from the student, an external program, or any other source. Generations of the questions was implemented in the expert system framework and when applicability conditions are met, an action is taken, wherein the action is the production of at least one question pertinent to the problem step or input. Question activation can be pedagogically based allowing for the possibility of assessment of whether a question is likely to be helpful or effective for the problem at hand, beyond simply whether the question is answerable. 
   Accordingly, the present invention generally is a computerized method for generating a question and answer dialogue for intelligent tutorials, comprising creating a question-generating rule class specific for a desired type of question (optionally from a base class), generating at least one question from said question-generating rule class provided an applicability condition is satisfied, and displaying said question in a user menu, wherein the user is permitted to direct an inquiry as to which question should be asked. 
   In operation then, in an intelligent tutor having a user interface including a display and a selection-device, a method for providing and selecting from a menu on said display is disclosed, wherein the method comprises retrieving a set of menu entries for the menu, each of the menu entries representing a question retrieved from a question-generating rule class; displaying the menu entries on the display; receiving a menu entry selection signal from the selection device pointing at one or more of the questions from the set of question-generating rules; and, in response to the signal, displaying an answer to the question, whereby a student using the intelligent tutor has directed an inquiry to a step of a problem. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a flow diagram representing the program flow for generating a dynamic question set. 
       FIG. 2  shows an example of the user menu that presents the generated-questions. 
       FIG. 3  shows an example of the dialogue produced as the computer is working the problem after the student asks the questions to get the details of each step. 
   

   The flow charts represent the program or logic flow that can be implemented on computer readable media tangibly embodying the program of instructions executable by standard, general-purpose computers, or which can be implemented as specialized devices. The present invention is not limited as to the type of computer on which it runs. The computer-typically includes an input device such as a touch-pad, mouse, and/or keyboard which may guide a selection device such as a pointer, and a display device such as a monitor. The computer also typically comprises a random access memory (RAM), a read only memory (ROM), a central processing unit (CPU), and a storage device such as a hard disk drive or a floppy disk drive. Embodied as computer-readable medium, the present methodology may reside in run-time medium such as the RAM or ROM, and in distribution media such as floppy disks, CD-ROMs, and hard disks capable of being executed by the computer or specialized device. 
   TERMS 
   Use of a “base class” as defined in the art is a common technique in the C++ programming language for specifying a common interface in this application for all of the rules. This technique is not central to the ability to implement the present methodology, thus other programming languages may be used. 
   “Consistency rules” refer to rules programmed to allow an educational response to an error even if an input is not anticipated or does not correspond to a specific misconception. 
   A “question-generating rule” is the rule that generates a question when an applicability condition is satisfied. 
   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
   Operationally, question-and-answer (Q&amp;A) capability can be implemented through a modified type of artificial intelligence (AI) rule used to generate the questions. In general, AI rules are typically programmed as logical implications:
         If A then B.
 
Here A represents a logical proposition defined on the current state of the tutoring session, for example including but not limited to the steps taken on the problem by the student. If proposition A is true, the logical condition  10  is satisfied and the rule is said to be applicable  20 . The implied action B can be carried out. In the present computerized methodology, a question-generating rule taken from a question-generating rule class  40  differs in its definition of the action B to be taken upon satisfaction of the applicability condition A. The differences of various rule types can be characterized as below in table 1:
       

   
     
       
             
             
             
           
         
             
                 
               TABLE 1 
             
             
                 
                 
             
             
                 
               Rule 
               Action 
             
             
                 
                 
             
           
           
             
                 
               Production Rule 
               A next step in solving the 
             
             
                 
                 
               problem is generated 
             
             
                 
               Consistency Rule 
               A constraint imposed on the 
             
             
                 
                 
               solution by a fundamental 
             
             
                 
                 
               principle is evaluated 
             
             
                 
               Question Rule 
               A question relevant to the 
             
             
                 
                 
               current state of the tutoring 
             
             
                 
                 
               session is generated 
             
             
                 
                 
             
           
        
       
     
   
   While all three rule types assess applicability by examination of the current state, the applicability decision for question rules can also contain purely pedagogical considerations. Such conditions can be included as needed as part of the definition of the applicability condition A for the rule, thus generating or producing this question  30 . 
   For example, in a chemistry tutor on balancing chemical equations, one reasonable question would concern how much of a certain product can be made from a given amount of a certain reactant. Though this question can always be answered from the balanced equation, for pedagogical considerations such as simplicity of illustration, it might be desirable that this question be generated only when there is a single reactant in the equation, in this case to avoid potential confusion or misconceptions involving the concept of limiting reagent which may be ancillary to the intended educational purpose of the question in this context. 
   In the present embodiment, a list of question-generating rules that make up the question-generating rule class  40  is added to the intelligent tutoring system, and the applicability or logical condition  10  of each rule in the list is tested upon every action that changes the state of the tutoring session, whereby a dynamic question set is implemented for the display  70  to the student. With use of the present computerized methodology, a student is permitted to direct the inquiry by selecting a question to ask  80  after the generation of the user menu shown in  FIG. 2 . 
   As shown, the questions are listed independently in a window, which window could be similar to a window found within a web page interface downloaded from a network such as the Internet. It should be understood that the questions generated by the rules can be assigned not only independently, but in categories (not shown), if desired. In this manner, the questions will be displayed in the user interface organized into these labeled categories. This is useful for presenting questions to the student organized according to the concepts studied, for example. 
   As can be seen, many different paths of inquiry are made possible for the same problem, and the system can answer questions and display the answer  90  with a thorough depth of coverage about its own work. The above menu is dynamic inasmuch as it is adapted to change after each input or next step performed  100 . Operationally and in conjunction with the flow chart, the mechanisms behind the dynamic question generation are demonstrated by the below example 1. 
   EXAMPLE 1 
   
     
       
             
           
             
             
           
             
           
             
             
           
             
           
             
             
           
             
           
             
             
           
             
           
             
             
           
             
             
           
         
             
                 
             
           
           
             
               Interface for creating question generating rule classes: 
             
             
               //Base class for question generating rules 
             
             
               class QuestionRule : public Proposition&lt;MCFTProblemContext&gt;{ 
             
             
               //QuestionRule is a specific type of logical proposition 
             
             
               public: 
             
             
               std:vector&lt;QandA&gt; getQandA(const Context&amp; C, const VariableMap&amp; 
             
             
               VM) 
             
             
               const; //Called to generate Q&amp;A from current context information 
             
             
               private: 
             
           
        
         
             
                 
               virtual void implQandA(std::vector&lt;QandA&gt;&amp; QA, const 
             
           
        
         
             
               Context&amp; C, const 
             
             
               VariableMap&amp; VM) const = 0;//Private function implementing Q&amp;A 
             
             
               generation for this rule-implementation of this function is different for 
             
             
               each rule 
             
             
               }; 
             
             
               Various concrete question-generating rule classes can be created 
             
             
               with this interface as derived classes: 
             
             
               class HowManyAtomsQuestionRule : public QuestionRule {//Concrete 
             
             
               rule class for questions about how many atoms are in a chemical formula 
             
             
               private: 
             
           
        
         
             
                 
               bool implA(const Context&amp; C) const; //Implementation of logical 
             
           
        
         
             
               applicability condition for this rule 
             
           
        
         
             
                 
               void implQandA(std::vector&lt;QandA&gt;&amp; QA, const Context&amp; C, 
             
           
        
         
             
               const   ? 
             
             
               VariableMap&amp; VM) const; //Implementation of Q&amp;A generation upon 
             
             
               satisfaction of logical applicability condition 
             
             
               }; 
             
             
               class FormulaMassQuestionRule public QuestionRule {//Concrete rule 
             
             
               class for questions about mass quantities associated with a chemical 
             
             
               formula 
             
             
               private: 
             
           
        
         
             
                 
               bool implA(const Context&amp; C) const; //Implementation of logical 
             
           
        
         
             
               applicability condition for this rule 
             
             
               void implQandA(std::vector&lt;QandA&gt;&amp; QA, const Context&amp; C const 
             
             
               VariableMap&amp; VM) const; 
             
             
               //Implementation of Q&amp;A generation upon satisfaction of logical 
             
             
               applicability condition 
             
             
               }; 
             
             
               After the question-generating rules are implemented, below is an 
             
             
               example of how they can be used: 
             
           
        
         
             
                 
               std::vector&lt;smart_ptr&lt;QuestionRule&gt; &gt; Rules; //Set up list of 
             
             
                 
               question-generating rules 
             
             
                 
               Rules.push_back(new HowManyAtomsQuestionRule); 
             
             
                 
               Rules.push_back(new FormulaMassQuestionRule); 
             
             
                 
               Rules.push_back(new PercentCompositionQuestionRule) 
             
             
                 
               Rules.push_back(new MoleConceptQuestionRule); 
             
             
                 
               Rules.push_back(new EmpiricalFormulaQuestionRule) 
             
             
                 
               . . . 
             
             
                 
               //After each user action, generate questions and answers 
             
             
                 
               for (int i = 0; i &lt; Rules.size( ); ++i) //For each rule in the system 
             
             
                 
               if(Rules[i]−&gt;applies(P)) //If the rule is applicable to the current 
             
             
                 
               state of tutoring session 
             
           
        
         
             
                 
               QandA::Add(Rules[i]−&gt;getQandA(P,VM)); //Generate the 
             
             
                 
               appropriate questions and answers 
             
             
                 
                 
             
           
        
       
     
   
   A question relevant to the state of the tutoring session is generated from context-specific and pedagogical factors and displayed for consideration as the user works on each step with the intelligent tutor. In this way, instead of step generation as output for a user&#39;s input as is known for production rule implementation, the action in the present methodology is a question, adapted to have its own particular answer. 
   The code snippet above for question generation depicts the case of the question-generating rules in a stand-alone list. In addition, question-generating rules can also be incorporated as part of the implementation of the other AI rule types in the system (production rules and consistency rules, see table 1) as a means of augmenting and enhancing the actions of these rules with question-and-answer capability. For example, within a production rule a question rule can be used to generate questions that can be answered for the student about a step the expert system took, and in a buggy rule (a production rule used to model anticipated student errors) questions can be provided that allow students to follow up by asking the system in-depth questions about specific errors they have made. 
     FIG. 3  demonstrates the dialogue produced as output by the program after the above questions are generated and prompted for use by the student. The example problem shown for  FIG. 3  is a problem involving the balancing of chemical equations. The present methodology is not limited to this particular type of problem and can be used, for example, with all content areas such as mathematical, measurement conversion, and ratio/proportion problems. The question generation adds to the depth of interaction between an intelligent tutor and the user by implementing this question-and-answer approach as an integral tool for asking both elementary and advanced questions during the learning process. 
   In conjunction with  FIG. 3  then, the table 2 below addresses what the program is doing after either each input from the student is received or the tutor has presented its own step, and why this is important. 
   
     
       
             
             
           
         
             
               TABLE 2 
             
             
                 
             
           
           
             
               1) 
               After balancing nitrogen, upon counting the oxygen atoms from 
             
             
                 
               left to right it might be thought that the two oxygen atoms 
             
             
                 
               from the two NO molecules “already” balance the two O atoms 
             
             
                 
               on the reactants side. Thus it might seem that oxygen can be 
             
             
                 
               successfully balanced by setting the coefficient of H 2 O to 
             
             
                 
               zero to eliminate the three “unnecessary” O atoms. The 
             
             
                 
               answer relates the interpretation of the notation to the 
             
             
                 
               representation of a definite chemical reaction, and directly 
             
             
                 
               connects to the meaning underlying the notation. 
             
             
               2) 
               Another step that seems to balance oxygen is setting the 
             
             
                 
               coefficient of NO to −1. This answer underscores that atoms 
             
             
                 
               and molecules are physical entities and points out the 
             
             
                 
               implication of this on the solution. It also emphasizes that 
             
             
                 
               balancing chemical equations is more than just solving an 
             
             
                 
               algebra problem or puzzle. As in (1), it shows that the 
             
             
                 
               reason for positive coefficients is chemical, not 
             
             
                 
               mathematical. Such “why didn&#39;t you” questions provide a 
             
             
                 
               particularly powerful way to model good cognitive strategies. 
             
             
               3) 
               This question gives a reminder of what the equation notation 
             
             
                 
               represents, and focuses attention on the chemical meaning, 
             
             
                 
               rather than on superficially manipulating symbols and 
             
             
                 
               numbers. 
             
             
               4) 
               This directly illustrates the mass conservation concept on 
             
             
                 
               the student&#39;s own problem. The fact that seemingly unrelated 
             
             
                 
               numbers add up to the same value is compelling evidence that 
             
             
                 
               something must be at work. If the student does not yet 
             
             
                 
               understand why this happens, he or she is primed by this 
             
             
                 
               demonstration for the explanation that follows. An important 
             
             
                 
               point is that this is a good example of a question the 
             
             
                 
               student is not likely to think of on his/her own, yet is 
             
             
                 
               beneficial for study. 
             
             
               5) 
               This question gives a timely reinforcement of why we would 
             
             
                 
               need to do this type of problem. 
             
             
                 
             
           
        
       
     
   
   The above examples show several useful attributes of the question-and-answer approach. It illustrates that many different paths of inquiry are made possible for the same problem, with the student leading the inquiry. In fact, the student can lead the inquiry even if the tutor itself is providing the step, instead of the student. This would be useful if the student does not know any steps that he or she can take in solving the problem, and they would rather watch the tutor perform the problem and ask questions as to why the steps are performed in such a manner.