Abstract:
An interactive computer system and method collects, aggregates and distributes information derived from free-text responses to questions. The system and method collect free-text responses from a subject user and aggregates them with free-text responses from other users. The system and method then uses these free-text responses in learning methodologies (such as temporal spacing) and styles to facilitate long-term learning and knowledge retention.

Description:
RELATED APPLICATION(S) 
       [0001]    This application claims the benefit of U.S. Provisional Application No. 61/348,887, filed on May 27, 2010. The entire teachings of the above application are incorporated herein by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    Effective education should generate long-term improvements in physicians&#39; knowledge and practice patterns, not just short-term gains that quickly dissipate over time. Current methods of education often do not achieve this end. By clumping education into ‘bolus’ learning events without subsequent reinforcement, much of the material learned at lectures, seminars, hands-on practice sessions, and board-review courses will be quickly forgotten. This is not the fault of the learner—forgetting is a well-studied, natural physiological process that should be expected. 
         [0003]    Applicants have found that a new form of online education (termed in the art ‘spaced education’) that can substantially improve long-term retention of learning. Spaced education courses consist of a series of questions and explanations that are presented to learners via email on computers and mobile devices. Based on each learner&#39;s performance, the spacing, reinforcement, and content of the subsequent material adapts for that individual until mastery of the content is achieved. 
         [0004]    Spaced education is based on two core psychology research findings, the spacing and testing effects. The ‘spacing effect’ refers to the finding that information which is presented and repeated over spaced intervals is learned and retained more efficiently than information presented at a single time-point. The spacing effect appears to have a distinct neurophysiologic basis. A protein tyrosinephosphatase has been identified which regulates the spacing effect in Drosophila (Pagani, M. R., Oishi K., Gelb, B. D. et al.: The Phosphatase SHP2 regulates the spacing effect for long-term memory induction. Cell, 139: 189, 2009). In addition, spaced learning by rats was found to improve neuronal longevity in the hippocampus (a primary memory center in the brain). See Sisti, H. M., Glass, A. L., Shors, T. J.: Neurogenesis and the spacing effect: Learning over time enhances memory and the survival of new neurons. Learn Mem, 14: 368, 2007. By presenting material in a question-answer testing format, spaced education also harnesses the educational merits of the ‘testing effect’. The ‘testing effect’ refers to the psychological finding that the process of testing does not merely assess the knowledge levels of individuals. Rather, testing alters the learning process itself so that new knowledge is retained more effectively (Karpicke, J. D., Roediger, H. L., 3 rd : The critical importance of retrieval for learning. Science, 319: 966, 2008). 
         [0005]    To date, more than 10 large randomized trials have been conducted to show that spaced education improves knowledge acquisition, (Kerfoot, B. P., Kearney, M. C., Connelly, D. et al.: Interactive spaced education to assess and improve knowledge of clinical practice guidelines: a randomized controlled trial. Ann Surg, 249: 744, 2009) boosts learners&#39; abilities to self-assess their performance, and improves knowledge retention (Kerfoot, B. P.: Learning benefits of on-line spaced education persist for 2 years, J Urol, 181: 2671, 2009). In one trial of 1067 medical students at 4 medical schools, spaced education increased longer-term retention by 170%. In another with 95 clinicians at 8 medical centers, spaced education reduced inappropriate prostate cancer screening by 40% for more than a year after the intervention. In a trial of 724 urology residents across the US, spaced education improved long-term learning efficiency 4-fold and was preferred 3:1 over web-based teaching modules. 
         [0006]    Spaced education to date has utilized question types of the following formats: multiple-choice question (single correct answer), multiple correct answer, and fill-in-the-blank. These question formats are appropriate for questions that have a correct answer, but many important questions (e.g. in the fields of ethics, politics, religion, etc) do not have distinct correct answers. Given this, a new question format is needed that can accommodate a range of potential answers without the need for there to be a correct answer designated by the question writer. Applicants hypothesize that free-text question format would be ideal since it could capture a broad range of nuanced responses. Psychological research on the testing effect has also shown that questions utilizing non-cued free-text responses generate greater long-term retention of knowledge than cued-response question formats (Kang, S. H. K., McDermott, K. B., Roediger, H. L.: Test format and corrective feedback modify the effect of testing on long-term retention. European Journal of Cognitive Psychology, 19 528, 2007). 
       SUMMARY OF THE INVENTION 
       [0007]    The present invention addresses the above shortcomings of the art. The present invention provides a computer based method and system that enhances knowledge acquisition and retention using free-text format questions and responses. In particular, embodiments provide computer methods and systems of collecting, aggregating and distributing information derived from free-text responses to questions, including concepts, principles, facts, methods, plans, and the like. The computer based system and method of the present invention includes delivering a question to a user, recording the user&#39;s free-text response to the question, determining potentially-equivalent answers to the question, presenting a selected list of prior responses to the user, and having the user select which answer on the list is the best answer to the question. Users may include students, on-line users, chat room participants, and the like. The question may be delivered via electronic media such as an email, a facsimile, instant messaging, XML, web page, a posting, a chat room discussion, and the like. 
         [0008]    One novel aspect of the method and system is that it incorporates a computer system by which the free-text responses can be aggregated into meaningful summary data. This is accomplished by calculating (via a processor or computer) a list of potentially-equivalent responses to a user&#39;s response to the question, and then having the user determine whether his response is in fact equivalent to others on the list. The user is the best person to understand the nuances of his response and thus can provide a very effective means to determine shades of meaning. 
         [0009]    Another novel aspect of the invention method and system is that the system translates the summary response data into a multiple-choice question format. The user then selects the response that he feels is the best answer to that question. This method and system taps the collective wisdom of the users and generates a summary list of the best responses. 
         [0010]    In one embodiment, the system and method of the present invention provides the user with feedback and educational material to foster learning about the pertinent concepts. The present invention, by combining a free-text question-answer process with the provision of feedback, improves the learner&#39;s ability to learn, recall, and retain items of information. 
         [0011]    In another embodiment, the user is asked to report whether his selected answer from the calculated list is a correct answer to the question. This can be an important step, since the popularity of an answer does not necessarily correlate with its correctness. In embodiments, correctness can also be determined by one or more computers or one or more other people. 
         [0012]    In another embodiment, the spacing, difficulty, and/or content of subsequent questions and educational material is customized for each learner based on the correctness of his answer. 
         [0013]    While the primary focus of this method and system is to improve the educational process, it has not escaped Applicant&#39;s notice that this method and system is also an excellent means to aggregate free-text responses to questions. This is of particular value now that online search engines are aiming to be able to answer submitted questions, not just provide a list of relevant web sites. The method and system in this disclosure allow for the population of a database with the answers to questions, with weightings given to the answers based on the popularity and correctness of the answers. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention. 
           [0015]    The accompanying drawings illustrate an embodiment of the invention and depict the above-mentioned and other features of this invention and the manner of attaining them. In the drawings: 
           [0016]      FIG. 1  shows a flowchart illustrating a process of collection and distribution of free-text information in accordance with the present invention. 
           [0017]      FIG. 2  shows the presentation of a question to a user to which the user can submit a free-text response. 
           [0018]      FIG. 3  shows the presentation of a list of potentially-equivalent free text responses to the user, and asking him to select if one or more prior responses on the list is (are) equivalent to his response or if none on the list is equivalent. 
           [0019]      FIG. 4  shows the presentation to the user of the correct answer(s) to the question(s) and/or educational material to foster learning about the topic covered by the question. 
           [0020]      FIG. 5  shows the re-presentation of the question and a newly-calculated list of selected responses to the user and electronically recording which response on this list the user selects as the best response to the question. 
           [0021]      FIG. 6  shows the presentation to the user of a list of answer(s) to the question(s) and/or educational material to foster learning about the topic covered by the question. The user may be asked to determine whether his selected answer is correct or incorrect or whether the question has no correct/incorrect answer. 
           [0022]      FIG. 7  shows the presentation to the user of a list of answer(s) to the question(s) and/or educational material to foster learning about the topic covered by the question. 
           [0023]      FIG. 8  is a schematic view of a computer network in which embodiments of the present invention are deployed. 
           [0024]      FIG. 9  is a block diagram of a computer node in the network. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0025]    A description of example embodiments of the invention follows. The entire teachings of PCT Publication no. WO 2008/008370 published Jan. 17, 2008 for “Adaptive Spaced Teaching Method and System” are herein incorporated. 
         [0026]      FIG. 8  illustrates a computer network or similar digital processing environment in which the present invention may be implemented. An exemplary system  100  is shown and described. 
         [0027]    Client computer(s)/devices  50  and server computer(s)  60  provide processing, storage, and input/output devices executing application programs and the like. Client devices  50  may be, for example, a desktop or laptop computer, a portable media device, a personal digital assistant, a mobile communication device and the like. Client computer(s)/devices  50  can also be linked through communications network  70  to other computing devices, including other client devices/processes  50  and server computer(s)  60 . Communications network  70  can be part of a remote access network, a global network (e.g., the Internet), a worldwide collection of computers, Local area or Wide area networks, and gateways that currently use respective protocols (TCP/IP, Bluetooth, etc.) to communicate with one another. Other electronic device/computer network architectures are suitable. 
         [0028]      FIG. 9  is a diagram of the internal structure of a computer (e.g., client processor/device  50  or server computers  60 ) in the computer system of  FIG. 8 . Each computer  50 ,  60  contains system bus  79 , where a bus is a set of hardware lines used for data transfer among the components of a computer or processing system. Bus  79  is essentially a shared conduit that connects different elements of a computer system (e.g., processor, disk storage, memory, input/output ports, network ports, etc.) that enables the transfer of information between the elements. Attached to system bus  79  is I/O device interface  82  for connecting various input and output devices (e.g., keyboard, mouse, displays, printers, speakers, etc.) to the computer  50 ,  60 . Network interface  86  allows the computer to connect to various other devices attached to a network (e.g., network  70  of  FIG. 8 ). Memory  90  provides volatile storage for computer software instructions  92  and data  94  used to implement an embodiment of the present invention (e.g., system process  100  of  FIG. 1  and user interface operations of  FIGS. 2 through 7  and supporting code detailed below). Disk storage  95  provides non-volatile storage for computer software instructions  92  and data  94  used to implement an embodiment of the present invention. Central processor unit  84  is also attached to system bus  79  and provides for the execution of computer instructions. 
         [0029]    In one embodiment, the processor routines  92  and data  94  are a computer program product (generally referenced  92 ), including a computer readable medium (e.g., a removable storage medium such as one or more DVD-ROM&#39;s, CD-ROM&#39;s, diskettes, tapes, etc.) that provides at least a portion of the software instructions for the invention system. Computer program product  92  can be installed by any suitable software installation procedure, as is well known in the art. In another embodiment, at least a portion of the software instructions may also be downloaded over a cable, communication and/or wireless connection. In other embodiments, the invention programs are a computer program propagated signal product  107  embodied on a propagated signal on a propagation medium (e.g., a radio wave, an infrared wave, a laser wave, a sound wave, or an electrical wave propagated over a global network such as the Internet, or other network(s)). Such carrier medium or signals provide at least a portion of the software instructions for the present invention routines/program  92 . 
         [0030]    In alternate embodiments, the propagated signal is an analog carrier wave or digital signal carried on the propagated medium. For example, the propagated signal may be a digitized signal propagated over a global network (e.g., the Internet), a telecommunications network, or other network. In one embodiment, the propagated signal is a signal that is transmitted over the propagation medium over a period of time, such as the instructions for a software application sent in packets over a network over a period of milliseconds, seconds, minutes, or longer. In another embodiment, the computer readable medium of computer program product  92  is a propagation medium that the computer system  50  may receive and read, such as by receiving the propagation medium and identifying a propagated signal embodied in the propagation medium, as described above for computer program propagated signal product. 
         [0031]    Generally speaking, the term “carrier medium” or transient carrier encompasses the foregoing transient signals, propagated signals, propagated medium, storage medium and the like. 
         [0032]    Illustrated in  FIG. 1  is a flow diagram of a process of collection and distribution of free text information in one embodiment (system  100 ) of the present invention. At Step  1  a subject user of a device  50  receives a question provided by host web page server  60 . Email, text message or other electronic delivery techniques are employed. Optionally in Step  2  the user connects from an email application to the web page (server  60 ) hosting the question. Hyperlink or equivalent technology may utilized. In turn, the user answers the question with a free-text response through his electronic device  50  (at Step  3 ). Device  50  and server  60  communicate with each other over communications network  70  using known or common protocols and techniques.  FIG. 2  is illustrative of a user interface screen view implementing Steps  1 ,  2  and  3 . 
         [0033]    In Step  4 , server  60  calculates a list of potentially equivalent prior responses and presents this list to the subject user (through device  50 ) such as shown in  FIG. 3 . Identification of equivalent prior responses may be by a computer or with human intervention. This may be accomplished by keyword matching or other techniques. In Step  5 , through device  50  (in working communication with server  60 ), the subject user selects if one or more prior responses on the presented list ( FIG. 3 ) is/are equivalent to his response, or indicates if none on the presented list is equivalent. 
         [0034]    In response, server  60  at Step  6  then calculates a selected list of responses to the question. In one embodiment, this includes correct answer(s) to the question and/or educational material fostering learning about the topic covered by the question.  FIG. 4  is illustrative. 
         [0035]    In another embodiment, server  60  calculates the selected list of responses to the question by tallying the most popular answers to the question as indicated by the number of equivalent answers submitted by users. 
         [0036]    Step  7  (server  60  through device  50 ) then re-presents to the user the question and the calculated selected list of responses, such as in the illustrated screen view of  FIG. 5 . Step  8  (device  50 ) enables the user to select which response on the list as presented by Step  7  ( FIG. 5 ) is the best response to the question. This may be by device  50  (as supported by server  60 ) presenting to the user a screen view having a list of answers to the questions and/or educational material on topic as in  FIGS. 6 and 7 . Optionally, in one embodiment, the screen view may ask the user to determine whether his answer is correct or incorrect. 
         [0037]    In embodiments, server  60  and device  50  are configured at Step  7  so that the question and calculated selected list of responses to the question are re-presented to the user over spaced intervals of time until the user reaches a pre-established level of proficiency. The server  60 /device  50  determine the temporal spacing of the question re-presentation by whether the user indicates that his selected answer (Step  8 ) is correct or incorrect or whether the question has no correct/incorrect answer. 
         [0038]    In other embodiments, the temporal spacing of the question re-presentation is determined by whether one or more computers determines that the user&#39;s selected answer (Step  8 ) is correct or incorrect or whether the question has no correct/incorrect answer. 
         [0039]    In yet further embodiments, the temporal spacing of the question re-presentation is determined by whether one or more persons determines that the user&#39;s selected answer (Step  8 ) is correct or incorrect or whether the question has no correct/incorrect answer. 
         [0040]    Other techniques are suitable for determining the temporal spacing and hence delivery interval of the question/calculated selected list of responses to the question re-presentation of Step  7 . 
         [0041]    Server  60  calculates a spacing-based factor of a new one of the delivery intervals where the new delivery interval for delivery of a subsequent question is increased for questions that have been answered correctly, and the new delivery interval for delivery of a subsequent question is decreased for questions that have been answered incorrectly. 
         [0042]    When the user meets a pre-established level of proficiency on content of a given difficulty level, server  60  delivers subsequent questions that are of a greater difficulty level. To accomplish this, server  60  may calculate a difficulty-based factor of the new level of difficulty whereby the new level of difficulty of a subsequent question is increased for questions that have been answered correctly and the new level of difficulty of a subsequent question is decreased for questions that have been answered incorrectly. 
         [0043]    When the user meets a pre-established level of proficiency in a given content area, server  60  delivers subsequent questions that pertain to a new content area. To accomplish this, server  60  may calculate a content area-based proficiency factor whereby the subsequent question includes a new content area when the user meets a pre-established or threshold content area-based proficiency factor. 
         [0044]    In other embodiments, when the calculated new level of difficulty, the calculated new delivery interval, and the calculated new content area for a subsequent question are unchanged, server  60  provides the subsequent question at the same level of difficulty, delivered at the same delivery interval, and pertaining to the same content area as an initial question. 
         [0045]    When at least one of the calculated new level of difficulty, the calculated new delivery interval, and the calculated new content area for the subsequent question includes a change, server  60  provides at least one of a new level of difficulty, a new delivery interval, or a new content area for the subsequent question. 
         [0046]    Employing the above, invention system  100  electronically delivers the question to the user in a manner that adapts to the learning needs of the user using temporal spacing. In this way, a user with a lower baseline knowledge level in a content area receives a subsequent question more frequently than a user having a higher baseline knowledge level in the same content area. 
         [0047]    In embodiments, server  60  stores the responses to the questions and their metadata (including but not limited to popularity and equivalent answers) in a searchable electronic database (e.g., at  94  in  FIG. 9 ). For example, questions are internally represented by content (in free-text format) and characteristics such as level of difficulty, proficiency category (level), content area, key words and correct and equivalent answers (e.g., links or pointers thereto) and the like metadata. Answers are internally represented by content (e.g. in free text format) and characteristics such as keywords, best answer indicator, popular answer count, link/pointer to equivalent answers, and the like metadata. Various data structures (at  94  in  FIG. 9 ) are suitable for storing questions, their metadata/characteristics, answers and their metadata/characteristics. Similarly users may be internally represented in working memory  90 ,  95  by a user identifier, indication of operative proficiency level and working aspects such as current question, delivery time period (temporal spacing)/frequency, user&#39;s current submitted selections and free-text responses, prior responses, prior questions and the like metadata. Suitable data structures in data  94  are in the purview of one skilled in the art. 
         [0048]    Server  60  and client device  50  may iterate Steps  6 ,  7  and  8  in order to refine which response is effectively equivalent to the user&#39;s response and/or which response the user selects is the best response to the question. In embodiments, server  60 /device  50  repeat Steps  7  and  8  one or more times over spaced intervals of time, each time presenting the same calculated selected list of responses to the question from which the user selects the best answer. Or server  60 /device  50  repeat Steps  6 - 8  one or more times over spaced intervals of time, each time presenting a newly-calculated selected list of responses to the question from which the user selects the best answer. The first and/or newly-calculated selected list may present the most popular responses to the question. The first and/or newly-calculated selected list may present prior responses selected based on multiple variables, including but not limited to popularity, syntax, and key words. Known techniques are used to accomplish this. 
         [0049]    In other embodiments, the newly-calculated selected list may contain the same prior responses as when it was presented to the user previously. 
         [0050]    In accordance with the foregoing, system  100  collects the user&#39;s responses (in free-text) and aggregates them with other user&#39;s free-text responses. This results in information in a useful free-text format, especially for enhancing knowledge retention and learning as illustrated above throughout the various embodiments. 
         [0051]    The teachings of all patents, published applications and references cited herein are incorporated by reference in their entirety. 
         [0052]    While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims. 
         [0053]    For example, the foregoing describes a client-server architecture implementing one embodiment of the present invention. Other computer architectures, configurations and/or arrangements for implementing embodiments of the invention are suitable. 
         [0054]    Further, the foregoing refers to questions generally. One example are test items (test questions). Other types of questions and queries are suitable. 
         [0055]    In another example, the user&#39;s free text responses are limited to a threshold number of characters, such as 140 characters. 
         [0056]    In yet a further example, server  60  is a central processing component (host-server) of system  100 . The questions, responses and/or educational material are delivered on-line over a communications or other network  70 . 
         [0057]    In another example embodiment, server  60  employs a pre-determined interval of time to separate the initial presentation of the question for a free-text response by the user (Steps  1 ,  2 ,  3 ) and the re-presentation of the question and the selected list of responses of Steps  7  and  8 .