Abstract:
A system of recollection using the search of records of the past, includes a database (DB) device storing records of individual&#39;s past events and things, a search computing device for making a search request in response to a user&#39;s request, a server computing device for searching the DB device in response to the search request to extract result data, and a presentation computing device for integrating and presenting the result data extracted from the server computing device by using multisensory elements.

Description:
CROSS-REFERENCE(S) TO RELATED APPLICATION(S) 
       [0001]    The present invention claims priority of U.S. Provisional Patent Application No. 61/142,667, filed on Jan. 6, 2009, which is incorporated herein by reference. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates to a system and method of recollection using the search of records of the past, which searches automatic or manual records of past events or things on a keyword or category basis and presents the results by various means, such as the senses of sight, hearing, touch, or smell in order to use them as the media for recollecting an individual&#39;s past. 
         [0003]    This work was supported by the IT R&amp;D program of MIC/IITA [2006-S-032-02, Development of an Intelligent Service Technology based on the Personal Life Log]. 
       BACKGROUND OF THE INVENTION 
       [0004]    People tend to recollect their past more easily through media associated with past events and things, such as photos, music, smells, and so on. For example, things used in the past, music, data in various types of albums, or the like serve as media for recollecting specific memories of the past. 
         [0005]    In recent years, since the present is recorded and managed as personal data through various digital devices, such as cameras and camcorders, or life logging devices, people can more easily get present digital records that will be the past. 
         [0006]    With the recent development of technology, it is made easy to collect individual&#39;s record data. Thus, there is an ongoing study which applies the collected record data to a recollection method for enabling patients at the initial stage of Alzheimer&#39;s or aged people whose memory is falling to effectively recollect their past. 
       SUMMARY OF THE INVENTION 
       [0007]    It is an object of the present invention to provide a system and method of recollection using the search of records of the past which helps an individual to recollect their past by extracting required information by various searches from automatic or manual records of past events or things and presenting it through a sensory media. 
         [0008]    In accordance with an aspect of the present invention, there is provided a system of recollection using the search of records of the past, including: 
         [0009]    a database (DB) device storing automatic or manual records of individual&#39;s past events and things; 
         [0010]    a search computing device for making a search request in response to a user&#39;s request; 
         [0011]    a server computing device for searching the DB device in response to the search request from the search computing device to extract result data to the search request; and 
         [0012]    a presentation computing device for integrating and presenting the result data extracted from the server computing device by using multisensory elements. 
         [0013]    In accordance with another aspect of the present invention, there is provided a method of recollection using the search of records of the past, including: 
         [0014]    building a database (DB) device of a person to therapy by collecting data of records of the past of the person; 
         [0015]    requesting, at a search computing device, a server computing device to search for clues that can be used as media for helping the person subject to therapy to recollect his or her past; 
         [0016]    searching for, at the server computing device, result data of records of the past that is to be used for recollecting the past by searching the DB device of the person subject to therapy in response to the search request; 
         [0017]    providing, at the search computing device, the searched result data in the form of a list; and 
         [0018]    providing, at the server computing device, data corresponding to a record of the past selected from the list to a presentation computing device for presenting the data. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0019]    The above and other objects and features of the present invention will become apparent from the following description of preferred embodiments, given in conjunction with the accompanying drawings, in which: 
           [0020]      FIG. 1  illustrates a block diagram of a system of recollection using the search of records of the past in accordance with an exemplary embodiment of the present invention; and 
           [0021]      FIG. 2  provides a flow chart showing a procedure of a recollection therapy application to which the present invention is applied. 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
       [0022]    Hereinafter, an exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings. The exemplary embodiment of the present invention describes a system and method of recollection which search automatic or manual records of past events or things on a keyword or category basis and presents the results by various means, such as the senses of sight, hearing, touch, or smell in order to use them as the media for recollecting the past of a person subject to therapy. 
         [0023]    Referring now to  FIG. 1 , there is shown a block diagram of a system of recollection using the search of records of the past in accordance with an exemplary embodiment of the present invention. The inventive system includes a database (DB) device  100 , a server computing device  200 , a search computing device  300 , and a presentation computing device  400 . 
         [0024]    The DB device  100  includes a media DB  101  that stores media data representing records of past events and things of an individual by the senses of sight, hearing, touch, and smell and a metadata DB  102  that stores information about individual records, elementary events and individual objects, for example, metadata such as names, meanings, representative words and so on. Each of the data stored in each of the DBs  101  and  102  contains values for variables corresponding to time, locations, characters, activities, and sensible information, and is classified to categorize for each variable and managed by a DB controller. 
         [0025]    The server computing device  200  is composed of a DB interworking/query unit  201  and a streaming server unit  202 . The DB interworking/query unit  201  receives a query request from the search computing device  300 , searches required information from each of the DBs  101  and  102  in the DB device  100 , and then provides searched records of the past to the search computing device  300 . In some cases, a streaming service of the searched records of the past may be requested. In this case, the streaming server unit  202  plays the role of a streaming server, and the search computing device  300  or the presentation computing device  400  serves as a streaming client. 
         [0026]    The search computing device  300  provides a user interface through which the user can search records of the past and supports various search methods. To this end, the search computing device  300  includes a user interface unit  301 , a search unit  302 , and a presentation controller  303 . The user interface unit  301  enables a category-based search as well as a simple keyword-based search. Here, the categories are classified into time, locations, characters, activities, and sensible information. In some cases, the number of data contained in each category is provided to the user so that he or she can understand the distribution of data. In addition, the user interface unit  301  enables to perform a keyword search within the range of a category, and also supports a multiple keyword search, a multiple category search, and a cross search of the category search and the keyword search. The search unit  302  interworks the server computing device  200  to combine and deliver a search formula to the DB interworking/query unit  201 , and provides the searched records of the past to the user interface unit  301  by which the searched records of the past are easily presented. 
         [0027]    Further, the search computing device  300  provides a list of the searched records of the past which are in the form of representative words and representative images through the user interface unit  301  and the search unit  302 , retrieves data corresponding to events or things regarding a specific record of the past selected from the list of the searched past records by the user  500 , and transmits the retrieved data to the presentation computing device  400 . The presentation controller  303  in the search computing device  300  provides control command for controlling the retrieved data to be presented through the presentation computing device. The control command includes command signals to start, pause, and stop the presentation of the retrieved data, command signals to individually select the reproduction of visual, auditory, tactile, and olfactory data, and command signals to individually select presentation intensities of the visual, auditory, tactile, and olfactory data. 
         [0028]    The presentation computing device  400  is composed of a device management unit  401 , an integration/synchronization control unit  402 , and a streaming client unit  403 . The streaming client unit  403  receives the retrieved data from the server computing device  200 , and the integration/synchronization control unit  402  receives control command from the search computing device  300 . Here, the retrieved data is data associated with the events or things regarding the specific search result selected by the user  500 . The retrieved data includes media data representing the senses of sight, hearing, touch, and smell, and metadata about the integrated presentation and synchronization of the media data. Further, the presentation computing device  400  serves to as a streaming client, and the device management unit  401  performs the function of controlling and managing the operations of devices for multisensory presentation, for example, a visual device  600 , an auditory device  700 , an olfactory device  800 , and a tactile device  900 . 
         [0029]    The DB device  100 , the server computing device  200 , the search computing device  300 , and the presentation computing device  400  have a scheme to operate in conjunction with one another via wired and wireless networks for enabling data transmission and reception between them. 
         [0030]    The system of the present invention so configured will be explained below based on an application scenario employing a recollection therapy for patients at the initial stage of Alzheimer&#39;s. 
         [0031]    First, a database is built which records basic information about a patient subject to therapy and the past of the patient. As to the recording of the past, records of events or things taking place in everyday life are continuously collected by a logging device provided to the patient, or data is collected by using a separate logging device, such as a camera and camcorder, when a specific event takes place or the user wants to record. For aged people, data may be collected by getting records of the past from albums, photographs, or the like and converting them into digital data. As the collected data are of a raw data that is not refined, they are sorted in units of records of the past by using an automatic or manual editing tool and significant data of record of the past alone are extracted. Then, for a subsequent search or data utilization, metadata containing representative words, images, search-based category information are created from the past record data. 
         [0032]    Through such a collection process of records of the past, the media DB  101  and the metadata DB  102  are prepared for the person subject to therapy, and the basic information about the person subject to therapy is also stored therein, thereby establishing the DB device  100 . 
         [0033]    Upon completion of these processes, a recollection therapy is performed on the person subject to therapy, a procedure of which will be described below with reference to  FIG. 2 . 
         [0034]      FIG. 2  provides a flow chart showing a procedure of a recollection therapy application to which the present invention is applied. 
         [0035]    As shown in  FIG. 2 , in step S 200 , when there is a patient subject to therapy, e.g., an Alzheimer&#39;s patient, the user  500  playing the role of a therapist, makes a request to search for information about the patient (e.g., profile search of the patient). That is, the definition of data desired to be requested is given by the user interface unit  301  and the search unit  301  in the search computing device  300 , and the user  500  and the DB device  100  interwork with each other through the DB interworking/query unit  201  and the DB control unit  103 . Next, required data is collected by a combination of proper queries and then the result to the queries is provided to the user  500 . 
         [0036]    Here, in the recollection therapy application, the user interface unit  301  provides an interface and information for people who can assist with the recollection therapy, like family and therapists, rather than for the patient. 
         [0037]    Thereafter, through the search computing device  300 , the user  500  searches for clues that can be used as media for helping the person subject to therapy to recollect his or her past. This process provides a search method of records of the past, following category-based and keyword-based classifications through the user interface unit  301  of  FIG. 1 . By this search method, a search keyword is entered and a category is selected in step S 202 . The categories are composed of time, locations, characters, activities, and sensible information. For example, the user  500  can search in the activity category by using a keyword ‘wedding’ and search in the sensible information category by using keywords ‘snow’ and ‘cold’. 
         [0038]    Next, in steps S 204 , the search unit  302  combines search keywords of the user  500  in a form suitable for query, and then transmits the same to the DB interworking/query unit  201 . In response thereto, the DB interworking/query unit  201  searches for past record data of the patient in the DB device  100  through the DB control unit  103 . 
         [0039]    And then, in step S 206 , the search results, i.e., a list of records of the past, is provided to the search unit  302 . At this time, the DB interworking/query unit  201  may perform a deduction and integration process of data by adding a recognition and AI (artificial intelligence) algorithm thereto, if necessary. Further, if there exist a plurality of persons subject to therapy, the DB device  100  may be implemented with separate devices as much. In this regard, the list of the search results is composed of representative images and representative words and is provided to the user  500  through the search computing device  300 . 
         [0040]    Subsequently, in step S 208 , when the user  500  selects one specific record of the past from the result list, the search unit  302  requests the server computing device  200  to search all data associated with selected record of the past. 
         [0041]    Then, in step S 210 , the DB interworking/query unit  201  receives all of the data from the DB device  100  in response to the request to search all data associated with selected record of the past. 
         [0042]    The server computing device  200  prepares to provide all the data provided from the DB device  100  to the client through the streaming server unit  202 . Thereafter, in step S 212 , the user  500  makes a request to present the data of record of the past. The presentation control unit  303  then delivers the type of media to be controlled and control commands to the integration/synchronization control unit  402  of the presentation computing device  400 , and receives the data of record of the past in a streaming fashion in conjunction with the streaming server unit  202  through the streaming client unit  403 . 
         [0043]    In step S 214 , the integration/synchronization control unit  402  controls the operations of the visual, auditory, tactile, and olfactory devices  600 ,  700 ,  800 , and  900 , which are multisensory devices, in response to control command from the presentation control unit  303  so as to achieve integrated presentation and synchronization of the past record data, and instructs the device management unit  401  to operate the multisensory devices individually. Accordingly, the device management unit  401  controls and operates the visual device  600 , the auditory device  700 , the tactile device  800 , and the olfactory device  900  individually to represent the past record data. 
         [0044]    Through this procedure, a clue serving as a medium of recollection is presented to a person subject to therapy, and the therapist repetitively performs this procedure partly to be consistent with the flow of therapy. 
         [0045]    In accordance with the embodiment of the present invention, it is possible to retrieve automatic or manual records of past events or things on a keyword or category basis and present the results by various means, such as the senses of sight, hearing, touch, or smell in order to use them as the media for recollecting the past of a person subject to therapy. As a result, the present invention can achieve the effect of enhancing an individual&#39;s memory of the past by retrieving records of the past of the individual in various ways and reproducing the corresponding records similarly through a multisensory presentation device so as to play the role of a more effective medium. 
         [0046]    Furthermore, the system of the present invention may be effectively used to remedy the symptom of an Alzheimer&#39;s patient by applying it to a recollection therapy. 
         [0047]    While the invention has been shown and described with respect to the particular embodiments, it will be understood by those skilled in the art that various changes and modification may be made without departing from the scope of the invention as defined in the following claims.