Patent Document

CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims priority from Japanese Patent Application No. 2013-058982, filed on Mar. 21, 2013, the entire contents of which are hereby incorporated by reference. 
     BACKGROUND 
     1. Technical Field 
     The present disclosure relates to an information processing apparatus, an information processing method, and a computer readable medium. 
     2. Description of the Related Art 
     In the related art, in case of a presentation for merchandise explanation, a person who made the presentation would analyze the situation at the time of the presentation and record information such as what the person presented chiefly actually in materials used in the presentation or what clients paid attention to in the materials. 
     As apparatuses through which participants can easily confirm contents serving as essential parts in a presentation or the like, there have been conceived a pointer pointed at a material which is displayed on a screen by a projector, and an image processing apparatus in which such a pointer and the position thereof are detected and options to select the position of the detected pointer are displayed on a touch panel so that whenever an option is selected on the touch panel, a material corresponding to the position of the pointer associated with the selected option is displayed on the touch panel (for example, see JP-A-2010-261989). 
     A data retrieval apparatus holds objects displayed on a display device in association with information about the display positions of the objects. When a user then touches a touch panel and draws a retrieval region thereon, an object associated with the display position of the retrieval region is retrieved and displayed on the display device (for example, see JP-A-2012-008917). 
     In the related art, there has been considered a technique in which a position designated in response to an operation of a user on a screen where some information is displayed is stored in association with contents of information displayed at the position, so that the information of the contents corresponding to the position on the screen can be displayed in response to an operation which will be performed by the user. 
     In such a related art technique, however, of contents displayed in the past, only the information of contents the user paid attention to can be simply displayed again as it is, based on the position where the information of the contents the user paid attention to had been displayed. 
     Therefore, it is impossible to know which information drew attention among the information which had been displayed in the past. 
     SUMMARY OF THE INVENTION 
     One of illustrative aspects of the present invention is to provide an information processing apparatus and an information processing method, which can record and display attention keyword among display information displayed in the past, and a computer readable medium storing a control program for the information processing apparatus and the information processing method. 
     According to one or more illustrative aspects of the present invention, there is provided an information processing apparatus comprising: a display device comprising a touch panel configured to receive a user operation and detect a touch position where the touch panel is touched by the user operation, wherein display information is displayed on the display device; a display information storage unit configured to store the display information; an information display controller configured to display the stored display information on the display device; a touch position acquisition unit configured to acquire the detected touch position in response to the user operation on the display information displayed on the display device; a region setting unit configured to set a certain region in the display information based on the acquired touch position; a target information extraction unit configured to extract a target information from the certain region of the display information; a target information storage unit configured to store the extracted target information; and a target information display controller configured to display the stored target information on the display device. 
     According to one or more illustrative aspects of the present invention, there is provided an information processing method for controlling a computer comprising a display device comprising a touch panel configured to receive a user operation and detect a touch position where the touch panel is touched by the user operation, wherein display information is displayed on the display device. The method comprises: (a) storing the display information in a display information memory; (b) displaying the stored display information on the display device; (c) acquiring the detected touch position in response to the user operation on the display information displayed on the display device; (d) setting a certain region in the display information based on the acquired touch position; (e) extracting a target information from the certain region of the display information; (f) storing the extracted target information in a target information memory; and (g) displaying the stored target information on the display device. 
     According to one or more illustrative aspects of the present invention, there is provided a non-transitory computer-readable medium storing a program for causing a computer to perform predetermined operations, wherein the computer comprises a display device comprising a touch panel configured to receive a user operation and detect a touch position where the touch panel is touched by the user operation, wherein display information is displayed on the display device. The predetermined operation comprises: (a) storing the display information in a display information memory; (b) displaying the stored display information on the display device; (c) acquiring the detected touch position in response to the user operation on the display information displayed on the display device; (d) setting a certain region in the display information based on the acquired touch position; (e) extracting a target information from the certain region of the display information; (f) storing the extracted target information in a target information memory; and (g) displaying the stored target information on the display device. 
     Other aspects and advantages of the present invention will be apparent from the following description, the drawings and the claims. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a functional block diagram showing the configuration of a keyword extractor  10  according to an embodiment of an information processing apparatus according to the invention; 
         FIG. 2  is a view showing an example of a presentation screen  14   g  stored in a presentation material storage unit  14  of the keyword extractor  10 ; 
         FIG. 3  is a view showing user operation information stored in an operation information storage unit  15  of the keyword extractor  10 ; 
         FIG. 4  is a view showing touch operation region data stored in a touch operation region data storage unit  16  of the keyword extractor  10 ; 
         FIG. 5  is a view showing keyword data stored in a keyword data storage unit  19  of the keyword extractor  10 ; 
         FIG. 6  is a flow chart showing a keyword extraction process (first part) executed in accordance with a keyword extraction process program PR 1  of the keyword extractor  10 ; 
         FIG. 7  is a flow chart showing the keyword extraction process (second part) executed in accordance with the keyword extraction process program PR 1  of the keyword extractor  10 ; 
         FIG. 8  is a flow chart showing the keyword extraction process (third part) executed in accordance with the keyword extraction process program PR 1  of the keyword extractor  10 ; 
         FIGS. 9A and 9B  are views showing a specific display operation on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10 , wherein  FIG. 9A  is a view showing touch operation positions T 1  to T 3  touched by a finger, a touch operation position P 1  touched by a pen and an enlargement operation region Z 1  on the presentation screen  14   g  displayed on a display device  12 , and  FIG. 9B  is a view showing an enlarged display screen  14   g Z of the enlargement operation region Z 1 ; 
         FIG. 10  is a view showing a specific example in which touch operation information are obtained on the presentation screen  14   g  in  FIGS. 9A and 9B  in accordance with the keyword extraction process of the keyword extractor  10 ; 
         FIG. 11  is a view showing a specific example in which touch operation region information are created correspondingly to the touch operation information in  FIG. 10  respectively in accordance with the keyword extraction process of the keyword extractor  10  while expanded region information is created when the regions of the touch operation region information overlap with each other; 
         FIG. 12  is a view showing a specific example of a touch region image GT extracted from a touch operation region  16 Tn on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10 ; 
         FIG. 13  is a view showing a specific example of an enlarged region image GZ extracted from an enlargement operation region  16 Zn on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10 ; 
         FIG. 14  is a view showing specific examples of respective keywords extracted from the respective operation regions on the presentation screen  14   g  and registered in the keyword data storage unit  19  in accordance with the keyword extraction process of the keyword extractor  10 ; 
         FIG. 15  is a flow chart showing a keyword recording for editing and recording the keywords registered in the keyword data storage unit  19  by the keyword extraction process of the keyword extractor  10 ; 
         FIG. 16  is a view showing a keyword recording form Fk used in the keyword recording of the keyword extractor  10 ; 
         FIG. 17  is a flow chart showing a presentation material creation process for creating new presentation materials based on the keyword recording form Fk recorded by the keyword recording of the keyword extractor  10 ; and 
         FIG. 18  is a view showing a presentation material form Fp used in the presentation material creation process of the keyword extractor  10 . 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     Exemplary embodiments of the present invention will be described below with reference to the drawings. 
       FIG. 1  is a functional block diagram showing the configuration of a keyword extractor  10  according to an embodiment of an information processing apparatus according to the invention. 
     The keyword extractor  10  is implemented by an information instrument such as a tablet-type PDA or PC which will be described later, a projector system which can detect a position designated on a screen by a user, etc. 
     The keyword extractor  10  according to the embodiment is constituted by a tablet-type PDA. The keyword extractor  10  is provided with an overall controller  11 , a touch region controller  20 , a pen region controller  21 , an enlarged region controller  22 , and a character recognition controller  23 , which consist of a computer (CPU). 
     These respective controllers  11 ,  20 ,  21 ,  22  and  23  control operations of respective units in accordance with an apparatus control program PR stored in a not-shown memory such as a hard disk or an ROM. The apparatus control program PR is started up in accordance with an input signal from an input device  13  of a transparent touch panel, which is, for example, provided on top of a screen of a backlighted color liquid crystal display device  12 . 
     The input device  13  is a double-layered touch panel in which an electromagnetic induction type touch panel as a first touch panel and an electrostatic capacitance type touch panel as a second touch panel are provided on a display screen of the display device  12 . 
     The electromagnetic induction type touch panel detects an operation performed by a special pen generating a magnetic field on the display screen of the display device  12  based on an electromagnetic induction system. 
     The electrostatic capacitance type touch panel detects an operation performed by a finger on the display screen of the display device  12  based on an electrostatic capacitance system. 
     A keyword extraction process program PR 1  for extracting a keyword from a region designated in accordance with a touch operation performed by a finger, a touch operation performed by a pen or an enlargement touch operation on a presentation screen (image) displayed on the display device  12 , a keyword recording program PR 2  for editing and recording the extracted keyword on a predetermined keyword recording form Fk, a presentation material creation process program PR 3  for creating new presentation materials based on the recorded keyword, etc. are stored as the apparatus control program PR in addition to the system program for controlling the whole of the apparatus. 
     The keyword extractor  10  is provided with a presentation material storage unit  14 , an operation information storage unit  15 , a touch operation region data storage unit  16 , a pen operation region data storage unit  17 , an enlarged operation region data storage unit  18  and a keyword data storage unit  19 , whose storage regions are allocated in a not-shown memory such as a hard disk or an RAM. 
     A presentation material screen (presentation screen)  14   g  which is created in advance or which is created in accordance with the presentation material creation process program PR 3  is stored in the presentation material storage unit  14 . 
       FIG. 2  is a view showing an example of the presentation screen  14   g  stored in the presentation material storage unit  14  of the keyword extractor  10 . 
     Information about a position or a region on the presentation screen  14   g  in accordance with a touch operation performed by a user in the input device (touch panel)  13  is stored in the operation information storage unit  15  in accordance with the control of the overall controller  11 . 
       FIG. 3  is a view showing user operation information stored in the operation information storage unit  15  of the keyword extractor  10 . 
     A touch operation information address  15 T, an enlargement operation information address  15 Z and a pen operation information address  15 P are allocated in the operation information storage unit  15 . 
     Coordinates (X, Y)  15 T 1 ,  15 T 2  or the like of touch positions 1, 2 or the like designated by touch operation on the presentation screen  14   g  with a finger by the user are stored in the touch operation information address  15 T. In addition, coordinates (X, Y, SX, SY)  15 Z 1 ,  15 Z 2  or the like of enlarged regions 1, 2 or the like enlarged by multitouch-operation on the same presentation screen  14   g  with fingers by the user are stored in the enlargement operation information address  15 Z. In addition, coordinates (X, Y)  15 P 1 ,  15 P 2  or the like of pen touch positions 1, 2 or the like designated by touch operation on the same presentation screen  14   g  with a pen by the user are stored in the pen operation information address  15 P. 
     In the input device (touch panel)  13 , a touch operation performed by a finger is detected by the electrostatic capacitance type touch panel and a touch operation performed by a pen is detected by the electromagnetic induction type touch panel. 
     Incidentally, when the input device (touch panel)  13  consists of one layer of the electrostatic capacitance type touch panel, detection of a touch operation performed by a finger or detection of a touch operation performed by a pen may be determined based on whether the center (X, Y) of the touch position is detected with a spread not smaller than a threshold or not (or detected as a pin point or not). 
       FIG. 4  is a view showing touch operation region data stored in the touch operation region data storage unit  16  of the keyword extractor  10 . 
     Coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like indicating regions 1, 2 or the like of rectangular sizes set in advance correspondingly to the coordinates (X, Y)  15 T 1 ,  15 T 2  or the like of the positions 1, 2 or the like designated by the touch operation with the finger by the user in accordance with the control of the overall controller  11  and the touch region controller  20  are stored in the touch operation region data storage unit  16 . 
     Coordinates (X, Y, SX, SY) indicating regions 1, 2 or the like of rectangular sizes set in advance correspondingly to the coordinates (X, Y)  15 P 1 ,  15 P 2 , . . . of the positions 1, 2 or the like designated by the touch operation with the pen by the user in accordance with the control of the overall controller  11  and the pen region controller  21  are stored in the pen operation region data storage unit  17  in the same manner as the touch operation region data ( 16 ). 
     The coordinates (X, Y, SX, SY)  15 Z 1 ,  15 Z 2  or the like of the enlarged regions 1, 2 or the like enlarged by the multi-touch operation of the user in accordance with the control of the overall controller  11  and the enlarged region controller  22  are transferred to and stored in the enlargement operation region data storage unit  18  as they are with the same contents as those stored in the enlargement operation information address  15 Z (see  FIG. 3 ). 
     Characters in an image extracted from the presentation screen  14   g  in accordance with the coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like of the touch operation regions 1, 2 or the like stored in the touch operation region data storage unit  16 , or the coordinates (X, Y, SX, SY) of the pen touch operation regions 1, 2 or the like stored in the pen operation region data storage unit  17 , or the coordinates (X, Y, SX, SY)  15 Z 1 ,  15 Z 2  or the like of the enlarged regions 1, 2 or the like stored in the enlargement operation region data storage unit  18  are recognized in accordance with the control of the character recognition controller  23 . A character string consisting of the recognized characters is stored as a keyword in the keyword data storage unit  19 . 
     On this occasion, assume that a user using his/her finger or fingers to perform a touch operation or an enlargement operation on the presentation screen  14   g  displayed on the display device  12  is regarded as a presenting side user (for example, a “salesclerk”) and a user using the pen to perform a touch operation on the same presentation screen  14   g  is regarded as a presented side user (for example, a “client”) in the embodiment. 
     Therefore, for example, as shown in  FIG. 5 , character recognition results of images extracted from the presentation screen  14   g  in accordance with the coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like of the touch operation regions 1, 2 or the like performed by the finger and character recognition results of images extracted from the presentation screen  14   g  in accordance with the coordinates (X, Y, SX, SY)  15 Z 1 ,  15 Z 2  or the like of the enlarged regions 1, 2 or the like, are stored as presenting side (salesclerk) keywords  19 S 1 ,  19 S 2  or the like in the keyword data storage unit  19 . In addition, character recognition results of images extracted from the presentation screen  14   g  in accordance with the coordinates (X, Y, SX, SY) of the touch operation regions 1, 2 or the like performed by the pen is stored as presented side (client) keywords  19 C 1 ,  19 C 2  or the like in the keyword data storage unit  19 . 
       FIG. 5  is a view showing keyword data stored in the keyword data storage unit  19  of the keyword extractor  10 . 
     The respective controllers (CPU)  11 ,  20 ,  21 ,  22  and  23  control operations of respective units in accordance with commands described in the respective programs PR 1 , PR 2  and PR 3  so that software and hardware can operate in cooperation with each other. In this manner, the keyword extractor  10  configured thus implements functions which will be described in the following operation description. 
     Next, operation of the keyword extractor  10  having the aforementioned configuration will be described. 
     (Keyword Extraction Function) 
       FIG. 6  is a flow chart showing a keyword extraction process (first part) executed in accordance with the keyword extraction process program PR 1  of the keyword extractor  10 . 
       FIG. 7  is a flow chart showing the keyword extraction process (second part) executed in accordance with the keyword extraction process program PR 1  of the keyword extractor  10 . 
       FIG. 8  is a flow chart showing the keyword extraction process (third part) executed in accordance with the keyword extraction process program PR 1  of the keyword extractor  10 . 
     For example, in a presentation operation mode set in response to a touch operation performed on the input device (touch panel)  13  by the presenting side user, an operation information recording (keyword extraction process) is started (Step S 1 ). 
     When a desired presentation screen  14   g  stored in the presentation material storage unit  14  is selected in accordance with an operation of the user, the selected presentation screen  14   g  is displayed on the display device  12  (Step S 2 ). 
     When the user performs a touch operation on the input device  13  in the presentation screen  14   g  displayed on the display device  12 , it may be determined that the operation is a touch operation performed with a finger (Step S 4  (touch)). In this case, coordinates (X, Y)  15 T 1 ,  15 T 2  or the like of the position the finger touches are recorded in the touch operation information address  15 T of the operation information storage unit  15  every time the touch operation is determined (Step S 5 T). 
     Further, it may be determined that the operation is a touch operation performed with a pen (Step  4  (pen)). In this case, coordinates (X, Y)  15 P 1 ,  15 P 2  or the like of the position the pen touches are recorded in the pen operation information address  15 P of the operation information storage unit  15  every time the touch operation is determined (Step S 5 P). 
     On the other hand, it may be determined that the operation is an enlargement operation performed by multi-touch (Step S 4  (enlarge)). In this case, coordinates (X, Y, SX, SY)  15 Z 1 ,  15 Z 2  or the like of a region subjected to the enlargement operation are recorded in the enlargement operation information address  15 Z of the operation information storage unit  15  every time the enlargement operation is determined (Step S 5 Z). 
     In this manner, in accordance with the touch operations using the input device (touch panel)  13  on the presentation screen  14   g  displayed on the display device  12 , the coordinates (X, Y)  15 T 1 ,  15 T 2  or the like designated by the touch operation with the finger on the presenting side and the coordinates (X, Y, SX, ZY)  15 Z 1 ,  15 Z 2  or the like of the region subjected to the enlargement operation are recorded in the touch operation information address  15 T and the enlargement operation information address  15 Z of the operation information storage unit  15  respectively, and the coordinates (X, Y)  15 P 1 ,  15 P 2  or the like designated by the touch operation with a pen on the presented side are recorded in the pen operation information address  15 P of the same operation information storage unit  15 . 
     When it is then determined that an operation to terminate the presentation has been performed (“Yes” in Step S 3 ), it is determined whether the touch operation information of the finger has been recorded in the touch operation information address  15 T of the operation information storage unit  15  or not (Step S 6 ). 
     Here, when it is determined that the touch operation information of the finger has been recorded in the touch operation information address  15 T of the operation information storage unit  15  (“Yes” in Step S 6 ), the coordinates (X, Y)  15 T 1 ,  15 T 2  or the like which are the touch operation information of the finger are acquired (Step S 7 ). 
     A region surrounded by a rectangular size of ±50 dots in both an X direction and a Y direction with respect to the coordinates (X, Y)  15 T 1 ,  15 T 2  or the like acquired from the touch operation information address  15 T in accordance with the coordinates (X, Y) is created as coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like indicating a touch operation region 1, 2 or the like and stored in the touch operation region data storage unit  16  (Steps S 8  to S 11 ). 
     On this occasion, it is determined whether a touch operation region ( 16 Tm) located on the same presentation screen  14   g  and already stored in the touch operation region data storage unit  16  has an overlapping region or not with a touch operation region ( 16 Tn) created this time (Step S 9 ). 
     When it is determined that the touch operation region ( 16 Tm) has no overlapping region (“No” in Step S 9 ), the coordinates (X, Y, SX, SY)  16 Tn of the touch operation region 1, 2 or the like created this time are additionally stored in the touch operation region data storage unit  16  (Step S 10 ). 
     On the other hand, when it is determined that the touch operation region ( 16 Tm) has an overlapping region (“Yes” in Step S 9 ), coordinates ( 16 Tm+Tn) of an expanded region obtained by adding the touch operation region ( 16 Tn) created this time to the touch operation region ( 16 Tm) which has been already stored and which is determined to have the overlapping region are stored for update in the touch operation region data storage unit  16  (Step S 11 ). 
     When it is determined that next touch operation information ( 15 Tn) has been recorded in the touch operation information address  15 T of the operation information storage unit  15 , the same process of creating and storing touch operation region information ( 16 Tn) is repeated (Steps S 6  to S 11 ). 
     Then, it is determined whether touch operation information of the pen has been recorded in the pen operation information address  15 P of the operation information storage unit  15  or not (Step S 12 ). 
     Here, when it is determined that the touch operation information of the pen has been recorded in the pen operation information address  15 P of the operation information storage unit  15  (“Yes” in Step S 12 ), coordinates (X, Y)  15 P 1 ,  15 P 2  or the like which are the touch operation information of the pen are acquired (Step S 13 ). 
     A region surrounded by a rectangular size of ±50 dots in both the X direction and the Y direction with respect to the coordinates (X, Y)  15 P 1 ,  15 P 2  or the like acquired from the pen operation information address  15 P in accordance with the coordinates (X, Y) are created as coordinates (X, Y, SX, SY) indicating the pen operation region 1, 2 or the like and stored in the pen operation region data storage unit  17  (Step S 14 ). 
     On this occasion, it is determined whether a pen operation region located on the same presentation screen  14   g  and already stored in the pen operation region data storage unit  17  has an overlapping region or not with the pen operation region created this time (Step S 15 ). 
     When it is determined that the pen operation region has no overlapping region (“No” in Step S 15 ), the coordinates (X, Y, SX, SY) of the pen operation region 1, 2 or the like created this time are additionally stored in the pen operation region data storage unit  17  (Step S 16 ). 
     On the other hand, when it is determined that the pen operation region has an overlapping region (“Yes” in Step S 15 ), coordinates of an expanded region obtained by adding the pen operation region created this time to the pen operation region which is determined to have the overlapping region and which has already been stored are stored for update in the pen operation region data storage unit  17  (Step S 17 ). 
     When it is determined that next pen operation information has been recorded in the pen operation information address  15 P of the operation information storage unit  15 , the same process of creating and storing pen operation region information is repeated (Step S 12  to S 17 ). 
     Next, it is determined whether touch operation region information have been stored in the touch operation region data storage unit  16  or not (Step S 18 ). 
     Here, when it is determined that touch operation region information have been recorded in the touch operation region data storage unit  16  (“Yes” in Step S 18 ), the coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like which are the touch operation region information are acquired (Step S 19 ). 
     Then, in accordance with the coordinates (X, Y, SX, SY)  16 T 1 ,  16 T 2  or the like of each of the touch operation regions acquired thus, characters are recognized in an image extracted from the presentation screen  14   g  correspondingly to the coordinates (X, Y, SX, SY) of the region by the character recognition controller  23  (Step S 20 ). 
     Strings of characters recognized thus are registered as presenting side keywords  19 S 1 ,  19 S 2  or the like in the keyword data storage unit  19  (see  FIG. 5 ) (Step S 21 ). 
     Next, it is determined whether pen operation region information have been stored in the pen operation region data storage unit  17  or not (Step S 22 ). 
     Here, when it is determined that the pen operation region information have been recorded in the pen operation region data storage unit  17  (Yes in Step S 22 ), coordinates (X, Y, SX, SY) or the like which are the pen operation region information are acquired (Step S 23 ). 
     Then, in accordance with the coordinates (X, Y, SX, SY) or the like of each of the pen operation regions acquired thus, characters are recognized in an image extracted from the presentation screen  14   g  correspondingly to the coordinates (X, Y, SX, SY) of the region by the character recognition controller  23  (Step S 24 ). 
     Strings of characters recognized thus are registered as presented side (client) keywords  19 C 1 ,  19 C 2  or the like in the keyword data storage unit  19  (see  FIG. 5 ) (Step S 25 ). 
     Then, it is determined whether enlargement operation region information have been stored in the enlargement operation region data storage unit  18  or not (Step S 26 ). 
     Here, when it is determined that the enlargement operation region information have been recorded in the enlargement operation region data storage unit  18  (Yes in Step S 26 ), coordinates (X, Y, SX, SY) or the like which are the enlargement operation region information are acquired (Step S 27 ). 
     Then, in accordance with the coordinates (X, Y, SX, SY) or the like of each of the enlargement operation regions acquired thus, characters are recognized in an image extracted from the presentation screen  14   g  correspondingly to the coordinates (X, Y, SX, SY) of the region by the character recognition controller  23  (Step S 28 ). 
     Strings of characters recognized thus are registered as presenting side keywords  19 S 1 ,  19 S 2  or the like in the keyword data storage unit  19  (see  FIG. 5 ) (Step  29 ). 
     &lt;An Example of Keyword Extraction Function&gt; 
       FIGS. 9A and 9B  are views showing a specific display operation on the presentation screen  14   g  in accordance with a keyword extraction process of the keyword extractor  10 .  FIG. 9A  is a view showing touch operation positions T 1  to T 3  touched by a finger, a touch operation position P 1  touched by a pen and an enlargement operation region Z 1  on the presentation screen  14   g  displayed on the display device  12 .  FIG. 9B  is a view showing an enlarged display screen  14   g Z of the enlargement operation region Z 1 . 
       FIG. 10  is a view showing a specific example in which touch operation information are obtained on the presentation screen  14   g  in  FIGS. 9A and 9B  in accordance with the keyword extraction process of the keyword extractor  10 . 
       FIG. 11  is a view showing a specific example in which touch operation region information are created correspondingly to the touch operation information in  FIG. 10  respectively in accordance with the keyword extraction process of the keyword extractor  10  while expanded region information is created when the regions of the touch operation region information overlap with each other. 
     When, for example, the presenting side touches three desired points T 1  to T 3  with a finger on the presentation screen  14   g  displayed on the display device  12  as shown in  FIG. 9A  (Steps S 1  and S 2 ), XY coordinates (X=50, Y=200)  15 T 1  corresponding to the touch operation position T 1 , XY coordinates (X=100, Y=220)  15 T 2  corresponding to the touch operation position T 2  and XY coordinates (X=80, Y=170)  15 T 3  corresponding to the touch operation position T 3  are recorded in the touch operation information address  15 T of the operation information storage unit  15  as shown in  FIG. 10  (Steps S 4  and S 5 T). 
     In addition, when, for example, the presenting side touches a region Z 1  to be enlarged, coordinates (X=200, Y=250, SX=150, SY=100)  15 Z 1  corresponding to the upper left of the enlargement operation region Z 1  and a range extending therefrom in the X direction and the Y direction are recorded in the enlargement operation information address  15 Z of the operation information storage unit  15  as shown in  FIG. 10  (Steps S 4  and S 5 Z). 
     Further, when, for example, the presented side (client) touches one desired point P 1  with the pen, XY coordinates (X=200, Y=220)  15 P 1  corresponding to the pen operation position P 1  are recorded in the pen operation information address  15 P of the operation information storage unit  15  as shown in  FIG. 10  (Steps S 4  and S 5 P). 
     When an operation to terminate the presentation is performed (Yes in Step S 3 ), the coordinates (X=50, Y=200)  15 T 1  corresponding to the first touch operation position T 1  are first acquired from the touch operation information address  15 T of the operation information storage unit  15  so that coordinates (X=0, Y=150, SX=100, SY=100)  16 T 1  of a touch operation region surrounded by a rectangular size of ±50 dots in both the X direction and the Y direction are created as shown in (A) of  FIG. 11  (Steps S 6  to T 8 ). 
     In this case, there is no other touch operation region overlapping with the first touch operation region (X=0, Y=150, SX=100, SY=100)  16 T 1 . Accordingly, the touch operation region  16 T 1  is stored in the touch operation region data storage unit  16  as it is (Steps S 9  and S 10 ). 
     Then, the coordinates (X=100, Y=220)  15 T 2  corresponding to the second touch operation position T 2  are acquired from the touch operation information address  15 T of the operation information storage unit  15  so that coordinates (X=50, Y=170, SX=100, SY=100)  16 T 2  of a touch operation region surrounded by a rectangular size of ±50 dots in both the X direction and the Y direction are created as shown in (B) of  FIG. 11  (Steps S 6  to S 8 ). 
     In this case, a region overlapping with the second touch operation region (X=50, Y=170, SX=100, SY=100)  16 T 2  created this time corresponds to the first touch operation region (X=0, Y=150, SX=100, SY=100)  16 T 1  which has been already stored in the touch operation region data storage unit  16 . Accordingly, coordinates (0, 150, 150, 120)  16 T 1 +T 2  of an expanded touch operation region obtained by adding the second touch operation region  16 T 2  to the existing and overlapping touch operation region  16 T 1  are stored for update in the touch operation region data storage unit  16  (Step S 11 ). 
     Then, the coordinates (X=80, Y=170)  15 T 3  corresponding to the third touch operation position T 3  are acquired from the touch operation information address  15 T of the operation information storage unit  15  so that coordinates (X=30, Y=120, SX=100, SY=100)  16 T 3  of a touch operation region surrounded by a rectangular size of ±50 dots in both the X direction and the Y direction are created as shown in (C) of  FIG. 11  (Steps S 6  to S 8 ). 
     In this case, a region overlapping with the third touch operation region (X=30, Y=120, SX=100, SY=100)  16 T 3  created this time corresponds to the expanded touch operation region (0, 150, 150, 120)  16 T 1 +T 2  which has been already stored in the touch operation region data storage unit  16 . Accordingly, coordinates (0, 120, 150, 150)  16 T 1 +T 2 +T 3  of an expanded touch operation region obtained by adding the third touch operation region  16 T 3  to the existing and overlapping touch operation region  16 T 1 +T 2  are stored for update in the touch operation region data storage unit  16  (Step S 11 ). 
     Also as for the coordinates (200, 220)  15 P 1  (see  FIG. 10 ) corresponding to the pen operation position P 1  which has been stored in the pen operation information address  15 P of the operation information storage unit  15 , a pen operation region corresponding to the pen operation position P 1  is created in the same manner as in the process of creating and storing the touch operation region  16 Tn corresponding to the touch operation position  15 Tn, and stored in the pen operation region data storage unit  17  (Steps S 12  to S 17 ). 
       FIG. 12  is a view showing a specific example of a touch region image GT extracted from a touch operation region  16 Tn on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10 . 
       FIG. 13  is a view showing a specific example of an enlarged region image GZ extracted from an enlarged operation region  16 Zn on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10 . 
       FIG. 14  is a view showing specific examples of respective keywords extracted from respective operation regions on the presentation screen  14   g  in accordance with the keyword extraction process of the keyword extractor  10  and registered in the keyword data storage unit  19 . 
     When the coordinates (0, 120, 150, 150)  16 T 1 +T 2 +T 3  of the touch operation region stored in the touch operation region data storage unit  16  are acquired as shown in  FIG. 11  (Steps S 18  and S 19 ), a touch region image GT corresponding to the touch operation region  16 T 1 +T 2 +T 3  is extracted from the presentation image  14   g  (see  FIG. 9A ) as shown in  FIG. 12  so that characters can be recognized in the touch region image GT by the character recognition controller  23  (Step S 20 ). 
     Then, a character string “acquisition of more clients at ABC•DEF” consisting of the characters recognized thus from the touch region image GT is registered as a presenting side (T) keyword  19 S 1  in the keyword data storage unit  19  as shown in  FIG. 14  (Step S 21 ). 
     When a pen operation region which has been stored in the pen operation region data storage unit  17  is acquired (Steps S 22  and S 23 ), a pen region image corresponding to the pen operation region is extracted from the presentation image  14   g  so that characters can be recognized in the pen region image by the character recognition controller  23  (Step S 24 ). 
     Then, a character string consisting of the characters recognized thus from the pen region image is registered as a presented side (client) (P) keyword  19 C 1  in the keyword data storage unit  19  (Step S 25 ). 
     When coordinates (200, 250, 150, 100)  15 Z 1  of an enlargement operation region which has been stored in the enlargement operation region data storage unit  18  are acquired (Steps S 26  and S 27 ), an enlarged region image GZ corresponding to the enlargement operation region  15 Z 1  is extracted from the presentation image  14   g  (see  FIGS. 9A and 9B ) as shown in  FIG. 13  so that characters can be recognized in the enlarged region image GZ by the character recognition controller  23  (Step S 28 ). 
     Then, a character string “support for various electronic moneys” consisting of the characters recognized thus from the enlarged region image GZ is registered as a presenting side (Z) keyword  19 S 2  in the keyword data storage unit  19  as shown in  FIG. 14  (Step S 29 ). 
     &lt;Keyword Recording Function&gt; 
       FIG. 15  is a flow chart showing a keyword recording for editing and recording keywords which have been registered in the keyword data storage unit  19  by the keyword extraction process of the keyword extractor  10 . 
       FIG. 16  is a view showing a keyword recording form Fk used in the keyword recording of the keyword extractor  10 . 
     When a keyword corresponding to each operation region of the user (the presenting side (salesclerk)/the presented side (client)) on the presentation screen  14   g  is extracted and registered in the keyword data storage unit  19  in accordance with the keyword extraction process (see  FIGS. 6 to 14 ), the keyword recording in  FIG. 15  is executed. 
     When the keyword recording is started up, the keyword recording form Fk (see  FIG. 16 ) corresponding to the presentation this time and stored in the keyword data storage unit  19  is read (Step A 1 ) so that it can be determined whether presented side (client) keywords (P) have been registered in the same keyword data storage unit  19  (see  FIG. 14 ) or not (Step A 2 ). 
     Here, when it is determined that presented side (client) keywords (P)  19 C 1  to  19 Cn have been registered (Yes in Step A 2 ), the keywords (P)  19 C 1  to  19 Cn are acquired sequentially (Step A 3 ) and added and recorded sequentially in a client keyword region Kc in the keyword recording form Fk (Step A 4 ). 
     When it is determined that there is no presented side (client) keyword (P) which should be acquired from the keyword data storage unit  19  (“No” in Step A 2 ), it is then determined whether presenting side (salesclerk) keywords (T) and (Z) have been registered in the same keyword data storage unit  19  (see  FIG. 14 ) or not (Step A 5 ). 
     Here, when it is determined that presenting side (salesclerk) keywords (T)  19 S 1  and (Z)  19 S 2  have been registered (“Yes” in Step A 5 ), the keywords (T)  19 S 1  and (Z)  19 S 2  are acquired sequentially (Step S 6 ) and added and recorded sequentially in a salesclerk keyword region Ks in the keyword recording form Fk (Step A 7 ). 
     When it is determined that there is no presenting side (salesclerk) keyword (T) or (Z) which should be acquired from the keyword data storage unit  19  (“No” in Step A 5 ), the keyword recording form Fk in which the presented side (client) keyword (P)  19 C 1  is classified and recorded into the client keyword region Kc and the presenting side (salesclerk) keywords (T)  19 S 1  and (Z)  19 S 2  are classified and recorded into the salesclerk keyword region Ks is registered in the keyword data storage unit  19 . 
     &lt;Presentation Material Creation Function&gt; 
       FIG. 17  is a flow chart showing a presentation material creation process for creating a new presentation material based on a keyword recording form Fk which has been recorded by the keyword recording of the keyword extractor  10 . 
       FIG. 18  is a view showing a presentation material form Fp used in a presentation material creation process of the keyword extractor  10 . 
     When a presentation material creation process in  FIG. 17  is started up in response to an operation of a user (Step B 1 ) after a keyword recording form Fk in which presented side (client) keywords (P) are classified into a client keyword region Kc and presenting side (salesclerk) keywords (T) and (Z) are classified into a salesclerk keyword region Ks has been created in accordance with the keyword recording ( FIGS. 15 and 16 ), a message encouraging the user to display a list of keywords which have been recorded in the keyword recording form Fk or to terminate the process is displayed (Step B 2 ). 
     Here, when the message to display the list of keywords is selected (“Yes” in Step B 3 ), it is determined whether there are or not any keywords which have been recorded in the client keyword region Kc or the salesclerk keyword region Ks of the keyword recording form Fk (Step B 4 ). 
     When it is first determined that presented side (client) keywords (P)  19 C 1  to  19 Cn have been recorded in the client keyword region Kc of the keyword recording form Fk (Yes in both Steps B 4  and B 5 ), all the presented side (client) keywords (P)  19  C 1  to  19 Cn which have been recorded are read out and added onto a pen keyword list screen Lk displayed in a window on the display device  12  as shown in  FIG. 18  (Step B 6 ). 
     When it is then determined that presenting side (salesclerk) keywords (T) and (Z)  19 S 1  to  19 Sn have been recorded in the salesclerk keyword region Ks of the keyword recording form Fk (“Yes” in Step B 7 ), all the presenting side (salesclerk) keywords (T) and (Z)  19 S 1  to  195   n  which have been recorded are read out and added onto a touch keyword list screen Mk displayed in a window on the display device  12  (Step B 8 ) as shown in  FIG. 18 . Then, the pen keyword list screen Lk and the touch keyword list screen Mk are compared with each other so that keywords which are coincident between a list on the pen keyword list screen Lk and a list on the touch keyword list screen Mk are highlighted and displayed in the lists. 
     In this manner, all the presented side (client) keywords (P)  19 C 1  to  19 Cn and the presenting side (salesclerk) keywords (T) and (Z)  19 S 1  to  195   n  which have been classified and recorded in the keyword recording form Fk are classified and displayed in the respective lists on the pen keyword list screen Lk and the touch keyword list screen Mk which are displayed in the respective windows on the display device  12 . Accordingly, the contents which drew client&#39;s attention and the contents which were emphasized by the salesclerk when the presentation was made by means of the presentation screen  14   g  can be compared with each other. In addition, the coincident keywords are highlighted so that consistent points and different points between the client&#39;s thought and the salesclerk&#39;s thought can be made clear. Thus, keywords regarded as highly important for future presentation can be known. 
     Then, when a keyword desired to be used as a new presentation material is selected from the respective keywords displayed in the lists on the pen keyword list screen Lk and the touch keyword list screen Mk (“Yes” in both Steps B 9  and B 10 ), the selected keyword is inputted into and displayed on a presentation material form Fp which is displayed adjacently to the pen keyword list screen Lk and the touch keyword list screen Mk (Step B 11 ). 
     In this manner, keywords selected desirably from the keyword list screen Lk in which keywords regarded as important are listed and displayed sequentially in order of importance are inputted into and displayed on the presentation material form Fp as they are. Thus, new and suitable presentation materials can be created easily without any omission of the important keywords. 
     Accordingly, according to the keyword extraction function of the keyword extractor  10  having the aforementioned configuration, the presentation screen  14   g  is displayed on the display device  12  provided with the touch panel type input device  13 . The presenting side (salesclerk) performs a touch operation or an enlargement operation with a finger or fingers on the presentation screen  14   g , or the presented side (client) performs a touch operation with the pen likewise. In the touch operation with the finger or the touch operation with the pen, characters are recognized from an image of a region in a fixed range corresponding to coordinates of the touch position in the touch operation, and a keyword of a character string consisting of the recognized characters is extracted. In the enlargement operation, characters are recognized from an image of an enlarged region designated by the enlargement operation, and a keyword of a character string consisting of the recognized characters is extracted. The keyword extracted in accordance with the presenting side (salesclerk) operation region and the keyword extracted in accordance with the presented side (client) operation region are classified and stored in a keyword recording form Fk having a client keyword region Kc and a salesclerk keyword region Ks respectively. 
     Therefore, information which drew attention in a presentation can be recorded in a form which can serve for making materials for a future presentation. 
     Regions corresponding to touch operation positions designated with a finger on the presentation screen  14   g  may overlap with one another, or regions corresponding to touch operation positions designated with a pen likewise may overlap with one another. In this case, according to the keyword extraction function of the keyword extractor  10  having the aforementioned configuration, the region corresponding to the touch operation performed this time is added to the region corresponding to the touch operation performed last time so that a keyword consisting of characters recognized from an image of the operation region expanded thus can be extracted. 
     Therefore, it is possible to eliminate such a disadvantage that characters of one and the same keyword are recognized and extracted severally from an image of a portion where regions corresponding to a plurality of touch operations overlap with each other. 
     In addition, in each of the embodiments, the extracted keywords are used for a new presentation material form. The invention is not limited thereto. When one is selected from the extracted keywords, a keyword in the presentation screen  14   g  coincident with the selected keyword may be highlighted. In this case, a keyword extracted in response to a touch operation may be highlighted and displayed in another way than a keyword extracted in response to a pen operation. In this manner, keywords in which lots of clients are interested, keywords which have to be explained, etc. for a next presentation can be known so that the quality of the presentation can be improved. Even when another salesclerk performs operation, the quality of presentation can be maintained. 
     Incidentally, in each of the embodiments, keywords are extracted from one presentation. The invention is not limited thereto. Keywords which have been extracted a large number of times may be displayed in a list whenever presentation is made several times. In this case, keywords are extracted whenever a presentation is terminated. When there is a keyword coincident with one of the extracted keywords, a counter of the keyword is counted up. Thus, keywords which has been extracted a larger number of times can be displayed. 
     Incidentally, each of the techniques of the respective processes performed by the keyword extractor  10  described in the aforementioned embodiments, that is, each of the techniques of the keyword extraction process shown in the flow charts of  FIGS. 6 to 8 , the keyword recording shown in the flow chart of  FIG. 15 , the presentation material creation process shown in the flow chart of  FIG. 17 , etc. may be distributed as a program which is stored in a medium of an external storage device such as a memory card (an ROM card, an RAM card, etc.), a magnetic disk (a floppy disk, a hard disk, etc.), an optical disk (a CD-ROM, a DVD, etc.), or a semiconductor memory so that the program can be executed by a computer. In a computer (control device) of an electronic instrument provided with a display device  12 , the program stored in the medium of the external storage device is read into a storage device so that operation of the electronic instrument can be controlled by the read program. In this manner, the keyword extraction function, the keyword recording function and the presentation material creation function in response to presentation operations described in the embodiments can be implemented so that the same processes based on the aforementioned techniques can be executed. 
     Incidentally, in the embodiments, keywords are extracted based on touched places. However, information to be extracted is not limited thereto. Images may be extracted or data associated with the touched places may be extracted. 
     In addition, data of the program for implementing each of the techniques may be transmitted as a form of program codes on a communication network. The data of the program can be imported into the computer (control device) of the electronic instrument provided with the display device  12  from a computer apparatus (program server) connected to the communication network and stored into the storage device so that the keyword extraction function, the keyword recording function and the presentation material creation function in response to presentation operations can be implemented. 
     While the present invention has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It is aimed, therefore, to cover in the appended claim all such changes and modifications as fall within the true spirit and scope of the present invention.

Technology Category: 3