SYSTEM AND METHOD FOR RAPID SERIAL VISUAL PRESENTATION WITH AUDIO

Abstract: A system and method for providing enhanced computer-aided reading are disclosed. A rapid serial visual presentation (RSVP) based text file includes portions having text and at least one code portion (3) that links to a particular text portion (4) one or more audio files (4) or other files (4) that stimulate a reader's senses. When a computer-based reading device reads the next portion (2) and corresponding code portion (3), the audio file (4) is retrieved and executed substantially in synchronicity with the presentation of the text in the text portion (2). In this way, the reader is presented with an enhanced reading experience.

Description: SYSTEM AND METHOD FOR RAPID SERIAL VISUAL PRESENTATION WITH AUDIO BACKGROUND OF THE INVENTION Technical Field of the Invention The present invention relates to rapid serial visual presentation (RSVP) with audio and/or other sensory information, and particularly to providing and utilizing RSVP -based text files with audio information. Background of the Invention Reading text, such as an electronic book, from a conventionally sized computer monitor involves much of the same activity as in reading printed material. Several lines of text are simultaneously presented on the computer monitor and the reader reads each line of text before scrolling or paging to the next set of lines of text. A problem exists, however, when attempting to read a relatively large amount of text from a relatively small display, such as a display screen for a PDA or Smartphone. In this case, the reader cannot efficiently read a relatively large amount of text by utilizing the scrolling and other features utilized in reading from a large computer screen. Consequently, users of devices having small display screens are forced to transport the text to a computer or other device having a conventionally sized computer screen in order to read the text. Advancements have been made to allow users of devices having small displays to effectively read large amounts of text. Rapid serial visual presentation, or RSNP, involves the brief display of consecutive words at a fixed location. Typically, the rate of presentation per word is set by the experimenter or the reader, and each word is shown for an equivalent amount of time. In most of the research with RSNP, the display duration of each word is very brief, as implied by the word "rapid" in the acronym. Regardless of the rate of presentation, the functional relevance of RSVP lies in the fact that a small amount of text, such as a word or small number of words, is presented to the reader at any one time. This allows for use of small screens, like screens used in PDAs or wireless telephones. By presenting text as a stream of text to a reader such that only a small amount of text is presented to the reader at a time, users of PDAs, wireless telephones and other devices may easily and efficiently read relatively large amount of text. Like reading printed material, computer-aided reading is typically a silent activity. However, with the advancements in computer technology, computers are now capable of providing a wide variety of multi-media related tasks. It would be a benefit to incorporate multi-media and other audio-visual features to computer-aided reading so as to enhance the reading experience. SUMMARY OF THE INVENTION The present invention overcomes shortcomings in prior computer-aided reading systems and solves a significant need for such a system that enhances the traditional silent reading experience. According to a preferred embodiment of the present invention, a reading file is provided that includes one or more text portions having text therein that forms a stream of text that is presentable to a reader in groups of one or more words at a time. The reading file further includes at least one sense code portion associated with one or more words in an adj acent text portion. The value of the sense code portion identifies a sensory presentation file, such as an audio file. When the text from the adjacent text portion is visually presented to the reader on the visual text display unit 23, the identified sensory presentation file is executed so that execution of the identified sensory presentation file occurs substantially synchronously with the text visual presentation. In this way, the reading experience is enhanced to include sounds or other sensory presentations that are related to the particular text presented to the reader, such as taste, smell and tactile presentations. An embodiment of the present invention includes a display device, such as a portable, hand-held communication device having a small display, including a processing unit that reads the text portion and a sense code portion associated therewith, identifies the sensory presentation file associated with the text portion, determines the amount of the sensory presentation file to be presented to the reader based upon the selected reading speed, and presents the sensory presentation file substantially synchronously with the presentation of the associated text portion. The display device may include a wizard that creates the reading file from a text file, such as an RS VP-based text file, by identifying words in the text file that are related to a particular sound or other sense, and placing a tag code to each identified word whose value identifies an audio file. The audio file is subsequently executed when the tagged word is presented to the reader. BRIEF DESCRIPTION OF THE DRAWINGS A more complete understanding of the system and method of the present invention may be obtained by reference to the following Detailed Description when taken in conjunction with the accompanying Drawings wherein: Figure 1 is a portion of a reading file and related sensory presentation file in accordance with a preferred embodiment of the present invention; Figure 2 is a function block diagram of a display device for presenting the information in the reading file of Figure 1; Figure 3 is a flow chart illustrating an operation of the display device according to a preferred embodiment of the present invention; and Figure 4 is a flow chart illustrating another operation of the display device in creating the reading file, in accordance with a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EXEMPLARY EMBODIMENTS The present invention will now be described more fully hereinafter with reference to the accompanying drawings in which a preferred embodiment of the invention is shown. This invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Referring to Figures 1-2, there is shown a signal format and corresponding device for presenting text-based information to a reader. The text-based information, forming a reading file 1 , is presented as a stream of words that is displayed on a screen one or a few words at a time. In a preferred embodiment of the present invention, the reading file 1 utilizes the rapid serial visual presentation (RSNP) technique. It is understood, however, that the reading file 1 may utilize other techniques for presenting a stream of text on a display one or a few words at a time. Reading file 1 includes text portions 2 having therein text that forms a stream of text that is presented on a display in one or a few words at a time, as explained above. Reading file 1 further includes one or more sense code portions 3 having therein a code that identifies a particular sensory presentation file 4. Sense code portion 3 is sized to suitably identify one of a plurality of sensory presentation files 4. A sense code portion 3 is located within reading file 1 adjacent and/or proximally to a text portion 2 with which sense code portion 3 is associated. Each sense code portion 3 links a particular sensory presentation file 4 to a word or group of words in an adjacent text portion 2, and is available to be executed substantially simultaneously with the presentation of the associated reading file 1. In this way, the execution of a sensory presentation file stimulates a human sense in synchronicity with the presentation of the associated text to the reader. In a preferred embodiment of the present invention, sensory presentation files 4 include audio files that are audibly presented to a reader when played. Audio files 4 may be MP3 audio files, but it is understood that audio files 4 may be other audio files. It is understood that reading file 1 and sensory presentation files 4 may be combined to form a single multi-media file and/or experience file, hi this scenario, each sense code portion 3 may link an executable sensory portion of the multi-media file to a specific text portion 2 thereof. In other embodiments of the present invention, sensory presentation files 4 may instead include files that stimulate other human senses when played and or executed. For instance, sensorypresentation files 4 may include video (MPEG) orpicture (JPEG) files that are presented simultaneously with the particular text to which sensory presentation files 4 are linked. Alternatively, sensory presentation files 4 may include files that stimulate a reader's sense of taste, touch or smell when executed. In presenting a stream of text to a reader, it is understood that the speed of the text stream is controllable by the reader. In order to synchronously present the execution of a sensory presentation file 4 with the presentation of the associated text linked thereto, it is necessary to control the length of play of each sensory presentation file 4 accordingly. Specifically, if sensory presentation files 4 are linked so that the duration of execution thereof corresponds to a relative slow reading pace, such as 100 wpm, it is desired to shorten the duration of execution of sensory presentation files 4 for faster reading speeds. In a preferred embodiment of the present invention, each sensory presentation file 4 is partitioned into one or more regions 5 of sensory information. Each region 5 is prioritized relative to each other. The regions 5 of sensory information having the relatively low priorities assigned thereto are not presented to the reader at higher reading speeds. As a result, both the content and duration of a sensory presentation file 4 are synchronized to the text linked thereto by controlling the duration of the sensory presentation file based upon the speed at which text is presented to the reader. In a preferred embodiment of the present invention, each region 5 of sensory information of sensory presentation file 4 includes a priority code 6 having a value that identifies the relative ranking and/or priority of region 5. It is understood, however, that priority codes 6 may be located elsewhere, such as in sense code portion 3 of reading file 1. A preferred embodiment of the present invention includes a display device 20 (Figure 2) that is adapted to receive reading file 1 and present text and other sensory information to the reader. Display device 20 may include a processing unit 21 for reading reading file 1 stored in memory 22 and presenting text therein as a stream of text on display unit 23 one word or a group of words at a time. In addition, display device 20 includes one or more sensory presentation units 24 for presenting the contents of sensory presentation files 4 to the reader. Sensory presentation files 4 are stored in memory 22. A sensory presentation unit 24, for example, may be a speaker for playing/executing an audio file 4. Display device 20 may be a portable, hand-held communication device, such as a Smartphone or PDA. Accordingly, display device 20 may further include a receiving unit 25 that receives reading file 1 and sensory presentation files 4 linked thereto as one or more wireless signals for storage in memory 22. For example, reading file 1 and corresponding sensory presentation files 4 are downloaded substantially simultaneously from a source (not shown) using the wireless application protocol (WAP). When retrieving and decoding reading file 1 from memory 22, processing unit 21 parses text from text portions 2 for presenting on display unit 23 in groups of one or more words. Processing unit 21 decodes the code value in a sense code control portion 3 and thereby identifies a sensory presentation file 4 to execute substantially simultaneously with the display of text to which the identified sensory presentation file 4 is linked. Processing unit 21 determines the duration of and/or number of regions 5 of sensory information of the identified sensory presentation file 4 to execute and/or present to the reader, based in part upon the speed at which the stream of text is displayed on display unit 23. The (reading) speed is provided to processing unit 21 from the reader/user of display device 20. As stated above, both the reading file 1 and corresponding sensorypresentation files 4 are downloaded to display device 20. This scenario allows for a composer to add sense code control portions 3 to an RSVP-based text file to create reading file 1. In another preferred embodiment of the present invention, a text-to-audio wizard 27 may be utilized to substantially automatically add sense code control portions 3 to reading file 1. In particular, application wizard 27, stored in memory 22, causes processing unit 21 to analyze an RSVP-based text file and identify text therein that can be presented audibly. For example, wizard 27 may cause processing unit 21 to identify verbs that may be associated with a sound. Wizard 27 and/or processing unit 21 place a tag and/or code control portion 3 on and/or near the identified text in the RSVP- based text file that identify an audio file 4 in a bank 28 of audio files stored in memory 22. When the RSVP-based text file (having code control portions 3 therein) is subsequently retrieved from memory 22 for presentation to the reader and the audio file 4 identified in the sense code control portion 3 exists in memory 22, the identified audio file 4 is executed substantially synchronously with the presentation of the tagged text. The operation of the preferred embodiment(s) of the present invention will be described with reference to Figures 3 and 4. initially, it is assumed that a reading file 1 is downloaded into memory 22 of display device 20 at step 30 together with sensory presentation files associated with reading file 1. Next, the reader provides to processing unit 21 at step 31 the desired reading speed at which the stream of text from reading file 1 is to be presented on visual text display unit 23. Text portion 2 and a corresponding sense code control portion 3 of reading file 1 are retrieved from memory 22 and decoded at step 32. Processing unit 21 prepares text from retrieved text portion 22 for presentation and identifies sensory presentation file 4 from control code portion 3 at step 33. Based upon the reading speed provided during step 31, processing unit 21 determines the duration and/or the number of regions 5 of the identified sensory presentation file 4 to be executed at step 34. The prepared text is presented on display unit 23 and the region(s) 5 of the identified sensory presentation file 4 is executed substantially simultaneously with the presentation of the prepared text at step 35. Processing unit 21 then repeats steps 32-35 until the entire contents of reading file 1 has been retrieved and presented to the reader or until the reader signals to processing unit 21 to stop the presentation. In the instance in which display device 20 includes a text-to-audio software wizard 27, the operation of preparing reading file 1 so as to include one or more links to a sensory presentation file 4 will be described with reference to Figure 4. At step 40, an RSVP-based text file is read by processing unit 21 and words identified therefrom that may be associated with a sound. Next, each identified word is tagged at step 41 by adding a control code thereto that indicate a particular sensory presentation file 4 in bank 28 of sensory presentation files 4. Upon a match existing between each tagged control code and a corresponding sensory presentation file 4 existing in bank 28, the corresponding sensory presentation files 4 are thereby linked to the RSVP-based text file. The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.

Claims: What is claimed is: 1. A reading file interpretable by a computer-aided reading device, comprising: a plurality of text portions, each text portion including text so as to form a stream of text that is presentable to a reader a group of one or more words at a time; and at least one sense code portion associated with at least one text portion, the at least one sense code portion having a code therein that identifies an executable sensory presentation file and links the executable sensory presentation file to text in the text portion associated therewith. 2. The reading file of claim 1, wherein: the sensory presentation file comprises an audio file. 3. The reading file of claim 1, further comprising: a plurality of sense code portions, each sense code portion being associated with a distinct text portion and including a code therein that identifies an executable sensory presentation file. 4. The reading file of claim 1, wherein: the sensory presentation file comprises an audio file of a sound that corresponds to at least one word in the text portion associated therewith. 5. A method of presenting information from a reading file to a reader, comprising the steps of: retrieving a first portion of the reading file having text information therein; retrieving a second portion of the reading file associated with the first portion and including a code therein; determining a sensory presentation file identified by the code in the second portion; visually presenting the text from the first portion; and executing the sensory presentation file substantially synchronously with the step of visually presenting the text from the first portion. 6. The method of claim 5, wherein: the sensory presentation file comprises an audio file. 7. The method of claim 5, further comprising the step of: receiving a desired reading speed from the reader; and wherein the method of executing the sensory presentation file comprises the step of determining a duration of the sensory presentation file execution based upon the desired reading speed. 8. The method of claim 7, wherein the step of determining comprises the step of: determining regions that partition the sensory presentation file; determining priority information corresponding to each region of the sensory presentation file; and executing one or more regions of the sensory presentation file having the higher priority information based upon the desired reading speed. 9. The method of claim 5, further comprising the step of: initially receiving the reading file and related presentation files using the wireless application protocol. 10. A computer program product for presenting text from a reading file on a display device including a processing unit, memory and a display unit, the computer program product including instructions for: retrieving from memory in the display device a first portion of the reading file having text information therein; retrieving from memory in the display device a second portion of the reading file associated with the first portion and including a code therein; determining a sensory presentation file stored in memory identified by the code in the second portion; visually presenting the text from the first portion on the display unit; and executing the sensory presentation file substantially synchronously with the text from the first portion being presented on the display unit. 11. The computer program product of claim 10, wherein: the display device comprises a speaker; and the sensory presentation file comprises an audio file so that executing the sensory presentation file comprises playing the audio file using the speaker. 12. The computer program product of claim 10, further including instructions for: receiving a desired reading speed from the reader at the processing unit; and wherein executing the sensory presentation file comprises determining a duration of the sensory presentation file execution based upon the desired reading speed. 13. The computer program product of claim 10, wherein the instructions for executing comprises instructions for: determining regions that partition the sensory presentation file; determining priority information corresponding to each region of the sensory presentation file; and executing one or more regions of the sensory presentation file having only the higher priority information based upon the desired reading speed. 14. The computer program product of claim 10, further including instructions for: initially receiving the reading file using a wireless protocol. -il¬ ls. The computer program product of claim 10, further including instructions for: determining regions that partition the sensory presentation file and determining priority information corresponding to each region of the sensory presentation file based upon contents of the sensory presentation file. 16. A method of creating a reading file for displaying a stream of text from an electronic device, comprising the steps of: identifying words in an RSVP-based text file corresponding to a sound; for each identified word, determining a code value therefor that links an audio file to the identified word in the RSVP-based text file, a sound generated by executing the audio file being substantially the same as the sound corresponding to the identified word; and adding each code value to the RSVP-based text file. 17. The method of claim 16, wherein: each code value is placed within the RSVP-based text file substantially proximally to the corresponding identified word. 18. A device for presenting a stream of text to a reader, comprising: a display unit for displaying text thereon; memory for storing a reading file having a plurality of text portions containing text information and at least one sense code portion containing a code therein; and a processing unit for reading the reading file, processing the text portions for presenting the text therein as a stream to the display unit, identifying the sense code portion and the text portion corresponding thereto, retrieving a sensory presentation file from the memory identified by the code in the sense code portion, and executing the sensory presentation file substantially synchronously with the presentation of the text from the corresponding text portion. 19. The device of claim 18, further comprising: a speaker; wherein the sensory presentation file comprises an audio file, execution of the audio file being played on the speaker. 20. The device of claim 19, wherein: the processing unit determines an amount of the audio file to be played on the speaker based upon the amount of time the text from the corresponding text portion is displayed on the display unit. 21. The device of claim 19, wherein: the device comprises a wireless telephone.

Inventors: ANNEROTH MIKAEL, GOLDSTEIN MIKAEL, LUNDBERG TORBJOERN, SICHERITZ KARIN, ANNEROTH, MIKAEL, GOLDSTEIN, MIKAEL, LUNDBERG, TORBJOERN, SICHERITZ, KARIN
Applicants: ERICSSON TELEFON AB L M, TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
Application year: 2001
Main classification: G06F15
IPC codes: G09B17