COMPUTER-BASED PATIENT RECORD AND MESSAGE DELIVERY SYSTEM

Abstract: A Computer-based Patient Record (CPR) system includes user equipment devices which are configured for speech synthesis in response to speech markup language text and which are connected via a network to a middle tier of a server system. The CPR system further includes a message delivery facility for delivery of textual messages to any of pager, electronic mail, or voice mail (after text-to-speech synthesis) message delivery vehicles. The server system accesses a user specific data store containing speech synthesis profiles which include prosodic information of the voices and speech of users, and message delivery profiles which specify which of the aforementioned message delivery vehicles are to be used and in what order. The stored speech synthesis information associated with an originator of a message and the stored message delivery information associated with the recipient of message are provided by the server to user equipment or a reminder generator to produce speech markup files containing information needed to synthesize the vocal and speech characteristics of the originator accompanied by delivery instructions reflecting the message delivery preferences of the recipient.

Description: COMPUTER-BASED PATIENT RECORD AND MESSAGE DELIVERY SYSTEM. The present invention relates generally to a computer system for enabling access of records in electronic form via a network, such as an intranet, which includes a facility for delivery of messages. The invention further relates to a method for message delivery. The invention is particularly useful in the medical domain in conjunction with Computer-based Patient Record (CPR) systems and in the provision of alerts to physicians and other health care professionals. CPR systems maintain multimedia patient records of one or more health care institutions such as histories, reports, charts, and images in electronic digital form in a server system so that authorized users may access the records remotely employing user equipment, such as desktop or portable computing devices, coupled to the server system via wired and/or wireless network paths. Speech interfaces for computer systems can employ speech recognition for command and control, thereby providing for hands-free navigation by the user. Also, useful information can be optimally arranged on screen and the user can remain focussed thereon by eliminating distracting and space consuming Graphical User Interface (GUI) control objects such as buttons, hot spots and/or menus that are typically used with a mouse or other pointing device. Speech recognition for dictation can enable health care personnel to enter reports when their hands are busy or where touching a keyboard would be a hygiene risk. Text-to-speech synthesis can provide background notification of events and status changes, such as printer activity, by speech synthesized messages thereby avoiding visual distraction of the user. The use of synthesized speech for audible messages also minimizes storage requirements for the messages. A Java® Speech Application Programming Interface (API) is substantially available to support speech synthesis, command-and-control recognizers and dictation systems. The Java® Speech API includes a Java® Speech Markup Language (JSML) for speech synthesis and a Java® Speech Grammar Format (JSGF) for speech recognizers. JSGF provides rule-based grammar indicating possible alternative commands, actions, objects, and their relative probabilities. JSML supports explicit specifications for delimited words and phrases such as structure (paragraph and sentence), special handling of dates and times, breaks, and prosodic information (speaking rate, volume, baseline pitch, pitch range). In hospital environments, health care personnel are not always at their user equipment devices. It is a drawback of the known system that it is not possible for health care personnel to issue alerts and reminders via the CPR system which can be received via a variety of message delivery vehicles, such as voice mail, pager and electronic mail. It is an object of the present invention to provide a message delivery facility in a computer network system which includes speech synthesis of text specified in a speech markup language, such as JSML. Thereto, a system according to the invention is defined in Claim 1 in this system the server system has access to a storage system containing user specific information including speech synthesis profiles of users of the system, each profile of a user including prosodic information for synthesizing vocal characteristics of the user, such as baseline pitch and pitch range, and including message delivery profiles of users indicating which of a plurality of possible message delivery vehicles are to be used, and in what order. The computer system has the capability of generating messages in the form of speech markup language text which includes prosodic information of a predetermined person associated with the message, usually the originator, to be initiated and supplied to a message delivery system for delivering the message to the recipient in accordance with the recipient's stored message delivery profile. It is a further object of the present invention that the text specified in speech markup language include speaker profile information so that the message may be personalized with prosodic characteristics of the voice of a predetermined person, for example the originator of the message, in order to invoke appropriate attention by the recipient. It is yet another object of the present invention that the computer system provide for delivery of messages in accordance with a message delivery profile of the recipient indicating which message delivery vehicles should be used and in what order. A further advantage of the system according to the invention is that alerts and reminders when delivered as audible messages have appropriate clarity, structure, and prosodic characteristics, so they are easily understandable and receive attention appropriate to the nature of the information conveyed. The message delivery system includes a speech synthesizer for receiving the speech markup language text and synthesizing therefrom an audible message, which is similar in vocal characteristics to the predetermined person, and also appears clear and natural due to the use of delimiters in the markup language text specifying emphasis or volume, speaking rate, and items to be said as dates and times. The message delivery system includes as one of the message delivery vehicles, a voice mail facility which receives synthesized speech from the speech synthesizer. Other message delivery vehicles of the message delivery system include a pager system, and an electronic mail system. The invention further relates to a method for message delivery, a method according to the invention is defined in Claim 8. Other objects, features and advantages of the present invention will become apparent upon perusal of the following detailed description when taken in conjunction with the appended drawing, wherein: Figure 1 is a schematic diagram of a speech enabled CPR system in accordance with the present invention which is capable of generating alerts, reminders, and reports in the form of a speech markup language. Referring to Figure 1 of the drawing, there is shown a computer system 10 for delivery of Computer-based Patient Records (CPRs) to users at user equipment devices, or client computers 12, of which one is illustrated, from a server system generally comprising servers 14 and 16, via network paths such as 18. Server 14 may be one or a plurality of servers configured to together perform the functions of a Hypertext Transfer Protocol (HTTP) server and also a Common Object Request Broker Architecture (CORBA) server, using Internet-Inter-Orb Protocol (HOP), which act as a middle tier between the user equipment devices or client computers 12 and the back end server 16 which is a Database Management System in communication with the various information systems of the health care institution(s), including Picture Archiving and Communications System (PACS), Hospital Information System (HIS), Radiology Information System (RIS), laboratory computing system, and pharmacy computing system, in order to supply requested patient information objects to middle tier server 14. This middle tier server provides services to HTTP clients and CORBA clients via an intranet accessible via network paths 18. CPR information may be presented on the display 20 of the user equipment 12 employing web-based tools such as Hyper Text Mark-up Language (HTML) pages and Java® (a trademark of Sun Microsystems, Inc.) applets that are downloaded to the user equipment by server 14 also acting as a Java® Object Request Broker (ORB), which downloaded applets and run thereon inside a web browser. The browser/ Java® applet combination is represented by the element 22. User equipment 12, and the applets downloaded thereto, are preferably speech- enabled. Thus, user equipment 12 includes a speech synthesizer 24 which feeds a speaker 26, and a speech recognition engine or recognizer 28 which is fed by a microphone 30. However, the usual input devices, such as mouse and keyboard 25 should also be provided. As is usual, speech synthesizer 24 and speech recognition engine 28 may be realized by software. Recognizer 28, which is preferably used both for command and control purposes and for dictation purposes, includes a lexicon and may be further constrained by a context appropriate to the subject or specialty of the user. For purposes of illustration, synthesizer 24 and recognizer 28 are shown separately coupled to applet/browser combination 22 via Java® Speech Application Programming Interface (API) 32. The connection 34 between recognition engine 28 and the Java® Speech API is shown as bi-directional, because grammar in Java® Speech Grammar Format (JSGF) is sent to the recognizer 28 for command and control recognition purposes. Preferably only the subset thereof applicable to the current screen display is sent from the applet to the recognizer to improve the recognition accuracy. In addition to command and control necessary to access patient records, users input text which may be medical reports of particular patients or messages, such as alerts, for particular recipients. This text may be inputted and placed into appropriate fields of an associated downloaded template by the user via the microphone 30 and speech recognition engine 28, via mouse and keyboard 25, or a combination thereof. User equipment 12 also contains a Java® Speech Markup Language (JSML) converter 33 which is preferably implemented by software under control of the applet and is shown functionally in bi-directional communication with the browser/applet combination 22. Its purpose is to receive user inputted or application generated text and a template which is associated with the text or in which the text is embedded and to produce a JSML text file therefrom. The template, which has previously been downloaded from server 14, has fields or defined locations in which text is embedded or with which text is associated and respective JSML delimiters are associated in one to one correspondence with respective fields. Consequently, through the use of JSML delimiters, such things as volume or emphasis, speaking rate, pitch change, and whether text should be said as a date or said as a time can be individually specified for the text of each field. In accordance with the invention, the JSML converter 33 has the capability of producing a JSML file in which prosodic information such as the baseline pitch and pitch range (and may include other information such as age and sex of the speaker) which are taken from user specific speech synthesis profile information downloaded to user equipment 12 by server 14 are specified for the entire JSML file. Thus, the JSML file also contains information which allows a speech synthesizer, such as 24, to synthesize speech having the vocal characteristics of a predetermined person associated with the text, such as the originator or author thereof. The prosodic characteristics which can be specified to approximate the voice and speech of a predetermined speaker are not necessarily limited to those specifically provided for in JSML as possible delimiters, since JSML may also contain native instructions to a specified speech recognizer. An application data store 34 and a user specific data store 36 are coupled to or accessible by server 14. Stores 36 may be implemented together in a single storage device or in a combination of such devices. Application data store 34 includes at area 38, Hypertext Markup Language (HTML) pages and templates for server 14 to form HTML pages in accordance with personalization information stored in user specific data store 36 at area 40. This personalization information includes the specialty or title and affiliation of the user, specifications of the user's equipment 12 including display type or resolution, and the limiting bandwidth in the network path 18 between server 14 and user equipment 12. After the user logs on to the system at user equipment 12 and presents his ID and password and requests patient formation or a list of patients, server 14 retrieves the personalization information and uses it to generate HTML pages to be supplied to user equipment 12 which are appropriate to the user, his relationship to the patient(s), and the capability of his equipment and network connection. The inclusion of such capability information allows the server 14 to limit the use of or number of pixels in graphical objects in the HTML pages where the display is low resolution (or text only) or the limiting bandwidth includes a relatively low baud rate telephone or wireless path which would otherwise produce unacceptably long download times for such objects. Application data store 34 also contains applets in area 42 and associated speech recognition grammar files in area 44 which are read by server 14 and downloaded to user equipment 12 when needed. Further, application data store 34 includes alert, reminder and report templates at area 46 and the user specific data store includes speech synthesis profiles at area 48 which are used by JSML converter 33 to produce JSML text files which are personalized to the voice of a predetermined speaker. The reminder templates and speech synthesis profiles are used by a reminder generator and JSML formatter 49. The latter device is configured to maintain a store of scheduled reminders from particular originators to particular recipients at predetermined times, having predetermined text in fields of an associated reminder template. At some time prior to the scheduled delivery time generator and formatter 49 produces a JSML text file therefrom in a similar manner as the JSML converter 33 operates. Then, alert/reminder profile data stored in area 50 of specific data store is accessed by generator and formatter 49 via server 14 to determine which message delivery vehicles are to be used, their characteristics, and in which order they should be tried. Generator and formatter 49 uses this information to produce delivery instructions which are sent along with the JSML text file to alert/reminder output devices system 51 via server 14. Output devices system includes a pager system 52 for delivery of plain text messages to designated pagers carried by users, an electronic mail system 54 which can be accessed by users from a variety of types of wired or wireless communications or computing devices (e.g. Personal Digital Assistants (PDAs), cellular or wireless local loop phones, screen telephones), and a voicemail facility 56. The latter includes a speech synthesizer 58 (including the necessary parts of Java® Speech API) and a conventional voice mail system, which users may access by telephone to hear audible messages. In the case of message delivery to electronic mail system 54, it is expected that the delivery instructions will include whether or not the textual message should include JSML delimiters or whether they should be removed. In a similar manner, when alerts are composed by a user at user equipment 12, server 14 will send to the user equipment an alert/reminder profile of the intended recipient so that delivery instructions may be produced by user equipment. The alert with accompanying delivery can be sent from user equipment 12 to output device system 51 via server 14. It should now be appreciated that the objects of the present invention are satisfied. While the present invention has been described in particular detail, it should also be appreciated that numerous modifications are possible within the intended spirit and scope of the invention.

Claims: CLAIMS: 1. A computer system (10) comprising: a server system (14,16); a plurality of user equipment devices (12) coupled to the server system via a network (18); a storage system (36) accessible to the network contain user specific information including speech synthesis profiles of users of the system, each profile (48) of a user including prosodic information for synthesizing vocal characteristics of the user, and including message delivery preferences (150) of users indicating which of a plurality of possible message delivery vehicles are to be used; means (33) at user equipment for an originator to initiate production of a message for a recipient, said message being in the form of speech markup language text which includes prosodic information of a predetermined person associated with the message; and a message delivery system (51) for delivering the message to the recipient in accordance with the recipient's stored message delivery profile (50). 2. The system as claimed in Claim 1 , wherein the message delivery system includes a speech synthesizer (24) for receiving the speech markup language text and synthesizing therefrom an audible message, which is similar in vocal characteristics to the predetermined person. 3. The system as claimed in Claim 2, wherein the message delivery system includes as one of said message delivery vehicles, a voice mail facility (56) which receives synthesized speech from said speech synthesizer. 4. The system as claimed in Claim 3, wherein the message delivery system includes as another of said message delivery vehicles, a pager system (52). 5. The system as claimed in Claim 4, wherein the message delivery system includes as another of said message delivery vehicles, an electronic mail system (54). 6. The system of Claim 1, wherein said predetermined person is the originator of the message. 7. The system of Claim 1, wherein said prosodic information includes for each user, baseline pitch, and pitch range. 8. A method for message delivery comprising: obtaining a filled-in message template comprising message text, specifying message content, and message delimiters, specifying characteristics for guiding subsequent speech synthesis, retrieving speech synthesis profile information for a predetermined person specifying prosodic characteristics for synthesizing the voice of the predetermined person, providing a message containing speech markup language text generated from said filled-in message template and said speech profile information, retrieving a message delivery profile of a recipient specifying which one or more of a plurality of message delivery vehicles, and their order, are to be used for delivery of said message to said recipient, and transmitting said message to said one or more of the plurality of message delivery vehicles. 9. The method of claim 8 wherein said obtaining a filled-in message template comprises the further steps of: retrieving a message template comprising fields with delimiters for specifying characteristics guiding subsequent speech synthesis providing message text, and embedding said message text in said template to produce said filled-in message template. 10. The method of claim 9 wherein said message text either is provided by user input, or is provided be application generation, or is predetermined. 11. The method of claim 8 wherein the method comprises a further step of receiving said message at said one or more of the plurality of message delivery vehicles. 12. The method of claim 11 wherein at least one of said one or more of the plurality of message delivery vehicles comprises a speech synthesizer, and wherein said receiving said message at said one message delivery vehicle comprises synthesizing said message into an audible message for said recipient, whereby the audible message appears clear, natural and similar in vocal characteristics to the predetermined person due synthesis guidance by said speech profile information and by said message template delimiters. 13. The method of claim 8 wherein said prosodic characteristics for a predetermined person include baseline pitch and pitch range. 14. The method of claim 8 wherein said speech markup language is JAVA® speech markup language.

Inventors: GLICKSMAN ROBERT A, MOSHFEGHI MEHRAN, GLICKSMAN, ROBERT, A., MOSHFEGHI, MEHRAN
Applicants: KONINKL PHILIPS ELECTRONICS NV, PHILIPS SVENSKA AB, KONINKLIJKE PHILIPS ELECTRONICS N.V., PHILIPS AB
Application year: 1999
Main classification: G06F17
IPC codes: G06F3, G06F17, G06F19, G06Q50, G10L13, G10L15, G10L19