Remote control of medical devices using instant messaging infrastructure

Systems and methods for remote control and management of medical workstations using an instant messaging infrastructure. A remote client, such as a mobile phone, laptop, tablet, or other computing device, is used to generate instructions or information requests in one or more data packets. The remote client sends the one or more data packets using the instant messaging infrastructure to a medical workstation at another location. A service application in communication with the medical workstation receives the data packets and causes the medical workstation to retrieve the requested information or execute the instruction. The communications between the remote client and the service application are encrypted and signed to ensure secure communications.

FIELD OF THE INVENTION

The present invention relates to remote control of medical devices.

SUMMARY

In some embodiments, the invention provides a system for remote control and management of medical workstations. The system includes a remote client, a web server, an instant messaging (IM) server, a medical workstation, and a remote instant message client (BOT) that communicate using an instant messaging infrastructure. The remote client includes a client IM application and the medical workstation includes a service application, each being used to form a connection therebetween and to transfer data packets over the IM infrastructure.

In some embodiments, the invention provides a method of transferring medical images from a remote medical image database to a destination device using an instant messaging (IM) infrastructure. The method includes establishing a session between a local device and an IM server; sending a presence message to a remote medical workstation via the IM server, wherein the remote medical workstation communicates with a medical image database; and receiving, by the local device, presence information for the remote medical workstation from the IM server. The method further includes the local device sending an image availability query message to the remote medical workstation via the IM server and obtaining image availability data from the remote medical workstation via the IM server. The image availability data indicates an image that is available on the remote medical image database. The local device further displays a representation of the image available on the remote medical image database and receives a user selection of the image. The method also includes sending an image command message, based on the user selection, to the remote medical workstation via the IM server. The image command message instructs the remote medical workstation to transmit a copy of the image to the destination device. The destination device may be the local device or another medical workstation.

In some embodiments, the invention provides a method of transmitting medical images from a medical image database to a remote device using an instant messaging (IM) infrastructure. The method includes establishing a first session between an IM server and the remote device, and a second session between the IM server and a medical image workstation. The medical image workstation is in communication with a medical image database, for instance, to obtain medical images for display on the medical image workstation and to provide them to other devices. The method further includes the IM server sending roster and presence information to the remote device based on a roster request message received from the remote device; forwarding an image availability query message from the remote device to the medical image workstation; and forwarding image availability data received from the medical image workstation to the remote device. The image availability data includes a representation of an image available on the medical image database. The IM server also forwards an image command message with an image selection received from the remote device to the medical image workstation. The image command message instructs the medical image workstation to transfer a copy of the selected image to a destination device.

In some embodiments, the invention provides a method of transmitting medical images from a medical image database to a remote device using an instant messaging (IM) infrastructure. The method includes establishing a session with an IM server. The method also includes the medical workstation receiving presence information for the remote device from the IM server and receiving an image availability query message from the remote device via the IM server. The medical workstation is in communication with the medical image database, for instance, to obtain medical images for display on the medical workstation and to provide them to other devices. The medical workstation transmits image availability data to the remote device via the IM server. The image availability data includes a representation of an image available on the medical image database. The method further includes receiving an image command message with an image selection from the remote device via the IM server. The image selection selects the image available on the medical image database for transmission. The medical image database then transmits a copy of the image to a destination device in response to the image command message.

In some embodiments, the invention provides an instant messaging (IM) server enabling instant message communication between a medical image workstation and a requesting device. The medical image workstation is in communication with a medical image database, for instance, to obtain medical images for display on the medical image workstation and to provide them to other devices. The IM server includes a memory, a session establishing module, and a message routing module. The memory includes a requesting device roster including an identification for the medical image workstation; a medical image workstation roster including an identification for the requesting device; and presence information indicating presence status of the requesting device and the medical image workstation. The session establishing module establishes a first instant message session between the IM server and the requesting device, and establishes a second instant message session between the IM server and the medical image workstation. The message routing module forwards an image availability query message from the requesting device to the medical image workstation, forwards image availability data received from the medical image workstation to the requesting device, and forwards an image command message with an image selection from the requesting device to the medical image workstation. The image availability data includes a representation of an image available on the medical image database. The image command message instructs the medical image workstation to transfer a copy of the selected image to a destination device.

In some embodiments, the invention provides a method of establishing a secure instant message (IM) session between a local device and a remote medical workstation. The method includes creating an IM account for the local device on an IM server by submitting registration information to a registration server. The registration server communicates with the IM server to create an IM account identifier using the registration information, to add the IM account identifier to a first roster associated with a registration BOT maintained by the IM server, and to add a BOT identifier to a second roster associated with the IM account identifier. The method also includes the local device receiving a temporary unique ID from the remote medical workstation. The registration BOT provides the temporary unique ID to the remote medical. The local device also requests that the registration BOT add the remote medical workstation to the second roster and add the local device to a third roster associated with the remote medical workstation. The request, however, is denied unless the temporary unique ID is provided to the registration BOT. The method further includes the local device sending a message via the IM server to the remote medical workstation. The message requests transmission of an image on the remote medical workstation to a destination device.

In some embodiments, the invention provides a client instant messaging (IM) application stored on a computer readable medium. When executed by the processor of a local device, the client IM application performs multiple steps. The client IM application establishes a session between the local device and an IM server and sends a presence message to a remote medical workstation via the IM server. The remote medical workstation communicates with a medical image database, for instance, to obtain medical images for display on the remote medical workstation and to provide them to other devices. The client IM application also receives presence information for the remote medical workstation from the IM server and sends an image availability query message to the remote medical workstation via the IM server. The client IM application obtains image availability data from the remote medical workstation via the IM server. The image availability data indicates an image that is available on a remote medical image database. The client IM application then displays, on a display screen of the local device, a representation of the image available on the remote medical image database. The user selects an image and the selection is received by the client IM application. The client IM application then sends an image command message, based on the user selection, to the remote medical workstation via the IM server. The image command message instructs the remote medical workstation to transmit a copy of the image to a destination device. The computer readable medium includes, for instance, a computer hard drive, compact disc, floppy disc, flash drive, or other memory device.

In some embodiments, the invention provides a medical workstation that is remotely controllable by a remote device via an instant messaging (IM) server. The medical workstation includes a processor and a service application, which, when executed by the processor, performs multiple steps. The service application establishes a session with the IM server and receives presence information for the remote device from the IM server. The service application also receives an image availability query message from the remote device via the IM server and transmits image availability data to the remote device via the IM server. The image availability data includes a representation of an image available on the medical image database. The service application further receives an image command message with an image selection from the remote device via the IM server. The image selection selects the image available on the medical image database for transmission. In response to the image command message, the service application transmits a copy of the image to a destination device.

DETAILED DESCRIPTION

Embodiments of the invention allow for secure control and management of a medical workstation by a remote device over an instant messaging (IM) infrastructure. An IM infrastructure enables at least two client devices to communicate messages according to an IM protocol via an IM server. The IM infrastructure provides near real time delivery of queries, requests, and commands, which eliminates delays found in systems where requests are queued and polled for by the receiver. Among other functions, the IM server maintains a roster for each client device, maintains presence information indicating the availability of each client device, provides certain roster and presence information to appropriate client devices, and routes instant messages between two or more client devices. The IM protocol provides a set of rules governing the format of messages that are exchanged between the client devices and the IM server. The IM protocol also specifies when and how a client device or IM server should send a message or react to a received message.

In some embodiments, a mobile phone is operable to request via an IM infrastructure that an image accessible to the medical workstation be sent back to the mobile phone or to another workstation. The use of an IM infrastructure to control a remote medical workstation enables a simpler configuration and network management relative to, for instance, use of web browser-based connections or other network connections between devices. Additionally, embodiments of the system and method are secure as the network traffic is encrypted and the data packets including medical information are both encrypted and signed. The encrypted and signed data packet ensures that the data is being received from the correct sender and that only the intended recipient can decrypt the data.

FIG. 1depicts a system100in which a remote device105remotely controls and manages a medical workstation110. The remote device105accesses and/or controls a medical workstation110operating in a remote location via a network115. For instance, the medical workstation110is often located in a different room, building, campus, city, or country. The network115is one or more of the Internet, a local area network (LAN), a wide area network (WAN), and other computer networks. The devices that are coupled to the network115, such as the remote device105and medical workstation110, are directly coupled or indirectly coupled by way of, e.g., a hub, router, or similar device. Such couplings include wired connections (universal serial bus (USB), Ethernet, etc.), wireless connections (e.g., Bluetooth, WiFi, cellular, etc.) or a combination thereof.

The system100further includes an instant messaging (IM) server120, a web server125, and a remote instant message client (BOT)130. The IM server120enables communications between the remote device105and the medical workstation110. The web server125and BOT130are used to register a remote device105and a medical workstation110with the IM server120and to alter roster information for the remote device105and medical workstation110. The system100is expandable to include multiple remote devices105and multiple medical workstations110. For instance, system100includes a second medical workstation135. Thus, in some embodiments, one or more remote clients105are operable to control one or more medical workstations110via the IM server120.

Remote Device

The remote device105is one of a personal computer, kiosk, tablet, laptop, mobile phone device (e.g., an iPhone™, Blackberry™, Droid™, etc.), or other computing device with an ability to connect to the network115. As shown inFIG. 2, the remote device105includes a processor140that executes a client IM application145, a web browser150, and other software155stored in memory160. In some embodiments, one or more of the software programs are stored remotely from the remote device105or are implemented partially or completely in hardware (e.g., using a field programmable gate array (FPGA) or application specific integrated circuit (ASIC)). The remote device105also stores security keys165within the memory160, which are used for secure communications and will be explained in greater detail below. The remote device105further includes a power source170, a display175(e.g., a touch screen display), user inputs180(e.g., push buttons, scroll wheels, etc.), audio in/out module185(e.g., including a microphone and speaker), and a network communications interface190for interfacing with the network115. The power source170is, for instance, a battery that provides power to the components of the remote device105. In some instances, the power source170receives power from an external battery, wall outlet, or other external power source, and provides the power to components of the remote device105.

A user of remote device105uses the client IM application145to register with the IM server120and to send and receive queries, requests, and commands to other devices connected to the IM server120(e.g., the medical workstation110). In some embodiments, the client IM application145is pre-loaded on the remote device105. In other embodiments, the client IM application145is downloaded from an online repository (e.g., App Store™ for iPhone™ mobile phones, Android Market™ for Droid™ mobile phones) directly accessible by the remote device105. In still other embodiments, the client IM application145is downloaded from an online repository to an independent computer using, for example, iTunes™ or Google Chrome™ and, thereafter, loaded from the independent computer to the remote device105.

The client IM application145provides a graphical user interface (GUI) on the display175of the remote device105. The GUI enables the user to interact with the client IM application145by way of the display175, user input180, and audio in/out185.

Medical Workstation

FIG. 3Adepicts the medical workstation110according to some embodiments of the invention. The medical workstation110includes a processor200and a memory205. The memory205includes a service application210, a service application-to-IM application program interface (API)215, a web browser220, other software225, security keys230, and a medical image database235. The service application210, API215, web browser220, and other software are all executable by the processor200. The medical workstation110includes a power source240, display245, user input250, audio in/out255, and network communications interface260, which, in general, each function similarly to their counterparts in the remote device105.

The medical workstation110is usable as a stand-alone device for viewing and/or manipulating medical images stored in the medical image database235on the display245. For instance, the service application210and/or other software225is executed on the processor200, which provides a GUI on the display245that a user interacts with to retrieve images within the medical image database235. In some instances the user is a medical professional that evaluates images using the medical workstation110to diagnosis a patient depicted in the image. The user is also able to interact with the GUI to move images from the medical image database235to a database of another medical workstation or other destination. In some instances, the medical workstation110is a workstation in an electronic picture archiving and communication system (PACS).

The images within the medical image database235include computed tomography (CAT scan) images, magnetic resonance imaging (MRI) images, x-ray images, and other graphical depictions of patient medical information.

The service application210, in combination with the API215, is executed by the processor200and is used for interfacing with the IM server120to communicate with the remote device105. Instant messages sent to the medical workstation110from the IM server120are received and translated by the API215in a protocol used by the service application210. The instant messages are then forwarded to the service application210. Vice versa, outgoing instructions and data from the service application210to the IM server120are translated by the API215into the IM protocol of the IM server120.

Although the service application210, API215, web browser220, other software225, security keys230, and medical image database235are all described as being stored on memory205locally, one or more of these items may be stored in an external memory in communication with the medical workstation110. For instance, in some embodiments, the medical image database235is stored on an external USB hard drive or on a network hard drive. In some embodiments, one or more of the service application210, API215, web browser220, other software225, and security keys230are implemented partially or completely in hardware (e.g., using an FPGA or ASIC).

FIG. 3Billustrates an account tab261of GUI262for user interaction with the service application210. The GUI262includes a status indicator264that indicates whether a session has been established with the IM server120. The GUI262also includes a login section264enabling a user to enter login information. Once the user information is entered into section264, the user selects the activate button266, which starts a session with the IM server120. The GUI262also has a log window268that displays recent activity by the service application210. The peers tab272and ID request tab270will be discussed in greater detail below with respect toFIGS. 5B and 10B, respectively.

In some instances, the medical workstation110is a medical image server that is not intended for image viewing but, rather, is intended as a medical image repository that interacts with the IM server120and other medical workstations that are intended for image viewing. For example, the medical image server implementation may be an image archive in a PACS.

IM Server

FIG. 4depicts the IM server120according to some embodiments of the invention. The IM server120generally handles the flow of messages between client devices of the IM infrastructure, including the remote device105and the medical workstation110. For instance, the IM server120maintains a roster for each client device, maintains presence information indicating the availability of each client device, provides certain roster and presence information to appropriate client devices, and routes instant messages between client devices.

To perform these and other functions, the IM server120includes an IM server memory275and an IM service module280. The IM server memory275includes account information285, rosters290, presence information295, security keys300, and an offline message queue305. The IM service module280includes an account management module310, roster management module315, presence module320, message routing module325, and session establishing module330.

Upon registration of a client device, the account management module310creates an account entry within the account information285. The account entry includes, for instance, a unique identification number (UID) for the client device, a personal identification number (PIN), an email address, a password, a phone number, and a nickname.

A client-specific roster is also created upon registration of each client device and the roster is stored in rosters290. Each roster within rosters290is associated with a client device and is identifiable by the client device's UID. Each roster is initially set up to include the BOT130as a contact (also known as a “friend” or “buddy”). The roster management module315is used to create, edit, and delete roster information within rosters290. To add additional contacts to a roster, a secure registration process is used, which is described in more detail below. A roster entry includes the associated client device's UID, a list of UIDs for contacts of that client device, and various permissions or restrictions for each contact.

The presence module320maintains presence information295, which indicates the availability of client devices indexable using their respective UIDs.FIG. 5Aillustrates the communications flow of presence messages332and334between the remote device105, IM server120, and medical workstation110. Upon establishing a session between a client device (e.g., remote device105) and the IM server120, the remote device105sends a presence message332to the IM server120(seeFIG. 5A). In response, the presence module320updates the presence information295for the remote device105. In particular, the entry for the remote device105is updated to indicate that the remote device105is available for IM communications. The presence module320also determines when the session has ended between the IM server120and the remote device105, for example, by receiving a presence message from the remote device105or by polling after a predetermined amount of time and not receiving an answer from the remote device105.

The presence module320uses the roster management module315to retrieve roster information to identify the client devices that should be made aware of particular changes to presence information. For example, the presence module320determines which client devices coupled to the IM server120have the remote device105as a contact on their associated roster within rosters290. Those contacts identified by the presence module320are sent the presence message334to indicate a change in presence status of the remote device105to available or unavailable.FIG. 5Billustrates the peers tab271of GUI262on the medical workstation110. The peers tab271lists the roster of the medical workstation110and the related presence information. For instance, peers tab271includes a contacts column335listing the contacts on the roster of the medical workstation110, a status column336listing the presence information for each contact on the roster, and a permissions column337listing the permission level assigned to each contact on the roster. A user can also remove contacts from the roster of the medical workstation110and edit permission levels using the remove button338and edit permissions button339, respectively, on the peers tab271.

Turning back toFIG. 3, the security keys300include public keys published by client devices. The public keys for each client device are stored in the security keys300by the account management module310upon registration. As is discussed in greater detail below, the public keys are used by client devices to encrypt and decrypt messages and the identities of message senders.

The session establishing module330is used to establish IM communication sessions between the IM server120and client devices. For instance, the session establishing module330receives initial handshaking messages from the remote device105requesting to start a session and identifying itself. The session establishing module330then verifies that the remote device105is permitted to establish a session by comparing provided information (e.g., UID and password) with data within the account information285.

The message routing module325is responsible for receiving messages from client devices and routing the messages appropriately. For instance, a presence message received from a client device is routed to the presence module320. A message from one client device (e.g., the remote device105) addressed to another client device (e.g., to the medical workstation110) is routed by the message routing module325to the addressed recipient (e.g., the medical workstation110).

The offline message queue305is used by the message routing module325if the IM server120receives a message for an intended recipient that is not available, which is determined via the presence module320. In the case of an unavailable recipient, the message is temporarily stored in the offline message queue305by UID until the intended recipient becomes available. The offline message queue305purges messages in the offline message queue305if the intended recipient does not become available within a certain amount of time. Upon creation of a session with a client device, the IM server120determines whether a message for that client device is being stored in the offline message queue305and forwards the message to the client device. In some embodiments, the client device has the responsibility to poll the offline message queue305upon creation of a session with the IM server120.

Registration and Roster Management

Before the IM server120will forward messages between two client devices, each client device is registered with the IM server120and each device has the other added as a roster contact. A message from a sending client device will not be forwarded by the IM server120to a receiving client device unless 1) the sending client device is on the receiving client device's roster and 2) the receiving client device is on the sending client device's roster. A client device works with the web server125to create a new account on the IM server120and works with the BOT130and another client device to add contacts to their associated rosters within rosters290.

FIG. 6depicts the web server125according to some embodiments of the invention. The web server125is used 1) to register client devices, such as the remote device105and medical workstation110, with the IM server120and 2) to alter a client device's roster information stored in rosters290. The web server125includes a web service335with website340and an IM server communications module345with an account creation module350.

FIG. 7depicts the BOT130according to some embodiments of the invention. The BOT130is a client device of the IM server120and communicates with the IM server120according to the IM protocol of the IM server120. The BOT130is included on the roster of each client device in the rosters290automatically upon creation of an account for each device. The BOT130is used to securely add client devices to rosters to enable communication therebetween. For example, the BOT130is used to add the remote device105to the roster of the medical workstation110and vice versa.

FIG. 8depicts a method400of registering the remote device105using the web server125and updating the remote device105and medical workstation110rosters using the BOT130. The method400is separated into two sub-methods, account creation method402and roster update method404. The method400is also applicable to registering other client devices for communications via the IM infrastructure. For example, the method402is used for both remote devices, such as remote device105, and medical devices, such as medical workstation110. The method404is used for updating the rosters of a remote device and medical workstation pair, to enable communications therebetween.

In step406, a user of remote device105enters new account information into the website340. The website340includes a collection of web pages viewable on a web browser, such as web browser150and web browser220. The website340displays a form in which a user enters account information for the remote device105being registered. The account information includes, for instance, a PIN, email address, password, phone number, and a nickname for the remote device105.

The web service335of the web server125enables the website340to be accessible at a website address (i.e., at a uniform resource locator (URL) address). The website address is supplied to potential client devices including the remote device105. In some instances, the website address is pre-loaded into the client IM application145and a user is able to select a register button via the GUI of the remote device105, which causes the launch of the web browser150and display of the website340. In other instances, a user manually types the website address into the web browser150to load the website340. In still other instances, a user accesses the website340with a web browser on a computer independent of the remote device105, although some modifications to the method402may be needed. For example, the independent computer will provide some of the data generated during the method402to the remote device105.

Also in step406, the remote device105generates a public/private key pair. The private key is stored locally on the remote device105and the public key is sent to the web server125as part of the account information.

In step408, the IM server120receives a new account request from the web server125. The new account request includes account information entered via the website340(the user-entered account information). The account creation module350communicates the user-entered account information to the account management module310of the IM server120.

In step410, the account management module310uses the received user-entered account information to generate a new UID and to create a new account entry within the account information285. Also in step410, the public key of the remote device105is stored in security keys300. Additionally in step410, the roster management module315creates a new roster for the UID and includes a UID of the BOT130as a contact on the new roster. Upon successful creation of an account and roster, the IM server120communicates a confirmation message to the web server125, which forwards the confirmation to the remote device105. For instance, the confirmation is displayed on the website340for viewing by the user on the web browser150.

In addition, the account information, including the private key, is stored locally on the remote device105. Before being stored, this account information is encrypted using the PIN. In some embodiments, the encrypting and local storing occurs in step406rather than step410.

FIG. 9Aillustrates communications between the remote device105, IM server120and web server125used to complete the account creation method402. Although described with respect to remote device105, the account creation method402is also applicable to adding accounts for other client devices, such as medical workstation110.

In some embodiments, the remote device105communicates with the website340and web server125without using web browser150. For instance, the client IM application145provides a GUI for a user to input registration information and communicates the registration information to the web server125.FIGS. 9B and 9Cillustrate screen shots of an exemplary GUI for receiving user registration information. In some instances, the registration information is sent over multiple messages from the remote device105to the web server125. For example, a first message includes a user name, password, and email address, while a second message includes a PIN.

Roster Update

After creation of its account on the IM server120, the remote device105includes BOT130as a contact on its rosters within rosters290. However, the remote device105and medical workstation110are not listed as contacts on each others' rosters and, therefore, are not yet able to exchange instant messages via the IM server. The roster update method404ofFIG. 8includes steps used to add the remote device105to the medical workstation110rosters and vice versa.FIG. 10Aillustrates the communications between the remote device105, medical workstation110, IM server120, and BOT130used to complete the roster update method404ofFIG. 8.

In step412, the BOT130receives a request from the medical workstation110for a unique connection ID (generate ID message413). For instance, a user or another individual present at the medical workstation110uses the service application210to request a unique connection ID. The service application210GUI provides a form for entering a nickname of the medical workstation110and specifying access rights for the remote device105, and a request button that submits the request upon activation (e.g., using a mouse click). For instance,FIG. 10Billustrates the ID request tab270of the GUI262. On the ID request tab270, a user at medical workstation110can enter a nickname in box415, set a permission level using check box417, and request an ID using the request ID button419for the remote device105.

In response to the request, the BOT130uses the unique connection ID generator360to generate a unique connection ID with a set lifespan (e.g., ten minutes). The BOT130stores the generated unique connection ID with the UID of the requesting medical workstation110and the set lifespan in the temporary unique ID storage365. The lifespan may be stored by calculating and storing the absolute end time of the unique connection ID (e.g., 10:33 am) or by storing the creation time (e.g., 10:23 am) and the duration of the lifespan (e.g., ten minutes).

In step414, the BOT130replies to the medical workstation110with a generate ID response message421including the generated unique connection ID and the lifespan. The medical workstation110then displays the generated unique connection ID and the lifespan on display245. For instance, the ID request tab270ofFIG. 10Bincludes an ID display423that displays the generated unique connection ID. In some instances, the lifespan is displayed as a timer that counts down the remaining time of the lifespan. The generated unique connection ID and nickname are provided to a user of the remote device105via some other out-of-band communication in addition to or in place of displaying them on the display245. An out-of-band communication is a communication that occurs independent of the IM server120, such as an email, phone call, text message, etc.

In step416, the remote device105sends a register ID message425to the BOT130requesting a connection to the medical workstation110. The request message includes the generated unique connection ID and a nickname to describe the medical workstation110.FIG. 10Cillustrates an exemplary GUI for remote device105used to enter a nickname for the medical workstation110and the generated unique connection ID in boxes427and429respectively. After entering the nickname and generated unique connection ID, a user selects the connect button431to cause the remote device105to generate and send the register ID message425to the BOT130.

In step418, the verification module370of the BOT130determines whether the unique connection ID and nickname match an entry within the temporary unique ID storage365. The verification module370also determines whether the lifespan of the unique connection ID has expired in step418. If the verification module370cannot verify the existence of a matching entry or an unexpired lifespan for the unique connection ID, it returns a failure message to the remote device105in step420. Upon verification of a matching entry and an unexpired lifespan, however, the BOT130sends a roster update request message433to the IM server120to update rosters290in step422. In step422, the roster update module375is used to communicate with the roster management module315of the IM server120to update rosters290. In particular, the roster of the remote device105is updated to include the medical workstation110and the roster of the medical workstation110is updated to include the remote device105. In some instances, the BOT130or the IM server120sends a confirmation message indicating that the roster update was successful. Thereafter, the remote device105and medical workstation110are operable to communicate with each other using instant messages via the IM server120.

The BOT130communicates with the roster management module315using instant messages according to the IM protocol of the IM server120. In some instances, however, the BOT130has the ability to directly manipulate the rosters290. Although the BOT130is depicted as an independent computing device inFIG. 1, the BOT130is deployed in the web server125or IM server120in other embodiments.

During the roster update performed by the BOT130, the remote device105is assigned one of two permission/access levels, full access or limited access. The user at the medical workstation110chooses which permission level to assign the remote device105. With full access, the remote device105is able to obtain and transfer any image in the medical image database235of medical workstation110. With limited access, the remote device105has access to no image data by default. Instead, access to images are granted by the medical workstation110(e.g., via service application210) on a patient-by-patient basis or image-by-image basis, for example. In some instances, other permission levels can be assigned to the remote device105. For instance, various permission levels can be used to selectively enable and disable features (e.g., moving images between workstations).

IM Protocol

After registering the remote device105and medical workstation110and adding each device on the other's roster as a contact, the devices are able to communicate via the IM server120. Communications within system100occur in “conversations” between two client devices, such as between the remote device105and medical workstation110. The conversations include transfers of text and data that conform to an IM protocol of the IM infrastructure. The conversations are used, for instance, by the remote device105to query and control the medical workstation110using instant messages.

In embodiments of the invention, the conversations between client devices occur in the background out of a user's view. For instance, the client IM application145GUI on the remote device105provides a user-friendly interface for navigating the medical image database235of the medical workstation110. When a user selects a particular image, queries the medical workstation110, or takes some other action on the GUI that requires interaction with the medical workstation110, the client IM application145generates an instant message that is sent to the medical workstation110via the IM server120. The instant message is not viewable by the user of the remote device105or the medical workstation110. Rather, the instant message is transparent to the user. The instant message is generated in the background by the client IM application145and is received and acted upon by the service application210(via the API215). Thus, no “chat window” or the like is generally displayed for the user of the remote device105to converse with the medical workstation110. Additionally, unlike some instant messaging systems, users are not made aware of their associated instant messaging account information. In some embodiments, however, users are made aware of their associated instant messaging account information.

The messages communicated between the remote device105and the medical workstation110follow an IM protocol of the IM server120. In some embodiments, the IM protocol is an enhanced version of the Extensible Messaging and Presence Protocol (XMPP) protocol including both standard XMPP message formats and new message formats. The IM protocol includes several message types including: presence messages, generateID/registerID messages, roster messages, publish/subscription messages, query messages, get destinations messages, image move messages, and image retrieve messages. The presence messages332and334were described above with respect toFIG. 5Aand the generated/registerID messages (generate ID message413, generate ID response message421, register ID message425, and roster update request message433) were described above with respect toFIG. 10Aand the roster update method404.

In addition to the public and private key secure communications described below, an additional layer of secure communication occurs by use of a Transport Layer Security (TLS) or Secure Sockets Layer (SSL) cryptographic protocol. The TLS and SSL protocols allow applications on the remote device105, IM server120, medical workstation110, web server125, and BOT130to communicate across the network115in a way designed to prevent eavesdropping and tampering. The TLS and SSL protocols provide security for communications over the networks115by encrypting segments of network115connections at an application layer to ensure secure end-to-end transit at the transport layer. Thus, even for those messages that may not be encrypted and signed as described below (e.g., a presence message), the messages may be securely communicated using the SSL or TLS protocol. For those messages that are encrypted and signed as described below (e.g., a query message), the messages may be securely communicated using the SSL or TLS protocol as an additional layer of security.

Request Roster Messages

The roster messages used in the system100may be similar to XMPP roster messages. Roster messages are generated and sent by client devices, such as the remote device105or medical workstation110, to determine which contacts of the client device are available for communication. The client devices are operable to send roster messages to the IM server120automatically (e.g., after a session is started) and periodically.

The communication flow for roster messages between the remote device105and the IM server120is depicted inFIG. 11A. The remote device105generates and sends to the IM server120a roster request message450. The roster management module315accesses the roster of the remote device105within the rosters290to determine which client devices are listed as contacts on the particular roster (step452). Thereafter, the presence module320obtains the presence information from the presence information295for each listed contact (step454). The IM service module280then generates a response roster message456including the contacts of the remote device105and their associated presence information (i.e., availability).

The response roster message456is communicated to the remote device105. The client IM application145is then operable to display on the display175a list of available and/or unavailable contacts of the remote device105.FIG. 11Billustrates an exemplary GUI screen for the remote device105that is used to display roster information returned from the IM server120in the response roster message456. Each contact on the roster of the remote device105(except the BOT) is displayed and a scrolling action allows a user to view contacts that do not fit onto the display175. Each contact includes an indicator to indicate the contact's availability. For instance, inFIG. 11B, medical workstation110and “webserver” are available while “mark laptop” and “pascal” are not available. The GUI inFIG. 11Balso includes an add contact button457, which brings a user to the screen ofFIG. 10C.

Subscription and Publication Messages

As noted above, each client device generates a public/private key pair when registering with the IM server120. The private key is stored locally on the client device (e.g., in security keys165or security keys230) and the public key is stored in the IM server120in security keys300. The public/private key pair is part of a public-key cryptographic infrastructure used for secure data transfers between client devices. In public-key cryptography, a private key is kept secret by one user and the public key may be made widely available. Messages are encrypted with the intended recipient's public key and can only be decrypted with the corresponding private key. In the context of system100, the remote device105obtains a public key for the medical workstation110and vice versa. A message from the medical workstation110to the remote device105is encrypted by the medical workstation110with the public key of the remote device105. Upon receipt of the message, the remote device105decrypts the message using the secret private key. If the encrypted message is intercepted, it cannot be read as the intercepting device will not have the private key necessary for decryption. Other cryptographic approaches are used in some embodiments of the invention to maintain secure communications between the remote device105and medical workstation110.

Additionally, certain messages include a digital signature encrypted with the sender's private key (also referred to as a certificate). The recipient uses the sender's public key to decrypt the signature and verify the authenticity of the message's origin. In some embodiments, the public keys are published, retrieved, and managed using the XMPP Extension XEP-0189: Public Key Publishing. Per XEP-0189, the certificates are X.509 certificates and are encoded according to Distinguished Encoding Rules (DER).

The publish/subscription messages enable client devices to publish their public keys to the IM server120and to determine their contacts' published public keys.FIG. 12illustrates the communication flow for a publication message sent by the medical workstation110and a subscription message sent by the remote device105. A publish message460including a public key is sent from the medical workstation110to the IM server120. The IM server120stores the public key within the security keys300portion of the IM server memory275. In some instances, a confirmation message is returned to the medical workstation110.

Thereafter, the remote device105sends a subscribe message462to the IM server120requesting to subscribe to the medical workstation110. The IM server120adds a subscription entry to the IM server memory275that notes the subscription of the remote device105to the medical workstation110(e.g., using each device's UID). The subscription entry is stored in rosters290in some instances. The IM server120sends a subscribe response message464to the remote device105with the public key of the medical workstation110. If the medical workstation110later publishes a new public key to the IM server120, the IM server120will send the new public key to the remote device105.

The publish message is automatically sent during registration of a client device in step406of method402. The subscription message is automatically sent to the IM server120while adding a contact in the roster update method404. In some instances, the BOT130generates the subscription message on behalf of a client device. In other instances, the client device sends the subscription message during or after the roster update method404.

Query Messages

The query message is used by the remote device105to determine the images on the medical workstation110that are available to the remote device105. In some embodiments, after the remote device105creates a session with the IM server120, the GUI of the client IM application145lists the available contacts, including the medical workstation110. Upon selection of the medical workstation110, a user is able to specify a query message470to be generated by the client IM application145. For instance, after selecting the medical workstation110, the GUI of the client IM application145displays at least two options: a study list and a series list. If a user selects the study list, the query message470with default settings is generated and sent to the medical workstation110automatically. Default settings may include, for instance, a limit to the number of results desired, a request for results by date (e.g., most recently added to the medical workstation110), and/or restrictions for only certain image types (e.g., x-ray image). In other instances, the user has an opportunity to alter or add to the default settings before the query message470is generated and sent. For example, the user can specify a patient identifier (name, ID, etc.), patient characteristics (sex, age), and a date/time of the image. Additionally, the user is operable to enter key words that are used as search terms to search a description of the image or other image/patient characteristics.

The query message470is encrypted with the medical workstation's public key, and a digital signature encrypted with the private key of the remote device105is included. In some instances, the query message with default settings is sent automatically upon selection of the medical workstation110.

FIG. 13illustrates the communication flow for the generated query message470sent from the remote device105to the medical workstation110via the IM server120. The remote device105sends the query message470to the IM server120. The message routing module325forwards the query message470to the medical workstation110. The medical workstation110verifies that it has the remote device105as a contact on its roster. In some embodiments, the IM server120performs the roster verification.

The API215and service application210of the medical workstation110verify the digital signature and decrypt the query message470. The service application210also verifies that the remote device105has permission to receive a response to the particular query message470. If so, the service application210performs a search of the medical image database235and generates an image results list according to the specifics of the query message470. In some instances, the image results list includes thumbnails or other visual representations of the images. In other instances, the image results list includes a textual description of each image (e.g., patient name, image date, image type, etc.).

The service application210and API215generate a query response message472including the image results list. The query response message472also includes an indication of whether there are more image results than allowed by the numeric limit of the query message470. The query response message472is then encrypted using the public key of the remote device105and a digital signature encrypted with the private key of the medical workstation110is included. The encrypted and signed query response message472is sent to the IM server120, which forwards the query response message472to the remote device105.

The remote device105verifies the signature of the query response message472and decrypts the message using the public key of the medical workstation110. The GUI of the client IM application145displays the image results list in whole or in part. For instance, the remote device105displaying the image results list is illustrated inFIG. 14A. The remote device105includes an outer case474, speaker255a, and microphone255b, and is displaying a study list475listing four studies. The study for patient3includes a “local” identifier indicating that the study was previously obtained and is stored locally on the remote device105. Therefore, a local identifier indicates that images are viewable without needing to execute additional steps to obtain them from the medical workstation110.

Upon selection of a study from the study list475, a new query message is sent to the medical workstation110, similar to query message470, that requests the series associated with the selected study. The medical workstation110responds with a query response message, similar to query response message472, including a list of the series associated with the selected study and a thumbnail image for each series. For example, inFIG. 14B, the remote device displays the series list associated with the patient1study from study list475. The patient1study includes four associated series and, therefore, four image thumbnails476-479are displayed. Thumbnail477has a “local” identifier attached to indicate that the study was previously requested and is already stored locally on the remote device105. Additional items in the results list that are not visible are accessible via a scrolling action. Beneath each thumbnail is a brief description of the image associated with the thumbnail. In some embodiments, text replaces thumbnails476-479and other information may be provided for each series item.

FIG. 14Cillustrates an exemplary GUI screen for the remote device105that is used to display study information provided by the IM server in a query response message472. The studies are grouped by patient. If a patient has more than one study, such as patient3, a user can scroll left and right between that patient's studies while the other patients remain in the same vertical position. For instance, remote device105ofFIG. 14Cis displaying one study for patient2(study480) while the user is moving between two different studies of patient3(studies481and482).

The GUI inFIG. 14Calso includes a search button483, which causes the GUI screen ofFIG. 14Dto be displayed.FIG. 14Dillustrates an exemplary GUI screen for the remote device105that is used to search the images within a remote medical image database, such as medical image database235. A user can specify three search criteria in the GUI screen ofFIG. 14D: study date range, modality, and patient name. The results of such a search may be displayed similar to the studies shown inFIG. 14C.

A user is also able to generate a query message for a series without first generating a query for a study list and selecting a study. Similar to the default query message to obtain the available studies on the medical workstation110, a default query message to obtain the available series on the medical workstation110is also generated and sent by the remote device105in accordance with a user command. Also, a user can further specify the parameters of the default query message for a series similar to the customizable query message for a study list described above.

Image Retrieval Messages

A user can select an image of the series list ofFIG. 14Bfor transfer to the remote device105or to another medical workstation, such as the second medical workstation135.FIG. 15Aillustrates the communication flow to cause the transfer of the image associated with thumbnail476to the remote device105. Upon a user selection of thumbnail476and receipt of user input indicating a desire to retrieve the associated image (image488), the client IM application145generates an image retrieve message484with an identifier486of an image488associated with the thumbnail476. The image retrieve message484is encrypted using the public key of the medical workstation110and signed using the private key of the remote device105. The image retrieve message484is then sent to the IM server120, which forwards the image retrieve message484to the medical workstation110.

The API215and service application210of the medical workstation110verify the digital signature and decrypt the image retrieve message484. The service application210verifies that the remote device105has permission to access the image488and, if so, retrieves the image488from the medical image database235. The API215and service application210generate a retrieve response message490, which is encrypted and signed and sent to the remote device105via the IM server120. The retrieve response message490does not include the image488. Rather, the medical workstation110and remote device105form a SOCKS5 proxy connection with a proxy server492. The image488is sent from the medical workstation110to the remote device105via the SOCKS5 proxy connections and proxy server492. In some instances, the proxy server492is included in the IM server120, but still communicates according to the SOCKS5 standard.

In some instances, the image488is a Digital Imaging and Communications in Medicine (DICOM) image and the medical workstation110converts the image488to an image file type that requires less memory space, such as a Portable Network Graphics (PNG) file, before sending the image488to the remote device. In other instances, the image488is sent to the remote device105in its original format.

Upon receipt, the image488is stored locally on the remote device105within the memory160. In some embodiments, the client IM application145stores images in the memory160in a cache-style such that, when a predetermined amount of memory space for images is exceeded, the client IM application145deletes the images least recently accessed. For example,FIG. 15Billustrates an exemplary GUI screen for the remote device105that is used to alter local memory settings. A user is able to specify the maximum amount of memory160space used to store images by adjusting the cache memory scroll bar491, clear the memory160of stored images using the clear cache button493, and review cache information. The cache information displayed includes the amount of memory space currently used to store images (cache size) and the total free memory space available on the remote device105(free space).

In some embodiments, the user is able to search, browse, view, and delete the images stored locally in the memory160using the client IM application145even when not connected to any external device, such as the IM server120. In some instances, the client IM application145will still require the user to enter his or her PIN number before allowing access to stored images.

After receipt of the image488, the image488is displayed on the display175of the remote device105via an image viewer of the client IM application145. The user is able to manipulate the display of the image488by using the touch screen capabilities of display175or the user input250. For instance, the user is able to zoom in/out, pan up/down/left/right, and reset to the original view of the image. The user is also able to select to view the image488text information overlaid on the image or on a text bar outside the image boundaries (e.g., below the image). In the case of a series with multiple images, the user is also able to scroll between the related images of the series.

FIG. 15Cprovides another illustration of the communication flow ofFIG. 15A, without the retrieve response message490. Although a doctor is depicted using the remote device105, other persons, such as medical professionals and patients, are potential users of the remote device105.

Image Move Messages

As noted above, a user can select an image of the image results list for transfer to another medical workstation, such as the second medical workstation135. First, however, the remote device105is used to determine which other medical workstations are available as destinations for a selected image.FIG. 16illustrates the communication flow between the remote device105, medical workstation110, and IM server120when a remote device105seeks the available destinations.

Upon receipt of user input indicating a desire to transfer an image between medical workstations, the client IM application145generates a get destinations message494. The get destinations message494is encrypted and signed and sent to the IM server120. The IM server120forwards the get destinations message494to the medical workstation110.

The API215and service application210of the medical workstation110verify the digital signature and decrypt the get destinations message494. The service application210verifies that the remote device105has permission to get destinations and, if so, retrieves a list of medical workstations available for image transfer. The list of medical workstations may be a generally static list or a dynamically updated list similar to a roster within rosters290, maintained locally by the medical workstation110or another resource coupled to a network of the medical workstation110. The list is stored in the memory205of the medical workstation110in some instances.

The API215and service application210generate a get destinations response message496including the list of medical workstation available for image transfer. The list includes an identifier for each medical workstation available, such as a nickname or a network address. The get destinations response message496is sent to the IM server120, which forwards it to the remote device105. The remote device105displays the list of available medical workstations, which includes the second medical workstation135.

Thereafter, a user is able to select one of the available workstations to receive the image488.FIG. 17Aillustrates the communication flow between the remote device105, medical workstation110, IM server120, and the second medical workstation135when a user of the remote device105selects to transfer the image488to the second medical workstation135.

The client IM application145generates an image move message500specifying the second medical workstation135as the destination and the image488as the image to be moved. The image move message500is encrypted and signed and sent to the IM server120. The IM server120forwards the image move message500to the medical workstation110.

The API215and service application210of the medical workstation110verify the digital signature and decrypt the image move message500. The service application210verifies that the remote device105has permission to move the image488. If so, the API215and service application210generate a move image response message502acknowledging the image move message500and indicating the intent to move the image488. The move image response message502does not indicate that the image488has actually been moved. The medical workstation110moves the image488to the second medical workstation135immediately or at some later time. The original image488is generally retained on the medical workstation110, while a copy of the image488is sent to the second medical workstation135. The image488is moved between workstations using, for instance, a LAN connection or some other suitable communication link.

FIG. 17Bprovides a different illustration of the communication flow ofFIG. 17A, without the move image response message502. Although a doctor is depicted using the remote device105, other persons, such as medical professionals and patients, are potential users of the remote device105.

FIG. 17Cillustrates an exemplary GUI screen for the remote device105that is used to select the second workstation135to receive an image or study on the medical workstation110. The workstations504listed were retrieved from the medical workstation110using the get destinations message494. Upon selection of one of the available workstations504, the image move message500is generated and sent by the remote device105to the medical workstation110as described above.

FIG. 18Aillustrates an exemplary GUI screen for the remote device105that is used when displaying an image506. When the image zoom/pan button508is enabled, a user can pan and zoom the image506using the touch screen display or user input250. When the image adjust button510is enabled, a user can adjust the brightness and contrast using the touch screen display or user input250. Additionally, for studies that include multiple images, a user can scroll through the images using the image scroll bar512. In some embodiments, the image scroll bar512also functions as a download status indicator to indicate how much of the image506has been received from the medical workstation110. The information button514causes the remote device105to display image information516for image506in an exemplary GUI screen as depicted inFIG. 18B. The image information516includes items such as a patient identifier number, a patient name, patient sex, date of study, study identifier number, study description, and study modality.

In some embodiments, the systems and methods described herein enable remote control of imaging workstations, remote signing of reports by radiologists, remote routing of imaging studies between locations, and/or remote access to radiology information systems and/or hospital information systems. In some embodiments, the systems and methods enable a specialist (e.g., a radiologist) or other doctor to access their system remotely. In some embodiments, the systems and methods are used by referring physicians to gain fast access to results from referred specialists and vice versa. Embodiments of the invention also enable a medical professional to send patient images to colleagues (e.g., members of a surgical team), a technologist, or other medical professionals. Additionally, embodiments of the invention provide system administrators remote access to troubleshoot a particular medical workstation110. Furthermore, embodiments of the invention provide patients access to their own information directly from a specialist or referring physician or to send images to a particular medical professional.

Although the above embodiments are directed to image browsing, searching, transferring, etc., embodiments of the invention also include remote control of the medical workstation110and other devices to perform non-image related tasks. For instance, in some embodiments, the IM infrastructure is used for communicating and/or signing medical reports.

Thus, embodiments of the invention provide, among other things, systems and methods for remote control and management of medical workstations over an IM infrastructure.