Patent Publication Number: US-2023139360-A1

Title: Medical device system performance index

Description:
INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS 
     Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are hereby incorporated by reference under 37 CFR 1.57. 
     BACKGROUND OF THE INVENTION 
     Field of the Invention 
     As technology becomes increasingly computer-based, medical devices are more commonly utilizing electronic features that interact with a larger system. For example, a hospital information system can transfer data to and from a medication management unit, which can facilitate communication with a plurality of specific hospital beds or medical devices to record and transmit treatment parameters for patients. In another example, medical devices can be programmed to notify clinicians when certain alarms or occurrences of a particular event are triggered. Furthermore, medical facilities routinely utilize electronic databases such as drug libraries and bar code systems to improve the administration of medication and prevent human errors. Thus, an electronic network system can be quite extensive and complex in a typical hospital setting due to the interaction of the various system components. 
     Infusion pumps are one type of medical device and are used for intravenous delivery of medicines such as insulin, analgesics, sedatives, vasopressors, heparin and anti-arrhythmics to patients. Correct delivery of these medications is important for avoiding adverse events, particularly in critically ill patients. Smart infusion pumps, which include drug libraries and integrated decision support software in their medication delivery systems, have decreased errors in administration of medications by incorporating features such as hard and soft alarm limits, clinician messaging, and medication barcode input. Smart pumps are also able to utilize electronic medical records and inputs customizable for specific clinical care areas, wards or to improve safety for individual patients. Other infusion systems have incorporated features for a specific disease, such as algorithms to change the rates of insulin delivery based on a patient&#39;s glucose level, or to offer procedures specifically for advanced cardiac life support. 
     SUMMARY OF THE INVENTION 
     A distributed network system and method includes a processing unit configured to manage safety data for a plurality of medical devices, a database software component in communication with the processing unit, and a monitoring software component in communication with the processing unit. The monitoring software component is configured to monitor a number of messages between a number of medical devices and the processing unit, to process performance parameters to generate an overall performance index, and to generate an output that is viewable by a user. The output includes relative contributions of each of the performance parameters to the overall performance index, where the overall performance index is generated using a weighting factor associated with each of the performance parameters. The performance parameters include the number of messages waiting to be processed, which has the largest weighting factor, and a disk queue length, which has the smallest weighting factor. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Each of the aspects and embodiments of the invention described herein can be used alone or in combination with one another. The aspects and embodiments will now be described with reference to the attached drawings. 
         FIG.  1    shows a system context diagram of an exemplary medical device network system; 
         FIG.  2    is a technical block diagram of a medical device network, in one embodiment; 
         FIG.  3    is an exemplary block diagram of a single server configuration; 
         FIG.  4    is an exemplary block diagram of a distributed server configuration; 
         FIG.  5    shows an exemplary flow chart for a method of generating a performance index; 
         FIG.  6    shows an equation for calculating a performance index, in one embodiment; 
         FIG.  7    provides an exemplary graphical output of a performance index analysis; and 
         FIG.  8    provides another graphical output with tabular results of a performance index analysis. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
       FIG.  1    provides a system context diagram of an exemplary medical device network system  100 . The system  100  includes a medical device management component  110  that is configured to manage safety data for a plurality of medical devices. In this figure, the management component  110  is shown as Hospira MedNet™ Software, manufactured and sold by Hospira, Inc., the assignee of the present disclosure. However, other medical device management systems may be utilized instead. Similarly, the medical devices described herein shall be referred to as infusers or infusion pumps; however, other types of medical equipment with electronic data interfaces such as hospital beds, patient monitoring units, or surgical devices are applicable. 
     In  FIG.  1   , the medical device management component  110  has a software server (Hospira MedNet™ Software Server “HMSS”) that interfaces to a medical device such as an infuser  120 , and also provides communication with other systems such as a third-party clinical system  130  and a pharmacy information system  140 . Third-party clinical system  130  may be, for example, a bar code medication administration (BCMA), hospital information system (HIS), or electronic medical record (EMR). Thus, the medical device management component  110  manages information, such as IV infusion information, for medical treatment of a patient and helps to reduce medication errors, improve quality of care, streamline clinical workflows and deliver potential cost savings. 
     The configurations for medical device network system  100  can vary widely in scope, depending on many factors. For instance, the size of the facility in which network system  100  is used may vary from a private practice, with a few medical devices  120 , to a large hospital with a large number of devices  120  linked to the network. Similarly, the hardware associated with the system  100  can vary widely in configuration such as in the number of processors, the memory capacity, and processing capability of the hardware. Furthermore, the amount of transactions generated within the system  100  will depend on the operational needs of that particular facility. Thus, it would be desirable to be able to evaluate or to predict the performance of a particular configuration of a medical device management system to improve or optimize the configuration for a particular application or facility. 
       FIG.  2    is a technical block diagram of an embodiment of a medical device network system  200  that includes a medical device management component  210  (e.g., Hospira MedNet™ Software Server) having an application server  220  and a database management server  230 . The application server  220  is a processing unit that may operate on, for example, a JBoss™ or Java™ platform. The database management server  230  is in communication with the application server  220  and may be operate using any database programming language, for example, Structured Query Language (SQL). In this embodiment, database server  230  communicates with application server  220 , with a drug library  212  (embodied here as Hospira MedNet™ Meds™), and/or with a pharmacy information system  214  via an integration engine  216 . Thus, application server  220  manages data for medical devices  250   a  and  250   b  by accessing medication information from drug library  212  and pharmacy information system  214 , and by interfacing with database  230  to retrieve and store data. 
     Application server  220  may include various components such as a user interface  222  for communicating with a client device  240 , a data access component  224  to communicate with the database management server  230 , and additional interfaces  226 . Additional interfaces  226  may be configured to communicate with medical devices  250   a/b , a third-party clinical system, a lightweight directory access protocol (LDAP) directory service  270 , and a third-party asset tracking system  280 . Medical devices  250   a  and  250   b , embodied here as infusers, are outside the medical management system  210  and may interface with medical management system  210  using protocols that are health device profile (HDP) compatible or non-HDP compatible. 
     In  FIG.  2   , it can be seen that system  200  involves many mutually, dependent programs that operate within an interdependent computer system. Since medical device management component  210 , medical devices  250   a/b , third-party clinical system  260 , and other components may each be provided from different suppliers, the communication between all the various components may require interfacing between multiple program languages and communication protocols. For example, in  FIG.  2   , medical network system  200  accommodates SQL and Java™ languages, using HTTP/HTML, HTTP/XML, HTTP/SOAP and LDAP. Thus, the present systems and methods provide evaluation of medical device network system in which the system involves various platforms integrated together. Furthermore, the present systems and methods provide performance data for the overall network, and not just individual transactions. Data is gathered from the network and then processed and presented in a manner that allows a user to identify where bottlenecks or problems are occurring in the system. 
     The ability to evaluate or predict the performance of the system for various hardware and software configurations of a medical device system network, as described above, requires the integration of many parameters. Furthermore, identifying what parameters to evaluate, and how to integrate them to produce meaningful metrics can be burdensome. In the present disclosure, a performance index is described that not only derives an overall index value for a medical device network system, but also provides information on the relative contributions of the various performance parameters to the index. These relative contributions can be outputted, for example, in a graphical display to facilitate interpretation of the analysis by a user. The ability to view the impact of the performance parameters on the overall system enables a user to correct for detected problems. For example, a user can make adjustments to the medical device management software (e.g., Hospira MedNet™) to address specific problems. In one exemplary adjustment, a user may, tune a SQL statement to help it improve the utilization of SQL to retrieve data, to further minimize latency around processing of messages. In other types of adjustments, the machine configuration within a computer may be altered to improve processing of messages, such as by modifying the hardware to increase the number of cores and threads. Thus, the evaluation can demonstrate value from a total cost of ownership standpoint, such as by identifying whether hardware for a particular network system needs to be changed to meet performance goals. For instance, the thread count, core count and memory of a system could be simulated and evaluated to accommodate a desired number of active objects running at the same time. The evaluations provided by the present methods can also enable performance comparisons of the impact that various medical device products have on system performance when connected to the network. 
     The performance index is generated by a monitoring software component that is in communication with the processing unit of the medical device management component. The monitoring software component is configured to i) monitor a number of messages between a number of medical devices and the processing unit, ii) process performance parameters to generate an overall performance index, and iii) generate an output that is viewable by a user, wherein the output includes relative contributions of each of the performance parameters to the overall performance index. The processing unit, the database software component, and the monitoring software component are housed on at least one computer. For example, in some embodiments the processing unit, the database software component and the monitoring software component may all be on separate computers. In other embodiments, the medical device management system processing unit and the database software component may be on one computer while the monitoring software component may be housed on a separate computer. In still further embodiments, the database software component, monitoring software component, and medical device management system processing unit are all located on separate computers. 
       FIG.  3    depicts a diagram of a medical device network system  300  in a single server configuration. Similar to the previous figures, medical device network system  300  has a medical device management system  310  that includes an application server  320  and a database server  330 . The application server  320  is a processing unit that is in communication with and is hosted together on a single server with database server  330 . Application server  320  is also in communication with third-party clinical system/integration engine  360 , as well as with medical device  350 , which is embodied here as an infusion pump. Medical device  350  in diagram  300  may refer to one or more medical devices. Third-party clinical system  360  may provide auto programs to application server  320 , and may receive auto documentation from application server  320 . For instance, auto programs and auto documentation may assist in automatically programming the medical device  350  and charting of that information. Medical device  350  may receive auto program requests from application server  320 , and may send device and therapy data to application server  320 . 
       FIG.  4    is similar to  FIG.  3    but for a medical device network system  400  having a distributed server configuration. That is, application server  420  is housed on a first computer, while database server  430  is on a second computer. Medical device  450  and third-party clinical system/integration engine  460  are the same as in  FIG.  3   . 
     In a network system, an important aspect of performance is the efficiency of processing messages through the system. This efficiency can be evaluated in many ways, depending on what measurement elements are utilized in the calculations. In the development of the present methods, parameters that affect performance were first identified, such as the central processing unit (CPU) consumption, processor queue length (PQL), disk queue length (DQL), and number of messages waiting in the medical device management software queue. For the purposes of this disclosure, the medical device management software will be described as a Java Messaging Service™ (JMS) system, although other types of programs may be substituted. After the performance parameters were identified, they were placed in order of importance. In one embodiment, the categories were ordered as JMS backlog, CPU, Memory usage, then Disk Input/Output. Based on their importance, a weighting percentage was distributed accordingly, for formulation of the performance index. 
     In some embodiments, the monitoring software component monitors actual traffic—that is, messages—within an operational medical device network system. The output results of performance index and contributions of the various performance parameters can then be used to identify, for instance, if the system has sufficient capacity for a certain number of medical devices, and where bottlenecks in system efficiency may be occurring. In other embodiments, the monitoring software simulates a number of medical devices that would be connected to the network, and simulates a number of messages generated by the medical devices. For such a simulation, the monitoring software enables a user to, for example, vary an anticipated number of medical devices connected to the network and the number of messages generated from the medical devices, to determine performance for projected configurations and optimize the system accordingly. For simulation scenarios, medical device  350  or  450  of  FIGS.  3  and  4    may be a server that houses the monitoring software, external to the medical device management component, to simulate the load on the medical device network system during operation. 
     Performance parameters that are used for calculating system performance metrics involve factors related to the management software component, the database component, and the medical devices. In this disclosure, the following terminology shall be used:
         CPU=Central Processing Unit or Processor of a machine.   DQL=Disk Queue Length. The average disk queue lengths represent the average number of both outstanding read and write requests at a given time.   HMSS=Hospira MedNet™ Server Suite, which represents a medical device management component, such as a software program housed on a first processing unit.   JMS=Java Messaging Service™. This is a set of interfaces for sending messages between two or more clients. [0030] PQL=Processor Queue Length. The processor queue length is an indicator for the number of threads waiting to be processed.   SQL=Structured Query Language. In this disclosure, the term shall be used to reference the database server, although in other embodiments, other database languages may be substituted.   Perfmon=Windows™ Performance Monitor. This is a tool within the server that holds the Hospira Mednet™ software, that is used to collect the data and statistics of the network system. In other embodiments, other tools such Unix™ or Linux™ equivalents are possible.       

     The following variables for calculation of the performance index in this disclosure are described below:
         HmssCPU=Observed HMSS CPU consumption. (CPU of the medical device management system) The consumption value is normalized to a percentage, such as 45% CPU Usage.   HmssPQL=Observed HMSS processor queue length. (PQL of the medical device management system) This value is taken from Perfmon. A maximum allowable value is set according to desired goals, such as number of cores*1.5. For example, if the CPU of the machine/VM have 4 cores, the maximum allowable queue length is 6. In other embodiments, the allowable tolerance may be 2-4 times the number of cores.   HmssDQL=Observed HMSS disk queue length. (DQL of the medical device management system) This value is taken from Perfmon. A maximum allowable value is set according to desired goals, such as a recommended queue length of less than 2. Higher values portend application performance degradation due to input/output ( 110 ) latency.   Sq1CPU=Observed SQL (or other database) CPU consumption. This is the same as HmssCPU but measured for the database server.   Sq1PQL=Observed SQL (or other database) processor queue length. This is the same as HmssPQL but measured for the database server.   Sq1DQL=Observed SQL (or other database) disk queue length. This is the same as HmssDQL but measured for the database server.   NumOfJMSMsg=Observed number of JMS messages waiting to be processed. This value is taken from the JMS queue. A maximum allowable value is set according to desired goals, such as the maximum value being the Total Number of Infusers* 10 . For example, if a test consists of 100 infusers, JMS backlog of 1000 or more as measured at the end of the test is considered a failure.   MemUsage=HMSS (or other medical device management system) memory usage.   MaxIndexNumber=The total maximum value used for the Performance Index.       

     In some embodiments, a lower value indicates better performance. 
       FIG.  5    shows an exemplary flow diagram  500  of the monitoring software in some embodiments. In step  510 , performance parameters are monitored and gathered for evaluation. Step  510  may be conducted by custom software or by pre-packaged software. In one embodiment, step  510  utilizes Windows™ PerfMon to specify collector sets, such as queue length. Use of a program such as PerfMon can enable the monitoring software to be transportable to any type of server. The performance parameters may be gathered from a distributed network, such as a network including processing unit acting as a medical device management server, a database server, medical devices outside the medical device management server, and third-party interfaces. In step  520 , the medical devices can optionally be simulated by the monitoring software program, such as by simulating the number of medical devices connected to the network, the type of devices, and the number of messages being generated by the devices. Because different types of devices, such as different models of infusions pumps, manifest traffic in different ways, the monitoring software can be programmed for a user to test different types and combinations of these devices. In other embodiments where performance of an actual physical system is being evaluated, step  520  may be omitted. 
     In step  530 , the data is processed. For example, a set of data gathered within a specified time period may be aggregated for calculation of a set of performance metrics. Calculation of the performance parameters utilizes weighting factors associated with each performance parameter, as shall be described in more detail below, to generate an overall performance index. In step  540 , an output is generated that is viewable by a user. The output includes relative contributions of the performance parameters to the overall performance index. For example, the output may take the form of numerical data tables, vertical or horizontal bar charts, such as stacked or grouped columns. 
     Calculation of performance parameters, such as in step  530  of  FIG.  5   , shall now be described in more detail.  FIG.  6    shows an equation 600 for calculating a performance index, in one embodiment. In equation 600, the functions f(x) are defined by the entities to the right of that, indicated by curly braces. The equation can be generalized as 
     
       
         
           
             
               ( 
               
                 MaxIndexNumber 
                 * 
                 
                   ∑ 
                   
                     
                       NotedCategoryTakenResult 
                       MaxCateboryAllowedValue 
                     
                     * 
                     Category 
                     ⁢ 
                         
                     Weighted 
                     ⁢ 
                         
                     % 
                   
                 
               
               ) 
             
             . 
           
         
       
     
     That is, the performance index is generated by summing entities for all the performance parameters, the entity for each performance parameter being the performance parameter value divided by a maximum allowed value and multiplied by the weighting factor associated with the performance parameter. For example, the first entity in equation 600 adds the maximum and average HMSS CPU data, divides it by the allowed value of 2, and multiplies the total by its weighting factor “HmssCPU %.” The other performance parameters of Sq1CPU, Number of JMS Messages, HmssPQL, HmssDQL, Sq1DQL and Memory Usage are similarly calculated and summed. Note that the equation takes into account the server configuration format. Thus, for non-distributed scenarios the PQL and DQL are taken only once and the weighted percentage is combined. For example, if the system PQL=2.2 in a four core all-in-one machine, the formula would be (2.2/(4*1.5))*(0.125+0.1). 
     The weighting factor for each variable is carefully determined in relation to its contribution on system performance. A set of weighting factors, in some embodiments, is listed in Table 1 below, with a description of the weighting factors being presented in the subsequent paragraphs. 
     
       
         
           
               
               
               
             
               
                   
                 TABLE 1 
               
               
                   
                   
               
               
                   
                 Variable 
                 Weighting Factor 
               
               
                   
                   
               
             
            
               
                   
                 HmssCPU 
                 12.5% 
               
               
                   
                 HmssPQL 
                 12.5% 
               
               
                   
                 HmssDQL 
                   5% 
               
               
                   
                 SqlCPU 
                 12.5% 
               
               
                   
                 SqlPQL 
                 12.5% 
               
               
                   
                 SqlDQL 
                   5% 
               
               
                   
                 NumOfJMSMsg 
                     30% 
               
               
                   
                 MemUsage 
                     10% 
               
               
                   
                 MaxIndexNumber 
                  100% 
               
               
                   
                   
               
            
           
         
       
     
     NumOfJMSMsg—This was deemed to have the largest weighting factor, such as about 30%, because the majority of the functionality of the Hospira MedNet™ software has to do with message processing and the timeliness of when the messages are processed. An example is processing status messages sent by the infuser and in turn sending updated statuses to a third party integrator. If there is a build up of messages and a messages stays in the queue longer than desired, by the time the message is sent out it is already stale. The JMS message counter can also reveal issues within Hospira MedNet™ software that otherwise seems like Hospira MedNet™ is performing optimally. There are times when both CPU and PQL counters are low but JMS number is high. This is usually a clear indicator of a deadlock on either the JAVA side, SQL side or both. 
     CPU and PQL—These factors in Table 1 have intermediate weighting, such as about 12.5%. Raw processing and how long a process needs to wait for CPU time are the next factors. In an optimal operating environment when there are no blockages CPU is usually the limiting factor. The weighting is equally spread between CPU and PQL on both HMSS and SQL servers. 
     MemUsage—MemUsage has intermediate weighting, such as about 10%. This was deemed important because in order to have peak performance, having memory and not needing to use I/O is very crucial. A weighting factor of about 10% provides a good indicator if HMSS has a memory leak. Memory leak deals with memory clean-up and allocation, which can cause a system to fail if memory space becomes insufficient. 
     DQL—In the embodiment of Table 1 DQL has the smallest value, such as about 5%. This weighting will reveal any I/O bottlenecks in the system when running Hospira MedNet™ software, providing a relative indication of the importance of other problems. 
       FIGS.  7 - 8    show exemplary outputs generated by the performance index calculations. In  FIG.  7   , a stacked horizontal bar chart  700  shows relative contributions of various performance factors (HMSS CPU, SQL CPU, etc.), as indicated by the colored segments for each bar. The various bars represent performance component differentiation, such as from different models of infusers. In this sample chart, a comparison of the second and third bars from the top shows that the “5.81 Vanilla Plum Mix with IVCI” (IV with clinical integration) base Plum™ infuser model resulted in better system performance—that is, had a performance index that was smaller in value—than the “5.81 Vanilla Symbiq IVCI” or base Symbig™ infuser model with IVCI. Looking at the sub-segments of these bars, it can be seen that the purple “HMSS PQL” component was a primary contributor to the worsened performance of the “Vanilla” Symbig™ infuser with IVCI compared with the “Vanilla” Plum™ infuser Mix with IVCI. Consequently, a user could use these results to target how to improve processor queue length of the medical device management software (HMSS) when using the Vanilla Symbiq infuser model. 
     In  FIG.  8   , another graphical output is presented as a vertical bar chart  800  showing comparison of execution between functional differentiations; that is, between system configurations. For example, phases 1-5 represent changes made within the Hospira MedNet™ system, and the side-by-side bars of chart  800  show comparisons of the base or Vanilla Symbig™ infuser with and without IVCI when run with the different phases. As can be seen, Vanilla Symbig™ infuser with IVCI (blue bars) had higher performance index values and thus worse performance than Vanilla Symbig™ infuser without IVCI (red bars). Table 850 shows the generated output in a numerical format, corresponding to the graphical output of chart  800 . 
     Thus,  FIGS.  7 - 8    show the ability of the performance index to evaluate the effect of different medical devices or management software configurations on the overall performance of a medical device network system. In other embodiments, the performance index could be used to compare different host configurations, with hardware and/or software adjustments. The various outputs may be displayed in different forms viewable by the user, such as, but not limited to graphical displays, numerical listings or tables, or text descriptions. The outputs may be generated as, for example, computer files, printed data, and/or displays on a computer monitor. 
     While the specification has been described in detail with respect to specific embodiments of the invention, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily conceive of alterations to, variations of, and equivalents to these embodiments. These and other modifications and variations to the present invention may be practiced by those of ordinary skill in the art, without departing from the scope of the present invention. Furthermore, those of ordinary skill in the art will appreciate that the foregoing description is by way of example only, and is not intended to limit the invention. Thus, it is intended that the present subject matter covers such modifications and variations.