Patent ID: 12246188

Like reference characters refer to like elements throughout the figures and description.

DETAILED DESCRIPTION

FIG.1is a block diagram illustrating a system10for monitoring a patient for cardiac arrhythmias in accordance with the techniques of the disclosure. System10includes a medical device16. One example of such a medical device is an implantable medical device (IMD), as shown inFIG.1. As illustrated by example system10inFIG.1, medical device16may, in some examples, be an implantable cardiac monitor, an implantable cardiac pacemaker, implantable cardioverter/defibrillator (ICD), or pacemaker/cardioverter/defibrillator, for example. In some examples, medical device16is a non-implantable medical device, such as a non-implantable cardiac monitor (e.g., a Holter monitor).

In the example ofFIG.1, medical device16is connected to leads18,20and22and is communicatively coupled to external device27, which in turn is communicatively coupled to computing system24over communication network25. Medical device16senses electrical signals attendant to the depolarization and repolarization of heart12, e.g., a cardiac electrogram (EGM), via electrodes on one or more leads18,20and22or the housing of medical device16. Medical device16may also deliver therapy in the form of electrical signals to heart12via electrodes located on one or more leads18,20and22or a housing of medical device16. The therapy may be pacing, cardioversion and/or defibrillation pulses. Medical device16may monitor EGM signals collected by electrodes on leads18,20or22, and based on the EGM signal, diagnose, and treat cardiac arrhythmias.

In some examples, medical device16includes communication circuitry17including any suitable circuitry, firmware, software, or any combination thereof for communicating with another device, such as external device27ofFIG.1. For example, communication circuitry17may include one or more processors, memory, wireless radios, antennae, transmitters, receivers, modulation and demodulation circuitry, filters, amplifiers, or the like for radio frequency communication with other devices, such as computing system24. Medical device16may use communication circuitry17to receive downlinked data to control one or more operations of medical device16and/or send uplinked data to external device27.

Leads18,20,22extend into the heart12of patient14to sense electrical activity of heart12and/or deliver electrical stimulation to heart12. In the example shown inFIG.1, right ventricular (RV) lead18extends through one or more veins (not shown), the superior vena cava (not shown), and right atrium26, and into right ventricle28. Left ventricular (LV) lead20extends through one or more veins, the vena cava, right atrium26, and into the coronary sinus30to a region adjacent to the free wall of left ventricle32of heart12. Right atrial (RA) lead22extends through one or more veins and the vena cava, and into the right atrium26of heart12.

While example system10ofFIG.1depicts medical device16, in other examples, the techniques of the disclosure may be applied to other types of medical devices that are not necessarily implantable. For example, a medical device in accordance with the techniques of the disclosure may include a wearable medical device or “smart” apparel worn by patient14. For example, such a medical device may take the form of a wristwatch worn by patient14or circuitry that is adhesively affixed to patient14, such as the Seeq™ Mobile Cardiac Telemetry System commercially available from Medtronic plc of Dublin, Ireland. In another example, a medical device as described herein may include an external medical device with implantable electrodes.

In some examples, external device27takes the form of an external programmer or mobile device, such as a mobile phone, a “smart” phone, a laptop, a tablet computer, a personal digital assistant (PDA), etc. In some examples, external device27is a CareLink™ monitor available from Medtronic, Inc. A user, such as a physician, technician, surgeon, electro-physiologist, or other clinician, may interact with external device27to retrieve physiological or diagnostic information from medical device16. A user, such as patient14or a clinician as described above, may also interact with external device27to program medical device16, e.g., select or adjust values for operational parameters of medical device16. External device27may include processing circuitry, a memory, a user interface, and communication circuitry capable of transmitting and receiving information to and from each of medical device16and computing system24.

In some examples, computing system24takes the form of a handheld computing device, computer workstation, server or other networked computing device, smartphone, tablet, or external programmer that includes a user interface for presenting information to and receiving input from a user. In some examples, computing system24may include one or more devices that implement a machine learning system, such as a neural network, a deep learning system, or another type of machine learning system. A user, such as a physician, technician, surgeon, electro-physiologist, or other clinician, may interact with computing system24to retrieve physiological or diagnostic information from medical device16. A user may also interact with computing system24to program medical device16, e.g., select values for operational parameters of the IMD. Computing system24may include a processor configured to evaluate EGM and/or other sensed signals transmitted from medical device16to computing system24.

Network25may include one or more computing devices (not shown), such as one or more non-edge switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection, and/or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices such as cellular phones or personal digital assistants, wireless access points, bridges, cable modems, application accelerators, or other network devices. Network25may include one or more networks administered by service providers and may thus form part of a large-scale public network infrastructure, e.g., the Internet. Network25may provide computing devices, such as computing system24and medical device16, access to the Internet, and may provide a communication framework that allows the computing devices to communicate with one another. In some examples, network25may be a private network that provides a communication framework that allows computing system24, medical device16, and EMR database66to communicate with one another but isolates computing system24, medical device16, and EMR database66from external devices for security purposes. In some examples, the communications between computing system24, medical device16, and EMR database66are encrypted.

External device27and computing system24may communicate via wired and/or wireless communication over network25using any techniques known in the art. In some examples, computing system24is a remote device that communicates with external device27via an intermediary device located in network25, such as a local access point, wireless router, or gateway. While in the example ofFIG.1, external device27and computing system24communicate over network25, in some examples, external device27and computing system24communicate with one another directly. Examples of communication techniques may include, for example, communication according to the Bluetooth® or BLE protocols. Other communication techniques are also contemplated. Computing system24may also communicate with one or more other external devices using a number of known communication techniques, both wired and wireless. In some examples, computing system24and network25may implement and provide access to the Carelink™ system administered by Medtronic plc.

EMR database66stores EMR data for patient14. EMR database66may include processing circuitry and one or more storage mediums (e.g., random access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), or flash memory. In some examples, EMR database66is a cloud computing system. In some examples, the functions of EMR database66are distributed across a number of computing systems.

In one example, computing system24receives patient data collected by medical device16of patient14. In some examples, the patient data includes physiological data for patient14, such as one or more of an activity level of patient14, a heart rate of patient14, a posture of patient14, a cardiac electrogram of patient14, a blood pressure of patient14, a pulse transit time of patient14, a respiration rate of patient14, a hypopnea index or apnea of patient14, accelerometer data for patient14, features derived from accelerometer data of patient14, such as activity counts, posture, statistical control process variables, etc., a raw electromyogram of patient14, one or more features derived from a raw electromyogram of patient14, such as heart rate variability, t-wave alternans, QRS morphology, etc., interval data and features derived from interval data, heart sounds, potassium levels, glycemic index, a temperature of patient14, or any data derivable from the aforementioned parametric data, or any other types of patient parametric data. In some examples, medical device16and/or other devices may automatically generate the patient parametric data by processing information from one or more sensors. For example, sensors of medical device16and/or other devices may determine that patient14has fallen down, patient14is frail or suffers an illness, that patient14is suffering an instance of sleep apnea.

In some examples, the patient data includes environmental data such as, air quality measurements, ozone levels, particulate counts, or pollution levels proximate to patient14, an ambient temperature, or daylight hours. In some examples, one of medical device or external device27may sense, via one or more sensors, the environmental data. In another example, the environmental data is received by external device27via an application, such as a weather application, executing on external device27, and uploaded to computing system24over network25. In another example, computing system24collects the environmental data directly from a cloud service that has location-based data for patient14.

In some examples, the patient data includes patient symptom data that is uploaded by patient14via an external device, such as external device27. For example, patient14may upload the patient symptom data via an application executing on a smart phone. In some examples, patient14may upload the upload the patient symptom data via a user interface (not depicted inFIG.1), such as by touchscreen, keyboard, graphical user interface, voice commands, etc.

In some examples, the patient data includes device-related data, such as one or more of an impedance of one or more electrodes of the medical device, a selection of electrodes, a drug delivery schedule for the medical device, a history of electrical pacing therapy delivered to the patient, or diagnostic data for the medical device. In some examples, the medical device that collects the patient data is an IMD. In other examples, the medical device that collects the patient data is another type of patient device, such as a wearable medical device or a mobile device (e.g., a smartphone) of patient14. In some examples, computing system24receives the patient data on a periodic, e.g., daily, basis.

In some examples, computing system24further receives EMR data for patient14from EMR database66. The EMR data may be considered another form of patient data. In some examples, the EMR data stored by EMR database66may include many different types of historical medical information about patient14. For example, EMR database66may store a medication history of the patient, a surgical procedure history of the patient, a hospitalization history of the patient, potassium levels of the patient over time, one or more lab test results for patient14, a cardiovascular history of patient14, or co-morbidities of patient14such as atrial fibrillation, heart failure, or diabetes, as examples.

Computing system24applies one or more machine learning models, trained using patient data for a plurality of patients, to the patient data for patient14to monitor patient14for occurrences of cardiac arrhythmias. For example, computing system24may receive patient data for patient14. The patient data be collected by one or more medical devices, such as medical device16. Furthermore, in this example, computing system24may apply a machine learning model to the patient data to determine a current probability value that indicates a probability that patient14has experienced an occurrence of a cardiac arrhythmia. In this example, computing system24may further determine whether the current probability value exceeds a probability threshold for patient14. In response to determining that the current probability value is greater than or equal to the probability threshold for patient14, computing system24may generate a notification indicating that patient14has likely experienced an occurrence of a cardiac arrhythmia. The probability value may be a value between 0 and 1; or a value in another range where the value is indicative of a probability or likelihood. Probability thresholds may be in such ranges as well.

How the probability threshold is set may determine a review burden and a diagnostic yield associated with notifications generated by computing system24. A user who reviews the notifications generated by computing system24may experience a review burden associated with the notifications because the user may need to spend time reviewing the notifications and determining whether patient14actually experienced the cardiac arrhythmias indicated by the notifications. The diagnostic yield associated with the notifications may correspond to the percentage or other measures of how many of the notifications generated by computing system24actually yield valuable diagnostic information. Should the review burden be too great, it may be impractical for the user to use computing system24to monitor patient14. Likewise, if the diagnostic yield is too low, the user may not find utility in using computing system24to monitor patient14.

Hence, in accordance with techniques of this disclosure, computing system24may enable the user to adjust a probability threshold in a way that may change the review burden and diagnostic yield associated with notifications generated by computing system24. Accordingly, in some examples of this disclosure, computing system24may generate a set of sample probability values by applying a machine learning model to a sample set of patient data. The machine learning model may be trained using patient data for a plurality of patients. Furthermore, the sample set may comprise a plurality of temporal windows. For each respective temporal window of the plurality of temporal windows, the machine learning model is configured to determine a respective probability value in the set of sample probability values that indicates a probability that a cardiac arrhythmia occurred during the respective temporal window. In some examples, each of the temporal windows of the sample set comprises a respective cardiac EGM strip.

Computing system24may generate graphical data based on the sample probability values and may output a user interface for display on a display device. The user interface may comprise the graphical data. As described in this disclosure, the graphical data may include one or more receiver-operator curves (ROCs), graphs, and/or other types of graphical data. Computing system24may receive, via the user interface, an indication of user input to select a probability threshold for patient14. For instance, computing system24may receive indications of clicking, typing, tapping, sliding, dragging, pinching, or other types of user input directed to one or more features of the user interface. Computing system24may receive patient data for patient14and may apply the machine learning model to the patient data to determine a current probability value that indicates a probability that patient14has experienced an occurrence of a cardiac arrhythmia. Computing system24may then determine whether the current probability value exceeds the probability threshold set for patient14.

In this example, in response to determining that the current probability value is greater than or equal to the probability threshold set for patient14, computing system24may generate a notification indicating that patient14has likely experienced the occurrence of the cardiac arrhythmia. Computing system24may store copies of the generated notifications. By enabling the user to appropriately set the probability threshold, computing device24may not generate and store unneeded notifications. This may conserve storage space on computer-readable media of computing system24. Furthermore, in some examples, computing system24may transmit the notifications to devices used by one or more users who are monitoring patient14. By not generating and transmitting numbers of notifications inappropriate for the diagnostic purposes of the user, network bandwidth may be conserved and battery life of the receiving devices may be conserved.

FIG.2is a conceptual diagram illustrating medical device16and leads18,20,22of system10ofFIG.1in greater detail. In the illustrated example, bipolar electrodes40and42are located adjacent to a distal end of lead18, and bipolar electrodes48and50are located adjacent to a distal end of lead22. In addition, four electrodes44,45,46and47are located adjacent to a distal end of lead20. Lead20may be referred to as a quadrapolar LV lead. In other examples, lead20may include more or fewer electrodes. In some examples, LV lead20comprises segmented electrodes, e.g., in which each of a plurality of longitudinal electrode positions of the lead, such as the positions of electrodes44,45,46and47, includes a plurality of discrete electrodes arranged at respective circumferential positions around the circumference of lead.

In the illustrated example, electrodes40and44-48take the form of ring electrodes, and electrodes42and50may take the form of extendable helix tip electrodes mounted retractably within insulative electrode heads52and56, respectively. Leads18and22also include elongated electrodes62and64, respectively, which may take the form of a coil. In some examples, each of electrodes40,42,44-48,50,62, and64is electrically coupled to a respective conductor within the lead body of its associated lead18,20,22and thereby coupled to circuitry within medical device16.

In some examples, medical device16includes one or more housing electrodes, such as housing electrode4illustrated inFIG.2, which may be formed integrally with an outer surface of hermetically-sealed housing8of medical device16or otherwise coupled to housing8. In some examples, housing electrode4is defined by an uninsulated portion of an outward facing portion of housing8of medical device16. Other divisions between insulated and uninsulated portions of housing8may be employed to define two or more housing electrodes. In some examples, a housing electrode comprises substantially all of housing8.

Housing8encloses signal generation circuitry that generates therapeutic stimulation, such as cardiac pacing, cardioversion, and defibrillation pulses, as well as sensing circuitry for sensing electrical signals attendant to the depolarization and repolarization of heart12. Housing8may also enclose a memory for storing the sensed electrical signals. Housing8may also enclose communication circuitry17for communication between medical device16and computing system24.

Medical device16senses electrical signals attendant to the depolarization and repolarization of heart12via electrodes4,40,42,44-48,50,62, and64. Medical device16may sense such electrical signals via any bipolar combination of electrodes40,42,44-48,50,62, and64. Furthermore, any of the electrodes40,42,44-48,50,62, and64may be used for unipolar sensing in combination with housing electrode4.

The illustrated numbers and configurations of leads18,20and22and electrodes are merely examples. Other configurations, i.e., number and position of leads and electrodes, are possible. In some examples, system10may include an additional lead or lead segment having one or more electrodes positioned at different locations in the cardiovascular system for sensing and/or delivering therapy to patient14. For example, instead of or in addition to intercardiac leads18,20and22, system10may include one or more epicardial or extravascular (e.g., subcutaneous or substernal) leads not positioned within heart12.

Medical device16send patient data to computing system24(e.g., by way of external device27). The patient data may include data based on the electrical signals detected by electrodes4,40,42,44-48,50,62, and/or64. For example, medical device16may gather and send cardiac EGM and other data to computing system24. In accordance with the techniques of this disclosure, computing system24may use the patient data to determine probability values that indicate probabilities that patient14has experienced occurrences of one or more cardiac arrhythmias.

Although described herein in the context of an example medical device16that provides therapeutic electrical stimulation, the techniques for disclosed herein may be used with other types of devices. For example, the techniques may be implemented with an extra-cardiac defibrillator coupled to electrodes outside of the cardiovascular system, a transcatheter pacemaker configured for implantation within the heart, such as the Micra™ transcatheter pacing system commercially available from Medtronic PLC of Dublin Ireland, an insertable cardiac monitor, such as the Reveal LINQ™ ICM, also commercially available from Medtronic PLC, a neurostimulator, a drug delivery device, a wearable device such as a wearable cardioverter defibrillator, a fitness tracker, or other wearable device, a mobile device, such as a mobile phone, a “smart” phone, a laptop, a tablet computer, a personal digital assistant (PDA), or “smart” apparel such as “smart” glasses or a “smart” watch.

FIG.3is a block diagram of example medical device16according to the techniques of the disclosure. In the illustrated example, medical device16includes processing circuitry58, memory59, communication circuitry17, sensing circuitry50, therapy delivery circuitry52, sensors57, and power source54. Memory59includes computer-readable instructions that, when executed by processing circuitry58, cause medical device16and processing circuitry58to perform various functions attributed to medical device16and processing circuitry58herein (e.g., performing short-term prediction of cardiac arrhythmias, delivering therapy, such as anti-tachycardia pacing, bradycardia pacing, and post-shock pacing therapy, etc.). Memory59may include any volatile, non-volatile, magnetic, optical, or electrical media, such as a random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), electrically-erasable programmable ROM (EEPROM), flash memory, or any other digital or analog media.

Processing circuitry58may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or analog logic circuitry. In some examples, processing circuitry58may include multiple components, such as any combination of one or more microprocessors, one or more controllers, one or more DSPs, one or more ASICs, or one or more FPGAs, as well as other discrete or integrated logic circuitry. The functions attributed to processing circuitry58herein may be embodied as software, firmware, hardware or any combination thereof.

Processing circuitry58controls therapy delivery circuitry52to deliver stimulation therapy to heart5according to therapy parameters, which may be stored in memory59. For example, processing circuitry58may control therapy delivery circuitry52to deliver electrical pulses with the amplitudes, pulse widths, frequency, or electrode polarities specified by the therapy parameters. In this manner, therapy delivery circuitry52may deliver pacing pulses (e.g., ATP pulses, bradycardia pacing pulses, or post-shock pacing therapy) to heart5via electrodes34and40. In some examples, therapy delivery circuitry52may deliver pacing stimulation, e.g., ATP therapy, bradycardia therapy, or post-shock pacing therapy, in the form of voltage or current electrical pulses. In other examples, therapy delivery circuitry52may deliver one or more of these types of stimulation in the form of other signals, such as sine waves, square waves, or other substantially continuous time signals.

Therapy delivery circuitry52is electrically coupled to electrodes34and40carried on the housing of medical device16. Although medical device16may only include two electrodes, e.g., electrodes34and40, in other examples, medical device16may utilize three or more electrodes. Medical device16may use any combination of electrodes to deliver therapy and/or detect electrical signals from patient12. In some examples, therapy delivery circuitry52includes a charging circuit, one or more pulse generators, capacitors, transformers, switching modules, and/or other components capable of generating and/or storing energy to deliver as pacing therapy, cardiac resynchronization therapy, other therapy or a combination of therapies. In some examples, therapy delivery circuitry52delivers therapy as one or more electrical pulses according to one or more therapy parameter sets defining an amplitude, a frequency, a voltage or current of the therapy, or other parameters of the therapy.

Sensing circuitry50monitors signals from one or more combinations (also referred to as vectors) of two or more electrodes from among electrodes4,40,42,44-48,50,62(FIG.2), and64(FIG.2) in order to monitor electrical activity of heart12, impedance, or other electrical phenomenon. In some examples, sensing circuitry50includes one or more analog components, digital components or a combination thereof. In some examples, sensing circuitry50includes one or more sense amplifiers, comparators, filters, rectifiers, threshold detectors, analog-to-digital converters (ADCs) or the like. In some examples, sensing circuitry50converts sensed signals to digital form and provides the digital signals to processing circuitry58for processing or analysis. In one example, sensing circuitry50amplifies signals from electrodes4,40,42,44-48,50,62, and64and converts the amplified signals to multi-bit digital signals by an ADC.

In some examples, sensing circuitry50performs sensing of the cardiac electrogram to determine heart rates or heart rate variability, or to detect arrhythmias (e.g., tachyarrhythmias or bradycardia) or to sense other parameters or events from the cardiac electrogram. Sensing circuitry50may also include switching circuitry to select which of the available electrodes (and the electrode polarity) are used to sense the heart activity, depending upon which electrode combination, or electrode vector, is used in the current sensing configuration. Processing circuitry58may control the switching circuitry to select the electrodes that function as sense electrodes and their polarity. Sensing circuitry50may include one or more detection channels, each of which may be coupled to a selected electrode configuration for detection of cardiac signals via that electrode configuration. In some examples, sensing circuitry50compares processed signals to a threshold to detect the existence of atrial or ventricular depolarizations and indicate the existence of the atrial depolarization (e.g., P-waves) or ventricular depolarizations (e.g., R-waves) to processing circuitry58. Sensing circuitry50may comprise one or more amplifiers or other circuitry for comparison of the cardiac electrogram amplitude to a threshold, which may be adjustable.

Processing circuitry58may include a timing and control module, which may be embodied as hardware, firmware, software, or any combination thereof. The timing and control module may comprise a dedicated hardware circuit, such as an ASIC, separate from other processing circuitry58components, such as a microprocessor, or a software module executed by a component of processing circuitry58, which may be a microprocessor or ASIC. The timing and control module may implement programmable counters. If medical device16is configured to generate and deliver bradycardia pacing pulses to heart12, such counters may control the basic time intervals associated with DDD, VVI, DVI, VDD, AAI, DDI, DDDR, VVIR, DVIR, VDDR, AAIR, DDIR and other modes of pacing.

Memory59may be configured to store a variety of operational parameters, therapy parameters, sensed and detected data, and any other information related to the therapy and treatment of patient12. In the example ofFIG.3, memory58may store sensed cardiac EGMs, e.g., associated with detected or predicted arrhythmias, and therapy parameters that define the delivery of therapy provided by therapy delivery circuitry52. In other examples, memory58may act as a temporary buffer for storing data until it can be uploaded to computing system24.

Communication circuitry17includes any suitable circuitry, firmware, software, or any combination thereof for communicating with another device, such as computing system24via network25ofFIG.1. For example, communication circuitry17may include one or more antennae, modulation and demodulation circuitry, filters, amplifiers, or the like for radio frequency communication with other devices, such as computing system24via network25. Under the control of processing circuitry58, communication circuitry17may receive downlink telemetry from and send uplink telemetry to computing system24with the aid of an antenna, which may be internal and/or external. Processing circuitry58may provide the data to be uplinked to computing system24and the control signals for the telemetry circuit within communication circuitry17, e.g., via an address/data bus. In some examples, communication circuitry17may provide received data to processing circuitry58via a multiplexer.

Power source54may be any type of device that is configured to hold a charge to operate the circuitry of medical device16. Power source54may be provided as a rechargeable or non-rechargeable battery. In other example, power source54may incorporate an energy scavenging system that stores electrical energy from movement of medical device16within patient12.

In accordance with the techniques of the disclosure, medical device16collects, via sensing circuitry50and/or sensors57, patient data of patient14. Sensors57may include one or more sensors, such as one or more accelerometers, pressure sensors, optical sensors for O2 saturation, etc. In some examples, the patient data includes one or more of an activity level of the patient, a heart rate of the patient, a posture of the patient, a cardiac electrogram of the patient, a blood pressure of the patient, accelerometer data for the patient, or other types of patient parametric data. Medical device16uploads, via communication circuitry17, the patient parametric data to computing system24over network25. In some examples, medical device16uploads the patient parametric data to computing system24on a daily basis. In some examples, the patient parametric data includes one or more values that represent average measurements of patient14over a long-term time period (e.g., about 24 hours to about 48 hours). For example, one or more other devices, such as a wearable medical device or a mobile device (e.g., a smartphone) of patient14, may collect the patient parametric data and upload the patient parametric data to computing system24.

Although described herein in the context of example medical device16that provides therapeutic electrical stimulation, the techniques for short-term prediction of cardiac arrhythmia disclosed herein may be used with other types of devices. For example, the techniques may be implemented with a transcatheter pacemaker configured for implantation within the heart, such as the Micra™ transcatheter pacing system commercially available from Medtronic PLC of Dublin Ireland, an insertable cardiac monitor, such as the Reveal LINQ™ ICM, also commercially available from Medtronic PLC, a neurostimulator, a drug delivery device, a wearable device such as a wearable cardioverter defibrillator, a fitness tracker, or other wearable device, a mobile device, such as a mobile phone, a “smart” phone, a laptop, a tablet computer, a personal digital assistant (PDA), or “smart” apparel such as “smart” glasses or a “smart” watch.

FIG.4is a block diagram illustrating an example computing system24that operates in accordance with one or more techniques of the present disclosure. In one example, computing system24includes processing circuitry402for executing applications424that include monitoring system450or any other applications described herein. Although shown inFIG.4as a stand-alone computing system24for purposes of example, computing system24may be any component or system that includes processing circuitry or other suitable computing environment for executing software instructions and, for example, need not necessarily include one or more elements shown inFIG.4(e.g., communication circuitry406; and in some examples components such as storage device(s)408may not be co-located or in the same chassis as other components). In some examples, computing system24may be a cloud computing system distributed across a plurality of devices.

As shown in the example ofFIG.4, computing system24includes processing circuitry402, one or more input devices404, communication circuitry406, one or more output devices412, one or more storage devices408, and user interface (UI) device(s)410. Computing system24, in one example, further includes one or more application(s)424such as machine learning model(s)450, and operating system416that are executable by computing system24. Each of components402,404,406,408,410, and412are coupled (physically, communicatively, and/or operatively) for inter-component communications. In some examples, communication channels414may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data. As one example, components402,404,406,408,410, and412may be coupled by one or more communication channels414.

Processing circuitry402, in one example, is configured to implement functionality and/or process instructions for execution within computing system24. For example, processing circuitry402may be capable of processing instructions stored in storage device408. Examples of processing circuitry402may include, any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry.

One or more storage devices408may be configured to store information within computing system24during operation. Storage device408, in some examples, is described as a computer-readable storage medium. In some examples, storage device408is a temporary memory, meaning that a primary purpose of storage device408is not long-term storage. Storage device408, in some examples, is described as a volatile memory, meaning that storage device408does not maintain stored contents when the computer is turned off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art. In some examples, storage device408is used to store program instructions for execution by processing circuitry402. Storage device408, in one example, is used by software or applications424running on computing system24to temporarily store information during program execution.

Storage devices408, in some examples, also include one or more computer-readable storage media. Storage devices408may be configured to store larger amounts of information than volatile memory. Storage devices408may further be configured for long-term storage of information. In some examples, storage devices408include non-volatile storage elements. Examples of such non-volatile storage elements include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.

Computing system24, in some examples, also includes communication circuitry406. Computing system24, in one example, utilizes communication circuitry406to communicate with external devices, such as IMD17and EMR database66ofFIG.1. Communication circuitry406may include a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device that can send and receive information. Other examples of such network interfaces may include 3G, 4G, 5G, and WI-FI™ radios.

Computing system24, in one example, also includes one or more user interface devices410. User interface devices410, in some examples, are configured to receive input from a user through tactile, audio, or video feedback. Examples of user interface devices(s)410include a presence-sensitive display, a mouse, a keyboard, a voice responsive system, video camera, microphone or any other type of device for detecting a command from a user. In some examples, a presence-sensitive display includes a touch-sensitive screen.

One or more output devices412may also be included in computing system24. Output device412, in some examples, is configured to provide output to a user using tactile, audio, or video stimuli. Output device412, in one example, includes a presence-sensitive display, a sound card, a video graphics adapter card, or any other type of device for converting a signal into an appropriate form understandable to humans or machines. In some examples, output device(s)412include a display device. Additional examples of output device412include a speaker, a cathode ray tube (CRT) monitor, a liquid crystal display (LCD), or any other type of device that can generate intelligible output to a user.

Computing system24may include operating system416. Operating system416, in some examples, controls the operation of components of computing system24. For example, operating system416, in one example, facilitates the communication of one or more applications424and monitoring system450with processing circuitry402, communication circuitry406, storage device408, input device404, user interface devices410, and output device412.

Application422may also include program instructions and/or data that are executable by computing system24. Example application(s)422executable by computing system24may include monitoring system450. Other additional applications not shown may alternatively or additionally be included to provide other functionality described herein and are not depicted for the sake of simplicity.

In accordance with the techniques of the disclosure, applications424include a monitoring system450. Monitoring system450is configured to receive patient data, evaluate the patient data, and generate notifications when monitoring system450determines that it is likely that patient14(FIG.1) has experienced one or more occurrences of one or more cardiac arrhythmias.

As shown in the example ofFIG.4, monitoring system450may include a set of machine learning model(s)452. Together, machine learning model(s)452may be referred to as a library of machine learning models or a suite of machine learning models. Each of machine learning model(s)452may be configured to generate, on the basis of patient data provided to the machine learning model, probability values that indicate probabilities of patient14having experienced occurrences of one or more cardiac arrhythmias. In some examples, each of machine learning model(s)452is implemented using one or more neural network systems, deep learning systems, or other type of supervised or unsupervised machine learning systems. For example, a machine learning model may be implemented by a feedforward neural network, such as a convolutional neural network, a radial basis function neural network, a recurrent neural network, a modular or associative neural network. In some examples, monitoring system450trains machine learning model(s)452with patient data for a plurality of patients to generate the probability values for cardiac arrhythmias. In some examples, after a machine learning model has been pre-trained with the patient data (and, in some examples, EMR data) for the plurality of patients, monitoring system450may further train the machine learning model with patient data specific to patient14.

In some examples, monitoring system450trains one or more of machine learning model(s)452with the patient data for the plurality of patients, determines an error rate of the machine learning model, and then feeds the error rate back to the machine learning model so as to allow the machine learning model to update its predictions based on the error rate. Monitoring system450may use a backpropagation algorithm, such as a gradient descent algorithm, to feed to error rate back to the machine learning model. The error rate may correspond to differences between probability values determined by the machine learning model based on input data and prelabeled probability values for the same input data. In some examples, monitoring system450may use an error function to determine the error rate. The error function may be implemented using signal processing techniques and heuristics in the manner conventionally used to detect occurrences of cardiac arrhythmias. In some examples, the error function may return a vector of elements, each indicating whether the machine learning model correctly identified an occurrence of a respective cardiac arrhythmia.

In some examples, monitoring system450may receive, from patient14or a clinician, feedback indicating whether a predicted cardiac arrhythmia occurred in patient14within a particular time period. In some examples, monitoring system450may receive, from medical device16, a message indicating that medical device16has detected (or has not detected) an occurrence of a cardiac arrhythmia in patient14. In some examples, monitoring system450may obtain the feedback in other ways, such as by periodically checking the EMR data to determine if a cardiac arrhythmia occurred. Monitoring system450may update the machine learning model with the feedback. Thus, the training process may occur iteratively so as to incrementally improve the data generated by the machine learning model by “learning” from correct and incorrect data generated by the machine learning model in the past. Further, the training process may be used to further fine-tune a machine learning model that is trained using population-based data to provide more accurate predictions for a particular individual. In some examples, personnel of a monitoring service may provide the feedback.

In accordance with techniques of this disclosure, different ones of machine learning model(s)452may correspond to different monitoring reasons. In some examples, different ones of machine learning model(s)452correspond to different codes defined in the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10). Different ICD-10 codes may correspond to different reasons for prescribing medical device16to patient14. For example, there may be different ICD-10 codes for atrial fibrillation (AF) management, stroke, syncope, and other medical conditions. In some examples, monitoring system450may train one or more of machine learning model(s)452using training data generated for different ICD-10 codes.

Monitoring system450may be used for different use-cases for which the same model might not provide optimal cardiac arrhythmia detection performance. Thus, a suite of machine learning models may be generated and applied to identify occurrences of cardiac arrhythmias of interest. For instance, in one example, machine learning model(s)452may include a machine learning model to detect any occurrences of AF or sinus bradycardia or asystole (e.g., pause) cardiac arrhythmias. In this example, the user may not be concerned about other cardiac arrhythmias (e.g., normal sinus rhythm (NSR), premature atrial contraction (PAC), premature ventricular contraction (PVC), sinus tachycardia) with respect to patient14. In this example, this machine learning model may be used by a general practitioner to look for “important” arrhythmias. In another example, monitoring system450may use a machine learning model configured to determine probability values indicating probabilities that patient14has experienced any cardiac arrhythmias belonging to the sinus bradycardia or AV blocks types. In this example, the user may not be concerned above other arrhythmias with respect to patient14. In this example, this machine learning model can be used for post transcatheter aortic valve replacement (TAVR) monitoring.

The input data used by machine learning model(s)452may include patient data. The patient data may include data representing one or more electrical signals, such as EGM signals. In some examples, the patient data may include data regarding the patient's physiological status (e.g., patient physiological statuses such as activity, posture, respiration, etc.), which may also be captured by medical device16. Training data corresponding to different physiological conditions (e.g., rest, resting at night, resting at night with high posture angle, etc.) can be used as additional parameters for model training or input data for machine learning model(s)452. Using such data may enable monitoring system450to detect cardiac arrhythmias during other disease conditions (e.g., a sensitive model for tachycardia during rest can be used to monitor heart failure (HF) patients; a model for bradycardia during activity can be used to monitor patients for chronotropic incompetence).

Once one or more of machine learning model(s)452have been trained, monitoring system450may use machine learning model(s)452to detect occurrences of cardiac arrhythmias experienced by patient14. For example, monitoring system450(which may be executed by processing circuitry402) may receive patient data via communication circuitry406. The patient data may be collected in whole or part by medical device16of patient14. In some examples, the patient data includes one or more of EGM data, an activity level of the patient, a heart rate of the patient, a posture of the patient, a cardiac electrogram of the patient, a blood pressure of the patient, accelerometer data for the patient, EMR data from EMR database66, and/or other types of patient data. In some examples, medical device16is an IMD. In other examples, medical device that16is another type of patient device, such as a wearable medical device or a mobile device (e.g., a smartphone) of patient14. In some examples, monitoring system450receives the patient data from medical device16on a daily basis.

In some examples, monitoring system450receives, via communication circuitry406, EMR data for patient14from EMR database66. In some examples, the EMR data stored by EMR database66may include many different types of historical medical information about patient14. For example, EMR database66may store a medication history of the patient, a surgical procedure history of the patient, a hospitalization history of the patient, potassium levels of the patient over time, or one or more lab test results for the patient, etc. The EMR data may form part of the patient data used as input to one or more of machine learning model(s)452.

In some examples, each of machine learning model(s)452converts the patient data into one or more vectors and tensors (e.g., multi-dimensional arrays) that represent the patient data. The machine learning model may apply mathematical operations to the one or more vectors and tensors to generate a mathematical representation of the patient data. The machine learning model may determine different weights that correspond to identified relationships between the patient data and the occurrence of cardiac arrhythmias. The machine learning model may apply the different weights to the patient data to generate the probability values.

FIG.5is a flowchart illustrating an example operation in accordance with the techniques of the disclosure. For convenience,FIG.5is described with respect toFIG.1. The flowcharts of this disclosure are presented as examples. Other examples in accordance with techniques of this disclosure may include more, fewer, or different actions, or actions may be performed in different orders or in parallel.

In the example ofFIG.5, computing system24may generate a set of sample probability values by applying a machine learning model to a sample set of patient data (500). The machine learning model may be one of machine learning models452(FIG.4). As discussed above, the machine learning model may be trained using patient data for a set of sample patients. The set of sample patients may include a single sample patient or a plurality of sample patients. The set of sample patients may or may not include a current patient who is going to be monitored for one or more cardiac arrhythmias.

The sample set comprises a plurality of temporal windows. In some examples, the temporal windows overlap. In other examples, the temporal windows do not overlap. Each temporal window of the sample set may comprise one or more series of samples of at least one cardiac electrical waveform of a patient in the set of sample patients. In examples where the temporal windows overlap, the same sample may be in two or more temporal windows.

For each respective temporal window of the plurality of temporal windows, the machine learning model is configured to determine a respective probability value in the set of sample probability values that indicates a probability that a cardiac arrhythmia occurred during the respective temporal window. As discussed elsewhere in this disclosure, the machine learning model may comprise a neural network that has been trained to generate probability values indicating probabilities that a patient has experienced occurrences of one or more cardiac arrhythmias. Inputs to the neural network may include data corresponding to the respective temporal time window.

Furthermore, in the example ofFIG.5, computing system24may generate graphical data based on the sample probability values (502). As discussed elsewhere in this disclosure, computing system24may generate one or more of various types of graphical data based on the sample probability values. For instance, in some examples, computing system24generates a graphical representation of a ROC.FIG.6, which is described in detail below, is an example flowchart that includes actions for generating a ROC.FIG.9, which is also described in detail below, is an example flowchart that includes actions for generating a graph of probabilities of occurrences of cardiac arrhythmias plotted against time.

Computing system24may output a user interface for display on a display device (504). In the example ofFIG.5, the user interface comprises the graphical data. For instance, in an example where the graphical data comprises a ROC, the user interface may include a graph showing the ROC. In an example where the graphical data comprises a graph of probabilities of occurrences of cardiac arrhythmias plotted against time, the user interface may include the graph. The display device may be one of output device(s)412, one of user interface device(s)410, or another device for display data.

Computing system24may output the user interface in one or more of various forms. For example, computing system24may render and output a webpage that contains the graphical data. In another example, computing system24may output a graphical user interface for a local application.

Additionally, in the example ofFIG.5, computing system24may receive, via the user interface, an indication of user input to select a probability threshold for patient14(FIG.1) (506). For instance, in an example where the user interface includes a graph showing the ROC, computer system24may receive an indication of user input to select a point on the ROC. In an example where the user interface includes a graph of probabilities of occurrences of cardiac arrhythmias plotted against time, the user interface may further include a threshold indicator and computing system24may receive an indication of user input to position the threshold indicator at a location in the graph corresponding to a probability threshold for patient14. In some examples, computing system24does not require an indication of user input to select the probability threshold for patient14. Rather, in the absence of an indication of user input to select the probability threshold for patient14, computing system24may set the probability threshold to, or leave the probability threshold at, a default probability threshold. In some such examples, the default probability threshold may be a probability threshold that maximizes diagnostic yield irrespective of the anticipated review burden.

Computing system24may receive patient data for patient14(508). The patient data is collected by one or more medical devices, such as medical device16(FIG.1), and/or other types of devices. As discussed elsewhere in this disclosure, the patient data may include one or more series of samples of at least one cardiac electrical waveform of patient14.

Furthermore, in the example ofFIG.5, computing system24may apply the machine learning model to the patient data to determine a current probability value that indicates a probability that patient14has experienced an occurrence of a cardiac arrhythmia (510). For instance, in an example where the cardiac arrhythmia is atrial fibrillation, computing system24may apply the machine learning model to the patient data and determine that there is a 0.98 probability that patient14experienced an occurrence of atrial fibrillation.

Additionally, computing system24may determine whether the current probability value exceeds the probability threshold for patient14(512). In response to determining that the current probability value is greater than or equal to the probability threshold for patient14(“YES” branch of512), computing system24may generate a notification indicating that the patient has likely experienced the occurrence of the cardiac arrhythmia (514). Computing system24may generate the notification in any of one or more ways. For instance, in one example, computing system24may send a message (e.g., a text message, SMS message, instant message, email message, in-app message, voice message, video message, etc.) to a monitoring user. In this example, the message notifies the monitoring user that patient14likely experienced an occurrence of the cardiac arrhythmia. In some examples, computing system24does not generate a notification for each occurrence of a cardiac arrhythmia but may instead generate notifications for groups of occurrences of cardiac arrhythmias. In some examples, a user interface may present a list of the generated notifications.

Otherwise, in the example ofFIG.5, in response to determining that the current probability value is not greater than or equal to the probability threshold for patient14(“NO” branch of512), computing system24does not generate the notification and may continue to receive patient data for the patient (508). In other examples, computing system24may perform other actions in response to determining that the current probability value is not greater than or equal to the probability threshold for patient14. In some examples, computing system24may perform the operation ofFIG.5for each respective cardiac arrhythmia in a plurality of cardiac arrhythmias.

As noted above, in some examples, computing system24does not receive an indication of user input to select the probability threshold for a patient, such as patient14. Thus, in such examples, computing system24may receive patient data for a patient, wherein the patient data is collected by one or more medical devices. In this example, computing system24may apply the machine learning model to the patient data to determine a current probability value that indicates a probability that the patient has experienced an occurrence of the cardiac arrhythmia. Furthermore, in this example, computing system24may determine that the current probability value exceeds a default probability threshold, wherein the default probability threshold is set to maximize diagnostic yield. In response to determining that the current probability value is greater than or equal to the default probability threshold, computing system24may generate a notification indicating that the patient has likely experienced the occurrence of the cardiac arrhythmia.

FIG.6is a flowchart illustrating a first example operation for generating graphical data and receiving an indication of user input to select a probability threshold, in accordance with techniques of this disclosure. The example ofFIG.6provides example details regarding how computing system24may generate graphical data in action (502) ofFIG.5and how computing system24may receive an indication of user input to select a probability threshold for patient14in action (506) ofFIG.5.

In the example ofFIG.6, as part of generating graphical data based on the sample probability values, computing system24may generate a ROC (600). The ROC is a curve that plots sensitivity values against specificity values. In general, when sensitivity values are high, the probability of computing system24not generating a notification indicating that patient14experienced an occurrence of the cardiac arrhythmia when patient14actually experienced an occurrence of the cardiac arrhythmia is low. Thus, when sensitivity values are high, there may be more false positives, but computing system24is unlikely to miss occurrences of the cardiac arrhythmia. When specificity values are high, the probability of computing system24generating a notification indicating that patient14experienced an occurrence of the cardiac arrhythmia when patient14did not actually experience an occurrence of the cardiac arrhythmia is low. Thus, when specificity values are high, there are likely to be few false positives, but computing system24is likely to miss more occurrences of the cardiac arrhythmia. Accordingly, the sensitivity values decrease as specificity values increase, and vice versa.

As part of generating the ROC, computing system24may perform actions (602) through (608) for each respective probability threshold in a set of evaluation probability thresholds. The set of evaluation probability thresholds may include two or more evaluation probability thresholds. In general, the greater number of evaluation probability thresholds used, the more data is available to computing system24to generate the ROC.

In the example ofFIG.6, computing system24may determine a sensitivity value for a respective evaluation probability threshold (602). As discussed above with respect toFIG.5, computing system24may generate a set of sample probability values by applying a machine learning mode to a sample set of patient data. Each of the sample probability values indicates a probability that an occurrence of the cardiac arrhythmia happened during the respective temporal window. Computing system24may determine the sensitivity value for the respective evaluation probability threshold as a ratio of: (i) a total number of sample probability values in the set of sample probability values that are greater than or equal to the respective evaluation probability threshold to (ii) a total number of temporal windows in the sample set that actually contain occurrences of the cardiac arrhythmia.

Furthermore, in the example ofFIG.6, computing system24may determine a specificity value for the respective evaluation probability threshold (604). Computing system24may determine the specificity value for the respective evaluation probability threshold as a ratio of: (i) the total number of the sample probability values that are not greater than or equal to the respective evaluation probability threshold to (ii) the total number of the temporal windows in the sample set that do not actually contain occurrences of the cardiac arrhythmia.

Computing system24may then determine a point on the ROC that corresponds to the respective probability value (606). The point on the ROC that corresponds to the respective probability value is based on the sensitivity value for the respective probability threshold and the specificity value for the respective probability threshold. For instance, the point may be defined by a pair of coordinates, one of which is the sensitivity value for the respective probability threshold and one of which is the specificity value for the respective evaluation probability threshold. In some examples, computing system24may apply one or more functions to transform the sensitivity value and specificity value for the respective evaluation probability threshold to determine the coordinates of the point.

Computing system24may determine whether there are any remaining evaluation probability thresholds to evaluate (608). If there are any remaining evaluation probability thresholds to evaluate (“YES” branch of608), computing system24may repeat actions (602) through (606) with respect to another one of the evaluation probability thresholds. Otherwise (“NO” branch of608), computing system24may continue the operation ofFIG.5. In some examples, after determining that there are no remaining evaluation probability thresholds to evaluate, computing system24may generate (e.g., using interpolation, extrapolation, and/or regression) a smooth or discrete-valued curve based on the determined points.

Subsequently, computing system24may receive an indication of user input to select a probability threshold for patient14(506). In the example ofFIG.6, as part of receiving the indication of user input to select the probability threshold for patient14, computing system24may receive an indication of user input to select a point on the ROC that corresponds to the probability threshold for patient14(610). Computing system24may receive the indication of user input to select the point on the ROC in one of various ways. For instance, in one example, the user interface may include an indicator element that is slidable along the ROC. In this example, computing system24may receive tapping, sliding, or dragging input to reposition the indicator element to a position along the ROC. In some examples, computing system24may receive an indication of user input specifying the probability threshold for patient14, in which case, computing system24may update the position of the indicator element such that the indicator element is located at a position on the ROC corresponding to the specified probability threshold for patient14. In some examples, computing system24may receive an indication of user input specifying a sensitivity value or a specificity value, in which case, computing system24may update the position of the indicator element such that the indicator element is located at a position on the ROC corresponding to the specified sensitivity value or the specified specificity value.

FIG.7is a flowchart illustrating an example process for model and operating point selection based on a reason for monitoring, in accordance with techniques of this disclosure. A user may determine an ICD-10 code corresponding to a reason for monitoring patient14(700). Furthermore, computing system24may be configured to use a set of machine learning models. The set of machine learning models may be referred to as a library of machine learning models. Each machine learning model may include one or more neural networks configured to determine probability values for one or more cardiac arrhythmias. Each of the machine learning models may be associated with a different ICD-10 code, or otherwise associated with a different reason for monitoring the patient. Computing system24and/or the user may select a machine learning model in the library based on the ICD-10 code associated with the machine learning model (702).

Furthermore, in the example ofFIG.7, computing system24may present a ROC704to the user to enable the user to select one or more operating points (706). An operating point may correspond to a probability threshold for patient14. Computing system24may generate the ROC based on thresholding the arrhythmia likelihood (e.g., as described in the example ofFIG.6). For example, if the reason for monitoring is to detect atrial fibrillation (AF) in stroke patients, computing system24may present a ROC for AF detection from which the user (e.g., a prescribing physician) can choose a threshold that corresponds to a very high sensitivity and low-specificity for AF detection. In another example, if the reason for monitoring is AF management, the user (e.g., a prescribing physician) can choose, from the ROC, an operating point that has a high specificity and low sensitivity for AF. In another example, if the reason for monitoring is syncope, the physician can choose a high sensitivity for asystole and sinus brady and a balanced sensitivity and specificity for AF and other arrhythmias from the ROC.

In the example ofFIG.7, the user (e.g., the prescribing physician) may iterate on a review burden versus diagnostic yield for the chosen machine learning models and operating points. As discussed elsewhere in this disclosure, review burden may refer to the burden of reviewing notifications and diagnostic yield may refer to the amount of diagnostically valuable information derived from such notifications. For instance, if the user is not using a monitoring service (“NO” branch of708), the user may evaluate the review burden versus the diagnostic yield one or more times before settling on an acceptable balance between review burden and diagnostic yield.

In some examples, computing system24may present (e.g., output for display on a display device) data indicating the anticipated review burden versus the anticipated diagnostic yield for one or more operating points. An operating point for a cardiac arrhythmia may correspond to a probability threshold. In some examples, computing system24may store historical data (e.g., from a monitoring center or database) for a patient population. For each patient in the patient population, the historical data may indicate an operating point for the patient, a review burden for the patient, and a diagnostic yield for the patient. In this example, when an operating point is being set for a particular patient, computing system24may identify similar patients in the patient population with whom the operating point was used and determine the anticipated review burden and anticipated diagnostic yield for the particular patient based on review burdens and diagnostic yields of the identified patients. For instance, computing system24may calculate averages of the review burdens and diagnostic yields of the identified patients.

In some examples, computing system24may estimate the anticipated review burden and anticipated diagnostic yield for an operating point from prevalence and algorithm performance numbers. For instance, in one example, suppose the patient population being monitored has an AF prevalence of 20% (i.e., 20% of AF-triggered ECGs from patients in this population actually have AF). This prevalence can be obtained either from the literature or from historical values at the monitoring center. Furthermore, in this example, suppose that a user would like to get a diagnostic yield and review burden for 2 AF detection operating points (i) 95% sensitivity and 70% specificity and (ii) 70% sensitivity and 95% specificity, and the corresponding algorithm-detected AF episodes are reported. An AF episode may correspond to a temporal window in which a device (e.g., medical device16) reported that an AF might be present. In this example, the “baseline” review burden is where all episodes are reviewed. In this example, with the first operating point, the review burden is 43% of the baseline. That is, assume 1000 episodes are presented to the algorithm. Accordingly, the number of true positives (TP) among the 1000 episodes is expected to be TP=0.95*200=190; the number of false positives (FP) among the 1000 episodes is expected to be FP=(1−0.7)*800=240; the total number of detections=190+240=430; hence 430/1000=43%. 95% (190 out of 200) of the actual episodes are captured and about 44% (190 out of 430) of the reviewed episodes are true AF. With the second operating point, the review burden is 18% of the baseline. That is, assume 1000 episodes being presented to the algorithm; TP=0.7*200=140; FP=(1−0.95)*800=40; total detections=140+40=180; hence, 180/1000=18%. Here, only 70% (140 out of 200) of the actual episodes are captured and about 78% (140 out of 180) of the reviewed episodes are true AF. Thus, in this example, the review burden and diagnostic yield for the first operating point is higher than that of the second one.

In some examples, the user may adjust the review burden versus the diagnostic yield of the chosen cardiac arrhythmias by changing the probability thresholds for one or more of the chosen cardiac arrhythmias (e.g., in the manner described elsewhere in this disclosure). This step can help users (e.g., physicians) tune their operating models to best choose one that provides optimal balance between review burden and diagnostic yield (e.g., the review burden might be exceedingly high if the sensitivity (e.g., probability threshold) is set at 99% but might be manageable if the sensitivity is set at 98%). Computing system24may then use the chosen model and operating points are used for arrhythmia detection and notification.

If the user is using a monitoring service (“YES” branch of708) or after evaluating the reviewing burden versus diagnostic yield, computing system24may start a monitoring process with the chosen machine learning model(s) and operating point. The monitoring process may receive patient information, apply the selected machine learning model, and generate notifications (e.g., as described with respect to actions (508) through (514) ofFIG.5. The monitoring service may be a service that reviews notifications on behalf of the user (e.g., physician).

FIG.8is a flowchart illustrating an example process for model and operating point selection based on cardiac arrhythmias for monitoring, in accordance with techniques of this disclosure. In the example ofFIG.8, a user (e.g., a prescribing physician) may determine a set of cardiac arrhythmias to monitor (800). The set of cardiac arrhythmias to monitor may be those cardiac arrhythmias for which the user wants to receive notifications. In this way, the user may choose one or more cardiac arrhythmias from a set of cardiac arrhythmias. The chosen cardiac arrhythmias may be cardiac arrhythmias that the user is interested in monitoring.

Furthermore, as shown in the example ofFIG.8, machine learning models802A-802N (collectively, “machine learning models802”) may be developed for different groups of cardiac arrhythmias. For instance, machine learning model802A may generate probability values indicating probabilities that patient14has experienced occurrences of atrial fibrillation (AF), pause, and sinus bradycardia (sinus brady); machine learning model802B may generate probability values indicating probabilities that patient14has experienced occurrences of sinus bradycardia, pause, and atrioventricular (AV) block; machine learning model802C may generate probability values indicating probabilities that patient14has experienced occurrences of atrial fibrillation, atrial flutter, and supraventricular tachycardia (SVT); machine learning model802N may generate probability values indicating probabilities that patient14has experienced occurrences of sinus tachycardia (sinus tachy), sinus bradycardia (sinus brady), atrial fibrillation (AF), atrial flutter, supraventricular tachycardia (SVT), atrioventricular (AV) blocks, and intraventricular conduction delay (IVCD). In the example ofFIG.8, each of machine learning models802corresponds to a different class of cardiac arrhythmias. In other examples, there may be different machine learning models for different cardiac arrhythmias instead of different classes of cardiac arrhythmias. In some examples, each of machine learning models802may be a different one of machine learning models452(FIG.4). Each of machine learning models802may be implemented using one or more neural networks. The developed machine learning models802may together form a “library” of machine learning models.

The user may select one or more of machine learning models from the library of machine learning models802(804). For instance, if the user chooses to monitor sinus bradycardia, AV block, and atrial flutter, the user may choose machine learning models802B and802C. Note that in some instances, the selected machine learning models may be configured to generate probability value for one or more cardiac arrhythmias that the user does not necessarily want to monitor. In some examples, computing system24may receive an indication of user input to select one or more of machine learning models802from the library of machine learning models802.

Furthermore, in the example ofFIG.8, computing system24may present one or more ROCs808to the user to enable the user to select one or more operating points for patient14(806). Each of the operating points for patient14may correspond to the probability threshold for use in identifying occurrences of different cardiac arrhythmias. Thus, based on the chosen cardiac arrhythmias for monitoring, the corresponding machine learning models may be chosen and the corresponding ROCs may be presented to enable the user to select an operating point. The user may select a different operating point on each of the ROCs. In the example ofFIG.8, the user may select an operating point on a ROC by providing user input to move an indicator element809on the ROC, thereby providing computing system24with an indication of user input to select a point on the ROC that corresponds to the probability threshold of patient14for the cardiac arrhythmia.

In the example ofFIG.8, the user (e.g., the prescribing physician) may iterate on a review burden versus diagnostic yield for the chosen machine learning models and operating point. As discussed elsewhere in this disclosure, review burden may refer to the burden of reviewing notifications and diagnostic yield may refer to the amount of diagnostically valuable information derived from such notifications. For instance, if the user is not using a monitoring service (“NO” branch of810), the user may evaluate the review burden versus the diagnostic yield one or more times before settling on an acceptable balance between review burden and diagnostic yield. In some examples, the user may adjust the review burden versus the diagnostic yield by changing the probability thresholds for one or more of the chosen cardiac arrhythmias (e.g., in the manner described elsewhere in this disclosure). If the user is using a monitoring service (“YES” branch of810) or after evaluating the reviewing burden versus diagnostic yield, computing system24may start a monitoring process with the chosen machine learning model(s) and operating point. The monitoring process may receive patient information, apply the selected machine learning model, and generate notifications (e.g., as described with respect to actions (508) through (514) ofFIG.5. In some examples, computing system24may receive subsequent indications of user input to update the probability threshold for patient14.

FIG.9is a flowchart illustrating a second example operation for generating graphical data and receiving an indication of user input to select a probability threshold, in accordance with techniques of this disclosure. The example ofFIG.9provides example details regarding how computing system24may generate graphical data in action (502) ofFIG.5and how computing system24may receive an indication of user input to select a probability threshold for patient14in action (506) ofFIG.5.

As shown in the example ofFIG.9, as part of generating graphical data based on the sample probability values, computing system24may generate a graph that plots sample probability values against time (908). As discussed above with respect toFIG.5, computing system24may generate a set of sample probability values by applying a machine learning model to a sample set of patient data. Each of the sample probability values indicates a probability that a cardiac arrhythmia belonging to a cardiac arrhythmia occurred during the respective temporal window. In some examples, computing system24may generate a smooth or discrete curve based on the sample probability values.

Furthermore, in the example ofFIG.9, computing system24may generate a threshold indicator (902). Additionally, in the example ofFIG.9, as part of receiving an indicator of user input to select the probability threshold for patient14(506), computing system24may receive an indication of user input to position the threshold indicator at a location in the graph corresponding to the probability threshold for patient14(904).

Computing system24may generate the threshold indicator in any of one or more ways. For instance, in one example, the threshold indicator comprises a threshold bar superimposed on the graph and oriented parallel to a time axis of the graph. In this example, the threshold bar may be superimposed on the graph such that the threshold bar appears over or underneath a curve or data points based on the set of sample probability values. Furthermore, in some examples where the threshold indicator comprises a threshold bar, computing system24may receive indications of user input to slide the threshold bar in a direction perpendicular to the time axis in order to position the threshold indicator at a location in the graph corresponding to the desired probability threshold for patient14. In some examples, computing system24may receive an indication of user input to specify the desired probability threshold for patient14(e.g., in the form of text) and computing system24may update the location of the threshold indicator to a location in the graph corresponding to the desired probability threshold for patient14. In some examples, the threshold indicator may comprise an arrow, pointer, or other type of graphical element in or adjacent to the graph.

FIG.10is a conceptual diagram that includes an example graph1000of a raw cardiac electrical waveform during a time period and an example graph1002of probabilities of cardiac arrhythmia events during the same time period, in accordance with techniques of this disclosure. In the example ofFIG.10, graph1000corresponds to a cardiac electrogram (EGM) that indicates an overall magnitude of the heart's electrical potential as recorded over the time period. In the example ofFIG.10, the time period is approximately 45 seconds in duration.

Graph1002includes waveforms corresponding to different cardiac arrhythmias in a set of cardiac arrhythmias. For instance, graph1002may include waveforms corresponding to different ICD-10 types). In some examples, the set of cardiac arrhythmias may be selected based on physician interest or patient condition.

In the example ofFIG.10, the set of cardiac arrhythmias includes 1stdegree atrioventricular block (AVB), atrial fibrillation, premature ventricular contractions (PVCs), sinus rhythm, supraventricular tachycardia, noise/non-physiological signal segments (labeled, “artifact” inFIG.10), atrial flutter, sinus bradycardia, and sinus tachycardia. For each of the cardiac arrhythmias, the waveform corresponding to the cardiac arrhythmia is based on sample probability values that indicate probabilities that an occurrences of the cardiac arrhythmia occurred during temporal windows ending at time values corresponding to the sample probability values. The waveforms shown in the example ofFIG.10are determined based on the waveform of graph1000. In the example ofFIG.10, probability values are mapped (e.g., linearly scaled) to index values for ease of interpretation. In the example ofFIG.10, the index values are labeled as “waveform heatmap” values.

In the example ofFIG.10, a threshold bar1004is superimposed on graph1002. Threshold bar1004is positioned at a location on graph1002that corresponds to a desired probability threshold for patient14. In the example ofFIG.10, threshold bar1004is positioned at a location on graph1002that corresponds to an index value of 2. If the probability value of one of the cardiac arrhythmias rises above the probability threshold corresponding to the position indicated by threshold bar1004, computing system24may generate a notification indicating that patient14is likely to have experienced an occurrence of the cardiac arrhythmia. For instance, in the example ofFIG.10, given the position of threshold bar1004, computing system24may determine that the patient is likely to have experienced six occurrences of PVC, two or more occurrences of a sinus rhythm arrhythmia, and an occurrence of atrial fibrillation.

As shown in the example ofFIG.10, computing system24may receive an indication of user input to update the position of threshold bar1004to a higher position in graph1002. Thus, given the updated position of threshold bar1004, computing system24may still determine that patient14is likely to have experienced six occurrences of PVC, but does not determine that patient14has experienced an occurrence of the sinus rhythm arrhythmia or the occurrence of atrial fibrillation.

In some examples, there may be different threshold bars for different cardiac arrhythmias. For instance, there may be a first threshold bar for 1stdegree AVC, a second threshold bar for atrial fibrillation, a third threshold bar for PVCs, and so on. Thus, a user may be able to set different probability thresholds for patient14for different cardiac arrhythmias. For instance, the user may require a high probability threshold for atrial flutter if patient14is known to frequently experience atrial flutter without serious effects but may require a lower probability threshold for sinus bradycardia.

In some examples, the techniques of the disclosure include a system that comprises means to perform any method described herein. In some examples, the techniques of the disclosure include a computer-readable medium comprising instructions that cause processing circuitry to perform any method described herein.

The following is a non-limiting list of examples that are in accordance within one or more techniques of this disclosure.

Example 1. A method comprising: generating, by a computing system that comprises processing circuitry and a storage medium, a set of sample probability values by applying a machine learning model to a sample set of patient data, wherein: the machine learning model is trained using patient data for a plurality of patients, the sample set comprises a plurality of temporal windows, and for each respective temporal window of the plurality of temporal windows, the machine learning model is configured to determine a respective probability value in the set of sample probability values that indicates a probability that a cardiac arrhythmia occurred during the respective temporal window; generating, by the computing system, graphical data based on the sample probability values; outputting, by the computing system, a user interface for display on a display device, the user interface comprising the graphical data; receiving, by the computing system, via the user interface, an indication of user input to select a probability threshold for a patient; receiving, by the computing system, patient data for the patient, wherein the patient data is collected by one or more medical devices; applying, by the computing system, the machine learning model to the patient data to determine a current probability value that indicates a probability that the patient has experienced an occurrence of a cardiac arrhythmia; determining, by the computing system, that the current probability value exceeds the probability threshold for the patient; and in response to determining that the current probability value is greater than or equal to the probability threshold for the patient, generating, by the computing system, a notification indicating that the patient has likely experienced the occurrence of the cardiac arrhythmia.

Example 2. The method of example 1, wherein: generating the graphical data comprises generating, by the computing system, a receiver operating curve (ROC), wherein generating the ROC comprises: for each evaluation probability threshold of a plurality of evaluation probability thresholds: determining, by the computing system, a sensitivity value for the respective evaluation probability threshold as a ratio of: (i) a total number of sample probability values in the set of sample probability values that are greater than or equal to the respective evaluation probability threshold to (ii) a total number of the temporal windows in the sample set that actually contain occurrences of the cardiac arrhythmia that actually occurred in the sample set; determining, by the computing system, a specificity value for the respective probability value as a ratio of: (i) a total number of the sample probability values that are not greater than or equal to the respective evaluation probability threshold to (ii) a total number of the temporal windows in the sample set that do not actually contain occurrences of the cardiac arrhythmia; and determining, by the computing system, a point on the ROC that corresponds to the respective probability value, wherein the point on the ROC that corresponds to the respective probability value is based on the sensitivity value for the respective evaluation probability threshold and the specificity value for the respective evaluation probability threshold; and receiving the indication of user input to select the probability threshold for the patient comprises: receiving, by the computing system, an indication of user input to select a point on the ROC that corresponds to the probability threshold for the patient.

Example 3. The method of any of examples 1 or 2, wherein: generating the graphical data comprises: generating, by the computing system, a graph that maps the sample probability values against time; and generating, by the computing system, a threshold indicator; and receiving the indication of user input comprises receiving, by the computing system, an indication of user input to position the threshold indicator at a location in the graph corresponding to the probability threshold for the patient.

Example 4. The method of example 3, wherein the threshold indicator comprises a threshold bar superimposed on the graph and oriented parallel to a time axis of the graph.

Example 5. The method of any of examples 1-4, wherein receiving the patient data for the patient comprises receiving, by the computing system, cardiac electrical waveform data for the patient.

Example 6. The method of any of examples 1-5, further comprising receiving, by the computing system, an indication of user input to update the probability threshold for the patient.

Example 7. The method of any of examples 1-6, wherein the medical device comprises a wearable medical device or an implantable medical device (IMD).

Example 8. The method of any of examples 1-7, wherein: the cardiac arrhythmia is a first cardiac arrhythmia in a plurality of cardiac arrhythmias, and the method comprises, for each respective cardiac arrhythmia of the plurality of cardiac arrhythmias: generating, by the computing system, a respective set of sample probability values by applying a respective machine learning model to a respective sample set of patient data, wherein: the respective machine learning model is trained using patient data for the plurality of patients, the respective sample set comprises a respective plurality of temporal windows, and for each respective temporal window of the respective plurality of temporal windows, the respective machine learning model is configured to determine a respective probability value in the respective set of sample probability values that indicates a probability that the respective cardiac arrhythmia occurred during the respective temporal window; generating, by the computing system, respective graphical data based on the respective set of sample probability values; outputting, by the computing system, the user interface for display on the display device such that the user interface comprises the respective graphical data; receiving, by the computing system, via the user interface, an indication of user input to select a respective probability threshold for the patient; applying, by the computing system, the machine learning model to the patient data to determine a respective probability value that indicates a probability that the patient has experienced an occurrence of the respective cardiac arrhythmia; determining, by the computing system, that the respective probability value exceeds the respective probability threshold; and in response to determining that the respective probability value is greater than or equal to the respective probability threshold, generating, by the computing system, a notification indicating that the patient has likely experienced the occurrence of the respective cardiac arrhythmia.

Example 9. The method of any of examples 1-8, further comprising presenting, by the computing system, data indicating the anticipated review burden versus the anticipated diagnostic yield for the probability threshold for the patient.

Example 10. A computing system comprising processing circuitry and a storage medium, the computing device configured to perform the methods of any of examples 1-9.

Example 11. A method as described in the specification.

It should be understood that various aspects and examples disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the techniques). In addition, while certain aspects of this disclosure are described as being performed by a single module, unit, or circuit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units, modules, or circuitry associated with, for example, a medical device.

In one or more examples, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” or “processing circuitry” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

Various examples have been described. These and other examples are within the scope of the following claims.