KEY FRAME IDENTIFICATION FOR INTRAVASCULAR ULTRASOUND BASED ON PLAQUE BURDEN

The present disclosure provides to process intravascular ultrasound (IVUS) images to identify key frames such as the proximal key frame, a distal key frame, and a minimal key frame from the IVUS images based on the raw lumen area, vessel area, and plaque burden. Ones of the key frames can be re-identified based on manipulation of other ones of the key frames.

TECHNICAL FIELD

The present disclosure generally relates to intravascular ultrasound (IVUS) imaging system. Particularly, but not exclusively, the present disclosure relates to identifying key frame markers using the plaque burden represented in IVUS images

BACKGROUND

Ultrasound devices insertable into patients have proven diagnostic capabilities for a variety of diseases and disorders. For example, intravascular ultrasound (IVUS) imaging systems have been used as an imaging modality for diagnosing blocked blood vessels and providing information to aid medical practitioners in selecting and placing stents and other devices to restore or increase blood flow.

IVUS imaging systems includes a control module (with a pulse generator, an image acquisition and processing components, and a monitor), a catheter, and a transducer disposed in the catheter. The transducer-containing catheter is positioned in a lumen or cavity within, or in proximity to, a region to be imaged, such as a blood vessel wall or patient tissue in proximity to a blood vessel wall. The pulse generator in the control module generates electrical pulses that are delivered to the transducer and transformed to acoustic pulses that are transmitted through patient tissue. The patient tissue (or other structure) reflects the acoustic pulses and reflected pulses are absorbed by the transducer and transformed to electric pulses. The transformed electric pulses are delivered to the image acquisition and processing components and converted into images displayable on the monitor.

However, it can be difficult for physicians to visualize the complete structure of the patient lumen (e.g., vessel) from the raw IVUS images. For example, it is difficult to determine an overall plaque burden, the appropriate size of stent (e.g., diameter and/or length) to use in correcting any strictures in the lumen, as well as the location to land the stent. Thus, there here is a need for systems and methods to process, annotate, visualize, and/or display images of a lumen of a patient based on IVUS images.

BRIEF SUMMARY

In general, the present disclosure provides to process raw IVUS images, automatically detected lumen and vessel borders, and identify regions of interest, or more particularly, starting and ending points between which include frames of interest in a series of IVUS images.

In some implementations, the present disclosure be embodied as a method, for example, a method for an intravascular ultrasound (IVUS) imaging system, comprising receiving a series of intravascular ultrasound (IVUS) images of a vessel of a patient, the series of IVUS images comprising a proximal IVUS frame, a distal IVUS frame, and a plurality of interior IVUS frames; determining a raw lumen area represented in each of the plurality of interior IVUS frames; generating, for each of the plurality of interior IVUS frames, a smooth lumen area based on a sampling filter and the lumen area; identifying as a smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the least smooth lumen area of the plurality interior IVUS frames; selecting a subset of the plurality of interior IVUS frames, the subset of interior IVUS frames comprising the smooth lumen frame and at least one other of the plurality of interior IVUS frames, wherein the smooth lumen frame and the at least one other of the plurality of interior IVUS frames are co-linear; and identifying as a minimum key frame, the one of the subset of the plurality of interior IVUS frames having a raw lumen area less than or equal to the lumen area of the subset of the plurality interior IVUS frames.

Alternatively, or additionally in any of the embodiments of a method above, the subset of the plurality of interior IVUS frames comprises 21 frames.

Alternatively, or additionally in any of the embodiments of a method above, the subset of the plurality of interior IVUS frames comprises ones of the series of IVUS images representing a distance along the vessel.

Alternatively, or additionally in any of the embodiments of a method above, the distance is less than or equal to 2 millimeters.

Alternatively, or additionally in any of the embodiments of a method above, the interior IVUS frames are disposed between the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of a method above, the series of IVUS images comprise a plurality of IVUS image frames and the method can comprise designating the first frame in the series of IVUS images as the distal key frame; and designating the last frame in the series of IVUS images as the proximal key frame.

Alternatively, or additionally any of the embodiments of a method above can comprise receiving, from an input device, an indication of the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of a method above, the series of IVUS images comprise a plurality of IVUS image frames and wherein identifying as the smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the smooth lumen area of the plurality interior IVUS frames can comprise designating each one of the plurality IVUS images disposed between the proximal IVUS frame and the distal IVUS frame as interior IVUS frames; identifying the least smooth lumen area of the plurality of interior IVUS frames; identifying the one or more of the plurality of interior IVUS frames having a smooth lumen area equal to the least smooth lumen area; and designating a one of the identified one or more of the plurality of interior IVUS frames disposed central to the proximal IVUS frame and the distal IVUS frame as the minimum smooth lumen frame.

Alternatively, or additionally in any of the embodiments of a method above, the sampling filter comprises a first sampling filter and the method can comprise generating, for each of the plurality of IVUS images, a smoother lumen area based on a second sampling filter and the smooth lumen area, wherein the second sampling filter is different than the first sampling filter.

Alternatively, or additionally in any of the embodiments of a method above, the series of IVUS images comprise a plurality of IVUS image frames and the method can comprise identifying an initial proximal search frame and an initial distal search frame; identifying an IVUS image frame from the one of the plurality of IVUS image frames located between the initial proximal search frame and the initial distal search frame having the smallest smoother lumen area; identifying, as a central search frame, the one of the plurality of IVUS image frames located within a first distance of the IVUS image frame having the smallest raw lumen area; identifying, as the distal key frame, the one of the plurality of IVUS image frames located within a second distance distal from the central search frame having a smooth plaque burden and a raw plaque burden less than a threshold value; and identifying, as the proximal key frame, the one of the plurality of IVUS image frames located within the second distance proximal from the central search frame having a smooth plaque burden and a raw plaque burden less than the threshold value.

Alternatively, or additionally in any of the embodiments of a method above, the threshold value is between 40 and 60 percent.

Alternatively, or additionally in any of the embodiments of a method above, the second distance is less than or equal to 5 millimeters.

Alternatively, or additionally in any of the embodiments of a method above, the first distance is less than or equal to 1 millimeter.

In some implementations, the present disclosure be embodied as an apparatus, comprising a processor of an intravascular ultrasound (IVUS) imaging system coupled to a memory, the memory comprising instructions executable by the processor, the processor configured to execute the instructions, which instructions when executed cause the processor to implement the method of any combination of the examples above.

In some implementations, the present disclosure can be embodied as at least one machine readable storage device, comprising a plurality of instructions that in response to being executed by a processor of an intravascular ultrasound (IVUS) imaging system cause the processor to implement the method of any combination of the examples above.

In some implementations, the present disclosure be embodied as an apparatus for an intravascular ultrasound (IVUS) imaging system, comprising circuitry to couple to an intravascular ultrasound (IVUS) system; a memory device storing instructions; and a processor coupled to the circuitry and the memory device, the processor configured to execute the instructions, which when executed cause the computing apparatus to: receive a series of intravascular ultrasound (IVUS) images of a vessel of a patient, the series of IVUS images comprising a proximal IVUS frame, a distal IVUS frame, and a plurality of interior IVUS frames; determine a raw lumen area represented in each of the plurality of interior IVUS frames; generate, for each of the plurality of interior IVUS frames, a smooth lumen area based on a sampling filter and the lumen area; identify as a smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the least smooth lumen area of the plurality interior IVUS frames; select a subset of the plurality of interior IVUS frames, the subset of interior IVUS frames comprising the smooth lumen frame and at least one other of the plurality of interior IVUS frames, wherein the smooth lumen frame and the at least one other of the plurality of interior IVUS frames are co-linear; and identify as a minimum key frame, the one of the subset of the plurality of interior IVUS frames having a raw lumen area less than or equal to the lumen area of the subset of the plurality interior IVUS frames.

Alternatively, or additionally in any of the embodiments of an apparatus above, the subset of the plurality of interior IVUS frames comprises 21 frames.

Alternatively, or additionally in any of the embodiments of an apparatus above, the subset of the plurality of interior IVUS frames comprises ones of the series of IVUS images represent a distance along the vessel.

Alternatively, or additionally in any of the embodiments of an apparatus above, the distance is less than or equal to 2 millimeters.

Alternatively, or additionally in any of the embodiments of an apparatus above, the interior IVUS frames are disposed between the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of an apparatus above, the series of IVUS images comprise a plurality of IVUS image frames and the memory device can further comprise instructions that when executed by the processor cause the IVUS imaging system to designate the first frame in the series of IVUS images as the distal key frame; and designate the last frame in the series of IVUS images as the proximal key frame.

Alternatively, or additionally in any of the embodiments of an apparatus above, the memory device can further comprise instructions that when executed by the processor cause the IVUS imaging system to receive, from an input device, an indication of the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of an apparatus above, the series of IVUS images comprise a plurality of IVUS image frames and wherein identifying as the smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the smooth lumen area of the plurality interior IVUS frames comprises: designate each one of the plurality IVUS images disposed between the proximal IVUS frame and the distal IVUS frame as interior IVUS frames; identify the least smooth lumen area of the plurality of interior IVUS frames; identify the one or more of the plurality of interior IVUS frames having a smooth lumen area equal to the least smooth lumen area; and designate a one of the identified one or more of the plurality of interior IVUS frames disposed central to the proximal IVUS frame and the distal IVUS frame as the minimum smooth lumen frame.

Alternatively, or additionally in any of the embodiments of an apparatus above, the sampling filter comprises a first sampling filter, the instructions when executed cause the computing apparatus to generate, for each of the plurality of IVUS images, a smoother lumen area based on a second sampling filter and the smooth lumen area, wherein the second sampling filter is different than the first sampling filter.

Alternatively, or additionally in any of the embodiments of an apparatus above, the series of IVUS images comprise a plurality of IVUS image frames and the memory device can further comprise instructions that when executed by the processor cause the IVUS imaging system to identify an initial proximal search frame and an initial distal search frame; identify an IVUS image frame from the one of the plurality of IVUS image frames located between the initial proximal search frame and the initial distal search frame having the smallest smoother lumen area; identify, as a central search frame, the one of the plurality of IVUS image frames located within a first distance of the IVUS image frame having the smallest raw lumen area; identify, as the distal key frame, the one of the plurality of IVUS image frames located within a second distance distal from the central search frame having a smooth plaque burden and a raw plaque burden less than a threshold value; and identify, as the proximal key frame, the one of the plurality of IVUS image frames located within the second distance proximal from the central search frame having a smooth plaque burden and a raw plaque burden less than the threshold value.

Alternatively, or additionally in any of the embodiments of an apparatus above, the threshold value is between 40 and 60 percent.

Alternatively, or additionally in any of the embodiments of an apparatus above, the first distance is less than or equal to 1 millimeter (mm) and the second distance is less than or equal to 5 mm.

In some implementations, the present disclosure be embodied as at least one machine readable storage device, comprising a plurality of instructions that in response to being executed by a processor of an intravascular ultrasound (IVUS) imaging system cause the processor to receive a series of intravascular ultrasound (IVUS) images of a vessel of a patient, the series of IVUS images comprising a proximal IVUS frame, a distal IVUS frame, and a plurality of interior IVUS frames; determine a raw lumen area represented in each of the plurality of interior IVUS frames; generate, for each of the plurality of interior IVUS frames, a smooth lumen area based on a sampling filter and the lumen area; identify as a smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the least smooth lumen area of the plurality interior IVUS frames; select a subset of the plurality of interior IVUS frames, the subset of interior IVUS frames comprising the smooth lumen frame and at least one other of the plurality of interior IVUS frames, wherein the smooth lumen frame and the at least one other of the plurality of interior IVUS frames are co-linear; and identify as a minimum key frame, the one of the subset of the plurality of interior IVUS frames having a raw lumen area less than or equal to the lumen area of the subset of the plurality interior IVUS frames.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the subset of the plurality of interior IVUS frames comprises 21 frames.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the subset of the plurality of interior IVUS frames comprises ones of the series of IVUS images represent a distance along the vessel.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the distance is less than or equal to 2 millimeters.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the interior IVUS frames are disposed between the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the series of IVUS images comprise a plurality of IVUS image frames and the instructions in response to being executed by the processor can further cause the processor to designate the first frame in the series of IVUS images as the distal key frame; and designate the last frame in the series of IVUS images as the proximal key frame.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the instructions in response to being executed by the processor can further cause the processor to receive, from an input device, an indication of the proximal IVUS frame and the distal IVUS frame.

Alternatively, or additionally in any of the embodiments of an at least one machine readable storage device above, the series of IVUS images comprise a plurality of IVUS image frames and wherein identifying as the smooth lumen frame, the one of the plurality of interior IVUS frames having a smooth lumen area less than or equal to the smooth lumen area of the plurality interior IVUS frames comprises: designate each one of the plurality IVUS images disposed between the proximal IVUS frame and the distal IVUS frame as interior IVUS frames; identify the least smooth lumen area of the plurality of interior IVUS frames; identify the one or more of the plurality of interior IVUS frames having a smooth lumen area equal to the least smooth lumen area; and designate a one of the identified one or more of the plurality of interior IVUS frames disposed central to the proximal IVUS frame and the distal IVUS frame as the minimum smooth lumen frame.

DETAILED DESCRIPTION

The foregoing has broadly outlined the features and technical advantages of the present disclosure such that the following detailed description of the disclosure may be better understood. It is to be appreciated by those skilled in the art that the embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. The novel features of the disclosure, both as to its organization and operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description and is not intended as a definition of the limits of the present disclosure.

As noted, the present disclosure relates to IVUS images and lumens (e.g., vessels) of patients and to processing an IVUS recording, or said differently, processing a series of IVUS images. As such, an example IVUS imaging system, patient vessel, and series of IVUS images is described.

Suitable IVUS imaging systems include, but are not limited to, one or more transducers disposed on a distal end of a catheter configured and arranged for percutaneous insertion into a patient. Examples of IVUS imaging systems with catheters are found in, for example, U.S. Pat. Nos. 7,246,959; 7,306,561; and 6,945,938; as well as U.S. Patent Application Publication Numbers 2006/0100522; 2006/0106320; 2006/0173350; 2006/0253028; 2007/0016054; and 2007/0038111; all of which are incorporated herein by reference.

FIG.1illustrates one embodiment of an IVUS imaging system100. The IVUS imaging system100includes a catheter102that is couplable to a control system104. The control system104may include, for example, a processor106, a pulse generator108, and a drive unit110. The pulse generator108forms electric pulses that may be input to one or more transducers (not shown) disposed in the catheter102.

With some embodiments, mechanical energy from the drive unit110can be used to drive an imaging core (also not shown) disposed in the catheter102. In at least some embodiments, electric signals transmitted from the one or more transducers may be input to the processor106for processing. In at least some embodiments, the processed electric signals from the one or more transducers can be used to form a series of images, described in more detail below. For example, a scan converter can be used to map scan line samples (e.g., radial scan line samples, or the like) to a two-dimensional Cartesian grid, which can be used as the basis for a series of IVUS images that can be displayed for a user.

In at least some embodiments, the processor106may also be used to control the functioning of one or more of the other components of the control system104. For example, the processor106may be used to control at least one of the frequency or duration of the electrical pulses transmitted from the pulse generator108, the rotation rate of the imaging core by the drive unit110. Additionally, where IVUS imaging system100is configured for automatic pullback, the drive unit110can control the velocity and/or length of the pullback.

FIG.2illustrates an image200of a vessel202of a patient. As described, IVUS imaging systems (e.g., IVUS imaging system100, or the like) are used to capture a series of images or a “recording” or a vessel, such as, vessel202. For example, an IVUS catheter (e.g., catheter102) is inserted into vessel202and a recording, or a series of IVUS images, is captured as the catheter102is pulled back from a distal end204to a proximal end206. The catheter102can be pulled back manually or automatically (e.g., under control of drive unit110, or the like).

FIG.3AandFIG.3Billustrates two-dimensional (2D) representations of IVUS images of vessel202. For example,FIG.3Aillustrates IVUS images300adepicting a longitudinal view of the IVUS recording of vessel202between proximal end206and distal end204.

FIG.3Billustrates an image frame300bdepicting an on-axis (or short axis) view of vessel202at point302. Said differently, image frame300bis a single frame or single image from a series of IVUS images that can be captured between distal end204and proximal end206as described herein. As introduced above, the present disclosure provides systems and techniques to process raw IVUS images to identify regions of interest, such as, for example starting and ending points between which include frames of interest in a series of IVUS images.

For example, IVUS images300adepicts an entire series of IVUS images taken of vessel202between distal end204and proximal end206. However, not all these images may be of interest to a physician. The present disclosure provides to identify “key frames” such as, a proximal key frame and a distal key frame as well as a minimum key frame.

FIG.4illustrates an IVUS images visualization system400, according to some embodiments of the present disclosure. In general, IVUS images visualization system400is a system for processing, annotating, and presenting IVUS images. IVUS images visualization system400can be implemented in a commercial IVUS guidance or navigation system, such as, for example, the AVVIGO® Guidance System available from Boston Scientific®. The present disclosure provides advantages over prior or conventional IVUS navigation systems in that the automatic detection of key frames will reduce the time needed for patients to be in treatment. For example, the present disclosure can be implemented in an IVUS navigation system used in a percutaneous coronary intervention (PCI). The disclosure can be provided to provide a physician rapidly and automatically with information including graphical information elements that provide more information than the raw IVUS images alone. This information (e.g., key frame identification and graphical representation) can be determined automatically as part of a pre-PCI, peri-PIC, or post-PCI while the user (e.g., physician, or the like) is using the system and while the patient is in treatment. For example, where the physician is placing a stent, the present disclosure can be used to present images to the physician (e.g., on a display) and provide a navigation framework for the physician to identify the type, size, and location of the stent more quickly and easily. Thereby reducing the time in which the patient is under treatment.

With some embodiments, IVUS images visualization system400could be implemented as part of control system104. Alternatively, control system104could be implemented as part of IVUS images visualization system400. As depicted, IVUS images visualization system400includes a computing device402. Optionally, IVUS images visualization system400includes IVUS imaging system100and display404.

Computing device402can be any of a variety of computing devices. In some embodiments, computing device402can be incorporated into and/or implemented by a console of display404. With some embodiments, computing device402can be a workstation or server communicatively coupled to IVUS imaging system100and/or display404. With still other embodiments, computing device402can be provided by a cloud based computing device, such as, by a computing as a service system accessibly over a network (e.g., the Internet, an intranet, a wide area network, or the like). Computing device402can include processor406, memory408, input and/or output (I/O) devices410, network interface412, and IVUS imaging system acquisition circuitry414.

The processor406may include circuitry or processor logic, such as, for example, any of a variety of commercial processors. In some examples, processor406may include multiple processors, a multi-threaded processor, a multi-core processor (whether the multiple cores coexist on the same or separate dies), and/or a multi-processor architecture of some other variety by which multiple physically separate processors are in some way linked. Additionally, in some examples, the processor406may include graphics processing portions and may include dedicated memory, multiple-threaded processing and/or some other parallel processing capability. In some examples, the processor406may be an application specific integrated circuit (ASIC) or a field programmable integrated circuit (FPGA).

The memory408may include logic, a portion of which includes arrays of integrated circuits, forming non-volatile memory to persistently store data or a combination of non-volatile memory and volatile memory. It is to be appreciated, that the memory408may be based on any of a variety of technologies. In particular, the arrays of integrated circuits included in memory120may be arranged to form one or more types of memory, such as, for example, dynamic random access memory (DRAM), NAND memory, NOR memory, or the like.

I/O devices410can be any of a variety of devices to receive input and/or provide output. For example, I/O devices410can include, a keyboard, a mouse, a joystick, a foot pedal, a display, a touch enabled display, a haptic feedback device, an LED, or the like.

Network interface412can include logic and/or features to support a communication interface. For example, network interface412may include one or more interfaces that operate according to various communication protocols or standards to communicate over direct or network communication links. Direct communications may occur via use of communication protocols or standards described in one or more industry standards (including progenies and variants). For example, network interface412may facilitate communication over a bus, such as, for example, peripheral component interconnect express (PCIe), non-volatile memory express (NVMe), universal serial bus (USB), system management bus (SMBus), SAS (e.g., serial attached small computer system interface (SCSI)) interfaces, serial AT attachment (SATA) interfaces, or the like. Additionally, network interface412can include logic and/or features to enable communication over a variety of wired or wireless network standards (e.g.,502.11communication standards). For example, network interface412may be arranged to support wired communication protocols or standards, such as, Ethernet, or the like. As another example, network interface412may be arranged to support wireless communication protocols or standards, such as, for example, Wi-Fi, Bluetooth, ZigBee, LTE, 5G, or the like.

The IVUS imaging system acquisition circuitry414may include circuitry including custom manufactured or specially programmed circuitry configured to receive or receive and send signals between IVUS imaging system100including indications of an IVUS run, a series of IVUS images, or a frame or frames of IVUS images.

Memory408can include instructions416. During operation processor406can execute instructions416to cause computing device402to receive (e.g., from IVUS imaging system100, or the like) a recording of an “IVUS run” and store the recording as IVUS images418in memory408. For example, processor406can execute instructions416to receive information elements from IVUS imaging system100comprising indications of IVUS images captured by catheter102while being pulled back from distal end204to proximal end206, which images comprising indications of the anatomy and/or structure of vessel202including vessel walls and plaque. It is to be appreciated that IVUS images418can be stored in a variety of image formats or even non-image formats or data structures that comprise indications of vessel202. Further, IVUS images418includes several “frames” or individual images that, when represented co-linearly can be used to form an image of the vessel202, such as, for example, as represented by IVUS images300aand/or300b.

The present disclosure provides to process IVUS images to identify key frames from the frames in IVUS images418. For example, the present disclosure provides to identify a distal key frame420, a proximal key frame422, and a minimum key frame424. Processor406can execute instructions416to identify the minimum key frame424. In some instances, processor406can execute instructions416to further identify the distal key frame420and/or the proximal key frame422. However, with other instances, processor406can execute instructions416to receive an indication (e.g., via I/O devices410, or the like) of the distal key frame420and/or proximal key frame422. In some embodiments, processor406can execute instructions416to identify distal key frame420, proximal key frame422, and minimum key frame424. Subsequently, processor406can execute instructions416to receive an indication of an updated distal key frame420and/or proximal key frame422and can re-identify minimum key frame424based on the updated distal key frame420and/or proximal key frame422as outlined herein.

In general, processor406can execute instructions416to identify the key frames based on the raw lumen boundaries as well as smoothed lumen boundaries. More specifically, processor406can execute instructions416to determine raw lumen area426for frames in IVUS images418. Further, processor406can execute instructions416to determine a smooth lumen area428from raw lumen area426. With some embodiments, the smooth lumen area428can be determined based on a moving average filter or an n-sample median filter. As a specific example, processor406can execute instructions416to determine smooth lumen area428based on a 21-sample median filter applied to IVUS images418and raw lumen area426. Further, with some embodiments, smooth lumen area428can be determined based on a first filter and a second filter where the first filter is an n-sample median filter, and the second filter is an n-distance median filter. This is described in greater detail below. However, in general, where IVUS images418are captured via a manual pullback operation, smooth lumen area428can be determined based on a single filter (e.g., n-sample median filter). Conversely, where IVUS images418are captured via an automatic pullback operation, smooth lumen area428can be determined based on multiple filters (e.g., n-sample median filter and n-distance median filter).

Processor406can execute instructions416to determine a minimum region subset of IVUS images430from the frames in IVUS images418located co-linearly between the distal key frame420and the proximal key frame422by identifying a smooth lumen frame432based on the smooth lumen areas428and then identify the minimum key frame424from the frames in the minimum region subset of IVUS images430based on the raw lumen area426. This provides an advantage in that errors in processing IVUS images418and determining raw lumen area426(e.g., automatic border detection errors, or the like) do not lead to mis-identification of minimum key frame424.

In some embodiments, processor406can execute instructions416to identify the distal key frame420and the proximal key frame422based on a plaque burden, or rather, a ratio of raw lumen area426to vessel area434. In such examples, processor406can execute instructions416to determine vessel area434for frames in IVUS images418and determine the distal key frame420and the proximal key frames422based on a ratio of raw lumen area426over vessel area43or of the smooth lumen areas428over the smooth vessel areas436. This is described in greater detail below.

FIG.5illustrates a logic flow500to identify key frames from an IVUS recording, according to some embodiments of the present disclosure. The logic flow500can be implemented by IVUS images visualization system400and will be described with reference to IVUS images visualization system400for clarity of presentation. However, it is noted that logic flow500could also be implemented by an IVUS guidance system different than IVUS images visualization system400.

Logic flow500can begin at block502. At block502“receive a series of intravascular ultrasound (IVUS) images of a vessel of a patient” a series of IVUS images captured via an IVUS catheter percutaneously inserted in a vessel of a patent can be received. For example, information elements comprising indications of IVUS images418can be received from IVUS imaging system100where catheter102is (or was) percutaneously inserted into vessel202. The IVUS images418can comprise frames of images representative of images captured while the catheter102is pulled back from distal end204to proximal end206. Processor406can execute instructions416to receive information elements comprising indications of IVUS images418from IVUS imaging system100, or directly from catheter102as may be the case.

Continuing to block504“determine a lumen area represented in each of the IVUS images” a lumen area represented in each frame of the IVUS images can be determined. For example, the raw lumen area426represented in each frame of the IVUS images418can be determined. Processor406can execute instructions416to determine raw lumen area426for IVUS images418. Further, with some embodiments, at block504processor406can execute instructions416to determine vessel area434for IVUS images418.

For example,FIG.6illustrates an image600showing an on-axis view of a frame of IVUS images418depicting a portion of vessel202. Vessel area434and raw lumen area426can be determined for the portion of vessel202represented in the depicted frame. Continuing to block506“smooth the raw lumen area” the raw lumen areas can be smoothed. For example, the raw lumen areas can be smoothed. In some embodiments, processor406can execute instructions416to determine smooth lumen areas428based on raw lumen areas426and an n-sample median filter. Smooth lumen areas428can be based on a 21-sample median filter applied to raw lumen areas426. With other embodiments, processor406can execute instructions416to determine smooth lumen areas428based on raw lumen areas426and an n-sample median filter as well as an n-distance median filter. For example, smooth lumen areas428can be determined from raw lumen areas426and an n-sample median filter (e.g., 21-sample, or the like). Subsequently, “smoother” smooth lumen areas428can be determined from the smooth lumen areas428and an n-distance median filter. The n-distance median filter can be a 1 millimeter (mm) median filter, a 2 mm median filter, a 3 mm median filter, or a 4 mm median filter. It is noted that the “smoother” smooth lumen areas428can be used where the IVUS images418are based on an automatic pullback IVUS run.

Continuing to decision block508“distal and proximal key frames known” a determination of whether the distal key frame420and proximal key frame422are known is made. For example, processor406can execute instructions416to determine whether indications of the distal key frame420and proximal key frame422have been received (e.g., via I/O devices410, or the like). From decision block508, logic flow500can continue to either block510or proceed to block512. Logic flow500can continue from decision block508to block510based on a determination at decision block508that the distal key frame420and proximal key frame422are unknown while logic flow500can proceed from decision block508to block512based on a determination at decision block508that the distal key frame420and proximal key frame422are known.

At block510“identify distal key frame and/or proximal key frame” the distal key frame420and/or the proximal key frame422can be determined. In some examples, both the distal key frame420and the proximal key frame422can be determined at block510. In some instances, an indication of one of the key frames (e.g., distal key frame420or proximal key frame422) may have been received and as such, the other key frame can be identified at block510. In general, the distal key frame420and the proximal key frame422can be identified based in part on the smooth lumen area428. However, specific examples of determining the distal key frame420and the proximal key frame422are provided below. For example,FIG.7andFIG.8depict logic flows700and800, respectively, which can be implemented to identify the distal key frame420and the proximal key frame422for IVUS images418captured based on a manual pullback IVUS run (e.g., logic flow700) or an automatic pullback IVUS run (e.g., logic flow800).

Continuing to block512“identify a smooth lumen frame from the IVUS images” a frame from the series of IVUS images can be identified and designated as the smooth lumen frame. For example, processor406can execute instructions416to identify smooth lumen frame432from IVUS images418based on the distal key frame420, proximal key frame422, and smooth lumen areas428of each IVUS images418. With some embodiments, processor406can execute instructions416to identify the frame of IVUS images418located between distal key frame420and proximal key frame422that has the smallest smooth lumen area428. Where multiple frames of IVUS images418all have equal smooth lumen areas428, which are the smallest smooth lumen area428, processor406can identify the middle frame as the smooth lumen frame432.

Continuing to block514“identify a subset of the IVUS images, the subset comprising the smooth lumen frame and a number of co-linear frames” a subset of the IVUS images can be identified. More particularly, the minimum region subset of IVUS images430can be determined from IVUS images418based on the smooth lumen frame432. For example, processor406can execute instructions416to identify several frames of IVUS images418(at least 2, at least 11, several frames representing a selected distance along vessel202, or the like) that are co-linearly located with the smooth lumen frame432.

With some embodiments, for example, where IVUS images418are captured based on a manual pullback IVUS run, processor406can execute instructions416to identify the minimum region subset of IVUS images430based on identifying (or designating, marking, or flagging) the smooth lumen frame432as well as the m (e.g., 10, between 5 and 15, or the like) frames co-linearly distal and co-linearly proximal to the smooth lumen frame as the minimum region subset of IVUS images430.

In some embodiments, for example, where IVUS images418are captured based on an automatic pullback IVUS run, processor406can execute instructions416to identify the minimum region subset of IVUS images430based on identifying (or designating, marking, or flagging) the smooth lumen frame432as well as the frames from IVUS images418that are captured within x distance (e.g., 1 mm, 1.5 mm, 2 mm, between 0.5 and 2 mm, or the like) of the smooth lumen frame432as the minimum region subset of IVUS images430.

In some embodiments, where the distal key frame420or the proximal key frame422would be identified as being included in the minimum region subset of IVUS images430, the minimum region subset of IVUS images430can be truncated to the frame just distal to the proximal key frame422or just proximal to the distal key frame420.

Continuing to block516“identify a minimum key frame from the subset of the plurality of IVUS images” the frame with the smallest lumen area, or the minimum key frame, can be identified from the subset of the IVUS images and the raw lumen areas. For example, the minimum key frame424can be identified from the smooth lumen frame432based on the raw lumen areas426. Processor406can execute instructions416to identify the frame from within the minimum region subset of IVUS images430that has the smallest raw lumen area426.

FIG.7illustrates the logic flow700, which can be implemented to identify the distal key frame420and/or the proximal key frame422for IVUS images418captured based on a manual IVUS run. As noted above, with some embodiments, logic flow500can implement logic flow700at block510. Logic flow700can begin at block702. At block702“identify the first frame in the recording as the distal key frame” the first frame in the IVUS recording can be designated or identified as the distal key frame. For example, processor406can execute instructions416to identify the first frame of the IVUS images418as the distal key frame420. Continuing to block704“identify the last frame in the recording as the proximal key frame” the last frame in the IVUS recording can be designated or identified as the proximal key frame. For example, processor406can execute instructions416to identify the last frame of the IVUS images418as the proximal key frame422.

FIG.8illustrates the logic flow800, which can be implemented to identify the distal key frame420and/or the proximal key frame422based on the smooth lumen area428for IVUS images418captured based on a manual IVUS run. As noted above, with some embodiments, logic flow500can implement logic flow800at block510. Logic flow800can begin at block802. At block802“identify an initial distal search frame” an initial distal search frame can be identified. For example, processor406can execute instructions416to identify an initial distal search frame as either (1) the first frame in the series of IVUS images418or (2) the most distal frame in the series of IVUS images418where the smooth lumen area428less than or equal to the smooth lumen area428of the next distal frame plus 0.5 mm2.

Continuing to block804“identify an initial proximal search frame” an initial proximal search frame can be identified. For example, processor406can execute instructions416to identify an initial proximal search frame as the frame that is most distal to (1) a specified distance (e.g., 1 mm, 1.1 mm, between 0.5 mm and 1.5 mm, or the like) distal to the last frame in the series of IVUS images418, (2) the most proximal frame prior to the start of the guide catheter, or (3) consecutive frames with descending smooth lumen areas428.

Continuing to block806“identify, as the central search frame, the frame between the initial proximal and distal search frames with the smallest raw lumen area located within a first distance from the frame with the smallest smooth lumen area” a central search frame can be identified as the frame between the initial proximal and distal search frames with the smallest raw lumen area located within a first distance from the frame with the smallest smooth lumen area. For example, processor406can execute instructions416to identify and designate as the central search frame the frame with the smallest raw lumen area426located within the first distance (e.g., 1 mm, 1.1 mm, between 0.5 mm and 1.5 mm, or the like) of the frame located between the initial proximal and distal search frames with the smallest smooth lumen area428. As noted above, in some examples, the smooth lumen area can be determined based on multiple types of sampling filters. Said differently, a smooth lumen area may be determined, and a smoother lumen area may be determined. With some examples, the central search frame identified at block806can be identified based on the smoother lumen area and the raw lumen area as described above.

Continuing to block808“smooth the vessel area” the vessel area can be smoothed. For example, the processor406can execute instructions416to determine the smooth vessel areas436based on vessel areas434and an n-sample median filter. Continuing to block810“identify, as the distal and proximal key frames, the frames at least a second distance away from the central search frame with a smooth and a raw plaque burden less than or equal to a threshold” the distal and proximal key frames can be identified as the most proximate frames to the central search frame having a smooth and a raw plaque burden less than or equal to 50%. Processor406can execute instructions416to identify the distal key frame420as the frame at least a specified distance distal to the central search frame that has a smooth plaque burden (e.g., smooth lumen area428over smooth vessel area436) less than or equal to a threshold (e.g., 50%, 60%, between 40 and 60%, or the like). In some examples, processor406can execute instructions416to determine that no frame distal to the central search frame has a raw and smooth plaque burden less than or equal to 50% and in such a case identify the initial distal search frame as the distal key frame420.

Similarly, processor406can execute instructions416to identify the proximal key frame422as the frame at least a specified distance proximal to the central search frame that has a smooth plaque burden (e.g., smooth lumen area428over smooth vessel area436) less than or equal to a threshold (e.g., 50%, 60%, between 40 and 60%, or the like). In some examples, processor406can execute instructions416to determine that no frame proximal to the central search frame has a raw and smooth plaque burden less than or equal to 50% and in such a case identify either (1) the last frame in the run or (2) the frame just distal to the guide catheter, whichever is more distal, as the proximal key frame422.

FIG.9illustrates an annotated image900representative of an annotation of the image200depicting vessel202, according to some embodiments of the present disclosure. As described herein, the present disclosure provides systems and techniques to automatically (e.g., without user input) identify key frames such as, the distal key frame420, proximal key frame422, and minimum key frame424. With some examples, an image of the vessel202of the patient can be annotated with indications of the key frames as well as the start frames of the IVUS run. For example, processor406can execute instructions416to generate a graphical information element to be rendered and displayed on display404that represents annotated image900. As depicted, the annotated image900includes a depiction of the vessel202, the distal end204, the proximal end206, the distal key frame420, the proximal key frame422, and the minimum key frame424.

FIG.10illustrates graphical element1000, which can be generated and presented to a user according to some embodiments of the present disclosure. Processor406can execute instructions416to generate the graphical element1000including a cartoon depiction of the vessel area434and the raw lumen area426as well as the distal key frame420, proximal key frame422, and minimum key frame424.

Annotated image900depicted inFIG.9and/or graphical element1000depicted inFIG.10can be generated by IVUS images visualization system400and displayed on display404as part of a PCI as outlined above.

FIG.11illustrates computer-readable storage medium1100. Computer-readable storage medium1100may comprise any non-transitory computer-readable storage medium or machine-readable storage medium, such as an optical, magnetic or semiconductor storage medium. In various embodiments, computer-readable storage medium1100may comprise an article of manufacture. In some embodiments, computer-readable storage medium1100may store computer executable instructions1102with which circuitry (e.g., processor106, processor406, IVUS imaging system acquisition circuitry414, and the like) can execute. For example, computer executable instructions1102can include instructions to implement operations described with respect to instructions416, logic flow500, logic flow700, logic flow800, annotated image900, and/or graphical element1000. Examples of computer-readable storage medium1100or machine-readable storage medium may include any tangible media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth. Examples of computer executable instructions1102may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like.

FIG.12illustrates a diagrammatic representation of a machine1200in the form of a computer system within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein. More specifically,FIG.12shows a diagrammatic representation of the machine1200in the example form of a computer system, within which instructions1208(e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine1200to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions1208may cause the machine1200to execute logic flow500ofFIG.5, logic flow700ofFIG.7, logic flow800ofFIG.8, or the like. More generally, the instructions1208may cause the machine1200to automatically determine key frames during a pre-PCI, peri-PCI, or post-PCI using IVUS. It is noted that the present disclosure provides specific and discrete implementations of identifying key frames (e.g., distal key frame420, proximal key frame422, and/or minimum key frame424) which is a significant improvement over the prior art. In particular, the present disclosure provides an improvement to computing technology in that key frames (e.g., particularly the minimum key frame424) is identified while an IVUS imaging system is in use and can be re-identified based on updated factors (e.g., a user moving the distal key frame420or proximal key frame422).

The instructions1208transform the general, non-programmed machine1200into a particular machine1200programmed to carry out the described and illustrated functions in a specific manner. In alternative embodiments, the machine1200operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine1200may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine1200may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions1208, sequentially or otherwise, that specify actions to be taken by the machine1200. Further, while only a single machine1200is illustrated, the term “machine” shall also be taken to include a collection of machines1200that individually or jointly execute the instructions1208to perform any one or more of the methodologies discussed herein.

The machine1200may include processors1202, memory1204, and I/O components1242, which may be configured to communicate with each other such as via a bus1244. In an example embodiment, the processors1202(e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor1206and a processor1210that may execute the instructions1208. The term “processor” is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. AlthoughFIG.12shows multiple processors1202, the machine1200may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

The memory1204may include a main memory1212, a static memory1214, and a storage unit1216, both accessible to the processors1202such as via the bus1244. The main memory1204, the static memory1214, and storage unit1216store the instructions1208embodying any one or more of the methodologies or functions described herein. The instructions1208may also reside, completely or partially, within the main memory1212, within the static memory1214, within machine-readable medium1218within the storage unit1216, within at least one of the processors1202(e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine1200.

Communication may be implemented using a wide variety of technologies. The I/O components1242may include communication components1240operable to couple the machine1200to a network1220or devices1222via a coupling1224and a coupling1226, respectively. For example, the communication components1240may include a network interface component or another suitable device to interface with the network1220. In further examples, the communication components1240may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices1222may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

The various memories (i.e., memory1204, main memory1212, static memory1214, and/or memory of the processors1202) and/or storage unit1216may store one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions1208), when executed by processors1202, cause various operations to implement the disclosed embodiments.

The instructions1208may be transmitted or received over the network1220using a transmission medium via a network interface device (e.g., a network interface component included in the communication components1240) and utilizing any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions1208may be transmitted or received using a transmission medium via the coupling1226(e.g., a peer-to-peer coupling) to the devices1222. The terms “transmission medium” and “signal medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signal medium” shall be taken to include any intangible medium that can store, encoding, or carrying the instructions1208for execution by the machine1200, and includes digital or analog communications signals or other intangible media to facilitate communication of such software. Hence, the terms “transmission medium” and “signal medium” shall be taken to include any form of modulated data signal, carrier wave, and so forth. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a matter as to encode information in the signal.

Terms used herein should be accorded their ordinary meaning in the relevant arts, or the meaning indicated by their use in context, but if an express definition is provided, that meaning controls.

By using genuine models of anatomy more accurate surgical plans may be developed than through statistical modeling.

Terms used herein should be accorded their ordinary meaning in the relevant arts, or the meaning indicated by their use in context, but if an express definition is provided, that meaning controls.