Patent Description:
The invention provides a method as set out in claim <NUM> and a device as set out in claim <NUM>. The disclosure is generally directed to automatically aligning representations of intravascular data obtained during two or more pullbacks. The pullbacks may be performed using the same and/or different modalities. For example, a first pullback may record intravascular images and a second pullback may record intravascular data measurements, such as pressure measurements. A representation of each of the pullbacks may be vertically aligned such that corresponding segments of the vessel in which the pullbacks were performed are vertically aligned.

One or more extraluminal images, such as angiograms, x-rays, or the like, may be taken before, during, or after a guide catheter is inserted into a target vessel. A plurality of pullbacks may be performed by a physician using, for example, an OCT probe, IVUS probe, pressure wire, micro-OCT probe, near-infrared spectrometry (NIRS) sensor, etc. A distance between an end point of each pullback and a proximal tip of the catheter may be determined, for example, by measuring the pixels between the end of each pullback and the proximal tip of the catheter. The start point of each pullback may be a distal-most point where intravascular data is captured, and the end point may be a proximal-most point. The proximal tip of the catheter may be, in some examples, referred to as a junction point, wherein the junction point is where the catheter and the guide wire meet. In some examples, the junction point is a point that is the same for each pullback. For example, the catheter may not move between each pullback such that the proximal tip of the catheter, or the junction point, is in the same location for each pullback. A difference between the end point of each pullback and the junction point may be determined, for example, by subtracting the distance of the end of the first pullback to the junction point from the distance of the end of the second pullback to the junction point. Such difference may correspond to an offset distance, the offset distance being a distance by which an end point of the representation of the second pullback is horizontally offset from an end point of the representation of the first pullback when the first and second representations are vertically aligned at the junction point. For example, if the second pullback has an end point further way from the junction point than the first pullback, the end point of the representation of the second pullback may be horizontally offset from the end point of the representation of the first pullback by the offset difference.

Aspects of the disclosed technology can include any combination of the features described herein.

One aspect of the disclosure provides a method, comprising receiving, by one or more processors, a plurality of extraluminal images of a target blood vessel, detecting, by the one or more processors, a junction point in the plurality of extraluminal images, receiving, by the one or more processors, a first set of intravascular data taken during a first pullback of a first intravascular device, determining, by the one or more processors, a first distance between the junction point and a first distal end point of the first pullback, receiving, by the one or more processors, a second set of intravascular data taken during a second pullback of a second intravascular device, determining, by the one or more processors, a second distance between the junction point and a second distal end point of the second pullback; and aligning, by the one or more processors based on the first distance and the second distance, a first representation of the first set of intravascular data with a second representation of the second set of intravascular data. The method may further include outputting for display, by the one or more processors, the first representation and the second representation, wherein first representation is vertically aligned with the second representation. Aligning the first representation and the second representation may include determining, by the one or more processors, a difference between the second distance and the first distance, and horizontally offsetting, by the one or more processors based on the determined difference, a distal end of the second representation from a distal end of the first representation. The junction point may be a proximal point of the guide catheter. The first intravascular may be the same device as the second intravascular device. The first and second sets of intravascular data may include intravascular images. The plurality of extraluminal images may be taken during both the first pullback and the second pullback. Detecting the junction point may further include determining, by the one or more processors based on the plurality of extraluminal images, the set of intravascular data, or the second set of intravascular data, at least one artificial intelligence ("AI") mask. Determining the first distal end point of the first pullback and the second distal end point of the second pullback may further include determining, based on the plurality of extraluminal images, the first set of intravascular data, or the second set of intravascular data, at least one wire mask image frame. According to some examples, the method may further include co-registering, by the one or more processors, the first set of intravascular data and the second set of intravascular data to the plurality of extraluminal images.

Another aspect of the disclosure provides a device, comprising one or more processors, The one or more processors may be configured to receive a plurality of extraluminal images of a target blood vessel, detect a junction point in the plurality of extraluminal images, receive a first set of intravascular data taken during a first pullback of a first intravascular device, determine a first distance between the junction point and a first distal end point of the first pullback, receive a second set of intravascular data taken during a second pullback of a second intravascular device, determine a second distance between the junction point and a second distal end point of the second pullback, and align, based on the first distance and the second distance, a first representation of the first set of intravascular data with a second representation of the second set of intravascular data. The one or more processors may be further configured to output for display the first representation and the second representation, wherein first representation is vertically aligned with the second representation. When aligning the first representation and the second representation the one or more processors may be further configured to determine a difference between the second distance and the first distance, and horizontally offset, based on the determined difference, a distal end of the second representation from a distal end of the first representation. The junction point may be a proximal point of the guide catheter. The first intravascular device may be the same device as the second intravascular device. The first and second sets of intravascular data may include intravascular images. The plurality of extraluminal images may be taken during both the first pullback and the second pullback. When detecting the junction point, the one or more processors may be further configured to determine, based on the plurality of extraluminal images, the set of intravascular data, or the second set of intravascular data, at least one artificial intelligence ("AI") mask. When determining the first distal end point of the first pullback and the second distal end point of the second pullback, the one or more processors may be further configured determine, based on the plurality of extraluminal images, the first set of intravascular data, or the second set of intravascular data, at least one wire mask image frame. The one or more processors may be further configured to co-register the first set of intravascular data and the second set of intravascular data to the plurality of extraluminal images.

Figure 1A illustrates a data collection system <NUM> for use in collecting intravascular and extravascular data. The system may include a data collection probe <NUM>. The data collection probe <NUM> may be, for example, an OCT probe, an IVUS catheter, micro-OCT probe, near infrared spectroscopy (NIRS) sensor, or any other device that can be used to image a blood vessel <NUM>. In some examples, the data collection probe <NUM> may be a pressure wire, a flow meter, etc. While the examples provided herein refer to an intravascular imaging device, such as an OCT probe, the use of an OCT probe is not intended to be limiting. For example, an IVUS catheter, a pressure wire, or another intravascular data collection device may be used in conjunction with or instead of the OCT probe. A guidewire, not shown, may be used to introduce the probe <NUM> into the blood vessel <NUM>. The probe <NUM> may be introduced and pulled back along a length of a blood vessel while collecting data. The intravascular data sets, or frames of image data, may be used to identify features, such as the cross-sectional area of the bolus.

The probe <NUM> may be connected to a subsystem <NUM> via an optical fiber <NUM>. The subsystem <NUM> may include a light source, such as a laser, an interferometer having a sample arm and a reference arm, various optical paths, a clock generator, photodiodes, and other OCT, IVUS, micro-OCT, NIRS, and/or pressure wire components.

The probe <NUM> may be connected to an optical receiver <NUM>. According to some examples, the optical receiver <NUM> may be a balanced photodiode based system. The optical receiver <NUM> may be configured to receive light collected by the probe <NUM>. The probe <NUM> may be coupled to the optical receiver <NUM> via a wired or wireless connection.

The system <NUM> may further include, or be configured to receive data from, an external imaging device <NUM>. The external imaging device may be, for example, an imaging system based on angiography, fluoroscopy, x-ray-, nuclear magnetic resonance, computer aided tomography, etc. The external imaging device <NUM> may be configured to noninvasively image the blood vessel <NUM>. According to some examples, the external imaging device <NUM> may obtain one or more images before, during, and/or after a pullback of the data collection probe <NUM>.

The external imaging device <NUM> may be in communication with subsystem <NUM>. According to some examples, the external imaging device <NUM> may be wirelessly coupled to subsystem <NUM> via a communications interface, such as Wi-Fi or Bluetooth. In some examples, the external imaging device <NUM> may be in communication with subsystem <NUM> via a wire, such as an optical fiber. In yet another example, external imaging device <NUM> may be indirectly communicatively coupled to subsystem <NUM> or computing device <NUM>. For example, the external imaging device <NUM> may be coupled to a separate computing device (not shown) that is in communication with computing device <NUM>. As another example, image data from the external imaging device <NUM> may be transferred to the computing device <NUM> using a computer-readable storage medium.

The subsystem <NUM> may include a computing device <NUM>. One or more steps may be performed automatically or without user input to navigate images, input information, select and/or interact with an input, etc. In some examples, one or more steps may be performed based on receiving a user input by mouse clicks, a keyboard, touch screen, verbal commands, etc..

The computing device may include one or more processors <NUM>, memory <NUM>, instructions <NUM>, data <NUM>, and one or more modules <NUM>.

The one or more processors <NUM> may be any conventional processors, such as commercially available microprocessors. Alternatively, the one or more processors may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although <FIG> functionally illustrates the processor, memory, and other elements of device <NUM> as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage media located in a housing different from that of device <NUM>. Accordingly, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

Memory <NUM> may store information that is accessible by the processors, including instructions <NUM> that may be executed by the processors <NUM>, and data <NUM>. The memory <NUM> may be a type of memory operative to store information accessible by the processors <NUM>, including a non-transitory computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, read-only memory ("ROM"), random access memory ("RAM"), optical disks, as well as other write-capable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions <NUM> and data <NUM> are stored on different types of media.

Memory <NUM> may be retrieved, stored or modified by processors <NUM> in accordance with the instructions <NUM>. For instance, although the present disclosure is not limited by a particular data structure, the data <NUM> may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data <NUM> may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. By further way of example only, the data <NUM> may be stored as bitmaps comprised of pixels that are stored in compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. Moreover, the data <NUM> may comprise information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data.

The instructions <NUM> can be any set of instructions to be executed directly, such as machine code, or indirectly, such as scripts, by the processor <NUM>. In that regard, the terms "instructions," "application," "steps," and "programs" can be used interchangeably herein. The instructions can be stored in object code format for direct processing by the processor, or in any other computing device language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance.

The modules <NUM> may include a co-registration module, a wire mask module, a junction point detection module, and an alignment module.

The computing device <NUM> may be adapted to co-register intravascular data with a luminal image. For example, computing device <NUM> may access co-registration module <NUM> to co-register the intravascular data with the luminal image. The luminal image may be an extraluminal image, such as an angiograph, x-ray, or the like. The co-registration module <NUM> may co-register intravascular data, such as an intravascular image, pressure readings, virtual flow reserve ("VFR"), fractional flow reserve ("FFR"), resting full-cycle ration ("RFR"), flow rates, etc. with the extraluminal image. In some examples, the co-registration module <NUM> may co-register intravascular data with an intraluminal image, such as an intraluminal image captured by an OCT probe, IVUS probe, micro-OCT probe, or the link.

In one example, the co-registration module <NUM> may co-register intraluminal data captured during a pullback with one or more extraluminal images. For example, the extraluminal image frames may be pre-processed. Various matrices such as convolution matrices, Hessians, and others can be applied on a per pixel basis to change the intensity, remove, or otherwise modify a given angiography image frame. As discussed herein, the preprocessing stage may enhance, modify, and/or remove features of the extraluminal images to increase the accuracy, processing speed, success rate, and other properties of subsequent processing stages. A vessel centerline may be determined and/or calculated. In some examples, the vessel centerline may be superimposed or otherwise displayed relative to the pre-processed extraluminal image. According to some examples, the vessel centerline may represent a trajectory of the collection probe <NUM> through the blood vessel during a pullback. In some examples, the centerline may be referred to as a trace. Additionally or alternatively, marker bands or radiopaque markers may be detected in the extraluminal image frames. According to some examples, the extraluminal image frames and the data received by collection probe <NUM> may be co-registered based on the determined location of the marker bands.

The computing device <NUM> may be adapted to determine a wire mask of an extraluminal image. For example, the computing device <NUM> may access a wire mask module <NUM> to determine, or create, a wire mask image frame of the extraluminal image. According to some examples, the wire mask module <NUM> may process the extraluminal images to determine, or create, a mask of a catheter and/or guidewire within the vessel.

Determining a wire mask image frame may include smoothing the image, such as by normalizing one or more image frames. Normalizing the image frames may convert the pixels in the image frame into a numerical range between zero (<NUM>) and one (<NUM>). According to some examples, normalizing the image frames allows the trained AI network to recognize the pixels as various structures, such as the catheter and guide wire. Smoothing the extraluminal image may enhance elongated structures in image. According to some examples, the structures being elongated include one or more of vessels, guidewires, ribs, the catheter, or other edge containing elements in the image.

In some examples, one or more morphological filters may be applied to the image to eliminate wide structures in the image. Wide structures may be, for example, bone structures in the image frames. According to some examples, the morphological filter may be a bottom hat filter or any filter configured or constrained to enhance or select small scale features, such as thin elements. The morphological filter allows for the enhancement of dark elongated elements in the image that have typical scale to the structure element used in a given morphological filter, such as for example, a bottom hat filter. In some examples, the morphological filter can be replaced by a median filter to produce a similar result.

In some examples, a ridge enhancing filter or detector or a vessel segmentation filter may be applied to the image. The ridge enhancing filter may be a Frangi filter, a Hessian filter, or other ridge or edge detectors. The ridge enhancing filter may extract thin elongated features in the image. The ridge enhancing filter output may be thresholded to produce a binary image containing thin and elongated dark elements that appear as bright pixels in the thresholded image.

In some examples, after thresholding the ridge enhanced image, adaptive thresholding may be performed on the image. An adaptive binary threshold may be applied in order to reject image areas with intensity values that are not of interest. In some examples, bright areas that have an intensity greater than a threshold associated with an intensity value or range of values corresponding to dark values may be rejected.

The thresholded ridge enhanced image and the image produced after applying the adaptive binary threshold may be merged using a pixel-wise AND operator, to obtain a merged metallic wire mask component. The wire mask module <NUM> may then connect and filter wire fragments that are detected in the images.

The guidewire may be extracted in fragments and other components, such as the catheter, junction point, and/or markers on the guidewire, may be detected. The wire fragments may be joined using a combined measurement of a takeoff angle and a distance between the fragments.

According to some examples, the wire mask module <NUM> may perform post filtering of components and/or thinning of components may be performed to remove elements from the surrounding area that may have joined during the wire detection.

The wire mask module <NUM> may output and/or store in memory <NUM> a wire mask of the detected catheter <NUM>, guidewire <NUM>, and/or junction point <NUM>.

The computing device <NUM> may be adapted to determine the location of the proximal point of a guide catheter in one or more image frames. The proximal point of the guide catheter may be the junction point. For example, the computing device <NUM> may access a junction point detection module <NUM> to determine the location. According to some examples, the junction point detection module <NUM> may determine the location of the junction point using the wire mask output by wire mask module <NUM>.

In some examples, the junction point detection module <NUM> may determine the location of the junction point using a trained artificial intelligence ("AI") network. The AI network may be trained using one or more extraluminal images and/or annotations as input. The extraluminal images may be, for example, angiography image, CT images, MRI images, etc. The annotations may be annotations identifying the location of the proximal tip of the catheter and/or the location of the guidewire. The trained AI network may output an AI mask, as shown in <FIG>. The AI mask 100A may identify the catheter and/or the guidewire in the image. The AI mask 100A may additionally or alternatively identify the junction point <NUM>. The junction point may be, for example, the proximal tip of catheter <NUM>. In some examples, the junction point <NUM> is the location where catheter <NUM> and guide wire <NUM> meet or intersect. The junction point detection module <NUM> may be used the trained AI network to predict and/or identify the location of the junction point on subsequent image frames.

The computing device <NUM> may be adapted to align representations of two or more pullbacks. For example, computing device <NUM> may access alignment module <NUM> to align the representations. The alignment of the representations of two or more pullbacks further include vertically aligning a portion of a first representation to a portion of the second representation. The portion of the first representation and the second representation may correspond to the junction point, a location in the pullback, etc. Thus, the portion of the first representation and the second longitudinal may correspond to the same location in the vessel.

In some examples, to vertically align the representations such that portions of each representation corresponding to the same location within the vessel are vertically aligned, alignment module <NUM> may determine an offset distance. The offset distance may be a difference in the distance between the end of the first pullback and the junction point "L1" and the distance between the end of the second pullback and the junction point "L2". For example, the difference "ΔL" may be calculated by subtracting "L1" from "L2. " The difference between the ends of each pullback and the junction point may be the offset distance in which the second longitudinal representation is horizontally offset when vertically aligned with the first longitudinal representation.

According to some examples, the modules made additionally or alternatively include a video processing software module, a preprocessing software module, an image file size reduction software module, a catheter removal software module, a shadow removal software module, a vessel enhancement software module, a blob enhancement software module, a Laplacian of Gaussian filter or transform software module, a guidewire detection software module, an anatomic feature detection software module, stationary marker detection software module, a background subtraction module, a Frangi vesselness software module, an image intensity sampling module, a moving marker software detection module, iterative centerline testing software module, a background subtraction software module, a morphological close operation software module, a feature tracking software module, a catheter detection software module, a bottom hat filter software module, a path detection software module, a Dijkstra software module, a Viterbi software module, fast marching method based software modules, a vessel centerline generation software module, a vessel centerline tracking module software module, a Hessian software module, an intensity sampling software module, a superposition of image intensity software module and other suitable software modules as described herein.

The subsystem <NUM> may include a display <NUM> for outputting content to a user. The display <NUM> may be integrated with the computing device <NUM>, or it may be a standalone unit electronically coupled to the computing device <NUM>. The display <NUM> may output intravascular data relating to one or more features detected in the blood vessel and/or obtained during a pullback. For example, the output may include, without limitation, cross-sectional scan data, longitudinal scans, diameter graphs, image masks, lumen border, plaque sizes, plaque circumference, visual indicia of plaque location, visual indicia of risk posed to stent expansion, flow rate, etc. The display <NUM> may identify features with text, arrows, color coding, highlighting, contour lines, or other suitable human or machine-readable indicia.

According to some examples the display <NUM> may include a graphic user interface ("GUI"). The display <NUM> may be a touchscreen display in which a user can provide an input to navigate images, input information, select and/or interact with an input, etc. In some examples, the display <NUM> and/or computing device <NUM> may include an input device, such as a trackpad, mouse, keyboard, etc. that allows a user to navigate images, input information, select and/or interact with an input, etc. The display <NUM> alone or in combination with computing device <NUM> may allow for toggling between one or more viewing modes in response to user inputs. For example, a user may be able to toggle between different intravascular data, images, etc. recorded during each of the pullbacks.

In some examples, the display <NUM>, alone or in combination with computing device <NUM>, may present one or more menus as output to the physician, and the physician may provide input in response by selecting an item from the one or more menus. For example, the menu may allow a user to show or hide various features. As another example, there may be a menu for selecting blood vessel features to display.

The output may include aligned representations of intravascular data received from a plurality of pullbacks. For example, each pullback may be output as a longitudinal representation of the vessel, a graphical representation of the intravascular data, etc. According to some examples, the representation of the first pullback and the representation of the second pullback may be vertically aligned. The intravascular data from each of the pullbacks may be aligned based on a difference between the end of each pullback and the junction point, as discussed herein with respect to <FIG>. For example, a first pullback may end a distance "L1" from the junction point and a second pullback may end a distance "L2" from the junction point. The difference "ΔL" between "L2" and "L1" may be the distance that the end of the representation of the second pullback is horizontally offset from the end of the representation of the first pullback.

<FIG> illustrates an image obtained by external imaging device <NUM>. The image <NUM> may be one of a plurality of angiography images taken by external imaging device. In an example where the image <NUM> is an angiography image, the image <NUM> may be obtained before contrast is injected into the blood vessel, while contrast is injected into the blood vessel, or after contrast is injected in the blood vessel. As shown, the image <NUM> is obtained prior to contrast being inject into the blood vessel and after catheter <NUM> and guide wire <NUM> have been inserted into the vessel. Image <NUM> may show one or more blood vessels, the catheter <NUM>, and guide wire <NUM>. While figure <NUM> is an angiography image, the external imaging device may be, for example, a CT or MRI machine such that the images are CT or MRI images.

<FIG> illustrates the identification of a junction point <NUM> on image <NUM>. The junction point <NUM> is the proximal point of the catheter within the vessel. According to some examples, imaging processing techniques, machine learning "(ML"), and/or artificial intelligence ("AI") may identify the junction point <NUM>. In some examples, the wire mask module <NUM> and/or junction point detection module <NUM> may identify junction point <NUM>. The wire mask module and/or junction point detection module may use image processing techniques, ML, and/or AI to identify junction point <NUM>. For example, a trained AI network may be used to predict and/or identify the location of junction point <NUM>. Junction point <NUM> may be the proximal point of the guide catheter <NUM>.

<FIG> illustrates an example of an image taken by an external imaging device in which the start and end points of a first pullback have been identified. Image <NUM> may be a composite image, such that one or more external images taken during the pullback have been combined and/or superimposed with each other to be able to identify the start and end point of the first pullback. According to some examples, contrast may be injected prior to and/or during the first pullback. The contrast may enhance, identify, and/or be used to detect the vascular structure in the region of interest.

As shown, image <NUM> shows the location of the catheter <NUM>, the junction point <NUM>, and details regarding the first pullback. The location of catheter <NUM> and junction point <NUM> in image <NUM>, taken during the first pullback, may correspond or substantially correspond to the location of catheter <NUM> and junction point <NUM> in image <NUM>, taken before the pullback. That is, the catheter <NUM> and, therefore, the junction point <NUM> does not change its position or location within the blood vessel once catheter <NUM> is inserted into the blood vessel. The details regarding the first pullback may include the start point, or distal point <NUM>, of the first pullback, the end point, or proximal point <NUM>, of the first pullback, and the trace <NUM>, or path, of the pullback. The proximal point <NUM>, distal point <NUM>, and trace <NUM> may be identified, for example, by determining a wire mask image frame for each image frame. As described above, the wire mask module <NUM> may determine and/or create a wire mask image frame. The wire mask image frame may identify, highlight, or extract proximal point <NUM> of the pullback, distal point <NUM> of the pullback, trace <NUM>, catheter <NUM>, junction point <NUM>, and/or radiopaque markers (not shown) on the probe or along trace <NUM> image <NUM>.

According to some examples, image <NUM> may be a co-registered image. For example, image data recorded by the external imaging device <NUM> may be co-registered with intravascular data recorded by collection probe <NUM>. The collection probe <NUM> may be, for example, an OCT probe, IVUS probe, pressure wire, micro-OCT probe, NIRS sensor, etc. In some examples the image data recorded by the external imaging device <NUM> may be angiographic images and the intravascular data recorded by the collection probe <NUM> may be OCT images, IVUS images, pressure readings from a pressure wire, etc. The intravascular data being recorded by collection probe <NUM> may be displayed as part of a graphic user interface. The intravascular data may be recorded at a certain location within the vessel. The extraluminal image recorded by the external imaging device <NUM> may include the location at which the intravascular data was recorded. The intravascular data may be related to the extraluminal image such the intravascular data and the extraluminal image display the same vessel segment with different views and/or data.

<FIG> illustrates the distance between the junction point <NUM> and the proximal point <NUM> of the first pullback. The distance "L1" may be automatically determined based on the identified junction point <NUM> and proximal point <NUM>. For example, the location of junction point <NUM> in image <NUM> may be determined and/or identified by the trained AI network and the location of the proximal point <NUM> of the first pullback in image <NUM> may be identified by the wire mask module <NUM>. Knowing the location of junction point <NUM> and proximal point <NUM> in image <NUM>, a distance between the junction point <NUM> and proximal point <NUM> may be determined. The distance may be measured, for example, in pixels.

<FIG> illustrates an example of an image taken by the external imaging device in which the start and end points of a second pullback have been identified. Image <NUM> may be a composite image of a plurality of external images taken during the second pullback of an intravascular device. The second pullback may be completed using the same intravascular device or a different intravascular device than was used during the first pullback. For example, the first pullback may have used an OCT imaging probe while the second pullback may have used an IVUS imaging probe or a pressure wire.

According to some examples, the first pullback may be taken before any pre-coronary intervention ("PCI") is taken while the second pullback may be taken after PCI. In some examples, both the first and second pullbacks may occur before PCI or after PCI. For example, where the first pullback records OCT images and the second pullback records pressure measurements, both the first and second pullbacks may occur before and/or after PCI to be able to co-register pre- and/or post-PCI information.

Image <NUM> may identify catheter <NUM> and junction point <NUM>. Catheter <NUM> and junction point <NUM> may be in the same position or location within blood vessel as in images <NUM>, <NUM>. That is, the location or position of catheter <NUM> may not change after being inserted into the blood vessel and/or after a first pullback. Image <NUM> may additionally or alternatively identify distal point <NUM>, proximal point <NUM>, and trace <NUM> of the second pullback. The distal point <NUM>, proximal point <NUM>, and trace <NUM> may be identified, for example, by determining a wire mask image frame.

<FIG> illustrates the distance between the junction point <NUM> and the proximal point <NUM> of second first pullback. The distance "L2" may be automatically determined based on the identified junction point <NUM> and proximal point <NUM>. The distance "L2" may be more, less, or equal to the distance "L1.

<FIG> illustrates an alignment of a first representation of the intravascular data received during the first pullback and a second representation of intravascular data received during the second pullback. As shown, the first and second representations are each longitudinal representations of the vessel. However, if the first and second pullbacks are different modalities, such as a first pullback of an OCT probe and a second pullback of a pressure wire, the first representation may be a longitudinal representation of the vessel and the second representation may be a graphical representation of the pressure within the vessel. According to some examples, the different modalities may be an OCT probe, IVUS probe, pressure wire, micro-OCT probe, NIRS sensor, etc. The first representation and the second representation may be vertically and/or horizontally aligned based a difference between the first distance "L1" and the second distance "L2.

As shown in <FIG>, the first longitudinal representation <NUM> and the second longitudinal representation <NUM> may be vertically aligned such that a portion of the vessel in the first longitudinal representation <NUM> is vertically aligned with the same portion of the vessel in the second longitudinal representation <NUM>. The second longitudinal representation <NUM> may be horizontally offset from the first longitudinal representation <NUM> by an offset distance. The offset difference may be the difference "ΔL" between the second distance "L2" and the first distance "L1. " For example, the difference "ΔL" may be the difference between the distance from junction point <NUM> to proximal point <NUM> and the distance from junction point <NUM> to proximal point <NUM>. Therefore, the proximal end <NUM> of the second longitudinal representation <NUM> may be horizontally offset from the proximal end <NUM> of the first longitudinal representation <NUM> by the difference "ΔL. " The proximal end <NUM> of the second longitudinal representation <NUM> may correspond to proximal point <NUM> of the second pullback and proximal end <NUM> of the first longitudinal representation <NUM> may correspond to proximal point <NUM> of the first pullback.

According to some examples, the first pullback may occur before a user or physician performs any percutaneous coronary intervention ("PCI"). For example, the first pullback may be performed to receive intravascular data prior to any PCI in order for the physician to determine whether any additional testing and/or measurements are needed or to determine an appropriate intervention. The second pullback may be performed to obtain additional measurements. In some examples, the second pullback may be taken after PCI. By keeping the catheter in the same location within the vessel, the first and second pullbacks can be automatically aligned. The automatic alignment of the first and second pullbacks may allow the physician to easily see and/or compare the intravascular data from the first and second pullbacks without providing any user input.

<FIG> illustrates an example method of automatically aligning a first and second pullback. The following operations do not have to be performed in the precise order described below. Rather, various operations can be handled in a different order or simultaneously, and operations may be added or omitted.

For example, in block <NUM>, the system may receive a plurality of extraluminal images of a target blood vessel. The extraluminal images may be, for example, angiographic images. According to some examples, the extraluminal images may be taken after a guide catheter is inserted into the target blood vessel.

In block <NUM>, the system may detect a junction point in the plurality of extraluminal images. The junction point may be the proximal point of the guide catheter. In some examples, the junction point may be the location where the guide catheter meets the guide wire. The system may detect the junction point using a trained AI network. The trained AI network may output an AI mask for each of the extraluminal images. The AI mask may identify, highlight, or extract the guidewire, guide catheter, junction point, and/or radiopaque markers on the probe or guidewire.

In block <NUM>, the system may receive a first set of intravascular data taken during a first pullback of a first intravascular device. The intravascular data may be recorded by an OCT probe, IVUS probe, pressure wire, micro-OCT probe, NIRS sensor, etc. The first pullback may have a start point and an end point. The start point may be the distal point of the pullback and the end point may be the proximal point of the pullback. According to some examples, the system may determine the proximal point and/or distal point of the pullback by determining and/or creating a wire mask image frame. The intravascular data may be, for example, intravascular images, pressure measurements, flow measurements, etc..

In block <NUM>, the system may determine a first distance between the junction point and a first proximal end point of the first pullback. For example, the system may determine the distance between the end of the first pullback and the proximal end of the guide catheter.

In block <NUM>, the system may receive a second set of intravascular data taken during a second pullback of a second intravascular device. According to some examples, the first and second intravascular devices may be the same intravascular device. In another example, the first and second intravascular devices may be different intravascular devices. In such an example, the first intravascular device may be an OCT probe and the second intravascular device may be a pressure wire.

According to some examples, the first pullback may occur before PCI and the second pullback may occur after PCI. In some examples, both the first and second pullbacks may occur before or after PCI.

In block <NUM>, the system may determine a second distance between the junction point and a second distal end point of the second pullback. For example, the system may determine the distance between the end of the second pullback and the proximal end of the guide catheter.

In block <NUM>, the system may align a first representation of the first pullback and a second representation of the second pullback. Aligning the first representation and the second representation may include determining a difference between the second distance and the first difference and offsetting, based on the determined different, the distal end of the second pullback from the distal end of the first pullback. For example, the first and second representations may each be output for display as a longitudinal representation. The first longitudinal representation of the first pullback may be vertically aligned with the second longitudinal representation of the second pullback. The distal end of the second representation may be offset, or horizontally shifted, with respect to the distal end of the first representation. According to other examples, the proximal end of the second representation may be offset from the proximal end of the first representation. The offset may be the difference between the second distance and the first distance.

The aspects, embodiments, features, and examples of the disclosure are to be considered illustrative in all respects and are not intended to limit the disclosure, the scope of which is defined only by the claims. Other embodiments, modifications, and usages will be apparent to those skilled in the art without departing from the scope of the claimed invention.

Throughout the application, where compositions are described as having, including, or comprising specific components, or where processes are described as having, including or comprising specific process steps, it is contemplated that compositions of the present teachings also consist essentially of, or consist of, the recited components, and that the processes of the present teachings also consist essentially of, or consist of, the recited process steps.

In the application, where an element or component is said to be included in and/or selected from a list of recited elements or components, it should be understood that the element or component can be any one of the recited elements or components and can be selected from a group consisting of two or more of the recited elements or components. Further, it should be understood that elements and/or features of a composition, an apparatus, or a method described herein can be combined in a variety of ways without departing from the scope of the present teachings, whether explicit or implicit herein.

The use of the terms "include," "includes," "including," "have," "has," or "having" should be generally understood as open-ended and non-limiting unless specifically stated otherwise.

The use of the singular herein includes the plural (and vice versa) unless specifically stated otherwise. Moreover, the singular forms "a," "an," and "the" include plural forms unless the context clearly dictates otherwise. In addition, where the use of the term "about" is before a quantitative value, the present teachings also include the specific quantitative value itself, unless specifically stated otherwise. As used herein, the term "about" refers to a ±<NUM>% variation from the nominal value. All numerical values and ranges disclosed herein are deemed to include "about" before each value.

Claim 1:
A method, comprising:
receiving (<NUM>), by one or more processors, a plurality of extraluminal images (<NUM>, <NUM>, <NUM>) of a target blood vessel;
detecting (<NUM>), by the one or more processors, a junction point (<NUM>) in the plurality of extraluminal images;
receiving (<NUM>), by the one or more processors, a first set of intravascular data taken during a first pullback of a first intravascular device;
determining, by the one or more processors, a first proximal end point (<NUM>) of the first pullback in one of the plurality of extraluminal images, or in a composite image of a plurality of the external images;
determining (<NUM>), by the one or more processors, a first distance (L1) between the junction point (<NUM>) and the first proximal end point (<NUM>) of the first pullback;
receiving (<NUM>), by the one or more processors, a second set of intravascular data taken during a second pullback of a second intravascular device;
determining, by the one or more processors, a second proximal end point (<NUM>) of the second pullback in one of the plurality of extraluminal images, or in a composite image of a plurality of the external images;
determining (<NUM>), by the one or more processors, a second distance (L2) between the junction point (<NUM>) and the second proximal end point (<NUM>) of the second pullback; and
aligning (<NUM>), by the one or more processors based on the first distance (L1) and the second distance (L2), a first representation (<NUM>) of the first set of intravascular data with a second representation (<NUM>) of the second set of intravascular data.