Patent Description:
One medical imaging technique is ultrasound imaging, which is a non-invasive medical imaging technique that uses sound waves, typically produced by piezoelectric transducers, to image a tissue in a patient. The ultrasound probe focuses the sound waves, typically producing an arc-shaped sound wave which travels into the body and is partially reflected from the layers between different tissues in the patient. The reflected sound wave is detected by the transducers and converted into electrical signals that can be processed by the ultrasound scanner to form an ultrasound image of the tissue.

Other medical imaging processes, such as mammography and tomography, rely primarily on x-ray radiation and are particularly useful tools for imaging breasts to screen for, or diagnose, cancer or other lesions with the breasts. Tomosynthesis, generally, produces a plurality of x-ray images, each of discrete layers or slices of the breast, through the entire thickness thereof. In contrast to typical two-dimensional (2D) mammography systems, a tomosynthesis system acquires a series of x-ray projection images, each projection image obtained at a different angular displacement as the x-ray source moves along a path, such as a circular arc, over the breast.

It is with respect to these and other general considerations that the aspects disclosed herein have been made. Also, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving the specific problems identified in the background or elsewhere in this disclosure.

US patent application <CIT> discloses a method of evaluating an extent of intraductal disease in a breast of a subject. The method comprises identifying a target duct of the breast of a subject, wherein the target duct comprises or is likely to comprise at least one abnormality, accessing the target duct through a corresponding ductal orifice of the breast using at least one access device, delivering a volume of a fluid within the target duct and imaging the breast using an imaging device to visualize and evaluate the extent of intraductal disease of the target duct.

European patent application <CIT> discloses a method of providing multimodal medical images. The method includes steps of loading first image data and second image data obtained by two different imaging modalities; generating object modeling data based on at least one of the first image data and the second image data; and displaying, on a display, the first image data, the second image data, and the object modeling data. The object modeling data is a virtual model of a body tissue of an object.

US patent application <CIT> discloses a method and an apparatus for processing a plurality of ultrasound images of ductolobular systems in a breast. The operator scans each particular duct system in the breast and saves these axial scan segments. Further, the operator scans the ductolobular systems transaxially. The imaging system, interactively with the operator, aligns the axial scan segments of the duct. This is accomplished by electronically noting the radial direction of the axial scan or by identifying two like image points on respective, sequential axial scan segments.

Examples of the present disclosure describe systems and methods for mapping the ducts of a breast and localization of abnormalities through one or more medical imaging techniques.

In the present invention, the technology relates to a method for locating an abnormality within a breast. The method includes acquiring first imaging data for a breast from a first imaging modality, wherein the first imaging modality is at least one of an x-ray-based imaging modality or a magnetic resonance imaging (MRI) modality, and acquiring second imaging data for the breast from a second imaging modality, wherein the second imaging modality is at least one of an ultrasound imaging modality or a thermal imaging modality. The method further includes co-registering the first imaging data from the first imaging modality with the second imaging data from the second imaging modality, such that the first imaging data from the first imaging modality and the second imaging data from the second imaging modality share a common coordinate space; mapping, based on the second imaging data from the second imaging modality, a plurality of ducts within the breast to generate a mapping of the plurality of ducts; locating, from first imaging data, the abnormality in the breast; and concurrently displaying the mapping of the plurality of ducts and the located abnormality in the breast. In an example, the method further includes determining that the located abnormality is within one of the plurality of ducts based on the mapping of the plurality of ducts. In another example, the abnormality is a calcification. In yet another example, displaying the mapping of the plurality of ducts and the located abnormality in the breast includes displaying the abnormality as an overlay of a portion of the mapping of the plurality of ducts. In still another example, the mapping is a three-dimensional mapping. In still yet another example, the first imaging data is three-dimensional imaging data acquired from one of tomosynthesis, computed tomography, or MRI. In another example, the first imaging data is mammogram data and the second imaging data is ultrasound imaging data.

In an aspect not covered by the present invention, the technology relates to a method for imaging a breast. The method includes receiving ultrasound data for a breast scanned with an ultrasound probe; executing an image analysis technique to remove at least a portion of non-ductal tissue from the ultrasound data to generate ductal image data; generating, from the ductal image data, a mapping of the ducts of the breast in a three-dimensional volume; analyzing the mapping of the ducts to determine a statistical correlation between the mapping of the ducts and data for an aggregation of ductal structures for other breasts; and based on the determined statistical correlation, generating a risk assessment for the breast. In an example, the method further includes scanning the breast with the ultrasound probe to generate the ultrasound data; tracking the location of the ultrasound probe during scanning of the breast; and providing visual feedback regarding progress of the scanning. In another example, the risk assessment indicates whether additional diagnostic procedures should be performed for the breast. In yet another example, the image analysis technique comprises an artificial-intelligence technique. In still another example, the method further includes receiving x-ray imaging data for the breast; locating an abnormality in the x-ray imaging data for the breast; and displaying the abnormality in the x-ray imaging data concurrently with at least a portion of the mapping of the ducts. In still yet another example, the method further includes displaying the x-ray imaging data; receiving a selection of a region of interest in the x-ray imaging data; and based on receiving the selection of the region of interest, displaying a portion of the mapping of the ducts corresponding to the selected region of interest. In another example, the method further includes determining that the located abnormality is within one of the plurality of ducts based on the mapping of the ducts.

In another aspect not covered by the invention, the technology relates to a system for imaging ducts of a breast. The system includes a display; at least one processor operatively connected to the display; and memory, operatively connected to the at least one processor, storing instructions that when executed by the at least one processor cause the system to perform a set of operations. The set of operations includes receiving ultrasound data during a scan of the breast with an ultrasound probe; based on the ultrasound data, generating a three-dimensional mapping of the ducts of the breast; receiving x-ray imaging data for the breast; locating an abnormality in the x-ray imaging data for the breast; and displaying the abnormality in the x-ray imaging data concurrently with at least a portion of the three-dimensional mapping of the ducts. In an example, the operations further include determining that the located abnormality is within one of the ducts of the breast based on the three-dimensional mapping of the ducts. In another example, the operations further include tracking the location of the ultrasound probe during the scan of the breast; and providing visual feedback regarding progress of the scanning during the scan of the breast. In yet another example, the operations further include displaying the x-ray imaging data; receiving a selection of a region of interest in the x-ray imaging data; and based on receiving the selection of the region of interest, displaying a portion of the three-dimensional mapping of the ducts corresponding to the selected region of interest. In still another example, the operations further include analyzing the three-dimensional mapping of the ducts to determine a statistical correlation between the mapping of the ducts and data for an aggregation of ductal structures for other breasts; and based on the determined statistical correlation, generating a risk assessment for the breast. In still yet another example, the risk assessment indicates whether additional diagnostic tests should be performed for the breast.

In another aspect not covered by the present invention, the technology relates to a method for locating an abnormality within a breast. The method includes acquiring first imaging data for a breast from a first imaging modality, wherein the first imaging modality is at least one of an x-ray-based imaging modality or a magnetic resonance imaging (MRI) modality, and acquiring second imaging data for the breast from a second imaging modality, wherein the second imaging modality is at least one of an ultrasound imaging modality or a thermal imaging modality. The method also includes, based on the second imaging data from the second imaging modality, generating a model of the one or more structures within the breast to generate a mapping of the one or more structures; locating, from at least one of the first imaging data or the second imaging data, the abnormality in the breast; and based at least on the generated model of the one or more structures, determining a location of the abnormality relative to the modeled one or more structures within the breast.

In an example, the method further includes displaying at least a portion of a visual representation of the model concurrently with the abnormality. In another example, the one or more structures are breast ducts. In yet another example, the one or more structures are at least one of breast ducts, lobules, lymph nodes, vascular structures, or Cooper's ligaments. In a further example, determining the location of abnormality relative to modeled one or more structures within the breast includes determining whether the abnormality is within one of the one or more structures. In still another example the first imaging data is three-dimensional imaging data acquired from one of tomosynthesis, computed tomography, or MRI. In still yet another example, the first imaging data is mammogram data and the second imaging data is ultrasound imaging data. In another example, the method further comprises: co-registering the first imaging data from the first imaging modality with the second imaging data from the second imaging modality, such that the first imaging data from the first imaging modality and the second imaging data from the second imaging modality share a common coordinate space.

In another aspect not covered by the present invention, the technology relates to a method for imaging a breast. The method includes receiving ultrasound data for a breast scanned with an ultrasound probe; executing an image analysis technique to identify one or more anatomical structures of the breast; generating, from the identified one or more anatomical structures, a mapping of the one or more structures of the breast; analyzing the mapping of the one or more anatomical structures to determine a statistical correlation between the mapping of the one or more anatomical structures and data for an aggregation of mappings of the one or more anatomical structures for other breasts; and based on the determined statistical correlation, generating a risk assessment for the breast.

In an example not covered by the present invention, the method further includes scanning the breast with the ultrasound probe to generate the ultrasound data; tracking the location of the ultrasound probe during scanning of the breast; and providing visual feedback regarding progress of the scanning. In another example, the risk assessment indicates whether additional diagnostic procedures should be performed for the breast. In yet another example, the image analysis technique comprises an artificial-intelligence technique. In still another example, the one or more anatomical structures are breast ducts. In still yet another example, the method further includes extracting from the generated mapping, quantitative values at least one of the number of ducts, a regularity pattern for the ducts, or a termination regularity for the ducts; and wherein the statistical correlation is based on the extracted quantitative values.

In another example, the one or more anatomical structures are at least one of breast ducts, lobules, lymph nodes, vascular structures, or Cooper's ligaments. In a further example, the ultrasound data is 3D ultrasound data for the whole breast.

In another aspect not covered by the present invention, the technology relates to a system for imaging ducts of a breast. The system includes at least one processor; and memory, operatively connected to the at least one processor, storing instructions that when executed by the at least one processor cause the system to perform a set of operations. The set of operations include receiving ultrasound data for a breast scanned with an ultrasound probe; executing an image analysis technique to identify one or more anatomical structures of the breast; generating, from the identified one or more anatomical structures, a mapping of the one or more anatomical structures of the breast; extracting at least one feature from the mapping of the one or more anatomical structures; comparing the extracted at least one feature to a threshold value; and based on the comparison of the extracted at least one feature to the threshold value, generating a risk assessment for the breast.

In an example, the threshold is based on an aggregate of mapping for the one or more anatomical structures. In another example, the one or more anatomical structures are at least one of breast ducts, lobules, lymph nodes, vascular structures, or Cooper's ligaments. In yet another example, the extracted at least one feature is represented by a quantitative value.

Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.

Detection and localization of abnormalities within a breast may be an important part in diagnosing a type of abnormality, or in some examples, a type of cancer. For instance, the location of a lesion or a calcification in relation other structures of the breast may provide additional information that may be useful for diagnostics. The relative location of abnormalities, such as lesions or calcifications, to structures such as breast ducts lobules, Cooper's ligaments, dense tissue, fat, skin, vascular structures, and/or lymph nodes may all provide additional diagnostic information. As an example, whether an abnormality is located in a duct of a breast may be informative as to what type of cancer the abnormality may correspond. In particular, the location of the abnormality relative to the breast ducts in useful in the classification as ductal carcinoma in situ (DCIS). DCIS is a non-invasive cancer where abnormal cells are found in a duct of the breast. If the abnormal cells are confined within the duct, the cancer is generally very treatable by a variety of treatment options. In contrast, if abnormal cells are located outside of the breast ducts, the cancer is likely to be more invasive and spread more quickly. Currently, DCIS is often diagnosed based on a pattern of abnormalities displaying as bright dots within a mammogram. Depending on the shape or pattern of the dots, a prediction is made as to whether the patient has DCIS. There is no determination, however, as to whether the abnormalities are actually confined to a breast duct. As such, it would be beneficial to be able to identify through non-invasive medical imaging whether an abnormality is located inside or outside of a breast duct.

Current medical imaging systems are limited in their ability to provide such an indication or relationship between abnormalities and other breast structures. For example, while x-ray imaging systems are generally effective for identifying some abnormalities (such as calcifications), the identification of other structures (such as breast ducts) through x-ray imaging is difficult. In contrast, ultrasound imaging systems are generally effective at identifying tissue such as ducts, but may not be as effective at identifying abnormalities. X-ray based imaging may also be somewhat limited in dense tissue, whereas ultrasound imaging often performs well in dense tissue. To leverage the benefits of both imaging modalities, the present technology provides for combining x-ray imaging data with ultrasound imaging data to provide an indication or determinations regarding the location of abnormalities in relation to other structures or features of the breast. For instance, the present technology may be used to provide an indication or determination as to whether abnormalities are located inside or outside the ducts of the breast. For example, a tomosynthesis system may be used to image a breast of a patient and an ultrasound system may also be used to image the breast. The imaging data from the tomosynthesis system may be co-registered with the imaging data from the ultrasound imaging system, such that a location in the tomosynthesis imaging data may be correlated with imaging data from the ultrasound imaging system. The structures of the breast may be also be mapped to form a 3D mapping of the structures of the breast. For example, the ducts of the breast may be mapped so as to form a 3D mapping of the ducts in the breast. An abnormality may be located or identified in the x-ray imaging data. The abnormality may then be overlaid, visually or mathematically, on the mapping of the ducts to determine whether the abnormality lies inside or outside one the structures, such as a duct.

In addition, the mapping of the breast structures, such as ducts, may also be used to determine a risk factor for different types of cancers or other conditions. Particular patterns and configurations of structures within a breast may be indicative of a higher risk for invasive cancers, whereas other patterns and configurations of structures may indicate a lower risk for such invasive cancers. Accordingly, the present technology may analyze the 3D mapping of the ducts to determine a statistical correlation between the mapping of structures and data for an aggregation of the same type of structures from other breasts. Based on the determined statistical correlation, a risk assessment for the analyzed breast may be determined. If the risk is considered high, additional procedures may be recommended for the patient to determine if any cancerous cells are present in the breast.

In describing examples and embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this patent specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that operate in a similar manner.

<FIG> depicts an example medical imaging system <NUM> for breast mapping and abnormality localization. System <NUM> includes an x-ray image acquisition system <NUM>, a tracking system <NUM>, an ultrasound imaging system <NUM>, a navigation system <NUM> and a display system <NUM>, all representatively connected via communication network <NUM>. It should be noted that, although the 'systems' are shown in <FIG> as functional blocks, different systems may be integrated into a common device, and the communication link may be coupled between fewer than all of the systems; for example, the tracking system <NUM>, navigation system <NUM> and display system <NUM> may be included in an acquisition work station or a technologist work station which may control the acquisition of the x-ray images in a radiology suite. Alternatively, the navigation system <NUM> and tracking system <NUM> may be integrated into the ultrasound system <NUM>, or provided as standalone modules with separate communication links to the display <NUM>, x-ray acquisition system <NUM> and ultrasound system <NUM>. Similarly, skilled persons will additionally appreciate that communication network <NUM> can be a local area network, wide area network, wireless network, internet, intranet, or other similar communication network.

In one embodiment, x-ray image acquisition system <NUM> is a tomosynthesis acquisition system which captures a set of projection images of a patient's breast as an x-ray tube scans across a path over the breast. The set of projection images is subsequently reconstructed to a three-dimensional volume which may be viewed as slices or slabs along any plane. The three-dimensional volume may be stored locally on x-ray imaging system <NUM> or in some embodiments in a database or other storage means. Additional details regarding an example x-ray image acquisition system are depicts in <FIG>.

X-ray imaging system <NUM> may transmit the three-dimensional x-ray image volume to navigation system <NUM> via communication network <NUM>, where such x-ray image can be stored and viewed. Skilled persons will understand that the x-ray image of a patient can, in alternative embodiments, be stored locally on x-ray imaging system <NUM> and accessed remotely by navigation system <NUM> via communications network <NUM>, and in other embodiments can be stored on a server in communication with navigation system <NUM> via communications network <NUM>. Navigation system <NUM> displays the x-ray image obtained by x-ray imaging system and once reconstructed for display on navigation system <NUM> the x-ray image can be reformatted and repositioned to view the image at any plane and any slice position or orientation. In some embodiments navigation system <NUM> displays multiple frames or windows on the same screen showing alternative positions or orientations of the x-ray-image slice.

Skilled persons will understand that the x-ray image volume obtained by x-ray imaging system <NUM> can be transmitted to navigation system <NUM> at any point in time and is not necessarily transmitted immediately after obtaining the x-ray image volume, but instead can be transmitted on the request of navigation system <NUM>. In alternative embodiments, the x-ray image volume is transmitted to navigation system <NUM> by a transportable media device, such as a flash drive, CD-ROM, diskette, or other such transportable media device.

Ultrasound imaging system <NUM> obtains an ultrasound image of a tissue of a patient, typically using an ultrasound probe, which is used to image a portion of a tissue of a patient within the field of view of the ultrasound probe. For instance, the ultrasound imaging system <NUM> may be used to image a breast, and more specifically, structures such as the ducts of a breast. Ultrasound imaging system <NUM> obtains and displays an ultrasound image of a patient's anatomy within the field of view of the ultrasound probe and typically displays the image in real-time as the patient is being imaged. In some embodiments, the ultrasound image can additionally be stored on a storage medium, such as a hard drive, CD-ROM, flash drive or diskette, for reconstruction or playback at a later time. Additional details regarding the ultrasound imaging system are depicted in <FIG>.

In some embodiments, navigation system <NUM> can access the ultrasound image, and in such embodiments ultrasound imaging system <NUM> is further connected to communication network <NUM> and a copy of the ultrasound image obtained by ultrasound imaging system <NUM> can be transmitted to navigation system <NUM> via communication network <NUM>. In other embodiments, navigation system <NUM> can remotely access and copy the ultrasound image via communication network <NUM>, and in alternative embodiments, a copy of the ultrasound image can be stored on a server in communication with navigation system <NUM> via communications network <NUM> and accessed remotely by navigation system <NUM>.

Tracking system <NUM> is in communication with navigation system <NUM> via communications network <NUM> and may track the physical position in which ultrasound imaging system <NUM> is imaging the tissue of the patient. In some embodiments, tracking system <NUM> can be connected directly to navigation system <NUM> via a direct communication link or wireless communication link. Tracking system <NUM> tracks the position of transmitters connected to ultrasound imaging system <NUM> and provides navigation system <NUM> with data representing their coordinates in a tracker coordinate space. In some embodiments, tracking system may be an optical tracking system comprising an optical camera and optical transmitters, however skilled persons will understand that any device or system capable of tracking the position of an object in space can be used. For example, skilled persons will understand that in some embodiments an RF tracking system can be used, comprising an RF receiver and RF transmitters.

Ultrasound imaging system <NUM> may be configured for use with navigation system <NUM> by a calibration process using tracking system <NUM>. Transmitters that are connected to the ultrasound probe of ultrasound imaging system <NUM> may transmit their position to tracking system <NUM> in the tracker coordinate space, which in turn provides this information to navigation system <NUM>. For example, transmitters may be positioned on the probe of ultrasound imaging system <NUM> so that tracking system <NUM> can monitor the position and orientation of the ultrasound probe and provide this information to navigation system <NUM> in the tracker coordinate space. Navigation system <NUM> may use this tracked position to determine the position and orientation of the ultrasound probe, relative to the tracked position of the transmitters.

In some examples, configuration occurs using a configuration tool. In such example, the position and orientation of the configuration tool may be additionally tracked by tracking system <NUM>. During configuration the configuration tool contacts the transducer face of the ultrasound probe of ultrasound imaging system <NUM> and tracking system <NUM> transmits information representing the position and orientation of the configuration tool in the tracker coordinate space to navigation system <NUM>. Navigation system <NUM> may determine a configuration matrix that can be used to determine the position and orientation of the field of view of the ultrasound probe in the tracker coordinate space, based on the tracked position of the transmitters connected to the ultrasound probe. In alternative embodiments, a database having configuration data of a plurality of brands or models of various ultrasound probes can be used to pre-load a field of view configuration into navigation system <NUM> during configuration.

Once ultrasound imaging system <NUM> is configured with navigation system <NUM>, the tissue of a patient can be imaged with ultrasound imaging system <NUM>. During ultrasound imaging, tracking system <NUM> monitors the position and orientation of the ultrasound probe of ultrasound imaging system <NUM> and provides this information in the tracker coordinate space to navigation system <NUM>. Since ultrasound imaging system <NUM> has been configured for use with navigation system <NUM>, navigation system <NUM> is able to determine position and orientation of the field of view of the ultrasound probe of ultrasound imaging system <NUM>.

Navigation system <NUM> can be configured to co-register an ultrasound image with an x-ray image. In some embodiments, navigation system <NUM> can be configured to transform the position and orientation of the field of view of the ultrasound probe from the tracker coordinate space to a position and orientation in the x-ray image, for example, to x-ray system coordinates. This can be accomplished by tracking the position and orientation of the ultrasound probe and transmitting this positional information in the tracker coordinate space to navigation system <NUM> and relating this positional information to the x-ray coordinate system. For example, in some embodiments, a user can select an anatomical plane within the x-ray image, and the user can then manipulate the position and orientation of a tracked ultrasound probe to align the field of view of the ultrasound probe with the selected anatomical plane. Once alignment is achieved, the associated tracker space coordinates of the ultrasound image can be captured. Registration of the anatomic axes (superior-inferior (SI), left-right (LR) and anterior-posterior (AP)) between the x-ray image and the tracker coordinate space can be determined from the relative rotational differences between the tracked ultrasound field of view orientation and the selected anatomical plane using techniques known to those of skill in the art.

This configuration may further include the selection of landmark within the x-ray image, for example, using an interface permitting a user to select an anatomical target. In some embodiments, the landmark can be an internal tissue landmark, such as veins or arteries, and in other embodiments, the landmark can be an external landmark, such as a fiducial skin marker or external landmark, such as a nipple. The same landmark selected in the x-ray image can be located with the ultrasound probe, and upon location, a mechanism can be provided for capturing coordinates of the representation of the target in the tracker coordinate space. The relative differences between the coordinates of the target in the x-ray image and the coordinates of the target in the tracker coordinate space are used to determine the translational parameters required to align the two co-ordinate spaces. The plane orientation information acquired previously can be combined with the translation parameters to provide a complete <NUM>×<NUM> transformation matrix capable of co-registering the two coordinate spaces.

Navigation system <NUM> can then use the transformation matrix to reformat the x-ray image being displayed so that the slice of tissue being displayed is in the same plane and in the same orientation as the field of view of the ultrasound probe of ultrasound imaging system <NUM>. Matched ultrasound and x-ray images may then be displayed side by side, or directly overlaid in a single image viewing frame. In some embodiments, navigation system <NUM> can display additional x-ray images in separate frames or positions on a display screen. For example, the x-ray image can be displayed with a graphical representation of the field of view of ultrasound imaging system <NUM> wherein the graphical representation of the field of view is shown slicing through a 3D representation of the x-ray image. In other embodiments annotations can be additionally displayed, these annotations representing, for example, the position of instruments imaged by ultrasound imaging system <NUM>, such as biopsy needles, guidance wires, imaging probes or other similar devices.

In other embodiments, the ultrasound image being displayed by ultrasound imaging system <NUM> can be superimposed on the slice of the x-ray image being displayed by navigation system <NUM> so that a user can view both the x-ray and ultrasound images simultaneously, overlaid on the same display. In some embodiments, navigation system <NUM> can enhance certain aspects of the super imposed ultrasound or x-ray images to increase the quality of the resulting combined image.

An exemplary method and system which may be used to navigate between a three dimensional image data set and an ultrasound feed, and to align coordinate systems to enable display of common reference points is described in further detail in <CIT>, titled "System and Method for Fusing Three Dimensional Image Data from a Plurality of Different Imaging Systems for Use in Diagnostic Imaging,". Additional details may also be found in <CIT>, titled "Systems and methods for tracking positions between imaging modalities and transforming a displayed three-dimensional image corresponding to a position and orientation of a probe,". In addition, while the system <NUM> is generally described as having an x-ray image acquisition system <NUM>, in some examples the system <NUM> may have a magnetic resonance imaging (MRI) system in place of, or in addition to, the x-ray image acquisition system <NUM>. Further, while the system <NUM> is generally described as having an ultrasound imaging system <NUM>, in some examples the system <NUM> may have an optical and/or thermal imaging system in place of, or in addition to, the ultrasound imaging system <NUM>. In some examples, the optical and/or thermal imaging system is incorporated in to the x-ray image acquisition system <NUM>.

<FIG> and <FIG> illustrate portions of a non-limiting example of a multi-mode breast x-ray imaging system operable in a CT mode but also configured to selectively operate in a tomosynthesis mode including a wide angle tomosynthesis mode and a narrow angle tomosynthesis mode, and in a mammography mode. For clarity of illustration, a patient shield for use in the CT mode is omitted from <FIG> and <FIG> but examples are illustrated in <FIG> and <FIG>. A support column <NUM> is secured to a floor and houses a motorized mechanism for raising and lowering a horizontally extending axle <NUM>, which protrudes through an opening 100a in column <NUM>, and for rotating axle <NUM> about its central axis. Axle <NUM> in turn supports a coaxial axle 102a that can rotate with or independently of axle <NUM>. Axle <NUM> supports a breast immobilization unit comprising an upper plate 104a and a lower plate 104b such that each plate can move up and down along the long dimension of support <NUM> together with axles <NUM> and 102a, at least one of the plates can move toward the other, and unit <NUM> can rotate about the common central axis of axles <NUM> and 102a. In addition, axle <NUM> supports a gantry <NUM> for two types of motorized movement: rotation about the central axis of axle <NUM>, and motion relative to axle <NUM> along the length of gantry <NUM>. Gantry <NUM> carries at one end an x-ray source such as a shrouded x-ray tube generally indicated at <NUM>, and at the other end a receptor housing <NUM> enclosing an imaging x-ray detector or receptor <NUM>.

When operating in a CT mode, the system of <FIG> and <FIG> immobilizes a patient's breast between plates 104a and 104b. To this end, unit <NUM> is raised or lowered together with axle <NUM> to the height of the breast while the patient is upright, e.g., standing or sitting. The patient leans toward unit <NUM> from the left side of the system as seen in <FIG>, and a health professional, typically an x-ray technician, adjusts the breast between plates 104a and 104b while pulling tissue to the right in <FIG> and moving at least one of plates 104a and 104b toward the other to immobilize the breast and keep it in place, preferably with as much as practicable of the breast tissue being inside unit <NUM>. In the course of taking x-ray measurements representing real projection x-ray images, from which to reconstruct images of respective breast slices, gantry <NUM> rotates about the central axis of axle <NUM> while the breast remains immobilized in unit <NUM>. Imaging receptor <NUM> inside housing <NUM> remains fixed relative to x-ray tube <NUM> during the rotation of gantry <NUM>. A pyramid shaped beam of x-rays from tube <NUM> traverses the breast immobilized in unit <NUM> and impinges on imaging receptor <NUM>, which in response generates a respective two-dimensional array of pixel values related to the amount of x-ray energy received for each increment of rotation at respective pixel positions in an imaging plane of the receptor. These arrays of pixel values for real projection images are delivered to and processed by a computer system to reconstruct slice images of the breast. Gantry <NUM> may be configured for motorized movement toward column <NUM>, to facilitate the x-ray technician's access to the patient's breast for positioning the breast in unit <NUM>, and away from column <NUM> to ensure that x-ray tube <NUM> and imaging receptor <NUM> inside housing <NUM> can image the appropriate breast tissue. Alternatively, gantry <NUM> can maintain a fixed distance from column <NUM>, to the left of the position seen in <FIG>, so that the imaging x-ray beam can pass through as much as practical of the breast immobilized in unit <NUM>, in which case there would be no need for a mechanism to vary that distance.

A unique challenge arises because of the upright position of the patient and the rotation of x-ray tube <NUM> and receptor housing <NUM> through a large angle in the CT mode of operation. As known, CT scanning typically involves a rotation of the source and receptor through an angle of <NUM>° plus the angle subtended by the imaging x-ray beam, and preferably a rotation through a greater angle, e.g., <NUM>°. However, if the rotation includes the <NUM>° position of x-ray source <NUM> as seen in <FIG> and <FIG>, the patient's head may be too close to x-ray source <NUM>. Collision of rotating assemblies with the patient, and concern with such collision, can be avoided by the use of a shield separating the patient from assemblies rotating even the full <NUM>, as discussed below in this patent specification, although depending on the design of the shield and the rotating assemblies in particular embodiments this may require the patient to arch her body such that both her head and legs are away from the system, to the left as seen in <FIG>. An alternative, also discussed below, is to exclude from the rotation a sector or segment around the position of x-ray source <NUM> seen in <FIG> and <FIG>. As a non-limiting example, if the position of x-ray tube <NUM> seen in <FIG> and <FIG> is designated the <NUM>° position, then the rotation for CT imaging excludes positions of x-ray source <NUM> in the <NUM>° sector or segment between <NUM>° and <NUM>°, or in the <NUM>° sector or segment between <NUM>° and <NUM>°, or in some other sector or segment that is sufficient to clear the patient's head position while taking x-ray CT data over a sufficient angle of rotation for the reconstruction of high quality slice images. While the rotation of x-ray tube <NUM> and receptor housing <NUM> still has to clear the lower part of the patient's body, it is generally easier for a patient to keep the lower part of her body away from the rotating components, to the left as seen in <FIG> (and preferably behind a shield), than to arch back her head and shoulders.

An example of such a shield is illustrated in <FIG> and <FIG>. <FIG> is a side elevation that is otherwise the same as <FIG> but additionally illustrates a patient shield <NUM> having a central opening 114c. Shield <NUM> may be completely circular in front elevation, as illustrated by the circle that includes an arc in broken line in <FIG>, in front elevation. In that case, gantry <NUM> can rotate through a complete circle in the CT mode. As an alternative, shield <NUM> can leave open a sector or segment 114a illustrated in <FIG> as the area below the broken line arc and between the solids line of shield <NUM>. In that case, gantry <NUM> can rotate in the CT mode only through an angle that is less than <NUM>°, but the patient can have space for her head and perhaps a shoulder and an arm in the V-shaped cutout 114b of shield <NUM>, for a more comfortable body posture. Specifically, as illustrated in <FIG>, gantry <NUM> can rotate only within the portion of shield <NUM> that is outside V-shaped cutout 114b. One of the possible positions of gantry <NUM> and tube <NUM> and receptor housing <NUM> is shown in solid lines. Another possible position is shown in broken lines, and designated as gantry <NUM>', carrying x-ray source <NUM>' and receptor housing <NUM>'. <FIG> illustrates a possible shape of shield <NUM> in side elevation.

Use of the system in a tomosynthesis mode is illustrated in <FIG> and <FIG>, which are otherwise the same as <FIG> and <FIG> respectively, except that gantry <NUM> is in a different position relative to breast immobilization unit <NUM> and axle <NUM> and column <NUM>, and no shield <NUM> is shown. In particular, x-ray source <NUM> is further from unit <NUM> and column <NUM>, and receptor housing <NUM> is closer to unit <NUM>. In the tomosynthesis mode, the patient's breast also is immobilized between plates 104a and 104b, which remain in place during imaging. In one example, x-ray tube <NUM> and receptor housing <NUM> may undergo a rotation about the immobilized breast that is similar to that in the CT mode operation but is through a smaller angle. A respective two-dimensional projection image Tp taken for each increment of rotation while x-ray tube <NUM> and imaging receptor <NUM> inside housing <NUM> rotate as a unit, fixed with respect to each other, as in the CT mode or as illustrated in principle in commonly assigned <CIT>. Alternatively, the motions of x-ray tube <NUM> and receptor <NUM> relative to the immobilized breast can be as in said system offered under the trade name Selenia® Dimensions® of the common assignee, certain aspect of which are described in commonly owned <CIT>. In this alternative case, x-ray tube rotates about the central axis of axle <NUM>, but receptor housing <NUM> remains in place while imaging receptor <NUM> rotates or pivots inside housing <NUM> about an axis that typically passes through the image plane of the receptor, is parallel to the central axis of axle <NUM>, and bisects imaging receptor <NUM>. The rotation or pivoting of receptor <NUM> typically is through a smaller angle than the rotation angle of x-ray tube <NUM>, calculated so that a normal to the imaging plane of receptor <NUM> can continue pointing at or close to the focal spot in x-ray tube <NUM> from which the imaging x-ray beam is emitted, and so that the beam continues to illuminate all or most of the imaging surface of receptor <NUM>.

In one example of tomosynthesis mode operation, x-ray tube <NUM> rotates through an arc of about ±<NUM>° while imaging receptor rotates or pivots through about ±<NUM>° about the horizontal axis that bisects its imaging surface. During this motion, plural projection images RP are taken, such as <NUM> or <NUM> images, at regular increments of rotation angle. The central angle of the ±<NUM>° arc of x-ray source <NUM> rotation can be the <NUM>° angle, i.e., the position of the x-ray source <NUM> seen in FIGS. <NUM> and <NUM>, or some other angle, e.g., the angle for the x-ray source position typical for MLO imaging in conventional mammography. In the tomosynthesis mode, the breast may be immobilized in unit <NUM> but, alternatively, lower plate 104b may be removed so that the breast is supported between the upper surface of receptor housing <NUM> and upper plate 104a, in a manner analogous to the way the breast is immobilized in said system offered under the trade name Selenia®. In the tomosynthesis mode, greater degree of breast compression can be used under operator control than in the CT mode. The same concave plates 104a and 104b can be used, or generally flat plates can be substituted, or a single compression paddle can be used while the breast is supported by the upper surface of receptor housing <NUM>, as used in said system offered under the Selenia® trade name.

When operating in a tomosynthesis mode, the system of <FIG> and <FIG> provides multiple choices of that mode, selectable by an operator, for example a narrow angle mode and a wide angle mode. In the narrow angle tomosynthesis mode, x-ray source <NUM> rotates around unit <NUM> and the patient's breast immobilized therein through an angle such as ±<NUM>°, while in the wide angle tomosynthesis mode x-ray tube <NUM> rotates through an angle such as in the range of about ±<NUM>° to ±<NUM>°. The wide angle mode may involve taking the same number of projection images RP as the narrow angle mode, or a greater number. As a non-limiting example, if the narrow angle mode involves taking a total or <NUM> or <NUM> tomosynthesis projection images RP as x-ray source <NUM> moves through its arc around the breast, the wide angle mode may involve taking the same number of images RP or a greater number, such as <NUM> or <NUM> or some other number, typically at regular angular increments. The examples of angles of rotation of x-ray source <NUM> are not limiting. The important point is to provide multiple modes of tomosynthesis operations, where one mode involves x-ray source rotation through a greater angle around the breast than another tomosynthesis mode. Additional details regarding the structure and operation of image system of <FIG> are provided in <CIT>. The methods and systems described herein may be implemented in digital breast tomosynthesis (DBT) procedures as well as multi-modality imaging (MMI) procedures. MMI procedures generally refers to the use of a combination of different imaging modes or techniques, such as DBT acquisitions with varying dosage levels and/or angular coverage, computerized tomography (CT) of a compressed breast, and/or a combination of the two.

In some examples, the system <NUM> may also include one or more optical and/or thermal imaging devices, such as digital cameras. The optical and/or thermal imaging devices may be mounted or incorporated in the gantry <NUM>. In such examples, the optical and/or thermal imaging devices may be mounted or incorporated near, or proximate to, the x-ray tube <NUM>. By incorporating the optical and/or thermal imaging devices into the gantry <NUM>, optical and thermal imaging data of the breast may be captured. The optical and thermal imaging data of the breast may be captured concurrently with the capture of the tomosynthesis and/or mammogram images. A map of the structures, such as ducts, of the breast and, in some examples, a vascular map of the breast may be generated from the optical and/or thermal imaging data. The optical and/or thermal imaging data may also be used to map the structures of the breast in combination with, or as a substitute for, ultrasound imaging data. The optical and/or thermal imaging data may also be co-registered with the x-ray data captured by the system <NUM>. In some examples, the co-registration of the optical and/or thermal imaging data with the x-ray data is simplified due to the optical and/or thermal imaging devices being attached to the gantry <NUM> near the x-ray tube <NUM>. In such examples, the optical and/or thermal imaging devices move with the x-ray tube <NUM>.

<FIG> depicts an example of an ultrasound imaging system <NUM>. The ultrasound localization system <NUM> includes an ultrasound probe <NUM> that includes an ultrasonic transducer <NUM>. The ultrasonic transducer <NUM> is configured to emit an array of ultrasonic sound waves <NUM>. The ultrasonic transducer <NUM> converts an electrical signal into ultrasonic sound waves <NUM>. The ultrasonic transducer <NUM> may also be configured to detect ultrasonic sound waves, such as ultrasonic sound waves that have been reflected from internal portions of a patient, such as ducts within a breast. In some examples, the ultrasonic transducer <NUM> may incorporate a capacitive transducer and/or a piezoelectric transducer, as well as other suitable transducing technology.

The ultrasonic transducer <NUM> is also operatively connected (e.g., wired or wirelessly) to a display <NUM>. The display <NUM> may be a part of a computing system, including processors and memory configured to produce and analyze ultrasound images. Further discussion of a suitable computing system is provided below with reference to FIG. The display <NUM> is configured to display ultrasound images based on an ultrasound imaging of a patient. The ultrasound imaging performed in the ultrasound localization system <NUM> is primarily B-mode imaging, which results in a two-dimensional ultrasound image of a cross-section of a portion of the interior of a patient. The brightness of the pixels in the resultant image generally corresponds to amplitude or strength of the reflected ultrasound waves. Other ultrasound imaging modes may also be utilized. For example, the ultrasound probe may operate in a 3D ultrasound mode that acquires ultrasound image data from a plurality of angles relative to the breast to build a 3D model of the breast. In some examples, ultrasound images may not be displayed during the acquisition process. Rather, the ultrasound data is acquired and a 3D model of the breast is generated without B-mode images being displayed.

The ultrasound probe <NUM> may also include a probe localization transceiver <NUM>. The probe localization transceiver <NUM> is a transceiver that emits a signal providing localization information for the ultrasound probe <NUM>. The probe localization transceiver <NUM> may include a radio frequency identification (RFID) chip or device for sending and receiving information as well as accelerometers, gyroscopic devices, or other sensors that are able to provide orientation information. For instance, the signal emitted by the probe localization transceiver <NUM> may be processed to determine the orientation or location of the ultrasound probe <NUM>. The orientation and location of the ultrasound probe <NUM> may be determined or provided in three-dimensional components, such as Cartesian coordinates or spherical coordinates. The orientation and location of the ultrasound probe <NUM> may also be determined or provided relative to other items, such as an incision instrument, a marker, a magnetic direction, a normal to gravity, etc. With the orientation and location of the ultrasound probe <NUM>, additional information can be generated and provided to the surgeon to assist in guiding the surgeon to a lesion within the patient, as described further below. While the term transceiver is used herein, the term is intended to cover both transmitters, receivers, and transceivers, along with any combination thereof. Additional details of examples of systems and components for localization and co-registration of an ultrasound probe are provided in <CIT>, titled "System and Method for Fusing Three Dimensional Image Data from a Plurality of Different Imaging Systems for Use in Diagnostic Imaging".

<FIG> depicts an example of the ultrasound imaging system <NUM> in use with breast <NUM> of a patient. The ultrasound probe <NUM> is in contact with a portion of the breast <NUM>. In the position depicted in <FIG>, the ultrasound probe <NUM> is being used to image a structure of the breast. In the example depicted, the ultrasound probe <NUM> is being used to image a duct <NUM> of the breast <NUM>. To image duct <NUM>, the ultrasonic transducer <NUM> emits an array of ultrasonic sound waves <NUM> into the interior of the breast <NUM>. A portion of the ultrasonic sound waves <NUM> are reflected off internal components of the breast, such as the duct <NUM> when the duct is in the field of view, and return to the ultrasound probe <NUM> as reflected ultrasonic sound waves <NUM>. The reflected ultrasonic sound waves <NUM> may be detected by the ultrasonic transducer <NUM>. For instance, the ultrasonic transducer <NUM> receives the reflected ultrasonic sound waves <NUM> and converts the reflected ultrasonic sound waves <NUM> into an electric signal that can be processed and analyzed to generate ultrasound image data on display <NUM>. The depth of the duct <NUM> or other objects in an imaging plane may be determined from the time between a pulse of ultrasonic waves <NUM> being emitted from the ultrasound prove <NUM> and the reflected ultrasonic waves <NUM> being detected by the ultrasonic probe <NUM>. For instance, the speed of sound is well-known and the effects of the speed of sound based on soft tissue are also determinable. Accordingly, based on the time of flight of the ultrasonic waves <NUM> (more specifically, half the time of flight), the depth of the object within an ultrasound image may be determined. Other corrections or methods for determining object depth, such as compensating for refraction and variant speed of waves through tissue, may also be implemented. Those having skill in the art will understand further details of depth measurements in medical ultrasound imaging technology. Such depth measurements and determinations may be used to build a 3D model of the breast <NUM>, and more specifically, a 3D model of the ducts <NUM> of the breast <NUM>. For instance, a whole breast <NUM> may be imaged with the ultrasound probe <NUM>. By imaging the whole breast <NUM> with 3D ultrasound techniques, 3D models of different structures, such as ducts <NUM>, may be generated.

In addition, multiple frequencies or modes of ultrasound techniques may be utilized. For instance, real time and concurrent transmit and receive multiplexing of localization frequencies as well as imaging frequencies and capture frequencies may be implemented. Utilization of these capabilities provide information to co-register or fuse multiple data sets from the ultrasound techniques to allow for visualization of ducts <NUM> and other medical images on the display <NUM>. The imaging frequencies and capture sequences may include B-mode imaging (with or without compounding), Doppler modes (e.g., color, duplex), harmonic mode, shearwave and other elastography modes, and contrast-enhanced ultrasound, among other imaging modes and techniques.

<FIG> depicts an example method <NUM> for locating an abnormality within a breast. At operation <NUM>, first imaging data for a breast from a first imaging modality is acquired or received, which may be an x-ray-based imaging modality and/or a magnetic resonance imaging (MRI) modality. The first imaging data may be two-dimensional imaging data or three-dimensional imaging data. In some examples, the first imaging data may be acquired from a tomosynthesis imaging system and/or a computed tomography system. In such an example, the first imaging data may be 3D imaging data. In other examples, the first imaging data may be 2D imaging data, such as mammography imaging data. At operation <NUM>, second imaging data is acquired for the breast from a second imaging modality. The second imaging modality may be an ultrasound imaging modality. In other examples, the second imaging modality may be an optical and/or thermal imaging modality. In some examples, the second imaging modality may include both the ultrasound imaging modality and the optical and/or thermal imaging modality. The first imaging data and the second imaging data may then be co-registered at operation <NUM>, such that the first imaging data from the first imaging modality and the second image data from the second imaging modality share a common coordinate space. Co-registering the imaging data from the different modalities at operation <NUM> may be accomplished through any of the means discussed above.

At operation <NUM>, a plurality of ducts for the breast are mapped to generate a mapping of the plurality of ducts. The mapping of the breast ducts may be generated from the second imaging data, such as ultrasound imaging data, acquired at operation <NUM>. The mapping of the breast ducts may be 3D mapping or a plurality of 2D mappings of the breast ducts and may include a mapping of all the ducts in the breast. In some instances, the mapping of the breast ducts may achieved through artificial-intelligence image analysis techniques. Such image analysis techniques may analyze the second image data to identify ductal structures within the image data for the imaged breast. In some examples, the non-ductal tissue (i.e., the tissue of the breast other than the ducts) may be removed from the imaging data to form the mapping of the breast ducts. The image analysis techniques may be trained using a dataset of image data where ductal structures have been previously identified, such as by manual identification. Once the image analysis techniques have been trained, the image analysis techniques are able to identify ducts within image data. Once the ducts are identified, the mapping of the breast ducts is generated.

At operation <NUM>, an abnormality is located or identified in the first imaging data from the first modality that was received or acquired in operation <NUM>.

In examples where the first imaging data is x-ray data in the form of tomography or mammography image data, the abnormality may appear brighter in the image data, such as when the abnormality is a calcification. For example, pixels in the image data having a higher value (i.e., brighter) may correspond to an abnormality. The abnormality may be identified through the use of image analysis techniques that analyze the image data based on pixel values or the patterns of pixels. The image analysis techniques may be performed in the spatial, transform, or frequency domains. For example, image analysis techniques in the spatial domain generally operate based on the pixel values in the imaging data. Image analysis techniques within the transform or frequency domains generally operate based on mathematical transforms, such as a Fourier or Laplace transform, of the pixel data from imaging data. For instance, in the frequency domain, the image analysis techniques may be based on a rate of change of pixel values within the spatial domain. In other examples, an abnormality may be identified through the assistance of a medical professional. For instance, locating the abnormality may include the medical professional selecting the abnormality on a screen displaying the imaging data.

At operation <NUM>, at least a portion of the mapping of the breast ducts generated in operation <NUM> is displayed concurrently with in the abnormality that was located in operation <NUM>. The concurrent display of the mapping of the breast ducts and the abnormality allows for a determination as to whether the abnormality is located inside or outside of one of the breast ducts. For example, the abnormality may be displayed as an overlay of a portion of the mapping of the plurality of ducts. Due to the first imaging data being co-registered with the second imaging data, an abnormality that is located in the first imaging data may be displayed in a mapping of the breast ducts in the proper location even though the mapping of the breast ducts was generated from the second imaging data. In some examples, imaging data displaying the abnormality maybe displayed, and a selection of a region of interest, such as a region containing the abnormality, may be selected from the displayed imaging data. Based on receiving the selection of the region of interest, a portion of the mapping of the ducts corresponding to the selected region of interest may be displayed.

At operation <NUM>, a determination may be made as to whether the abnormality is inside or outside of a breast duct based on the mapping of the breast ducts. In one example, the determination may be made based on the concurrent display of the abnormality and the duct mapping. The determination may be made through image analysis techniques that analyze the concurrent display of the abnormality and the duct mapping to identify whether the abnormality is inside or outside of a breast duct. The determination may also be made based on the location of the abnormality and the locations of the breast ducts. For example, a location of the abnormality may be determined and represented in coordinates of the shared coordinate space between the first imaging data and the second imaging data. The structures of the ducts may also be represented in coordinates of the shared coordinate space. Accordingly, a determination may be made as to whether the location of the abnormality falls inside or outside the ducts.

<FIG> depicts another example method <NUM> for locating an abnormality within a breast. At operation <NUM>, first imaging data for a breast from a first imaging modality is acquired or received, which may be an x-ray-based imaging modality and/or a magnetic resonance imaging (MRI) modality. The first imaging data may be two-dimensional imaging data or three-dimensional imaging data. In some examples, the first imaging data may be acquired from a tomosynthesis imaging system and/or a computed tomography system. In such an example, the first imaging data may be 3D imaging data. In other examples, the first imaging data may be 2D imaging data, such as mammography imaging data. At operation <NUM>, second imaging data is acquired for the breast from a second imaging modality. The second imaging modality may be an ultrasound imaging modality. The second imaging data may include 3D ultrasound data of a whole breast. For instance, during an ultrasound imaging procedure, the ultrasound imaging system may acquire a series of 2D slice and stitches and/or stacks the 2D slices together to form a set of 3D data or a 3D ultrasound volume of the breast. In other examples, the second imaging modality may be an optical and/or thermal imaging modality. In some examples, the second imaging modality may include both the ultrasound imaging modality and the optical and/or thermal imaging modality. The first imaging data and the second imaging data may then be co-registered at operation <NUM>, such that the first imaging data from the first imaging modality and the second image data from the second imaging modality share a common coordinate space. Co-registering the imaging data from the different modalities at operation <NUM> may be accomplished through any of the means discussed above.

At operation <NUM>, one or more structures of the breast are mapped or modeled based on the first imaging data and/or the second imaging data. For example, the mapping or modeling of the structures may be generated from the second imaging data, such as ultrasound imaging data, acquired at operation <NUM>. The mapping or modeling of the breast structures may be a 3D model or mapping or a 2D model of mapping. The mapping or model may also be for the whole breast. In some examples, the mapping or modeling of the breast structures may be achieved through artificial-intelligence image analysis techniques. Such image analysis techniques may analyze the first imaging data and/or the second imaging data to identify particular structures within the breast. For example, the structures may be ducts, lobules, Cooper's ligaments, dense tissue, fat, skin, vascular structures and/or lymph nodes. The particular structure(s) that are modeled may be selected by a clinician or automatically set by the imaging system. In addition, multiple models or mappings may be generated. For instance, a first model or mapping of ducts may be generated and a second model or mapping for lobules may be generated. In some examples, the tissue of the breast that is not the structure being modeled (such as non-ductal tissue) may be removed from the imaging data when generating the mapping or modeling of the breast structure(s). The image analysis techniques may be trained using a dataset of image data where the desired structures have been previously identified, such as by manual identification. Once the image analysis techniques have been trained, the image analysis techniques are able to identify ducts within image data. For example, the echogenicity of different structures as well as patterns and textures of those structures allows for the structures to be identified within the image data. Once the structures are identified, the mapping or model of the structures is generated.

At operation <NUM>, an abnormality is located or identified in the first imaging data from the first modality that was received or acquired in operation <NUM> and/or the second imaging data from the second imaging modality that was received or acquired in operation <NUM>. In examples where the first imaging data is x-ray data in the form of tomography or mammography image data, the abnormality may appear brighter in the image data, such as when the abnormality is a calcification. For example, pixels in the image data having a higher value (i.e., brighter) may correspond to an abnormality. The abnormality may be identified through the use of image analysis techniques that analyze the image data based on pixel values or the patterns of pixels. The image analysis techniques may be performed in the spatial, transform, or frequency domains. For example, image analysis techniques in the spatial domain generally operate based on the pixel values in the imaging data. Image analysis techniques within the transform or frequency domains generally operate based on mathematical transforms, such as a Fourier or Laplace transform, of the pixel data from imaging data. For instance, in the frequency domain, the image analysis techniques may be based on a rate of change of pixel values within the spatial domain. In other examples, an abnormality may be identified through the assistance of a medical professional. Locating the abnormality may include the medical professional selecting the abnormality on a screen displaying the imaging data.

At operation <NUM>, the abnormality that was located in operation <NUM> is compared to the mapping or model of the breast structures generated in operation <NUM>. The comparison of the abnormality to the mapping or model may include displaying at least a portion of a visual representation of the mapping concurrently with the abnormality. The concurrent display may include an overlapped display such that the abnormality is displayed within the mapping or model of the breast structure. In other examples, the comparison may include a mathematical or numerical comparison. For instance, the locations of the structures may be defined mathematically in the mapping or model. Similarly, the location of the abnormality may be also be defined mathematically. At operation <NUM>, the location of abnormality relative to the mapped or modeled structure(s) is determined. For example, a determination may be made as to whether the abnormality is within the duct of a breast or within or attached to a lobule. Such information can provide additional information for identifying the type or risk of the abnormality. The determined relative location may be displayed or presented in a variety of manners. For instance, the relative location may be displayed as a distance from the center or perimeter of the abnormality to the center or perimeter of one or more of the modeled structures.

<FIG> depicts an example method <NUM> for imaging a breast and generating a risk assessment. At operation <NUM> the breast is scanned with an ultrasound probe. For instance, an ultrasound technician may contact the probe to the surface of the breast and begin scanning the breast. At operation <NUM>, the location of the ultrasound probe is tracked during the scan of the breast. Tracking the location of the ultrasound probe may also include tracking the orientation of the probe as well. By tracking the location and orientation of the ultrasound probe, a 3D mapping of the breast ducts may later be generated and the image data acquired from the ultrasound imaging may be co-registered with imaging data from another imaging modality, such as x-ray imaging data. Feedback regarding progress of the scan may also be provided during the scan at operation <NUM>. The feedback may be visual feedback that indicates to the ultrasound technician whether the breast has been fully imaged. For example, to map all the ducts in the breast, the entire breast may need to imaged, or at least the portion of the breast that includes ducts. The visual feedback may be in the form of a graphical user interface element that updates during the scan. The graphical user interface element may be a circle or ellipse that is generally representative of the breast. As portions of the breast are scanned, the equivalent portions of the circle will change color or appear shaded to indicate that the corresponding portion of the breast has been scanned. As such, the ultrasound technician is able to see what portions of the breast still need to be scanned. Other visual indicators that indicate similar information may also be used. At operation <NUM>, a determination is made as to whether the scan has completed. The determination may be based on the ultrasound technician providing input indicating the scan is complete. In other examples, the system may make the determination based on the tracking of the ultrasound probe and the progress feedback. For instance, if the progress feedback indicates that the breast has not been fully or entirely scanned, the determination at operation <NUM> is that the scan is not complete. If it is determined that the scan is not complete at operation <NUM>, an alert may be provided to the ultrasound technician to continue scanning until the scanning is complete. In addition, the method <NUM> flows back to operation <NUM> where the ultrasound scanning continues until the full breast is scanned. If it is determined at operation <NUM> that the scan is complete, the method <NUM> flows to operation <NUM>.

At operation <NUM>, an image analysis technique is executed to identify the ducts within the breast. In some instances, the identification and mapping of the breast ducts may achieved through artificial-intelligence image analysis techniques. Such image analysis techniques may analyze image data from the ultrasound scan to identify ductal structures within the image data for the imaged breast. In some examples, the non-ductal tissue (i.e., the tissue of the breast other than the ducts) may be removed from the imaging data to generate ductal image data. The image analysis techniques may be trained using from a dataset of image data where ductal structures have been previously identified, such as by manual identification. Once the image analysis techniques have been trained, the image analysis techniques are able to identify ducts within image data.

At operation <NUM>, a mapping of the ducts of the breast may be generated. The mapping of the ducts may be a 3D mapping of the ducts or a plurality of 2D mapping of the ducts. In some examples, the mapping of the ducts of the breast is generated from the ductal image data generated in operation <NUM>. At operation <NUM>, the mapping of the ducts generated in operation <NUM> is analyzed to determine a statistical correlation between the mapping of the ducts and data for an aggregation of ductal structures for other breasts. As discussed above, the pattern or structure of the ducts of a breast may be indicative of potential invasive cancers. As such, comparing the mapping of the ducts for a patient to prior ductal structures of patients having different types of cancers may reveal potential risk factors. As an example, data may be aggregated for a number of patients that have been diagnosed with different types of cancers. The data may include the type of cancer and a mapping of the ducts for the patient. A statistical correlation may then be determined for a current mapping of a breast to the aggregated data for prior duct mappings of other patients. Based on statistical correlation determined at operation <NUM>, a risk assessment for the breast may be generated at operation <NUM>. The risk assessment may indicate whether the breast is at a high or low risk for certain types of cancers. The risk assessment may also indicate whether additional diagnostic procedures, such as mammography or tomography procedures, biopsy procedures, or other diagnostic procedures should be performed.

While method <NUM> is generally discussed as utilizing ultrasound imaging techniques, operations <NUM>-<NUM> may also be performed with optical and/or thermal imaging data. For instance, optical and/or thermal imaging data may be acquired instead of, or in addition to, the ultrasound imaging data. The identification of ducts in operation <NUM> and the generation of the mapping of the ducts in operation <NUM> may then be performed based on the optical and/or thermal imaging data.

<FIG> depicts another example method <NUM> for imaging a breast and generating a risk assessment. The method <NUM> begins with scanning of the breast with the ultrasound probe in operation <NUM>. During the scanning of the breast, the location of the ultrasound probe is tracked at operation <NUM>. Feedback regarding progress of the scan may also be provided during the scan at operation <NUM>. At operation <NUM>, a determination is made as to whether the scan has been completed. Operations <NUM>-<NUM> may be substantially the same as operations <NUM>-<NUM> in method <NUM> depicted in <FIG>.

Once the scan is complete, in operation <NUM>, one or more anatomical structures of the breast are identified within the image data generated by the scan. In some instances, the mapping of the breast ducts may achieved through artificial-intelligence image analysis techniques. Such image analysis techniques may analyze image data from the ultrasound scan to identify the desired or selected structures within the image data for the imaged breast. The image analysis techniques may be trained from a dataset of image data where the selected or desired structures have been previously identified, such as by manual identification. Once the image analysis techniques have been trained, the image analysis techniques are able to identify the selected structures within image data.

At operation <NUM>, a mapping or model of the selected or desired structures of the breast may be generated. The mapping or model of the structures may be a 3D mapping of the structures or a plurality of 2D mapping of the structures and may be for substantially the whole breast. In some examples, the mapping or model may be for a portion of the breast. The mapping or model of the structures of the breast may be generated from the structures identified in operation <NUM>. At operation <NUM>, the mapping or model of the structures generated in operation <NUM> is analyzed to determine and generate a risk assessment in operation <NUM>. The analysis of the structures may include analyzing patterns within the mapping or model to extract features or values associated with the structures. For example, from the mapping or model of the structures, quantitative values for features may be extracted. In the example of a model or mapping of ducts, the number of ducts, a regularity pattern for the ducts, and/or a termination regularity for the ducts may be extracted from the mapping or model. Those extracted features or structures may then be compared to one or more thresholds, known values, or benchmarks. The analysis may also include a statistical correlation between the mapping or model of the structure(s) and data for an aggregation of the same type of structure(s) for other breasts. For instance, comparing the mapping of the structure for a patient to prior mappings of the same type of structures for patients having different types of cancers may reveal potential risk factors. As an example, data may be aggregated for a number of patients that have been diagnosed with different types of cancers. The data may include the type of cancer and a mapping of the structure for the patient. A statistical correlation may then be determined for a current mapping of a breast to the aggregated data for prior duct mappings of other patients. Based on that analysis, a risk assessment for the breast may be generated at operation <NUM>.

The analysis of the structures may different depending on the type of structure that is being analyzed. For example, features of breast ducts that may increase a risk assessment for a patient may include features indicative of ductal ectasia (dilation), solid material or masses within a duct, an abrupt change in the caliber of the duct, and/or an abrupt termination of a duct. Such features may be extracted from a mapping or model of the ducts and/or from the imaging data itself generated during the scan. In examples where the selected or desired structure is a lobule, features that may increase a risk assessment for a patient may include features indicative of dilation of a lobule, solid material or masses within the lobule, and/or a distortion of a lobular cluster. Such features may be extracted from a mapping or model of the lobules and/or from the imaging data itself generated during the scan. In examples where the selected or desired structure is a Cooper's ligament, features that may increase a risk assessment for a patient may include distortion of a normal undulating flow, an abrupt interruption of a Cooper's ligament, and/or a thickening of the ligament. In examples where the selected or desired structure is dense tissue, features that may increase a risk assessment for a patient may include areas of prominent shadowing and/or asymmetry of the dense tissue in the breast. In examples where the selected or desired structure is fat, features that may increase a risk assessment for a patient may include a lack of homogeneity, such as stranding in normally hypoechoic fat (indicating edema), retroglandular fat (near the pectoralis muscle) stranding (suggesting tumor invasion), and/or edema or inflammation. In examples where the selected or desired structure is skin, features that may increase a risk assessment for a patient may include a thickening of the skin (such as greater than about <NUM>) and/or an interruption of a deep dermis. In examples where the selected or desired structure is a vascular structure, features that may increase a risk assessment for a patient may include a region of increased vascularity. In examples where the selected or desired structure is a lymph node, features that may increase a risk assessment for a patient may include a thickened cortex of the lymph node (such as greater than <NUM>) and/or a lack of a fatty central hilum.

Values for the above discussed features of the respective structures may be determined based on aggregated patient data. For instance, for the feature of a thickened cortex of a lymph node, the value of the thickness of a lymph node that may be indicative of a high likelihood for an abnormality may be determined based on an aggregate of lymph node mappings from patients having cancer or other abnormalities. The risk assessment may indicate whether the breast is at a high or low risk for certain types of cancers. The risk assessment may also indicate whether additional diagnostic procedures, such as mammography or tomography procedures, biopsy procedures, or other diagnostic procedures should be performed.

The methods described above may be performed by the systems described herein. For example, a system may include one or more imaging systems, such as a tomosynthesis, mammography, MRI, ultrasound, thermal, and/or optical imaging systems. One or more of those imaging systems may be operatively connected to a computing device, such as a computing device incorporating the operating environment discussed below with reference to <FIG>. The imaging data may be received by the computing device and the computing device may then perform the operations of the methods described herein. For example, the computing device may include at least one processor and memory that is operatively connected to the at least one processor. The memory may store instructions that when executed by the at least one processor cause the system to perform the operations described herein.

<FIG> illustrates one example of a suitable operating environment <NUM> in which one or more of the present embodiments can be implemented. This operating environment may be incorporated directly into the imaging systems disclosed herein, or may be incorporated into a computer system discrete from, but used to control, the imaging systems described herein. This is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality. Other computing systems, environments, and/or configurations that can be suitable for use include, but are not limited to, imaging systems, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics such as smart phones, network PCs, minicomputers, mainframe computers, tablets, distributed computing environments that include any of the above systems or devices, and the like.

In its most basic configuration, operating environment <NUM> typically includes at least one processing unit <NUM> and memory <NUM>. Depending on the exact configuration and type of computing device, memory <NUM> (storing, among other things, instructions to perform the image acquisition and processing methods disclosed herein) can be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in <FIG> by dashed line <NUM>. Further, environment <NUM> can also include storage devices (removable, <NUM>, and/or non-removable, <NUM>) including, but not limited to, magnetic or optical disks or tape. Similarly, environment <NUM> can also have input device(s) <NUM> such as touch screens, keyboard, mouse, pen, voice input, etc., and/or output device(s) <NUM> such as a display, speakers, printer, etc. Also included in the environment can be one or more communication connections <NUM>, such as LAN, WAN, point to point, Bluetooth, RF, etc..

Operating environment <NUM> typically includes at least some form of computer readable media. Computer readable media can be any available media that can be accessed by processing unit <NUM> or other devices comprising the operating environment. By way of example, and not limitation, computer readable media can comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, solid state storage, or any other tangible medium which can be used to store the desired information. Communication media embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media. A computer-readable device is a hardware device incorporating computer storage media.

The operating environment <NUM> can be a single computer operating in a networked environment using logical connections to one or more remote computers. The remote computer can be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above as well as others not so mentioned. The logical connections can include any method supported by available communications media. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

In some embodiments, the components described herein comprise such modules or instructions executable by computer system <NUM> that can be stored on computer storage medium and other tangible mediums and transmitted in communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Combinations of any of the above should also be included within the scope of readable media. In some embodiments, computer system <NUM> is part of a network that stores data in remote storage media for use by the computer system <NUM>.

The embodiments described herein may be employed using software, hardware, or a combination of software and hardware to implement and perform the systems and methods disclosed herein. Although specific devices have been recited throughout the disclosure as performing specific functions, one of skill in the art will appreciate that these devices are provided for illustrative purposes, and other devices may be employed to perform the functionality disclosed herein without departing from the scope of the disclosure.

This disclosure describes some embodiments of the present technology with reference to the accompanying drawings, in which only some of the possible embodiments were shown. Other aspects may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments were provided so that this disclosure was thorough and complete and fully conveyed the scope of the possible embodiments to those skilled in the art. Further, as used herein and in the claims, the phrase "at least one of element A, element B, or element C" is intended to convey any of: element A, element B, element C, elements A and B, elements A and C, elements B and C, and elements A, B, and C.

Claim 1:
A method for locating an abnormality within a breast, the method comprising:
acquiring (<NUM>) first imaging data for a breast from a first imaging modality, wherein the first imaging modality is at least one of an x-ray-based imaging modality or a magnetic resonance imaging (MRI) modality;
acquiring (<NUM>) second imaging data for the breast from a second imaging modality, wherein the second imaging modality is at least one of an ultrasound imaging modality or a thermal imaging modality;
co-registering (<NUM>) the first imaging data from the first imaging modality with the second imaging data from the second imaging modality, such that the first imaging data and the second imaging data share a common coordinate space;
based on the second imaging data from the second imaging modality, generating (<NUM>) a model of one or more structures within the breast to generate a mapping of the one or more structures;
locating (<NUM>) from the first imaging data, the abnormality in the breast; and
based at least on the generated model of the one or more structures, determining (<NUM>) a location of the abnormality relative to modeled one or more structures within the breast.