Patent ID: 12185921

DETAILED DESCRIPTION

Various implementations and aspects of the present disclosure will be described with reference to the below details. The following description and drawings are illustrative of the present disclosure and are not to be construed as limiting the present disclosure. Numerous specific details are described to provide a thorough understanding of various implementations of the present disclosure. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of implementations of the present disclosure.

The systems and methods described herein may be useful in the field of neurosurgery, including oncological care, neurodegenerative disease, stroke, brain trauma, and orthopedic surgery; however, extending these concepts to other conditions or fields of medicine is also encompassed by the present disclosure. The surgical process is applicable to surgical procedures for brain, spine, knee and any other suitable region of the body.

Various apparatuses and processes are below described to provide examples of implementations of the disclosed herein system. No implementation below described limits any claimed implementation and any claimed implementations may cover processes or apparatuses that differ from those below described. The claimed implementations are not limited to apparatuses or processes having all of the features of any one apparatus or process below described or to features common to multiple or all of the apparatuses or processes below described. It is possible that an apparatus or process described below is not an implementation of any claimed subject matter.

Furthermore, numerous specific details are set forth in order to provide a thorough understanding of the implementations herein described. However, the implementations described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the implementations described herein.

As used herein, elements may be described as “configured to” perform one or more functions or “configured for” such functions. In general, an element that is configured to perform or configured for performing a function is enabled to perform the function, or is suitable for performing the function, or is adapted to perform the function, or is operable to perform the function, or is otherwise capable of performing the function.

For the purpose of this present disclosure, language of “at least one of X, Y, and Z” and “one or more of X, Y and Z” may be construed as X only, Y only, Z only, or any combination of two or more items X, Y, and Z, e.g., XYZ, XY, YZ, ZZ, and the like. Similar logic may be applied for two or more items in any occurrence of “at least one . . . ” and “one or more . . . ” language.

Referring toFIG.1, this diagram illustrates a navigation system100is shown to support minimally invasive access port-based surgery, in accordance with an embodiment of the present disclosure. A neurosurgeon101conducts a minimally invasive port-based surgery on a patient102in an operating room (OR) environment. The navigation system100includes an equipment tower, tracking system, displays and tracked instruments to assist the surgeon101during the procedure. An operator103may also be present to operate, control and provide assistance for the navigation system100.

Referring toFIG.2, this diagram illustrates components of an example medical navigation system200, according to embodiments of the present disclosure. The medical navigation system200illustrates a context in which a surgical plan including equipment, e.g., tool and material, tracking, such as that described herein, may be implemented. The medical navigation system200includes, but is not limited to, one or more monitors205,211for displaying a video image, an equipment tower201, and a mechanical arm202, which supports an optical scope204. The equipment tower201is mounted on a frame, e.g., a rack or cart, and may contain a computer or controller (examples provided with reference toFIGS.3and6), planning software, navigation software, a power supply, software to manage the mechanical arm202, and tracked instruments. In one example non-limiting implementation, the equipment tower201comprises a single tower configuration with dual display monitors211,205, however other configurations may also exist, e.g., dual tower, single display, etc. Furthermore, the equipment tower201is configured with a universal power supply (UPS) to provide emergency power, in addition to a regular AC adapter power supply.

Still referring toFIG.2, a patient's anatomy may be held in place by a holder. For example, in a neurosurgical procedure, the patient's head may be held in place by a head holder217, and an access port206and an introducer210may be inserted into the patient's head. The introducer210may be tracked using a tracking camera213, which provides position information for the navigation system200. The tracking camera213may also be used to track tools and/or materials used in the surgery, as below described in more detail. In one example, the tracking camera213comprises a 3D (three-dimensional) optical tracking stereo camera, similar to one made by Northern Digital Imaging (NDI®), configured to locate reflective sphere tracking markers212in 3D space.

Still referring toFIG.2, in another example, the tracking camera213may comprise a magnetic camera, such as a field transmitter, where receiver coils are used to locate objects in 3D space, as is also known in the art. Location data of the mechanical arm202and access port206may be determined by the tracking camera213by detection of tracking markers212placed on these tools, for example, the introducer210and associated pointing tools. Tracking markers may also placed on surgical tools or materials to be tracked. The secondary display205may provide output of the tracking camera213. In one example non-limiting implementation, the output may be shown in axial, sagittal, and coronal views as part of a multi-view display. The introducer210may include tracking markers212for tracking. The tracking markers212may comprise reflective spheres in the case of an optical tracking system and/or pick-up coils in the case of an electromagnetic tracking system. The tracking markers212may be detected by the tracking camera213and their respective positions are inferred by the tracking software.

Still referring toFIG.2, a guide clamp218(or more generally a guide) for holding the access port206may be provided. The guide clamp218may optionally engage and disengage with the access port206without needing to remove the access port206from the patient. In some examples, the access port206may be moveable relative to the guide clamp218, while in the guide clamp218. For example, the access port206may be able to slide up and down, e.g., along the longitudinal axis of the access port206, relative to the guide clamp218while the guide clamp218is in a closed position. A locking mechanism may be attached to or integrated with the guide clamp218, and may optionally be actuatable with one hand, as described further below. Furthermore, an articulated arm219may be provided to hold the guide clamp218. The articulated arm219may have up to six degrees of freedom to position the guide clamp218. The articulated arm219may be lockable to fix its position and orientation, once a desired position is achieved. The articulated arm219may be attached or attachable to a point based on the patient head holder217, or another suitable point, e.g., on another patient support, such as on the surgical bed, to ensure that when locked in place, the guide clamp218does not move relative to the patient's head.

Referring toFIG.3, a block diagram is shown illustrating a control and processing unit300that may be used in the navigation system200, as shown inFIG.2, e.g., as part of the equipment tower, in accordance with an embodiment of the present disclosure. In one example non-limiting implementation, control and processing unit300may include one or more processors302, a memory304, a system bus306, one or more input/output interfaces308, a communications interface310, and storage device312. In particular, one or more processors302may comprise one or more hardware processors and/or one or more microprocessors. Control and processing unit300is interfaced with other external devices, such as tracking system321, data storage device342, and external user input and output devices344, which may include, but is not limited to, one or more of a display, keyboard, mouse, foot pedal, and microphone and speaker. Data storage device342may comprise any suitable data storage device, including, but not limited to a local and/or remote computing device, e.g., a computer, hard drive, digital media device, and/or server, having a database stored thereon.

Still referring toFIG.3, the data storage device342includes, but is not limited to, identification data350for identifying one or more medical instruments360and configuration data352that associates customized configuration parameters with one or more medical instruments360. Data storage device342may also include, but is not limited to, preoperative image data354and/or medical procedure planning data356. Although data storage device342is shown as a single device inFIG.3, in other implementations, data storage device342may be provided as multiple storage devices.

Still referring toFIG.3, the medical instruments360may be identifiable using control and processing unit300. Medical instruments360may be connected to and controlled by control and processing unit300, and/or medical instruments360may be operated and/or otherwise employed independently of control and processing unit300. Tracking system321may be employed to track one or more of medical instruments360and spatially register the one or more tracked medical instruments360to an intraoperative reference frame. In another example, a sheath may be placed over a medical instrument360and the sheath may be connected to and controlled by control and processing unit300.

Still referring toFIG.3, the control and processing unit300may also interface with a number of configurable devices and may intraoperatively reconfigure one or more of such devices based on configuration parameters obtained from configuration data352. Examples of devices320, as shown inFIG.3, include, but are not limited, one or more external imaging devices322, one or more illumination devices324, a robotic arm, one or more projection devices328, and one or more displays305,311.

Still referring toFIG.3, aspects of the present disclosure may be implemented via processor(s)302and/or memory304. For example, the functionalities described herein may be partially implemented via hardware logic in processor302and partially using the instructions stored in memory304, as one or more processing modules370and/or processing engines. Example processing modules include, but are not limited to, user interface engine372, tracking module374, motor controller376, image processing engine378, image registration engine380, procedure planning engine382, navigation engine384, and context analysis module386. While the example processing modules are shown separately, in one example non-limiting implementation the processing modules370may be stored in the memory304and the processing modules may be collectively referred to as processing modules370.

Still referring toFIG.3, the system is not intended to be limited to the components as shown. One or more components of the control and processing unit300may be provided as an external component or device. In one example non-limiting implementation, navigation engine384may be provided as an external navigation system that is integrated with control and processing unit300.

Still referring toFIG.3, some implementations may be implemented using processor302without additional instructions stored in memory304. Some implementations may be implemented using the instructions stored in memory304for execution by one or more general purpose microprocessors. Thus, the present disclosure is not limited to a specific configuration of hardware and/or software.

While some implementations may be implemented in fully functioning computers and computer systems, various implementations are capable of being distributed as a computing product in a variety of forms and are capable of being applied regardless of the particular type of machine or computer readable media used to actually effect the distribution.

At least some aspects disclosed may be embodied, at least in part, in software. That is, the techniques may be carried out in a computer system or other data processing system in response to its processor, such as a microprocessor, executing sequences of instructions contained in a memory, such as ROM, volatile RAM, non-volatile memory, cache and/or a remote storage device.

A computer readable storage medium, and/or a non-transitory computer readable storage medium, may be used to store software and data which, when executed by a data processing system, causes the system to perform various methods. The executable software and data may be stored in various places including for example ROM, volatile RAM, non-volatile memory and/or cache. Portions of this software and/or data may be stored in any one of these storage devices.

Examples of computer-readable storage media include, but are not limited to, recordable and non-recordable type media such as volatile and non-volatile memory devices, read only memory (ROM), random access memory (RAM), flash memory devices, floppy and other removable disks, magnetic disk storage media, optical storage media, e.g., compact discs (CDs), digital versatile disks (DVDs), etc., among others. The instructions may be embodied in digital and analog communication links for electrical, optical, acoustical and/or other forms of propagated signals, such as carrier waves, infrared signals, digital signals, and the like. The storage medium may comprise the internet cloud, storage media therein, and/or a computer readable storage medium and/or a non-transitory computer readable storage medium, including, but not limited to, a disc.

At least some of the methods described herein are capable of being distributed in a computer program product comprising a computer readable medium that bears computer usable instructions for execution by one or more processors, to perform aspects of the methods described. The medium may be provided in various forms such as, but not limited to, one or more diskettes, compact disks, tapes, chips, USB (Universal Serial Bus) keys, external hard drives, wire-line transmissions, satellite transmissions, internet transmissions or downloads, magnetic and electronic storage media, digital and analog signals, and the like. The computer useable instructions may also be in various forms, including compiled and non-compiled code.

According to one aspect of the present application, one purpose of the navigation system200, which may include control and processing unit300, is to provide tools to a surgeon and/or a neurosurgeon that will lead to the most informed, least damaging neurosurgical operations. In addition to removal of brain tumors and intracranial hemorrhages (ICH), the navigation system200may also be applied to a brain biopsy, a functional/deep-brain stimulation, a catheter/shunt placement procedure, open craniotomies, endonasal/skull-based/ENT, spine procedures, and other parts of the body such as breast biopsies, liver biopsies, etc. While several examples have been provided, aspects of the present disclosure may be applied to other suitable medical procedures.

Referring toFIG.4, this diagram illustrates a port-based brain surgery procedure using a video scope, in accordance with an embodiment of the present disclosure. An operator404, for example, a surgeon, may align video scope402to peer down port406. Video scope402is attached to an adjustable mechanical arm410. Port406may have a tracking tool408attached to it where tracking tool408is tracked by a tracking camera of a navigation system. Even though the video scope402may comprise an endoscope and/or a microscope, these devices introduce optical and ergonomic limitations when the surgical procedure is conducted over a confined space and conducted over a prolonged period such as the case with minimally invasive brain surgery.

Referring toFIG.5, this diagram illustrates the insertion of an access port12into a human brain10, in order to provide access to interior brain tissue during a medical procedure, in accordance with an embodiment of the present disclosure. The access port12is inserted into a human brain10, providing access to interior brain tissue. Access port12may include, but is not limited to, instruments such as catheters, surgical probes, and/or cylindrical ports such as the NICO BrainPath®. Surgical tools and instruments may then be inserted within a lumen of the access port12in order to perform surgical, diagnostic or therapeutic procedures, such as resecting tumors as necessary. However, the present disclosure applies equally well to catheters, DBS needles, a biopsy procedure, and also to biopsies and/or catheters in other medical procedures performed on other parts of the body. In the example of a port-based surgery, a straight and/or linear access port12is typically guided down a sulci path of the brain. Surgical instruments and/or surgical tools would then be inserted down the access port12.

Referring toFIG.6, this schematic diagram illustrates a system600comprising a flexible high-resolution endoscope601that could be used with access port12, in accordance with an embodiment of the present disclosure. Elements of system600are not drawn to scale, but are depicted schematically to show functionality. Endoscope601comprises: a plurality of optical fiber bundles603; a plurality of lenses605in a one-to-one relationship with plurality of optical fiber bundles603; and, a plurality of cameras607in a one-to-one relationship with the plurality of optical fiber bundles603, each respective optical fiber bundle603, of the plurality of optical fiber bundles603, having a respective lens605, of the plurality of lenses605, located at a respective distal end609, and a camera607, of the plurality of cameras607, located at a respective proximal end611, plurality of optical fiber bundles603being coupled together at a common distal end609, and otherwise being uncoupled from one another, a bending radius of endoscope601defined by a largest respective bending radius of each of the plurality of optical fiber bundles603. The plurality of lenses605are each formed in a common optical element613located at common distal end609, which also couples together plurality of optical fiber bundles603at common distal end609. Respective distal ends609of each optical fiber bundle603are coincident with common distal end609, such each of distal ends609and common distal end609are similarly numbered.

Still referring toFIG.6, the system600further comprises a controller615, coupled with each of cameras607, and a display device626, as below described in more detail. In general, endoscope601is configured to acquire a plurality of images of a tissue sample620, which can include, but is not limited to, a tissue sample accessible via access port12. In particular, respective distal ends609of the plurality of optical fiber bundles603, and respective lenses605located at respective distal ends609, are spaced apart from one another to provide different views of objects (such as tissue sample620) in front of the respective distal ends609. In some of these implementations, endoscope601can thereby form a plenoptic camera.

Still referring toFIG.6, while only one of each of plurality of optical fiber bundles603, plurality of lenses605, and plurality of cameras607, the endoscope601comprises four optical fiber bundles603, four respective lenses605and four respective cameras607. However, endoscope601comprises as few as two of each of optical fiber bundles603, lenses605and cameras607, and comprises more than four of each of optical fiber bundles603, lenses605and cameras607. However, at a minimum, endoscope601comprises the plurality of optical fiber bundles comprises a first optical fiber bundle603and a second optical fiber bundle603, each having the respective lens605located at the respective distal end609, and the respective camera607located at the respective proximal end611, which can thereby form a three-dimensional camera.

Still referring toFIG.6, each optical fiber bundle603comprises an optical fiber having a respective diameter of are less than or equal to about 2 mm (however, optical fiber bundles603need not have all the same diameter). In particular, each optical fiber bundle603can have a diameter that can convey images from respective lenses605to respective cameras607with resolutions similar to cameras607. For example, 2 mm optical fiber bundles can convey images of resolutions of about 18 kilopixels, and hence cameras607can produce digital images have resolutions of about 18 kilopixels.

Still referring toFIG.6, furthermore, as each optical fiber bundle603is free to bend independent from every other optical fiber bundle603, other than at common distal end609, the bending radius of endoscope601is determined and/or defined by the individual bending radii of each optical fiber bundle603rather than a total bending radii if optical fiber bundles603were coupled together along their entire length. Put another way, a bending radius of endoscope601is defined by a largest respective bending radius of each of the plurality of optical fiber bundles603. As such, optical fiber bundles603of any suitable diameter are within the scope of present implementations; for example, a specified bending radius of endoscope601is used to select individual optical fibers that will meet this present disclosure, rather than selecting optical fibers that, when coupled together, will meet this present disclosure.

Still referring toFIG.6, in other words, a plurality of optical fiber bundles603are coupled together at common distal end609, e.g., by way of common optical element613, and are otherwise uncoupled from one another. Indeed, each of plurality of optical fiber bundles603can bend from common distal end609independent of the other optical fiber bundles603. As such plurality of optical fiber bundles603are not attached to each other than at common distal end609.

Still referring toFIG.6, each optical fiber bundle603can have a length that is commensurate with insertion through an access port (including, but not limited to, access port12), as well as port-based surgery, such that common distal end609is inserted through an access port, and optical fiber bundles603join respective lenses605to respective cameras607such that cameras607do not block surgical access to access port12, e.g., cameras607do not block access of a surgeon (and the like) and/or surgical tools (and the like) to access port12. For example, each optical fiber bundle603can be greater than about a half meter long. Furthermore, optical fiber bundles603need not all be the same length, and some optical fiber bundles603can be longer or shorter than other optical fiber bundles603.

Still referring toFIG.6, each lens605is formed in common optical element613located at common distal end609. Common optical element613comprises one or more of optical glass and optical plastic, at least at a tissue facing side of common optical element613. Each lens605is formed in the common optical element613using, for example, laser processing (including, but not limited to, femtosecond laser processing, and the like) to form profiles in the index of refraction in the glass and/or plastic of common optical element613which correspond to each lens605. However, lenses605can also be tiled together using one or more of a mechanical assembly, adhesives, and the like.

Still referring toFIG.6, two or more of plurality of lenses605can have one or more of: different depths of field, different fields of view of objects in front of the plurality of lenses605: and different angular views of objects in front of the plurality of lenses605. Hence, when endoscope601is imaging tissue sample620, tissue sample620can be imaged using at least two different depths of field and/or at least two different fields of view and/or at least two different angular views.

Still referring toFIG.6, each camera607can include, but is not limited to one or more of a charge-coupled device (CCD) camera, a digital camera, an optical camera, and the like, and is generally configured to acquire digital images, and in particular digital images received from a respective lens605via a respective optical fiber bundle603. While not depicted, each camera607can further include one or more respective lenses for focusing light from a respective optical fiber603onto a respective imaging element (such as a CCD). While not depicted, endoscope601can include one or more devices for coupling optical fiber bundles603to a respective camera607. Furthermore, each camera607can have a resolution of about 18 kilopixels.

Still referring toFIG.6, the controller615comprises any suitable combination of computing devices, processors, memory devices and the like. In particular, controller615comprises one or more of a data acquisition unit, configured to acquire data and/or images at least from cameras607, and an image processing unit, configured to process data and/or images from cameras607for rendering at display device626. Hence, controller615is interconnected with cameras607and display device626. In some implementations, controller615comprises control and processing unit300, as shown inFIG.3, and/or controller615communicates with control and processing unit300, as shown inFIG.3, and/or controller615can be under control of communication with control and processing unit300, as shown inFIG.3.

Still referring toFIG.6, in some implementations, however, controller615can be a component of endoscope601such that endoscope601comprises controller615. In these implementations, endoscope601can be provided as a unit with controller615which can be interfaced with control and processing unit300depicted inFIG.3, and the like. The display device626comprises any suitable display device including, but not limited to, cathode ray tubes, flat panel displays, and the like. For example, display device626comprises one or more of monitors205,211, as shown inFIG.2, and/or displays305,311shown inFIG.3.

Referring toFIG.7, this flow diagram illustrates a method700for combining images from cameras, according to non-limiting implementations, in accordance with an embodiment of the present disclosure. In order to assist in the explanation of method700, assumed is that method700is performed using system600, and specifically by controller615. Indeed, the method700is one way in which system600and/or controller615can be configured. Furthermore, the following discussion of method700will lead to a further understanding of controller615, and system600and its various components. However, the system600and/controller615and/or method700can be varied, and need not work exactly as discussed herein in conjunction with each other, and that such variations are within the scope of present disclosure.

Still referring toFIG.7, regardless, the method700need not be performed in the exact sequence as shown, unless otherwise indicated; and likewise various blocks are performed in parallel rather than in sequence; hence the elements of method700are referred to herein as “blocks” rather than “steps”. The method700can be implemented on variations of system600as well. At block701, controller615receives respective images from each of plurality of cameras607. At an optional block703(indicated by block703being shown in broken lines), controller615removes dead pixels from the respective images. In some implementations, block703is not performed. Furthermore, in other implementations, when no dead pixels are in the respective images, block703is not performed. At block705, controller615can combine the respective images into a single higher resolution image. At block707, controller615can combine the respective images into a depth-map of objects in front of the plurality of lenses605using, for example, light field processing techniques, and the like. In some implementations, controller615can implement both blocks705,707, for example, in parallel with each other and/or one after the other. In other implementations, controller615can implement one of blocks705,707. In some implementations, controller615can implement blocks703in conjunction with one or more of blocks705,707.

Still referring toFIG.7, back toFIG.6, and ahead toFIG.8, the method700, respective distal ends609and respective proximal ends611are enlarged, and the entire length of each optical fiber bundle603is not depicted. InFIG.8, light801,802,803,804representing different respective views of tissue sample620is collected by respective lenses601and conveyed to respective cameras607by respective optical fiber bundles603. Cameras607convert light801,802,803,804into respective digital images811,812,813,814of tissue sample620, which are received at controller615, e.g. at block701of method700.

Still referring toFIG.7, back toFIG.6, and ahead toFIG.8, the controller615processes digital images811,812,813,814to optionally remove dead pixels in each of digital images811,812,813,814, e.g., at block703of method700, and combine digital images811,812,813,814into one or more of single higher resolution image820of tissue sample620, e.g., at block705of method700, and a depth-map830of tissue sample620, e.g., at block707of method700. Controller615can be configured to produce higher resolution image820and/or depth-map830from digital images811,812,813,814using light field processing. Controller615can then provide higher resolution image820and/or depth-map830to display device626for rendering thereupon. Controller can optionally provide one or more of digital images811,812,813,814to display device626for rendering thereupon (not depicted).

Still referring toFIG.7, back toFIG.6, and ahead toFIG.8, dead pixels of digital images811,812,813,814and pixels of digital images811,812,813,814can be combined and/or interlaced and/or used to produce interpolated pixels to produce image820which has a higher resolution of each of digital images811,812,813,814taken alone. Hence, if each of digital images811,812,813,814has a resolution of about 18 kilopixels, image820can have a resolution can at least about double 18 kilopixels. Thus, endoscope601is configured to images having resolutions similar to those produced by existing high-resolution endoscopes, but without the attendant problems with bending radius suffered by those existing high-resolution endoscopes. Using light field processing of the separate digital images811,812,813,814from cameras607, a depth-map of tissue sample620(or any other object) imaged by lenses605can be reconstructed, which can allow structures with differing depth to be more easily detected and/or see. When endoscope601is configured for omni-focusing (having all objects imaged by lenses605in focus), selective post-acquisition focusing, and depth of field control can be possible, both in post-acquisition and real-time. This post-processing can also allow for removal of dead pixels which can be caused by broken fibers within fiber bundles without significant loss of detail. In other words, block703can be performed in conjunction with either of blocks705,707.

Still referring toFIG.7, back toFIG.6, and ahead toFIG.8, in some implementations, two or more of digital images811,812,813,814can be combined into a stereo image of tissue sample620. Indeed, a plurality of pairs of digital images811,812,813,814can be combined to produce a plurality of stereo images of tissue sample620, for example from different angles and/or different fields of view and/or different depths of field. In yet further implementations, where lenses605have different depths of field, but a similar field of view, digital images811,812,813,814can be combined into a plenoptic image of tissue sample620such that a depth of field of the plenoptic image can be selected by a user interacting with controller615, display device626and an input device (not depicted). Indeed, in these implementations, lenses605can be configured for omni-focusing, where all objects, e.g., including, but not limited to tissue sample620, imaged by lenses605are in focus; while each individual lens605may not have all objects in focus, collectively lenses605can image all objects in focus such that, collectively, all images produced by cameras607include all objects imaged by lenses605in focus, at least in one of the images.

Referring toFIG.9, this diagram illustrates a common optical element613. Common optical element613is generally configured to both provide lenses605and couple together the plurality of optical fiber bundles603at common distal end609, in accordance with an embodiment of the present disclosure. Hence, common optical element613comprises lenses605and, as depicted, respective slots901for receiving a respective optical fiber bundle603on a proximal side, each slot901in a body of common optical element613, and each slot901terminating at a respective lens605at distal end609. Hence, each slot901has a diameter that is similar to a diameter of a respective optical fiber bundle603such that each slot901can receive a respective optical fiber bundle603and seat a distal end of each respective optical fiber bundle603at a respective lens605. While not depicted, common optical element613can further comprise a mechanism for fixing each respective optical fiber bundle603within a respective slot901; alternatively, adhesives (including, but not limited to optical adhesives) can be used to fix a respective optical fiber bundle603within a respective slot901.

Referring toFIG.10, this diagram illustrates an optical element613a, substantially similar to optical element613, with like elements having like numbers, however with an “a” appended thereto, in accordance with an embodiment of the present disclosure. Hence, optical element613acomprises a plurality of lenses605aat a distal end609a. However, in contrast to optical element613, optical element613acomprises eight lenses605a, and one slot901aconfigured to receive a plurality of optical fiber bundles. However, optical element613acomprises fewer than eight lenses605aand more than eight lenses605a.

Referring toFIG.11, which depicts slot901aof optical element613areceiving a plurality of optical fiber bundles603a, in a one-to-one relationship with plurality of lenses605a, optical fiber bundles603abeing coupled together a common distal end609a, and otherwise being uncoupled from one another, in accordance with an embodiment of the present disclosure. While only a portion of optical fiber bundles603ais depicted, it is assumed that each optical fiber bundle603ais coupled to a respective camera at a proximal end, similar to implementations depicted inFIG.6. In particular, coupling together of optical fiber bundles603aat common distal end609aresults in a total diameter of coupled optical fiber bundles603athat is about a same diameter as slot901a. While not depicted, distal ends of optical fiber bundles603aare aligned with a respective lens605a, as in system600. For example, a geometry of distal ends of optical fiber bundles603acan be selected so that when distal ends of optical fiber bundles603aare coupled together, they form a geometric pattern, and lenses605acan be arranged into a similar pattern. Hence, when distal ends of optical fiber bundles603aare inserted into slot901a, one or more of distal ends of optical fiber bundles603aand common optical element613acan be rotated until alignment with lenses605aoccurs. Such alignment can be determined by one or more of processing and viewing images from cameras to which each optical fiber bundle603ais coupled.

Still referring toFIG.11, alternatively, distal ends of optical fiber bundles603acan have a cross-section of a given geometric shape, for example, a geometric shape having at least one flat side, and a respective cross-section slot901acan have a similar shape; hence, when distal ends of optical fiber bundles603aare inserted into slot901athe flat sides align which can cause the distal ends of optical fiber bundles603ato align with lenses605a. Alternatively, a mechanical assembly can be used to couple together distal ends of optical fiber bundles603aand further space distal ends of optical fiber bundles603ain a pattern similar to lenses605a; in these implementations, slot901acan be adapted to receive the distal ends of optical fiber bundles603aand the mechanical assembly.

Referring toFIG.12, which depicts common optical element613abeing used with an optical fiber bundle603bhaving a similar diameter to that of slot901a, and can include a plurality of optical fiber bundles coupled together at common distal end609a, as well as along their length, in accordance with an embodiment of the present disclosure. Hence, optical fiber bundle603bis configured to convey images from lenses605ato one or more cameras at a common proximal end. Indeed, individual optical fiber bundles of optical fiber bundle603bneed not be aligned with lenses605aas a proximal end of optical fiber bundle603bcan have a diameter that can receive light from all of lenses605a. The bending radius of optical fiber bundle603bis larger than a bending radius of endoscope601, however such difference in bending radius does not preclude use of common optical element613awith more typical endoscopes.

Referring toFIG.13, which depicts an alternative system1300that includes an example of a flexible high-resolution endoscope1301that could be used with access port12to image tissue sample620, in accordance with an embodiment of the present disclosure. System1300is substantially similar to system600, with like elements having like numbers, but in a “1300” series rather than a “600” series. However, in contrast to endoscope601, optical fiber bundles of endoscope1301are coupled together at both a common distal end and a common proximal end, and are otherwise uncoupled, and cameras used with endoscope1301are not necessarily in a one-to-one relationship with the optical fiber bundles.

Still referring toFIG.13, hence, endoscope1301comprises: a plurality of optical fiber bundles1303; a plurality of lenses1305in a one-to-one relationship with plurality of optical fiber bundles1303; and, one or more cameras1307. Each respective optical fiber bundle1303, of the plurality of optical fiber bundles1303, has a respective lens1305, of the plurality of lenses1305, located at a respective distal end1309. One or more cameras1307are located at a common proximal end1311of the plurality of optical fiber bundles1303. Plurality of optical fiber bundles1303are coupled together at a common distal end1309and at common proximal end1311, and are otherwise uncoupled from one another. As depicted, plurality of lenses1305are each formed in a common optical element1313similar to common optical element613. As depicted, system1300further comprises a controller1315, coupled to each of one or more cameras1307, and a display device1326.

Still referring toFIG.13, while endoscope1301, as depicted, includes three cameras1307, in other implementations endoscope1301could include as few as one camera1307and more than three cameras1307, including more than four cameras1307. It is assumed, however, that cameras1307of endoscope1301are collectively configured to receive light from all of optical fiber bundles1303, and that each of one or more cameras1307can be arranged to receive images from one or more of plurality of optical fiber bundles1303. Hence, in these implementations, respective alignment of distal and proximal ends of optical fiber bundles1303with lenses1305and one or more cameras1307is less important than in system600, as each of one or more cameras1307can be arranged to receive images from one or more of plurality of optical fiber bundles1303. Controller1315can hence be configured to separate and/or combine images from each of one or more cameras1307into images corresponding to fields of view of each of lenses1305.

Still referring toFIG.13, indeed while, as depicted, each of distal ends of plurality of optical fiber bundles1303is aligned with a respective lens1305, in other implementations, more than one of plurality of optical fiber bundles1303can be arranged to receive light from one or more of lenses1305, such that plurality of optical fiber bundles1303functions optically as a larger optical fiber bundle similar to that depicted inFIG.12. Indeed, in some implementations, common optical element1313can be replaced with common optical element613ahaving one larger slot901ainstead of individual slots. However, as each of plurality of optical fiber bundles1303are free to bend individually, other than at ends1309,1311, a bending radius of endoscope1301is determined by the bending radii of individual optical fiber bundles1303rather than all of optical fiber bundles1303bending together.

Referring toFIG.14, this block diagram illustrates an endoscope apparatus A, in accordance with an embodiment of the present disclosure. The endoscope apparatus A comprises: a plurality of optical fiber bundles1401; a plurality of lenses1402in a one-to-one correspondence with the plurality of optical fiber bundles1401, each lens1402of the plurality of lenses1402comprising a distinct depth of field and a distinct angle of view in relation to another lens1402of the plurality of lenses1402; and a plurality of cameras1403in a one-to-one correspondence with the plurality of optical fiber bundles1402. The apparatus A further comprises a common optical element1404operable with the plurality of lenses1402, the common optical element1404disposed at a common distal end of the plurality of optical fiber bundles1402, and the common optical element1404being at least one of removable and disposable.

Still referring toFIG.14, in the apparatus A, each lens1402of the plurality of lenses1402is disposed in the common optical element1404. Each lens1402of the plurality of lenses1402is correspondingly disposed at a distal end of each optical fiber bundle1401of the plurality of optical fiber bundles1401. Each camera1403of the plurality of cameras1403is correspondingly disposed at a proximal end of each optical fiber bundle1401of the plurality of optical fiber bundles1401. The plurality of optical fiber bundles1401is coupled, together, at the common distal end. Each optical fiber bundle1401of the plurality of optical fiber bundles1401comprises a largest bending radius defining a largest bending radius of the apparatus A. The plurality of optical fiber bundles1401comprises a first optical fiber bundle and a second optical fiber bundle. The plurality of optical fiber bundles1401, the plurality of lenses1402, and the plurality of cameras1403, together, forming a three-dimensional camera. Each optical fiber bundle distal end is spaced apart from another optical fiber bundle distal end; and each lens1402is spaced apart from another lens1402to provide a plurality of distinct views.

Still referring toFIG.14, in the apparatus A, each lens of the plurality of lenses1402further comprises a distinct field of view in relation to another lens1402of the plurality of lenses1402. The apparatus A further comprises a controller1405configured to: receive at least one image from each camera1403of the plurality of cameras1403; and combine the at least one image from each camera1403of the plurality of cameras1403into a single higher resolution image. The controller1405is further configured to remove dead pixels from the at least one image from each camera1403of the plurality of cameras1403. The controller1405is further configured to provide a depth map by combining the at least one image from each camera1403of the plurality of cameras1403. The controller1405is further configured to provide the depth map by using light field processing. Each optical fiber bundle1401of the plurality of optical fiber bundles1401comprises a diameter in a range of up to approximately 2 mm. The common optical element1404comprises a body having a proximal end and a distal end, the distal end of the body configured to accommodate the plurality of lenses1402, and the proximal end of the body comprising a plurality of slots configured to correspondingly receive the plurality of optical fiber bundles1401, each slot of the plurality of slots correspondingly terminating at each lens of the plurality of lenses1402at the distal end of the body.

Referring toFIG.15, this flow diagram illustrates a method M1of providing an endoscope apparatus A, as shown inFIG.14, in accordance with an embodiment of the present disclosure. The method M1comprises: providing a plurality of optical fiber bundles1401, as indicated by block1501; providing a plurality of lenses1402in a one-to-one correspondence with the plurality of optical fiber bundles1401, providing the plurality of lenses1402comprising providing each lens1402of the plurality of lenses1402with a distinct depth of field and a distinct angle of view in relation to another lens1402of the plurality of lenses1402, as indicated by block1502; and providing a plurality of cameras1403in a one-to-one correspondence with the plurality of optical fiber bundles1401, as indicated by block1503. The method M1further comprises providing a common optical element1404operable with the plurality of lenses1402, providing the common optical element1404comprising disposing the common optical element1404at a common distal end of the plurality of optical fiber bundles1401, and providing the common optical element1404comprising providing the common optical element1404as at least one of removable and disposable, as indicated by block1504.

Still referring toFIG.15, in the method M1, providing the plurality of lenses1402, as indicated by block1502, comprises disposing each lens in the common optical element1404. Providing the plurality of lenses1402, as indicated by block1502, comprises correspondingly disposing each lens1402of the plurality of lenses1402at a distal end of each optical fiber bundle1401of the plurality of optical fiber bundles1401. Providing the plurality of cameras1403, as indicated by block1503, comprises correspondingly disposing each camera1403at a proximal end of each optical fiber bundle1401of the plurality of optical fiber bundles1401. Providing the plurality of optical fiber bundles1401, as indicated by block1501, comprises coupling, together, the plurality of optical fiber bundles1401at the common distal end. Providing the plurality of optical fiber bundles1401, as indicated by block1501, comprises providing each optical fiber bundle1401with a largest bending radius defining a largest bending radius of the apparatus A. Providing the plurality of optical fiber bundles1401, as indicated by block1501, comprises providing a first optical fiber bundle and providing a second optical fiber bundle. Providing the plurality of optical fiber bundles1401, as indicated by block1501, providing the plurality of lenses1402, as indicated by block1502, and providing the plurality of cameras1403, as indicated by block1503, together, providing a three-dimensional camera. Providing the plurality of optical fiber bundles1401, as indicated by block1501, comprises spacing-apart each optical fiber bundle distal end from another optical fiber bundle distal end and providing the plurality of lenses1402comprises spacing-apart each lens1402from another lens1402to provide a plurality of distinct views.

Still referring toFIG.15, in the method M1, providing the plurality of lenses1402, as indicated by block1502, comprises providing each lens1402of the plurality of lenses1402with a distinct field of view in relation to another lens1402of the plurality of lenses1402. The method M1further comprises providing a controller1405configured to: receive at least one image from each camera1403of the plurality of cameras1403; and combine the at least one image from each camera1403of the plurality of cameras1403into a single higher resolution image, as indicated by block1505. Providing the controller1405, as indicated by block1505, comprises further configuring the controller1405to remove dead pixels from the at least one image from each camera1403of the plurality of cameras1403. Providing the controller1405, as indicated by block1505, further comprises configuring the controller1405to provide a depth map by combining the at least one image from each camera1403of the plurality of cameras1403. Providing the controller1405, as indicated by block1505, further comprises configuring the controller1405to provide the depth map by using light field processing. Providing the plurality of optical fiber bundles1401, as indicated by block1501, comprises providing each optical fiber bundle1401with a diameter in a range of up to approximately 2 mm. Providing the common optical element1404, as indicated by block1504, comprises providing a body having a proximal end and a distal end. Providing the body comprises configuring the distal end of the body to accommodate the plurality of lenses1402. Providing the body comprises providing the proximal end of the body with a plurality of slots configured to correspondingly receive the plurality of optical fiber bundles1401, each slot of the plurality of slots correspondingly terminating at each lens of the plurality of lenses1402at the distal end of the body.

Referring toFIG.16is a flow diagram illustrates a method M2of imaging by way of an endoscope apparatus A, as shown inFIG.14, in accordance with an embodiment of the present disclosure. The method M2comprises: providing the apparatus A, as indicated by block1600, providing the apparatus A comprising: providing a plurality of optical fiber bundles1401, as indicated by block1601; providing a plurality of lenses1402in a one-to-one correspondence with the plurality of optical fiber bundles1401, providing the plurality of lenses1402comprising providing each lens1402of the plurality of lenses1402with a distinct depth of field and a distinct angle of view in relation to another lens1402of the plurality of lenses1402, as indicated by block1602; and providing a plurality of cameras1403in a one-to-one correspondence with the plurality of optical fiber bundles1401, as indicated by block1603; and operating the apparatus A by way of a controller, e.g., the controller1405, as indicated by block1606. The method M2further comprises: providing a common optical element1404operable with the plurality of lenses1402, providing the common optical element1404comprising disposing the common optical element1404at a common distal end of the plurality of optical fiber bundles1401, and providing the common optical element1404comprising providing the common optical element1404as at least one of removable and disposable, as indicated by block1604; and providing a controller configured to: receive at least one image from each camera1403of the plurality of cameras1403; and combine the at least one image from each camera1403of the plurality of cameras1403into a single higher resolution image, as indicated by block1605.

Hence, provided herein is a flexible endoscope that comprises of multiple optical fiber bundles which can each have about 18 kilopixels resolution, each coupled to a multi-lens array at a distal end and multiple cameras at a proximal end. Each lens on the array can convey a separate image to the distal end of each optical fiber bundle and cameras coupled to the proximal end of the optical fiber bundles acquire separate pixelated images. These lower resolution images, acquired by each of the cameras, can be merged and/or combined, and reconstructed using principles of light field imaging and processing, to produce a super-resolution image. This can allow for much higher resolution imaging than conventional endoscopes, which can allow for better diagnosis and treatment.

Furthermore, using light field processing of the separate images from the cameras, a depth-map of objects imaged by the lenses can be reconstructed, which can allow structures with differing depth to be more easily detected and/or seen. By taking advantage of the underlying optics of the method, omni-focusing (having all object in the scene in-focus), selective post-acquisition focusing, and depth of field control is possible post-acquisition and real-time. This post-processing can allow for removal of “dead” pixels which can be caused by broken fibers within fiber bundles without significant loss of detail. Using separate fiber bundles can also resolve flexibility issues associated with having one large fiber bundle since each smaller fiber bundle can move independently from one another and not seize when bent.

While devices and methods described herein have been described with respect to surgical and/or medical applications, devices and methods described herein can be used in other fields, such as in engineering and defense, particularly in scenarios where high-resolution three-dimensional imaging of a space and/or objects occurs through a small, (and even convoluted) access port. The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments may be susceptible to various modifications and alternative forms. The claims are not intended to be limited to the particular forms disclosed, but, rather, to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure.