Patent ID: 12204085

Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description; however, the description is not limited to the examples and/or implementations provided in the drawings.

DETAILED DESCRIPTION OF EXAMPLES

Disclosed herein are example systems and methods for rotating very small objects to facilitate three-dimensional imaging or modeling of such objects. Disclosed herein are example systems and methods that rotate objects for three-dimensional imaging or modeling with less complexity and cost. Disclosed herein are example systems and methods that apply a non-rotating nonuniform electric field so as to apply a dielectrophoretic torque to a three-dimensional object so as to rotate the three-dimensional object.

The example systems and methods facilitate rotation of the very small objects by suspending the very small objects in a fluid, such as a liquid. As a result, the system the methods are well suited for the imaging of objects where direct physical contact and direct manipulation the objects is difficult. The example systems and methods for rotating objects may employ as few as two spaced electrodes to form the nonrotating nonuniform electric field that creates a dielectrophoretic torque and that rotate the three-dimensional object.

The example systems and methods facilitate rotation of the very small objects by suspending the very small objects in a fluid, such as a liquid. As a result, the system the methods are well suited for the imaging of objects where direct physical contact and direct manipulation the objects is difficult. The example systems and methods for rotating objects may employ as few as to spaced electrodes to form the nonrotating nonuniform electric field that creates a dielectrophoretic torque and that rotate the three-dimensional object.

The example systems and methods for rotating objects facilitate rotation of the object about a rotational axis that is parallel to planes of the electrodes. In some implementations, this facilitates rotation of the objects about a rotational axis that is also parallel to a plane of a microfluidic chip, slide or a platform/stage containing the fluid that suspends the object during their rotation. The rotation of the object by the nonrotating nonuniform electric field facilitates rotation the object about the rotational axis that is perpendicular to an optical axis of the camera or imager capturing images of the object during his rotation. Because the rotation axis perpendicular to the optical axis, the overall imaging system may be more compact and less complex.

In some implementations, the three-dimensional object being rotated, imaged and modeled may comprise biological elements, such as cells. In some implementations, three-dimensional object being rotated may comprise a cellular object. For purposes of this disclosure, a cellular object comprises a 3D culture or an organoid. 3D cultures are cells grown in droplets or hydrogels that mimic a physiologically relevant environment. Organoids are miniature organs grown in a lab derived from stem cells and clusters of tissue, wherein the specific cells mimic the function of the organ they model. 3-D cultures and organaids may be used to study basic biological processes within specific organs or to understand the effects of particular drugs. 3-D cultures and organoids may provide crucial insight into mechanisms of cells and organs in a more native environment.

Disclosed herein is an example three-dimensional object modeling method. The method may include applying a nonrotating nonuniform electric field to apply a dielectrophoretic torque to a three-dimensional object to rotate the three-dimensional object. Images are captured of the object at different angles during rotation of the object. A three-dimensional model of the object is formed based on the captured images.

Disclosed is an example three-dimensional object modeling system. The system may include a first electrode, a second electrode, a power supply connected to the first electrode and the second electrode, a camera and a controller. The controller may output control signals controlling the power supply such that the first electrode and the second electrode cooperate to apply a nonrotating nonuniform electric field to an object suspended in the fluid so as to rotate the object. The controller may further output control signals controlling the camera to capture images of the object at different angles during rotation of the object, wherein the controller is to form a three-dimensional model of the object based on the captured images.

Disclosed herein is an example cellular object rotation system for use with a cellular object imaging system. The cellular object rotation system may include a first electrode, a second electrode, a power supply connected to the first electrode and the second electrode and a controller to output control signals controlling the power supply such that the first electrode and the second electrode cooperate to apply a nonrotating nonuniform electric field to a cellular object suspended in the fluid so as to rotate the object.

FIG.1schematically illustrates portions of an example object rotation system20for use with a three-dimensional imaging system. In one implementation, the object rotation system20comprises a cellular object rotation system for use with a cellular object imaging system. Object rotation system20facilitates low cost and less complex rotation of objects, such as cellular objects, as such object are being imaged for three-dimensional modeling. Object rotation system20comprises electrodes60, power supply61and controller62.

Electrodes60comprise a pair of spaced electrodes that cooperate to form a nonrotating nonuniform electric field through a cellular object suspension region50. The cellular object suspension region50comprises a volume of fluid54in which a three-dimensional object, such as a cellular object52is suspended. In the example illustrated, electrodes60comprise a pair of electrodes located on one side of the cellular object52. In implementations where imaging is performed through the plane or planes containing electrodes60, such electrodes60may be formed from a transparent electrically conductive material such as indium tin oxide. In other implementations, electrodes may be formed from other electrically conductive materials. In one implementation electrodes62each comprise a flat planar electrode, wherein the electrodes60are coplanar. As a result, object rotation system20may be more compact.

Power supply61comprise a source of power for allegedly charging at least one of electrodes60. In one implementation, power supply61supplies power to electrodes60under the control of controller62.

Controller62comprises a processing unit that follows instructions contained in a non-transitory computer-readable medium. In one implementation, controller62may comprise an application-specific integrated circuit. In one implementation, controller62serves as a signal generator controlling the frequency and voltage of the nonrotating nonuniform electric field.FIG.2is a schematic diagram illustrating the application of the nonrotating nonuniform electric field to the example cellular object52. As indicated by arrow63, the electric field applies a dielectrophoretic torque to object52so as to rotate object52about a rotational axis65. Such rotation facilitates the capturing of images of the cellular object52at different angles to facilitate three-dimensional reconstruction or modeling of cellular object52for analysis. In one implementation, controller62outputs control signals such that electrodes60apply a sinusoidal nonrotating nonuniform alternating current electric field having a frequency of at least 30 kHz and no greater than 500 kHz. In one implementation, the nonrotating nonuniform electric field has a voltage of at least 0.1 V rms and no greater than 100 V rms. Between taking consecutive images, the cellular object must have rotated a distance that at least equals to the diffraction limit dlim of the imaging optics. The relationship between minimum rotating angle θmin, radius r and diffraction limit distance dlim is θmin=dlim/r. For example, for imaging with light of λ=500 nm and a lens of 0.5 NA, the diffraction limit dlim=λ/(2NA)=500 nm. In the meanwhile, the cellular object cannot rotate too much that there is no overlap between consecutive image frames. So the maximum rotating angle between consecutive images θmax=180−θmin.

In one implementation, the nonuniform nonrotating electric field produces a dielectrophoretic torque on the cellular object so as to rotate the cellular object at a speed such that an image may be captured every 2.4 degrees while producing output in a reasonably timely manner. In one implementation where the capture speed of the imager is 30 frames per second, the produced dielectrophoretic torque rotates the cellular object at a rotational speed of at least 12 rpm and no greater than 180 rpm. In one implementation, the produced dielectrophoretic torque rotates the cellular object at least one pixel shift between adjacent frames, but where the picture shift is not so great so as to not be captured by the imager280. In other implementations, cellular object52may be rotated at other rotational speeds.

FIG.3illustrates portions of another example three-dimensional object rotation system120. As with object rotation system20, object rotation system120is well-suited for rotating very small objects while such small objects are suspended in a fluid. As with object rotation system20, object rotation system120well-suited for rotating cellular objects as such objects are being imaged to form three-dimensional reconstructions or models of the cellular objects. System120is similar to system20described above except that system120comprises an alternative arrangement having electrodes160in lieu of electrodes60. Those remaining components of system120which correspond to components of system20are numbered similarly inFIG.3or are shown inFIG.1.

Electrodes160comprise a pair of electrodes that cooperate to form a nonrotating nonuniform electric field through the cellular object suspension region50. In the example illustrated, electrodes160comprise a pair of electrodes located on opposite sides of the cellular object52with the electrodes160facing one another. In implementations where imaging of the rotating cellular object is with an imager or having an optical axis that passes through or intersects either of the two electrodes, such electrodes may be formed from a transparent or translucent electrically conductive material such as indium tin oxide. In other implementations, electrodes160may form from other electrically conductive materials.

In operation, object rotation system120may perform in a similar fashion as compared object rotation system20. Controller62(shown inFIG.1) outputs control signals or generate signals such that electrodes160apply a nonrotating nonuniform electric field about the suspended object, shown as cellular object52. In one implementation, controller162outputs control signals such that electrodes160apply a sinusoidal nonrotating nonuniform alternating current electric field having a frequency of at least 30 kHz and no greater than 500 kHz. In one implementation, the nonrotating nonuniform electric field has a voltage of at least 0.1 V rms and no greater than 100 V rms.

The electric field is applied such that it applies a dielectrophoretic torque to the object52so as to rotate the object52as indicated by arrow163about an axis165. The rotational axes165of object52is parallel to the slide, stage or platform containing the fluid and may be perpendicular to the optical axis of the image or camera capturing images of object52at different angular positions. Because the rotation axis is perpendicular to the optical axis, the overall imaging system may be more compact and less complex.

FIG.4schematically illustrates portions of an example three-dimensional object modeling system210. Modeling system210facilitates the rotation of an object, such as a cellular object, as the object is being imaged to facilitate the output of a three-dimensional reconstruction or model of the object. Modeling system210facilitates such rotation of the object with an architecture that is less complex and more compact, potentially reducing cost. Modeling system210comprises object rotation system220, controller270and imager280, in the form of a camera.

Object rotation system220is similar to object rotation system20described above except that controller270controls power supply61and imager280. Controller270comprises processing unit272and a non-transitory computer readable medium in the form of memory274. Processing unit272follows instructions contained in memory274. Memory274contains instructions that direct processing unit272to control the operation of electrodes60,160and imager280. For example, controller270outputs control signals controlling the rate at which so the object52is rotated during imaging. As with controller62, controller270serves as a signal generator controlling the frequency and voltage of the nonrotating nonuniform electric field such as shown inFIG.2or3. The electric field applies a dielectrophoretic torque to object52so as to rotate object52about a rotational axis65. Such rotation facilitates the capturing of images of the cellular object52at different angles to facilitate three-dimensional reconstruction or modeling of cellular object52for analysis. In one implementation, controller270outputs control signals such that electrodes60,160apply a sinusoidal nonrotating nonuniform alternating current electric field having a frequency of at least 30 kHz and no greater than 500 kHz. In one implementation, the nonrotating nonuniform electric field has a voltage of at least 0.1 V rms and no greater than 100 V rms. Between taking consecutive images, the cellular object must have rotated a distance that at least equals to the diffraction limit dlimof the imaging optics. The relationship between minimum rotating angle θmin, radius r and diffraction limit distance dlimis θmin=dlim/r. For example, for imaging with light of λ=500 nm and a lens of 0.5 NA, the diffraction limit dlim=λ/(2NA)=500 nm. In the meanwhile, the cellular object cannot rotate too much that there is no overlap between consecutive image frames. So the maximum rotating angle between consecutive images θmax=180−θmin.

In one implementation, the nonuniform nonrotating electric field produces a dielectrophoretic torque on the cellular object so as to rotate the cellular object at a speed such that an image may be captured every 2.4 degrees while producing output in a reasonably timely manner. In one implementation where the capture speed of the imager is 30 frames per second, the produced dielectrophoretic torque rotates the cellular object at a rotational speed of at least 12 rpm and no greater than 180 rpm. In one implementation, the produced dielectrophoretic torque rotates the cellular object at least one pixel shift between adjacent frames, but where the picture shift is not so great so as to not be captured by the imager280. In other implementations, cellular object52may be rotated at other rotational speeds.

Imager280comprise at least one camera to capture images of the rotating cellular object52at different angles during rotation of cellular object52about axis65. Imager280has an optical axis282which is perpendicular to axis65. In one implementation, imager280may comprise multiple cameras. Imager280captures images of the rotating cellular object52at different angular positions during his rotation to facilitate subsequent three-dimensional image reconstruction of the cellular object52as will be described hereafter. In one implementation imager280may comprise a camera having an optical lens282facility microscopic viewing and imaging of cellular object52.

In addition to outputting control signals to electrodes60,160so as to create the nonrotating nonuniform electric field that rotates cellular object52, controller270may additionally control imager280. Controller270receives images or image signals from imager280. Based upon the different images of the rotating cellular object52captured at different rotational angles, controller270triangulates identified points of the image to form a three-dimensional reconstruction or model290of cellular object52for analysis.

FIG.5is a flow diagram of an example three-dimensional object modeling method300. Modeling method300captures images of an object rotated with a nonrotating nonuniform electric field. Modeling method300may be implemented with less complex and lower-cost components. Although method300is described in the context of being carried out by three-dimensional object modeling system210having object rotation system220, it should be appreciated that method300may likewise be carried out with other similar modeling systems and other similar object rotation systems, such as object rotation systems20or120.

As indicated by block304, a nonrotating nonuniform electric field is applied so as to apply a dielectrophoretic torque to a three-dimensional object, such as a cellular object, to rotate the three-dimensional object. In one implementation, a sinusoidal nonrotating nonuniform alternating current electric field having a frequency of at least 30 kHz and no greater than 500 kHz is applied to the object or the object suspended in a fluid. In one implementation, the nonrotating nonuniform electric field has a voltage of at least 0.1 V rms and no greater than 100 V rms.

As indicated by block308, controller270outputs control signals causing camera282capture images of the object52at different angles during rotation of object52. As indicated by block312, upon receiving the captured images from imager280, controller270formed a three-dimensional reconstruction or model of the object52based upon the captured images.

FIG.6is a flow diagram of an example three-dimensional volumetric modeling method500that may be carried out by controller470using captured two-dimensional images of the rotating object52. As indicated by block504, a controller, such as controller470, receives video frames or two-dimensional images captured by the imager/camera60during rotation of object52. As indicated by block508, various preprocessing actions are taken with respect to each of the received two-dimensional image video frames. Such preprocessing may include filtering, binarization, edge detection, circle fitting and the like.

As indicated by block514, utilizing such edge detection, circle fitting and the like, controller470retrieves and consults a predefined three-dimensional volumetric template of the object52, to identify various internal structures of the object are various internal points in the object. The three-dimensional volumetric template may identify the shape, size and general expected position of internal structures which may then be matched to those of the two-dimensional images taken at the different angles. For example, a single cell may have a three-dimensional volumetric template comprising a sphere having a centroid and a radius. The three-dimensional location of the centroid and radius are determined by analyzing multiple two-dimensional images taken at different angles.

Based upon a centroid and radius of the biological object or cell, controller470may model in three-dimensional space the size and internal depth/location of internal structures, such as the nucleus and organelles. For example, with respect to cells, controller470may utilize a predefined template of a cell to identify the cell wall and the nucleus As indicated by block518, using a predefined template, controller470additionally identifies regions or points of interest, such as organs or organelles of the cell. As indicated by block524, controller470matches the centroid of the cell membrane, nucleus and organelles amongst or between the consecutive frames so as to estimate the relative movement (R, T) between the consecutive frames per block528.

As indicated by block534, based upon the estimated relative movement between consecutive frames, controller470reconstructs the centroid coordinates in three-dimensional space. As indicated by block538, the centroid three-dimensional coordinates reconstructed from every two frames are merged and aligned. A single copy of the same organelle is preserved. As indicated by block542, controller470outputs a three-dimensional volumetric parametric model of object52.

FIGS.7-11illustrate one example modeling process600that may be utilized by 3-D modeler70or controller470in the three-dimensional volumetric modeling of the biological object.FIG.7-11illustrate an example three-dimensional volumetric modeling of an individual cell. As should be appreciated, the modeling process depicted inFIGS.7-11may likewise be carried out with other biological objects.

As shown byFIG.6, two-dimensional video/camera images or frames604A,604B and604C (collectively referred to as frame604) of the biological object52(schematically illustrated) are captured at different angles during rotation of object52. In one implementation, the frame rate of the imager or camera is chosen such as the object is to rotate no more than 5° per frame by no less than 0.1°. In one implementation, a single camera captures each of the three frames during rotation of object52(schematically illustrated with three instances of the same camera at different angular positions about object52) in other implementations, multiple cameras may be utilized.

As shown byFIGS.7and8, after image preprocessing set forth in block508inFIG.5, edge detection, circle fitting another feature detection techniques are utilized to distinguish between distinct structures on the surface and within object52, wherein the structures are further identified through the use of a predefined template for the object52. For the example cell identify the cell, controller470identifies wall608, its nucleus610and internal points of interest, such as cell organs or organelles612in each of the frames (two of which are shown byFIGS.7and8).

As shown byFIG.9and as described above with respect to blocks524-538, controller470matches a centroid of a cell membrane, nucleus and organelles between consecutive frames, such as between frame604A and604B. Controller470further estimates a relative movement between the consecutive frames, reconstructs a centroid's coordinates in three-dimensional space and then utilizes the reconstructed centroid coordinates to merge and align the centroid coordinates from all of the frames. The relationship for the relative movement parameters R and T is derived, assuming that the rotation axis is kept still, and the speed is constant all the time. Then, just the rotation speed is utilized to determine R and T ({right arrow over (0102)}) as shown inFIG.9, where:

O1⁢O2→=OO1→·Rθ-OO1→Rθ=Ry⁡(θ)=[cos⁢⁢θ0sin⁢⁢θ011-sin⁢⁢θ1cos⁢⁢θ]
based on the following assumptions:θ is constant;|{right arrow over (001)}|=|{right arrow over (002)}|=|{right arrow over (003)}|=. . . ;rotation axis doesn't change (along y axis); and{right arrow over (001)} is known.
As shown byFIG.10, the above reconstruction by controller470results in the output of a parametric three-dimensional volumetric model of the object52, shown as a cell.

Although the present disclosure has been described with reference to example implementations, workers skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the claimed subject matter. For example, although different example implementations may have been described as including features providing one or more benefits, it is contemplated that the described features may be interchanged with one another or alternatively be combined with one another in the described example implementations or in other alternative implementations. Because the technology of the present disclosure is relatively complex, not all changes in the technology are foreseeable. The present disclosure described with reference to the example implementations and set forth in the following claims is manifestly intended to be as broad as possible. For example, unless specifically otherwise noted, the claims reciting a single particular element also encompass a plurality of such particular elements. The terms “first”, “second”, “third” and so on in the claims merely distinguish different elements and, unless otherwise stated, are not to be specifically associated with a particular order or particular numbering of elements in the disclosure.