Image sensor system

In one example, an image sensor module comprises one or more covers having at least a first opening and a second opening, a first lens mounted in the first opening and having a first field of view (FOV) centered at a first axis having a first orientation, a second lens mounted in the second opening and having a second FOV centered at a second axis having a second orientation different from the first orientation, a first image sensor housed within the one or more covers and configured to detect light via the first lens, and a second image sensor housed within the one or more covers and configured to detect light via the second lens. The first image sensor and the second image sensor are configured to provide, based on the detected light, image data of a combined FOV larger than each of the first FOV and the second FOV.

FIELD

This application is generally related to image sensor, and more specifically to techniques for extending field of view (FOV) of image sensor.

BACKGROUND

An image sensor can detect and convey information used to make an image. The image sensor can capture light reflected from a scene and convert the captured light into signals. The light can include, for example, visible light, infra-red light, etc. The signals can be used to generate an image of the scene to support various applications, such as depth-sensing, location tracking, augmented reality (AR)/virtual reality (VR)/mixed reality (MR) applications, etc.

One important performance metric of an imaging system is an extent of a scene that can be imaged by the image sensor. The extent can be affected by various parameters, such as a field of view (FOV) of the image sensor, which measures an angular extent of the scene that can be imaged by the image sensor. FOV can be used interchangeably with angle of view (AOV). It is desirable that an image sensor has a wide FOV, such that the image sensor can image a larger area of a scene and can capture more information about the scene in an image.

SUMMARY

Techniques are described for improving a field of view of an image sensor. The techniques can also be used to improve a field of illumination of an illuminator.

In some embodiments, an apparatus comprises: one or more covers having at least a first opening and a second opening; a first lens mounted in the first opening and having a first field of view (FOV) centered at a first axis having a first orientation; a second lens mounted in the second opening and having a second FOV centered at a second axis having a second orientation different from the first orientation; a first image sensor housed within the one or more covers and configured to detect light via the first lens; and a second image sensor housed within the one or more covers and configured to detect light via the second lens. The first image sensor and the second image sensor are configured to provide, based on the detected light, image data of a combined FOV larger than each of the first FOV and the second FOV.

In some aspects, the apparatus further comprises a support structure having a first surface perpendicular to the first axis and a second surface perpendicular to the second axis. The first image sensor is formed on the first surface. The second image sensor is formed on the second surface.

In some aspects, the apparatus further comprises a first circuit board on which the first image sensor and the second image sensor are formed. The first circuit board is bonded with the first surface and the second surface of the support structure.

In some aspects, the first circuit board is bonded with the first surface and the second surface of the support structure with an epoxy material.

In some aspects, the apparatus further comprises a second circuit board. Two ends of the first circuit board are bonded with the second circuit board such that the support structure is sandwiched between the first circuit board and the second circuit board.

In some aspects, the two ends of the first circuit board comprise first pads. The second circuit board comprises second pads. The first pads at the two ends of the first circuit board are soldered to the second pads of the second circuit board.

In some aspects, each of the first circuit board and the second circuit board includes a rigid-flex circuit board.

In some aspects, the apparatus further comprises a processor bonded with the second circuit board and electrically connected to the first image sensor and the second image sensor. The processor is configured to: receive a first image frame from the first mage sensor corresponding to the first FOV; receive a second image frame from the second image sensor corresponding to the second FOV; and generate a combined image frame corresponding to the combined FOV based on the first image frame and the second image frame.

In some aspects, the processor is configured to generate the combined image frame based on identifying pixels of an object captured in both the first image frame and the second image frame.

In some aspects, the processor is sandwiched between the support structure and the second circuit board.

In some aspects, the processor is electrically connected to the second circuit board via at least one of: flip chip connectors, or bond wires.

In some aspects, the one or more covers include one cover mounted on the second circuit board.

In some aspects, the one or more covers include a first cover having the first opening and a second cover having the second opening. The first cover is formed on the first surface of the support structure. The second cover is formed on the second surface of the support structure.

In some aspects, the apparatus further comprises a third cover having a transparent lid. The third cover is formed on the second circuit board. The transparent lid allows light to reach the first lens and the second lens.

In some aspects, the apparatus further comprises an illuminator configured to emit the light.

In some embodiments, a method of fabricating an image sensor module is provided. The method comprises: forming electrical connections between a processor and a first circuit board; bonding a support structure onto a surface of the processor; placing a first image sensor and a second image sensor on a second circuit board; bonding the second circuit board comprising the first and second image sensors on a first surface and a second surface of the support structure, the first surface and the second surface having different orientations; forming electrical connections between the first circuit board and the second circuit board bonded to the support structure; and placing a cover having a first lens and a second lens on the first circuit board to enclose the first and second image sensors, the first lens having a first field of view (FOV) centered at a first axis perpendicular to the first surface and the second lens having a second FOV centered at a second axis perpendicular to the second surface.

In some aspects, the electrical connections between the processor and the first circuit board comprise at least one of: flip-chip connections, or bond wires. The electrical connections between the first circuit board and the second circuit board comprise at least one of: flip-chip connections, or bond wires.

In some aspects, support structure is bonded onto the surface of the processor based on an epoxy material. Forming electrically connections between the first circuit board and the second circuit board bonded to the support structure comprises using a hot bar to simultaneously solder first pads on two ends of the second circuit board onto second pads on the first circuit board.

In some aspects, each of the first circuit board and the second circuit board include a rigid-flex circuit board.

DETAILED DESCRIPTION

As described above, one important performance metric of an imaging system is a field of view (FOV) of the image sensor, which measures an angular extent of the scene that can be imaged by the image sensor. An image sensor module typically includes a lens to focus incident light onto an image sensor, and the FOV/AOV of the image sensor can be increased by, for example, increasing the aperture size of the lens, reducing the focal length of the lens. Increasing the aperture size and/or reducing the focal length can also increase optical aberrations, such as Seidel aberrations, which can increase blurriness and reduce the resolution of the imaging operation.

Disclosed are techniques that can improve the field of view of an image sensor. In one example, an imaging module may include one or more covers having at least a first opening and a second opening. The imaging module may include a first lens mounted in the first opening and having a first field of view (FOV) centered at a first axis having a first orientation, and a second lens mounted in the second opening and having a second FOV centered at a second axis having a second orientation different from the first orientation. The imaging module may further include a first image sensor housed within the one or more covers and configured to detect light via the first lens and a second image sensor housed within the one or more covers and configured to detect light via the second lens. The first image sensor and the second image sensor are configured to provide, based on the detected light, image data of a combined FOV larger than each of the first FOV and the second FOV. In some examples, the imaging module may include more than two image sensors arranged in different orientations to further enlarge the combined FOV.

By arranging two or more image sensors in different orientations to detect light, and by combining the image data provided by the image sensors, the combined FOV can become larger than the individual FOV provided by each image sensor. The widening of the FOV also does not require increasing the aperture of each of the first lens and the second lens, which allows the FOV to be widened without incurring additional optical aberrations. The resolution of the imaging operation can be improved as result.

The image module can support various applications, such as a ranging application. For example, the image module can be integrated with an illuminator to project infra-red light (e.g., light pulses, structured light carrying specific patterns, etc.) onto a scene. The image module can capture the infra-red light reflected by one or more objects in the scene. A distance between the image module and the objects can be determined based on, for example, a time-of-flight of the infra-red light pulses, orientations and/or locations of the structured infra-red light, etc.

FIG. 1illustrates an image sensor module100according to embodiments of the present disclosure. Image sensor module100can be part of a mobile device such as, for example, a smart phone, a laptop, a camera, an Internet-of-Thing (IoT) device, etc. As shown inFIG. 1, sensor module100includes an image sensor102housed within a cover104. Cover104can be made of a polymer material to provide physical protection and insulation to image sensor102. Cover104may include an opening106in which a lens108can be mounted. Image sensor102can be configured to detect light110that passes through lens108. Image sensor module100may further include a cover glass112mounted on a light receiving surface114of image sensor102to protect the image sensor. In some examples, image sensor module100can further include an optical filter array (e.g., a Bayer filter array) to control the wavelength of light received by each pixel cells.

Image sensor102may include a pixel cell array formed below light receiving surface114. Each pixel cell within the array can pixel data representing an intensity of light110received by the pixel cell. The pixel data from the pixel cells can provide an image of a scene. Sensor module100further includes a processor120to process the pixel data for different applications. For example, processor120can operate an imaging application and can reconstruct the image of the scene based on the pixel data. Processor120can also operate a computer vision (CV) application, a machine learning (ML) application, etc. to analyze the image for various other applications, such as object detection and identification, performing ranging operation, tracking a location of the device that includes the sensor module, etc. In some examples, sensor102and processor120can be combined into the same chip (e.g., housed within the same package, monolithically integrated on the same substrate, etc.).

As shown inFIG. 1, to reduce a horizontal footprint (e.g., on the x-y plane) of image sensor module100, image sensor102and processor120(as well as cover glass112) can be arranged to form a vertical stack of devices (e.g., along the z-axis). Image sensor module100may include a circuit board130to provide electrical connections to image sensor102and processor120in the stack. For example, processor120can include flip-chip connectors (e.g., flip-chip connectors132), bond wires, etc., which can be soldered onto pads134of circuit board130. Circuit board130can include a rigid flex printed circuits board. Moreover, image sensor module100may include bond wires136aand136bwhich can be soldered onto pads138aand138bof circuit board130to provide electrical connections between image sensor102and circuit board130. Circuit board130can include circuitries to provide electrical connections between pads134and138to enable communication between image sensor102and processor120. Cover104can be mounted on circuit board130to enclose image sensor102and processor120to form a chip package. Circuit board130may include connectors140to provide electrical connections between image sensor module100and other components of the mobile device (e.g., power supply).

As shown inFIG. 1, image sensor module100can provide a field-of-view (FOV)150for imaging. FOV150can have an angle of θ around an axis160perpendicular to circuit board130. InFIG. 1, axis160can align with, for example, the z-axis when circuit board130is parallel with x-y plane. FOV150can be determined based on the geometric properties of lens108as well as the dimension of light receiving surface114of image sensor102.FIG. 2Aprovides an illustration of determination of FOV. As shown inFIG. 2A, with lens108having a focal length f, light receiving surface114having a dimension h on the horizontal plane (e.g., x-y plane), the FOV angle θ on the horizontal plane can be determined based on the following equation:

In Equation 1, arctan is inverse of the tangent function. FOV angle θ can be centered around principle axis202of lens108, which also goes through the center of lens108.

The FOV can be defined on different planes. In the example ofFIG. 2A, FOV angle θ can be a horizontal FOV (HOV) on the horizontal x-y plane. Referring toFIG. 2B, a vertical FOV (VFOV) can be defined on a vertical plane (e.g., on x-z or y-z plane), whereas a diagonal FOV (DFOV) can be defined on a diagonal plane formed on a diagonal axis of an image plane210that is parallel with the z-y plane. In a case where light receiving surface114of image sensor102is parallel with image plane210, the FOV angle θ of image sensor module100inFIG. 1can be an HOV.

As described above, it is desirable that image sensor102has a wide FOV, such that image sensor102can image a larger area of a scene and can capture more information about the scene in an image. Referring back to Equation 1, one way to increase the FOV is by decreasing the focal length of lens108, but doing so can increase optical aberrations, such as Seidel aberrations, which can increase blurriness and reduce the resolution of the imaging operation.

FIG. 3Aillustrates an example of an image module300which can provide a widened FOV based on combining FOVs of multiple lens. Image sensor module300can be part of a mobile device such as, for example, a smart phone, a laptop, a camera, an Internet-of-Thing (IoT) device, etc. As shown inFIG. 3A, sensor module300includes a cover304mounted on circuit board130. Cover304can have side surfaces306aand306beach forms an angle α with respect to circuit board130. Side surface306aincludes an opening308afor mounting a lens108a, whereas side surface306bincludes an opening308bfor mounting a lens108b. Side surfaces306aand306b(and/or openings308aand308b) are configured such that the principle axis202aof lens108ahas a different orientation from the principle axis202bof lens108b.

Image module300further includes image sensors102aand102bpositioned below, respectively, lens108aand108b. Image sensors102aand102bcan be oriented such that each is parallel with, respectively, side surfaces306aand306band forms angle α with respect to circuit board130. With such arrangements, light receiving surfaces140aand140bof image sensors102aand102bare perpendicular to, respectively, principle axes202aand202bof lens108aand lens108b. Image sensors102aand102bcan be supported on, respectively, surfaces310and312of a support structure314, which can be of a triangular shape, a prism shape, or other arbitrary shape. Support structure314can include materials such as polymer, glass, or other suitable material. Image sensor module300may further include a cover glass112aand a cover glass112bmounted on light receiving surfaces114aand114bto protect the image sensors. Image sensor module300may further include a filter array (not shown inFIG. 3A) to control the wavelength of light received by each pixel cell of the image sensors. Image module300further includes processor120sandwiched between support structure314and circuit board130to form a stack structure and to reduce the foot print of image module300.

Image sensor102acan detect light that passes through lens108a, which can provide an FOV150afor image sensor102aon surface310to generate pixel data. Image sensor102bcan light that passes through lens108b, which can provide an FOV150bfor image sensor102bon surface312to generate pixel data. In a case where image sensor module300is mounted on a vertical image plane (e.g., z-y plane, z-x plane, etc.), both FOVs150aand150bcan be horizontal FOVs. Processor120can combine the image data from image sensors102aand102bto generate a combined image having a combined FOV of FOVs150aand150b. The combined FOV can be wider than each of FOVs150aand150b.

FIG. 3Billustrates an example of widening of a FOV (e.g., HFOV) based on combining FOV150aand FOV150b. For example, referring toFIG. 3B, each of FOVs150aand150bmay have an angle θ centered around, respectively, principle axes202aand202b. In some examples, θ can have a range between 72-100 degrees.

Side surfaces306aand306b(and principle axes202aand202b) are oriented such that FOVs150aand150bhas an overlap angle t. In some examples, the overlap angle t can have a range between 6-44 degrees.

A combined FOV330, formed by combining the pixel data output by image sensors102aand102b, can have an angle θcombinebased on the following equation:
θcombine=2×θ−t(Equation 2)

The above Equation can be based on an assumption that the image plane being viewed is far compared to the distance between sensors306aand306b. In a case where θ is 72 degrees and t is 6 degrees, a combined FOV of 138 degrees (72×2−6) can be achieved. In a case where θ is 100 degrees and t is 44 degrees, a combined FOV of 156 degrees (100×2−44) can be achieved.

Processor120can post-process an image frame from each of image sensors102aand102b, each corresponding to respectively FOV150aand FOV150b, to generate a combined image frame corresponding to a combine FOV330. For example, processor120can identify pixels of an object that is captured in both image frames to be in the overlapped angle t of combined FOV330, while the rest of the pixels are in the non-overlapped portions of combined FOV330. Processor120can also perform transformation of the pixel data in the image frames from image sensors102aand102bto generate the combined image frame to account for the differences in the orientation of principle axis between, for example, image sensor102ofFIG. 1and image sensors102aand102bofFIG. 3A.

With such arrangements, each of image sensors102aand102bcan provide pixel data corresponding to, respectively, FOVs150aand150bto processor120, which can combine the pixel data to obtain an image corresponding to FOV330, which is wider than both of FOVs150aand150b. Moreover, each of lens108aand108bcan be identical to lens108ofFIG. 1and their geometric properties (e.g., focal lengths) need not be adjusted to widen the FOV, which enables increasing the overall FOV of image module300without introducing additional optical aberrations. AlthoughFIG. 3AandFIG. 3Billustrates that image sensor module300provides a widened HFOV, it is understood that the techniques disclosed in this disclosure can be applied to widen VFOV, DFOV, etc., based on adjusting the orientations of side surfaces306aand306b, which can set the orientations of the principle axes of lens108aand108b.

The orientations of side surfaces306aand306b, as well as the principle axes202aand202bof lens108aand108b, can be configured based on a target combined FOV330. For example, as shown inFIG. 3B, each of side surfaces306aand306bcan form an angle α with respect to x axis. Angle α can be chosen to set the orientations of the right boundary of FOV150aand the left boundary of FOV150b, which in turn set the overlap angle t between the right boundary of FOV150aand the left boundary of FOV150b.

Given the angle θ of FOVs150aand150band a target FOV330, the overlap angle t can be determined from Equation 2 above, and angle α can be set based on overlap angle t.FIG. 3Cillustrates example relationship between angle α and overlap angle t. As shown on the left ofFIG. 3C, with overlap angle t, the right boundary of FOV150aforms an angle of t/2 with respect to axis332which can be perpendicular to the x-y plane, and an angle of 90−t/2 with respect to x-y plane. Moreover, as shown on the right ofFIG. 3C, the right boundary of FOV150aalso forms an angle of 90−θ/2 with respect to surface306a, which forms an angle of α with respect to the x-y plane. Therefore, the right boundary of FOV150acan form an angle of 90−θ/2+α with respect to the x-y plane. A relationship between angle t and α can be based on the following equation:
t/2=θ/2−α  (Equation 3)

In a case where θ is 72 degrees and t is 6 degrees, α can be equal to 33 degrees. In a case where θ is 100 degrees and t is 44 degrees, α can be equal to 28 degrees.

Moreover, the size of angle α can also be constrained such that there is at least some overlap between the right boundary of FOV150aand the left boundary of FOV150b(e.g., t has to be at least zero or positive).

Referring back toFIG. 3A, image module300may include a circuit board340as well as circuit board130to provide electrical connections to image sensors102aand102bas well as processor120. Circuit board340can also include a rigid flex circuit board and can be bended to conform to the shape of support structure314. Circuit board340can be bonded to surfaces310and312with, for example, epoxy materials. Sensors102aand102bcan be electrically connected to circuit board340via, respectively, bond wires350aand350b. In some examples, sensors102aand102bcan also be electrically connected to circuit board340via flip chip. Two ends of circuit board340may include pads360aand360bthat can be soldered to pads138aand138bof circuit board130, such that support structure314becomes sandwiched between circuit board130and circuit board340.

Moreover, processor120can include flip-chip connectors (e.g., flip-chip connectors132), bond wires, etc., which can be soldered onto pads134of circuit board130. Circuit board130can include circuitries to provide electrical connections between pads134and138to enable communication between image sensor102and processor120. Cover304can be mounted on circuit board130to enclose image sensor102and processor120. Circuit board130may include connectors140to provide electrical connection among sensors102aand102b, processor120and other components of the mobile device (e.g., power supply).

AlthoughFIG. 3AandFIG. 3Billustrates that two lens108aand108bare used to provide a widened FOV, it is understood that more than two lens can be used.FIG. 4illustrates an example of an image module400including more than two lens. As shown inFIG. 4, image module400may include a cover402and include multiple steps including steps406a,406b, . . .406N. Each step can hold a lens108(e.g., lens108a,108b, . . .108N). Each lens have a FOV150(e.g., FOV150a,150b, . . .150N) each has an angle of θ. Each step406and lens108corresponds to an image sensor102(e.g., image sensors102a,102b,102N) positioned on a surface410(e.g.,410a,410b, . . .410N) of multi-faced support structure420. Image module400can provide a combined FOV equal to the sum of the FOVs of the lens minus the total overlap angles between pairs of neighboring lens (e.g., lens108aand108b, lens108band108c, etc.).

FIG. 5AandFIG. 5Billustrate other examples of an image module that can widen FOV. Image module500includes mostly the same components as image module300except that image module500includes two covers502and504mounted on circuit board340to cover, respectively, image sensors102aand102b, to form a dual sensor package. Circuit board340in turn is mounted on support structure314so that image sensor102aand102bhave different orientations to widen the combined FOV. Cover502includes an opening506to mount lens108a, whereas cover504includes an opening508to mount lens108b. In some examples, as shown inFIG. 5B, image module500can further include a lens carrier510having a transparent lid512mounted on circuit board130to protect the dual sensor package.

FIG. 6illustrates another example of an image module600. Image module600includes mostly the same components as image module300, including image sensors102aand102bmounted on circuit board340, which in turn is mounted on support structure314so that image sensor102aand102bhave different orientations to widen the combined FOV. Support structure314is mounted on circuit board130. In addition, image module600can include an illuminator602mounted on circuit board130. Illuminator602can emit, for infra-red light604which can include, for example, light pulses, structured light, etc. Illuminator602can also emit light of other wavelength ranges, such as monochrome visible light, visible light of a particular color (e.g., one of red/green/blue), etc.

In some examples, processor120can synchronize/coordinate the operation of illuminator602with the operation of image sensors102aand102bto perform a ranging operation. For example, in a case where illuminator602emits infra-red light pulses, processor120can set a first time when illuminator602emits an infra-red light pulse, and then determine a second time (and/or a third time) when each of image sensors102aand102breceives a reflected infra-red light pulse from an object. A time-of-flight of the infra-red light pulse can then be determined based on the difference between the first time and the second time (and third time), and the time-of-flight can be used to determine a distance between the image module and the object. As another example, processor120can control illuminator602to output structured infra-red light604that carries a specific pattern. From the outputs of image sensors102aand102b, processor120can determine orientations and/or locations of images of the infra-red light pattern, and determine a distance between the image module and the object based on the orientations and/or locations of infra-red light pattern image.

Image module600can include a cover606which includes openings308aand308bto mount, respectively, lens108aand108b. In addition, cover606further includes an opening608to let infra-red light604out of image module600. In some examples, a lens or other optical components (not shown inFIG. 6) can be mounted in opening608to set the property of infra-red light604such as, for example, focusing the light, filtering out components other than infra-red light, etc. In some examples, illuminator602can be external to cover606. In some examples, image module600may also include multiple illuminators formed on, for example, surfaces310and312of support structure314, and cover606can include additional openings on side surfaces306aand306bto let the light out of the illuminators.

FIG. 7AandFIG. 7Billustrate an example method700of fabricating an image sensor module, such as image sensor module300. Referring toFIG. 7A, in step702, electrical connections are formed between processor120and circuit board130. The electrical connection can be based on, for example, flip-chip, wire bonding, etc. Circuit board130can include a rigid flex circuit board.

In step704, support structure314is bonded with a surface of processor120to form a stack. Support structure314can be of a triangular shape, a prism-shape, or other arbitrary shape.

In step706, image sensors102aand102bare placed on circuit board340which can also include a rigid flex circuit board. Electrical connections are formed between each of image sensors102aand102band circuit board340. The electrical connections can be based on, for example, flip chip, wire bonding, etc.

In step710, circuit board340, which now includes image sensors102aand102b, can be bonded to surfaces310and312of support structure314. The bonding can be based on epoxy materials. Surfaces310and312have different orientations. Electrical connections can also be formed between circuit board340and circuit board130. The electrical connections can be formed by, for example, soldering pads360aand360bon two sides of circuit board340to, respectively, pads138aand138bof circuit board130. In some examples, the soldering of the two sides of circuit board340to circuit board130can be performed simultaneously using a hot bar. In some examples, the electrical connections can also be formed by bond wires between circuit board340and circuit board130.

In step712, cover304having lens108aand108bmounted in surfaces306aand306bcan be placed on circuit board130to enclose image sensors102aand102b. Lens108aand108bare oriented such that principle axis202aof lens108ais perpendicular to surface310and principle axis202bof lens108bis perpendicular to surface312.