The present disclosure relates to optical systems, vehicles, and methods that are configured to illuminate and image a wide field of view of an environment. An example optical system includes a camera having an optical axis and an outer lens element disposed along the optical axis. The optical system also includes a plurality of illumination modules, each of which includes at least one light-emitter device configured to emit light along a respective emission axis and a secondary optical element optically coupled to the at least one light-emitter device. The secondary optical element is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the optical system.

BACKGROUND

Conventional imaging systems can be configured to acquire electronic motion pictures. Such imaging systems can include video cameras, smartphones, web cams, digital single lens reflex (DSLR) cameras with video capabilities, etc.

In some scenarios, such imaging systems can be used in conjunction with artificial light sources. Such artificial light sources could serve as a primary or secondary light source in a given scene. Artificial light sources could include intermittent (e.g., an electronic flash unit) and/or continuous light sources (e.g., a light emitting diode panel).

SUMMARY

The present disclosure generally relates to imaging systems with illumination modules configured to provide properly-illuminated images over a wide field of view.

In a first aspect, an optical system is provided. The optical system includes at least one camera. The at least one camera includes an optical axis and an outer lens element disposed along the optical axis. The optical system also includes a plurality of illumination modules. Each illumination module includes at least one light-emitter device configured to emit light along a respective emission axis and a secondary optical element optically coupled to the at least one light-emitter device. The secondary optical element is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the optical system.

In a second aspect, a vehicle is provided. The vehicle includes a camera. The camera includes an optical axis and an outer lens element disposed along the optical axis. The vehicle also includes a first illumination module configured to illuminate a first portion of an environment of the vehicle and a second illumination module configured to illuminate a second portion of the environment of the vehicle. Each illumination module includes at least one light-emitter device configured to emit light along a respective emission axis. The illumination modules also include a secondary optical element optically coupled to the at least one light-emitter device. The secondary optical element is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the vehicle.

In a third aspect, a method is provided. The method includes causing at least one light-emitter device of at least one illumination module of a vehicle to emit light into an environment according to a light emission pattern. The at least one light-emitter device is configured to emit light along an emission axis. The at least one light-emitter device is optically coupled to a secondary optical element. The secondary optical element is configured to interact with the emitted light so as to provide a light emission pattern. The light emission pattern includes an azimuthal angle extent of at least 170 degrees. The method also includes causing a camera of the vehicle to capture at least one image of the portion of the environment. The camera includes an optical axis and an outer lens element disposed along the optical axis. The at least one light-emitter device is coupled to the camera by way of a modular attachment structure configured to interchangeably couple the plurality of illumination modules to the camera such that each respective emission axis forms a non-zero tilt angle with respect to the optical axis. An illumination intensity is at least 70% uniform within a field of view of the camera at a predetermined distance.

DETAILED DESCRIPTION

Thus, the example embodiments described herein are not meant to be limiting. Aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.

The present disclosure relates an optical system that includes a camera and an illumination module. In some embodiments, the optical system could be used in conjunction with semi- or fully-autonomous vehicles. However, other imaging applications are contemplated within the scope of the present disclosure.

In some embodiments, the camera could include an image sensor. The image sensor could be, for example, an APS-C format sensor. Other sensor size formats are possible and contemplated.

The illumination module could be configured to emit light towards an environment. For example, the illumination module may include one or more light-emitter devices and could be utilized to uniformly illuminate a scene at close range (e.g., up to 3 meters away). Such a system could be operable to provide illumination to, and imaging of, a scene so as to be able to detect and classify objects in low ambient light conditions. In some embodiments, the optical system could be configured to classify relatively small objects (e.g., 30-50 cm or less in height) at a range of approximately 3 meters away from the optical system. As an example of a real world application, an optical system could be configured to detect pedestrians, bicyclists, etc. and other objects near the optical system, under various lighting conditions, including night scenarios.

In the vehicle context, several optical systems (illumination modules and cameras) can be disposed on a given vehicle at different locations and/or different orientations so as to cover a full 360 degree azimuthal field of view without any occlusions. For example, two optical systems, each having a 190 degree azimuthal field of view, could be arranged on the vehicle (e.g., one optical system per side) to cover the full azimuthal field of view around the vehicle.

In some embodiments, the optical systems could be between 50-100 millimeters by 20-50 millimeters in length and width or vice versa. The optical systems could have a depth of approximately 20-50 millimeters. It will be understood that the optical system could have a different size in one or more dimensions.

The light-emitter devices could include devices configured to emit light in the infrared (e.g., 800-1000 nanometers), ultraviolet, and/or visible wavelengths. The light-emitter devices could include laser diodes, vertical cavity surface-emitting lasers (VCSELs), and/or light-emitting diodes (LEDs). However, other types of light-emitters are contemplated and possible. In some embodiments, the light-emitter devices could include a secondary optical element (e.g., a lens) configured to shape, focus, or otherwise modify characteristics of the emission light (e.g., light pattern, light intensity, wavelength, etc.). In some embodiments, the secondary optical element could include a refractive or diffractive optic element configured to broaden the emission light pattern. For example, in some scenarios, a LED could be configured to emit light within a cone having an extent of 130 degrees in azimuth. In such scenarios, the secondary optical element could be configured to spread or diffuse the emission cone to have an extent of 170 degrees in azimuth or more.

In some embodiments, each illumination module could include one or more LEDs. In such scenarios, at least one LED could be operable to emit visible light and at least one LED could be operable to emit infrared light or another wavelength not in the visible wavelengths. In some embodiments, the light-emitter devices could be configured to provide intermittent illumination. In other embodiments, the light-emitter devices could be configured to provide continuous light to the scene.

The illumination module could include light-emitter devices that are angled away from an optical axis of the camera. For example, in some embodiments, an emission axis of one or more light-emitter devices could form a 5-20 degree angle away from the optical axis of the camera. In some embodiments, a plurality of light-emitter devices could be disposed at approximately equal intervals around the optical axis of camera. For example, two light-emitter devices could be disposed approximately 10 millimeters from the optical axis of the camera, one on each side of an outer surface of the camera (e.g., to the right and left of the camera, or above and below the camera). In some embodiments, the light-emitter devices could partially or completely surround and/or encircle an outer portion of the camera (e.g., a lens opening).

In some embodiments, the light-emitter devices could be recessed with respect to an outer extent (e.g., outer/final camera lens element). For example, the light-emitter device could be recessed between 5 millimeters and 20 millimeters from the final camera lens element. In some embodiments, the light-emitter devices and/or the illumination module could include a flat or curved outer window so as to ease cleaning and/or improve durability. The outer window could include a hydrophilic or hydrophobic coating.

Such an arrangement of the light-emitter devices with respect to the camera could improve uniformity of the illumination within a given scene. Furthermore, such arrangements may reduce or eliminate the amount of stray light, which can reduce the performance of the camera. For example, no light rays emitted from the light-emitter devices directly enter the camera.

In some embodiments, the one or more illumination modules could be coupled to the camera by way of a modular attachment structure, which could include a bracket, slot, or another type of physical attachment structure configured to provide interchangeable replacement of the illumination module and/or the camera. In some embodiments, the illumination modules could be coupled to the left and right of the camera by way of the modular attachment structure. Additionally or alternatively, the modular attachment structure could be symmetric with respect to the camera lens opening. That is, illumination modules could be attached to the modular attachment structure in an interchangeable fashion. That is, a single illumination module could optionally be coupled to the modular attachment structure in a plurality of different locations with respect to the camera lens (e.g., right/left, right/left/top/bottom, etc.). The modular attachment structure could include, for example, a backplane attached to the camera and a plurality of attachment locations for illumination modules to attach to the modular attachment structure.

In some embodiments, the illumination module(s) and the corresponding light-emitter devices could be configured to emit light when the vehicle is operating at a speed below a predetermined low speed threshold. That is, the light-emitter devices could be configured to emit light when the vehicle is operating at low speeds (e.g., below 25 miles per hour). Additionally or alternatively, an operational mode of the illumination module and/or the light-emitter devices could be adjusted based on other considerations, such as vehicle energy usage and/or system temperature. For example, the illumination module could be disabled or operate at a lower temporal resolution when a system temperature is above a predetermined system temperature threshold. In some embodiments, the illumination could be at least 70% uniform across the field of view at a given range (e.g., 3 meters).

In some embodiments, the illumination module could operate in conjunction with various operations of the camera. For example, the illumination module could emit light only while the camera is actively capturing video. In other embodiments, the illumination module could emit intermittent light pulses in synchronization with one or more focal plane shutters associated with the camera.

II. Example Optical Systems

FIG. 1illustrates an optical system100, according to an example embodiment. The optical system100includes a camera110. The camera110includes an optical axis114and an outer lens element112disposed along the optical axis114. In an example embodiment, the camera110includes an image sensor116, such as a focal plane array or another type of multi-element photodetector sensor. For example, the image sensor116could include a plurality of charge-coupled device (CCD) elements and/or a plurality of complementary metal-oxide-semiconductor (CMOS) elements. In some embodiments, the camera110could include a plurality of image sensors.

The image sensor116could be configured (e.g., sized or dimensioned) according to an image sensor format. For example, the image sensor116could include a full-frame (e.g., 35 millimeter) format sensor. Additionally or alternatively, the image sensor116could include “crop sensor” formats, such as APS-C (e.g., 28.4 mm diagonal) or one inch (e.g., 15.86 mm diagonal) formats. In an example embodiment, the image sensor could include a 1/2.8″ format (e.g., ˜6.5 mm diagonal). Other image sensor formats are contemplated and possible within the scope of the present disclosure. It will be understood that embodiments with multiple cameras and/or multiple image sensors are possible and contemplated within the context of the present disclosure.

The optical system100includes a plurality of illumination modules120. Each illumination module120includes at least one light-emitter device122configured to emit light along a respective emission axis126. The light-emitter device122could include a light-emitting diode, a diode laser, or another type of light source (e.g., an incandescent or fluorescent light source). The light-emitter device122could be configured to emit light in the infrared wavelength band as well as other wavelength bands (e.g., the visible spectrum (about 400-700 nanometers) and/or the ultraviolet spectrum (about 10-400 nanometers)). In some embodiments, the illumination modules120could be shaped like a ring, which could be arranged about the optical axis114of the camera110(e.g., similar to a ring light). In such scenarios, the respective light-emitter devices122associated with such a ring-shaped illumination module120could be arranged at various locations about the optical axis114of the camera110. As will be described herein, other locations and/or arrangements of the illumination modules120and light-emitter devices122with respect to the camera110are possible and contemplated.

In some embodiments, the light-emitter device122could be configured to emit light at or around a wavelength of 850 nanometers. For example, the light-emitter device122could include an 850 nm LUXEON infrared LED (radiometric power of 1050 mW at 1000 mA). Such a light-emitter device122could have a beam divergence angle of approximately 150 degrees. As an example, the light-emitter device122could include a surface mount package and could be approximately 5×5×1.6 mm in dimension. However, other package formats and device sizes are contemplated and possible.

Each illumination module120also includes a secondary optical element124optically coupled to the at least one light-emitter device122. The secondary optical element124is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the optical system100. In some embodiments, the secondary optical element124could include a hemispherical or half-ball lens (e.g., Edmund TECHSPEC N-BK7 Half-Ball Lens). However, other types of diverging lenses are possible and contemplated. In some embodiments, the secondary optical element124could include a plano-convex lens, a prism lens, a cylindrical lens, a conical lens, and/or another type of non-hemispherical (e.g., oval) lens. Other custom secondary optical lenses are contemplated. Such custom secondary optical lenses could vary in diameter and shape, based on a desired horizontal field of view, a desired vertical field of view, and light emitter size.

In some embodiments, the illumination module(s)120could include one or more reflectors128. The reflectors128could include, for example, one or more optically reflective surfaces configured to modify and/or enhance the as-emitted light emission pattern from the at least one light-emitter device122. The reflectors128could include a surface coating of a highly reflective material, such as aluminum, gold, and/or chrome. In some embodiments, the reflectors128could be configured to improve light emission uniformity within the field of view of the camera110and/or increase or maximize the light intensity within the field of view. As an example, the reflectors128could include a three-dimensionally curved (e.g., parabolic) reflective surface arranged about the at least one light-emitter device122. Additionally or alternatively, the reflectors128could include a reflective surface having a different shape.

Example embodiments may include different types of light-emitter devices122, which may be utilized to obtain a desired spectral output from the illumination module120. For example, infrared LEDs could be used in conjunction with visible LEDS of various colors (e.g., red, green, blue, among other possibilities). In such scenarios, the secondary optical element124could be configured to “mix” the various spectra of emitted light of the various light-emitter devices122. In an example embodiment, the spectral emission, duty cycle, and/or emission intensity of light emitted by the illumination module120could be adjusted so as to achieve a desired spectral output characteristic that is observable by humans. For example, the desired spectral output characteristic could include that the emitted light from the illumination module120is observable by a human as being “white light”. In such an embodiment, the spectral output of the illumination module120could be adjusted so as to avoid human observation of substantially only red light, which may be distracting and/or may connote an undesired signal (e.g., unintentionally signaling slowing or stopping) when observed by others in a driving scenario. That is, by “mixing” various colors of LEDs, the spectral output of the illumination module120can be adjusted to avoid inadvertent warning signals to other human drivers.

Additionally or alternatively, some embodiments may include incorporating one or more optical dyes into at least one of the light-emitter devices122or the secondary optical element124. In some embodiments, the optical dye(s) could be dispersed within the secondary optical element124and/or coated onto one or more surfaces of the secondary optical element124. The optical dye(s) could be configured to absorb light at a first optical wavelength and emit light at a second optical wavelength. As an example, the secondary optical element124could include a Lumogen perylene-based dye configured to downconvert light (e.g., UV light) into one or more wavelengths in the visible spectrum. Alternatively, conventional inorganic dyes or pigments, chromophores, and/or colorants are contemplated and possible relating to the present disclosure.

In some scenarios, the optical system100also includes a housing160. The housing160could include the external surface of the optical system100and/or an external cover, a protective shell, and/or a coating. In some embodiments, the housing160could include an acrylic material formed in a sheet, dome, cylinder, or another shape. In some examples, an outer surface of each light-emitter device122is recessed between 0 millimeters and 50 millimeters with respect to a plane perpendicular to the optical axis114and that intersects the outer lens element112of the camera110. By recessing the light-emitter device122with respect to the outer lens element112, the amount of stray illumination light received by the camera110can be reduced.

In some examples, the system100could include one or more light-blocking elements162. The light-blocking elements162could include, for example, baffles, ink, apertures, stops, or other optically-opaque structures. In example embodiments, the light-blocking elements162could be arranged inside and/or adjacent to the secondary optical element124. For example, the blocking elements162could be arranged between the secondary optical element124and the camera110. Among other possibilities, the blocking elements162could take the form of a lens hood around an outer lens element112of the camera110. Other “go-between” or “gobo” arrangements of the blocking elements162with respect to the camera110and the light-emitter device122are possible and contemplated. In such scenarios, the blocking elements162could be configured to prevent light from the light-emitter device(s)122from directly entering the camera110. Accordingly, the blocking elements162may help reduce or minimize lens flare, emission light “pollution”, and optical ghosting, and/or otherwise improve image quality of the camera110.

In some example embodiments, the plurality of illumination modules120could include a first illumination module and a second illumination module (e.g., first illumination module120aand second illumination module120bshown inFIGS. 2A and 2B). The first illumination module120aand the second illumination module120bcould be arranged opposite one another with respect to the optical axis114of the camera110such that the respective emission axes126a-bof the illumination modules120a-bare tilted in opposite directions with respect to the optical axis. Put another way, the pointing angle of the camera110(e.g., optical axis114) could be different than the pointing angles of the respective illumination modules120(e.g., emission axis126aand emission axis126b). While embodiments described herein include two illumination modules, which could be arranged to the left and right of the camera110, it will be understood different numbers of illumination modules are possible and contemplated. For example, some embodiments may include four illumination modules, which could be respectively arranged to the left, right, above, and below the camera110.

In some embodiments, the first illumination module120aand the second illumination module120bcould each have a tilt angle with respect to the optical axis114that is within a range from 5 degrees to 20 degrees. That is, the respective emission axes126aand126bcould be tilted between 5 and 20 degrees from the optical axis114of the camera110. It will be understood that other tilt angles (e.g., between 0 and 30 degrees or between 10 and 45 degrees) are possible and contemplated. By angling the emission axes of the illumination modules away from the optical axis114, the amount of stray illumination light received by the camera110can be further reduced.

In various embodiments, the light emission pattern could provide an illumination intensity that is at least 70% uniform within a field of view of the illumination modules (e.g., first illumination module120aand/or second illumination module120b) at a predetermined distance (e.g., 3 meters) from the optical system100. In various examples, the predetermined distance could be, without limitation, a distance between 5 centimeters up to 50 meters. For example, the predetermined distance could be 3 meters, 10 meters, 30 meters, or another distance. In some embodiments, the illumination module120could be configured to emit light in a predetermined emission pattern. For example, the predetermined emission pattern may be selected or configured so as to compensate for light falloff in various optical elements in the optical system100(e.g., the outer lens element112). In other words, illumination module120could emit light such that the relative illumination level is maintained at a substantially constant or equal amount across the field of view of the camera110. In a specific example, in some embodiments, the outer lens element112of the camera110may transmit less light from the edges of the field of view as compared to the central portion of the field of view. Accordingly, the illumination module120could compensate for this effect by providing higher light intensity in regions near the edges of the field of view of the camera110. Other types of predetermined emission patterns are possible and contemplated.

In some embodiments, the optical system could additionally include a modular attachment structure130configured to interchangeably couple the plurality of illumination modules120to the camera110such that each respective emission axis126forms a non-zero tilt angle with respect to the optical axis114. In such scenarios, each illumination module (e.g., illumination modules120aand120b) of the plurality of illumination modules120is configured to interchangeably couple to each of a plurality of mounting locations132on the modular attachment structure130. The modular attachment structure130includes at least one registration surface134. The at least one registration surface134is configured to position an illumination module120according to at least one of: a known or desired orientation, a known or desired physical alignment position, and/or a known or desired optical alignment position. It will be understood that in some embodiments, illumination module120aand illumination module120bcould be physically coupled and/or could be incorporated into a single illumination module. In scenarios involving a vehicle, the modular attachment structure130could attach directly to the vehicle. Additionally or alternatively, the modular attachment structure130could be a vehicle bracket or another type of vehicle structure.

In some examples, the modular attachment structure130could beneficially enable one or more of the illumination modules120to be interchangeably attached to the camera110in scenarios where an illumination module fails or if different types of illumination modules are needed. For example, illumination modules with different illumination intensity levels and/or different illumination patterns could be used in different imaging scenarios and/or applications.

Furthermore, in some embodiments, the modular attachment structure130could include one or more mounts to interchangeably attach a different camera110. For example, different types and/or formats of cameras could be utilized based on a specific imaging scenarios and/or application. For example, a camera with a wide horizontal (e.g., azimuthal angle) field of view (e.g., 190 degree cone angle) could be attached to the modular attachment structure130in a first scenario and a camera with a narrower horizontal field of view (e.g., a 120 degree cone angle) could be attached in a second scenario. It will be understood that utilizing cameras with different vertical (e.g., elevation angle) fields of view is also possible. Other ways to interchangeably couple different cameras110and/or different illumination modules120are possible.

In some scenarios, the optical system100also includes a controller150. The controller150includes at least one of: a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a light-emitting diode (LED) driver circuit. Additionally or alternatively, the controller150may include one or more processors152and a memory154. The one or more processors152may be a general-purpose processor or a special-purpose processor (e.g., digital signal processors, etc.). The one or more processors152may be configured to execute computer-readable program instructions that are stored in the memory154. As such, the one or more processors152may execute the program instructions to provide at least some of the functionality and operations described herein.

The memory154may include or take the form of one or more computer-readable storage media that may be read or accessed by the one or more processors152. The one or more computer-readable storage media can include volatile and/or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which may be integrated in whole or in part with at least one of the one or more processors152. In some embodiments, the memory154may be implemented using a single physical device (e.g., one optical, magnetic, organic or other memory or disc storage unit), while in other embodiments, the memory154can be implemented using two or more physical devices.

As noted, the memory154may include computer-readable program instructions that relate to operations of optical system100. As such, the memory154may include program instructions to perform or facilitate some or all of the functionality described herein. The controller150is configured to carry out operations. In some embodiments, controller150may carry out the operations by way of the processor152executing instructions stored in the memory154.

In some embodiments, the illumination modules120and the light-emitter devices122themselves could be controlled in various ways. For example, the illumination modules120and the light-emitter devices122could be synchronized with various operations of the camera110. Additionally or alternatively, the illumination modules120and/or the individual light-emitter devices122could be operated in continuous (DC) mode or in pulsed mode. In such scenarios, the controller150could carry out operations so as to adjust one or more characteristics of the light emitted by the illumination modules120.

The operations could include causing the at least one light-emitter device122of at least one illumination module120to emit light according to a light emission pattern. In such scenarios, the light includes light having a wavelength of substantially 850 nm (e.g., between 845 nm and 855 nm or between 840 nm and 860 nm). In some embodiments, the illumination module120could be configured to emit visible light (e.g., between 390 nm and 700 nm). Additionally or alternatively, the illumination module120could be configured to simultaneously emit both visible light and infrared light. Yet further, in some embodiments, the illumination module120could trigger the visible and infrared light emitter devices together (e.g., simultaneously or synchronously) or separately (e.g., asynchronously). Other light emission wavelengths and/or wavelength ranges are possible and contemplated.

In some embodiments, the operations could additionally include causing the camera110to capture at least one image of the portion of the environment. For example, the operations could include capturing a plurality of images using the camera110so as to provide information about the environment. In some embodiments, the images could be used to determine objects and/or obstacles within the environment.

Additionally or alternatively, the operations could include operating the optical system100in a first operating mode. In some embodiments, the first operating mode could include a “normal” or “standard” operating mode.

In such a scenario, the operations may include receiving information indicative of an operating temperature of the optical system100. Such information could be obtained from a temperature sensor, such as a thermometer, a thermistor, a thermocouple, and/or a bandgap temperature sensor. In such scenarios, the temperature sensor could sense the temperature of the camera110and/or the illumination module(s)120.

In such example embodiments, the operations could include determining that the optical system100is operating at an elevated temperature. For example, the elevated temperature could include an operating temperature of the optical system100being above a predetermined threshold temperature (e.g., 30° C., 40° C., 60° C., or another threshold temperature).

For example, the operations could include, in response to determining the optical system100is operating at an elevated temperature, operating the optical system100in a second operating mode. In other words, in scenarios where the optical system100is operating above a threshold temperature, the optical system100could be adjusted so as to operate in the second operating mode.

In such scenarios, operating the optical system100in the second operating mode could include operating the plurality of illumination modules120at a reduced illumination rate (e.g., 75%, 50%, or 25% of nominal pulse rate or duty cycle). In some embodiments, the duty cycle and/or onset of illumination of the illumination modules120could be varied based on an exposure time of the camera110. For example, the duty cycle the illumination modules120could be synchronized to the exposure time and/or frame rate of the camera110. In such scenarios, the duty cycle could be selected so as to avoid improperly- or insufficiently-lit images. Additionally or alternatively, the second operating mode could include operating the plurality of illumination modules120at a reduced illumination intensity (e.g., 50% of the illumination intensity provided during the first operating mode). Yet further, the second operating mode could include operating a reduced set of the plurality of illumination modules120and/or a reduced set of the plurality of light-emitter devices122. In some embodiments, the second operating mode could include an operating mode that is configured to generate less heat than the first operating mode. For example, the illumination modules120could be operated so as to output less heat. In other embodiments, the second operating mode could include increasing the cooling rate of various thermal management devices, such as cooling fins, liquid cooling systems, heat sinks, etc.

In example embodiments, the operations could include synchronously operating the camera110and the plurality of illumination modules120based one or more focal plane shutter times associated with the camera110. That is, the illumination modules120could be synchronized with the camera110so as to provide illumination substantially only while the shutter of the camera110is open—either physically or “digitally” during image capture. Additionally or alternatively, the illumination modules120could be synchronized with the camera110by the controller150and/or by an external computing device. In some embodiments, such synchronization could help avoid “rolling shutter” effects, where different portions of a given image are illuminated by vastly different light levels.

In cases where the illumination modules120are operated in synchronization with the camera110, the respective image sensor116of the camera110could be a global shutter-type image sensor (e.g., an image sensor that captures light from an entire area of field of view simultaneously) or a rolling shutter-type image sensor (e.g., an image sensor that scans portions of the field of view in a sequential manner, such as in a line-by-line fashion). For global shutter-type image sensors, the illumination modules120could be triggered to illuminate the scene at the same time as the image capture is triggered at the image sensor116. The illumination provided from the illumination modules120could end after an exposure time has elapsed. In the case of a rolling shutter-type image sensor, the illumination modules120could be triggered to begin illuminating the scene when the image capture is triggered, but the illumination level will generally be maintained until all of the rows of the image sensor have been scanned. In such scenarios, the illumination modules120could be kept on (e.g., illuminated) for an exposure time+(a number of rows*readout time per row) (potentially plus some buffering time).

In some embodiments, the controller150could be configured to adjust a current and/or voltage signal provided to the illumination module120and/or specific light-emitter devices122to control their operation. For example, by adjusting a current and/or voltage input to the illumination module120, the emitted light intensity could be controllably varied. In some embodiments, the controller150could be operable to adjust an overall light intensity of the illumination modules120while maintaining a consistent overall light emission pattern. In yet other embodiments, the controller150could be operable to maintain the light emission pattern at substantially all times. It will be understood that other ways to control the illumination modules120are possible and contemplated.

The light intensity output of the illumination module120and/or specific light-emitter devices122could be controlled by way of continuous (DC) mode or a pulse-width modulation (PWM) mode. If a PWM mode is utilized with a rolling shutter-type image sensor, a frequency of the modulation of the respective light-emitter devices122should be high enough so that all of the image sensor pixels will be illuminated for their respective exposure and readout times.

For a system that includes multiple cameras110, illumination intensity could be controlled on a per camera basis. As background, the closer an object is from the camera, the more light it will generally receive from the illumination modules120. For example, for an object at 1 meter and an object at 3 meter, there are cases for which the dynamic range of the camera might not be enough (even in HDR mode) to be able to properly expose the 2 objects without saturating or under exposing them. However, by dynamically controlling the illumination intensity, the camera110can capture a first image with a well-exposed 1-meter object under a first illumination intensity and capture a second image with a well-exposed 3-meter object under a second illumination intensity. This control can also be performed by changing the camera settings when the intensity is not modifiable (e.g., changing the gain and/or exposure time).

While illumination intensity could be controlled in an “open-loop” fashion, some embodiments may include a “closed-loop” illumination intensity control. For example, prior images could be utilized to determine an adjustment amount to change the illumination intensity. For example, if a target object in a prior image is over- or under-exposed, the controller150could adjust a continuous voltage/current signal and/or a pulse-width modulation signal so as to adjust the illumination intensity provided by the illumination modules120so as to properly expose the target object in subsequent images. Other ways to utilize feedback to make changes in the illumination intensity are possible and contemplated.

In some embodiments, the camera110could operate at 10 Hz (e.g., 10 frames per second) and, in some cases, may run at all times. Within the scope of the present disclosure, the images from the camera110can be used in multiple ways to detect certain target objects. For instance, in some embodiments, a perception module could run one or more image classifier routines on images captured by the camera110to detect pedestrians, vehicles, obstacles, or other important objects. Such cameras and optical systems may help augment various applications, such as autonomous vehicles and/or machine vision. As an example, images captured by the camera110could be utilized to identify objects and understand if the autonomous vehicle can proceed safely. For example, the systems and methods described herein could help the perception module of an autonomous vehicle distinguish between vegetation and a pedestrian.

FIG. 2Aillustrates a side or cross-sectional view of an optical system200, according to an example embodiment. Optical system200could be similar or identical to optical system100as illustrated and described in reference toFIG. 1. For example, optical system200includes a camera110having an outer lens element112and an image sensor116.

Optical system200also includes a first illumination module120aand a second illumination module120b. As illustrated, the first illumination module120aincludes two light-emitting devices122aand122band secondary optical elements124aand124b. Furthermore, the second illumination module120bincludes two light-emitting devices122cand122dand secondary optical elements124cand124d.

The first illumination module120aand the second illumination module120bcould be attached to the camera110by way of a modular attachment structure130. The modular attachment structure130could include, for example, registration surfaces134aand134band/or mounting locations132aand132b.

FIG. 2Billustrates the optical system200ofFIG. 2A, according to an example embodiment. Specifically,FIG. 2Billustrates the respective emission angles of the light-emitter devices122a-d. For example, light-emitter device122ahas an emission cone125a, light-emitter device122bhas an emission cone125b, light-emitter device122chas an emission cone125c, and light-emitter device122dhas an emission cone125d. Furthermore,FIG. 2Billustrates a field of view115of camera110. In some embodiments, emission cones125a-dcould be greater than 170 degrees, although other cone angles are possible and contemplated. Furthermore, whileFIG. 2Billustrates “symmetric” emission cones, it will be understood that different arrangements and different types of emission cones are possible and contemplated. For example, the emission cones could be asymmetric (e.g., a flattened cone shape with wider angle coverage in azimuth than in elevation) so as to illuminate a wider azimuthal field of view. In some embodiments, a plurality of light-emitter devices122and/or illumination modules120each having an asymmetrical emission cone could be combined so as to provide a substantially symmetric light emission pattern with respect to the camera110. In one such scenario, three illumination modules120with individually asymmetric light emission patterns could be arranged so as to provide a symmetric light emission pattern about the optical axis114of the camera110.

FIGS. 3A and 3Billustrate an optical system300, according to an example embodiment. Optical system300could be similar or identical in various respects to optical systems100and200as illustrated and described in relation toFIGS. 1, 2A, and 2B. However, in contrast to optical system200, optical system300could include the illumination modules120aand120bhaving emission axes126aand126bthat are substantially parallel to the optical axis114of the camera110(as shown inFIG. 3B). That is, the light-emitter devices122a-dcould be arranged substantially along a plane302, which could be parallel to plane304, defined by the camera110and/or the outer lens element112(as shown inFIG. 3A).

In some embodiments, the light-emitter devices122a-dcould be recessed from the plane304by a recess distance306, which could be between 0 and 50 mm or more. Other recess distances are contemplated and possible. In some examples, the recess distance306could be selected based on a field of view extent of the camera110. For instance, the recess distance306could be selected to be 50 mm for a very wide field of view camera (e.g., 220 degree azimuthal angle extent). As described herein, recessing the light-emitter devices122a-dfrom the outer lens element112could reduce the amount of stray illumination light received by the camera110.

FIG. 3Bshows the respective emission angles of the light-emitter devices122a-d. For example, light-emitter device122ahas an emission cone125a, light-emitter device122bhas an emission cone125b, light-emitter device122chas an emission cone125c, and light-emitter device122dhas an emission cone125d. Furthermore,FIG. 3Billustrates a field of view115of camera110. In some embodiments, emission cones125a-dcould be greater than 170 degrees, although other cone angles are possible and contemplated.

In some embodiments, one or more optical systems100could be attached or otherwise mounted to a vehicle, as described below.

III. Example Vehicles

FIG. 4illustrates a vehicle400, according to an example embodiment. In some embodiments, the vehicle400could be a semi- or fully-autonomous vehicle. WhileFIG. 4illustrates vehicle400as being an automobile (e.g., a car), it will be understood that vehicle400could include another type of autonomous vehicle, robot, or drone that can navigate within its environment using sensors and other information about its environment.

The vehicle400may include one or more sensor systems402,404,406,408, and410. Some embodiments, sensor systems402,404,406,408, and410could include LIDAR sensors having a plurality of light-emitter devices arranged over a range of angles with respect to a given plane (e.g., the x-y plane).

One or more of the sensor systems402,404,406,408, and410may be configured to rotate about an axis (e.g., the z-axis) perpendicular to the given plane so as to illuminate an environment around the vehicle400with light pulses. Based on detecting various aspects of reflected light pulses (e.g., the elapsed time of flight, polarization, etc.,), information about the environment may be determined.

In an example embodiment, sensor systems402,404,406,408, and410may be configured to provide respective point cloud information that may relate to physical objects within the environment of the vehicle400. While vehicle400and sensor systems402and404are illustrated as including certain features, it will be understood that other types of sensor systems are contemplated within the scope of the present disclosure.

An example embodiment may include a system having a plurality of light-emitter devices. The system may include a transmit block of a LIDAR device. For example, the system may be, or may be part of, a LIDAR device of a vehicle (e.g., a car, a truck, a motorcycle, a golf cart, an aerial vehicle, a boat, etc.). Each light-emitter device of the plurality of light-emitter devices is configured to emit light pulses along a respective beam elevation angle. The respective beam elevation angles could be based on a reference angle or reference plane, as described elsewhere herein. In some embodiments, the reference plane may be based on an axis of motion of the vehicle400.

While LIDAR systems with multiple light-emitter devices are described and illustrated herein, LIDAR systems with fewer light-emitter devices (e.g., a single light-emitter device) are also contemplated herein. For example, light pulses emitted by a laser diode may be controllably directed about an environment of the system. The angle of emission of the light pulses may be adjusted by a scanning device such as, for instance, a mechanical scanning mirror and/or a rotational motor. For example, the scanning devices could rotate in a reciprocating motion about a given axis and/or rotate about a vertical axis. In another embodiment, the light-emitter device may emit light pulses towards a spinning prism mirror, which may cause the light pulses to be emitted into the environment based on an angle of the prism mirror angle when interacting with each light pulse. Additionally or alternatively, scanning optics and/or other types of electro-opto-mechanical devices are possible to scan the light pulses about the environment.

In some embodiments, a single light-emitter device may emit light pulses according to a variable shot schedule and/or with variable power per shot, as described herein. That is, emission power and/or timing of each laser pulse or shot may be based on a respective elevation angle of the shot. Furthermore, the variable shot schedule could be based on providing a desired vertical spacing at a given distance from the LIDAR system or from a surface (e.g., a front bumper) of a given vehicle supporting the LIDAR system. As an example, when the light pulses from the light-emitter device are directed downwards, the power-per-shot could be decreased due to a shorter anticipated maximum distance to target. Conversely, light pulses emitted by the light-emitter device at an elevation angle above a reference plane may have a relatively higher power-per-shot so as to provide sufficient signal-to-noise to adequately detect pulses that travel longer distances.

In some embodiments, the power/energy-per-shot could be controlled for each shot in a dynamic fashion. In other embodiments, the power/energy-per-shot could be controlled for successive set of several pulses (e.g.,10light pulses). That is, the characteristics of the light pulse train could be changed on a per-pulse basis and/or a per-several-pulse basis.

In some embodiments, the spectral emission of the light-emitter devices122could be configured and/or selected so as to avoid the wavelength or wavelengths utilized by the LIDAR sensors. For example, the LIDAR sensors could utilize various infrared wavelengths, such as 905 nm and/or 1050 nm. In such scenarios, the light-emitter devices122could be selected so as to emit light at a different wavelength (e.g., 850 nm) so as to minimize or eliminate cross-talk or optical interference between the two types of sensor systems. Furthermore, the temporal characteristics of the light emitted from the light-emitter devices122could be configured to be distinguishable from that of LIDAR light pulses. For example, the light-emitter devices122could emit light pulses with substantially longer “turn-on” times (e.g., microseconds or milliseconds) as compared to several nanosecond-range laser diode pulses utilized for LIDAR sensing. Additionally or alternatively, the duty cycle of the light-emitter devices122(e.g., greater than 20% duty cycle) could be distinguishable from LIDAR laser diode pulses (e.g., less than 5% duty cycle).

WhileFIG. 4illustrates various LIDAR sensors attached to the vehicle400, it will be understood that the vehicle400could incorporate other types of sensors, such as a plurality of cameras and corresponding illumination modules, as described below. Additionally or optionally, the light emissions from the illumination modules120and/or the LIDAR light pulses could be adjusted and/or regulated by system hardware and/or software so as to comply with local, federal, and/or international eye safety conventions, laws, and/or regulations.

FIG. 5illustrates the vehicle400ofFIG. 4, according to an example embodiment. In some embodiments, the vehicle400could include one or more instances of optical system100as illustrated and described in reference toFIGS. 1, 2A, 2B, 3A, and/or3B. For example, the vehicle400could include a camera (e.g., camera110as illustrated inFIGS. 1, 2A, 2B, 3A, and/or3B). In such examples, the camera110includes an optical axis114and an outer lens element112disposed along the optical axis114. The vehicle400may also include a first illumination module (e.g., illumination module120a) configured to illuminate a first portion of an environment of the vehicle400. The vehicle400may additionally include a second illumination module (e.g., illumination module120b), which could be configured to illuminate a second portion of the environment of the vehicle400.

In such embodiments, each illumination module120could include at least one light-emitter device (e.g., light-emitter devices122a-band/or122c-d) configured to emit light along a respective emission axis (e.g., emission axis126aor126b).

In some embodiments, each illumination module120could include a secondary optical element124optically coupled to the at least one light-emitter device122, wherein the secondary optical element124is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the vehicle400. It will be understood that light emission patterns with different azimuthal angle extents are possible. For example, the light emission pattern could have an azimuthal angle extent of 30 degrees, 45 degrees, 90 degrees, 120 degrees, 270 degrees, or 360 degrees. Other azimuthal angle extents are possible and contemplated.

The first illumination module120aand the second illumination module120bcould be arranged opposite one another with respect to the optical axis114of the camera110such that the respective emission axes126of the first illumination module120aand second illumination module120bare tilted in opposite directions with respect to the optical axis114. For example, emission axis126acould be tilted five degrees away from the optical axis114. In such a scenario, emission axis126bcould be tilted five degrees in the opposite direction from the optical axis114.

It will be understood that other tilt angles are possible and contemplated. For example, the first illumination module120aand the second illumination module120bcould have a tilt angle with respect to the optical axis that is within a range from 5 degrees to 20 degrees or from 0 degrees to 45 degrees.

As described herein, the light emission pattern provided by the illumination modules120could be configured to provide an illumination intensity that is at least 70% uniform within a field of view of the illumination modules120at a predetermined distance (e.g., 5, 10, 20, or 30 meters) from the optical system100.

In some example embodiments, vehicle400could include a modular attachment structure130configured to interchangeably couple the plurality of illumination modules120to the camera110such that each respective emission axis126forms a non-zero tilt angle with respect to the optical axis114. As an example embodiment, each of the first illumination module120aand the second illumination module120bcould be configured to interchangeably couple to each of a plurality of mounting locations132on the modular attachment structure130.

Additionally or alternatively, the modular attachment structure130could include at least one registration surface134. The at least one registration surface134could be configured to position an illumination module120according to at least one of: a known or desired orientation, a known or desired physical alignment position, and/or a known or desired optical alignment position.

In some embodiments described herein, the vehicle400could include a controller (e.g., controller150), which may include at least one processor and a memory. The controller could be configured to execute instructions stored in the memory so as to carry out operations. For example, the operations could include causing the at least one light-emitter device of at least one illumination module to emit light according to the light emission pattern. The light could include light having a wavelength of 850 nm (e.g., light with wavelength(s) between 840 nm and 860 nm.

In example embodiments, the operations could include causing the camera (e.g., camera110) to capture at least one image of the portion of the environment.

FIG. 6illustrates a vehicle400, according to an example embodiment. As illustrated, vehicle400could include a first optical system100aand a second optical system100b. The first optical system100acould be attached to a left side of the vehicle400and the second optical system100bcould be attached to a right side of the vehicle400. It will be understood that other arrangements of the respective optical systems are possible and contemplated. For example, the optical systems could be mounted in front/rear arrangement. Additionally or alternatively, while two optical systems are illustrated, it will be understood that more or fewer optical systems could be utilized with vehicle400.

As illustrated inFIG. 6, the respective fields of view of the optical systems100aand100bcould overlap in azimuth so that two optical systems could provide 360 degree imaging around the vehicle400. That is, field of view610aassociated with optical system100aand field of view610bassociated with optical system100bcould overlap a few meters in front of the vehicle400and a few meters behind the vehicle400. In such a manner, the two camera systems could provide 360 degree azimuthal coverage around the vehicle400.

In some embodiments, certain operations of the optical systems100aand/or100bcould be based on operations of the vehicle400. For example, the operations could include, while the vehicle400is operating below a threshold velocity (e.g., 25 miles per hour), operating the at least one light-emitter device122or the at least one illumination module120to emit light according to the light emission pattern. In such a scenario, the operations could also include, while the vehicle400is operating at or above the threshold velocity, discontinuing operation of the at least one light-emitter device122and the at least one illumination module120. In some embodiments, the threshold velocity could be a velocity between 10 and 30 miles per hour.

While the example embodiments described above relate to vehicles, it will be understood that the optical systems and methods could be applied in other contexts. For example, the described optical systems could be utilized for security applications (e.g., surveillance and/or safety applications). Additionally or alternatively, the optical systems and methods could be applied within the context of photography, videography, remote monitoring, among other possibilities.

IV. Example Methods

FIG. 7illustrates a method700, according to an example embodiment. It will be understood that the method700may include fewer or more steps or blocks than those expressly illustrated or otherwise disclosed herein. Furthermore, respective steps or blocks of method700may be performed in any order and each step or block may be performed one or more times. In some embodiments, some or all of the blocks or steps of method700may be carried out by controller150and/or other elements of optical system100and/or vehicle400as illustrated and described in relation toFIGS. 1, 2A, 2B, 3A, 3B, 4, 5, and 6.

Block702includes causing at least one light-emitter device of at least one illumination module of a vehicle to emit light into an environment according to a light emission pattern. The light emission pattern could correspond to a spatial illumination pattern and/or a temporal pattern (e.g., 2 kHz light pulse emission rate). In an example embodiment, the spatial illumination pattern could include a substantially uniform illumination intensity over a broad angle range (e.g., 170 degree cone angle). For example, the illumination intensity could be at least 70% uniform across the emission cone angle.

The at least one light-emitter device is configured to emit light along an emission axis. In such scenarios, the at least one light-emitter device is optically coupled to a secondary optical element. The secondary optical element is configured to interact with the emitted light so as to provide a light emission pattern. The light emission pattern includes an azimuthal angle extent of at least 170 degrees.

Block704includes causing a camera of the vehicle to capture at least one image of the portion of the environment. The camera includes an optical axis and an outer lens element disposed along the optical axis. The at least one light-emitter device is coupled to the camera by way of a modular attachment structure configured to interchangeably couple the plurality of illumination modules to the camera such that each respective emission axis forms a non-zero tilt angle with respect to the optical axis. In such examples, an illumination intensity (e.g., of the emitted light) is at least 70% uniform within a field of view of the illumination modules at a predetermined distance (e.g., 3 meters).

The relative illumination uniformity across the field of view of the illumination modules may provide improved imaging performance and object recognition based on the images captured by the camera. Providing a uniform illumination intensity across a ˜170 degree cone may additionally help reduce the negative effects of retroreflectors by more evenly distributing illumination light throughout the field of view, instead of concentrating it in a narrower cone.

In some embodiments, the method could include, while the vehicle is operating below a threshold velocity, operating the at least one light-emitter device or the at least one illumination module to emit light according to the light emission pattern. The light could include light having a wavelength of around 850 nm.

Additionally or alternatively, the method could include while the vehicle is operating at or above a threshold velocity, discontinuing operation of the at least one light-emitter device and the at least one illumination module. In some embodiments, the threshold velocity could be between 10 and 30 miles per hour. However, other threshold velocities are possible and contemplated.

The computer readable medium can also include non-transitory computer readable media such as computer-readable media that store data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer readable media can also include non-transitory computer readable media that store program code and/or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, compact-disc read only memory (CD-ROM), for example. The computer readable media can also be any other volatile or non-volatile storage systems. A computer readable medium can be considered a computer readable storage medium, for example, or a tangible storage device.

While various examples and embodiments have been disclosed, other examples and embodiments will be apparent to those skilled in the art. The various disclosed examples and embodiments are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.