Patent Description:
A common problem in radiometric remote sensing is the estimation of incident irradiance from the sun on arbitrary surfaces from the scattered and direct component of the sunlight. Traditionally, these components of sunlight are measured on the ground, using a shaded pyranometer for the scattered component and a pyrheliometer for the direct component. Both of these devices track the position of the sun during measurement. The pyrheliometer has a long tube that only allows direct light in and the tracking shading of the pyranometer blocks direct light, so that the instruments measure only the direct and scattered light, respectively. Both instruments have significant cost and are unsuitable for mounting on a small, rapidly moving platform such as a drone.

A single conventional light sensor can be used to measure both components of sunlight, if the sensor attitude is well determined and the attitude is varied over time. However, while such a sensor can be mounted on a drone, the precise attitude estimates for a moving platform are difficult to obtain, or require costly sensors and are prone to significant errors, particularly under changing light conditions, e.g. due to partial cloud cover.

Accordingly, in conventional remote sensing applications, such as multispectral imaging applications for determining the health of vegetation, ground-based calibration systems are typically employed for normalizing the effects of a variable light source (e.g., the sun) on multispectral images of a target. Such calibration systems commonly rely on the use of target calibration or reflectance panels having a known spectral reflectance that are placed in the field of view of a multispectral imaging device and can be used to calibrate the acquired image of the target. There are several drawbacks to such techniques, including that the calibration or reflectance panels are costly, cumbersome and do not accurately measure irradiance levels simultaneously with the acquired images.

<NPL>, discloses attitude stabilization strategies via output sensor feedback for micro aerial vehicles (MAVs). In order to compensate for the size and power limitations of MAVs, ocelli and halteres, the body orientation and rotation sensing mechanisms used by flying insects, are introduced. Attitude stabilization techniques based on these sensors are proposed and tested on an aerodynamic model for a micromechanical flying insect (MFI).

<CIT> discloses an attitude sensor, and in particular, a sun sensor using a multi-channel light guide fiber as a light introducer and a method of measuring the position thereof.

<CIT> discloses a window shading control system and method thereof based on decomposed direct and diffuse solar radiations.

<CIT> discloses a method and system of calibrating a multispectral camera on an aerial vehicle.

<CIT> discloses an imaging processing method, image processing system, and image processing program.

In a first aspect of the present invention, there is provided a device according to claim <NUM>.

In a second aspect of the present invention, there is provided a method according to claim <NUM>.

The dependent claims define preferred embodiments.

The present disclosure is directed to devices and methods for sensing irradiance from a light source, such as the sun, by an irradiance sensing device including a plurality of photo sensors arranged at differing orientations. By simultaneously sensing the irradiance with multiple photo sensors having different orientations, particular components of the irradiance, such as the direct and scattered components and the incidence angle, may be determined. These determined irradiance components may be used to compensate or normalize images of a target that are acquired at the same time by an imaging device. The irradiance sensing device and the imaging device may be carried on an aerial vehicle, such as a drone.

The present disclosure provides a device that includes an aerial vehicle and an irradiance sensing device. The irradiance sensing device includes a base structure mounted to the aerial vehicle, and the base structure includes a plurality of surfaces. The irradiance sensing device further includes a plurality of photo sensors, with each of the photo sensors being arranged on a respective surface of the base structure and having different orientations.

The present disclosure provides a method that includes: simultaneously sensing irradiance by a plurality of photo sensors, each of the photo sensors having a different sensing orientation; acquiring image information associated with a target object; determining, by a processor, direct and scattered components of the sensed irradiance; and determining a reflectance of the target object based on the determined direct and scattered components and the acquired image information.

The present disclosure provides a method that includes: simultaneously sensing irradiance by a plurality of photo sensors positioned on an aerial vehicle, each of the photo sensors having a different sensing orientation; transmitting information indicative of the sensed irradiance from the plurality of photos sensors to a processor; and determining, by the processor, direct and scattered components of the irradiance.

In the drawings, identical reference numbers identify similar elements. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale.

The present disclosure is directed to systems and methods for measuring solar irradiance in radiometric remote sensing applications. Irradiance from a light source, such as the sun, may be simultaneously sensed by a plurality of photo sensors arranged at differing orientations on an irradiance sensing device. Components of the irradiance, such as the direct and scattered components and the incidence angle, may thus be determined, and utilized to compensate or normalize images of a target that are acquired at the same time by an imaging device.

<FIG> illustrates an aerial vehicle <NUM> for sensing irradiance and simultaneously obtaining an image, for example, of a ground-based target, in accordance with one or more embodiments, and <FIG> illustrates further details of the aerial vehicle <NUM>. Referring to <FIG> and <FIG>, the aerial vehicle <NUM> includes an irradiance sensing device <NUM> and an imaging device <NUM> for imaging a physical area or scene (i.e., a target). The irradiance sensing device <NUM> and the imaging device <NUM> may collect, store and/or output the obtained irradiance and image information.

The aerial vehicle <NUM> may be any type of aerial vehicle, including any rotary or fixed wing aerial vehicle, and may be an unmanned vehicle (as shown in <FIG>) or manned aerial vehicle, such as an airplane or a drone. Additionally, the aerial vehicle <NUM> may be an autonomous vehicle, capable of autonomous flight (and autonomous acquisition of irradiance and image information), or may be a piloted vehicle (e.g., flown by a pilot in a manned vehicle, or by a remote pilot of an unmanned vehicle).

The imaged target (e.g., trees <NUM>, crops <NUM>, <NUM>, a field of grass, a body of water or the like) receives irradiance from a light source, such as the sun <NUM>. The target may be one or more distinct objects (e.g., a single tree, a building, a pond, etc.), an area or scene (e.g., a portion of a forest, a portion of a field of crops, a portion of a lake, etc.) or any other target for which the acquisition of an image may be desired.

The imaging device <NUM> may be a multispectral imaging device capable of acquiring spectral images of a target, and may include multiple imagers, with each such imager being tuned for capturing particular wavelengths of light that is reflected by the target. The imaging device <NUM> may be configured to capture reflected light in one or more of the ultraviolet, visible, near-infrared, and/or infrared regions of the electromagnetic spectrum.

Images acquired by such multispectral imaging devices may be utilized to measure or determine different characteristics of the target, such as the chlorophyll content of a plant, an amount of leaf area per unit ground area, an amount or type of algae in a body of water, and the like. According to the invention, a processor is configured to determine a reflectance of the target based on image information of the target acquired by the imaging device <NUM>.

The imaging device <NUM> is mounted to the aerial vehicle <NUM> and oriented in any manner as may be desired. For example, the imaging device <NUM> may be mounted to a lower surface of the aerial vehicle <NUM> and positioned such that images of ground-based targets may be obtained.

The irradiance sensing device <NUM> may be mounted to an upper surface of the aerial device <NUM>, and includes a plurality of photo sensors configured to simultaneously sense irradiance from a light source, such as the sun <NUM>, at various different orientations with respect to the light source.

By simultaneously sensing irradiance by multiple photo sensors having different orientations, it is possible to determine particular characteristics of the light source, namely the direct and scattered components of solar irradiance, as well as an angle of incidence α of the solar irradiance. Moreover, the irradiance sensing device <NUM> senses irradiance at the same time as images are acquired by the imaging device <NUM>, which enables normalization or compensation of the acquired images to account for variations in received irradiance by the imaged target. For example, an image of a target acquired by the imaging device <NUM> on a cloudy day can be correlated to an image acquired of the same target on a cloudless day, by accounting for the differences in the irradiance sensed by the irradiance sensing device <NUM> at the time of acquiring each image.

<FIG> illustrates the irradiance sensing device <NUM> in further detail, in accordance with one or more embodiments of the present disclosure. The irradiance sensing device <NUM> includes a plurality of photo sensors <NUM> arranged on different surfaces of a base <NUM>. The base <NUM> includes a lower surface <NUM> that may be mounted, for example, to an upper surface of the aerial vehicle <NUM>. Extending from the lower surface <NUM> is a plurality of inclined surfaces <NUM> on which the photo sensors <NUM> may be mounted. As shown in <FIG>, in one or more embodiments, the base <NUM> may have a truncated square pyramid shape, with four inclined surfaces <NUM> extending between the lower surface <NUM> and a flat upper surface <NUM>. One or more photo sensors <NUM> may be mounted on each of the inclined surfaces <NUM> and the upper surface <NUM>. The photo sensors <NUM> may thus be oriented to receive and sense varying amounts or components (e.g., direct and scattered components) of irradiance from a light source such as the sun <NUM>.

The base <NUM> may have any shape or form that includes a plurality of surfaces on which photo sensors <NUM> may be mounted and configured to sense irradiance from differing orientations. The irradiance sensing device <NUM> may preferably include at least four, and in accordance with the claimed invention, includes five photo sensors <NUM>. Accordingly, the base <NUM> may preferably include at least four, and in accordance with the claimed invention, five surfaces having different orientations for mounting the photo sensors <NUM>.

Each photo sensor <NUM> includes a housing <NUM> or some external packaging that houses electronic circuitry (such as one or more application specific integrated circuits, computer-readable memory and the like) for processing and/or storing received signals (e.g., signals indicative of the sensed irradiance), and a photo sensor surface <NUM> for sensing irradiance.

Each of the photo sensors <NUM> may include one or more ports <NUM> for communicating signals (e.g., one or more signals indicative of the sensed irradiance) to or from the photo sensors <NUM>. In one or more embodiments, the photo sensors <NUM> may be coupled to a processor (e.g., by one or more electrical wires or cables coupled to the ports <NUM>) that is included onboard the aerial vehicle <NUM>. The processor may similarly be communicatively coupled to the imaging device <NUM>. Accordingly, the processor may acquire the sensed irradiance by the photo sensors <NUM> at the same time as an image of a target is acquired by the imaging device <NUM>. The irradiance sensed by the irradiance sensing device <NUM> may thus be correlated with the image that is simultaneously acquired by the imaging device <NUM>.

Additionally or alternatively, the photo sensors <NUM> may store the sensed irradiance information as it is acquired during a flight of the aerial vehicle <NUM>. Similarly, the imaging device <NUM> may store images acquired during the flight. The image and irradiance information may later be uploaded to a computing system, which may correlate the stored irradiance and image information based on the time of acquisition of such information, which may be provided through a time stamp or similar information that may be included with the irradiance and image information.

The base <NUM> may be at least partially hollow or may otherwise include an inner cavity, which reduces the weight of the irradiance sensing device <NUM>. Further, additional components of the aerial vehicle <NUM>, such as any electrical or electronic components, may be housed within the inner cavity of the base <NUM>. For example, a processor and/or any other circuitry may be included within the base <NUM> and may be communicatively coupled to the photo sensors <NUM> and/or the imaging device <NUM>.

For irradiance sensing by an aerial vehicle, an irradiance sensing device should provide an instantaneous estimate of both the direct and scattered components, independent of sensor attitude estimates (e.g., which may be provided from an imprecise IMU) and large movements of the aerial vehicle itself. While a single sensor cannot provide such estimates, a multisensor array such as the irradiance sensing device <NUM> provided herein can.

As will be demonstrated below, the direct and scattered components of solar irradiance at any particular time are determined based on the sensed irradiance simultaneously acquired by a plurality of photo sensors <NUM> having different orientations.

For simplicity sake and without loss of generality, a sensor body coordinate system is assumed that has a Z-axis oriented towards the current sun position. In such a coordinate system, the incidence angle α between the sun and a sensor depends only on two angles (the azimuth angle and the zenith angle), since the irradiance is invariant under rotations around the Z-axis.

Rather than trying to directly measure these angles, the azimuth and zenith angles are treated as unknowns to be estimated along with the direct and scattered solar irradiance. Thus in total, we aim to determine four unknowns from a set of five (or more) independent irradiance measurements, which will give us five (or more) non-linear equations. Such a system is readily solvable by standard means, such as Newton's method or least squares.

A system of five sensors (e.g., as shown in <FIG>) having the following configuration provides good results in simulation and allows a stable determination of all unknown quantities.

Note that the only inputs in this method are the known fixed photo sensor orientations and the measured irradiances. No assumptions about the time course of the direct and scattered irradiance are required and no attitudes need to be measured. The estimates of the components of the irradiance are instantaneous and as an added benefit, the photo sensor attitudes are provided in the special solar coordinate system.

It is noted that there are some special circumstances in which this method may not suitably determine the components of irradiance. One such circumstance exists in the absence of any direct light, in which the number of independent equations collapses to just one. However this is a special case that can easily be identified, as in this case all photo sensor readings should be the same, and equal to the scattered irradiance. Also, no meaningful results can be expected when the incidence angle becomes greater than <NUM> degrees for any photo sensor, a case which can be determined by use of an IMU. Note that in this case no particularly high accuracy from the IMU is required, as it is only needed to determine this special threshold.

In view of the above, the irradiance sensing device <NUM> may have a known coordinate system, and a transformation exists and may be determined between the device coordinate system and the global coordinate system, as the position of the sun at any given time is known.

Accordingly, irradiance sensed simultaneously by each of the photo sensors <NUM> of the irradiance sensing device <NUM> is utilized to determine (e.g., by a processor) the direct and scattered components of solar irradiance (as well as the incidence angle α, the azimuth angle φ and the zenith angle θ) that is incident at a particular time on a target that may be imaged by the imaging device <NUM>.

<FIG> is a block diagram illustrating a system <NUM> for estimating or determining irradiance, based on the sensed irradiance from a plurality of photo sensors (e.g., as sensed by the irradiance sensing device <NUM>), and for determining the reflectance of an imaged target or object. The system <NUM> may include a processor <NUM> that is communicatively coupled to the imaging device <NUM> and the irradiance sensing device <NUM> (including photo sensors <NUM> to N).

As noted previously herein, the processor <NUM> may be included onboard the aerial vehicle <NUM> (e.g., housed within a cavity in the base <NUM>, or at any other location on the aerial vehicle <NUM>). In other embodiments, the processor <NUM> may be included as part of a post-processing computer to which the irradiance sensing device <NUM> and/or the imaging device <NUM> may be coupled after an imaging session by the aerial vehicle <NUM>. The post-processing computer may thus determine the components of the sensed irradiance based on the data collected and stored by the irradiance sensing device <NUM>. Similarly, the imaging device <NUM> may capture and store data, which may later be provided to and processed by the processor <NUM>.

Additionally, the processor <NUM> and/or instructions performed by the processor <NUM> (e.g., for determining irradiance components, reflectance values, etc.) may be located in the cloud, i.e., a remote distributed computing network that receives the collected data wirelessly from the imaging device <NUM> and the irradiance sensing device <NUM> or receives the data through a wired network once the imaging device <NUM> and irradiance sensing device <NUM> are coupled to a computer after the imaging session.

The processor <NUM> receives the sensed irradiance information from the irradiance sensing device <NUM> and the acquired image information from the imaging device <NUM>. The processor <NUM> may access an irradiance determination module <NUM>, which contains computer-readable instructions for determining the direct and scattered components of solar irradiance (and may further determine the incidence angle a, the azimuth angle φ and the zenith angle θ) based on the simultaneously sensed irradiance information from the plurality of photo sensors <NUM>, as described herein.

The processor <NUM> provides the determined direct and scattered components of solar irradiance to a reflectance determination module <NUM>, along with image information of a target that was acquired by the imaging device <NUM> at the same time that the irradiance information was acquired. The reflectance determination module <NUM> includes computer-readable instructions for determining the reflectance of the target based on the image information of the target (which may indicate, for example, an amount of light reflected by the target) and the determined components of irradiance at the time the image information was acquired. Accordingly, the determined reflectance of an imaged target is normalized or compensated to account for different irradiance levels that may be present at the time of imaging a target. For example, a determined reflectance for a target based on an image of the target acquired on a cloudy day will be the same or substantially the same as the reflectance for that same target that is determined based on an image of the target that was acquired on a cloudless day.

A compensation factor may thus be determined by the processor <NUM> (based on the determined components of irradiance) and may be applied by the reflectance determination module <NUM> for every image that is acquired by the imaging device <NUM>, in order to accurately determine the reflectance of the imaged target, regardless of the lighting conditions at the time the image was acquired.

<FIG> is a flowchart <NUM> illustrating a method of the present disclosure. At <NUM>, the method includes simultaneously sensing irradiance from a light source by an irradiance sensing device <NUM> including a plurality of photo sensors <NUM> having different orientations with respect to the light source. The photo sensors <NUM> may be arranged, for example, as shown in the irradiance sensing device <NUM> of <FIG>. The photo sensors <NUM> may be included onboard an aerial vehicle <NUM>, and the irradiance may thus be sensed while the aerial vehicle <NUM> is in flight.

At <NUM>, the method includes acquiring an image of a target object by an imaging device <NUM>. The image may be acquired at the same time as the irradiance sensing device <NUM> senses irradiance, and the image and irradiance information may thus be correlated.

At <NUM>, the method includes determining direct and scattered components of the sensed irradiance. And at <NUM>, the method includes determining the reflectance of the target object based on the determined direct and scattered components of the sensed irradiance, and the acquired image of the target object. This method thus provides inherently compensated or normalized reflectance measurements of a target, such as vegetation, that are independent of changes in irradiance from a variable light source (e.g., the sun), and does not require an IMU or calibration of the imaging device. The method may be performed for each image acquired by the imaging device <NUM>.

As is well known, different materials reflect and absorb incident irradiance differently at different wavelengths. Thus, targets can be differentiated based on their spectral reflectance signatures in remotely sensed images. Reflectance is a property of materials and is generally defined as the fraction of incident irradiance that is reflected by a target. The reflectance properties of a material depend on the particular material and its physical and chemical state (e.g., moisture), as well as other properties such as surface texture and other properties that may be known in the relevant field.

The various embodiments provided herein may be thus be utilized in a variety of applications in which determining reflectance of one or more imaged targets may be desirable. For example, by measuring or determining the reflectance of a plant at different wavelengths, areas of stress in a crop may be identified. Moreover, determined changes in reflectance of surface features such as vegetation, soil, water and the like can be utilized to determine the development of disease in crops, growth of algae in a body of water, changes in the chemical properties of ground or soil, and so on.

Various other applications are contemplated by the present disclosure. For example, embodiments provided herein may be utilized in navigational applications, since the orientation of the irradiance sensing device <NUM> is determined with respect to the position of the sun. That is, the orientation of the irradiance sensing device <NUM> may be mapped to the global or horizontal coordinate system, as described above, which may thus be used for navigational purposes by any vehicle including the irradiance sensing device <NUM>. Additionally, it will be appreciated that flight parameters of the aerial vehicle <NUM>, including pitch, heading and roll, may be determined based on the irradiance sensed by the photo sensors <NUM> of the irradiance sensing device <NUM>, and the determined components of the irradiance. As noted previously herein, the estimates of the components of the irradiance are instantaneous and the photo sensor attitudes are provided in the special solar coordinate system. Moreover, the orientation of the irradiance sensing device <NUM> may be mapped to the global or horizontal coordinate system, as described herein. Accordingly, the determined photo sensor attitudes (including, pitch, heading and roll information) provided in the special solar coordinate system may be mapped to the global or horizontal coordinate system for an indication of the aerial vehicle's <NUM> attitude with respect to the Earth. Additionally, changes in the determined photo sensor pitch, heading and roll during flight may be utilized for navigational purposes.

In the description, certain specific details are set forth in order to provide a thorough understanding of various embodiments of the disclosure. However, one skilled in the art will understand that the disclosure may be practiced without these specific details. In other instances, well-known structures have not been described in detail to avoid unnecessarily obscuring the descriptions of the embodiments of the present disclosure.

Unless the context requires otherwise, throughout the specification and claims that follow, the word "comprise" and variations thereof, such as "comprises" and "comprising," are to be construed in an open, inclusive sense, that is, as "including, but not limited to.

As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. It should also be noted that the term "or" is generally employed in its sense including "and/or" unless the content clearly dictates otherwise.

Claim 1:
A device, comprising:
an aerial vehicle (<NUM>); and
an irradiance sensing device (<NUM>) including:
a base structure (<NUM>) mounted to the aerial vehicle (<NUM>), the base structure (<NUM>) including five surfaces (<NUM>, <NUM>), and
five photo sensors (<NUM>) respectively arranged on the surfaces of the base structure (<NUM>) and having different orientations with respect to each other, wherein the five photo sensors (<NUM>) are configured to simultaneously sense irradiance from a light source, and to output signals indicative of the sensed irradiance, and
wherein the device further comprises a processor (<NUM>) coupled to the five photo sensors (<NUM>) and configured to receive the output signals and to determine a direct component and a scattered component of the sensed irradiance, and an azimuth and a zenith angle of the light source in a sensor body coordinate system based on the orientation of the photo sensors and the sensed irradiance,
wherein the direct component and the scattered component of the sensed irradiance, and the azimuth and the zenith angle of the light source are determined by solving at least five non-linear equations formed from a set of at least five independent irradiance measurements,
the device further comprising an imaging device (<NUM>) mounted to the aerial vehicle (<NUM>), wherein the processor (<NUM>) is coupled to the irradiance sensing device and the imaging device, the processor being configured to determine a reflectance of a target based on image information of the target acquired by the imaging device and the determined components of irradiance at the time the image information was acquired, and normalizing or compensating the determined reflectance to account for different irradiance levels at the time of imaging the target.