Aerial vehicle with replaceable camera modules

An aerial vehicle system, such as an unmanned aerial vehicle, that includes a camera system with replaceable camera modules that have different spectral, optical and/or sensing characteristics from one another. By replacing one camera module with another camera module, different information can be gathered and analyzed by the camera system of the aerial vehicle system.

FIELD

The technology described herein generally relates to imaging from aerial vehicles including, but not limited to, unmanned aerial vehicles (UAVs) using cameras.

BACKGROUND

Multispectral camera sensors have become a common tool used in aerial imaging applications such as in agricultural applications. In the case of agricultural applications, by photographing plants and isolating various color spectra, it is possible to learn more about the health of the plants than could be gained with the naked eye. However, depending on factors such as the type of plant, the growth stage of the plant, the season (for example, spring, summer, fall, winter), and the feature of the plant being detected, different spectral data could be required in order to permit a proper analysis of the plants. For example, different spectral bands are useful for different plants, different stages of plant growth, different seasons, and the like. Additionally, higher or lower resolution imagery of a plant may also be required, typically defined by ground sample distance (GSD). GSD defines how much area is covered by a single camera pixel. For example, if one is attempting to count emerging corn plants, a very low GSD is needed such that each corn plant is represented by several pixels in the resulting images. Conversely, if one is attempting to measure the overall health of a closed canopy corn field, a much larger GSD is sufficient.

SUMMARY

An aerial vehicle system is described that includes a unique imaging system, for example a camera, that reduces the size, complexity and cost of implementing an imaging system that is useful over a variety of aerial imaging applications such as, including but not limited to, in agricultural applications. For example, the imaging system is designed to be used with replaceable camera modules that have different spectral, optical and/or sensing characteristics from one another. By replacing one camera module with another camera module, different information can be gathered and analyzed.

The types of aerial vehicles that the concepts described herein can be implemented on include, but are not limited to, unmanned aerial vehicles (UAVs) such as quad-copter or quad-rotor UAVs, other rotorcraft UAVs, fixed-wing UAVs, or other types of UAVs.

In one embodiment, a system described herein includes an aerial vehicle, such as a UAV, and a camera system mounted on the aerial vehicle. The camera system includes a camera module mounting location, an interface connection located at the camera module mounting location, where the interface connection provides an interface for at least one of or both data and power, a processor electrically connected to the interface connection, and a plurality of camera modules. Each camera module can be detachably mountable in the camera module mounting location, and each camera module can electrically interface with the interface connection when mounted in the camera module mounting location to provide at least one of data and power between the interface connection and the camera module mounted in the camera module mounting location. In addition, each camera module has two or more spectral, optical and/or sensing characteristics, and the spectral, optical and/or sensing characteristics of the camera modules are different from one another. Therefore, by replacing one camera module with another camera module, the functioning of the camera system can be altered to extend the usefulness of the camera system over a variety of spectral bands, optical characteristics, and sensing capabilities.

In another embodiment described herein, a camera module of a camera system of an aerial vehicle such as a UAV is provided. The camera module can be detachably mountable in a camera module mounting location of the camera system. The camera module includes an assembly that has a lens, an optional optical filter, and a sensor, with the assembly being detachably mountable in the camera module mounting location of the camera system of the UAV. The assembly further includes an electrical interface that is connectable with an interface connection of the camera module mounting location to provide at least one of or both data and power between the interface connection and the camera module. In addition, the assembly has optical and sensing characteristics, and spectral characteristics if an optical filter is present, for the intended application of the camera system. For example, the optical, sensing and optional spectral characteristics can make the camera module suitable for imaging plants in a field. The camera module can be one of a number of similar camera modules, with each camera module having different optical, sensing and/or spectral characteristics.

In still another embodiment described herein, a method of gathering data on agricultural plants is described. The method can include detachably mounting a first camera module in a camera module mounting location of a camera system of an aerial vehicle such as a UAV, where the first camera module has first optical, sensing and/or spectral characteristics. The UAV is then flown over a field and agricultural plants are imaged using the first camera module. Thereafter, the first camera module is removed and a second camera module is installed in the camera module mounting location of the camera system of the UAV, where the second camera module has second optical, sensing and/or spectral characteristics that are different from the first optical, sensing and/or spectral characteristics. The UAV is then flown over a field, which could be the same field or a different field, and agricultural plants are imaged using the second camera module.

The camera module concepts described herein can be used in precision agriculture applications to improve farming management. For example, the camera modules can be used to image plants or crops in one or more fields and/or soil conditions in the field(s). The camera modules may also be used to image non-agricultural plants such as trees. However, the camera modules are not limited to imaging plants in agricultural or non-agricultural applications, and many other applications are possible.

DETAILED DESCRIPTION

Systems and methods are described where an aerial vehicle such as a UAV has a camera system (also referred to as an imaging system) that has replaceable camera modules that have different spectral, optical and/or sensing characteristics from one another. By replacing one camera module with another camera module in the camera system, different information can be gathered and analyzed when the camera system is used for imaging one or more targets.

To assist in describing the concepts herein, the aerial vehicle will be described as a UAV, and the UAV, the camera system, and the camera modules will be described as being used to image plants growing in a field(s) for precision agriculture to improve farming management. However, the UAV, the camera system, and the camera modules can be used to analyze other agronomic information, such as soil conditions, for precision agriculture to improve farming management. The UAV, the camera system, and the camera modules may also be used in non-agronomy applications for example imaging non-agricultural plants such as trees. Further, the UAV, the camera system, and the camera modules can be used in many other applications.

FIG.1illustrates an example environment5where a UAV10described herein can be used. In this example, a field12receives incident light14, for example sunlight from the sun16, with the incident light14being reflected by the field12(for example the soil20, plants/crops22growing in the soil20, other vegetation growing in the soil20, etc.) in the form of reflected light18to a camera system24(also referred to as an imaging system or an image capture device) mounted on the UAV10. The camera system24captures images of the field12which can then be analyzed to gather information about the field12, for example the condition of the plants22and/or the condition of the soil20.

FIG.2schematically depicts an example of the camera system24. The camera system24is designed and configured to be used with a plurality of replaceable camera modules30a,30b, . . .30nwhere the combined optical and sensing characteristics, and optionally spectral characteristics, of the camera modules are different from one another. The camera modules30a-n, which can be considered part of the camera system24or not part of the camera system24, are each detachably mountable in the camera system24to selectively change the optical, sensing and/or spectral characteristics of the camera system24by swapping out the camera modules30a,30b, . . .30n.

Spectral characteristics of the camera module refers to characteristics or capabilities imparted to the camera module by one or more filter elements in the camera module. Optical characteristics of the camera module refers to characteristics or capabilities imparted to the camera module by one or more lens elements in the camera module. Sensing characteristics of the camera module refers to characteristics or capabilities imparted to the camera module by one or more sensors in the camera module.

A camera module30a,30b, . . .30nas used herein is defined as an optical sub-system of the camera system24that is removably installable in the camera system24and that when installed on the camera system24provides unique spectral, optical, and/or sensing characteristics compared to another one of the camera modules30a,30b, . . .30n. The camera system24may or may not have what can be considered a fixed optical system that permits the camera system24to perform imaging functions in addition to and separate from imaging functions performed by the removable and replaceable camera modules30a,30b, . . .30n. However, the camera system24is configured to permit one or more of the camera modules30a,30b, . . .30nto be interchangeably and selectively removably mounted thereon to permit changes in the imaging capabilities of the camera system24and thus of the UAV10.

With reference toFIGS.3and4, in one embodiment each one of the camera modules30a,30b, . . .30ncan include at least one lens32, optionally at least one optical filter34, at least one sensor36, and means for transferring data and power between the camera system24and the camera module30a,30b, . . .30n. In the illustrated embodiment, the lens32, the optical filter34, and the sensor36are combined together into a single integrated unit forming the camera module30a,30b, . . .30n. The lens32, the optical filter34, and the sensor36may be combined together by being mounted together in a common housing38as best seen inFIG.4. Each one of the camera modules30a,30b, . . .30ncan include other elements not illustrated, such as a diffuser, coatings and other elements known in the art. The lens32, the optical filter34, and the sensor36should be sealed in the housing38and should stay tightly coupled to maintain the intended focus and to keep debris from entering the housing38.

In another embodiment, one or more of the camera modules30a,30b, . . .30nmay not include the optical filter34but includes the lens32and the sensor36. This is indicated by the dashed lines inFIGS.3and6which indicates the optical filter34as being an optional component of the camera modules30a,30b, . . .30n.

The lens32can be any type of lens having any shape and/or configuration depending upon the desired optical characteristics, such as the field of view or other optical characteristics, of the camera module30a,30b, . . .30n. In one non-limiting example, the lens32can be a focusing lens. The lens32can also be provided with zoom capability and/or adjustable iris. The lens32can be fixed in position or the lens32can be adjustable in position, for example in a directed toward and/or away from the optical filter34, as indicated by the arrow40inFIG.6.

The optical filter34, if used, can be any type of optical filtering element having any construction depending upon the desired optical filtering/spectral characteristics. For example, the optical filter34can be a bandpass filter that controls the spectrum of light that reaches the sensor36and/or the optical filter34can be a spectral shaping filter that shapes the light reaching the sensor36. The optical filter34can be fixed in position or the optical filter34can be adjustable in position, for example in a directed toward and/or away from the lens32or toward and/or away from the sensor36, as indicated by the arrow42inFIG.6.

FIGS.3and4illustrate each camera module30a,30b, . . .30nas having a single one of the optical filters34located a fixed distance between the lens32and the sensor36. However, with reference toFIG.5, other configurations are possible. For example, a first optical filter34a, for example a bandpass filter, and a second optical filter34b, for example a spectral shaping filter, can be located between the lens32and the sensor36. The first optical filter34acould be the spectral shaping filter while the second optical filter34bcould be the bandpass filter. In addition or alternatively, an optical filter34, for example a bandpass filter or a spectral shaping filter, can be located in front the lens32located within the housing38and/or located in front of the lens32outside the housing38.

Returning toFIGS.3and4, the sensor36can be any type of sensor that can sense light impinging thereon. In one non-limiting example, the sensor36can be a linear or area focal plane array, formed by an array of detection elements. The detection elements can be photoresistors, photodiodes, phototransistors or any other elements suitable for being arranged in an array and for detecting electromagnetic waves. In other embodiments, the sensor36could be a thermal sensor. One module can employ a higher resolution rolling shutter camera and one module can employ a lower resolution global shutter camera. The term “rolling shutter” refers to the way the image sensor scans the image, where the image is scanned sequentially from one side of the sensor to the other, line by line. Many complementary metal-oxide-semiconductor (CMOS) sensors use rolling shutters. The term “global shutter” refers to sensors that scan the entire area of the image simultaneously. The vast majority of charge-coupled device (CCD) sensors employ global shutter scanning. The sensor36can be fixed in position or the sensor36can be adjustable in position, for example in a directed toward and/or away from the optical filter34, as indicated by the arrow44inFIG.6.

A suitable data/power interface48is provided for passing data and power between the camera modules30a,30b, . . .30nand the rest of the camera system24. The data/power interface48can have any construction that is suitable for passing data and power between the camera modules30a,30b, . . .30nand the rest of the camera system24. For example, referring toFIG.4, the sensor36can be mounted on a sensor board46which forms part of the camera module30a,30b, . . .30n. The sensor board46can include one or more power and data vias, which are known, or other electrical constructions for passing power and data through or past the sensor board46to the sensor36and any other elements of the camera modules30a,30b, . . .30n. The sensor board46can electrically connect to a power and data distribution board50of the camera system24using one or more electrical connectors52that electrically interface with the vias or electrical connectors in the sensor board46on the backside thereof. In addition, a processor54of the camera system24that processes the images sensed by the sensor36and performs analytics on the images can electrically interface with the power and data interface48, for example using one or more electrical connectors56that electrically interface with vias or other electrical connectors in the power and data distribution board50on the backside thereof. In one embodiment as illustrated inFIG.5, the data/power interface48and the processor54can be combined into a single element.

With reference toFIG.4, the camera system24is formed with one or more camera module mounting locations60each of which provides a location for removably mounting one of the camera modules30a,30b, . . .30nto the camera system24. A single camera module mounting location60can be provided, or two or more camera module mounting locations60can be provided. Each camera module mounting location60is an area on the camera24that each one of the camera modules30a,30b, . . .30ncan removably mount into, and fix the position of the camera module30a,30b, . . .30nduring use of the UAV10. As depicted inFIG.4, the power and data distribution board50and the processor54are provided at a base of each one of the camera module mounting locations60, and the camera module30a,30b, . . .30ncan be installed in the camera module mounting location(s)60and fixed in position. When fully and properly installed, the camera module30a,30b, . . .30nelectrically interfaces with the power and data distribution board50and the processor54to pass power and data between the camera module30a,30b, . . .30nand the camera system24.

Any technique for removably securing each one of the camera modules30a,30b, . . .30nto the camera module mounting locations60can be utilized. In the non-limiting example illustrated inFIG.4, the housing38of each camera module30a,30b, . . .30ncan include a snap ring62that is engageable in a detent groove64formed on an interior wall of the camera system24that forms the camera module mounting location60. Each camera module30a,30b, . . .30ncan further include a seal66, for example an o-ring seal, that seals with the interior wall or other part of the camera system24when the camera module30a,30b, . . .30nis fully and properly installed in the camera module mounting location60as illustrated at the right-hand side ofFIG.4to prevent ingress of contaminants into the camera module mounting location60.

Different ones of the camera modules30a,30b, . . .30ncan be created each one having its own unique combination of two or more of spectral, optical and/or sensing characteristics or properties. This can be achieved by utilizing different types of, and different combinations of, the lenses32, optical filters34, and/or the sensors36. The camera modules could have different lenses32but the same optical filters34and sensors36; different optical filters34, but the same lenses32and sensors36; different sensors36, but the same lenses32and optical filters34; different lenses32and optical filters34, but the same sensors36; different lenses32and sensors36, but the same optical filter(s)34; different optical filter(s)34and sensors36, but the same lenses32; etc. Any combination can be used as long as one or more of the spectral, optical and/or sensing characteristics of the camera modules are different so that the combined spectral, optical and/or sensing characteristics of each one of the camera modules are different from one another.

The various camera modules30a,30b, . . .30ncan be swapped out from the camera system24as needed to change the imaging capabilities of the UAV10, while maintaining the same data/power interface board50and processor54. The user can attach the appropriate camera module(s)30a,30b, . . .30n, having the desired optical, sensing and/or spectral characteristics, for their specific need, for example given the plant type, growth stage, season, and desired data outcome in the case of an agricultural application. This allows the camera system24to be extremely versatile to support a variety of uses without the camera system24having to simultaneously support all optical, sensing and spectral options, thereby reducing the size, weight, and cost of the AUV10.

The camera modules30a,30b, . . .30ncan be a single assembly of the lens32, the optional optical filter34and the sensor36. Or the camera modules30a,30b, . . .30ncan be composed of two or more assemblies, with each assembly including the lens32, the optional optical filter34and the sensor36. For example, the camera system34inFIG.4could be configured with two of the camera module mounting locations60, and the two camera modules depicted inFIG.4can be paired with one another so that they are installed and/or removed as a unit (i.e. together). These paired camera modules could have a single interconnect to the data/power interface board50, or they could utilize two independent interconnects to the data/power interface board50. The camera system34can include more than two camera module mounting locations60receiving multiple ones of the paired camera modules, or a combination of one or more paired camera modules along with one or more individual or single camera modules.

In an agricultural application, the camera system24and the camera modules30a,30b, . . .30nwill face generally downward toward the field12in order to detect the reflected light18. However, in other applications, the camera system24and the camera modules30a,30b, . . .30ncould face sideways (or generally parallel to ground) and could even face upward.

In one example application of the system described herein, a method of gathering data on agricultural plants can include detachably mounting a first one of the camera modules30a,30b, . . .30nin the camera module mounting location60of the camera system24of the UAV10, where the first camera module has first optical, sensing and optional spectral characteristics. The UAV10can then be flown over a field and agricultural plants imaged using the first camera module. Thereafter, the first camera module can be removed and a second one of the camera modules installed in the camera module mounting location60of the camera system24, where the second camera module has second optical, sensing and optional spectral characteristics that are different from the first optical, sensing and spectral characteristics. The UAV10can then be flown over a field, which could be the same field or a different field, and agricultural plants can be imaged using the second camera module. A similar process can be repeated for third, fourth and any additional number of the camera modules30a,30b, . . .30n.

In the example of an agricultural application, the agricultural plants imaged by the first camera module can be the same type of agricultural plants imaged by the second camera module, in either the same or different fields. The agricultural plants imaged by the first camera module can be different from the agricultural plants imaged by the second camera module, in either the same field or in different fields.