Apparatus, system, and method for determining an object's location in image video data

Described herein is a method of displaying an object that includes detecting a first location of the object on a first image frame of image data. The method also includes determining a second location of the object on a second image frame of the image data on which the object is not detectable due to an obstruction. The method includes displaying a representation of the object on the second image frame.

FIELD

This disclosure relates generally to image sensing, and more particularly to determining an object's location in image video data.

BACKGROUND

Some image sensors are used to obtain image sequence data by perceiving objects in day and/or night settings. The clarity of the image sequence data may be affected by environmental factors such as fog, haze, sand-brownouts, smoke, rain, snow, steam, and so forth. Unclear image sequence data may be difficult to use.

SUMMARY

The subject matter of the present application has been developed in response to the present state of the art, and in particular, in response to shortcomings of conventional image processing techniques. For example, conventional image processing techniques do not provide sufficient image clarity under certain conditions, such as in degraded visibility environments where visibility is degraded due to fog, haze, sand-brownouts, smoke, rain, snow, steam, and so forth.

Accordingly, the subject matter of the present application has been developed to provide examples of an image enhancing apparatus, system, and method that overcome at least some of the above-discussed shortcomings of prior art techniques. More particularly, in some embodiments, described herein are apparatuses, systems, and methods for determining an object's location in image video data.

A method of displaying an object identified from image data captured from a mobile vehicle includes detecting a first location of the object on a first image frame of image data. The method also includes determining a second location of the object on a second image frame of the image data on which the object is not detectable due to an obstruction. The method includes displaying a representation of the object on the second image frame. The preceding subject matter of this paragraph characterizes example 1 of the present disclosure.

The image data includes infrared image data. The preceding subject matter of this paragraph characterizes example 2 of the present disclosure, wherein example 2 also includes the subject matter according to example 1, above.

The first location of the object is detected based on intensity values corresponding to the infrared image data. The preceding subject matter of this paragraph characterizes example 3 of the present disclosure, wherein example 3 also includes the subject matter according to any one of examples 1 or 2, above.

A method includes determining whether the object is stationary or moving. The preceding subject matter of this paragraph characterizes example 4 of the present disclosure, wherein example 4 also includes the subject matter according to any one of examples 1, 2, or 3, above.

A method includes, in response to the object determined to be stationary, determining the second location of the object on the second image frame based on the first location of the object and a change in position of a vehicle used to capture the image data. The preceding subject matter of this paragraph characterizes example 5 of the present disclosure, wherein example 5 also includes the subject matter according to any one of examples 1, 2, 3, or 4, above.

A method includes, in response to the object determined to be moving, determining the second location of the object on the second image frame based on the first location of the object, a predicted location of the object, and a change in position of a vehicle used to capture the image data. The preceding subject matter of this paragraph characterizes example 6 of the present disclosure, wherein example 6 also includes the subject matter according to any one of examples 1, 2, 3, 4, or 5, above.

A method includes synchronizing the image data with navigation data of a vehicle. The preceding subject matter of this paragraph characterizes example 7 of the present disclosure, wherein example 7 also includes the subject matter according to any one of examples 1, 2, 3, 4, 5, or 6, above.

Detecting a first location of the object includes detecting a first location of an object including a landing area on a first image frame of image data. The preceding subject matter of this paragraph characterizes example 8 of the present disclosure, wherein example 8 also includes the subject matter according to any one of examples 1, 2, 3, 4, 5, 6, or 7, above.

A method includes determining whether the object is stationary or moving, and in response to a determination that the object is stationary, determining the second location of the object on the second image frame in which the object is not detectable due to an obstruction, based on the first location of the object and a change in position of the mobile vehicle. The preceding subject matter of this paragraph characterizes example 9 of the present disclosure, wherein example 9 also includes the subject matter according to any one of examples 1, 2, 3, 4, 5, 6, 7, or 8, above.

Displaying the representation of the object on the second image frame includes displaying, on a display device associated with the mobile vehicle, a representation of the object comprising a landing area that is overlaid on the second image frame where the object is expected to be, such that the object that is obscured in the second image frame is displayed as a representation of the object to an operator of the mobile vehicle. The preceding subject matter of this paragraph characterizes example 10 of the present disclosure, wherein example 10 also includes the subject matter according to any one of examples 1, 2, 3, 4, 5, 6, 7, 8, or 9, above.

A mobile vehicle includes a sensor configured to detect an image and to produce image data associated with the image. The mobile vehicle also includes a processor operatively coupled to the sensor and configured to receive the image data. The mobile vehicle includes a memory that stores code executable by the processor to: detect a first location of an object on a first image frame of the image data; predict a second location of the object on a second image frame of the image data; and display a representation of the object on the second image frame. The preceding subject matter of this paragraph characterizes example 11 of the present disclosure.

The sensor includes an infrared image sensor. The preceding subject matter of this paragraph characterizes example 12 of the present disclosure, wherein example 12 also includes the subject matter according to example 11, above.

The mobile vehicle includes a display device operatively coupled to the processor and configured to display the representation of the object on the second image frame. The preceding subject matter of this paragraph characterizes example 13 of the present disclosure, wherein example 13 also includes the subject matter according to any one of examples 11 or 12, above.

A mobile vehicle includes a navigation unit that produces navigation data for the mobile vehicle. The preceding subject matter of this paragraph characterizes example 14 of the present disclosure, wherein example 14 also includes the subject matter according to any one of examples 11, 12, or 13, above.

A mobile vehicle in which the code is executable by the processor to synchronize frames of the image data with the navigation data. The preceding subject matter of this paragraph characterizes example 15 of the present disclosure, wherein example 15 also includes the subject matter according to any one of examples 11, 12, 13, or 14, above.

A mobile vehicle in which the code is executable by the processor to determine whether the object is stationary or moving. The preceding subject matter of this paragraph characterizes example 16 of the present disclosure, wherein example 16 also includes the subject matter according to any one of examples 11, 12, 13, 14, or 15, above.

A mobile vehicle in which the code is executable by the processor to, in response to the object determined to be stationary: determine a first position of the mobile vehicle corresponding to the first image frame; determine a second position of the mobile vehicle corresponding to the second image frame; and predict the second location of the object based on the first location of the object, the first position, and the second position of the mobile vehicle. The preceding subject matter of this paragraph characterizes example 17 of the present disclosure, wherein example 17 also includes the subject matter according to any one of examples 11, 12, 13, 14, 15, or 16, above.

A mobile vehicle in which the code is executable by the processor to, in response to the object determined to be moving: determine a first position of the mobile vehicle corresponding to the first image frame; determine a second position of the mobile vehicle corresponding to the second image frame; generate a motion model for the object based on the first location of the object and another location of the object detected before the first image frame; and predict the second location of the object based on the motion model for the object, the first position of the object. The preceding subject matter of this paragraph characterizes example 18 of the present disclosure, wherein example 18 also includes the subject matter according to any one of examples 11, 12, 13, 14, 15, 16, or 17, above.

The memory that stores code includes an object detection module executable by the processor to detect a first location of the object including a landing area on a first image frame of image data, and a prediction module executable by the processor to predict a second location of the object on a second image frame in which the object is not detectable due to an obstruction, based on the first location of the object and a change in position of the mobile vehicle. The preceding subject matter of this paragraph characterizes example 19 of the present disclosure, wherein example 19 also includes the subject matter according to any one of examples 11, 12, 13, 14, 15, 16, 17, or 18, above.

A mobile vehicle includes a display device associated with the mobile vehicle, for displaying the representation of the object on the second image frame that is overlaid on the second image frame where the object is expected to be, such that the object including a landing area that is obscured in the second image frame is displayed as a representation of the object to an operator of the mobile vehicle. The preceding subject matter of this paragraph characterizes example 20 of the present disclosure, wherein example 20 also includes the subject matter according to any one of examples 11, 12, 13, 14, 15, 16, 17, 18, or 19, above.

An apparatus includes an object detection module that detects a first location of an object on a first image frame of image data based on intensity values corresponding to the first image frame. The apparatus also includes a prediction module that determines a second location of the object on a second image frame of the image data on which the object is not detectable due to an obstruction. The apparatus includes a display module that displays a representation of the object on the second image frame. At least one of the object detection module, the prediction module, and the display module comprises one or more of hardware and executable code, the executable code stored on one or more non-transitory computer readable storage media. The preceding subject matter of this paragraph characterizes example 21 of the present disclosure.

An apparatus includes a linear motion module that generates a linear motion model in response to the object moving and provides information corresponding to the linear motion model to the prediction module to facilitate determining the second location of the object. The preceding subject matter of this paragraph characterizes example 22 of the present disclosure, wherein example 22 also includes the subject matter according to any one of example 21, above.

An apparatus includes a navigation module that generates navigation data for a vehicle used to capture the image data. The preceding subject matter of this paragraph characterizes example 23 of the present disclosure, wherein example 23 also includes the subject matter according to any one of examples 21 or 22, above.

An apparatus includes a mapping module that maps image frames of the image data to the navigation data. The preceding subject matter of this paragraph characterizes example 24 of the present disclosure, wherein example 24 also includes the subject matter according to any one of examples 21, 22, or 23, above.

Each sampling time period corresponding to the navigation data is mapped to an image frame of the image data. The preceding subject matter of this paragraph characterizes example 25 of the present disclosure, wherein example 25 also includes the subject matter according to any one of examples 21, 22, 23, or 24, above.

DETAILED DESCRIPTION

Referring toFIG. 1, and according to one embodiment, an environment100in which image data may be received and/or processed is illustrated. As used herein, image data may refer to one or more images, image video data, video data, a sequence of images, and/or image sequence data. In the illustrated environment100, a mobile vehicle105and/or a stationary platform110may be used to receive and/or process image data. In certain embodiments, the mobile vehicle105may be an aircraft such as an airplane, a helicopter, a jet, a drone, and so forth flyable above a ground surface115. In other embodiments, the mobile vehicle105may be a rocket, a satellite, a missile, and so forth. Moreover, in some embodiments, the stationary platform110may be part of a ground surveillance system positioned on the ground surface115.

Both the mobile vehicle105and the stationary platform110may include a sensor120used to detect or capture optical images of objects, such as an object125, and to convert the optical images of objects into image data associated with the images. As may be appreciated, the sensor120may be any suitable image sensor, such as an infrared (IR) sensor, a semiconductor charge-coupled device (CCD), an active pixel sensor, and so forth. The image data associated with the images may be produced and/or provided to another device. For example, in the illustrated embodiment, the sensor120may provide the image data to an image processing system130to process the image data and/or to enhance the quality of the image data. As illustrated, an imaging system135includes the sensor120and the image processing system130. The mobile vehicle105includes a navigation module137having a navigation unit that produces navigation data (e.g., position, orientation, pitch, roll, yaw, etc.) that may be used by the imaging system135.

As illustrated, a degraded visibility environment140may block (e.g., temporarily, momentarily, for a period of time) the sensor120from sensing a clear image of the object125, thus resulting in degraded image data. The degraded visibility environment140may be any type of environment that reduces the quality of the sensed image obtained from the sensor120. For example, the degraded visibility environment140may include fog, haze, sand-brownout, smoke, rain, snow, steam, and so forth. The image processing system130may be used to predict a position of the object125while the degraded visibility environment140is present. The object125may be located within or adjacent to the degraded visibility environment140. Similarly, the mobile vehicle105and stationary platform110also may be located within or adjacent to the degraded visibility environment140.

The image processing system130may be used to display the object125even when the degraded visibility environment140blocks the object125. To display the object125, the image processing system130may detect a first location of the object125on a first image frame of image data on which the object125is visible. The image processing system130may also determine a second location of the object125on a second image frame of the image data on which the object125is not detectable due to an obstruction. The image processing system130may display a representation of the object125on the second image frame. Accordingly, the image processing system130may display the object125even if image sequence data obtained by the sensor120is degraded.

FIG. 2is a schematic block diagram of an embodiment of the image processing system130. The image processing system130includes a processor200, memory205, communication hardware210, a display module215, and an image processing module220. The memory205may be a semiconductor storage device, a hard disk drive, an optical storage device, a micromechanical storage device, or combinations thereof. Furthermore, the memory205may store code and the processor200may be used to execute the code. In certain embodiments, the processor200may be operatively coupled to the sensor120and configured to receive image data from the sensor120. Moreover, the communication hardware210may communicate with other devices. The display module215may be a display device operatively coupled to the processor200and used to display data, such as image data and/or enhanced image data. The image processing module220may include various modules used to enhance image data received from the sensor120.

In certain embodiments, the memory205may store code executable by the processor to: detect a first location of an object on a first image frame of the image data; predict a second location of the object on a second image frame of the image data; display a representation of the object on the second image frame; synchronize frames of the image data with the navigation data; determine whether the object is stationary or moving; determine a first position of the mobile vehicle corresponding to the first image frame; determine a second position of the mobile vehicle corresponding to the second image frame; predict the second location of the object based on the first location of the object, the first position of the mobile vehicle, and the second position of the mobile vehicle; determine a first position of the object corresponding to the first image frame; determine another position of the object corresponding to the image frame before the first image frame; generate a motion model for the object based on the first location of the object and the another location of the object; and/or predict the second location of the object in the second image frame based on the motion model for the object, the first position of the object, and the sampling time of the second image frame.

FIG. 3is a schematic block diagram of one embodiment of the image processing module220. The image processing module220receives image sequence data300, such as image sequence data from a degraded environment. An object detection module305obtains the image sequence data300, such as by using the sensor120. The object detection module305may acquire multiple images that may be part of a video in an environment outside of a vehicle (e.g., mobile vehicle105, stationary vehicle110). Moreover, the object detection module305may detect a first location of an object125on a first image frame of image data based on intensity values corresponding to the first image frame. In various embodiments, the object detection module305may be used to detect and/or track an object125in a clear screen that is visible because the environment is not degraded to the point to obscure detection. The object detection module305may, for example, comprise a software module that, when executed by a processor, operates to detect an object in the first image frame based on intensity values in the first frame, as explained below.

In certain embodiments, when the landing ground115is in a clear condition, an object125may be detected and tracked from infrared image sequences. Because infrared images have no texture on objects, object detection in infrared images may be based on intensity values of the objects. Thresholding-intensity based detection and saliency based detection may be widely used in object detection from infrared images. In certain threshold based techniques: a threshold is predetermined; pixels with intensity values larger than the threshold are treated as object pixels; and objects are determined from the object pixels with some size constraint on the detected object. In saliency based object detection: some special filters may be applied to infrared images for computing saliency features; and then thresholding based techniques may be applied to the saliency features for extracting objects. In various embodiments, saliency based techniques may be more robust than intensity based value based thresholding techniques.

To track objects in infrared image sequences, prediction techniques may be used to track next positions of objects in image sequences. Kalman filtering and linear prediction models may be used for object tracking in infrared images. To apply Kalman filtering to object-tracking problem, an object tracking system may be modeled as a linear system by Equation 1.

where X=[r, v, a]Tis internal states; the variable r is object range; the variable v is object speed; the variable a is object acceleration; the variable Y is observation vector; the variables u and w are independent random noises. The state transition matrix A is given by Equation 2.

The variable T is a sampling time step. The transition matrix may assume that objects have constant accelerations. The observation matrix C is given by Equation 3.

In Kalman filtering, a procedure of predicting the states of time n using the observation at time n−1 may be given by the set of Equations 4-8.
{circumflex over (X)}n|n−1=A{circumflex over (X)}n−1|n−1Equation 4
{circumflex over (P)}n|n−1=A{circumflex over (P)}n−1|n−1AT+QuEquation 5
Kn=Pn|n−1CT[CPn|n−1CT+Qw]−1Equation 6
{circumflex over (X)}n|n={circumflex over (X)}n|n−1+Kn[YnC{circumflex over (X)}n|n−1]  Equation 7
Pn|n=[1−KnC]Pn|n−1Equation 8

where Qu=E[uuT] and Qw=E[wwT] are covariance matrixes for the noise processes, u(t) and w(t), respectively. The symbols K(n) and P(n) are the Kalman gain vector and the state covariance matrix. The Kalman filtering may be initiated by Equations 9 and 10.
{circumflex over (X)}0|0=E[X0]  Equation 9
P0|0=E[X0X0T]  Equation 10

The prediction of a future observation may be written as found in Equation 11.
Ŷn+1=C{circumflex over (X)}n|nEquation 11

In a linear prediction model, objects may be assumed to move in constant speeds. As a result, future positions of objects (whose locations have been detected in the first image frame and a number of subsequent image frames) may be predicted by Equation 12.

where rn+1is the future value of an object range; vn+1is the future value of object speed; and T is a sampling interval of time. The object speed, v(n)=[vrow(n), vcol(n)]T, may be estimated by object displacement as shown in Equations 13 and 14.

The vector p(n)=[i(n), j(n)]Tis the position vector of an object on the image plane at time n.

Navigation data310from a navigation unit and image data (e.g., image frames, image frame data, object data) from the object detection module305are provided to a mapping module315. The mapping module315maps (e.g., synchronizes) image frames of the image data to the navigation data. For example, in certain embodiments, each sampling time period corresponding to the navigation data310is mapped to an image frame of the image data for the associated time period.

In certain embodiments, infrared image frames and the navigation data310may have different data rates, which may mean that they may not be sampled at the same time interval. Accordingly, to use the navigation data310to process infrared image frames, the mapping module315may match the navigation data310to the infrared image frames in a time index (e.g., time stamps). In one embodiment, a time instant of navigation data310that is the closest to the time instant of an infrared image frame may be used as the matched time instant of the navigation data310for a respective infrared image frame. The mapping module315may use Equations 15 and 16 to synchronize the infrared image frames and the navigation data310. For Equations 15 and 16, let tnav(n) be an indexed time for navigation data and tIR(n) be an indexed time for infrared image frames. For the kthinfrared image frame, the navigation time index is computed as shown in Equations 15 and 16.

The navigation at time tnav(k*) may be used as the matched navigation data310for the kthinfrared image frame. The threshold η may be set to one millisecond because navigation data310may have a higher sampling rate than infrared image frames generally.

A prediction module320determines a location of the object125on an image frame of the image data on which the object is not detectable due to an obstruction (e.g., degraded visibility environment140). The prediction module320may determine the location of the object125for moving objects and/or for stationary objects, based on the detected location of the object125in preceding image frames that are not obscured as a result of a degraded visibility environment. An object identification module325uses object data from the object detection module305and predicted object locations from the prediction module320to identify predicted locations on images where objects are expected to be, when the objects in the image are obscured as a result of a degraded visibility environment. As a result of a degrade visibility environment, the objects are obscured in the images and their locations are not known, absent the present approach of identifying the predicted locations on the images where objects are expected or predicted to be. The object identification module325outputs enhanced image data330that may be displayed by the display module215. For example, the display module215may display a representation of the object on the images (by projection onto or overlaying the representation of the object onto the image) in a predicted location where the predicted object is expected to be.

FIG. 4is a schematic block diagram of one embodiment of the prediction module320. The prediction module320includes a stationary object prediction module400used to predict object locations for stationary objects, and a moving object prediction module405used to predict object locations for moving objects.

In various embodiments, the moving object prediction module405determines position changes of moving objects on the ground using two types of motions (e.g., the object125movement and movement of the mobile vehicle105). In a seeing phase (e.g., with a clear image frame), motion vectors of moving objects may be estimated. In some embodiments, it may be difficult to separate the two types of motion from the motion vectors. In other words, it may be difficult to separate the object125movement from the motion vectors corresponding to the mobile vehicle105. In images having degraded visibility environment140, the mobile vehicle105moving dynamics may be estimated from the navigation data310, but the object125moving dynamics may be unknown; therefore, it may be difficult to obtain the combined movements for the moving objects. In certain embodiments, combined motion of the object125and of the mobile vehicle105may be assumed to be a smooth linear motion with a constant speed. In some embodiments, the combined motion is modeled by a linear motion module410or a Kalman filter model.

The linear motion module410may generate a linear motion model in response to the object125moving and provides information corresponding to the linear motion model to the prediction module320to facilitate determining a location of the object125. The linear motion model may be estimated during a clear image (e.g., seeing phase) without degraded visibility. In an image that has degraded visibility (e.g., a remembering phase), the estimated model may be used for position prediction of moving objects125. In some embodiments, for the linear motion model, the next position of an object125may be given by Equation 17.
Pn+1=Pn+v×ΔtEquation 17

The speed v is estimated in the seeing phase; the variable Δt is the time interval between the new position and the old position.

For a Kalman filter model, a Kalman gain factor, K(n), is treated as a constant estimated in the seeing phase. The next position (e.g., range) of an object may be estimated using Equations 4 through 11. For an estimated moving direction angle, θ, the next position vector, p(n)=[i(n),j(n)]T, is calculated using Equations 18 and 19.
i(n)=r(n)sin(θ)  Equation 18
j(n)=r(n)cos(θ)  Equation 19

FIG. 5is a schematic block diagram of one embodiment of the stationary object prediction module400. The position changes of stationary objects on the image plane may be mainly determined by the movement of the mobile vehicle105because the locations of stationary objects on the ground are fixed, and only the viewing position changes due to the mobile vehicle105moving. The position changes of the mobile vehicle105may be calculated from navigation data. New positions of stationary objects on the image plane may be estimated by the movement of the mobile vehicle105.FIG. 5illustrates one embodiment of a procedure for the stationary object prediction module400to estimate new positions of stationary objects in degraded conditions.

Specifically, the stationary object prediction module400may receive synchronized navigation data505. A frame conversion module510may convert the synchronized navigation data505from a navigation frame to a universal transverse mercator (UTM) system. A vehicle position module515uses the output from the frame conversion module510to estimate a position change of the mobile vehicle105. A prior known object position520and output from the vehicle position module515are provided to a position mapping module525that maps the prior known object position520to a new object position530.

In various embodiments, to track stationary objects on the ground may be substantially equivalent to mapping an object from one image plane to another image plane because the objects are stationary and only the viewing position (e.g., image plane) changes. In some embodiments, a transformation matrix between the two image planes may be used. For example, if p(n−1) is a prior known object position in a prior image plane, and p(n) is a new object position in the new image plane, then Equations 20 and 21 result.
p(n)=sTp(n−1)  Equation 20
T=[Tr,Tt]  Equation 21

The variable s is a scale factor depending on the depth difference of the two image planes to the objects on the ground. The matrix Tris a rotation matrix and Ttis a translation vector. Both of these may be determined by movement of the mobile vehicle105and may be estimated from the navigation data. The navigation data may generally be based on the navigation frame (e.g., latitude, longitude). To calculate the translation vector of the mobile vehicle105movement, the navigation frame may need to be converted to a local coordinate system. As set forth above, the frame conversion module510converts the navigation frame into the UTM coordinate system. In some embodiments, the UTM system may use north-east-up (NEU) to represent a three-dimensional coordinate system. Mathematically, converting may be performed using a set of Equations 22 through 33. For these equations: φ represents latitude; and λ represents longitude. Moreover, the following constants are used. Equatorial radius: a=6378,137 km; inverse flattening: 1/f=298.257223563; k0=0.9996; N0=10000 km for southern hemisphere and 0 for northern hemisphere; and E0=500 km. Intermediate variables may be calculated using the following equations:

Then the UTM coordinates may be calculated by Equations 28 and 29.

With the UTM coordinate system, the translation vector may be calculated by Equations 30 through 33.
Tt=f[ΔAlt,ΔE,ΔN]TEquation 30
ΔAlt=Alt(n)−Alt(n−1)  Equation 31
ΔE=E(n)−E(n−1)  Equation 32
ΔN=N(n)−N(n−1)  Equation 33

The variable f is infrared camera focus length; and the variables Alt, E, and N, represent altitude, east, and north respectively. Because object position on an image plane is represented by homogeneous coordinates, the object position mapping from one plane to another one may be accurate up to a scale because of the different depths from two image planes to objects on the ground. In various embodiments, UTM coordinate values of the north axis are used to estimate a scale factor. For example, for a given navigation position, p(n)=[Alt(n), E(n), N(n)]T, the scale factor may be calculated by Equation 34.

In some embodiments, rotation matrix may be estimated by a Euler angles approach given by Equations 35 through 38.

The variables Δϕ, Δθ, and Δψ, are angle differences in yaw, pitch, and roll from one mobile vehicle105position to another position. Depending on position changes of the mobile vehicle105, a translation matrix and a rotation matrix may (individually or in combination) be included in a transformation matrix for mapping a location of an object from one image plane to another image plane. Where a location of an object is obscured in a latter image frame as a result of a degrade visibility environment, a representation of the object may be displayed (for example, overlaid on the image) in a predicted location of where the object is expected to be, using the transformation matrix for mapping a location of a visible object from one image plane to another image plane. The object detection module305, mapping module310, prediction module320, object identification module325, stationary object prediction module400, and frame conversion module510may comprise software modules that, when executed by a processor, operate to perform algorithms including the equations and steps described above. Where the mobile vehicle105is an aircraft that is airborne above a ground surface115, and an object such as a landing pad or landing ground115that is detected in a first image frame (obtained from a sensor on the vehicle) is obscured in a second image frame as a result of a degraded visibility environment, the display module215is configured to display on a display device associated with the vehicle a representation of the object on the second image frame in a predicted location where the object is expected to be. The representation of the object on the second image frame may be displayed by projecting or overlaying the representation of the object onto the second image frame in a predicted location where the object is expected to be, such that the representation of the object is displayed on a display device of the mobile vehicle to an operator of the vehicle. Accordingly, an operator of the mobile vehicle105may therefore better control operation of the vehicle during periods in which images are obscured as a result of a degraded visibility environment, by virtue of the display device displaying the representation of the object that is overlaid on the image in a predicted location where the object is expected to be.

FIG. 6is a schematic flow diagram of one embodiment of a method600for displaying an object (e.g., the object125). The method600may include detecting605a first location of the object on a first image frame of image data. In certain embodiments, the image data may be infrared data captured by an infrared sensor. In various embodiments, the first location of the object is detected based on intensity values corresponding to the infrared image data. In some embodiments, the image data includes multiple images in a video of an environment outside of a vehicle that are acquired in real time using infrared imaging equipment of the vehicle.

Furthermore, the method600may include determining610a second location of the object on a second image frame of the image data on which the object is not detectable due to an obstruction (e.g., degraded visibility environment140).

Moreover, the method600may include displaying615a representation of the object on the second image frame.

In certain embodiments, the method600may include determining whether the object is stationary or moving. In some embodiments, the method600includes, in response to the object determined to be stationary, determining the second location of the object on the second image frame based on the first location of the object and a change in position of a vehicle used to capture the image data. In various embodiments, the method600includes, in response to the object determined to be moving, determining the second location of the object on the second image frame based on the first location of the object, a predicted location of the object, and a change in position of a vehicle used to capture the image data. In various embodiments, the method600includes synchronizing the image data with navigation data of a vehicle.

The image processing module220may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. The image processing module220may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.