Characterising and predicting the movement of clouds using fractional‐order optical flow

IET Image ProcessingVolume 13, Issue 8 p. 1375-1381 Research ArticleFree Access Characterising and predicting the movement of clouds using fractional-order optical flow Snehlata Shakya, Corresponding Author Snehlata Shakya sneh022@gmail.com orcid.org/0000-0002-4469-9808 Department of Mathematics, IIT Roorkee, Roorkee, IndiaSearch for more papers by this authorSanjeev Kumar, Sanjeev Kumar Department of Mathematics, IIT Roorkee, Roorkee, IndiaSearch for more papers by this author Snehlata Shakya, Corresponding Author Snehlata Shakya sneh022@gmail.com orcid.org/0000-0002-4469-9808 Department of Mathematics, IIT Roorkee, Roorkee, IndiaSearch for more papers by this authorSanjeev Kumar, Sanjeev Kumar Department of Mathematics, IIT Roorkee, Roorkee, IndiaSearch for more papers by this author First published: 29 May 2019 https://doi.org/10.1049/iet-ipr.2018.6100Citations: 4AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Estimating cloud motion with complex background through the sequence of satellite images plays an important role in weather forecasting. This motion can be used for characterization of clouds and predicting storms. Optical flow is used here for motion estimation which gives horizontal and vertical velocity components. Velocity field vector alone is not sufficient to analyze the cloud behavior for predicting extreme weather conditions and there is a need to develop some visual features for enhancing the weather prediction strategies. In this paper, we utilize the optical flow to localize the high alert regions. To have a better localization and motion signatures, we develop a fractional order technique to compute optical flow. Also, the localization is characterized by brightness of image, magnitude, directions, vorticity and irrotational components of the optical flow. We did analysis on sequence of images for Mumbai, India heavy rain that happened during August 28–29, 2017, cyclonic data sets for May 16, 2018, September 19–20, 2018 and October 14, 2018. Visual features show different patterns for extreme and normal weather situations. A study on interpolation and extrapolation of the image sequence is also presented. Optical flow based interpolation and Advection-anisotropic-diffusion based extrapolation model give promising results. 1 Introduction Weather forecasting has always been studied to predict weather conditions. Severe weather situations may lead to loss of lives and economy if the predictions are not so accurate. There is a need for advancement of existing prediction techniques for analysing clouds especially with complex background. Recent advances in computer vision can be used to detect cloud with complex backgrounds from a sequence of dynamic satellite images. In computer vision, estimating motion using an image sequence is a fundamental research topic. In general, moving object's motion is approximated in terms of image flow fields, which is 3D velocity vector of objects (clouds). However, it is being an inverse problem in which we have to approximate the cloud motion using an image sequence. In image plane, this leads to a vector of velocity components (vertical and horizontal) called the optical flow field. In general, features in the image frames related to the motion of actual object play an important role and sometimes their detection is not easy. For example, there is no sign of movement in the rotating motion of a perfectly black metal ball in optical flow field. However, this rotation appears in the image flow field. The optical flow field provides a reasonable approximation and it can be used in a variety of applications. Here, our aim is to track the motion of clouds and prediction of extreme weather conditions using satellite imagery. Recently, few efforts are made to detect optical flow across the spectral (using infra-red (IR) and electro-optical (EO) sensors) for surveillance applications. However, a great bunch of efforts is needed to extend these methods for satellite images. The temperature between layers of clouds varies together with the altitude. Most of the severe storms are usually happening at the boundary between cold and warm air masses. Such a boundary is classified by a jet stream [1]. Hence, utilisation of thermal sensors together with usual electro-optical imaging can enhance the quality of prediction up to some extent. Given two images, we want to determine the mapping of all points from first image to new locations in second image using displacement vector field. Here, these two images with intensities and are captured at two consecutive time instances and , respectively. There exists a number of methods for computing optical flow between these two images. Original methods were proposed by Horn and Schunk [2] and Lucas and Kanade [3]. Since then, researchers have been continuously working on enhancing the quality of optical-flow-based methods. Brox et al. [4] have replaced the quadratic regularisation by smoothness constraints in Horn and Schunk's method which leads to piecewise smooth results. Further, Bruhn et al. [5] worked in this direction and introduced novel approaches (for ill-posed problem) by combining the properties of local and global methods. Ill-posedness problem was addressed by several researchers and different smoothing techniques and assumptions have been made to overcome this ill-posedness issue. Mostly, spatial and temporal image derivatives were used for optical flow estimation. A feedforward approach and perceptually weighted optical flow are also proposed which are comparatively better to fed-forward and non-weighted motion estimates, respectively [6, 7]. It can have several applications, at large, the motion of rigid bodies were captured by estimating the optical flow [5, 8]. These optical-flow-based methods have rarely been used to model the turbulent motion in case of the fluid dynamics. In the present study, we are using a fractional order variational model for estimating optical flow [9, 10]. Optical flow field is not sufficient to observe the extreme weather conditions [11]. It is, therefore, required to extract some visual features to classify the storm or heavy rain situations using optical flow components. Recently, Zhang et al. [1] have worked in this direction and proposed to use some visual features that can help in better predicting the storm conditions. We will use some of those criteria collectively for weather forecasting application purposes. We observe that images captured by satellite INSAT-3D were acquired at large amount of time interval. For better accuracy of weather prediction and analysing the pattern of cloud motion, it is necessary to acquire image in between the given time frame. Interpolation of satellite images in space and time was carried out by a particle-tracking technique [12]. Application of this technique was to estimate a continuous distribution of a satellite-derived water property in Lake Superior chlorophyll measurements. Optical-flow-based temporal interpolation was employed by Gurdan et al. [13]. They have applied backward warping to synthesise novel interpolated image. We will use optical-flow-based interpolation [13] for interpolating the high definition temporal image sequence captured by INSAT-3D satellite. Ehrhardt et al. [14] have proposed this method for temporal interpolation of magnetic resonance imaging while we will be utilising it for remote sensing application. Temporal interpolation of satellite image sequence will enhance the tracking of cloud motion or storm location. If some warning conditions are suggested by satellite images and we want to predict the weather in near future (before the satellite captures new image), extrapolation can be advantageous. Spatiotemporal autoregressive (STAR)-based, discrete Fourier transform (DFT)-based and hybrid DFT–STAR-Based techniques have been used previously for image prediction in near future for weather forecasting purposes [15]. Sakaino [11] has proposed an advection-anisotropic-diffusion (AAD)-based prediction model which integrates two physical effects of advection and anisotropic diffusion. We are incorporating it to predict the weather and will validate it with the available ground truth images. Following are the main objectives of present work: Visual feature extraction using optical flow components: Firstly, fractional order optical flow field is computed by modifying the Horn and Schunk's method. Secondly, these optical flow components are used to characterise clouds. Optical-flow-based temporal Interpolation: Temporal interpolation is applied using optical flow components which gives images at different time instances between the two given images at time and . Extrapolation for near future prediction: To predict the near future weather, we are using extrapolation. The model utilises advection and anisotropic diffusion property for this prediction. 2 Theory This section is divided into four parts: (i) optical flow, (ii) visual features, (iii) optical-flow-based interpolation and (iv) Near future weather prediction (Fig. 1). Fig. 1Open in figure viewerPowerPoint Illustration of working strategy 2.1 Optical flow In Horn and Schunck's method [2], the energy functional is minimised as (1)where I is image intensity, , , , , is a regularising parameter, , u and v are the optical flow components. An improved and robust version of variational model [10] is defined as (2)where is a constant of small values. The energy functional (2) combines the two models of Horn and Schunck [2] and Nagel and Enkelmann [16]. Fusion of both the models results in improved dense optical flow while preserving the discontinuities. However, we require a more generalise way in case of dense clouds for characterising an accurate motion. Therefore, we are adopting a generalisation of integer order gradient into fractional order. Another version of energy functional (2) with fractional order is defined as (3)where is left fractional derivative operator of Riemann–Liouville and . The proposed model can be treated as the generalisation of the integer order variational optical flow models for . For estimating the flow vector , the variational functional (3) needs to be minimised. To obtain the corresponding Euler–Lagrange equations of (3), we consider functions and that are desired such that for any test functions and , u and v are defined as (4)where . Thus, the Riemann–Liouville fractional derivative of (4) with respect to x and y are given as (5)With the use of (4) and (5) in (3), the functional (3) is obtained as a function of only for each and , and it is given as (6)We will be differentiating variational functional (6) with respect to and making , in order to obtain the extremum, and then equating the coefficients of and , we get (7) (8)This derivation composes a system of equations for each pixels of the image. The above system can be solved easily by a numerical scheme for the sparse linear system. Here, we used Krylov-subspace method for solving the above system. 2.2 Visual features The translation and rotation components are used to extract the visual signatures in basic cloud motion. The attributes of feature set are extracted using the properties of these two components of optical flow. 2.2.1 Localisation of vortex centre Vortex is defined as a region where the flow rotates around an axis line. In tropical cyclones, an eye forms at the centre of vortex [17]. Eye of a cyclone is the region of reversed downward flow in and around the axis of the vortex, it is uncovered of clouds. Thus, identification of vortex centre is an important step. In order to detect the centre of the vortex, we have used the rotation and divergence as two main terms for detecting severe weather conditions. In optical flow field, these two types of motions are defined by the curl and divergence operators. The divergence and curl of a flow field are given by and , respectively. Magnitude of the curl at any point is twice the angular velocity of itself. This criterion is used to detect the storm. To do this, the optical flow field is decomposed into solenoidal (divergence-free) and an irrotational (velocity-free) components. One of the methods [1] to achieve this is as following: Calculate the local deformation tensor . Decompose this tensor into symmetric (strain) part and rotation part . Calculate the Q-parameter as .where denotes the Frobenious matrix norm. Adopt the reasoning as: if Q is positive, then the storm is detected. Otherwise, no storm is detected. Regions, where , show more rotation motion than translation. Also those regions are typically the centres in vortexes. Neighbouring pixels also show strong vorticity (but they are less significant than vortex core) and they also take part in the vortex system. 2.2.2 Construction of vortex descriptors Prediction of storms will use the properties of vortex. However, based on the previous studies, it can be concluded that not all the vortex are related to the storms. Therefore, we need to characterise all the vortex by extracting a number of attributes. The descriptors (for each vortex) can be formed by these extracted attributes and subsequently will be used for classification and characterisation of the vortex. Consider as some vortex region in a satellite image with its area as . We define some visual features, from a single image and from the optical flow, which will be used as the descriptors of storms: These attributes listed in Table 1 and computed for each vortex from the optical flow data. Table 1. Descriptors and their formula Descriptors Formula Remark mean channels brightness is the brightness of pixel in the channel k image mean optical flow intensity is the optical flow at pixel maximal Q-value of the vortex is defined above 2.3 Optical-flow-based temporal interpolation Optical-flow-based interpolation had been employed before by researchers to compute intermediate images. Rakêt et al. [18] have proposed model to estimate the unknown frame using surrounding frames. They parametrised the optical flow objective function to model the interpolation assumption directly. Gurdan et al. [13] also worked in this direction of temporal interpolation. For this purpose, they have used optical-flow-based approach. In the present study, we will also rely on optical-flow-based interpolation and will follow the work by [14]. The brief description is given below, Let and are image intensities at position and time and . is the velocity field vector. To obtain image at some time t such that , weighted time average is used: (9) (10)Here, and time is normalised, i.e. . The vector is known and for computing we have employed pseudo-inverse. For comparing the results with linear interpolation, we follow Grevera and Udupa [19] for error measures: Mean difference (MD):(11)Here, is the set of pixels in frame . and represent original and interpolated image intensities at time frame , respectively. Number of sites of disagreement (NSD):(12)where, (13)Here, is the threshold and it is chosen 5% in present study. Largest difference per frame (LD):(14)Following [14], statistical relevance is used to compare the interpolation methods, which is defined as (15)Here suffixes OF and lin represent optical flow (OF) and linear (lin) interpolations, respectively. 2.4 Near future weather prediction Previously, several methods including tracking, extrapolation etc., have been employed for predicting next time image using previous images captured from satellite. In the present work, we will employ the AAD-based prediction model for predicting near future image. Detailed information about the model is available in [11], here we will present a brief description. The advection–diffusion equation is given as (16)The first term is the advection and the second term is associated with the diffusion. and are the second-order derivatives of image intensity I w.r.t. x and y, respectively. By discretising the derivative terms by the finite difference method, we can write (16) as follows: (17)Here, represents the pixel coordinates and n represents the time ( for future time). The terms involved in (17) can further be simplified as follows: Diffusion parameter is computed from the local image gradients: (18)where is a non-linear function of z with a control parameter . We will utilise skill score [15] for testing the performance of prediction method. It is defined as (19)where is defined as where and are the pixel values of observed and predicted images, respectively. n is the total number of pixels in image. is the persistence error computed as root mean square error between last acquired image at the time of now casting and observed image. If SS is positive then model is performing better than persistence by SS% [15]. 3 Results and discussion We have worked on high-definition images obtained from INSAT-3D satellite. It is an improved design of very high resolution radiometer (VHRR/2). It provides imagery data for six spectral bands; Visible (0.55–0.75 μm), short wave infrared (1.55–1.70 μm), mid-wave infrared (3.80–4.00 μm), water vapour (6.50–7.10 μm), thermal infrared-1 (10.3–11.3 μm) and thermal infrared-2 (11.5–12.5 μm). Spatial resolutions for visible, Mid Wave Infrared, water vapor and Thermal infrared are 1, 4, 8 and 4 km, respectively. The images of Earth disk are captured in every 26 min. The INSAT-3D imager gives information about outgoing long-wave radiation, quantitative precipitation estimation, sea surface temperature, snow cover, cloud motion winds etc. [20]. We choose to process the thermal infrared data with range of spectral channel 10.3–11.3 μm. Mumbai heavy rain on 28–29 August 2017 is analysed where extreme rainfall of 468 mm in 12 h led to flood situation. Cyclonic storms Sagar on 16 May 2018, Daye on 19–20 September 2018 and Luban on 14 October 2018 are also analysed. Sagar was formed on 16 May 2018 in east of the Guardafui Channel and it made a landfall in Somalia next day. A depression was observed on 19 September 2018 over the east central Bay of Bengal. It got intensified into a deep depression on 20 September 2018 over the west central Bay of Bengal. Luban was categorised as severe cyclonic storm. It made landfall in eastern Yemen on 14 October 2018. 3.1 Visual features The top left image of Fig. 2 is captured at 0059 h (GMT) on 28 August 2017. A circular cloud region is observed over Odisha and neighbouring regions. Moderate-to-heavy rains were expected over these regions. Top right image is captured at 0030 h on 29 August 2017. No circulation regions are identified visually but dense clouds can be seen over Mumbai and surrounding areas. Middle row left image is captured at 2300 h on 16 May 2018. Circular clouds are seen over east of the Guardafui Channel. Middle row right image is captured at 0630 h on 19 September 2018. Dense and circular clouds are observed over the east central Bay of Bengal. Bottom row left image is captured at 0430 h on 20 September 2018. Dense and circular clouds are moved toward west central Bay of Bengal. Bottom row right image is captured at 0030 h on 14 October 2018. Circular clouds are observed over Yemen. Fig. 2Open in figure viewerPowerPoint Maximum Q factor plotted for different high intensity regions. Satellite images with maximum Q-value for, First row: August 28 at time (GMT) 0059 h (left) and August 29 at time (GMT) 0030 h (right). Middle row: May 16 at time (GMT) 2300 h (left) and September 19 at time (GMT) 0630 h (right). Third row: September 20 at time (GMT) 0430 h (left) and October 14 at time (GMT) 0030 h (right) To observe the movement of clouds, we computed optical flow using those images. We used Horn and Schunk's [2] and fractional-order-based [9, 10] methods for computing optical flow vectors. We observed that when we use fractional-order-based method then the error in prediction is reduced by an order of to . So we choose to proceed with fractional-order-based method for estimating optical flow field. Several regions are selected where clouds are more dense because heavy rain or storm can only happen at these highly dense cloud regions. For those regions, the maximum Q-factors are computed. Some of the regions are ignored where the dense cloud patches are limited to small areas. For large dense cloud regions, the maximum Q-factor is plotted on satellite images which are displayed in Fig. 2. Yellow dot shows the vortex centre and the red colour shows the maximum Q-values. Top left and right images are captured on August 28 and 29, respectively. Maximum Q-factor is being observed on August 28 in the North Bay of Bengal region which shows the presence of rotating clouds. It can be treated as vortex centre. In July and August 2017, severe flooding happened in West Bengal. There is not much rotation is observed in Mumbai region for both the dates of August 28 and 29 but dense clouds can be seen in this region. As maximum Q-factor shows mainly rotation, no maximum Q-values are observed. Some other visual features or techniques are needed to forecast heavy rain regions more accurately. For the cyclonic image data, maximum Q values are observed over the cyclonic regions for all the four cases of May 16, September 19–20 and October 14. However, we found maximum Q-values at several other non-cyclonic regions. We further need to work on the advancement of algorithm to tackle the issue. We estimated other visual features, namely mean channel brightness (MCB) and mean optical flow Intensity (MOFI) for the local regions of dense clouds. These visual features including maximum Q-factor are displayed in Fig. 3. Plots in the top row are of 28 August 2017 for time 0059 (left), and 29 August 2017 for time 0030 h (right). Middle row left and right images display plots for May 16 (2300 h) and 19 September 2018 (0630 h). Bottom row left and right images display plots for September 20 (0430 h) and 14 October 2018 (0030 h). Values (MCB, MOFI and MaxQ) are normalised between 0 and 1. Visual signatures are locally computed for the small regions of dense clouds. It is observed that the mean optical flow intensity and maximum Q-values follow similar pattern for most of the cases. Other visual feature, mean channel brightness shows a different pattern than MOFI and MaxQ. In most cases, if MCB values goes up/down the other two show a down/up pattern. We need to investigate a large number of images to observe the pattern for MCB and other visual features. Fig. 3Open in figure viewerPowerPoint Display of Visual features namely mean channel brightness (MCB), mean optical flow intensity (MOFI) and maximum Q-factor. First row: August 28 at time (GMT) 0059 h (left) and August 29 at time (GMT) 0030 h (right). Middle row: May 16 at time (GMT) 2300 h (left) and September 19 at time (GMT) 0630 h (right). Third row: September 20 at time (GMT) 0430 h (left) and October 14 at time (GMT) 0030 h (right) 3.2 Optical-flow-based temporal interpolation Satellite captures images at a fixed amount of time. However, to analyse or track the cloud motion more accurately, one needs to have more intermediate images. Optical-flow-based interpolation (Section 2.3) is used for this purpose. A simple simulation has been done for validation. Fig. 4 shows the results. Images have a pixel resolution of 255 × 255. Top left image is object's position at time frame where the circular object has a centre at (0, 0). Top right image is generated with 8 pixels displacement towards right and bottom. This represent image at time frame . Top middle image is the original intermediate image at time with 4 pixels displacement towards right and bottom. Bottom left is the linearly interpolated image for time and bottom right image is obtained by applying optical-flow-based interpolation for time . We can see a shadowing effect with linear interpolation while optical-flow-based interpolation gives close resemblance to the original image. We are taking a small displacement as the cloud movement is also small in terms of pixel size (size of pixel is large). Fig. 4Open in figure viewerPowerPoint Top row: image at time frame (left most), image at time frame (right most) and intermediate image at . Bottom row: Linear interpolation (left) and optical-flow-based interpolation (right) Now, we apply interpolation on real satellite images. Fig. 5 shows satellite images captured on August 28. The left most image in Fig. 5a is the original satellite image captured at time 0229 h. Top and bottom images in the middle column of Fig. 5a are optical-flow-based and linearly interpolated. The last column images are the corresponding absolute differences between original and the interpolated images. Similarly, Fig. 5b shows interpolation for time 0259 h. Following [14], statistical relevance of error measures MD, NSD and LD are obtained for satellite images. Statistical relevance values are computed for comparing the results and they are given in Table 2. Positive values indicate that optical-flow-based interpolation is performing better. Table 2. Performance of interpolation methods Date, time 28, 0229 3.8 0.01 33.6 28, 0259 3.6 0.005 40.1 Fig. 5Open in figure viewerPowerPoint Satellite image captured on August 28. (a) First column: Original image at time 0229 h (GMT), mid-column top and bottom: optical-flow-based and linearly interpolated images, respectively, and the last column top and bottom: Absolute differences of original and respective interpolated images. Similarly image (b) is for interpolation at time 0259 h (GMT) In previous exercise, we did interpolation when ground truth was known. Now it is applied to get images at an interval of 10 min. The results are displayed in Fig. 6. Top and bottom rows have images of August 28 and 29, respectively. In top row, the left-most and right-most are the original images captured by satellite at times 0029 and 0059 h, respectively. Second and third images in top row are the interpolated at times 0039 and 0049 h. Similarly in bottom row, the results are of the August 29. Ground truth images are not available, thus error measures are not given. However, the cloud motion is continuous and clear in the images. Fig. 6Open in figure viewerPowerPoint Top row: Satellite images captured on August 28 at time 0029 (left most) and 0059 h (right most), Middle two images are optical-flow-based interpolated for time 0039 and 0049 h. Bottom row: left to right: Satellite images captured on August 29 at time 0230 (left most) and 0300 h (right most), middle two images are optical-flow-based interpolated for time 0240 and 0250 h 3.3 Near future weather prediction For weather forecasting, extrapolation is one of the important methods to predict the weather. We used advection–anisotropic–diffusion model for this purpose. Fig. 7 shows results for August 28. Top row is displaying original satellite image (left), predicted image (middle) and absolute image intensity difference (right) for time 0259 h. The predicted image at 0259 h is obtained for a lead time of about half-an-hour by using satellite images of time 0159 and 0229 h. Bottom row showing (left to right) original, predicted and their difference for time 0329 h. We compared the outcomes in terms of Skill Score [15, 21] as visual difference is not sufficient to check the performance of model. For both the above mentioned time for prediction, is positive. Model performing 2.8% and 6.77% better than persistence. We computed for about half-an-hour of lead time prediction for other diverse events (mentioned before) and the values are given in Table 3. By observing the values, we see that the is larger for cyclonic data in comparison to non-cyclonic data. It indicates that prediction is better for such cyclonic cases. Table 3. Performance of prediction method based on Skill Score Event 28 August 2017, time: 0259 h 2.83 28 August 2017, time: 0329 h 6.77 16 May 2018, time: 2330 h 12.98 19 September 2018, time: 0700 h 11.67 20 September 2018, time: 0500 h 6.16 14 October 2018, time: 0100 h 11.01 Fig. 7Open in figure viewerPowerPoint Top row: Original Satellite image captured on August 28 at time 0259 (left), predicted (middle) and their absolute difference (right). Bottom row: Original satellite image captured on August 28 at time 0329 (left), predicted (middle) and their absolute difference (right) We further computed for large number of data set to compare the model with [15]. We processed 45 satellite images of 16 May 2018, 39 images of 19 September 2018, 39 images of 20 September 2018 and 42 images of 14 October 2018. We first used hourly satellite images and predicted for one hour of lead time. Next we utilised two hours time interval images to predict two hours of lead time images. Then three hour time interval images are used to predict image for the lead time of three hours. Mean and standard deviations are computed for lead time of one, two and three hours. The plot of versus lead time is shown in Fig. 8. The mean values are nearly 9, 5 and 4 for lead time of 1, 2 and 3 h of time, respectively. Standard deviation values are nearly 3 for all three lead times. Shukla et al. [15] hybridised DFT and STAR techniques. They used 150 data sets and compared results with individual DFT and STAR techniques. The mean values with hybrid DFT–STAR model were nearly 4, 12 and 11 for lead times of 1, 2 and 3 h. However, the standard deviation values were roughly 5, 7, 10, respectively. By comparing the numbers, we can say that for a lead time of 1 h the AAD models gives better performance to hybrid DFT–STAR technique. For the prediction of lead time of 2 and 3 h, mean value with hybrid DFT–STAR technique is much better to AAD model. However, if we look into standard deviation values the AAD models seems pretty stable. DFT and STAR models show negative mean and standard values. Thus, our model performs better to these two models. The comparison of AAD-based prediction and hybrid DFT–STAR, DFT and STAR is not straightforward as datasets are different and lead time prediction is also done in different manner. However, this comparison gives an overall insight about model's performance. Fig. 8Open in figure viewerPowerPoint Performance of model over large data set Next we present prediction with brightness temperature (BT). We worked on cyclone Luban, 14 October 2018. BT images at time 0000 and 0030 h are used to predict BT image at time 0100 h. We selected a very small region of cyclonic area. These are displayed in Fig. 9. Top image is the full original image. Original cropped (left), predicted (middle) and difference (right) images are given in the bottom row. Blue large region in cropped original and predicted image seems to be the centre of cyclone. Predicted BT image is quite similar to original with a shrink blue region. Much of difference is observed at sharp boundaries where the gradient is large which can be seen in the difference image. In the present study, we have not worked to tackle that issue as the overall results found satisfactory. We have done several experiments to observe the cloud and vortex movement in near future. The movement with predicted image clearly shows a good match with the ground truth images. Here, we presented a very few results for the sake of brevity. Fig. 9Open in figure viewerPowerPoint Cyclone Luban: 14 October 2018, Time: 0100 h 4 Conclusion We presented a study on visual feature extraction that helped to identify the regions with more rotation that can possibly trigger a storm. Analysis is presented for heavy rain situations and few cyclones. We also applied interpolation and extrapolation on given satellite images. This exercise is important for tracking the cloud/storm by locating the vortexes more accurately. We made a comparison with ground truth images and the results are satisfactory. Further, we also tested the prediction algorithm for large number of satellite images with cyclonic storm conditions. 5 Acknowledgment We gratefully acknowledge the financial support of the Indian Space Research Organization (ISRO) through project RESPOND OGP-150 for carrying out this work. The authors would like to thank Space Application Center Ahmedabad for providing satellite image data. We would also like to thank Dr. Bipasha Paul Shukla, Scientist, SAC-ISRO for her valuable suggestions and feedback. 6 References 1Zhang Y., Wistar S., and Piedra-Fernandez J.A. et al: ' Locating visual storm signatures from satellite images'. Proc. of IEEE Int. Conf. on Big Data, Washington, DC, 2014, pp. 711– 720 2Horn B.K.P., and Schunck B.G.: 'Determining optical flow', Artif. Intell., 1981, 17, (1–3), pp. 185– 203 3Lucas B., and Kanade T.: ' An iterative image registration technique with an application to stereo vision'. Seventh Int. Joint Conf. on Artificial Intelligence, Vancouver, Canada, 1981, pp. 674– 679 4Brox T., Bruhn A., and Papenberg N. et al: ' High accuracy optical flow estimation based on a theory for warping'. Computer Vision – ECCV, Prague, Czech Republic, 2004, vol. 4, pp. 25– 36 5Bruhn A., Weickert J., and Schnorr C.: 'Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods', Int. J. Comput. Vis., 2005, 61, (3), pp. 211– 231 6Giaccone P.R., and Jones G.A.: ' Feed-forward estimation of optical flow'. IET Conf. Proc., Dublin, Ireland, 1997, pp. 204– 208 7Malo J., Gutierrez J., and Epifanio I. et al: 'Perceptually weighted optical flow for motion-based segmentation in MPEG-4 paradigm', Electron. Lett., 2000, 36, (20), pp. 1693– 1694 8Bors A.G., and Pitas I.: 'Optical flow estimation and moving object segmentation based on median radial basis function network', IEEE Trans. Image Process., 1998, 7, (5), pp. 693– 702 9Kumar P., Kumar S., and Balasubramanian R.: 'A fractional order variational model for the robust estimation of optical flow from image sequences', Optik – Int. J. Light Electr. Opt., 2016, 127, (20), pp. 8710– 8727 10Kumar P., and Kumar S.: 'A modified variational functional for estimating dense and discontinuity preserving optical flow in various spectrum', AEU – Int. J. Electron. Commun., 2016, 70, (3), pp. 289– 300 11Sakaino H.: 'Spatio-temporal image pattern prediction method based on a physical model with time-varying optical flow', IEEE Trans. Geosci. Remote Sens., 2013, 51, (5), pp. 3023– 3036 12Xue P., Schwab D.J., and Sawtell R.W. et al: 'A particle-tracking technique for spatial and temporal interpolation of satellite images applied to lake superior chlorophyll measurements', J. Gt. Lakes Res., 2017, 43, (3), pp. 1– 13 13Gurdan T., Oswald M.R., and Gurdan D. et al: ' Spatial and temporal interpolation of multi-view image sequences'. Pattern Recognition. (GCPR), Münster, Germany, 2014, (LNCS, 8753) 14Ehrhardt J., Säring D., Handels H.: ' Interpolation of temporal image sequences by optical-flow-based registration', in H. Handels, J. Ehrhardt, and A. Horsch et al (Eds): ' Bildverarbeitung für die Medizin' ( Informatik aktuell. Springer, Berlin, Heidelberg, 2006), pp. 256– 260 15Shukla B.P., Pal P.K., and Joshi P.C.: 'Extrapolation of sequence of geostationary satellite images for weather nowcasting', IEEE Geosci. Remote Sens. Lett., 2011, 8, (2), pp. 216– 219 16Nagel H.H., and Enkelmann W.: 'An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences', IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (5), pp. 565– 593 17Oruba L., Davidson P.A., and Dormy E.: 'Formation of eyes in large-scale cyclonic vortices', Phys. Rev. Fluids, Am. Phys. Soc., 2018, 3, (1), pp. 013502-1– 013502-18 18Rakêt L.L., Roholm L., and Bruhn A. et al: ' Motion compensated frame interpolation with a symmetric optical flow constraint'. Advances in Visual Computing (ISVC 2012), Crete, Greece, 2012, (LNCS, 7431), pp. 447– 457 19Grevera G.J., and Udupa J.K.: ' Shape-based interpolation of multidimensional grey-level images'. Proc. SPIE 2164, Medical Imaging: Image Capture, Formatting, and Display, Newport Beach, CA, United States, 1994 20Available at https://directory.eoportal.org/web/eoportal/satellite-missions/i/insat-3d 21Shukla B.P., Kishtawal C.M., and Pal P.K.: 'Prediction of satellite image sequence for weather nowcasting using cluster-based spatiotemporal regression', IEEE Geosci. Remote Sens. Lett., 2014, 52, (7), pp. 4155– 4160 Citing Literature Volume13, Issue8June 2019Pages 1375-1381 FiguresReferencesRelatedInformation