# Spatiotemporal Image Fusion in Remote Sensing
## 2 Fusion Methods to Increase Spatiotemporal Resolution of Satellite Images
### Reconstruction-Based Spatiotemporal Image Fusion Methods
The relationship between the fine resolution and coarse pixels is modeled using the following formula:  
\[FR(x,y,t_{i}) =CR\big{(}x,y,T_{j}\big{)}+\varepsilon(x,y,t_{i}) \tag{2}\]  
where \(x\), \(y\) represents the aligned fine resolution (\(FR\)) and coarse resolution (\(CR\)) pixels, \(T_{j}\) is the acquisition date for the pair-images and \(\varepsilon(x,y,t_{i})\) represents the differences between the pixels of the two images (i.e., errors) caused by e.g., viewing angle geometry. The prediction of the fine resolution pixels values (\(FR\)) at \(t_{pr}\) is performed as follows:  
\[FR\big{(}x,p,t_{pr}\big{)} =CR\big{(}x,y,t_{p}\big{)}+\Delta\big{(}x,y,t_{p}\big{)} \tag{3}\]  
where \(\Delta\big{(}x,y,t_{p}\big{)}\) represents the difference between the spectral reflectance values of the two input pair images at location \(x\), \(y\) and date \(t_{pr}\). Two types of temporal changes need to be considered when developing spatiotemporal fusion methods, namely the seasonal change of vegetation, i.e., vegetation phenology, and the land cover change, i.e., deforestation. While the first temporal change is successfully considered by the method proposed by [PERSON] et al. [32] and by [PERSON] et al. [46], the second temporal change required further developments. [PERSON] et al. [43] proposed a spatial temporal adaptive method for detecting reflectance changes associated with land cover change and disturbance. This image fusion method uses high spatial resolution tasseled cap transformation and high frequency of tasseled cap transformation to identify changes in reflectance. The developed method is simple and intuitive and requires at least two fine resolution images representing the beginning and the end of the user-defined time interval. These two images are used to identify the changes in the data. The changes are calculated using the so-called Disturbance Index (DI) which relies on three tasseled cap indices, namely brightness, greenness, and wetness. This index identifies the changes (or disturbances) in the coarse resolution pixel values between two consecutive dates. The presence of a disturbance event in the study area influences which pair images are used for predicting the reflectance value of the fine resolution images. For example, if the disturbance occurs after the prediction date, then the first fine-course resolution pair-images is used. If the pixel to be predicted lies after the occurrence of the disturbance, then the last pair-images is used. While this method is one of the initial image fusion efforts which consider landscape changes [7], there are studies which reported that it is suited only for forest disturbances scenario as the method selects the optimal fine resolution image date for prediction based on forest disturbance date detection from coarse resolution images. Therefore, this method is of limited value if when other types of land cover changes occur in the investigated areas. [PERSON] et al. [47] developed a method called robust adaptive spatial and temporal fusion method, which is capable to consider not only gradual changes, i.e., shape change (e.g., crop rotation), but also abrupt changes such as urban sprawl. To do this, the authors extended the spatiotemporal image fusion method of [PERSON] et al. [32] by replacing the Inverse Distance Weighted (IDW) with a local linear regression model to calculate the weights assigned to the pixels similar to the central pixel. To do this,the method (which uses only one prior pair of fine-coarse resolution images) follows three steps.