# Spatiotemporal Image Fusion in Remote Sensing
## 3 Synthesis: Challenges and Opportunities
### Spatiotemporal Image Fusion Methods for Sentinel Images  
The new program of the European Space Agency (ESA), namely the Sentinel missions, gained the attention of the remote sensing community due to the increasing spatial, spectral and temporal resolution [121]. In the perspective of combined use of Sentinel-2 and Landsat 8, there are several differences to be considered. For example, in a previous study [122] we found out relatively high discrepancies between Normalized Difference Vegetation Indeed (NDVI) computed from Sentinel-2 and Landsat-8. We concluded in that work that, in order to take advantage of the 10 m resolution of Sentinel-2 and use these data along with Landsat-8 data, it would be desirable to adjust the reflectance values of the two sensors. A useful method for this purpose could be the one presented by [PERSON] [123] or co-registration techniques such as phase correlation [124]. Besides differences in spectral values, previous studies also reported a misalignment of several pixels between Landsat 8 and Sentinel-2 [125]. Therefore, images coming from the two sensors need to be co-registered before any concurrent use. Further information on Sentinel-2 MSI and Landsat-8 OLI sensors characteristics is provided by [PERSON] et al. [126].  
[PERSON] et al. [48] described an advanced method to fuse MSI onboard Sentinel-2 platform and OLCI onboard Sentinel-3 platform which relies on regression model fitting to relate spectral reflectance from two acquisition times and spatial filtering to remove the artifacts in the regression model fitting prediction. Their proposed method accounts successfully for the temporal land cover changes when creating nearly daily Sentinel-2 images. Furthermore, it relies on a single Sentinel-3/Sentinel 2 images pair. Besides these studies dedicated to Sentinel missions, important research has been dedicated to developing more generic methods which can be used to fuse different sensors products. An example of such a method is those developed by [PERSON] et al. [46].  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Data Fusion Performance Metrics** & **Authors** \\ \hline Spectral angle mapper (SAM) & [PERSON] et al. [108] \\ \hline Peak Signal-to-noise-ratio (SNR) & [PERSON] et al. [116] \\ \hline Structural Similarity Index (SSIM) & [PERSON] et al. [117] \\ \hline Image quality index & [PERSON] et al. [110] \\ \hline Extended image quality index & [PERSON] et al. [111] \\ \hline Quality w/no reference index & [PERSON] et al. [114] \\ \hline Enhancement measure evaluation (EME) & [PERSON] et al. [115] \\ \hline Entropy & [PERSON] et al. [118] \\ \hline \hline Erreur Relative Globale Adimensionnelle de Synthese (ERGAS) & [PERSON] [113] \\ \hline \hline \end{tabular}
\end{table}
Table 2: Most common metrics used to evaluate the accuracy of the spatiotemporal fused product.  
It is expected that a large number of spatiotemporal image fusion methods will be developed in the future in order to leverage the available satellite data archive and to increase the spatial resolution of the latest satellite images such as those acquired by microsatellites (e.g., planet images).