# Assessment of Spatiotemporal Fusion Algorithms for Planet and Worldview Images
## 2 Spatiotemporal Fusion Approaches
### Hybrid Color Mapping (HCM) Approach  
We will use WV and Planet images to illustrate the HCM approach [9]. Figure 2 shows two pairs of Planet (P1 and P2) and WV (W1 and W2) images. The pairs, (P1, W1) and (P2, W2) are collected on the same days. We have two observations. First, the Planet images can be treated as blurred versions of their WV counterparts. Second, the intensity relationship between the Planet pixels is somewhat similar to that of those WV pixels. If we can capture the intensity mapping between Planet images at two different times, then we can use that mapping to predict the WV image at time \(t_{p}\) using the WV image at \(t_{k}\). It turns out that, although the above idea is simple and intuitive, the prediction results using this idea are quite accurate. To further justify the above observation, we provide some statistics (means and standard deviations (std)) shown in Tables 1-3 for the Planet and Worldview image pairs. PSE and WVE are Planet and Worldview image pair at an earlier time and PSL and WVL are the image pair at a later time. The numbers are quite close. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & **Image** & **CANAL** & **FIELD** & **RUNWAY** \\ \hline Red Band & MEAN & PSE & 0.302202 & 0.383428 & 0.269307 \\  & & WVE & 0.303498 & 0.381238 & 0.263837 \\  & & PSL & 0.244982 & 0.356089 & 0.268762 \\  & & WVL & 0.244404 & 0.354388 & 0.243716 \\  & & PSE & 0.096934 & 0.084006 & 0.093859 \\  & & WVE & 0.099734 & 0.085258 & 0.093664 \\  & & PSL & 0.082842 & 0.089564 & 0.093986 \\  & & WVL & 0.083615 & 0.089764 & 0.088597 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Means and stds of Red band of two image pairs for the three scenarios in Section 3. Figure 1: Flowchart of the Flexible Spatiotemporal Data Fusion (FSDAF) fusion algorithm. Figure reproduced from [8]. Figure 3 illustrates the HCM approach. Based on the available Planet images collected at \(t_{k}\) and \(t_{p}\), we learn the pixel by pixel mapping between the two images. The learned matrix, \(F\), is then applied \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & **Image** & **CANAL** & **FIELD** & **RUNWAY** \\ \hline Green Band & MEAN & PSE & 0.303195 & 0.390444 & 0.273252 \\  & & WVE & 0.304829 & 0.387911 & 0.26864 \\  & & PSL & 0.251461 & 0.354358 & 0.272753 \\  & & WVL & 0.250843 & 0.353061 & 0.250539 \\  & & PSE & 0.090393 & 0.064076 & 0.093683 \\  & & WVE & 0.093879 & 0.064833 & 0.09259 \\  & & PSL & 0.075665 & 0.070846 & 0.093826 \\  & & WVL & 0.07582 & 0.069963 & 0.085556 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Means and stds of Green band of two image pairs for the three scenarios in Section 3. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & **Image** & **CANAL** & **FIELD** & **RUNWAY** \\ \hline Blue Band & MEAN & PSE & 0.321027 & 0.431879 & 0.295805 \\  & & WVE & 0.322543 & 0.42958 & 0.293019 \\  & & PSL & 0.269751 & 0.392876 & 0.295402 \\  & & WVL & 0.268997 & 0.390579 & 0.275519 \\  & & PSE & 0.084322 & 0.049597 & 0.0866 \\  & & WVE & 0.086584 & 0.050008 & 0.084989 \\  & & PSL & 0.069209 & 0.055778 & 0.086771 \\  & & WVL & 0.06806 & 0.055536 & 0.077771 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Means and stds of Blue band of two image pairs for the three scenarios in Section 3. Figure 2: Relationship between two pairs of Planet (top row) and Worldview-2 (WV) (bottom row) images. in the prediction step. Using similar notations in earlier equations, the prediction of the WV image at \(t_{p}\) can be achieved by  
\[W(x,y,t_{p})=F\times W(x,y,t_{k}) \tag{4}\]  
where \(W(\cdot,\cdot,\cdot)\) denotes a pixel vector (up to \(K\) with \(K\) being the number of bands) for this application and \(F\) is a pixel to pixel mapping/transformation matrix with appropriate dimensions. \(F\) can be determined by using the following relationship:  
\[P(x,y,t_{p})=F\times P(x,y,t_{k}) \tag{5}\]  
where \(P(\cdot,\cdot,\cdot)\) denotes a pixel vector (\(K\) bands).