# Dynamic Model Averaging in Economics and Finance with fDMA: A Package for R
## 7 Forecast Comparison
R> wti <- crudeoil[-1, 1] R> drivers <- (lag(crudeoil[, -1], k = 1))[-1, ] R> ld.wti <- (diff(log(wti)))[-1, ] R> ld.drivers <- (diff(log(drivers)))[-1, ] R> m <- fDMA(y = ld.wti, x = ld.drivers, + alpha = 0.99, lambda = 0.99, initvar=1) R> hit.ratio(y = ld.wti, y.hat = m$y.hat, d = TRUE)  
[1] 0.5466  
Another aspect of forecast comparison is to have some statistical test which could differentiate forecasts' qualities. A well-known and commonly used test to compare forecasts is the Diebold-Mariano test [180]. The great advantage of this test is that it is based on relatively weak assumptions. Therefore, it can be applied in relatively many situations. One version of this test is already implemented in the forecast package by [PERSON] and [PERSON] [20]. Indeed, dmtest, mdmtest and hmdmtest are wrappers for dm.test from forecast package.  
In short, if \(\epsilon_{i,1},\ldots,\epsilon_{i,T}\) and \(\epsilon_{j,1},\ldots,\epsilon_{j,T}\) are forecast errors from two alternative forecasting methods, then the quality of each forecast can be evaluated by some loss function \(g\). The null hypothesis is that the two methods have the same forecast accuracy, i.e., that \(\mathbb{E}(g(\epsilon_{i,t})-g(\epsilon_{j,t}))=0\) for all \(t\).  
All implemented functions assume that one-ahead forecasts are compared and the second power is used in the loss function. Moreover, it should be noticed that \"the Diebold-Mariano (DM) test was intended for comparing forecasts; it has been, and remains, useful in that regard. The DM test was not intended for comparing models\" [181].  
The function dmtest computes the original Diebold-Mariano test [180]. The function mdmtest computes the modified Diebold-Mariano test. The modification is useful for small samples [182]. The function hmdmtest computes another modification of the Diebold-Mariano test. This modification is useful if the presence of ARCH effects is suspected in forecast errors. But it is also useful for small samples [183].  
In this package three versions of this test are implemented. Each of them as a separate function. Moreover, the outcomes are presented as a matrix. The purpose for such a choice is that the user can easily perform this test for a set of forecasts by one command.  
dmtest(y, f) mdmtest(y, f) hmdmtest(y, f)  
The arguments for all of these three functions are the same, i.e.,  
* y should be a vector of the dependent (forecasted) time-series,
* f should be a matrix of the predicted values from various models. The forecasts should be ordered by rows. The first row should correspond to the method that is compared with the alternative ones (corresponding to the subsequent rows).  
The outcomes are a matrix, in which the first column contains the tests statistics, the next columns contains \(p\)-values, respectively to the alternative hypothesis being that the alternative forecasts have different accuracy than the compared forecast, that the alternative forecasts are less accurate than the compared forecast, and that the alternative forecasts have greater accuracy than the compared forecast. The tests outcomes for different forecasts, which are compared against the selected one, are ordered by rows.  
R> wti <- crudeoil[-1, 1] R> drivers <- (lag(crudeoil[, -1], k = 1))[-1, ]  
R> ld.wti <- (diff(log(wti)))[-1, ] R> ld.drivers <- (diff(log(drivers)))[-1, ]  
R> ld.drivers.r <- ld.drivers[,1:3]  
R> m <- fDMA(y = ld.wti, x = ld.drivers.r, + alpha = 0.99, lambda = 0.99, initvar=1) R> m.y <- m$y.hat  
R> a <- altf2(y = ld.wti, x = ld.drivers.r, d = TRUE, initial.period = 50) R> a.y <- a$y.hat R> a.y <- matrix(unlist(a.y), nrow = length(a.y), byrow = TRUE)  
R> fc <- rbind(m.y, a.y)  
R> dm <- dmtest(y = as.vector(ld.wti), f = fc)  
R> dm std. DM p-val. different DM p-val. greater DM p-val.