# Dynamic Model Averaging in Economics and Finance with fDMA: A Package for R
## 5 Information-Theoretic Averaging  
Although, the particular Bayesian model averaging scheme is the main topic of this package, herein also information-theoretic averaging is implemented. The motivation is to provide a tool, which would be able to contain some competitive approach as, for example, a set of benchmark forecast. With the package described herein the presentation of the outcomes from information-theoretic averaging can be done is a very similar way as for DMA, DMS and Median Probability Model. And, indeed, such a comparison can be interesting for researchers [80]. Of course, some other packages are designed especially for information-theoretic averaging [19]. The aim of fDMA is in no sense to compete with them. But just to provide some simple and easy tool to compare DMA with the conventional method. Therefore, information-theoretic models are implemented just in very basic versions, as some kind of an add-on to fDMA package. First of all, in case of model averaging the Bayesian approach dominates in the literature. The conventional approach (i.e., frequentist one) is more popular in the context of model averaging in biological, ecological, and similar sciences, rather than in economics or finance. It should be noticed that this approach is based on completely different assumptions than the Bayesian one. For example, using all possible models is not a good approach. First, as being a conventional method (i.e., opposite to the Bayesian one), the user has to consider certain limitations of the number of observations. In other words, they should be large enough to provide correct estimations of parameters. Secondly, including models having no theoretical underlying in the averaging procedures, is widely criticized. In other words, the user should not relocate the uncertainty about the correct model to the model averaging procedure (in information-theoretic approach). He or she should rather first examine the models thoroughly, and in the context of the underlying (for example, economic) theory, and then select possibly few models to averaging [77, 78]. The most popular scheme in information-theoretic averaging is to ascribe weights basing on Akaike Information Criterion (AIC). This comes from the fact that AIC can be interpreted as the estimate of the difference between the Kullback-Leibler distance of two competing models. And Kullback-Leibler distance is a way to calculate how much information is lost when one approximates one distribution with another [80, 177]. Suppose there are \(k=1,\ldots,K\) competing models. And each of them is characterized by some \(AIC_{k}\). Denote the AIC of the model with the minimum one by \(\min_{k=1,\ldots,K}\{AIC_{k}\}\). And consider \(\Psi^{AIC}_{k}=AIC_{k}-\min_{k=1,\ldots,K}\{AIC_{k}\}\). The weights for information-theoretic model averaging [77] are then given by  
\[w^{AIC}_{k}=\frac{\exp(-0.5\Psi^{AIC}_{k})}{\sum_{i=1}^{K}\exp(-0.5\Psi^{AIC}_ {i})}\quad. \tag{22}\]  
The numerator can be interpreted as the relative likelihood of \(k\)-th model, and denominator is used for the normalisation. Sometimes Akaike Information Criterion with a correction (AICc) is suggested to be used [77]. For instance, if the ratio of number of observations (\(n\)) to the number of model's parameters (\(l\)) is less than 40. Then the weights are given by  
\[w^{AICc}_{k}=\frac{\exp(-0.5\Psi^{AICc}_{k})}{\sum_{i=1}^{K}\exp(-0.5\Psi^{AICc }_{i})}\quad, \tag{23}\]  
with \(\Psi^{AICc}_{k}=AIC_{k}-\min_{k=1,\ldots,K}\{AIC_{k}\}\). The distinction between AIC and AICc is given by \(AICc=AIC+\frac{2l(l+1)}{n-l-1}\). Sometimes, Bayesian Information Criterion (BIC) is also used [77]. Then the weights are given by  
\[w^{BIC}_{k}=\frac{\exp(-0.5\Psi^{BIC}_{k})}{\sum_{i=1}^{K}\exp(-0.5\Psi^{BIC}_ {i})}\quad, \tag{24}\]  
with \(\Psi^{BIC}_{k}=BIC_{k}-\min_{k=1,\ldots,K}\{BIC_{k}\}\). Significantly often, in certain applications, despite many other tries, equal weights lead to quite well-performing forecast in a sense of accuracy [178].