# Currency Crises Prediction Using Deep Neural Decision Trees
## 2 Methodology
### Deep Neural Decision Trees (DNDTs)  
Deep neural decision trees are DT models executed by deep-learning NNs, where a configuration of DNDT weightings corresponds to a specific decision tree and is thus interpretable [20]. Nevertheless, as DNDT is performed by the NN, it has several different properties of conventional DTs: DNDTscan be implemented from the NN structure in software such as Python (Pytorch). All parameters are optimized simultaneously with stochastic gradient descent (SGD) instead of a complex greedy splitting procedure; this allows large-scale processing with mini-batch-based learning and can be connected to any larger NN model for end-to-end learning with backward propagation. Continuing with this explanation, conventional DTs learn through a greedy and recursive division of characteristics [30]. This may have benefits with respect to the selection of functions; however, this greedy search may become inefficient [31]. Some recent work explores alternative approaches to train decision trees that aim to achieve better performance, for example, with a latent variable structured prediction [31]. On the other hand, a DNDT is much simpler, but we can still find the best solutions compared to conventional DT inductors when looking for the structure and parameters of the tree with SGD. Finally, while conventional DT inductors only use binary divisions to simplify, DNDT can also work with arbitrary cardinality divisions which can sometimes generate more interpretable trees. The algorithm begins by implementing a soft binning function to calculate the error rate for each node, making it possible to make decisions divided into DNDTs [32]. In general, the input of a binning function is a real scalar x which generates an index of the containers to which x belongs. Assuming x is a continuous variable, group it into \(n+1\) intervals. This requires \(n\) cut-off points which are trainable variables in this context. The cut-off points are denoted as (\(\beta_{1}\), \(\beta_{2}\),..., \(\beta_{n}\)) and are strictly ascending such that \(\beta_{1}<\beta_{2}\)\(<\ldots<\beta_{n}\). The activation function of the DNDT algorithm is implemented based on the NN defined in Equation (1). \[\pi=\text{fw,b,}\tau\left(\text{x}\right)=\text{softmax}((\text{wx}+\text{b}) /\tau) \tag{5}\]  
where w is a constant with value w = [1, 2,..., \(n+1\)], \(\tau>0\) is a temperature factor, and b is defined in Equation (6). \[\text{b}=[0,-\beta_{1},-\beta_{1},-\beta_{2},\ldots,-\beta_{1}-\beta_{2}- \ldots-\beta_{n}] \tag{6}\]  
The NN defined in Equation (1) gives a coding of the binning function x. Additionally, if \(\tau\) tends to 0 (often the most common case), the vector sampling is implemented using the Straight-Through (ST) Gumbel-Softmax method [33]. Given the binning function described above, the key idea is to build the DT using the Kronecker product. Assuming we have an input instance \(x\in R^{D}\) with D characteristics. Associating each characteristic \(x_{d}\) with its own NN \(\text{f}_{\text{d}}\) (x\({}_{\text{d}}\)), we can determine all the final nodes of the DT, in line with Equation (7). \[\text{z}=\text{f}_{1}(\text{x}_{1})\otimes\text{f}_{2}(\text{x}_{2})\otimes \ldots\otimes\text{f}_{\text{D}}(\text{x}_{\text{D}}) \tag{7}\]  
where z is now also a vector that indicates the index of the leaf node reached by instance x. We assume that a linear classifier on each leaf z classifies the instances that reach it. The number of cut points per feature is the complexity parameter of the model. The cut-off point values are not limited, which means that some of them may be inactive. For example, they are smaller than the minimum x\({}_{\text{d}}\) or greater than the maximum x\({}_{\text{d}}\). With the method described so far, we can route the input instances to the leaf nodes and classify them. Therefore, training a decision tree becomes a matter of training the cut-off points of the container and the sheet sorters.