## 4 Conclusions  
In this paper, we analyzed the feasibility of a novel type of encoder, the Causal Feature Extractor, as a part of an encoder-decoder deep neural network, in different problems of machine neural translation. The results obtained are very promising, achieving a 63.0% accuracy in bilingual translation, 96.6% in LaTeX decompilation and 60.1% in audio transcription. However, the best solution is always the specifically designed system, that has been adjusted and fine-tuned by the corresponding research groups over the years, with improvements of 6.6%, 0.2% and 10.8% in the cited problems, respectively.  
Figure 4: Attention matrices produced by the attention mechanism in the problem of bilingual translation, for the translation of the sentence (horizontal axis) “A guy works on a building”. (**a**) Attention matrix for manual CFE, the output (vertical axis) is “Ein Mann arbeitet an eiem Gebäude”. (**b**) Attention matrix for random CFE, the output is “Ein Mann arbeitet auf einem Gebäude”. (**c**) Attention matrix for Transformer model, the output is “Ein Typ arbeitet an eiem Gebäude”. (**d**) Attention matrix for LSTM, the output is “Ein Mann arbeitet an eiem Gebäude”.  
Therefore, the results obtained by our approach are close to that of other works that constitute the state of the art, especially in the image processing problem of LaTeX decompilation.  
Furthermore, the proposed model has the inherent advantages of convolutional networks with respect to recurrent and LSTM networks. On the one hand, it is a generic architecture that can be adapted to a large number of scenarios, while the use of recurrent networks is more restricted. On the other hand, convolutional networks are known for being parallelizable and highly optimized for training using GPUs, so improving the implementation of this architecture should be much faster than recurrent networks. This was observed in the average execution times per iteration, which is considerably faster for CFE than for the specific models. Those solutions require on average 70% more time than the proposed approaches.  
Clearly, there is still ample room for improvement in the application of CFE to the problems of natural language processing. For example, more complex attention mechanisms (such as multi-head attention or local attention) could be combined with the proposed CFE architecture. Also, elimination or relaxation of the use of dilations in the CFE architecture, which could be diluting the influence of the input data too much, could be beneficial. Finally, since the proposed CFE model is very generic, it could be interesting to analyze its application in other areas of computational learning.  
Conceptualization, A.J. and G.G.-M.; methodology, A.J. and G.G.-M.; software, A.J.; validation, A.J. and G.G.-M.; formal analysis, A.J.; investigation, A.J. and G.G.-M.; resources, [PERSON]; data curation, [PERSON]; writing--original draft preparation, [PERSON]; writing--review and editing, [PERSON] and [PERSON]-M.; visualization, [PERSON]; supervision, [PERSON]-M. All authors have read and agreed to the published version of the manuscript.  
This research was funded by Spanish Ministry of Science, Innovation and Universities, FEDER funds, under grant RTI2018-095855-B-I00 (G.G.-M).  
[PERSON] wants to acknowledge support from the Max Planck Institute for Intelligent Systems.  
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l} ACC & Accuracy \\ BLEU & Bilingual evaluation understudy \\ CFE & Causal feature encoder \\ CNN & Convolutional neural networks \\ FCNN & Fully connected neural network \\ LSTM & Long short-term memory \\ MNT & Machine neural translation \\ NLP & Natural language processing \\ PER & Perplexity \\ WER & Word error rate \\ RNN & Recurrent neural network \\ \end{tabular}