# Text Normalization Using Encoder-Decoder Networks Based on the Causal Feature Extractor
## 5 Conclusions  
In this paper, a new encoder-decoder architecture with attention mechanisms has been proposed for the problem of text normalization, using a character-level approach and introducing a new type of encoder. This encoder, called Causal Feature Extractor, is a novel technique designed to work properly in cooperation with the attention mechanisms. The experiments have empirically proven that this method is able to achieve very positive results, using the attention matrices more like it would be expected. Besides, it is able to work at least as good as the best of the compared encoders, and it brings all the benefits of using convolutional neural networks (e.g., computational efficiency and fast convergence). The last aspect that distinguishes this encoder from the traditional recurrent encoders is its simplicity to be adapted to other input layouts (for example, sound, images or video). Another contribution is the introduction of a new variation of the attention mechanisms, by using a context matrix instead of a vector.  
Regarding previous works, the initial results have shown to be close to the state of the art, with much room for future improvements. Despite getting worse accuracy than the method presented in [6] (92.74% vs. 99.8%), it does not critically suffer from unrecoverable errors, nor it seems to concentrate its errors on any particular semiotic class, since most errors are attention-based; besides, the proposed method is less computationally expensive and does not include additional rule-based filters.  
From a general point of view, an interesting result that can be extracted is the empirical proof that empowers the role of encoders in the encoder-decoder architectures. It has been shown that the system does not work correctly if it only takes features of single elements (without considering their neighborhood).  
Future research lines could focus on some aspects such as applying the proposed CFE encoder as a general-purpose encoder in different tasks of natural language processing. In particular, applications to audio and images are already being studied. It is also interesting to develop new methods for hyperparameter selection in order to obtain better results. The models have a large number of hyperparameters, and each execution of the system can take several hours. Finally, future works could consider conditioning the model to external factors, for example, to distinguish between British and American English.  
Conceptualization, [PERSON] and [PERSON]-M.; methodology, [PERSON] and [PERSON]-M.; software, A.J.; validation, [PERSON] and [PERSON]-M.; formal analysis, A.J.; investigation, A.J. and [PERSON]-M.; resources, A.J.; data curation, A.J.; writing-original draft preparation, A.J.; writing-review and editing, [PERSON] and G.G.-M.; visualization, A.J.; supervision, [PERSON]-M. All authors have read and agreed to the published version of the manuscript.  
This research was funded by Spanish Ministry of Science, Innovation and Universities, FEDER funds, under grant RTI2018-095855-B-I00 (G.G.-M.).  
We would like to express our gratitude to [PERSON] for his useful feedback on this article. Besides, [PERSON] acknowledges support from the Max Planck Institute for Intelligent Systems.  
The authors declare no conflict of interest.