# Implementation of the SoftMax Activation for Reconfigurable Neural Network Hardware Accelerators
## 7 Conclusions  
Presently, machine-learning algorithms are becoming an important part of many modern intelligent systems. However, the variety and computational complexity of these algorithms is a challenge for low-power devices, such as smartphones, IoT sensors, satellites, and many others. A possible solution to this problem is reconfigurable hardware accelerators. Their key feature is the capability to support a wide variety of algorithms and dynamically alter them by an external signal. This paper explores the reconfigurable hardware accelerators based on reconfigurable computing environments. RCE is a class of computing systems that consists of numerous similar processing elements connected in a grid pattern. To use RCE efficiently, the required algorithms need to be adapted to its particularities.  
This paper discusses the implementation of the SoftMax activation in the RCE. The implementation leverages several optimizations: using a piecewise linear approximation of the exponential function, modifying input values to limit their range, spatial distribution, and parallelization of computations. The proposed implementation of the piecewise linear approximation with variable step in the RCE is of great importance. It is a generic approach that can be easily reused to perform a wide variety of other functions in the RCE, greatly expanding its capabilities. The spatial distribution and parallel computing ensures high performance of the considered implementation of the SoftMax function.  
The characteristics of the proposed model were evaluated by simulations on FPGA. The simulation results were compared to other implementations, lacking the reconfiguration capability. The proposed implementation of the SoftMax demonstrates high performance (about 43 ns for three inputs or 1.12 Gbps throughput) and high accuracy (the maximum and average absolute errors are 0.002 and 0.00078, respectively). Only one of the three counterparts presented has a throughput of 2.4 Gbps, which noticeably outperforms the proposed implementation. The resource usage is also high--the model utilizes 54,718 FPGA logic elements, compared to 17,870 logic elements in the previously mentioned research.  
The high performance of the developed models proves the efficiency of using the RCE-based accelerators as a replacement for rigid ones in tasks that require variability and high flexibility of the implemented algorithms. The noticeably high resource consumption of the proposed model is the cost of its spatial distribution and the flexibility of the RCE. This is the price for all the previously mentioned benefits of the reconfiguration ability. The proposed implementation of the SoftMax function allows RCE-based accelerators to tackle a wide variety of classification tasks. The RCE-based reconfigurable accelerators seem to be a promising direction for the development of AI hardware.  
Conceptualization, V.S. and S.S.; Data curation, D.S.; Formal analysis, S.S.; Funding acquisition, D.S.; Investigation, V.S.; Methodology, D.S.; Project administration, S.S.; Resources, S.S.; Software, V.S.; Supervision, D.S.; Validation, D.S.; Visualization, V.S.; Writing--original draft, V.S.; Writing--review and editing, S.S. All authors have read and agreed to the published version of the manuscript.  
This study was supported by the Tomsk State University Development Programme (Priority-2030).  
Not applicable.  
The data presented in this study are available in article [[PERSON]; [PERSON]; [PERSON] Sigmoid Activation Implementation for Neural Networks Hardware Accelerators Based on Reconfigurable Computing Environments for Low-Power Intelligent Systems. Appl. Sci. 2022, 12, 5216. [[https://doi.org/10.3390/app12105216](https://doi.org/10.3390/app12105216)]([https://doi.org/10.3390/app12105216](https://doi.org/10.3390/app12105216))].  
The authors declare no conflict of interest.