— Renewable energy harvesting through solar photovoltaic with solar smart dome at rural area can help local farmers drying agricultural product such as coffee, spices, and dried fruit. To have a more viable and economical battery for the energy storage system, an accurate prediction battery State of Charge (SOC) is important to help control the battery charging and discharging, to extend the battery lifespan. This study expl ore correlation between SOC prediction with battery observable parameter such as voltage, current and temperature. Using Transformer Neural Network with comparison of Long -Short Term Memory (LSTM) and Gated Recurrent Unit (GRU), prediction model constructe d utilizing two different datasets of laboratory lithium battery LiFePO4 and actual lead acid battery OPzS to measure model accuracy and its training time with extreme condition. Result show that voltage having strong positive correlation with SOC predicti on for both battery type, while temperature having strong positive correlation only on lead acid battery. Current didn’t have direct correlation to SOC but have strong positive correlation with voltage for both battery dataset. Best prediction result gaine d from GRU at 45 epochs with MAE 0.642%, RMSE 0.885 %, R2 99.88% and training time of 10.74s. Transformer Neural Network accuracy placed third after LSTM with MAE 1.175%, RMSE 1.634%, R2 99.69% but it has faster training time at 7.13 second. Generalization capability of neural network in SOC prediction to produce great accuracy is proven in this study on GRU model with highest MAE of 1.19% given its challenge of limited data quantity, quality, and different battery type. Keywords — Battery State Prediction, Transformer Neural Network, State of Charge, Machine Learning, Smart Solar Dryer I. INTRODUCTION Development of renewable energy especially solar photovoltaic (PV) at rural area with no national power grid can help local farmers of its energy needs and suppo rt agricultural finishing drying process for product such as coffee, spices, and dried fruit. Drying is one of the techniques currently used in food processing to remove moisture to prevent mold and bacteria from ruining the food. Coffee for example requ ire good drying process to get better scent and taste before being sold at market [1]. Common practice of drying process with open space method heavily depend on daylight which can be solv ed using closed dome with solar PV such as Solar Smart Dome 4.0 (SSD) (see figure 1) [2]. Electrical energy that produced by Solar PV at daylight then stored in battery system to provide enough energy at night. This battery system is required to power up electrical devices at dome to enable 24/7 drying process. In its implementation, this standalone energy system need low cost battery stora ge system that have optimal energy management with good control of energy usage and longer lifetime [3], [4] . Newer battery technology in last decade have been vastly improved but it still has high cost to be used at low- cost project. But for old technology such as lead acid battery tend to have lower cost and higher capacity with constraint of shorte r lifecycle of 2,000 compared to lithium battery up to 10,000 cycle [5]. With such battery characteristic, lead acid bat tery usage needs to be optimized to ensure battery having longer lifetime. Optimization can be done by having accurate battery State Of Charge for maintain battery capacity and State Of Health as battery lifespan indicator [6], [7] . State Of Charge (SOC) of battery is affected by various paramete r such as current, voltage, temperature at certain time [7], [8] . Correlation between lead acid bat tery cell parameter is related to each other at both charging and discharging process. At discharging process, having lower temperature will make current and voltage more stable and efficient [9]. When charging process, as voltage increased there is chemical reaction need to be controlled to prevent battery damage from gassing stage. Current have direct correlation with battery capacity and directly affected by battery cel l temperature during discharging process [10]. International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 2 Each cell battery life time would be different in actual usage depend on energy management efficiency at charging and discharging process. Unbalance charging power at each battery cell at solar PV microgrid would reduce available battery capacity, faster battery degradation even safety issues [7]. This condition provides research opportunity with different approach to increase battery usage efficiency . Figure 1: SSD 4.0 coffee drying process (Source : Budiman et al (2021 ) [2]) Chemical complexity in battery dynamic that constantly changing its parameter making accurate SOC prediction more complicated considering battery aging factor and nonlinear cell battery char acteristic [11]. Given its complexity, SOC prediction can be done through adaptive approach of machine learning to achieve SOC prediction efficiently with good accuracy , real time and self -learning capability throughout the time while accommodate temperature and other parameter variance [12], [13] . Machine learning SOC prediction utilizing data based approach become a solution to achieve high accuracy that can be applied at any condition and battery type with prediction model using observable time step parameter of current, voltage and temperature [14] Popular machine learning approach such as Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) do have its limitation of high computational requirement, long training time and vanishing data gradient in the process [11]. Using deep learning Transformer neural network with self - supervised learning compared to other popular method can have high accuracy and small error with 5 epoch using only 20% of training data [15]. Transformer neural network ca n utilize all input data simultaneously to gain deeper or more complete information compared to conventional neural network method [16]. This research study relation between battery parameter such as voltage, current, temperature and other recorded parameter into SOC prediction. Furthermore, this study provides SOC prediction modelling using transformer neural network to support SSD implementation with lead acid battery type (OPzV). Prediction model using transformer neural network constructed using dataset from laboratory lithium -ion battery dataset contains mul tiple time -series to achieve its best accuracy and fastest training time. Prediction model challenged further through transfer learning into actual limited dataset of lead acid battery that similar with SDD battery type to ensure its performance. In summar y, the contributions of this study are: • To analyze relation between battery parameter into SOC prediction • Design prediction model architecture using transformer neural network using 2 time -series datasets under extreme condition such as limited data quanti ty and quality, different battery type (lithium ion and lead acid) and different condition (laboratories and actual) • To measure transformer neural network best prediction model result in term of accuracy and training time compared to LSTM and GRU This pape r is organized as follows: ( I) In the first section, the background of this study is elaborated; ( II) in the second section, we outline related theory; ( III) in the third section, we detail our research methodology; ( IV) in the fourth section, the result o f this study is reported; ( V) finally, in the last section, we conclude our study II. LITERATURE REVIEW To extend battery lifetime and improve its performance, battery state of charge and state of health estimation is important because battery is one of the m ost expensive part of energy storage system [14]. Several technique to extend battery lifetime in recent studies are : limiting SOC charge level to 50% can increased battery lifetime expectancy by 44-130% [17], smart AI integration feature on battery [18], cloud battery management system with online SOC and SOH [19], optimize battery charging method [20], equalize charge current distribution [21], remaining useful lifetime (RUL) prediction [22], [23] . All these studies shows that accurate SOC is needed in battery management system for cell balancing to prevents batteries from over dischar ge and overcharge which can degrade batter ies lifetime . SOC estimation methods can be classified into four group: direct measurement, bookkeeping estimation, adaptive systems, hybrid methods . International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 3 Aside from adaptive systems, all other methods require complex mathematical models and deep understanding over battery type which highly time variant, non-linear and affected by many batteries chemical factors . Adaptive system with machine learning approach recently become more popular to achieve great SOC estimation accuracy, robustness, and effectiveness despite different battery type without increasing models complexity and estimation procedures [12]. A. Lead Acid Battery & Machine Learnin g Lead acid has been known since 1859 and used at many fields like automotive, industrial, telecommunication and standalone power generator. Despite major advancement in lithium battery, until 2015 lead acid battery still hold 70% secondary battery in worl d [24]. Despite its mature technology and lower cost, lead acid battery has shorter lifespan and require better control of its capacity charge and discharge process to ensure i ts safety application and extend lifetime. On lead acid battery, self -discharge happen because of internal chemical reaction which its speed effected by temperature, chemical composition, electrolyte formulation and cell layer composition [24]. This discharge process needs to be controlled to maintain depth of discharge level does not fall below 50% and damaging battery permanently. Charging process of lead acid battery is consi sts of 3 phases : (1) efficient charging until battery reaches 70%-80% SOC, (2) mixed, SOC is between 75 - 100% and (3) gassing evolution, is happen when SOC already at 100% but still being charged. Few causes of lead acid battery broke down are (a) positive cell plate degradation , (b) unperfect charging process, (c) sulfation on negative plate due to overcharging, (d) unbalance SOC between cell [25]. It shows how important SOC in battery optimization to extend its lifespan and more economical usage. In SOC prediction, model based approach can produce very high accuracy but it requi res deep understanding of chemical reaction of battery , component, long time and complex mathematical model in its calculation [26]. On the other hand, data-based approach of machine learning utilizing observable parameter f rom battery cell without prior knowledge of battery requirement, as it was black box model concept to produce SOC prediction. B. Transformer Neural Network Machine learning is part of Artificial Intelligence (AI) to construct mathematical model based on sam ple data to produce prediction or decision [27]. Sample data then split into training data and test data both labelled or non labelled data with more data meaning higher accuracy prediction result. Machine learning approach using big scale of data require high computational power and long time to train [11], [13] . Breakthrough in machine learning arrived when transformer neural network was introduced back in 2017 by Vaswani [28] that promising faster algorithm and overcome limitation on popular Recurrent Neural Network (RNN) of LSTM and GRU model. Transformer neural network utilizing encoder -decoder and unique ―attention‖ mechanism on its architecture to do Natural Language Processing (NLP) at first. Few of latest usage on SOC prediction is done by Hannan et al in 2021 [15] with result of having high accuracy with MAE 0.44% on constant ambient temperature and MAE 0.7 on varying ambient temperature. Transformer neural network capable of produce high accuracy of SOC prediction with innovative immersion and invariance to achieve higher accuracy than popular baseline other machine learning model [16]. All data input in transformer mapped by positional encoding using this equation: With pos and i stand for position and dimension that feed into single head attention mechanism with SoftMax function to produce wei ghted sum of value using this equation: Transformer neural network capable of having multi head attention mechanism enable its model to read all data simultaneously regarding its sequence. In order to do better prediction on time -series data according to literature study [15], [16] , transformer in this study only using encoder without decoder with shortest epoch as possible depicted in figure 2 . International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 4 Figure 2: Transformer with encoder only C. Relation between voltage, current and temperature to SOC Prediction Discharging process on battery happen when there is flow of current out that measured with capacity in ampere hour. Battery voltage in time ste p will progressively have lower value and increasing battery cell temperature as current flow [24]. As temperature rise, it shows that battery capacity being drained that measu red by high current output until battery voltage reach its basic specification of 1.7V for lead acid battery. On charging process, voltage increased as charging current flow into battery and effecting temperature too due to internal battery chemical reacti on. Relation between current and voltage when charging process define various charging method such as constant current, constant voltage, taper, pulse, trickle, float and rapid charging , which each method directly impact to battery lifespan [24]. Based on charging and discharging process , it show that voltage, current and temperature definitely have correlation that need to be proven through Pearson correlation analysis [29]. From literature study, SOC prediction mostly use voltage, current and temperature as its input as shown in Table 1. Table 1. Summary of Input and Output Parameters for Battery State Prediction Input Parameter SOC Voltage [30]–[42] Current [30]–[42] Temperature [30], [31], [34] –[41], [43] III. METHODOLOGY A. Dataset In this study, we use two datasets from different battery type both lithium ion and lead acid. First dataset of lithium ion LiFePO4 battery by Sandia National Laboratories [44] with constant charging of 0.5C ranging from 0 to 100% SOC at various controlled room temperature 150C, 250C and 350C. LiFePO4 battery was chosen due to its common implementation in similar standalone energy storage system such as S DD. Various temperature of first dataset is selected to represent as close as possible with actual temperature at SDD various site. This first dataset consists of 3 years data with 1,320,946 rows with 11 attributes . Second dataset using OPzS lead acid bat tery data from one solar power plant owned by Indonesia National Power Plant (PLN). This dataset selected because it has same battery type used by some SDD site in Indonesia aside from OPzV type. This dataset has limited 3 days data of only 806 rows with 1 5 attributes to further challenge constructed prediction model with extreme condition of limited quality and quantity data. A. Models All tested models in this study were developed with similar architectures that receive the time series variables and output o f State of Charge (SOC) at the next time step. In total, we tested four deep learning models: Transformer with optimizer ADAM [45] (TF ADAM), Transformer with optimizer SGD [46] (TF SGD), LSTM and GRU. The architecture of these models is depicted in Figure 3. Figure 3: Architecture of developed model International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 5 Baseline transformer models first constructed using SNL dataset with 80% training data and 20% testing data, until prediction error reach below 2% before transferred to next phase. Error measurement in this study using Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and R -squared (R2) to evaluate prediction result. When transformer model achieves MAE <2%, same hyperparameter setting transferred to LSTM and GRU model to compare its prediction result and training time. All models further challenged by transfer learning to second dataset and repeating same step depicted in figure 4. SOC prediction with transfer learning into different battery type can be applied efficiently to get great predictio n result [47] even with limited data constraint [15]. Finally, summa ry of all models results for both datasets will be presented to be analyzed of its performance both accuracy and its training time. Figure 4: Develop model flow diagram B. Hyperparameter Settings Hyperparameter settings in this study being set at same value to have fair comparison on all models (see table 2). No regularization was applied to all models. Optimizer ADAM turned out to be best optimizer, therefore it applied on LSTM and GRU for both datasets . Table 2. Hyperparameter setting for all models Hyperparameter Value Time lag n = 32 Input Layer 4 Output Layer 1 Hidden Neuron 128 Batch Size 64 Dropout 0 Epoch 5, 15 , 30, 45 Learning rate 0,001 Weight decay 0 Optimizer ADAM (Transformer, LSTM, GRU) SGD (Transformer) IV. RESULT AND ANALYSIS A. Corre lation analysis of voltage, current and temperature to SOC prediction Research finding in this study shows that in: (a) Voltage is main attribute in SOC prediction (b) Battery cell temperature have very strong correlation with SOC but only on lead acid bat tery dataset, (c) Current didn’t have direct correlation with SOC, but it does have very strong correlation with voltage for both datasets . On first dataset of lithium -ion battery, voltage and charge capacity have same strong correlation score of 0.59 with SOC while current only at 0.053. For this study, charge capacity attribute not selected because it isn’t common observable parameter in many batteries management system hardware. Other attribute that can only be recorded by specific battery hardware teste r such as charge energy (Wh), discharge energy (Wh) is not used in this study. Battery cell temperature in this dataset have negative correlation value of -0.039 shown in figure 6 due to controlled condition in laboratory when measurement being recorded. International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 6 Figure 5a show there is linear correlation between ambient temperature with battery cell temperature. With constant charging rate of 0,5C and same discharging rate, this condition creating linear increase between ambient temperature with battery cell tempe rature with its mean temperature difference below 10C. On second dataset of PLN, SOC have quite strong correlation value of 0.20 with voltage while battery cell temperature and ambient temperature have strongest correlation with respective value of 0.45 a nd 0.35. It is totally opposite of first dataset with figure 5b show there is major fluctuation with battery cell temperature is not having linear relation with ambient temperature. Even at lowest ambient temperature of 23.9 0C, battery cell temperature is still at 29.3 0C. Ambient temperature having mean at 32.390C while battery cell temperature means at 34.020C. As this dataset come from actual usage of battery, this finding is having more valid to state that battery cell temperature plays important part in SOC prediction. Current on both datasets didn’t have strong direct correlation with SOC, but it has very strong correlation with voltage with value of 0.53 and 0.79. Based on battery principle, current is a flow of energy that affected temperature and voltage, therefore current is proven to be important parameter in SOC prediction. Figure 5: Ambient and Cell Temperature Plot (a) Dataset SNL (b) Dataset PLN Figure 6: Pearson Correlation Analysis (a) Dataset SNL (b) Dataset PLN B. Models Prediction Result All models able to achieve R2 of 0.99 on all epoch except on transformer model with SGD optimizer with R2 at 0.96 at 5 epochs . Table 3a show that Transformer ADAM can achieve MAE of 0.01135 at 5 epochs but LSTM and GRU have ev en better result. For best accuracy result, GRU is best choice at 45 epochs with MAE 0.000939, RMSE 0.001310 and R2 0.999979. LSTM and GRU widely recognized to have very good prediction result on sequential data especially on constant room ambient temperat ure [15]. But both GRU and LSTM does have longer training time compared to Transformer model in all epochs (see table 3b). (a) (b) (a) (b) International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 7 Table 3. Performance of all models on SOC prediction dataset SNL (a) Accuracy (b) Training Time With training time consideration, Transformer ADAM in this dataset can be considered as best option at 5 epochs with R2 achieving 0.9975 and fast training time of 1.43 second. Figure 7 show that Transformer ADAM model able to do prediction very well with its projected value over actual SOC value Figure 7: Prediction and actual SOC value graph dataset SNL C. Transfer L earning Prediction Result Result show that all models did not suffer overfitting or under fitting based on plots in figure 8. Therefore, generalization capability of all models is assured. Out of all remaining models, Transformer SGD is the only model th at having hard time on achieving stable convergence point. It is resulting that this model having lowest accuracy in this dataset with best R2 achieve only 0.9662 at 45 epochs compared to other models. Figure 8: Train and validation loss plot of al l models on dataset SNL Table 4 show that all models did not achieve intended MAE at 5 epochs . GRU able to achieve R2 at 0.99 level at 15 epochs while Transformer ADAM and LSTM at 30 epochs . For best accuracy result, GRU once more is best choice at 45 epoc hs with MAE 0.011981, RMSE 0.025941 and R2 0.997761. With training time consideration, GRU was still best model for this dataset at 30 epochs, followed by LSTM at 45 epoch and Transformer ADAM at 45 epochs . Due to transformer neural network nature of readi ng all data non sequential, this prediction model needs much more data which this dataset can’t provide therefore it explained its inferior result to other models. On the other hand, GRU & LSTM processed prediction by sequential time-series data where temp erature variance on limited dataset still able to produce high accuracy prediction [48]. (b) (a) International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 8 Table 4. Performance of all models on SOC prediction datas et PLN (a) Accuracy (b) Training Time A level -2 heading Figure 9 show that GRU model have best prediction with its projected value over actual SOC value. On the other hand, Transformer SGD struggled with inconsistent prediction result. Transformer ADA M performed quite good but it still inconsistent in some point, not as good as LSTM. Figure 9: Prediction and actual SOC value graph dataset PLN Figure 10: Summary Result (a) Average training time, ( b) Average MAE, (c) Average R2 V. CONCLUSION Best prediction result gained by GRU model at 45 epochs with average of MAE 0.645%, RMSE 0.885%, R2 99.88% and training time of 10.74 second. Transformer SGD have lowest accuracy compared to other models, but it has fastest training time on all epochs. Transfo rmer ADAM have faster training time compared to LSTM and GRU with good accuracy. Considering best training time and accuracy, GRU at 15 epoch is best prediction model with MAE 1.05%, RMSE 1.33%, R2 99.54% and training time of 3.78 second. (b) (a) (a) (b) (c) International Journal of Emerging Technology and Advanced Engineering Website: www.ijetae.com (E -ISSN 2250 -2459, Scopus Indexed, ISO 9001:2008 Certified Journal, Volume 12, Issue 10, October 2022) 9 This study als o proves that generalization capability of neural network to produce high accuracy prediction across battery type under extreme condition such as limited quantity and quality data can be done. To further improve this prediction model, this model can be tested on actual battery data from SDD once its available, and if possible, on actual usage data of LiFePO4. Acknowledgements This work is supported by Research and Technology Transfer Office, Bina Nusantara University as a part of Bina Nusantara Universi ty’s International Research Grant, entitled: Enabling Low -Cost Renewable Energy Systems for Localized, Self -Sufficient Power Production -Battery Charging Optimization using Machine Learning. With contract number: No.061/VR.RTT/IV/2022 and contract date: 8 A pril 2022.