# Environmental Fault Diagnosis of Solar Panels Using Solar Thermal Images in Multiple Convolutional Neural Networks

Authors: Tamilselvi Selvaraj; Ramasubbu Rengaraj; GiriRajanbabu Venkatakrishnan; SoundhariyaGanesan Soundararajan; Karuppiah Natarajan; PraveenKumar Balachandran; PrinceWinston David; Shitharth Selvarajan
Journal: International Transactions on Electrical Energy Systems
 (2022)

---

## Abstract

Every year, each solar panel suffers an efficiency loss of 0.5% to 1%. This degradation of solar panels arises due to environmental and electrical faults. A timely and accurate diagnosis of environmental faults reduces the damage caused by faults on the panel. In recent years, deep learning precisely convolutional neural networks have achieved wonderful results in many applications. This work is focused on finely tuning pretrained models of convolutional neural networks, especially AlexNet, GoogleNet, and SqueezeNet. Based on the performance metrics, SqueezeNet is used for training thermal images of solar panels and for the classification of environmental faults. The results obtained show that SqueezeNet has a significant testing accuracy of 99.74% and F1 score of 0.9818, which make the model successful in identifying environmental faults in solar panels and help users to protect the panels.


---

## Body

## 1. Introduction

Photovoltaic systems are one of the most distinguished and clean sources of energy which generate power by converting solar energy from the sun into direct current electricity. In 2019, solar power delivered was 2.7% of total worldwide electricity production. The International Energy Agency has stated by 2050, solar power would contribute up to 16% of the world’s electric energy production with solar being the largest renewable source of energy. Electric power generated by a 1 KW system of solar panels is roughly around 850 KWh per year. However, each year the solar panels suffer an efficiency loss of 0.5% to 1% resulting in reduced output power generation. This power loss in solar panels arises due to environmental and electrical faults [1].Environmental faults like shading, soiling, and snowing tend to cause a significant power loss in PV modules. Solar panels are expensive and require proper maintenance throughout the year. Hence, it is necessary that faults in the solar panels are detected and rectified in the preliminary stage [2].The goal of this research is to detect environmental faults in the solar panels accurately. For this purpose, different convolutional neural networks (CNN), namely, AlexNet, GoogleNet, and SqueezeNet are trained and their performance metrics are obtained. Based on the results, a suitable network is opted for training thermal images of solar panels and for the precise determination of environmental faults in solar panels.S.K. Firth conducted a survey and found that annually the different faults in photovoltaic systems reduce the power output by 19% [3]. To detect these faults, W. Chine proposed a feasible solution for the fault classification of photovoltaic system using Artificial Neural Network (ANN) [4]. In this proposed method, a simulation model is introduced for computing a number of parameters like current, voltage, and the number of peaks in the current-voltage (I-V) characteristics of the PV cells. Two ANN architectures are employed for environmental and electrical faults detection in PV systems: the Multilayer Perceptron (MLP) and the Radial basis function (RBF). For the MLP based model, the true and false classification rates achieved are 90.3% and 9.7%, respectively, and for RBF-based model, they are 68.4% and 31.6%, respectively.C. Mantel developed a machine learning model where electroluminescence images of photovoltaic panels were fed to the model [5]. Two architectures, Support Vector Machine (SVM) and Random Forest Model (RF), were employed. The results obtained from SVM had an accuracy of 0.997 and a recall of 0.274. The RF model had an accuracy of 0.967 and a recall of 0.193. The results demonstrated that SVM had an improvement of about 3% in terms of accuracy compared to the RF model. However, the high accuracy of both the classifiers (SVM and RF) makes them promising for detecting faults in PV modules from electroluminescence images.Furthermore, Natarajan studied the above model by using thermal images of photovoltaic systems [6]. Based on the fundamentals of thermal image processing, an algorithm is suggested for deriving the characteristics of the solar cells in operation. The images are classified by a classifier tool called SVM, which determines whether the solar modules are faulty or nonfaulty. The results obtained are 97% accurate with the comparison of test and training results. This fault classification technique is used in real time for the large PV system with very less computation time.Papadomanolaki on his research based on benchmarking different deep learning networks for classification of precise and sharp satellite multispectral data has trained CNN from scratch using huge datasets comprising large number of labeled data samples [7]. Therefore, employing a pretrained deep learning model has been proposed and successfully achieved in this paper.In addition, C. Szegedy has carried out a detailed study of different CNN architectures, namely, AlexNet, VGG Net, and GoogleNet in the computer vision community, and their versions are made available publicly [8].Diverse range of works in machine learning community have successfully manifested the generalization power of deep learning networks where large datasets have performed well in regards with classification of other datasets, even from different distinct. Motivated by these results, we apply pretrained models of AlexNet, GoogleNet, and SqueezeNet on a large-scale image classification dataset for fault classification in PV arrays [9–11].The following paper comprises various sections elaborating in detail about environmental fault diagnosis of solar panels using CNN. Section2 describes about the various environmental faults in solar panels while the Section 3 discusses about the identification of these faults. In Section 4, the mechanism of deep learning and different neural networks are elaborated followed by the explanation of confusion matrix algorithm in Section 5. Section 6 displays the corresponding experimental results for environmental fault diagnosis of solar panels using various CNN architectures.


## 2. Environmental Faults in Solar Panels

Solar panels are operated in the open air, making them vulnerable to environmental conditions [12]. Under these conditions, the PV module may fail to operate efficiently due to the following effects:(i)
Shading effect: shading can arise from direct shadows or temporary shadows. Direct shadows cause serious impact on the performance of the solar panel. Temporary shadows are caused due to shades of buildings, trees, snow, etc. Shading effect is classified into two types such as partial shading and fully shading. Partial shading leads to reduced current and voltage and the output power is dropped to half the nominal value while full shading leads to no output power extraction as shading of at least 1/36 of the cell reduces the output power by 75% [13].(ii)
Soiling effect: the aggregation of dust particles like sand, cement, mud, and leaves etc., on solar panel’s surface is called soiling. The various factors causing soiling and power loss are climatic conditions, tilt angle, and liquid used for cleaning solar panels [14].(iii)
Snowing effect: when a thick layer of snow is deposited on a solar panel, solar cells find it difficult to absorb solar radiation, and this affects the output power very badly. This affects more worse when the snow remains on the panel for a longer period of time [15].(iv)
Temperature rise: solar panels are temperature-sensitive. A temperature rise above the optimum temperature 35°C degrades the output power of the solar panel and causes excess heat emission, highly affecting the open circuit voltage of the solar panel.


## 3. Identification of Environmental Faults

The following section describes about the various stages involved in the identification of environmental faults in solar panels such as capturing and processing the thermal images of the solar panels, detecting hotspots in the images, and identifying those faults.


### 3.1. Thermography

Environmental faults like the temperature rise in solar panels and shading effects cannot be perceived by human eyes. In such cases, the solar panel fault can be detected by the principle of thermography. Each object having a temperature above the absolute zero point (0 Kelvin) emanates infrared radiation, which is directly proportional to its intrinsic temperature [16,17]. A thermal imager when placed at a distance of 1 m from the solar panel, the imager perceives the infrared radiation from the solar panel and determines the surface temperature of the panel. The imager converts the infrared radiation into electrical signals and displays these signals with varying temperature in different colors [18,19].


### 3.2. Thermal Image Processing

Thermal image processing helps to enhance the characteristics of the image data. The image processing method used in this research captures the accurate hotspot area of the panel, and the contrast level of the image is adjusted for better training of faults in neural networks. In Figure1, the actual thermal image of the panel is displayed, whereas in Figure 2, the contrast level of the panel is adjusted high using thermal image processing software.Figure 1
Thermal image of a solar panel with fault.Figure 2
Thermal image of the faulty solar panel with the contrast level adjusted high.


### 3.3. Hotspot Phenomenon

Hotspots are high temperature zones affecting a particular section of a solar panel, thereby reducing the localized efficiency and lowering the output power of the solar panel [20]. The hotspot phenomenon primarily occurs due to shading and dust accumulation on the panel. These preliminary damages in the solar panel can be detected using thermal images. In Figure 3, the black box shows the hotspot of the panel indicating the abnormal working condition of the solar panel. The remaining blue portion and faded yellow-green portion indicates the normal functioning of the solar panel.Figure 3
Hotspot location in the solar panel.


### 3.4. Fault Detection and Identification

Fault detection checks for any abnormal working condition in the solar panel. In this research, the faults are determined based on the location of the hotspot in the thermal image [21].


## 3.1. Thermography

Environmental faults like the temperature rise in solar panels and shading effects cannot be perceived by human eyes. In such cases, the solar panel fault can be detected by the principle of thermography. Each object having a temperature above the absolute zero point (0 Kelvin) emanates infrared radiation, which is directly proportional to its intrinsic temperature [16,17]. A thermal imager when placed at a distance of 1 m from the solar panel, the imager perceives the infrared radiation from the solar panel and determines the surface temperature of the panel. The imager converts the infrared radiation into electrical signals and displays these signals with varying temperature in different colors [18,19].


## 3.2. Thermal Image Processing

Thermal image processing helps to enhance the characteristics of the image data. The image processing method used in this research captures the accurate hotspot area of the panel, and the contrast level of the image is adjusted for better training of faults in neural networks. In Figure1, the actual thermal image of the panel is displayed, whereas in Figure 2, the contrast level of the panel is adjusted high using thermal image processing software.Figure 1
Thermal image of a solar panel with fault.Figure 2
Thermal image of the faulty solar panel with the contrast level adjusted high.


## 3.3. Hotspot Phenomenon

Hotspots are high temperature zones affecting a particular section of a solar panel, thereby reducing the localized efficiency and lowering the output power of the solar panel [20]. The hotspot phenomenon primarily occurs due to shading and dust accumulation on the panel. These preliminary damages in the solar panel can be detected using thermal images. In Figure 3, the black box shows the hotspot of the panel indicating the abnormal working condition of the solar panel. The remaining blue portion and faded yellow-green portion indicates the normal functioning of the solar panel.Figure 3
Hotspot location in the solar panel.


## 3.4. Fault Detection and Identification

Fault detection checks for any abnormal working condition in the solar panel. In this research, the faults are determined based on the location of the hotspot in the thermal image [21].


## 4. Convolutional Neural Network

Deep learning is a crucial fragment of machine learning where multiple layers of nodes perform complex operations like abstraction and representation, which perceive images, sound, and text. Figure4 represents the functioning of deep learning network in a flowchart.Figure 4
Flowchart representation of deep learning algorithm.Convolutional neural networks (CNNs) are a subset of deep neural networks which are designed to operate on visual imagery [22, 23]. The properties and features of the input images are extracted and are encoded by the CNN architecture resulting in the reduction of parameters in large quantities compared to normal neural networks.


### 4.1. Types OF CNN

(i)
AlexNet: Alex Krizhevsky designed AlexNet (AN) in 2012. This network consists of eight layers where the first five layers are convolutional layers succeeded by max-pooling layers and the last three layers are fully connected layers. For better training performance, AN prefers ReLu activation function over sigmoid and tanh [24].(ii)
GoogleNet: Google developed the GoogleNet (GN) architecture. This network consists of nine inception modules where in the twenty-two layers, four are convolutional layers and max-pooling layers, three layers are for average pooling followed by a section of five fully connected layers and three softmax layers [25].(iii)
SqueezeNet: SqueezeNet (SN) was developed by DeepScale in 2016 and has accuracy with 50x less parameter. This network contains eighteen layers which begins with a single convolution layer followed by eight fire modules and ends with a convolutional layer. Different activation functions like ReLu, tanh, and sigmoid can be used where ReLu provides a good boost in the performance [26].


## 4.1. Types OF CNN

(i)
AlexNet: Alex Krizhevsky designed AlexNet (AN) in 2012. This network consists of eight layers where the first five layers are convolutional layers succeeded by max-pooling layers and the last three layers are fully connected layers. For better training performance, AN prefers ReLu activation function over sigmoid and tanh [24].(ii)
GoogleNet: Google developed the GoogleNet (GN) architecture. This network consists of nine inception modules where in the twenty-two layers, four are convolutional layers and max-pooling layers, three layers are for average pooling followed by a section of five fully connected layers and three softmax layers [25].(iii)
SqueezeNet: SqueezeNet (SN) was developed by DeepScale in 2016 and has accuracy with 50x less parameter. This network contains eighteen layers which begins with a single convolution layer followed by eight fire modules and ends with a convolutional layer. Different activation functions like ReLu, tanh, and sigmoid can be used where ReLu provides a good boost in the performance [26].


## 5. Parameters for Determination of the Suitable Network

Confusion matrix is a predictive analysis method used for describing the performance of the classification model in deep learning [27]. The rows in the confusion matrix depict the true class, whereas the columns depict the predicted class. The confusion matrix reports true positive, false positive, true negative, and false negative. Furthermore, it helps to calculate statistical measures like accuracy, specificity, sensitivity, precision, and F1 score.True positive (TP): the output class matches with the true class. True negative (TN): the output class is not predicted for other classes except the true class. False positive (FP): the output class is predicted as the true class when it is not. False negative (FN): the output class is not predicted as the true class when it is. Figure5 shows the true positive, false positive, true negative, and false negative of a confusion matrix.Figure 5
Confusion matrix.


### 5.1. Statistical Measures

Accuracy: accuracy is the ability to detect the target class and the output class correctly, that is, to numerate the fraction of true positive and true negative in all the evaluated classes.(1)Accuracy=TP+TNTP+TN+FP+FN.Specificity: specificity refers to the ability to determine the target class correctly by the predicted class, that is, to calculate the proportion of true positive.(2)Specificity=TNTN+FP.Sensitivity: sensitivity refers to the ability to determine the proportion of true negative, that is, the fraction of true positive classes in all the positive assessments.(3)Sensitivity=TPTP+FN.Precision: precision is defined as the fraction of positive predictions in all the predicted positive classes.(4)Precision=TPTP+FP.F1 score: F1 score is the harmonic mean of precision and sensitivity and is a better measure than accuracy. A high value of F1 score determines that the neural network has better performance on positive classes.(5)F1score=2∗Precision×SensitivityPrecision+Sensitivity.Compared to accuracy performance metrics, statistical measures like precision, sensitivity, specificity, and F1 score provide better insights into the prediction accuracy of neural networks.


## 5.1. Statistical Measures

Accuracy: accuracy is the ability to detect the target class and the output class correctly, that is, to numerate the fraction of true positive and true negative in all the evaluated classes.(1)Accuracy=TP+TNTP+TN+FP+FN.Specificity: specificity refers to the ability to determine the target class correctly by the predicted class, that is, to calculate the proportion of true positive.(2)Specificity=TNTN+FP.Sensitivity: sensitivity refers to the ability to determine the proportion of true negative, that is, the fraction of true positive classes in all the positive assessments.(3)Sensitivity=TPTP+FN.Precision: precision is defined as the fraction of positive predictions in all the predicted positive classes.(4)Precision=TPTP+FP.F1 score: F1 score is the harmonic mean of precision and sensitivity and is a better measure than accuracy. A high value of F1 score determines that the neural network has better performance on positive classes.(5)F1score=2∗Precision×SensitivityPrecision+Sensitivity.Compared to accuracy performance metrics, statistical measures like precision, sensitivity, specificity, and F1 score provide better insights into the prediction accuracy of neural networks.


## 6. Experimental Results

The following sections discusses in detail about data collection and clustering of environmental faults in solar panels and provide a comparative analysis of the trained and tested fault images in various neural networks. Figure6 represents the various steps involved in the collection of fault images from solar panels.Figure 6
Methodology for data collection and clustering.


### 6.1. Data Collection and Clustering

A total of 1197 real images of solar panels for different fault conditions were captured. For training of various neural networks, the images were further divided in the percent ratio of 70 and 30 for training and testing of solar panel images. Table1 shows the number of images for each training class of real images. The total numbers of images for testing and training are 313 and 884, respectively.Table 1
Number of real images used in each training class.
S. no.Training classImage count1Normal panel1562Partially cement covered1573Fully cement covered1204Partially snow covered1445Fully snow covered1636Mud covered144Similarly, a total of 872 thermal images for various faults were captured from the infrared camera and divided in the percent ratio of 70 and 30 for training and testing, respectively. This clustering of images helps in proper and efficient training of neural networks. Table2 shows the number of images for each training class of thermal images. The total numbers of images for testing and training are 272 and 610, respectively.Table 2
Number of thermal images used in each training class.
S. no.Training classImage count1Normal panel1002Partially shaded1003Fully shaded1084Partially soil covered1025Completely soil covered1006Temperature rise100


### 6.2. Neural Network Analysis

Real images collected are trained in Deep Network Designer application in MATLAB software for various pretrained networks: SN, AN, and GN. For the proper fitting of the trained model, training options are set as shown in Table3.Table 3
Training options employed for the pretrained networks.
S. no.Training optionValue1Solver∗SGDM2Maximum epochs33Mini batch size84Max number of iterations2315Base learning rate0.0001∗Stochastic gradient descent with momentum.The execution environment employed for training is CPU. Table4 shows the intermediate results obtained from the training of real images in SN, AN, and GN. Figures 7–9 show the training progress of the various pretrained networks where the blue and red lines specify the training accuracy and loss while the black dotted line specifies the validation accuracy and loss of the trained model.Table 4
Intermediate results for SqueezeNet, AlexNet, and GoogleNet.
Type of networkEpochIterationTime elapsed (hh:mm:ss)Validation accuracyTraining accuracyValidation lossTraining lossSqueezeNet1100 : 00 : 2617.358518.953.31083.298727500 : 07 : 0792.37691.93450.17510.06371315500 : 14 : 101001000.04360.006683 (end)23100 : 20 : 441001000.02670.01164AlexNet1100 : 00 : 1813.584912.53.71244.531427500 : 04 : 1194.578399.8750.000214.575e-05315500 : 09 : 351001006.9009e-052.644e-053 (end)23100 : 14 : 161001009.0214e-050.0008GoogleNet1100 : 00 : 2818.490519.38342.639633.175127500 : 08 : 0796.981192.47640.137590.0445315500 : 14 : 5010099.15280.023650.0053 (end)23100 : 23 : 001001000.01240.0026Figure 7
Training progress of finely tuned SqueezeNet.Figure 8
Training progress of finely tuned AlexNet.Figure 9
Training progress of finely tuned GoogleNet.From the following figures, it can be seen that the dataset is finely trained by AN as it attains a training accuracy of 99.8% and 100% at epochs 2 and 3 at a faster rate of 14 min 47 s while SN takes a time of 21 min 15 s to train and attains a training accuracy of only 91.9% and 100% at corresponding epochs 2 and 3. Compared to the other two neural networks, GN takes a longer time of 23 min 31 s to reach a training accuracy of 92.48% and 99.15% at epochs 2 and 3, respectively. Hence, AN is more suitable for training solar panels for the fault classification of solar panels due to better training accuracy and shorter training time followed by SN next.A set of 100 images are selected from the testing images. The confusion matrix for these samples is plotted, and corresponding statistical measures such as accuracy, sensitivity, specificity, precision, and F1 score are calculated for SN, AN, and GN are given in the following. Figures10–12 are the confusion matrices of SN, AN, and GN while Figures 13–15 are the statistical measures of SN, AN, and GN.Figure 10
Confusion matrix for SqueezeNet.Figure 11
Confusion matrix for AlexNet.Figure 12
Confusion matrix for GoogleNet.Figure 13
Statistical measures for SqueezeNet.Figure 14
Statistical measures for AlexNet.Figure 15
Statistical measure for GoogleNet.Figures16–18 show the solar panel faults classified by various neural networks along with their testing accuracy for twenty testing images.Figure 16
Fault classified by SqueezeNet with testing accuracy.Figure 17
Fault classified by AlexNet with testing accuracy.Figure 18
Fault classified by GoogleNet with testing accuracy.Table5 shows the average of testing accuracy, specificity, sensitivity, precision and F1 score. For the selection of an appropriate neural network for fault to provide better acuity in prediction accuracy, it is necessary for the various statistical parameters such as testing accuracy, specificity, sensitivity and precision to complement each other and provide a good F1 score. From the three neural networks tested, SN provides a remarkable testing accuracy of 99.815% and a F1 score of 0.992. On the other hand, AN despite having better training accuracy, the testing accuracy of the network is 94.975 and F1 score is 0.9559 which is comparatively low compared to the other two networks as GN itself provides a testing accuracy of 92.285% and F1 score of 0.9832. Similarly, the order of specificity, sensitivity and precision results were obtained higher for SN as the corresponding values were 0.9978, 0.9907 and 0.9936 and the lowest values were obtained for AN as the values were 0.9897, 0.9515 and 0.9664. For GN, specificity, sensitivity and precision results were 0.9977, 0.975 and 0.9936. It can be seen that the specificity and precision of SN and GN are nearly equal. However, the F1 score and sensitivity of SN has an edge over GN.Table 5
Comparative measures of SN, GN, and AN.
Network typeTesting accuracy of imageAccuracy of predictionSpecificitySensitivityPrecisionF1 scoreSN99.820.9970.9980.9910.9940.992AN92.290.9830.9900.9520.9660.956GN94.980.9930.9980.980.9940.983Table6 represents the comparative results of different neural networks based on the number of layers, the number of images which can be processed, and their image size. Furthermore, SN requires a low memory size of 4.6 MB with 18 layers that can process a total of 1.24 million images. GN with a memory size of 27 MB can process up to 7 million images in 22 layers. AN can process up to 61 million images, and it requires a memory size of 227 MB and has only 8 layers.Table 6
Parameters of various neural networks.
Network TypeMemory size (MB)Image SizeImage capacity (Million)Layers countSN4.6227 × 2271.2418AN227227 × 227618GN27224 × 224722Based on factors such as testing and training accuracy, statistical measures, training time, and memory requirement, it can be seen that SN is efficient for the fault classification of solar PV panels with precision. Hence, a set of thermal images were trained in SN. Figure19 shows the training progress of thermal images of solar panels with faults. Table 7 indicates that the time required for training and validating 610 images is 11 min 53 s where the training accuracy attains a training accuracy of 99.861 and 100 at the end of epochs 2 and 3, respectively. Figures 20 and 21 present the classification of PV faults in SN for real and thermal images.Figure 19
Training progress of SqueezeNet for fault classification of thermal images of solar panels.Table 7
Intermediate results of thermal images trained in SN.
EpochIterationTime elapsed (hh:mm:ss)Validation accuracyTraining accuracyValidation lossTraining loss1100 : 00 : 10004.57395.012825400 : 04 : 5699.86197.6980.06750.1597310500::08 : 291001000.002360.0011315900 : 11 : 181001000.000540.00085Figure 20
Fault classified by SqueezeNet for real images of solar panels.Figure 21
Fault classified by SN for thermal images of solar panels.The confusion matrices obtained for a sample of hundred real and thermal images are shown in Figures22 and 23. From the confusion matrices, it can be seen that for real images, the desired class matches with the output class giving 100% accuracy while the confusion matrix for thermal images gives an accuracy of 98%. Table 8 presents the statistical measures of thermal images trained in SN where the neural network trained has an F1 score of 0.9818.Figure 22
Confusion matrix for real images with fault in solar panels.Figure 23
Confusion matrix for thermal images with fault.Table 8
Statistics of thermal images in SN.
ClassAccuracySpecificitySensitivityPrecisionF1 scoreNormal panel0.980.97110.9380.968Completely soil covered11111Partially soil covered11111Temperature rise0.9810.85710.923Fully shaded11111Partially shaded11111Average0.9930.9950.9760.9900.982Table9 depicts that the testing accuracy of thermal images is 99.74% as the testing accuracy of each image tested is above 0.95 while for real images the testing accuracy is comparatively low, that is, 94.42%. However, the prediction accuracy of real images by the pretrained SN is far better as the accuracy is 100% while for thermal images the accuracy is 98%. The decrease in the prediction accuracy of thermal images is due to the hotspots in the panels, which may arise due to the temperature rise in addition to the actual fault in the panel.Table 9
Prediction and testing accuracy of real and thermal images.
ParameterPrediction accuracy (%)Testing accuracy of image (%)Real image10094.42Thermal image9899.74


## 6.1. Data Collection and Clustering

A total of 1197 real images of solar panels for different fault conditions were captured. For training of various neural networks, the images were further divided in the percent ratio of 70 and 30 for training and testing of solar panel images. Table1 shows the number of images for each training class of real images. The total numbers of images for testing and training are 313 and 884, respectively.Table 1
Number of real images used in each training class.
S. no.Training classImage count1Normal panel1562Partially cement covered1573Fully cement covered1204Partially snow covered1445Fully snow covered1636Mud covered144Similarly, a total of 872 thermal images for various faults were captured from the infrared camera and divided in the percent ratio of 70 and 30 for training and testing, respectively. This clustering of images helps in proper and efficient training of neural networks. Table2 shows the number of images for each training class of thermal images. The total numbers of images for testing and training are 272 and 610, respectively.Table 2
Number of thermal images used in each training class.
S. no.Training classImage count1Normal panel1002Partially shaded1003Fully shaded1084Partially soil covered1025Completely soil covered1006Temperature rise100


## 6.2. Neural Network Analysis

Real images collected are trained in Deep Network Designer application in MATLAB software for various pretrained networks: SN, AN, and GN. For the proper fitting of the trained model, training options are set as shown in Table3.Table 3
Training options employed for the pretrained networks.
S. no.Training optionValue1Solver∗SGDM2Maximum epochs33Mini batch size84Max number of iterations2315Base learning rate0.0001∗Stochastic gradient descent with momentum.The execution environment employed for training is CPU. Table4 shows the intermediate results obtained from the training of real images in SN, AN, and GN. Figures 7–9 show the training progress of the various pretrained networks where the blue and red lines specify the training accuracy and loss while the black dotted line specifies the validation accuracy and loss of the trained model.Table 4
Intermediate results for SqueezeNet, AlexNet, and GoogleNet.
Type of networkEpochIterationTime elapsed (hh:mm:ss)Validation accuracyTraining accuracyValidation lossTraining lossSqueezeNet1100 : 00 : 2617.358518.953.31083.298727500 : 07 : 0792.37691.93450.17510.06371315500 : 14 : 101001000.04360.006683 (end)23100 : 20 : 441001000.02670.01164AlexNet1100 : 00 : 1813.584912.53.71244.531427500 : 04 : 1194.578399.8750.000214.575e-05315500 : 09 : 351001006.9009e-052.644e-053 (end)23100 : 14 : 161001009.0214e-050.0008GoogleNet1100 : 00 : 2818.490519.38342.639633.175127500 : 08 : 0796.981192.47640.137590.0445315500 : 14 : 5010099.15280.023650.0053 (end)23100 : 23 : 001001000.01240.0026Figure 7
Training progress of finely tuned SqueezeNet.Figure 8
Training progress of finely tuned AlexNet.Figure 9
Training progress of finely tuned GoogleNet.From the following figures, it can be seen that the dataset is finely trained by AN as it attains a training accuracy of 99.8% and 100% at epochs 2 and 3 at a faster rate of 14 min 47 s while SN takes a time of 21 min 15 s to train and attains a training accuracy of only 91.9% and 100% at corresponding epochs 2 and 3. Compared to the other two neural networks, GN takes a longer time of 23 min 31 s to reach a training accuracy of 92.48% and 99.15% at epochs 2 and 3, respectively. Hence, AN is more suitable for training solar panels for the fault classification of solar panels due to better training accuracy and shorter training time followed by SN next.A set of 100 images are selected from the testing images. The confusion matrix for these samples is plotted, and corresponding statistical measures such as accuracy, sensitivity, specificity, precision, and F1 score are calculated for SN, AN, and GN are given in the following. Figures10–12 are the confusion matrices of SN, AN, and GN while Figures 13–15 are the statistical measures of SN, AN, and GN.Figure 10
Confusion matrix for SqueezeNet.Figure 11
Confusion matrix for AlexNet.Figure 12
Confusion matrix for GoogleNet.Figure 13
Statistical measures for SqueezeNet.Figure 14
Statistical measures for AlexNet.Figure 15
Statistical measure for GoogleNet.Figures16–18 show the solar panel faults classified by various neural networks along with their testing accuracy for twenty testing images.Figure 16
Fault classified by SqueezeNet with testing accuracy.Figure 17
Fault classified by AlexNet with testing accuracy.Figure 18
Fault classified by GoogleNet with testing accuracy.Table5 shows the average of testing accuracy, specificity, sensitivity, precision and F1 score. For the selection of an appropriate neural network for fault to provide better acuity in prediction accuracy, it is necessary for the various statistical parameters such as testing accuracy, specificity, sensitivity and precision to complement each other and provide a good F1 score. From the three neural networks tested, SN provides a remarkable testing accuracy of 99.815% and a F1 score of 0.992. On the other hand, AN despite having better training accuracy, the testing accuracy of the network is 94.975 and F1 score is 0.9559 which is comparatively low compared to the other two networks as GN itself provides a testing accuracy of 92.285% and F1 score of 0.9832. Similarly, the order of specificity, sensitivity and precision results were obtained higher for SN as the corresponding values were 0.9978, 0.9907 and 0.9936 and the lowest values were obtained for AN as the values were 0.9897, 0.9515 and 0.9664. For GN, specificity, sensitivity and precision results were 0.9977, 0.975 and 0.9936. It can be seen that the specificity and precision of SN and GN are nearly equal. However, the F1 score and sensitivity of SN has an edge over GN.Table 5
Comparative measures of SN, GN, and AN.
Network typeTesting accuracy of imageAccuracy of predictionSpecificitySensitivityPrecisionF1 scoreSN99.820.9970.9980.9910.9940.992AN92.290.9830.9900.9520.9660.956GN94.980.9930.9980.980.9940.983Table6 represents the comparative results of different neural networks based on the number of layers, the number of images which can be processed, and their image size. Furthermore, SN requires a low memory size of 4.6 MB with 18 layers that can process a total of 1.24 million images. GN with a memory size of 27 MB can process up to 7 million images in 22 layers. AN can process up to 61 million images, and it requires a memory size of 227 MB and has only 8 layers.Table 6
Parameters of various neural networks.
Network TypeMemory size (MB)Image SizeImage capacity (Million)Layers countSN4.6227 × 2271.2418AN227227 × 227618GN27224 × 224722Based on factors such as testing and training accuracy, statistical measures, training time, and memory requirement, it can be seen that SN is efficient for the fault classification of solar PV panels with precision. Hence, a set of thermal images were trained in SN. Figure19 shows the training progress of thermal images of solar panels with faults. Table 7 indicates that the time required for training and validating 610 images is 11 min 53 s where the training accuracy attains a training accuracy of 99.861 and 100 at the end of epochs 2 and 3, respectively. Figures 20 and 21 present the classification of PV faults in SN for real and thermal images.Figure 19
Training progress of SqueezeNet for fault classification of thermal images of solar panels.Table 7
Intermediate results of thermal images trained in SN.
EpochIterationTime elapsed (hh:mm:ss)Validation accuracyTraining accuracyValidation lossTraining loss1100 : 00 : 10004.57395.012825400 : 04 : 5699.86197.6980.06750.1597310500::08 : 291001000.002360.0011315900 : 11 : 181001000.000540.00085Figure 20
Fault classified by SqueezeNet for real images of solar panels.Figure 21
Fault classified by SN for thermal images of solar panels.The confusion matrices obtained for a sample of hundred real and thermal images are shown in Figures22 and 23. From the confusion matrices, it can be seen that for real images, the desired class matches with the output class giving 100% accuracy while the confusion matrix for thermal images gives an accuracy of 98%. Table 8 presents the statistical measures of thermal images trained in SN where the neural network trained has an F1 score of 0.9818.Figure 22
Confusion matrix for real images with fault in solar panels.Figure 23
Confusion matrix for thermal images with fault.Table 8
Statistics of thermal images in SN.
ClassAccuracySpecificitySensitivityPrecisionF1 scoreNormal panel0.980.97110.9380.968Completely soil covered11111Partially soil covered11111Temperature rise0.9810.85710.923Fully shaded11111Partially shaded11111Average0.9930.9950.9760.9900.982Table9 depicts that the testing accuracy of thermal images is 99.74% as the testing accuracy of each image tested is above 0.95 while for real images the testing accuracy is comparatively low, that is, 94.42%. However, the prediction accuracy of real images by the pretrained SN is far better as the accuracy is 100% while for thermal images the accuracy is 98%. The decrease in the prediction accuracy of thermal images is due to the hotspots in the panels, which may arise due to the temperature rise in addition to the actual fault in the panel.Table 9
Prediction and testing accuracy of real and thermal images.
ParameterPrediction accuracy (%)Testing accuracy of image (%)Real image10094.42Thermal image9899.74


## 7. Limitations and Scope for Future Work

For efficient training of neural networks, a large number of thermal images of solar panels are required. This also increases the training time which may extend from 1 hour to 2.5 hours and requires a better execution environment, most preferably Graphical Processing Unit instead of CPU. In the future, the average testing accuracy of SN for thermal images can be increased by altering the number of layers in the neural network. Also, a mobile application can be created that executes the SN model for automatic classification of faults in solar panels, so users with scarce or no knowledge can employ the application to detect faults in solar panels effectively.


## 8. Conclusion

This research proposed a deep learning technique by comparing CNN pretrained models and fine-tuning them for the diagnosis of environmental faults in solar panels. The insight of this research focused on comparing the performance of AlexNet, GoogleNet, and SqueezeNet with different performance metrics and finding a suitable model for fault classification. The three models used in this research were capable of classifying five faults in solar panels from the productive class, in which the SqueezeNet model comprising 18 layers reached a testing accuracy of 99.815%. On the other hand, AlexNet despite its significant training accuracy obtained the lowest performance testing accuracy of 94.975% compared to the other architecture. Hence, the SqueezeNet model was used for training thermal images of solar panels which provided a testing accuracy of 99.74% and F1 score of 0.9818. The proposed method looks forward to make a significant contribution to the solar industry.



---

*Source: 2872925-2022-09-22.xml*