# Detection of Precipitation and Fog Using Machine Learning on Backscatter Data from Lidar Ceilometer
## 4 Weather Detection
### Under-Sampling  
The number of absent examples was much higher than that of present examples in the training data. Such highly imbalanced data can hinder the training process of machine learning algorithms, making the resulting prediction model rarely predict the present examples. Therefore, we used under-sampling to balance the training data as in [12; 31]. The training data comprised data from the first day to the 15 th day of each month, and the validation data were composed of data ranging from the 16 th to the last day of each month. Random forest was used to find the optimal under-sampling ratio by varying the presence to absence ratio from 1:1 to 1:7. Note that under-sampling is applied only to the training data. To validate the results, we calculated and compared the accuracy, precision, false alarm rate (FAR), recall (or probability of detection; POD), and F1 score, which are measures that are frequently used in machine learning and meteorology (see Table 10) [32]. Accuracy is the probability that the observed value will coincide with the predicted value among all data (\((a+d)/n\)). In precipitation detection, precision is the probability that the predicted precipitation is correct (\(a/(a+b)\)); the FAR is the number of false alarms over the total number of alarms or predicted precipitation samples (\(b/(a+b)\)), and recall (or POD) is the fraction of the total amount of precipitation occurrences that were correctly predicted (\(a/(a+c)\)). In fog detection, precision is the probability that the predicted fog is correct (\(a/(a+b)\)), the FAR is the number of false alarms over the total number of alarms or predicted fog samples (\(b/(a+b)\)), and recall (or POD) is the fraction of the total amount of fog occurrences that were correctly predicted (\(a/(a+c)\)). F1 score is an index that measures the accuracy of validation, and the harmonic mean of precision and recall (\(2\times(precision\times recall)/(precision+recall)\)). In imbalanced classification, accuracy can be a misleading metric. Therefore, the F1 score, which considers both precision and recall, is widely used as a major assessment criterion [33; 34]. \begin{table}
\begin{tabular}{c l l} \hline \hline
**Field Information** & & **Description or Value** \\ \hline \multirow{4}{*}{Hourly AWS data} & 1 & Month \\  & 2 & Day \\  & 3 & Backscatter coefficients \\  & 3 & on heights 10 m, 20 m, \(\dots\), 4500 m \\  & 4 & Presence of weather phenomenon (0 or 1) \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Field information on train data for weather phenomenon detection. In Tables 11 and 12, F1 score is the highest when the under-sampling ratio is 1:2. When the under-sampling ratio is 1:7, accuracy is high, but precision is very low. A high precision is good considering the accuracy in the case of precipitation or visibility sensor phenomenon. However, if precision is high, there is a tendency to only overestimate the corresponding phenomenon. Therefore, we selected the case of the highest F1 score whereby precision and recall were balanced. In other words, we under-sampled the precipitation and fog (low visibility) at the under-sampling ratio of 1:2. We also compared our results with two versions of random prediction (see the two bottom rows of Tables 11 and 12): one method called \"Rand\" evenly predicts the presence or absence of weather phenomenon at random, and the other method called \"W-rand\" randomly predicts the presence or absence of weather phenomenon with the weights according to the probability of actual observation of weather phenomenon. We could clearly see that random forest with under-sampling is superior to random prediction with respect to F1 score. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Predicted**} & \multicolumn{3}{c}{**Observed (OBS)**} \\ \cline{2-4}  & **Yes** & **No** & **Total** \\ \hline Yes & \(a\) & \(b\) & \(a+b\) \\ No & c & d & \(c+d\) \\ Total & \(a+c\) & \(b+d\) & \(a+b+c+d=n\) \\ \hline \hline \end{tabular}
\end{table}