# Cloud Detection and Tracking Based on Object Detection with Convolutional Neural Networks
## 2 Methodology  
This paper proposes a method based on low-cost cameras and computer vision techniques, specifically a CNN-based object detection technique, to estimate the available solar energy. The method works as follows: First, a low-cost camera takes a picture of the sky or the Sun, as shown in Figure 1, and projects the positions of the objects in the sky onto the camera plane. The image can cover the whole sky or only the sun region, such as the image used for Hel-IoT. The camera was set to full auto mode with a maximum resolution, allowing it to adjust parameters like exposure time according to the lighting conditions. Alternatively, other camera settings such as reducing the exposure time to prevent pixel saturation near the Sun can be used, but they require further investigation. The next step consists in the development of a machine learning (ML) model, capable of localizing and identifying multiple objects in a image using a modern CNN [21], in other words, detecting the Sun and cloud positions in the input image of the model. At this point, an algorithm developed for this work is used to analyze the previous results and assign an identifier number to each detected object according to the identifier number of the closest object of the same class in the previous images of the series of images. Then, the algorithm compares the results with previous results to compute the cloud and sun movement vectors  
Figure 1: Solar tower system and image taken by a camera located in a heliostat. (the white and green lines, respectively, in Figure 2). With the position and the movement vectors (speed and direction), the transient remaining time (RT in Figure 2 for each detected cloud) can be computed as the ratio between the distance in pixels to the Sun and the average of the last five speeds measured for each detected cloud. In Figure 2, in addition to RT and the movement vector, each object detected is also shown according to its class (the colored box), the confidence level of each detection, the id number, and the position of each detected cloud. In this way, a prediction of solar radiation cloud blocking events above the point where the camera is located can be obtained. Several cameras deployed in a region allow the algorithm to infer how the transient will affect each of the areas of the region in which the sensors are installed. For image acquisition and data processing, the presented methodology uses an open, low-cost hardware platform from the Raspberry Pi Foundation, a Raspberry Pi 4 with 4 Gb of RAM and a Picam, together with a fish eye lens with 8 Mpx of resolution, for a total hardware cost of less than EUR 100. The low-cost camera must be located close to the system that will use the cloud movement information, for example, in the case of Hel-IoT, it can be incorporated together with the smart solar tracking system [19; 20]. In others cases or other systems, such as PV systems or existing CSP systems, the proposed system can be located in a representative area of the field. Note that since this new approach can be implemented with low-cost hardware, a network of systems can be distributed over very large solar fields without incurring great cost. In this study, the low-cost camera was installed on a tripod in the northern part of the CESA field. The CESA field is one of the SCR systems at the Plataforma Solar de Almeria (PSA) and is composed of 300 heliostats and an 80-meter high tower. The PSA [22], belonging to the Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas (CIEMAT), is the largest research, development, and testing center dedicated to concentrated solar technologies. Figure 2: Cloud detection results. White, blue, and green boxes to show cloud, heliostat, and sun detection, respectively. The software was developed in Python 3.10 [23], using TensorFlow [24] and the object detection API [21]. TensorFlow is an open source platform for machine learning that lets researchers push the state of the art in ML by building and deploying ML-powered applications.