# Deep Learning-Powered System for Real-Time Digital Meter Reading on Edge Devices
## 2 Materials and Methods
### Automatic Meter Reading Pipeline
#### 2.2.1 Screen Detection  
The first module of the system receives the mobile-acquired images, crops a central square with a length equal to 90% of the shortest side, and detects the equipment's screen, enabling it to be extracted and projected onto a normal plane, as described above. The chosen approach is firstly based on a model that roughly detects a bounding box containing the screen. The detection is only accepted if the following requirements are satisfied: the  
Figure 3: Pipeline of the automatic digital meter reading system.  
Figure 2: Examples of input images of each step of the proposed pipeline: (**a**) original input image, (**b**) central cropped images that will serve as input for screen detection, (**c**) perspective corrected image after keypoint detection, (**d**) text regions obtained after text detection and template matching.  
confidence score is higher than 70%, the bounding box is not located on the corners of the image, and its ratio is within the range \(AvgRatio\pm 3 StdRatio\). If all conditions are met, a tolerance margin is added to this detected screen to face possible inaccuracies, and then a keypoint detection model is used to locate the corners of the screen.  
The model for screen bounding box detection was built with the TensorFlow Object Detection API. This tool provides several models with different architectures pre-trained on public datasets. As we aim to develop a model suitable for integration in low-end devices, the SSD MobileNetv2 [9] model pre-trained on the COCO [10] dataset is chosen due to the trade-off between the number of parameters, speed and performance. This model is trained on this work's dataset, returning the detection boxes and corresponding confidence scores. The input images are automatically resized to 320 \(\times\) 320.  
For keypoint detection, we implement a model based on the same backbone (MobileNetV2), from which image features are extracted and connected to four convolutional layers which calculate a regression that fits the coordinates of the desired points. Randomness is provoked in the position of the screen in order to introduce variability. We fine-tune the model by freezing the backbone layers and only training the regression layers. This model takes images sized \(224\times 224\) as input and returns the \(x\) and \(y\) coordinates of the screen corners.  
In non-ideal conditions of acquisition, the screen can assume distinct quadrilateral shapes (for example trapeziums). To overcome this, a perspective correction (perspective warp) methodology is employed to ensure that the corrected screen image maintains the graphical structure of the screen content, as the position of the text is crucial to understanding its context. Our approach uses OpenCV's getPerspectiveTransform function to obtain a 3 \(\times\) 3 matrix of a perspective transform so that the detected screen corners are transformed into the corresponding expected points in the destination image. Then, OpenCV's warp-Perspective applies the perspective transformation to the resulting image of the screen bounding box module.