# CNN-Based Crosswalk Pedestrian Situation Recognition System Using Mask-R-CNN and CDA
## 2 Related Works  
The first object detection network using deep learning was the R-CNN [20], announced in 2014. Object detection since developed into networks such as Fast R-CNN [21] and Faster R-CNN [22]. Mask R-CNN is a network that adds a fully convolutional network (FCN) based on Faster R-CNN. It consists of two stages, and the first is the region proposal network (RPN) which is a stage for extracting the object's location. The second stage is a parallel prediction of the binary mask, box offset, and class for each region of instance (RoI). Mask R-CNN added an FCN to the Faster R-CNN and applied FCN to the RoI extracted by the RPN. Therefore, it became possible to predict an M*M-sized mask without losing information. Faster R-CNN cannot split instances on a pixel basis, whereas Mask R-CNN does so. Furthermore, Mask R-CNN is capable of efficient computation, surpassing the performance of existing state-of-the-art (SOTA) networks. [PERSON] et al. [23] proposed and evaluated a dynamic passive pedestrian detection system (DPPD) over crosswalks using optical and thermal sensors. Their image analysis system with sensors determined the pedestrian's location in real time. Consequentially, an average accuracy of 89% and a standard deviation of 10% were exhibited in determining the location of pedestrians using thermal sensors. In addition, 82% average accuracy and a standard deviation of 8% were exhibited in establishing the location of pedestrians using optical sensors. Their system can also detect atypical pedestrians. This expression refers to unstructured pedestrians, such as pedestrians pulling strollers and those using umbrellas. Thus, [PERSON] et al. enabled further system improvement by creating and evaluating two new systems. In 2020, [PERSON] et al. [24] proposed a system for judging pedestrian traffic laws. They used the LSTM (Long Short Term Memory) neural network to predict pedestrian behavior. In their paper, the situation of the crosswalk was judged by predicting the pedestrian's behavior when a red light was displayed. They used nine characteristics, such as gender, walking direction, group behavior, etc. Their proposed system applied deep learning to predict pedestrians' unexpected behavior on crosswalks, consequentially exhibiting 91.6% accuracy. Their system allows for the prevention of collisions between pedestrians and vehicles on crosswalks. Other works that detect pedestrians include studies by [PERSON] et al. [25], [PERSON] et al. [26], [PERSON] and [PERSON] [27], [PERSON] and [PERSON] [28], and [PERSON] et al. [29]. [PERSON] et al. 's system distinguished whether pedestrians were on the road using the cascade classifier and histogram-of-oriented-gradient (HOG). [PERSON] and [PERSON] developed an algorithm that extracted edges by Hough line detection and color-based extraction, recognized crosswalks, and detected whether there were pedestrians in crosswalk areas using three classifiers. However, they did not apply deep learning in their system. Furthermore, [PERSON] et al. [30] applied YOLO to detect pedestrians, and [PERSON] et al. [31] used YOLOv5 for crosswalk detection. Using Mask R-CNN, [PERSON] [32] suggested pedestrian crosswalk detection. However, [PERSON] did not investigate the relationship between pedestrians and crosswalks. Thus far, the research focused on simple image analysis or prediction of pedestrians' positions within the crosswalk using devices such as thermal and optical sensors. Additionally, detection using YOLO has the disadvantage of not accurately representing the locations of pedestrians and crosswalks in pixels. Thus, we tested a new method using Mask R-CNN and the CDA. Our trained CNN enabled us to classify danger and safety based on the location of pedestrians and crosswalks.