# Time Classification Algorithm Based on Windowed-Color Histogram Matching
###### Abstract  
A web-based search system recommends and gives results such as customized image or video contents using information such as user interests, search time, and place. Time information extracted from images can be used as a important metadata in the web search system. We present an efficient algorithm to classify time period into day, dawn, and night when the input is a single image with a sky region. We employ the Mask R-CNN to extract a sky region. Based on the extracted sky region, reference color histograms are generated, which can be considered as the ground-truth. To compare the histograms effectively, we design the windowed-color histograms (for RGB bands) to compare each time period from the sky region of the reference data with one of the input images. Also, we use a weighting approach to reflect a more separable feature on the windowed-color histogram. With the proposed windowed-color histogram, we verify about 91% of the recognition accuracy in the test data. Compared with the existing deep neural network models, we verify that the proposed algorithm achieves better performance in the test dataset. time classification; sky region; windowed-color histogram; weighting approach +
Footnote †: journal: Applied Sciences  
**Citation:** [PERSON]; [PERSON]; [PERSON] Time Classification Algorithm Based on Windowed-Color Histogram Matching. _Appl. Sci. 2021_, _II_, 11997. [[https://doi.org/10.3390/appl112411997](https://doi.org/10.3390/appl112411997)]([https://doi.org/10.3390/appl112411997](https://doi.org/10.3390/appl112411997))  
what to do (action), with whom (relationship with other) and so on should be extracted from the image and video data [16; 17; 18]. In order to obtain high-level abstracted information, we focus on extracting time information from images. It can be used as important metadata in the search or recommendation system. To extract it from image data, we paid attention to the amount of sunlight because it causes the change of time during the day--dawn, morning, midday, evening, and night. In this aspect, the sky region represents the change of time in outdoor images well and can provide useful information about the environment, especially for the given time. Therefore, by extracting sky area and analyzing its characteristics, it is able to derive meaningful time information during the day. In this paper, we propose an efficient algorithm to classify time by analyzing the sky region with deep learning and color histogram. The proposed algorithm is trained by Mask R-CNN [19] to extract the sky region from the given image. Based on the extracted region, the reference color histograms are generated, which can be considered as the ground-truth. Time is classified by comparing the reference color histogram with a histogram of the sky region extracted from test dataset. We design the windowed-color histograms (for RGB bands) to compare each time period with the reference and input efficiently. In addition, we propose a weighted histogram representation which compares histograms by assigning large weights according to the degree of color importance. The main contribution of this study is the followings:  
* We focus on the sky area of the image because it has more information on light and discriminating feature;
* The designed weighted histogram comparison can pay attention on more important colors in each class;
* As a result, we simplify the problem and improve the time classification performance better than existing deep learning models. This study can have a wide range of applications in the real world. If time information is used, a web-based search system can categorize results into dawn, dusk, day, or night. In addition, it can recommend contents based on the related time of queries. We aim to provide more accurate search and recommendation results by using time information of image or video. The remaining part of this paper is organized as follows: In Section 2, we analyze some existing methods as the related works.