What is My Dog doing Now ? Modeling Dog Behavior from Live Stream

Posture and motion detection for human have been a fairly popular academic topic recently due to its wide application in downstream tasks. However, little has been done towards our fur friends/family members. In this project, we are going to explore the videos streams from a dog camera, applying computation vision techniques and trying to better understand of their behaviors when the owner is away from home. This feature would also turn our cameras into the smart ones that could alert abnormal behavior of pets. Motivation Lab experiments (Lucas, Powell, and Murphree 1977; Wauquier et al. 1979; Copley 1976) have shown that a typical dog sleeps 14 hours a day but the mean of sleep-wake cycle is about 83 min. When the owner is away from home, the dogs change their behavior and are switching between alert and drowsy even more frequently. While nowadays’ dog cameras provide a live perception of our fur friends, the product is not completed with crucial component missing: the lacks of semantic interpretation makes the owner upset and very hard to respond immediately unless they put up the live stream all the time. For example, when dogs are chewing a chocolate or jumping out of the play pen, it is better to notify the owner immediately rather than recording the video in the backend. It has been a couple of years since researcher applying deep neural net for human pose detection (Cao et al. 2017; Toshev and Szegedy 2014), but not until recently dog or animal focused projects (Ehsani et al. 2018) has incorporated and leveraged the prior knowledge for them. In this project, we try to close the gap by modeling dog postures as well as motions using image processing technologies. Dataset We are going to create a dataset by ourselves. We have one 11 month Shiba Dog, Keke, for this experiment. Keke stays in its play pen most time of the day and the entire area is covered by a YI Dome Camera 1080p HD camera1 24/7 with an FPS of 15. Sample images as the following2: For detailed specification, please see the appendix. We might change the angle and setup to accommodate different use case later. Figure 1: Sample image from the current camera setting; Keke is sleeping Task definition and scope Goals This project will focus on the detection and analysis of the dog postures and motions based on dog camera video streams. More advanced features including alerting the owner when abnormal behaviour of the dog is observed will be included depending on the project progress. Milestones 1. Collect images: Image will be sampled from the frames of 7 days (9072000 = 7 × 24 × 3600 × 15). Image processing techniques will be used in sampling to get more representative data. 2. Label data: Images need to be annotated with dog postures, as well as status, e.g. sleeping, standing and eating. 3. Detection: We will start the experiment with fixed camera angle/position. After detecting the dog position in the image, the part of image with dog will be extracted for further analysis. 4. Classification: Based on the detection result, we will try to classify the status of the dog. Analytical result will be summarized 5. Generalization: We will carry out similar experience with a different setting of cameras, background and maybe dog if we have enough time. 6. Deployment: We would like to engineer the speed and reliability of the algorithm and deploy it on PC for live demo.