Saliency based 360° Video Contents Encoding for Streaming Service (poster)

Video streaming service has been essential to the Internet ecosystem since a majority of Internet contents is consumed via streaming ser-vices, such as Netflix and Youtube [3]. Moreover, contents providers started to upload 4K and even 8K videos on streaming service to satisfy users' demand for higher quality of streaming service in accordance with the recent refinement on display technology. However, the available bandwidth in most of the developed countries can barely support single full HD contents streaming service [1]. Such bandwidth shortage intensifies in 360 video streaming service even at a larger scale, as 360 video contents providers started to upload 4K or 8K videos at the higher frame rate(60FPS or 120FPS). No current ISP (Internet service provider) can sustain the bandwidth needed for such scale of video contents [1]. To overcome the aforementioned network limitations, many re-searchers and engineers suggested ingenious mechanisms to reduce contents size. One of the mechanisms is viewport-only streaming service that streams a partial region in each video frame that fits in the exact viewport in an HMD device (e.g., HTC Vive, OculusVR, Google Daydream VR, and Samsung GearVR) in real-time [6].Assuming the size of the viewport is set to 90 degree, the bandwidth required for the viewport-only streaming service can be approximately reduced by an eighth of the original 360 video. Although the viewport-only streaming service can reduce con-tents size significantly, it is infeasible yet due to the existing streaming network latency. Even with the state-of-the-art content delivery network (CDN), viewport-only streaming service cannot satisfy the 10ms latency requirement in the standard Internet as demonstrated in other interactive multimedia systems [4]. Consequently, viewers can suffer discontinuity of streaming contents even in the highly optimized streaming service [8]. Thus, the research trend moved to the viewport adaptive stream-ing service that buffers segments of contents where a viewport Such reduction of contents size is achieved by the contents compression process that provides the original resolution in the Field ofView(FoV) while compromising the video quality outside the FoV. Despite the reduction of contents size and delivery of navigable 360 video contents, viewport adaptive streaming service cannot accommodate viewers' head movement within buffered video frames,especially when a viewer's focal point deviates from the viewport within the buffered frames. In this case, viewers can experience video distortions or degradation of resolutions [8]. Therefore, anew encoding mechanism that can handle viewers' deviation from the viewport while reducing contents size is required for viewers'immersive experience. To answer the requirements, we propose a saliency-based view-port adaptive streaming service, SALI360 that focuses on improving viewers' quality of perception in 360 video contents. To achieve high perception quality, we first adopt visual saliency model [9]to predict fixation regions in 360 video contents. Then we render the fixation regions on top of the geometry based encoded regions.Specifically, SALI360 encodes the peripheral regions in lower resolution to reduce the contents size, and encodes the fixation regions in higher resolution to increase the quality of perception.