3DTV Content Search (3DTVS)

Framework: FP7 — Cooperation
Programme: Information and Communication Technologies
Destination: Pervasive and Trusted Network and Service Infrastructures
Action type: Collaborative project
Countries involved: Greece,Germany,France,France,Germany,Germany
Call:  (FP7-ICT-2011-7, 2011)
Start date: 2011-11-01
End date: 2015-01-31
Duration (months): 39.0
Total project cost (EUR): 3517641.0
EU funding (EUR): 2570650.0
Participants: 7
Project status: CLOSED

As 3DTV emerges, 3DTV content started already being accumulated in 3DTV broadcasters and 3D video production houses. Its volume will soon rise tremendously. Such content already started being posted in the web, as already done for 2D audiovisual material. Furthermore, with the advent of 3D mobile devices, such content is expected to be posted soon in social networks or user-fed audiovisual archives (e.g. YouTube). Such 3DTV audiovisual content consists primarily of stereo, video+depth or multiple views video and multichannel 5.1 or 7.1 sound and must be properly archived, searched and retrieved. However, it cannot be accommodated well within classical audiovisual media archives, because such archives do not exploit depth information, when indexing, searching, retrieving and browsing 3DTV content. Certain tasks, e.g. creation of 3D video summaries and browsing are inherently different in 3D than in 2D content. Therefore, new efficient schemes must be devised for content-based 3DTV audiovisual (AV) content description, indexing, fast search and browsing. The goal of the 3DTVS project is to devise scalable 3DTV AV content description, indexing, search and browsing methods across open platforms, by using mobile and desktop user interfaces and to incorporate such functionalities in 3D audiovisual content archives. The major novelty in 3DTV content indexing/retrieval research will be on how to exploit 3D (depth) information for stereo and multiview video indexing, retrieval and browsing that could address semantic queries of the form ‘find stereo videos with shallow depth’ or ‘find stereo videos, where actor X approaches actor Y’. 3D multichannel audio analysis will perform better audio source (e.g. musical instrument) separation and localization that will be used in 3D audio/cross-modal indexing and retrieval. Multimodal 3D audiovisual content analysis will built on the results of 3D video and audio analysis. 3DTV content description and search mechanisms will be developed to enable fast reply to semantic queries. User friendly mobile/desktop search interfaces will be built that could also permit adaptation to the user needs and profile. The novel results that will be generated within 3DTVS will: • reinforce the position of the European ICT and digital media industry in the 3D television sector, by opening wider market opportunities; • provide greater creativity both to 3DTV professionals and to the average user, stimulated through technologies and tools to search/exchange professional and user generated 3DTV content, possibly also to be used in immersive and interactive applications; • provide innovative offers to 3DTV content aggregators with tools and interfaces audiovisual 3DTV content consumption.

Keywords: 3dtv content search and retrieval,multimedia search,3d video,3d audio,multimodal analysis,semantic 3d video analysis