XRECO

While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access to content for media creators, considering novel data monetization and rights management policies. A set of AI-based media transformation services are built around the NMR to produce novel media- and XR experiences, including 3D neural reconstructions, neural based device localisation, image stitching, de-/re-lighting and holoportation. The developed technology will be validated in use case scenarios for (i) the news media for XR-based broadcasting and automatic and customized multitarget news publishing, and for (ii) location-based information and entertainment content, with applications in tourism and the automotive industry.

Our team was tasked with: 1) Research in Neural Radiance Fields in-the-wild algorithms, 2) Research in Gaussian Splatting-based Avatar reconstruction, 3) Micro-service-based deployment of the developed algorithms.

Consortium

DEUTSCHE WELLE
FUNDACIO PRIVADA I2CAT, INTERNET I INNOVACIO DIGITAL A CATALUNYA
JOANNEUM RESEARCH FORSCHUNGSGESELLSCHAFT MBH
ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS ANAPTYXIS
ESTUDIO BAY SLU
MOG TECHNOLOGIES SA
SOUND HOLDING B.V.
UNIVERSIDAD POLITECNICA DE MADRID
FINCONS SPA
ATOS SPAIN SA
ZAUBAR UG (HAFTUNGSBESCHRAENKT)
VIRTUAL REALITY BERLIN-BRANDENBURG E.V.
RAI-RADIOTELEVISIONE ITALIANA SPA
MELLANOX TECHNOLOGIES LTD – MLNX
KATHOLIEKE UNIVERSITEIT LEUVEN
FFP FILM- & FERNSEHPRODUKTION GMBH
UNITY TECHNOLOGIES APS
CONTINENTAL AUTOMOTIVE ROMANIA SRL
CHIOU THEODOROS