A System Architecture for Live Immersive 3D-Media Transcoding over 5G Networks
Tele-Immersive (TI) applications are network based immersive media applications that enable multi-party real-time interactions of users located in different parts of the globe by placing them inside a shared virtual world. For each user participating in a TI application, their appearance is captured via multiple RGB-D sensors and a “3D replica” is created by utilizing full-body 3D-reconstruction algorithms that produce the person’s digitized 3D time varying mesh (TVM). TI applications produce large volumes of visual data in the form of multi-textured geometry, thus, creating a challenging networking scenario. Although TVM geometric data can be compressed per-frame via state-of-the-art static mesh compression algorithms, these techniques lack optimal performance since they do not exploit correlations of the data over time. As of today, TVM compression algorithms leveraging temporal exploitation of TVMs are very few and these TVM compression schemes are not quite ready yet to support real-time applications. Apart from the geometry aspect and in the simplest scenario, the texture components of TVMs are typically encoded as static images via typical image compression algorithms like JPEG2000. Thus, in order to cope with the large volume of data produced by a TI application, the invention of novel, efficient, real-time, and potentially adaptive, TVM compression schemes is highly demanded. However, due to the difficulties of developing such decent TVM codecs conforming to the aforementioned characteristics, an upgrade to a more efficient network infrastructure appears to be an additional necessity. TI applications have very high bandwidth requirements and ultra-low latency to allow for interactions and VR viewing as well as, in the future, delivery in high crowd-density areas. These comprise some of the main targets to be addressed by modern 5G networks.