Broadcasting stage events such as theater, music, or circus performances expands the potential audience to those unable to attend live, and allows re-experiencing the event at home. However, traditional media like television or cinema often fail to convey the immersive feeling of participation, making the broadcasted experience less thrilling. Head-mounted displays for virtual reality offer a promising solution to bridge this gap by surrounding the spectator with a complete and realistic environment, enhancing the sense of presence compared to 2D screens. While 360° video is commonly used, it lacks parallax, which can lead to motion sickness due to the absence of depth and occlusion information. Alternatively, modeling the event location in a 3D game engine can reproduce parallax effectively, but integrating live actors remains challenging. Often, green-screen or volumetric capture is used, or actors are replaced by avatars, diminishing naturalness. To address these issues, we developed a workflow for integrating live actors into a 3D environment, offering a fully immersive experience. Our approach balances realism and complexity by comparing full light-field rendering with stereo projection. Although stereo projection does not perfectly reproduce parallax, it provides convincing depth cues if the spectator is at a reasonable distance from the performance. Additionally, stereo content aligns closely with traditional broadcast media, allowing the reuse of existing infrastructure. However, this approach necessitates a distinct production methodology. Actors’ performances are transmitted via a stereo signal, while surrounding environments are recreated from digitized 3D models. New production and rendering guidelines are essential to maintain visual quality. This paper summarizes the lessons learned and the developed workflow to advance immersive media in stage events.
Understand the challenges of producing immersive media for stage events. Compare the advantages and drawbacks of stereo-based production compared to alternative solutions. Get to know about the guidelines and workflows for stereo content played back in VR applications. Know required capture and processing equipment. Be able to evaluate the next steps required for immersive media delivery for stage events.