Name
PTP Synchronization of GPU Processing
Date & Time
Tuesday, October 25, 2022, 4:00 PM - 4:30 PM
Thomas Kernen Thomas True
Description
GPU-based video processing applications that composite rendered graphics with live video require synchronization of the GPU processing to the video essence stream timebase to prevent visual artifacts caused by temporal misalignment. One such application is in-camera visual effects (ICVFX) in virtual production, the technique of capturing GPU-rendered visual effects composited with live action in-camera within an LED volume. As the media industry continues to move from a SDI to a ST 2110 infrastructure, virtual production environments for ICVFX utilize ST 2059-2 PTP locked Network Interface Controllers (NICs) or Data Processing Units (DPUs) for accurate packet pacing of the media essence streams according to the ST 2110-21 specifications.

GPU processing of video frames is typically performed at the display refresh frequency with updates aligned to frame boundaries. This paper describes a solution for synchronizing GPU display updates with a ST 2059-2 PTP locked NIC, where the PTP Hardware Clock (PHC) is disciplined by the PTP stack running on the NIC/DPU. The NIC generates a Transistor-Transistor-Logic (TTL) signal at the required frequency which is then used to align the GPU’s display engine frame boundaries with those of the PTP synchronized ST 2110-20 video essence streams to within nanosecond accuracy. Application Programming Interfaces (APIs) allow GPU-based video processing applications to align the generation and processing of frames across multiple GPUs, as well as perfectly synchronizing updates across all displays.

The ability to leverage the ST 2059-2 PTP profile used to synchronize the ST 2110 essence streams to additionally synchronize displays attached to GPUs, significantly simplifies the overall infrastructure deployment and management relative to traditional means of synchronizing displays using a Sync Pulse Generator (SPG) over coax infrastructure.
Location Name
Salon 2
Take-Aways from this Presentation
The NIC/DPU and PTP are leveraged for accurate timing in virtual production in-camera visual effects environments. Synchronization of the GPU and attached displays to the PTP frame boundaries is critical for the capture of in-camera visual effects to eliminate artifacts caused by temporal misalignment between the GPU render frames and the video essence streams. GPU synchronization to PTP frame boundary must be done in hardware as a software solution cannot provide sufficient accuracy or stability.