Nvidia unveils NeMo Video Curator for petabyte-scale video processing

Nvidia (NVDA) said the NVIDIA NeMo Curator team developed a flexible, GPU-accelerated streaming pipeline for large-scale video curation now available on NVIDIA DGX Cloud. “In this post, we explore the optimizations made in the pipeline including auto-scaling and load-balancing techniques to ensure optimized throughput across pipeline stages while fully using available hardware,” the company said. “The result? Higher throughput, lower total cost of ownership (TCO), and accelerated AI training cycles, speeding up end user outcomes… Our efforts with auto-scaling and load balancing are only part of the performance optimization work that has enabled us to achieve a 89x performance improvement to date above baseline, resulting in a pipeline capable of processing ~1M hours of 720p video data on 2k H100 GPUs in one day. The efficient use of GPU resources can be further demonstrated by tracking the worker utilization of GPU stages of the NeMo Curator pipeline during execution.” NeMo Video Curator is available through early access on NVIDIA DGX Cloud. NeMo Video Curator is also available through early access as a downloadable SDK, suitable for running on your own infrastructure, also through the Early Access Program, the company added. Meanwhile, Nvidia said that it has been actively working with a variety of early access account partners to enable best-in-class TCO for video data curation, with such partners including Getty Images (GETY), Canva, Uber (UBER), and others.