How Machine Learning May Modernize—and Monetize—Legacy Video Assets

As we detailed in the waning days of 2018, artificial intelligence is proving increasingly adept at upscaling still images to improve their resolution. The technology has become mainstream enough that consumer digital imaging software vendors are selling it for $100.

Now, the technology is rolling into the professional video market.

Upscaling, or interpolation, is a relatively common practice in the video market—available on hardware devices from DVD and Blu-ray players up to encoding systems in professional editing software. With the advent of machine learning, video upscaling appears poised to undergo the same renaissance it’s enjoying in the still photography market.

Unlike traditional approaches, machine learning-powered upscaling appears to deliver more realistic results. We have already seen some impressive proof of concepts from academic researchers and AI “enthusiasts” (for lack of a better word), including the delivery of footage recorded in 1895 (yes, that’s right) into a 4K/60p file. Now, companies like Pixop, iSize and GDFLAB are commercializing it. (Indeed, several of these companies were poised to tout their wares at NAB 2020 until the COVID-19 pandemic shuttered the show.)

Two key forces are driving the push for AI-powered upscaling.

The first is the flowering of dozens of streaming media services, which have collectively stoked a nearly insatiable demand for video content. While upstarts and established brands plow hundreds of millions of dollars into creating new content to satiate this demand, there’s renewed attention to older assets to fill the maw as well. Indeed, part of the early success of Disney Plus has surely been attributable to the company’s treasure trove of old films and shorts.

The second is the progression of display technology from HD to 4K/HDR and the coming of 8K video on the horizon. While large-budget Hollywood productions have had access to high-resolution sensors capable of creating master files that can keep pace with changing display technology, that’s just the tip of a much larger iceberg of content that’s stored at a lower resolution. In fact, even Hollywood hasn’t always been diligent about preserving high-quality original masters. Upscaling enable legacy content owners to deliver formerly lower-resolution content to today’s higher-resolution displays. It also allows them to restore degraded footage that may not have been restorable using traditional upscaling techniques.

As AI upscaling techniques mature, and as cloud computing resources continue to add GPU processing power at scale, the market for AI-powered upscaling looks bright.