Adobe’s Firefly Video, Finally in Beta: Why It Matters

Yesterday, Adobe released the beta version of its much-anticipated Firefly Video Model. The text-to-video and image-to-video generator offers the ability to produce 16×9 and 9×16 video clips up to five seconds in length at 1080p resolution, with the promise that 4K is coming soon.

Firefly is the brand name for Adobe‘s family of generative AI applications, first made publicly available in March 2023 with its still image generator. Today that’s part of its Creative Cloud applications suite and, according to the company, has produced over 18 billion assets. Developing Firefly Video took longer than the company expected. Its first significant steps forward came last fall with the introduction of Gen AI tools like Object Removal and Generative Extend to Premiere Pro. Adobe’s professional video editing software is widely used by independent filmmakers, including a majority of those at Sundance this year.

Firefly is not the first publicly available text-to-video generator and — we’ll wait for reviews, once users have time to fully test drive — it may not be the most powerful. The big selling point for Firefly Video is, unlike OpenAI’s Sora, it is commercially safe.

From day one, a “clean” model was the company’s AI selling point: Adobe doesn’t infringe on copyrighted material. Unlike so many Gen AI models that steal (and can we please start using this word) whatever it can scrape off the internet, Adobe trained Firefly using images and video it owns — in part by leveraging its stock photo and video business and relationships.

In conversations with IndieWire over the last year, Adobe executives told IndieWire that the “clean” approach slowed their Firefly video progress. It also means that its Gen AI generator may have initial gaps in its abilities. But for those who care about AI ethics (or, are simply paid professionals who can’t work commercially and violate copyright laws), Adobe’s approach is refreshing in the face of Silicon Valley thieves who believe that breakneck AI development will outpace the courts’ ability to legislate it.

Filmmakers working in Premiere Pro must exit that application and open the Firefly to generate video. However, once the video is in Premiere they can make key adjustments to the Firefly footage, rather than having to toggle between apps.

The beta version of Firefly Video also offers basic filmmaking and camera controls. It understands a “close-up,” and “wide shot,” and it can execute basic camera moves. This is a particularly powerful feature with the photo-to-video generator. In a demonstration for IndieWire, the controls work well; it also lets you avoid some of the madness involved with trying to get ChatGPT to write a paragraph that will allow Sora to understand a long shot.

As seems to be true with all things AI, the audio features are impressive today. The ability to generate narration or dubs in other languages is potentially huge for independent creators who want to reach a global audience.

Available today are two new Adobe Firefly plan offerings, Firefly Standard ($9.99) and Firefly Pro ($29.99). Firefly Standard provides 2,000 video/audio credits per month with up to 20 five-second 1080p video generations. Firefly Pro provides 7,000 video/audio credits per month with up to 70 five-second 1080p video generations. Coming soon: a Firefly Premium plan for high-volume professionals.