Generative AI in television
How I helped incorporate generative models into the #2 Amazon Prime series
Reading time about 4 minutes.
Welcome to Visually AI!
🔮AI News this Week
I can FINALLY share what I’ve been working on for the past 6 months!
I worked on the #2 series currently streaming on Amazon Prime: House of David
I joined the Wonder Project post-production VFX team for the show co-directed by Jon Erwin. Jon found a way to incorporate generative AI into the existing pipeline for certain shots.
The show is doing extraordinarily well on Amazon Prime, with 22 million viewers in the first 17 days!
To be clear - the show is a fully-staffed production shot on location and in a studio with a large cast and crew.
In post-production, I used a variety of new technology, including:
Midjourney, Mystic, Flux
Runway, Kling, Hailuo
Magnific, Krea, Freepik
I was privileged to be the only full-time AI artist working on Season 1, but now I’ve been working full-time on location at the production studio for Season 2 with a team of other AI artists and we do a lot more than prompting: Testing workflows, finding the right tools for specific VFX tasks, and pre-visualization to assist the cast and crew before they shoot scenes.
Again, this is not an AI series - there is a large cast and crew of more than 700 people. We are not replacing anyone and helping the show stay within a limited budget by creating assets that can be used during the shooting.
We work long hours, but we can visit the sets and sound stage in the building, and travel to the larger, outdoor set locations nearby:
I’ll share more about how we incorporate generative models into new workflows in the coming weeks.
This is a quick preview of the raw videos used for the opening scene in Episode 106:
📸 AI Snapshots
OpenAI 4o Image Model
OpenAI integrated advanced image generation capabilities into GPT-4o, so you can generate photorealistic images directly within ChatGPT including, complex visuals, including detailed text and diagrams, by leveraging GPT-4o's comprehensive understanding of both text and images.
My prompt: “Put my banner on a billboard in Times Square”
Reve
A new image model is here: Reve (also known as ‘Halfmoon’). Reve excels at photorealism, prompt adherence, and typography. You get 100 free credits when you sign up and 20 credits daily when you login.
Some of my first results:
Runway Gen-4 Video
Runway released Gen-4 video for all paid and Enterprise plans. The new model is currently available for image to video only, with improved ability to generate dynamic videos with realistic motion and better prompt adherence.
Here’s my first result with no upscaling:
🚀 My Recent Top AI Tool Picks
Kiko: Chat with Kiko in Kaiber Superstudio to generate and restyle images and videos on a canvas with the top models. It learns as you create - offering workflow suggestions and realtime feedback.
Gemini Co-Drawing: Built with Gemini 2.0 native image generation; lets you draw on the pad and prompt for changes to the image.
TrajectoryCrafter: New open source model uses diffusion models to redirect the trajectory of videos. I tried the demo with my videos - original on the left and results on the right side below:
Hunyuan 3D 2.0 Multiview: Upgraded 3D generation model with multi-view generation from Tencent.
Orpheus-3b TTS: Open source zero-shot voice training capable of generating consistent voices with empathy and expression.
🖼️ Image Prompts
Prompt: Macro shot of whiskey being poured over ice in slow motion, capturing the moment liquid hits the crystal-clear cubes. Amber liquid splashes and creates tiny droplets in the air. Product-photography perfection, hyperdetailed liquid physics, controlled studio lighting with dramatic highlights.
Prompt: Macro shot of a honeybee landing on a vibrant wildflower, pollen coating its legs. Water droplets on the petals catch morning light like tiny prisms. Extreme textural detail, shallow depth of field with bokeh background, natural sunlight creating translucent wings, photoreal scientific accuracy
🎥 Video Prompts
Here is a recent image to video model comparison I posted, using a Midjourney image and this prompt:
Prompt: As the fierce warrior turns to rally his troops, sunlight gleams off his chiseled arms, showcasing their taut muscle definition. In the foreground, he stands resolute, a harbinger of strength, while a dolly zoom reveals his warriors behind, poised with spears. Dust swirls around them under a vivid blue sky, capturing the impending surge of battle.
Thank you for reading.
Have a creative week!
Interesting article, IA is making progress in professional contexts. However, I don't understand how you're not replacing anyone. Helping the show stay in a limited budget by not hiring people to do the job is the literal definition of replacement.
Very beautiful article, Heather! Many thanks for sharing both the insights and your knowledge with us! Cheers from Zurich!