Hot summer of AI video: Luma & Runway drop amazing new models
Plus an amazing FREE video to sound app from ElevenLabs
Total reading time around 4 minutes.
Welcome to Visually AI!
🔮AI News This Week
AI Video
Immediately after we saw Sora-like videos from KLING, Luma AI’s Dream Machine video results overshadowed them.
Dream Machine reached 1 million users in 4 days.
Not long after Dream Machine’s launch, RunwayML shared teaser clips of the newest video model, named Gen-3.
Luma AI Dream Machine
Sora? Yet to be released.
Kling? Limited access.
The Dream Machine for AI video is here...
Dream Machine is a next-generation AI video model that creates high-quality, realistic shots from text instructions and images.
Available to everyone, it generates 5-second videos with seamless extensions in 5-second increments. The free plan allows up to 30 videos per month, with paid options for more. Try it here.
I had early access to Dream Machine and I was not prepared for the high-quality videos it could produce.
These are all clips I generated using text-only prompts or my Midjourney images plus text prompts.
No editing, including upscale:
You can see more examples in this YouTube video I posted this week.
RunwayML Gen-3
Runway introduced Gen-3 Alpha, a new model for high-fidelity, controllable video generation.
This model represents a significant improvement over its predecessor, Gen-2, in terms of fidelity, consistency, and motion. Gen-3 Alpha is trained on a new infrastructure designed for large-scale multimodal training, incorporating both videos and images.
It supports various tools such as Text to Video, Image to Video, and Text to Image, along with advanced control modes like Motion Brush, Advanced Camera Controls, and Director Mode.
Key features of Gen-3 Alpha include:
Fine-grained temporal control: Enables imaginative transitions and precise key-framing.
Photorealistic humans: Capable of generating expressive human characters with diverse actions and emotions.
Industry customization: Allows for stylistically controlled and consistent characters tailored to specific artistic and narrative needs.
The model also includes new safeguards, such as an improved visual moderation system and C2PA provenance standards.
Gen-3 Alpha was developed collaboratively by a team of research scientists, engineers, and artists, to interpret a wide range of styles and cinematic terminology.
Gen-3 is not available to the public yet, and they haven’t announced a release date.
Hedra Character-1
Introducing Character-1, Hedra's new foundation model for expressive talking, singing, and acting characters.
Available now on desktop and mobile, it offers a free open preview with 30-second durations and up to 90 seconds of generated content per 60 seconds.
The company says this marks the first step in its mission to create a multimodal studio for complete control over emotional dialogue, movement, and entire worlds.
I was impressed with the expressive characters and the ability to maintain the quality throughout the videos. Try the beta for free here.
These are a few of my results:
You could have your AI service, tool, or event seen by Visually AI’s community of over 15,000 subscribers:
🚀 This Week’s AI Tools
Video to Sound Effects: ElevenLabs' new tool analyzes your video to generate four matching sound effects and lets you download each selection with the sound attached to the video. (link)
The generated sounds are often accurate, as you can see in this video with one of my Krea Video clips:
OnePublish: Cross-publish your content directly from Notion to DEV, Hashnode, Medium, Ghost and more upcoming platforms. (link)
Unicorn Platform: AI-powered website builder designed for startups, solo entrepreneurs, and hackers to create responsive websites effortlessly using ready-made templates. (link)
Glif: A low-code platform for creating AI-powered "glifs" that transform user inputs like text and images into outputs such as text, images, or videos. (link)
Abacus AI: Comprehensive AI platform offering solutions for forecasting, anomaly detection, language processing, personalization, marketing, sales, vision, and fraud detection. (link)
OSSA: Converts your script into professionally edited short-form video, with no editing skills required. (link)
💻Visually AI on YouTube
Luma AI Dream Machine | Free AI Video
🖼️ Image Prompts
Prompt: Photorealistic waterfall at sunset with the sky in colors of salmon, gold, violet, dusky rose
Prompt: Design an abstract sphere using dot art, with a 3D effect making it appear to float above the surface, vibrant color contrasts to define its texture, and dynamic lighting to create shadows and highlights.
Thanks for reading, and have a creative week!