Multi-Modal AI: Video Creation Simplified
Simplify your video workflow with AI
Total reading time around 4 minutes.
Welcome to Visually AI!
🔮AI News this Week
Emerging Multi-Modal AI Video Creation Platforms
The rise of multi-modal AI platforms has revolutionized content creation, allowing users to research, write, and generate images in one app. Now, a new wave of platforms is extending these capabilities to video creation and editing.
Multi-modal video platforms combine various AI tools for tasks like writing, transcription, text-to-voice conversion, image-to-video generation, and lip-syncing. These platforms leverage open-source models like FLUX and LivePortrait, along with APIs from services such as ElevenLabs, Luma AI, and Gen-3.
Key Platforms
Krea AI
Krea recently added four AI video models (Gen-3, Luma, Kling, and Hailuo) to complement its FLUX Style Gallery and real-time image generator.
Kaiber
Kaiber's Superstudio offers multiple canvases for seamless image and video generation. Users can create images with FLUX, generate videos using Kaiber's model or Luma, and perform video-to-video style transfer.
Hunch
Hunch enables users to build AI 'Blocks' for individual tasks, which can be combined to perform complex operations, including image and video generation.
Each AI
Each is an open-source framework for building complex AI workflows or using preset templates to generate images, logos, or videos.
Developer Platforms
Developer-focused platforms like Fal, Replicate, Hugging Face, and Mystic offer a wide range of models for generating text, voice, images, character animations, and videos.
Fal recently added Gen-3, Luma, Kling, and Hailuo video models to its offerings.
Why should you care?
These multi-modal platforms allow users to access various AI tools and models in one place, eliminating the need to switch between multiple apps or maintain separate subscriptions for each service.
📸AI Snapshots
Adobe MAX 2024 showcased a significant focus on AI-powered creativity, with Adobe unveiling over 100 new features across Creative Cloud applications. Key highlights included examples of Adobe Firefly Video model (limited public beta), updates to existing apps like Photoshop and Illustrator for improved speed and precision, Generative Extend in Premiere Pro, and the introduction of Adobe GenStudio, a product designed to help enterprise teams automate and scale marketing content delivery.
Suno AI launched Scenes for iOS - create unique music from your photos or videos. The Android app will be available soon.
Amazon introduced AI Shopping Guides to help consumers quickly match buyer guidance and customer insights to the product type you’re looking for. The AI guides will appear automatically in search autocomplete suggestions.
Google’s Imagen 3 text to image model is available to all users in Gemini, with improved photorealism, understanding prompts, and fewer artifacts. I asked it to generate a long a detailed prompt and the result was very good:
🛠️ This Week’s AI Tools
Kick: ‘Self-driving bookeeping’ AI-powered accounting app categorizes transactions and learns your spending patterns in the background. I have been using it for my personal business and canceled QuickBooks because it was far more efficient with less work from me. And it’s free up to the first $25k business limit. (link)
Cartwheel: Text to 3D animation for VR, AR, video games, or social media posts. (link)
Vimmerse: Animate images with Gen-3, Haiper, and more with Vimmerse’s Canva app, called Image Animate. (link)
AI Workflows: Find expert AI video community members’ AI workflows for multiple image and video models. (link)
🖼️ Image Prompts
My friend, Ludovic Carli, shared a technique called Polaroid Lift Emulsion on 𝕏. It produces a beautiful effect, as you’ll see in these images below.
Note: Ludovic shares amazing prompts and workflows every day, and he actually uses far more cutting-edge generative AI tools than I do! Check him out on 𝕏 or LinkedIn.
Prompt: A still life of a typewriter with handwritten notes, laying beside it portrayed through a Polaroid emulsion lift on aged canvas, with visible brush strokes and faded sepia lighting.
Prompt: A horse galloping on the beach, as if captured on a manipulated instant film Polaroid, with peeling emulsion and a surreal sense of motion.
Prompt: Portrait of an elderly woman with a warm smile, captured in a Polaroid emulsion lift style, highlighting handcrafted textures and muted pastel tones.
🎬Video Prompt
This prompt is long, but it worked well on Hailuo AI:
Prompt: First person POV, dutch angle shot, watching an amazing female DJ on stage with shoulder length wavy hair in a razor-cut bob style is wearing a purple zipup hoodie sweatshirt, and high-waisted dark wash denim jeans. The woman is a DJ and standing at her turntables, with an intense look on her face and bobbing her head and moving hands from side to side on the turntable and mixer as she switches beats and smiles when she likes the result, with the beat as she mixes and scratches for an incredible hip hop fused with EDM style. Background is a rave festival stage background
Thanks for reading and have a creative week!
Big thank you to the extraordinary Heather Cooper @visuallyai
Thanks for great Stack content from the UK.