Kandinsky 6.0 Video: generating audio and motion together
A guide to the Lite and Pro models, synchronized audio, and the published implementation.
Kandinsky 6.0 Video release artwork. Credit: Kandinsky Lab. Original image ↗
Kandinsky 6.0 Video generates short clips with synchronized audio from text or an input image. The official repository provides the implementation and model links; the lab’s model overview explains the release’s intended capabilities.
Plan the image and sound as one scene
A short clip of someone closing a door has at least three requirements: the hand reaches the handle, the door moves consistently, and the closing sound coincides with contact. A model that produces attractive frames can still fail that timing test.
For a first comparison, choose one visible action and one identifiable sound. Describe the setting, motion, camera, and sound without introducing several competing events. Generate comparable samples and examine the frame where the sound should occur. This evaluation plan helps you separate synchronization from visual style; it is not an undocumented prompting guarantee.
Two model sizes
| Model | Parameters | Intended emphasis |
|---|---|---|
| Video Lite | 3 billion | Faster generation |
| Video Pro | 29 billion | Richer motion and camera control |
The release describes five-second clips, synchronized 44 kHz audio, lip synchronization, and super-resolution up to Full HD. These are specifications from the maintainers, rather than results from our own GPU tests.
Work through audio and video timing
Synchronization is a relationship between two clocks. A frame rate tells you how finely you can locate an event visually; an audio sample rate tells you how many waveform samples represent that same interval. Neither automatically establishes that a generated sound matches the visible action.
For a worked example, choose 25 frames per second. This frame rate is our illustrative analysis setting, not a required Kandinsky output mode. The release's 44 kHz audio specification then gives:
Each forty-millisecond frame interval spans 1,760 audio samples. A five-second waveform has 220,000 samples per channel at that rate. If a closing-door sound begins two frames after visible contact, the offset is 0.08 seconds, or 3,520 samples. That quantifies the discrepancy, but it does not decide whether a listener will accept it. Sound travel, editing style, and the action itself can affect an appropriate tolerance.
The graph converts hypothetical frame offsets to milliseconds. It is an analysis aid for reviewing a clip, not a synchronization benchmark. Use the actual exported frame and sample rates when evaluating an output; some players and containers expose timing differently. Keep sample counts per channel separate from byte size, which also depends on channel count and sample format.
Turn a prompt into an acceptance table
For an illustrative door-closing brief, record four checks: the hand makes contact, the door reaches its closed position, the sound occurs at the intended event, and the room remains coherent. A clip can pass three and still fail the task. A convincing noise without corresponding contact is not successful synchronization; a silent but visually coherent clip is a different failure.
Review multiple outputs using the same brief and output settings. Record a timing tolerance before comparing models, and retain the examples that fail it. Increasing resolution may improve visible detail while leaving a timing error untouched. Likewise, changing an audio instruction can improve the sound while introducing an unwanted event. This is why the Lite/Pro distinction and the full inference configuration should accompany any practical comparison. The model size does not by itself explain which dimension improved.
Convert frame offset to elapsed time
Focus the graph and use arrow keys, or tap a point, to inspect values. The complete data table is available below.
View data table
| Offset (frames) | Offset (ms) |
|---|---|
| 0 | 0 |
| 1 | 40 |
| 2 | 80 |
| 3 | 120 |
| 4 | 160 |
| 5 | 200 |
Choose an entry point
The repository links integrations for Diffusers, ComfyUI, and other inference frameworks. Use the instructions for the integration you intend to run. Hardware requirements and dependency versions can differ between them.
For a source installation, check the repository’s current prerequisites before creating the environment. Keep the model checkpoint, inference configuration, and integration version together in your experiment notes.
Follow the release’s actual setup path
The repository contains installation and inference instructions, model links, and a ComfyUI integration. Choose the Lite or Pro path before copying a configuration. Check the requested weights, runtime dependencies, and hardware notes together; parameter count alone is not a dependable estimate of the memory needed by an entire pipeline.
Keep the initial test at the documented output settings. Once that works, change one variable—such as the description or camera instruction—and retain the original for comparison. Save the model variant and inference configuration with the output so a later reviewer knows what produced it.
Review beyond the best sample
Listen with headphones, then inspect the clip without sound. Check that motion remains coherent in both views. Compare several outputs from the same brief and count the synchronization failures, rather than choosing a single attractive result. The publisher’s examples establish what it presents publicly; a dependable production workflow needs an acceptance test for your own subjects and output requirements.
