Dialogue and Lip Sync · öffentlicher Open Skill
Dialogue and Lip Sync
Dialogue and Lip Sync ist ein öffentlicher Open Skill auf Digen. Öffne ihn, um zu sehen, was dieser Workflow erstellt.
Custom
34 5 15
https://d7qlwi7tl2u27.cloudfront.net/skill-covers/prod/1856918/2026-09-03/mtcyh6sc.png
# Dialogue and Lip Sync
## When to use
- Activate **whenever we generate any video that contains speaking characters**: dialogue scenes, film/drama shots, short videos, MVs, interviews, talking-head or reaction shots.
- Use standalone (the user gives a line/scene and wants a spoken shot) **or** as an overlay on other generation workflows (movie-maker, short-maker, music-video-lovagen, etc.): the same dialogue and lip-sync rules apply to every generation step that contains speech.
- Do not use for fully silent videos with no spoken dialogue.
## Core principles
1. **Every character speaks the correct dialogue at the right moment** throughout the entire video. Each spoken line must match the character who is supposed to say it, and the timing of the speech must fit the scene, emotion, and action.
2. **Mouth movements must be synchronized with the spoken words as accurately as possible.** Lip movement should match the rhythm, timing, and intensity of the dialogue. Avoid random mouth movement, silent talking, delayed lip sync, or lips moving when the character is not speaking.
3. **The performance supports the line.** When a character speaks, their facial expression, eye movement, head movement, and body language should support the meaning and emotion of the line. Angry dialogue → tension in the face and mouth. Happy dialogue → mouth, eyes, and expression feel open and positive. Dramatic dialogue → timing feels cinematic and emotionally believable.
4. **Only the current speaker moves their lips.** Other characters react naturally through facial expressions, eye contact, small gestures, or body language — they must not move their mouths unless they are also speaking.
5. **Consistency.** Dialogue stays clear, natural, and synchronized with the visuals; spoken text, facial performance, and lip movement remain consistent across the entire video.
## Prompt building blocks
- **Positive frame**: "Only {speaker} speaks this line — \"{line}\" — at {timing/beat}. Their mouth moves in precise sync with the words, matching the rhythm, timing, and intensity of the delivery. The face, eyes, head, and body support the {emotion} of the line: {emotion/performance descriptors}. Other characters listen and react naturally through expression, eye contact, or small gestures, without moving their lips. The dialogue is clear, natural, and consistent with the visuals."
- **Guardrails to avoid** in the prompt: random mouth movement, silent talking, delayed lip sync, lips moving when the character is not speaking, characters mouthing lines they are not supposed to say, muffled or garbled speech.
- **More examples**: ready-to-paste prompts for dialogue scenes (argument, greeting, quiet dramatic line, surprise, singing/MV) live in `references/example-prompts.md`, plus a verification snippet.
## Steps
1. **Lock the dialogue map.** From the user's request or the hosting pipeline's script, build a per-line table: line text → speaker → timing/beat → emotion/intensity. If a voice track exists, identify which audio segment belongs to which character and line before generating.
2. **Compose the prompt with lip-sync + performance direction.** State who says which line, when, and how the mouth/face/body perform it, using the positive frame and guardrails above.
3. **Generate.** If acting as an overlay on another workflow, inject the composed prompt into that pipeline's generation step (image-to-video / director / clip assembly) and keep its channel choices. If generating standalone:
- For the **most accurate lip sync with real audio**, use audio-driven channels: `generate_video` with `model='ai2v'` (single image + driving audio) or `model='ref2v'` + `gen_mode='reference_to_video'` + `audio_url` (reference images + driving audio). Pass the voice track via `audio_url` (for `ref2v`) or as the driving audio (for `ai2v`); never stuff the audio into `input_urls`.
- For silent shots or video without a voice track: `model='i2v'` (single image → video), `'fl2v'` (fir