Music Video Lip-Sync Studio · skill pubblica Open Skill
Music Video Lip-Sync Studio
Turn one photo and one song into a finished music video. Music Video Lip-Sync Studio runs the whole production: • Reads the track — transcribes the audio to get the lyric text and section timing (verse, chorus, drop), then plans one shot per section. • Locks the performer — builds one identity reference image first (from your photo, or from a description you approve), then keeps every shot on that same reference so the face never drifts between cuts. • Picks the right channel per shot — audio-driven lip-sync where the mouth has to follow the track, reference-to-video for identity across scenes, image-to-video for simple motion. • Score and typography — original instrumental beds and stingers, plus lyric text laid out per section. • Stitches the final cut — shots assembled in narrative order, landscape or portrait. • Remembers your performer — your locked look and visual style carry over to the next video, so a second track stays consistent. Works from an uploaded audio file, a YouTube / TikTok / Instagram link, or music generated inside the skill. Output: a stitched MV in landscape or portrait, plus every shot as a separate clip you can re-cut. Note: video generation on this platform does not support adult content on any channel. Such requests are declined, with a non-explicit alternative offered instead.
Music
31 6 7
https://d7qlwi7tl2u27.cloudfront.net/skill-covers/prod/4267104/2026-10-12/ydwar7vc.png
# Music Video Lip-Sync Studio
Turn a song or an audio track into a finished music video. The user may bring a photo of a performer, a reference portrait, or nothing but audio. Output is a stitched MV with consistent identity, in landscape or portrait.
## When to use
- The user provides audio (upload or link) and wants visuals for it.
- The user wants a "music video", "MV", "lyric video", "singing video", or "make this photo sing".
- The user wants a multi-shot cinematic sequence cut to a track.
- The user asks to keep the same performer across several videos (continuity).
Do not use this skill for: talking-head narration with no music (that is a plain avatar/video task), pure audio generation with no visuals, or anything requiring adult/NSFW video — see Notes.
## Steps
0. **Set up**
Call `write_todos` with the plan: confirm audio → transcribe → lock the performer → shot list → generate shots → stitch → deliver.
Call `memory_whoami` and then `memory_query` with a query like "performer appearance and visual style". If the user has made videos before, reuse the locked look instead of inventing a new one, and say so ("I'll keep the same performer look as your last MV — say the word if you want a different one").
1. **Get the audio and its timing**
- If the user uploaded audio, use its asset URL directly.
- If the user gives a YouTube / TikTok / Instagram link, resolve it first with `fetch_video_info(url=...)` for metadata. Only if the audio itself is needed, note that `download_video` supports YouTube / TikTok / Instagram only — for Facebook / X / LinkedIn, ask the user to upload the file instead.
- Call `transcribe_audio(audio_url=<asset url>, model='asr')`. Use the returned `srt` (not just text) to get section boundaries, and use `text` as the lyric source.
- If the audio is longer than the per-clip generation limit, split it into sections by verse / chorus / drop. Plan one shot per section. Tell the user how many shots you planned before generating.
2. **Lock the performer's look (identity reference)**
This step is what makes the MV coherent. Always produce one locked reference image first.
- Photo supplied → `generate_image(model='i2i', input_urls=[<user photo asset>], name='Performer look', entity_type='characters', orientation='portrait')`
- No photo → `generate_image(model='t2i', name='Performer look', entity_type='characters', orientation='portrait')` with a description you confirm with the user first (age range, hair, wardrobe, era, mood). Do not silently invent a person.
- Show the reference to the user and get a yes before spending generation budget on shots. Every later step must reference this same asset so the face does not drift.
3. **Generate the shots**
Choose the channel from what drives the motion:
| Situation | Channel | Notes |
|---|---|---|
| Audio drives the performance (mouth/body moves with the track) | `ai2v` | Requires both an image and the driving audio: `input_urls=[look]`, `audio_url=<section audio>`, `audio_start_time=<seconds>` |
| Keep identity across different scenes, motion from text | `ref2v` | Default channel when `model` is omitted; requires at least 1 reference image |
| A single still that should move subtly | `i2v` | Cheapest single-image path |
| First-frame-to-last-frame transition | `fl2v` | `input_urls=[first, last]` |
| Pure text-to-video, no reference | `t2v` | Must pass `model='t2v'` explicitly or the call is rejected |
Call `generate_video` once per shot:
```
generate_video(
model='ai2v',
input_urls=[<performer look asset>],
audio_url=<section audio url>,
audio_start_time=<offset in seconds>,
name='Shot 3 — rooftop chorus',
entity_type='storyboard_videos',
orientation='portrait'
)
```
Rules:
- One shot per call. Never ask for a whole MV in one generation.
- Give every shot a distinct, meaningful name (it is the canvas label).
- Use `entity_type='storyboard